NetBSD Problem Report #53674

From wiz@yt.nih.at  Thu Oct 18 18:09:19 2018
Return-Path: <wiz@yt.nih.at>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 42C9C7A167
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 18 Oct 2018 18:09:19 +0000 (UTC)
Message-Id: <20181018170348.586692AC68C@yt.nih.at>
Date: Thu, 18 Oct 2018 19:03:48 +0200 (CEST)
From: Thomas Klausner <wiz@NetBSD.org>
Reply-To: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: Maxime Villard <max@m00nbsd.net>
Subject: kASAN: Unauthorized Access in file_ctor
X-Send-Pr-Version: 3.95

>Number:         53674
>Category:       kern
>Synopsis:       kASAN: Unauthorized Access in file_ctor
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 18 18:10:00 +0000 2018
>Closed-Date:    Mon Apr 08 18:47:20 +0000 2019
>Last-Modified:  Mon Apr 08 18:47:20 +0000 2019
>Originator:     Thomas Klausner
>Release:        NetBSD 8.99.25
>Organization:
Curiosity is the very basis of education and if you tell me that 
curiosity killed the cat, I say only that the cat died nobly.
- Arnold Edinborough
>Environment:


Architecture: x86_64
Machine: amd64
>Description:
When running a kernel with
        makeoptions     KASAN=1         # Kernel Address Sanitizer                                 
        options         KASAN                                            
        no options      SVS                                          
I got multiple kASAN warnings in file_ctor with similar backtraces.

kASan: Unauthorized Access In 0xffffffff80e92a80: Addr 0xffffbf0044d791e8 [8 bytes, write]
#0 0xffffffff80e92a80 in file_ctor <netbsd>
#1 0xffffffff80f18d60 in pool_cache_get_slow <netbsd>
#2 0xffffffff80f1ade2 in pool_cache_get_paddr <netbsd>
#3 0xffffffff80e958c2 in fd_allocfile <netbsd>
#4 0xffffffff80f7004d in fsocreate <netbsd>
#5 0xffffffff80f7807c in sys___socket30 <netbsd>
#6 0xffffffff80265278 in syscall <netbsd>


kASan: Unauthorized Access In 0xffffffff80e92a80: Addr 0xffffbf0121ef9228 [8 bytes, write]
#0 0xffffffff80e92a80 in file_ctor <netbsd>
#1 0xffffffff80f18d60 in pool_cache_get_slow <netbsd>
#2 0xffffffff80f1ade2 in pool_cache_get_paddr <netbsd>
#3 0xffffffff80e958c2 in fd_allocfile <netbsd>
#4 0xffffffff80f78446 in do_sys_accept <netbsd>
#5 0xffffffff80f7b3bd in sys_accept <netbsd>
#6 0xffffffff80265278 in syscall <netbsd>

kASan: Unauthorized Access In 0xffffffff80e92a80: Addr 0xffffbf004e2a69e8 [8 bytes, write]
#0 0xffffffff80e92a80 in file_ctor <netbsd>
#1 0xffffffff80f18d60 in pool_cache_get_slow <netbsd>
#2 0xffffffff80f1ade2 in pool_cache_get_paddr <netbsd>
#3 0xffffffff80e958c2 in fd_allocfile <netbsd>
#4 0xffffffff80fa1eb9 in do_open <netbsd>
#5 0xffffffff80fa2229 in do_sys_openat <netbsd>
#6 0xffffffff80fa23b4 in sys_open <netbsd>
#7 0xffffffff80265278 in syscall <netbsd>

kASan: Unauthorized Access In 0xffffffff80e92a80: Addr 0xffffbf00e1640d68 [8 bytes, write]
#0 0xffffffff80e92a80 in file_ctor <netbsd>
#1 0xffffffff80f18d60 in pool_cache_get_slow <netbsd>
#2 0xffffffff80f1ade2 in pool_cache_get_paddr <netbsd>
#3 0xffffffff80e958c2 in fd_allocfile <netbsd>
#4 0xffffffff80f3922f in pipe1 <netbsd>
#5 0xffffffff80f2f7d3 in sys_pipe <netbsd>
#6 0xffffffff80265278 in syscall <netbsd>

>How-To-Repeat:
Run a KASAN kernel, wait (with my usual load).
>Fix:
not provided.

>Release-Note:

>Audit-Trail:
From: "Maxime Villard" <maxv@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53674 CVS commit: src/sys/kern
Date: Thu, 27 Dec 2018 07:56:43 +0000

 Module Name:	src
 Committed By:	maxv
 Date:		Thu Dec 27 07:56:43 UTC 2018

 Modified Files:
 	src/sys/kern: uipc_domain.c

 Log Message:
 Fix apparent race.

 We're doing a LIST_FOREACH, but unlock filelist_lock in the middle of the
 loop and drop the reference to fp. We then read fp->...le_next, but it
 may have been freed by another thread.

 This is difficult to trigger and observe, probably only KASAN can see
 problems of this kind.

 Switch to LIST_FOREACH_SAFE, and re-fetch np after re-locking.

 May fix PR/53674.


 To generate a diff of this commit:
 cvs rdiff -u -r1.105 -r1.106 src/sys/kern/uipc_domain.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: maxv@NetBSD.org
State-Changed-When: Thu, 27 Dec 2018 08:02:08 +0000
State-Changed-Why:
Please tell me if the patch I committed fixes the issue.


From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/53674 (kASAN: Unauthorized Access in file_ctor)
Date: Tue, 1 Jan 2019 18:18:43 +0100

 No, the problem is not fixed. It  appeared again with a kernel from Dec 29 (with your patch included).

 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] kASan: Unauthorized Access In 0xffffffff80e8ac85: Addr 0xffffc400485064e8 [8 bytes, write]
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #0 0xffffffff80e8ac85 in file_ctor <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #1 0xffffffff80f177a1 in pool_cache_get_slow <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #2 0xffffffff80f197cc in pool_cache_get_paddr <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #3 0xffffffff80e8dae7 in fd_allocfile <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #4 0xffffffff80fa25f9 in do_open <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #5 0xffffffff80fa2969 in do_sys_openat <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #6 0xffffffff80fa2af4 in sys_open <netbsd>
 Dec 31 23:55:46 yt /netbsd: [ 91039.0390593] #7 0xffffffff802633a8 in syscall <netbsd>

  Thomas

State-Changed-From-To: feedback->open
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Tue, 01 Jan 2019 17:23:17 +0000
State-Changed-Why:
Feedback supplied


From: "Maxime Villard" <maxv@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53674 CVS commit: src/sys/kern
Date: Sun, 7 Apr 2019 08:37:38 +0000

 Module Name:	src
 Committed By:	maxv
 Date:		Sun Apr  7 08:37:38 UTC 2019

 Modified Files:
 	src/sys/kern: subr_pool.c

 Log Message:
 Fix tiny race in pool+KASAN, that resulted in occasional false positives.

 We were uselessly marking already valid areas as valid. When doing that,
 our KASAN code emits two calls to kasan_markmem, and there is a very small
 window where the area becomes invalid. So, if the area happens to be
 already globally referenced, and if another thread happens to read the
 buffer via this reference, we get a false positive.

 This happens only with pool_caches that have a pc_ctor that creates a
 global reference to the buffer, and there is one single pool_cache that
 does that: 'file_cache'.

 So now, two changes:

  - In pool_cache_get_slow(), the pool_get() has already redzoned the
    object, so no need to call pool_redzone_fill().

  - In pool_cache_destruct_object1(), don't re-mark the object. If there is
    no ctor pool_put is fine with already-invalid objects, if there is a
    ctor the object was not marked as invalid in the first place; so in
    either case, the re-marking is not needed.

 Fixes PR/53674. Although very rare and difficult to reproduce, a local
 quarantine patch of mine made the false positives recurrent.


 To generate a diff of this commit:
 cvs rdiff -u -r1.246 -r1.247 src/sys/kern/subr_pool.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: maxv@NetBSD.org
State-Changed-When: Mon, 08 Apr 2019 18:47:20 +0000
State-Changed-Why:
I managed to reproduce it, and it is now fixed. Close this PR.

(However, if you still see it, please re-open.)


>Unformatted:

Home	PR Database Search