NetBSD Problem Report #55493

From www@netbsd.org  Thu Jul 16 11:34:40 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 85B731A9213
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 16 Jul 2020 11:34:40 +0000 (UTC)
Message-Id: <20200716113439.4A68E1A921E@mollari.NetBSD.org>
Date: Thu, 16 Jul 2020 11:34:39 +0000 (UTC)
From: prlw1@cam.ac.uk
Reply-To: prlw1@cam.ac.uk
To: gnats-bugs@NetBSD.org
Subject: panic in radixtree
X-Send-Pr-Version: www-1.0

>Number:         55493
>Category:       kern
>Synopsis:       panic in radixtree
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    chs
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jul 16 11:35:00 +0000 2020
>Closed-Date:    Thu Aug 20 06:52:12 +0000 2020
>Last-Modified:  Thu Aug 20 06:52:12 +0000 2020
>Originator:     Patrick Welche
>Release:        NetBSD-9.99.69/amd64
>Organization:
>Environment:
>Description:
panic: kernel diagnostic assertion "*vpp == NULL" failed: file "/usr/src/local/netbsd/src/sys/lib/libkern/../../../common/lib/libc/gen/radixtree.c", line 591

A core file is available:

crash> bt
_KERNEL_OPT_NJOY_EAP() at 0
?() at ffffad54ce539298
sys_reboot() at sys_reboot
vpanic() at vpanic+0x15b
__x86_indirect_thunk_rax() at __x86_indirect_thunk_rax
radix_tree_insert_node() at radix_tree_insert_node+0x141
uvm_pagealloc_strat() at uvm_pagealloc_strat+0x55e
uao_get() at uao_get+0x524
ubc_fault() at ubc_fault+0x425
uvm_fault_internal() at uvm_fault_internal+0xc41
trap() at trap+0x501
--- trap (number 6) ---
copyout() at copyout+0x33
uiomove() at uiomove+0x84
ubc_uiomove() at ubc_uiomove+0x165
tmpfs_read() at tmpfs_read+0xbe
VOP_READ() at VOP_READ+0x40
vn_read() at vn_read+0x88
dofileread() at dofileread+0x79
sys_pread() at sys_pread+0xa0
syscall() at syscall+0x283
--- syscall (number 173) ---
syscall+0x283:

>How-To-Repeat:
Thanks to msaitoh's efforts in
http://mail-index.netbsd.org/current-users/2020/06/27/msg038985.html
I can now boot a LOCKDEBUG kernel.

To try to find the underlying issue of PR kern/55362 (dd to raidframe
fine, dump to raidframe fine (apparently, might be luck), dump to cgd
on raidframe not fine with differences on the platters - full dmesg over there) I was running dd zero to each of the raid disks simultaneously and doing a pbulk run. Just found the box had panicked and rebooted.
>Fix:

>Release-Note:

>Audit-Trail:
From: Patrick Welche <prlw1@cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/55493: panic in radixtree
Date: Fri, 17 Jul 2020 12:48:29 +0100

 This time with line numbers to the source which was a github
 checkout d1534d74f37a40398e2a9cb8129e38ea170bccb7:

 (gdb) thread apply all bt

 Thread 2.1 (<kvm>):
 #0  0xffffffff80225835 in cpu_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/local/netbsd/src/sys/arch/amd64/amd64/machdep.c:713
 #1  0xffffffff80cc660f in kern_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/local/netbsd/src/sys/kern/kern_reboot.c:73
 #2  0xffffffff80d07f0e in vpanic (fmt=0xffffffff81354908 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=ap@entry=0xffff88829ff843b8) at /usr/src/local/netbsd/src/sys/kern/subr_prf.c:290
 #3  0xffffffff80e7e796 in kern_assert (fmt=fmt@entry=0xffffffff81354908 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at /usr/src/local/netbsd/src/sys/lib/libkern/kern_assert.c:51
 #4  0xffffffff80e6750d in radix_tree_lookup_ptr (tagmask=0, alloc=true, path=0x0, idx=761, t=0xffffad54ce539298) at /usr/src/local/netbsd/src/sys/lib/libkern/../../../common/lib/libc/gen/radixtree.c:671
 #5  radix_tree_insert_node (t=t@entry=0xffffad54ce539298, idx=idx@entry=761, p=p@entry=0xffff888017fbb280) at /usr/src/local/netbsd/src/sys/lib/libkern/../../../common/lib/libc/gen/radixtree.c:668
 #6  0xffffffff80c75f9b in uvm_pageinsert_tree (pg=0xffff888017fbb280, uobj=0xffffad54ce539280) at /usr/src/local/netbsd/src/sys/uvm/uvm_page.c:243
 #7  uvm_pagealloc_strat (obj=obj@entry=0xffffad54ce539280, off=off@entry=3117056, anon=anon@entry=0x0, flags=0, strat=<optimized out>, free_list=<optimized out>) at /usr/src/local/netbsd/src/sys/uvm/uvm_page.c:1311
 #8  0xffffffff80c5258b in uao_pagealloc (flags=0, offset=3117056, uobj=0xffffad54ce539280) at /usr/src/local/netbsd/src/sys/uvm/uvm_aobj.c:537
 #9  uao_get (uobj=0xffffad54ce539280, offset=<optimized out>, pps=0xffff88829ff84650, npagesp=<optimized out>, centeridx=<optimized out>, access_type=<optimized out>, advice=0, flags=6146) at /usr/src/local/netbsd/src/sys/uvm/uvm_aobj.c:954
 #10 0xffffffff80c561af in ubc_fault (ufi=0xffff88829ff84790, ign1=<optimized out>, ign2=<optimized out>, ign3=<optimized out>, ign4=<optimized out>, access_type=1, flags=<optimized out>) at /usr/src/local/netbsd/src/sys/uvm/uvm_bio.c:382
 #11 0xffffffff80c5c987 in uvm_fault_internal (orig_map=orig_map@entry=0xffffffff81915700 <kernel_map_store>, vaddr=vaddr@entry=18446612692453982208, access_type=access_type@entry=1, fault_flag=fault_flag@entry=0) at /usr/src/local/netbsd/src/sys/uvm/uvm_fault.c:918
 #12 0xffffffff802273c6 in trap (frame=0xffff88829ff84a30) at /usr/src/local/netbsd/src/sys/arch/amd64/amd64/trap.c:520
 #13 0xffffffff8022008d in alltraps ()
 #14 0xffffffff8021f943 in copyout ()
 #15 0xffffffff80cf0fc1 in copyout_vmspace (vm=<optimized out>, kaddr=<optimized out>, uaddr=<optimized out>, len=<optimized out>) at /usr/src/local/netbsd/src/sys/kern/subr_copy.c:251
 #16 0xffffffff80cf1098 in uiomove (buf=buf@entry=0xffff88826aee8000, n=14919, uio=uio@entry=0xffff88829ff84de0) at /usr/src/local/netbsd/src/sys/kern/subr_copy.c:130
 #17 0xffffffff80c56c21 in ubc_uiomove (uobj=uobj@entry=0xffffad54ce539280, uio=uio@entry=0xffff88829ff84de0, todo=14919, advice=advice@entry=0, flags=265) at /usr/src/local/netbsd/src/sys/uvm/uvm_bio.c:775
 #18 0xffffffff80b93aec in tmpfs_read (v=<optimized out>) at /usr/src/local/netbsd/src/sys/fs/tmpfs/tmpfs_vnops.c:553
 #19 0xffffffff80d7dbcf in VOP_READ (vp=vp@entry=0xffffad5234bd5340, uio=uio@entry=0xffff88829ff84de0, ioflag=ioflag@entry=0, cred=cred@entry=0xffffad50d222c640) at /usr/src/local/netbsd/src/sys/kern/vnode_if.c:504
 #20 0xffffffff80d74ddc in vn_read (fp=<optimized out>, offset=0xffff88829ff84e58, uio=0xffff88829ff84de0, cred=0xffffad50d222c640, flags=0) at /usr/src/local/netbsd/src/sys/kern/vfs_vnops.c:572
 #21 0xffffffff80d18bc8 in dofileread (fd=fd@entry=17, fp=fp@entry=0xffffad54347e7d80, buf=0x7465900ee750, nbyte=62932, offset=offset@entry=0xffff88829ff84e58, flags=flags@entry=0, retval=retval@entry=0xffff88829ff84eb0) at /usr/src/local/netbsd/src/sys/kern/sys_generic.c:156
 #22 0xffffffff80d6af9e in sys_pread (l=<optimized out>, uap=0xffff88829ff84f00, retval=0xffff88829ff84eb0) at /usr/src/local/netbsd/src/sys/kern/vfs_syscalls.c:2947
 #23 0xffffffff80553363 in sy_call (rval=0xffff88829ff84eb0, uap=0xffff88829ff84f00, l=0xffffad514e13a240, sy=0xffffffff81889df8 <sysent+4152>) at /usr/src/local/netbsd/src/sys/sys/syscallvar.h:65
 #24 sy_invoke (code=173, rval=0xffff88829ff84eb0, uap=0xffff88829ff84f00, l=0xffffad514e13a240, sy=0xffffffff81889df8 <sysent+4152>) at /usr/src/local/netbsd/src/sys/sys/syscallvar.h:94
 #25 syscall (frame=0xffff88829ff84f00) at /usr/src/local/netbsd/src/sys/arch/x86/x86/syscall.c:138
 #26 0xffffffff8020a22d in handle_syscall ()

From: "Chuck Silvers" <chs@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55493 CVS commit: src/sys/uvm
Date: Wed, 19 Aug 2020 15:36:41 +0000

 Module Name:	src
 Committed By:	chs
 Date:		Wed Aug 19 15:36:41 UTC 2020

 Modified Files:
 	src/sys/uvm: uvm_aobj.c

 Log Message:
 in uao_get(), if we unlock the uobj to read a page from swap,
 we must clear the cached page array because it is now stale.
 also add a missing call to uvm_page_array_fini() if the I/O fails.
 fixes PR 55493.


 To generate a diff of this commit:
 cvs rdiff -u -r1.150 -r1.151 src/sys/uvm/uvm_aobj.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: kern-bug-people->chs
Responsible-Changed-By: chs@NetBSD.org
Responsible-Changed-When: Thu, 20 Aug 2020 06:52:12 +0000
Responsible-Changed-Why:
I fixed this.



State-Changed-From-To: open->closed
State-Changed-By: chs@NetBSD.org
State-Changed-When: Thu, 20 Aug 2020 06:52:12 +0000
State-Changed-Why:
thanks for the dump, I could reproduce this once I knew how to arrange the race.


>Unformatted:

Home	PR Database Search