NetBSD Problem Report #58430

From gson@gson.org  Mon Jul 15 16:47:10 2024
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits)
	 client-signature RSA-PSS (2048 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 280031A9239
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 15 Jul 2024 16:47:10 +0000 (UTC)
Message-Id: <20240715164702.652DD253EF9@guava.gson.org>
Date: Mon, 15 Jul 2024 19:47:02 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: Panic in vdead_check()
X-Send-Pr-Version: 3.95

>Number:         58430
>Category:       kern
>Synopsis:       Panic in vdead_check()
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jul 15 16:50:00 +0000 2024
>Last-Modified:  Tue Jul 16 07:55:01 +0000 2024
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current, source date 2024-07-01 17:47:24 Z
>Organization:
>Environment:
System: NetBSD
Architecture: i386
Machine: i386
>Description:

One of my NetBSD systems paniced and dumped a kernel core.
The backtrace shown by gdb is:

#0  0xc012ce85 in maybe_dump (howto=260) at /usr/src/sys/arch/i386/i386/machdep.c:725
#1  cpu_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/sys/arch/i386/i386/machdep.c:746
#2  0xc0c62338 in kern_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /usr/src/sys/kern/kern_reboot.c:91
#3  0xc0ca74a9 in vpanic (fmt=fmt@entry=0xc11b1054 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=0xe080ce98 "\273\017\033\301uo(\301O++\301+\b") at /usr/src/sys/kern/subr_prf.c:288
#4  0xc0e6e21c in kern_assert (fmt=<optimized out>, fmt@entry=0xc11b1054 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at /usr/src/sys/lib/libkern/kern_assert.c:51
#5  0xc0d1da69 in vdead_check (vp=vp@entry=0xc4b70000, flags=flags@entry=1) at /usr/src/sys/kern/vfs_vnode.c:2091
#6  0xc0d0716e in vfs_vnode_iterator_next1 (vni=0xcdb53000, f=f@entry=0xc0b9dd82 <ffs_sync_selector>, cl=cl@entry=0xe080cf2c, do_wait=false) at /usr/src/sys/kern/vfs_mount.c:508
#7  0xc0d080f9 in vfs_vnode_iterator_next (vni=<optimized out>, f=f@entry=0xc0b9dd82 <ffs_sync_selector>, cl=cl@entry=0xe080cf2c) at /usr/src/sys/kern/vfs_mount.c:530
#8  0xc0ba1c09 in ffs_sync (mp=0xc4b28000, waitfor=3, cred=0xc3d23040) at /usr/src/sys/ufs/ffs/ffs_vfsops.c:1996
#9  0xc0d0ce4a in VFS_SYNC (mp=0xc4b28000, a=3, b=0xc3d23040) at /usr/src/sys/kern/vfs_subr.c:1587
#10 0xc0d0cf50 in sched_sync (arg=0xc441b240) at ./machine/cpu.h:77
#11 0xc0102011 in lwp_trampoline ()

I can run additional gdb commands on request.

>How-To-Repeat:

>Fix:

>Audit-Trail:
From: "J. Hannken-Illjes" <hannken@mailbox.org>
To: NetBSD GNATS <gnats-bugs@netbsd.org>
Cc: 
Subject: Re: kern/58430: Panic in vdead_check()
Date: Mon, 15 Jul 2024 21:57:21 +0200

 If I get the trace right we have #6:

    506                  mutex_enter(vp->v_interlock);
    507                  if (vnis_marker(vp) ||
    508                      vdead_check(vp, (do_wait ? 0 : VDEAD_NOWAIT)) ||

 and #5:

   2087  int
   2088  vdead_check(struct vnode *vp, int flags)
   2089  {
   2090
   2091          KASSERT(mutex_owned(vp->v_interlock));

 I have absolute no idea why this assertion fires.  Could print as much
 as possible from this (0xc4b70000) vnode as "struct vnode_impl *"?

 --
 J. Hannken-Illjes - hannken@mailbox.org

From: Andreas Gustafsson <gson@gson.org>
To: "J. Hannken-Illjes" <hannken@mailbox.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/58430: Panic in vdead_check()
Date: Tue, 16 Jul 2024 09:11:46 +0300

 J. Hannken-Illjes wrote:
 >  I have absolute no idea why this assertion fires.  Could print as much
 >  as possible from this (0xc4b70000) vnode as "struct vnode_impl *"?

 See below.  If you need more details, please send the specific gdb
 commands you would like me to run.
 -- 
 Andreas Gustafsson, gson@gson.org

 (gdb) frame 5
 #5  0xc0d1da69 in vdead_check (vp=vp@entry=0xc4b70000, flags=flags@entry=1) at /usr/src/sys/kern/vfs_vnode.c:2091
 (gdb) set print pretty on
 (gdb) p /x (struct vnode_impl *) vp
 $6 = 0xc4b70000
 (gdb) p /x *((struct vnode_impl *) vp)
 $7 = {
   vi_vnode = {
     v_uobj = {
       vmobjlock = 0x84cb3fcd,
       pgops = 0x2d00bb01,
       uo_npages = 0x8172803c,
       uo_refs = 0x1d22002d,
       uo_pages = {
         t_root = 0xcb67255,
         t_height = 0xa96da933
       },
       uo_ubc = {
         lh_first = 0x427fbb78
       }
     },
     v_size = 0xc5d72e64acb72314,
     v_writesize = 0xeb773e7855c35c8a,
     v_cv = {
       cv_opaque = {0xde483000, 0xbc1a9ff8}
     },
     v_iflag = 0xce0295f6,
     v_uflag = 0x450008,
     v_usecount = 0x4100,
     v_numoutput = 0x11400040,
     v_writecount = 0xa84c3,
     v_holdcnt = 0xf09df801,
     v_cleanblkhd = {
       lh_first = 0x84cb3fcd
     },
     v_dirtyblkhd = {
       lh_first = 0x2d00bb01
     },
     v_vflag = 0xce9a01be,
     v_interlock = 0xdfed5098,
     v_mount = 0xf49ccf3b,
     v_op = 0xeeefc38c,
     v_un = {
       vu_mountedhere = 0xca0cb5,
       vu_socket = 0xca0cb5,
       vu_specnode = 0xca0cb5,
       vu_fifoinfo = 0xca0cb5,
       vu_ractx = 0xca0cb5
     },
     v_type = 0xee000000,
     v_tag = 0x535017,
     v_data = 0x0,
     v_klist = 0xde483000,
     v_segvguard = 0xbc1a9ff8
   },
   vi_key = {
     vk_mount = 0x84cb3fcd,
     vk_key = 0x2d00bb01,
     vk_key_len = 0x8166eb2a
   },
   vi_klist = {
     vk_klist = {
       slh_first = 0x1d22002d
     },
     vk_interest = 0x25b67255
   },
   vi_lrulisthd = 0x1b9396cb,
   vi_lrulist = {
     tqe_next = 0xc02795d8,
     tqe_prev = 0x53d1d21b
   },
   vi_synclist_slot = 0xc0d90a53,
   vi_lrulisttm = 0xf956313d,
   vi_synclist = {
     tqe_next = 0x5ac8146,
     tqe_prev = 0x94ff5ed0
   },
   vi_hash = {
     sle_next = 0x44a49d
   },
   vi_state = 0x94000000,
   vi_mntvnodes = {
     tqe_next = 0x535017,
     tqe_prev = 0x0
   },
   vi_nc_tree = {
     rbt_root = 0xde483000,
     rbt_ops = 0xbc1a9ff8,
     rbt_minmax = {0xce0295f6, 0x450008}
   },
   vi_nc_list = {
     tqh_first = 0x4100,
     tqh_last = 0x11400040
   },
   vi_nc_mode = 0xa84c3,
   vi_nc_uid = 0xf09df801,
   vi_nc_gid = 0x84cb3fcd,
   vi_nc_spare = 0x2d00bb01,
   vi_lock = {
     rw_owner = 0xfda2dc86
   },
   vi_nc_lock = {
     rw_owner = 0x84cb3fcd
   },
   vi_nc_listlock = {
     rw_owner = 0x2d00bb01
   }
 }
 (gdb) p vp->v_interlock
 $9 = (kmutex_t *) 0xdfed5098
 (gdb) p *vp->v_interlock
 Cannot access memory at address 0xdfed5098

From: "J. Hannken-Illjes" <hannken@mailbox.org>
To: NetBSD GNATS <gnats-bugs@netbsd.org>
Cc: 
Subject: Re: kern/58430: Panic in vdead_check()
Date: Tue, 16 Jul 2024 09:54:02 +0200

 > On 16. Jul 2024, at 08:15, Andreas Gustafsson <gson@gson.org> wrote:
 > 
 > The following reply was made to PR kern/58430; it has been noted by GNATS.
 > 
 > From: Andreas Gustafsson <gson@gson.org>
 > To: "J. Hannken-Illjes" <hannken@mailbox.org>
 > Cc: gnats-bugs@netbsd.org
 > Subject: Re: kern/58430: Panic in vdead_check()
 > Date: Tue, 16 Jul 2024 09:11:46 +0300
 > 
 > J. Hannken-Illjes wrote:
 >> I have absolute no idea why this assertion fires.  Could print as much
 >> as possible from this (0xc4b70000) vnode as "struct vnode_impl *"?
 > 
 > See below.  If you need more details, please send the specific gdb
 > commands you would like me to run.

 This vnode has been freed and its memory reused.
 I have no idea where we lost the vnode -- no more debugging possible here.

 --
 J. Hannken-Illjes - hannken@mailbox.org

Home	PR Database Search