NetBSD Problem Report #37034

From martin@duskware.de  Fri Sep 28 08:23:36 2007
Return-Path: <martin@duskware.de>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by narn.NetBSD.org (Postfix) with ESMTP id 53A5563B8C2
	for <gnats-bugs@gnats.netbsd.org>; Fri, 28 Sep 2007 08:23:36 +0000 (UTC)
Message-Id: <20070928074316.DB92263B97E@narn.NetBSD.org>
Date: Fri, 28 Sep 2007 07:43:16 +0000 (UTC)
From: netbsd@eq.cz
Reply-To: netbsd@eq.cz
To: netbsd-bugs-owner@NetBSD.org
Subject: Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
X-Send-Pr-Version: www-1.0

>Number:         37034
>Category:       kern
>Synopsis:       Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Sep 28 08:25:01 +0000 2007
>Closed-Date:    Sat Sep 29 11:35:55 +0000 2007
>Last-Modified:  Thu Oct 04 18:45:01 +0000 2007
>Originator:     rudolf
>Release:        NetBSD 4.0_RC1 (netbsd-4 20070927)
>Organization:
>Environment:
NetBSD bludicka.crlf.cz 4.0_RC1 NetBSD 4.0_RC1 (GENERIC_debug) #0: Thu Sep 27 23:52:18 CEST 2007  root@bludicka.crlf.cz:/mnt/B/obj/sys/arch/amd64/compile/GENERIC_debug amd64
>Description:
I have this backup MSDOS img. I can access the DOS filesystem with 'ls'. I can it access with 'ls -al', but only once. The second time I try, the process gets stucked in vnlock.

The t/t of one of the hanging processes looks like this:
ltsleep()
acquire()
lockmgr()
VOP_LOCK()
vn_lock()
vget()
deget()
msdosfs_root()
lookup()
namei()
sys___lstat30()
syscall_plain()

I have a kernel dump from single-user with everything mounted read-only with one stucked process, I can provide it at request. The kernel is GENERIC with the following changes:
--- /usr/src/sys/arch/amd64/conf/GENERIC        2007-09-23 12:30:48.000000000 +0200
+++ /usr/src/sys/arch/amd64/conf/GENERIC_debug  2007-09-23 12:04:05.000000000 +0200
@@ -83,20 +83,20 @@
 #options       BUFQ_PRIOCSCAN

 # Diagnostic/debugging support options
-#options       DIAGNOSTIC      # expensive kernel consistency checks
-#options       DEBUG           # expensive debugging checks/support
-#options       KMEMSTATS       # kernel memory statistics (vmstat -m)
+options        DIAGNOSTIC      # expensive kernel consistency checks
+options        DEBUG           # expensive debugging checks/support
+options        KMEMSTATS       # kernel memory statistics (vmstat -m)

 #
 # Because gcc omits the frame pointer for any -O level, the line below
 # is needed to make backtraces in DDB work.
 #
-#makeoptions   COPTS="-O2 -fno-omit-frame-pointer"
-#options       DDB             # in-kernel debugger
-#options       DDB_HISTORY_SIZE=512    # enable history editing in DDB
+makeoptions    COPTS="-O2 -fno-omit-frame-pointer"
+options        DDB             # in-kernel debugger
+options        DDB_HISTORY_SIZE=512    # enable history editing in DDB
 #options       KGDB            # remote debugger
 #options       KGDB_DEVNAME="\"com\"",KGDB_DEVADDR=0x3f8,KGDB_DEVRATE=9600
-#makeoptions   DEBUG="-g"      # compile full symbol table
+makeoptions    DEBUG="-g"      # compile full symbol table

 # Compatibility options
 options        COMPAT_15       # compatibility with NetBSD 1.5,

>How-To-Repeat:
The dump I have was acquired like this:

boot -s
mount -r /dev/wd1e /mnt
cd /mnt/dir_with_dos_img
vnconfig -r vnd0 msdos.img
mount -r -t msdos /dev/vnd0a /mnt2
ls /mnt2/
=> this can be repeated without a stuck
ls -al /mnt2/
=> This time I get the listing
ls -al /mnt2/
=> stuck
CTRL+ALT+ESC
reboot 0x104
=> lockmgr: no context panic
reboot 0x104
=> actually made the dump

>Fix:

>Release-Note:

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
Date: Fri, 28 Sep 2007 10:42:32 +0200

 Could you try with a kernel that has

  options 	LOCKDEBUG

 please?

 This should produce better debugging output.

 Martin

From: rudolf <netbsd@eq.cz>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted
 from img (vnconfig)
Date: Fri, 28 Sep 2007 11:25:35 +0200

 Martin Husemann wrote:
 >  Could you try with a kernel that has
 >   options 	LOCKDEBUG
 >  please?
 >  
 >  This should produce better debugging output.

 OK, now I get a panic: lockmgr: locking against myself.

 The dumps are here:
 http://sw.eq.cz/tmp/

 I also forget to mention in the report that the filesystem in the img is 
 slightly corrupted. I don't know if this could cause this problem, under 
 Linux I can access the fs, just not each file in it.

 r.

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: netbsd@eq.cz
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
Date: Fri, 28 Sep 2007 14:31:14 +0200

 On Fri, Sep 28, 2007 at 10:30:09AM +0000, rudolf wrote:
 >  OK, now I get a panic: lockmgr: locking against myself.

 Can you include the full output please? It should tell you where it last
 locked and unlocked e.t.c. - also a "trace" output would be helpfull.

 Thanks,

 Martin

From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@NetBSD.org
Cc: netbsd@eq.cz
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
Date: Fri, 28 Sep 2007 16:09:52 +0300

 On Fri Sep 28 2007 at 10:30:09 +0000, rudolf wrote:
 >  I also forget to mention in the report that the filesystem in the img is 
 >  slightly corrupted. I don't know if this could cause this problem, under 
 >  Linux I can access the fs, just not each file in it.

 Yes, that would likely be the cause.  If you can share the image, that
 would be great.

 Meanwhile, try if this helps:

 Index: msdosfs_denode.c
 ===================================================================
 RCS file: /cvsroot/src/sys/fs/msdosfs/msdosfs_denode.c,v
 retrieving revision 1.17
 diff -p -u -r1.17 msdosfs_denode.c
 --- msdosfs_denode.c	25 Nov 2006 12:17:30 -0000	1.17
 +++ msdosfs_denode.c	28 Sep 2007 13:09:17 -0000
 @@ -335,8 +335,10 @@ deget(pmp, dirclust, diroffset, depp)
  		/* leave the other fields as garbage */
  	} else {
  		error = readep(pmp, dirclust, diroffset, &bp, &direntptr);
 -		if (error)
 +		if (error) {
 +			vput(nvp);
  			return (error);
 +		}
  		DE_INTERNALIZE(ldep, direntptr);
  		brelse(bp);
  	}

 -- 
 Antti Kantee <pooka@iki.fi>                     Of course he runs NetBSD
 http://www.iki.fi/pooka/                          http://www.NetBSD.org/
     "la qualité la plus indispensable du cuisinier est l'exactitude"

From: rudolf <netbsd@eq.cz>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted
 from img (vnconfig)
Date: Fri, 28 Sep 2007 16:35:16 +0200

 Antti Kantee wrote:
 >  On Fri Sep 28 2007 at 10:30:09 +0000, rudolf wrote:
 >  >  I also forget to mention in the report that the filesystem in the img is 
 >  >  slightly corrupted. I don't know if this could cause this problem, under 
 >  >  Linux I can access the fs, just not each file in it.
 >  
 >  Yes, that would likely be the cause.  If you can share the image, that
 >  would be great.

 No problem, but it is quite big: 16 GB uncompressed. If you'll need it, 
 we can arrange it somehow.

 >  Meanwhile, try if this helps:
 [...]

 With this patch I could do 'ls -al /top_of_msdosfs/', 'find 
 /top_of_msdosfs/', ... and everything was ok. Then 'shutdown -r now' 
 caused this panic:

 syncing disks... done
 unmounting file systems...
 unmounting /mnt2 (/dev/vnd0a)... vrele: bad ref count: tag VT_UFS(1) 
 type VBLK(3), usecount -1, writecount 0, refcount 1858, flags(0<LOCKSWORK>)
 tag VT_UFS, ino 184765, on dev 0, 32 flags 0x0, effnlink 1, nlink 1
 mode 060640, owner 0, group 5, size 0 not locked
 panic: vrele: ref cnt vp 0xffff80004c203e20
 Stopped in pid 60.1 (reboot) at netbsd:cpu_Debugger+0x5: leave

 I have a core dump from this one too.

 r.

From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@NetBSD.org
Cc: netbsd@eq.cz
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
Date: Fri, 28 Sep 2007 18:00:25 +0300

 On Fri Sep 28 2007 at 14:40:02 +0000, rudolf wrote:
 >  No problem, but it is quite big: 16 GB uncompressed. If you'll need it, 
 >  we can arrange it somehow.

 Ok, let's try to avoid that.

 >  >  Meanwhile, try if this helps:
 >  [...]
 >  
 >  With this patch I could do 'ls -al /top_of_msdosfs/', 'find 
 >  /top_of_msdosfs/', ... and everything was ok. Then 'shutdown -r now' 
 >  caused this panic:
 >  
 >  syncing disks... done
 >  unmounting file systems...
 >  unmounting /mnt2 (/dev/vnd0a)... vrele: bad ref count: tag VT_UFS(1) 
 >  type VBLK(3), usecount -1, writecount 0, refcount 1858, flags(0<LOCKSWORK>)
 >  tag VT_UFS, ino 184765, on dev 0, 32 flags 0x0, effnlink 1, nlink 1
 >  mode 060640, owner 0, group 5, size 0 not locked
 >  panic: vrele: ref cnt vp 0xffff80004c203e20
 >  Stopped in pid 60.1 (reboot) at netbsd:cpu_Debugger+0x5: leave

 Ah, brilliant.  I'm not familiar enough with msdosfs to see the thread
 logic in the function, but this should fix that:

 Index: msdosfs_denode.c
 ===================================================================
 RCS file: /cvsroot/src/sys/fs/msdosfs/msdosfs_denode.c,v
 retrieving revision 1.17
 diff -p -u -r1.17 msdosfs_denode.c
 --- msdosfs_denode.c	25 Nov 2006 12:17:30 -0000	1.17
 +++ msdosfs_denode.c	28 Sep 2007 14:59:35 -0000
 @@ -335,8 +335,11 @@ deget(pmp, dirclust, diroffset, depp)
  		/* leave the other fields as garbage */
  	} else {
  		error = readep(pmp, dirclust, diroffset, &bp, &direntptr);
 -		if (error)
 +		if (error) {
 +			ldep->de_devvp = NULL;
 +			vput(nvp);
  			return (error);
 +		}
  		DE_INTERNALIZE(ldep, direntptr);
  		brelse(bp);
  	}

 -- 
 Antti Kantee <pooka@iki.fi>                     Of course he runs NetBSD
 http://www.iki.fi/pooka/                          http://www.NetBSD.org/
     "la qualité la plus indispensable du cuisinier est l'exactitude"

From: rudolf <netbsd@eq.cz>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted
 from img (vnconfig)
Date: Fri, 28 Sep 2007 18:15:21 +0200

 Antti Kantee wrote:
 >  On Fri Sep 28 2007 at 14:40:02 +0000, rudolf wrote:
 [...]
 >  >  With this patch I could do 'ls -al /top_of_msdosfs/', 'find 
 >  >  /top_of_msdosfs/', ... and everything was ok. Then 'shutdown -r now' 
 >  >  caused this panic:
 [...]
 >  Ah, brilliant.  I'm not familiar enough with msdosfs to see the thread
 >  logic in the function, but this should fix that:
 [...]

 :-) Are you a magician?

 I can see no more problems with this patch. Thanks.

 r.

From: Antti Kantee <pooka@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: PR/37034 CVS commit: src/sys/fs/msdosfs
Date: Fri, 28 Sep 2007 19:48:43 +0000 (UTC)

 Module Name:	src
 Committed By:	pooka
 Date:		Fri Sep 28 19:48:43 UTC 2007

 Modified Files:
 	src/sys/fs/msdosfs: msdosfs_denode.c

 Log Message:
 Release vnode being created in error branch instead of leaving it
 locked.  Ideally the function should be rewritten to do things in
 a different order, but this tries to keep changes minimal aiming
 for a possible netbsd-4 pullup.

 fixes PR kern/37034


 To generate a diff of this commit:
 cvs rdiff -r1.23 -r1.24 src/sys/fs/msdosfs/msdosfs_denode.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: pooka@netbsd.org
State-Changed-When: Fri, 28 Sep 2007 22:56:49 +0300
State-Changed-Why:
fix committed to -current


From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@NetBSD.org
Cc: netbsd@eq.cz
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted from img (vnconfig)
Date: Fri, 28 Sep 2007 22:56:22 +0300

 On Fri Sep 28 2007 at 16:20:02 +0000, rudolf wrote:
 >  :-) Are you a magician?

 No, I just sacrifice a lot of chickens.  A good way is to stuff truffle
 shavings under the skin and rub the chicken liberally with butter.  Then
 truss it.  Roast until golden brown & fully cooked (but not overdone!).

 >  I can see no more problems with this patch. Thanks.

 Nice.  Does this mean it fully works now, i.e. you can access the files?

 Could you please test this patch (which I committed to -current).  I made
 two more changes and I'd like to be sure it works before issuing a pullup
 request to netbsd-4.

 Index: msdosfs_denode.c
 ===================================================================
 RCS file: /cvsroot/src/sys/fs/msdosfs/msdosfs_denode.c,v
 retrieving revision 1.17
 diff -p -u -r1.17 msdosfs_denode.c
 --- msdosfs_denode.c	25 Nov 2006 12:17:30 -0000	1.17
 +++ msdosfs_denode.c	28 Sep 2007 19:50:20 -0000
 @@ -291,6 +291,7 @@ deget(pmp, dirclust, diroffset, depp)
  	 * need to it.
  	 */
  	vn_lock(nvp, LK_EXCLUSIVE | LK_RETRY);
 +	genfs_node_init(nvp, &msdosfs_genfsops);
  	msdosfs_hashins(ldep);

  	ldep->de_pmp = pmp;
 @@ -335,8 +336,12 @@ deget(pmp, dirclust, diroffset, depp)
  		/* leave the other fields as garbage */
  	} else {
  		error = readep(pmp, dirclust, diroffset, &bp, &direntptr);
 -		if (error)
 +		if (error) {
 +			ldep->de_devvp = NULL;
 +			ldep->de_Name[0] = SLOT_DELETED;
 +			vput(nvp);
  			return (error);
 +		}
  		DE_INTERNALIZE(ldep, direntptr);
  		brelse(bp);
  	}
 @@ -365,7 +370,6 @@ deget(pmp, dirclust, diroffset, depp)
  		}
  	} else
  		nvp->v_type = VREG;
 -	genfs_node_init(nvp, &msdosfs_genfsops);
  	VREF(ldep->de_devvp);
  	*depp = ldep;
  	nvp->v_size = ldep->de_FileSize;

 -- 
 Antti Kantee <pooka@iki.fi>                     Of course he runs NetBSD
 http://www.iki.fi/pooka/                          http://www.NetBSD.org/
     "la qualité la plus indispensable du cuisinier est l'exactitude"

From: rudolf <netbsd@eq.cz>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/37034: Processes stuck on vnlock: accessing msdos fs mounted
 from img (vnconfig)
Date: Fri, 28 Sep 2007 23:24:45 +0200

 Antti Kantee wrote:
 >  On Fri Sep 28 2007 at 16:20:02 +0000, rudolf wrote:
 >  >  :-) Are you a magician?
 >  
 >  No, I just sacrifice a lot of chickens.  A good way is to stuff truffle
 >  shavings under the skin and rub the chicken liberally with butter.  Then
 >  truss it.  Roast until golden brown & fully cooked (but not overdone!).

 :-)

 >  >  I can see no more problems with this patch. Thanks.
 >  
 >  Nice.  Does this mean it fully works now, i.e. you can access the files?
 >  
 >  Could you please test this patch (which I committed to -current).  I made
 >  two more changes and I'd like to be sure it works before issuing a pullup
 >  request to netbsd-4.
 [...]

 Yes, it works also with this last patch, I can access and read the 
 files. No stucked processes, umount without problems. Upon access to the 
 corrupted files I get "Invalid argument" and that's ok, I don't expect 
 them to be accessible. Thanks again.

 r.

From: Juan Romero Pardines <xtraeme@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: PR/37034 CVS commit: [netbsd-4] src/sys/fs/msdosfs
Date: Sat, 29 Sep 2007 11:33:36 +0000 (UTC)

 Module Name:	src
 Committed By:	xtraeme
 Date:		Sat Sep 29 11:33:36 UTC 2007

 Modified Files:
 	src/sys/fs/msdosfs [netbsd-4]: msdosfs_denode.c

 Log Message:
 Pull up following revision(s) (requested by pooka in ticket #909):
 	sys/fs/msdosfs/msdosfs_denode.c: revision 1.24

 Release vnode being created in error branch instead of leaving it
 locked.  Ideally the function should be rewritten to do things in
 a different order, but this tries to keep changes minimal aiming
 for a possible netbsd-4 pullup.
 fixes PR kern/37034


 To generate a diff of this commit:
 cvs rdiff -r1.17 -r1.17.2.1 src/sys/fs/msdosfs/msdosfs_denode.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: feedback->closed
State-Changed-By: pooka@netbsd.org
State-Changed-When: Sat, 29 Sep 2007 14:35:55 +0300
State-Changed-Why:
Submitter confirms fixed.

thanks for excellent testing feedback!


From: Manuel Bouyer <bouyer@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: PR/37034 CVS commit: [netbsd-3] src/sys/fs/msdosfs
Date: Thu,  4 Oct 2007 18:43:58 +0000 (UTC)

 Module Name:	src
 Committed By:	bouyer
 Date:		Thu Oct  4 18:43:58 UTC 2007

 Modified Files:
 	src/sys/fs/msdosfs [netbsd-3]: msdosfs_denode.c

 Log Message:
 Pull up following revision(s) (requested by pooka in ticket #1849):
 	sys/fs/msdosfs/msdosfs_denode.c: revision 1.24
 Release vnode being created in error branch instead of leaving it
 locked.  Ideally the function should be rewritten to do things in
 a different order, but this tries to keep changes minimal aiming
 for a possible netbsd-4 pullup.
 fixes PR kern/37034


 To generate a diff of this commit:
 cvs rdiff -r1.7.10.1 -r1.7.10.2 src/sys/fs/msdosfs/msdosfs_denode.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.