NetBSD Problem Report #26878

From merlin.buzzard.freeserve.co.uk
Received: (qmail 11782 invoked by uid 605); 7 Sep 2004 23:32:39 -0000
Message-Id: <200409071943.i87JhBfm001313@osprey.buzzard.freeserve.co.uk>
Date: Tue, 7 Sep 2004 20:43:12 +0100 (BST)
From: Richard Earnshaw <rearnsha@buzzard.freeserve.co.uk>
Sender: gnats-bugs-owner@NetBSD.org
Reply-To: rearnsha@buzzard.freeserve.co.uk
To: gnats-bugs@gnats.NetBSD.org
Subject: FFSv2 + softdep = livelock (no free ram)
X-Send-Pr-Version: 3.95

>Number:         26878
>Category:       kern
>Synopsis:       FFSv2 + softdep = livelock (no free ram)
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Sep 07 23:33:00 +0000 2004
>Closed-Date:    Wed Apr 01 04:08:27 +0000 2009
>Last-Modified:  Wed Apr 01 04:08:27 +0000 2009
>Originator:     Richard Earnshaw
>Release:        NetBSD 2.0_BETA
>Organization:
None.

>Environment:


System: NetBSD osprey.buzzard.freeserve.co.uk 2.0_BETA NetBSD 2.0_BETA (OSPREY) #0: Thu Aug 5 20:31:21 BST 2004 rearnsha@merlin.buzzard.freeserve.co.uk:/work/rearnsha/netbsd/build/cats/src/sys/arch/cats/compile/OSPREY cats
	Physical RAM 128M.  HDD: 80G Maxtor running in UDMA-1 mode.
Architecture: arm
Machine: cats
>Description:
	creating a FFSv2 filesystem and then mounting it with soft 
	dependencies can lead to the situation where the kernel runs out of
	free memory if there is heavy disk activity (eg untarring a large
	amount of data).  The system isn't truly dead (top continues to
	run), but shows that there is only ~4k (1 page) of memory on the
	free list.  However, no new processes can be started and it's 
	impossible to kill any existing ones.

>How-To-Repeat:
	As above.

>Fix:
	No idea.  Probably should ignore softdep flag if FFSv2 mode for now.

>Release-Note:
>Audit-Trail:

From: enami tsugutomo <enami@sm.sony.co.jp>
To: rearnsha@buzzard.freeserve.co.uk
Cc: gnats-bugs@gnats.NetBSD.org
Subject: Re: kern/26878: FFSv2 + softdep = livelock (no free ram)
Date: 04 Oct 2004 10:34:35 +0900

 Richard Earnshaw <rearnsha@buzzard.freeserve.co.uk> writes:

 > >Description:
 > 	creating a FFSv2 filesystem and then mounting it with soft 
 > 	dependencies can lead to the situation where the kernel runs out of
 > 	free memory if there is heavy disk activity (eg untarring a large
 > 	amount of data).  The system isn't truly dead (top continues to
 > 	run), but shows that there is only ~4k (1 page) of memory on the
 > 	free list.  However, no new processes can be started and it's 
 > 	impossible to kill any existing ones.

 I suspect this is same or related to PR#27057.

 enami.

From: Richard Earnshaw <Richard.Earnshaw@buzzard.freeserve.co.uk>
To: enami tsugutomo <enami@sm.sony.co.jp>
Cc: rearnsha@buzzard.freeserve.co.uk, gnats-bugs@gnats.NetBSD.org
Subject: Re: kern/26878: FFSv2 + softdep = livelock (no free ram) 
Date: Tue, 12 Oct 2004 23:27:00 +0100

 On 04 Oct 2004 10:34:35 +0900, enami tsugutomo wrote:
 > Richard Earnshaw <rearnsha@buzzard.freeserve.co.uk> writes:
 > 
 > > >Description:
 > > 	creating a FFSv2 filesystem and then mounting it with soft 
 > > 	dependencies can lead to the situation where the kernel runs out of
 > > 	free memory if there is heavy disk activity (eg untarring a large
 > > 	amount of data).  The system isn't truly dead (top continues to
 > > 	run), but shows that there is only ~4k (1 page) of memory on the
 > > 	free list.  However, no new processes can be started and it's 
 > > 	impossible to kill any existing ones.
 > 
 > I suspect this is same or related to PR#27057.
 > 
 > enami.

 I've tried the changes that were pulled up onto the 2.0 branch in relation 
 to that PR, but it doesn't solve a hang on my shark (with a FFSv1+softdep 
 filesystem).

 Also I've noted that with a similar vintage kernel on my PC the system 
 sometimes goes catatonic for a few seconds (even the mouse won't move) 
 before a burst of file-system activity.

 Is there a pull-up missing?

 R.


From: Andrew Doran <ad@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/26878 CVS commit: src
Date: Sun, 22 Feb 2009 20:28:07 +0000 (UTC)

 Module Name:	src
 Committed By:	ad
 Date:		Sun Feb 22 20:28:07 UTC 2009

 Modified Files:
 	src/doc: CHANGES
 	src/lib/libp2k: p2k.c
 	src/sbin/fsck_lfs: lfs.c
 	src/sbin/mount: mount.8
 	src/sbin/newfs_lfs: make_lfs.c
 	src/sbin/tunefs: tunefs.8 tunefs.c
 	src/sys/arch/vax/conf: VAX780
 	src/sys/conf: files
 	src/sys/kern: sys_aio.c vfs_bio.c vfs_subr.c vfs_syscalls.c
 	src/sys/miscfs/specfs: spec_vnops.c
 	src/sys/miscfs/syncfs: sync_subr.c
 	src/sys/modules/ffs: Makefile
 	src/sys/rump/fs/lib/libffs: Makefile
 	src/sys/rump/include/rump: rump.h
 	src/sys/rump/librump/rumpvfs: rump_vfs.c vm_vfs.c
 	src/sys/sys: buf.h vnode.h
 	src/sys/ufs: files.ufs
 	src/sys/ufs/ffs: ffs_alloc.c ffs_balloc.c ffs_extern.h ffs_inode.c
 	    ffs_snapshot.c ffs_vfsops.c ffs_vnops.c ffs_wapbl.c
 	src/sys/ufs/lfs: lfs_rfw.c lfs_vfsops.c lfs_vnops.c
 	src/sys/ufs/ufs: inode.h ufs_dirhash.c ufs_extern.h ufs_inode.c
 	    ufs_lookup.c ufs_readwrite.c ufs_vnops.c ufs_wapbl.c
 	src/sys/uvm: uvm_pager.c
 Removed Files:
 	src/sys/rump/librump/rumpkern/opt: opt_softdep.h
 	src/sys/ufs/ffs: ffs_softdep.c ffs_softdep.stub.c softdep.h

 Log Message:
 PR kern/26878 FFSv2 + softdep = livelock (no free ram)
 PR kern/16942 panic with softdep and quotas
 PR kern/19565 panic: softdep_write_inodeblock: indirect pointer #1 mismatch
 PR kern/26274 softdep panic: allocdirect_merge: ...
 PR kern/26374 Long delay before non-root users can write to softdep partitions
 PR kern/28621 1.6.x "vp != NULL" panic in ffs_softdep.c:4653 while unmounting a softdep (+quota) filesystem
 PR kern/29513 FFS+Softdep panic with unfsck-able file-corruption
 PR kern/31544 The ffs softdep code appears to fail to write dirty bits to disk
 PR kern/31981 stopping scsi disk can cause panic (softdep)
 PR kern/32116 kernel panic in softdep (assertion failure)
 PR kern/32532 softdep_trackbufs deadlock
 PR kern/37191 softdep: locking against myself
 PR kern/40474 Kernel panic after remounting raid root with softdep

 Retire softdep, pass 2. As discussed and later formally announced on the
 mailing lists.


 To generate a diff of this commit:
 cvs rdiff -r1.1191 -r1.1192 src/doc/CHANGES
 cvs rdiff -r1.8 -r1.9 src/lib/libp2k/p2k.c
 cvs rdiff -r1.29 -r1.30 src/sbin/fsck_lfs/lfs.c
 cvs rdiff -r1.65 -r1.66 src/sbin/mount/mount.8
 cvs rdiff -r1.13 -r1.14 src/sbin/newfs_lfs/make_lfs.c
 cvs rdiff -r1.37 -r1.38 src/sbin/tunefs/tunefs.8 src/sbin/tunefs/tunefs.c
 cvs rdiff -r1.1 -r1.2 src/sys/arch/vax/conf/VAX780
 cvs rdiff -r1.942 -r1.943 src/sys/conf/files
 cvs rdiff -r1.22 -r1.23 src/sys/kern/sys_aio.c
 cvs rdiff -r1.215 -r1.216 src/sys/kern/vfs_bio.c
 cvs rdiff -r1.368 -r1.369 src/sys/kern/vfs_subr.c
 cvs rdiff -r1.388 -r1.389 src/sys/kern/vfs_syscalls.c
 cvs rdiff -r1.122 -r1.123 src/sys/miscfs/specfs/spec_vnops.c
 cvs rdiff -r1.36 -r1.37 src/sys/miscfs/syncfs/sync_subr.c
 cvs rdiff -r1.2 -r1.3 src/sys/modules/ffs/Makefile
 cvs rdiff -r1.6 -r1.7 src/sys/rump/fs/lib/libffs/Makefile
 cvs rdiff -r1.9 -r1.10 src/sys/rump/include/rump/rump.h
 cvs rdiff -r1.1 -r0 src/sys/rump/librump/rumpkern/opt/opt_softdep.h
 cvs rdiff -r1.12 -r1.13 src/sys/rump/librump/rumpvfs/rump_vfs.c
 cvs rdiff -r1.3 -r1.4 src/sys/rump/librump/rumpvfs/vm_vfs.c
 cvs rdiff -r1.110 -r1.111 src/sys/sys/buf.h
 cvs rdiff -r1.200 -r1.201 src/sys/sys/vnode.h
 cvs rdiff -r1.18 -r1.19 src/sys/ufs/files.ufs
 cvs rdiff -r1.121 -r1.122 src/sys/ufs/ffs/ffs_alloc.c
 cvs rdiff -r1.51 -r1.52 src/sys/ufs/ffs/ffs_balloc.c
 cvs rdiff -r1.74 -r1.75 src/sys/ufs/ffs/ffs_extern.h
 cvs rdiff -r1.102 -r1.103 src/sys/ufs/ffs/ffs_inode.c
 cvs rdiff -r1.91 -r1.92 src/sys/ufs/ffs/ffs_snapshot.c
 cvs rdiff -r1.116 -r0 src/sys/ufs/ffs/ffs_softdep.c
 cvs rdiff -r1.23 -r0 src/sys/ufs/ffs/ffs_softdep.stub.c
 cvs rdiff -r1.242 -r1.243 src/sys/ufs/ffs/ffs_vfsops.c
 cvs rdiff -r1.110 -r1.111 src/sys/ufs/ffs/ffs_vnops.c
 cvs rdiff -r1.11 -r1.12 src/sys/ufs/ffs/ffs_wapbl.c
 cvs rdiff -r1.11 -r0 src/sys/ufs/ffs/softdep.h
 cvs rdiff -r1.11 -r1.12 src/sys/ufs/lfs/lfs_rfw.c
 cvs rdiff -r1.269 -r1.270 src/sys/ufs/lfs/lfs_vfsops.c
 cvs rdiff -r1.219 -r1.220 src/sys/ufs/lfs/lfs_vnops.c
 cvs rdiff -r1.55 -r1.56 src/sys/ufs/ufs/inode.h
 cvs rdiff -r1.27 -r1.28 src/sys/ufs/ufs/ufs_dirhash.c
 cvs rdiff -r1.60 -r1.61 src/sys/ufs/ufs/ufs_extern.h
 cvs rdiff -r1.77 -r1.78 src/sys/ufs/ufs/ufs_inode.c
 cvs rdiff -r1.100 -r1.101 src/sys/ufs/ufs/ufs_lookup.c
 cvs rdiff -r1.93 -r1.94 src/sys/ufs/ufs/ufs_readwrite.c
 cvs rdiff -r1.172 -r1.173 src/sys/ufs/ufs/ufs_vnops.c
 cvs rdiff -r1.4 -r1.5 src/sys/ufs/ufs/ufs_wapbl.c
 cvs rdiff -r1.93 -r1.94 src/sys/uvm/uvm_pager.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Wed, 01 Apr 2009 04:08:27 +0000
State-Changed-Why:
softdep (softupdates) has been removed.


>Unformatted:

Home	PR Database Search