NetBSD Problem Report #45982

From www@NetBSD.org  Sat Feb 11 06:14:35 2012
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id EE1E263E04C
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 11 Feb 2012 06:14:34 +0000 (UTC)
Message-Id: <20120211061434.449AE63E043@www.NetBSD.org>
Date: Sat, 11 Feb 2012 06:14:34 +0000 (UTC)
From: perseant@hhhh.org
Reply-To: perseant@hhhh.org
To: gnats-bugs@NetBSD.org
Subject: LFS metadata space estimate can cause false "filesystem full" condition
X-Send-Pr-Version: www-1.0

>Number:         45982
>Category:       kern
>Synopsis:       LFS metadata space estimate can cause false "filesystem full" condition
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Feb 11 06:15:01 +0000 2012
>Closed-Date:    Sun Mar 18 15:44:48 +0000 2012
>Last-Modified:  Sun Mar 18 15:44:48 +0000 2012
>Originator:     Konrad Schroder
>Release:        5.99.64 userland (including RUMP kernel) on 5.99.61 host
>Organization:
>Environment:
NetBSD wetbar.hitl.washington.edu 5.99.61 NetBSD 5.99.61 (GENERIC) #1: Sat Jan 28 13:57:59 PST 2012  perseant@wetbar.hitl.washington.edu:/alt/src-current/sys/arch/i386/compile/obj.i386/GENERIC i386

>Description:
Because LFS allocates metadata, specifically inode blocks, on an as-needed basis rather than pre-allocating them the way FFS does, it estimates how much of the rest of the disk *would* be filled with inode blocks, segment headers, and other metadata if usage continues the same as it has for the disk thus far, compared to the number of data blocks.

If the proportion of inode blocks is extreme---as it is for the t_renamerace test---the filesystem can report "full" even when there is almost nothing actually stored on the disk.
>How-To-Repeat:
Fix the crashy part of t_renamerace (see kern/43582), and then run "t_renamerace lfs_renamerace_dirs".
>Fix:
Use nonlinear mixing of the actual metadata use with the projection. When  the disk is empty, the sample size that generates the projection is small, and we should avoid reporting disk full; but when the disk approaches capacity, the projection is more accurate (assuming a uniform pattern of use) and it is more important to avoid reporting free space erroneously.

>Release-Note:

>Audit-Trail:
From: "Konrad Schroder" <perseant@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/45982 CVS commit: src
Date: Thu, 16 Feb 2012 02:47:56 +0000

 Module Name:	src
 Committed By:	perseant
 Date:		Thu Feb 16 02:47:56 UTC 2012

 Modified Files:
 	src/sbin/newfs_lfs: make_lfs.c
 	src/sys/ufs/lfs: lfs.h lfs_alloc.c lfs_bio.c lfs_segment.c lfs_vfsops.c
 	    lfs_vnops.c
 	src/tests/fs/vfs: t_renamerace.c t_rmdirrace.c

 Log Message:
 Pass t_renamerace and t_rmdirrace tests.

 Adapt dholland@'s fix to ufs_rename to fix PR kern/43582.  Address several
 other MP locking issues discovered during the course of investigating the
 same problem.

 Removed extraneous vn_lock() calls on the Ifile, since the Ifile writes
 are controlled by the segment lock.

 Fix PR kern/45982 by deemphasizing the estimate of how much metadata
 will fill the empty space on disk when the disk is nearly empty
 (t_renamerace crates a lot of inode blocks on a tiny empty disk).


 To generate a diff of this commit:
 cvs rdiff -u -r1.18 -r1.19 src/sbin/newfs_lfs/make_lfs.c
 cvs rdiff -u -r1.135 -r1.136 src/sys/ufs/lfs/lfs.h
 cvs rdiff -u -r1.111 -r1.112 src/sys/ufs/lfs/lfs_alloc.c
 cvs rdiff -u -r1.121 -r1.122 src/sys/ufs/lfs/lfs_bio.c
 cvs rdiff -u -r1.223 -r1.224 src/sys/ufs/lfs/lfs_segment.c
 cvs rdiff -u -r1.293 -r1.294 src/sys/ufs/lfs/lfs_vfsops.c
 cvs rdiff -u -r1.239 -r1.240 src/sys/ufs/lfs/lfs_vnops.c
 cvs rdiff -u -r1.24 -r1.25 src/tests/fs/vfs/t_renamerace.c
 cvs rdiff -u -r1.8 -r1.9 src/tests/fs/vfs/t_rmdirrace.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/45982 CVS commit: [netbsd-6] src
Date: Sat, 17 Mar 2012 17:40:08 +0000

 Module Name:	src
 Committed By:	bouyer
 Date:		Sat Mar 17 17:40:08 UTC 2012

 Modified Files:
 	src/sbin/newfs_lfs [netbsd-6]: make_lfs.c
 	src/sys/ufs/lfs [netbsd-6]: lfs.h lfs_alloc.c lfs_bio.c lfs_segment.c
 	    lfs_vfsops.c lfs_vnops.c
 	src/tests/fs/vfs [netbsd-6]: t_renamerace.c t_rmdirrace.c

 Log Message:
 Pull up following revision(s) (requested by perseant in ticket #116):
 	sys/ufs/lfs/lfs_alloc.c: revision 1.112
 	tests/fs/vfs/t_rmdirrace.c: revision 1.9
 	tests/fs/vfs/t_renamerace.c: revision 1.25
 	sys/ufs/lfs/lfs_vnops.c: revision 1.240
 	sys/ufs/lfs/lfs_segment.c: revision 1.224
 	sys/ufs/lfs/lfs_bio.c: revision 1.122
 	sys/ufs/lfs/lfs_vfsops.c: revision 1.294
 	sbin/newfs_lfs/make_lfs.c: revision 1.19
 	sys/ufs/lfs/lfs.h: revision 1.136
 Pass t_renamerace and t_rmdirrace tests.
 Adapt dholland@'s fix to ufs_rename to fix PR kern/43582.  Address several
 other MP locking issues discovered during the course of investigating the
 same problem.
 Removed extraneous vn_lock() calls on the Ifile, since the Ifile writes
 are controlled by the segment lock.
 Fix PR kern/45982 by deemphasizing the estimate of how much metadata
 will fill the empty space on disk when the disk is nearly empty
 (t_renamerace crates a lot of inode blocks on a tiny empty disk).


 To generate a diff of this commit:
 cvs rdiff -u -r1.18 -r1.18.2.1 src/sbin/newfs_lfs/make_lfs.c
 cvs rdiff -u -r1.135 -r1.135.2.1 src/sys/ufs/lfs/lfs.h
 cvs rdiff -u -r1.111 -r1.111.8.1 src/sys/ufs/lfs/lfs_alloc.c
 cvs rdiff -u -r1.121 -r1.121.2.1 src/sys/ufs/lfs/lfs_bio.c
 cvs rdiff -u -r1.223 -r1.223.2.1 src/sys/ufs/lfs/lfs_segment.c
 cvs rdiff -u -r1.293 -r1.293.2.1 src/sys/ufs/lfs/lfs_vfsops.c
 cvs rdiff -u -r1.239 -r1.239.2.1 src/sys/ufs/lfs/lfs_vnops.c
 cvs rdiff -u -r1.24 -r1.24.4.1 src/tests/fs/vfs/t_renamerace.c
 cvs rdiff -u -r1.8 -r1.8.4.1 src/tests/fs/vfs/t_rmdirrace.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 18 Mar 2012 15:44:48 +0000
State-Changed-Why:
Fixed and pulled up.


>Unformatted:

Home	PR Database Search