NetBSD Problem Report #46629

From tsutsui@ceres.dti.ne.jp  Mon Jun 25 14:06:55 2012
Return-Path: <tsutsui@ceres.dti.ne.jp>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 71FE463B85F
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 25 Jun 2012 14:06:55 +0000 (UTC)
Message-Id: <201206251406.q5PE6p6T006433@mirage.localdomain>
Date: Mon, 25 Jun 2012 23:06:51 +0900 (JST)
From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Reply-To: tsutsui@ceres.dti.ne.jp
To: gnats-bugs@gnats.NetBSD.org
Cc: tsutsui@ceres.dti.n.ejp
Subject: newfs(8): proper block/fragment defaults for modern AFT disks
X-Send-Pr-Version: 3.95

>Number:         46629
>Category:       install
>Synopsis:       newfs(8): proper block/fragment defaults for modern AFT disks
>Confidential:   no
>Severity:       non-critical
>Priority:       high
>Responsible:    install-manager
>State:          closed
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Mon Jun 25 14:10:00 +0000 2012
>Closed-Date:    Fri Oct 26 16:51:22 +0000 2012
>Last-Modified:  Fri Oct 26 16:51:22 +0000 2012
>Originator:     Izumi Tsutsui
>Release:        NetBSD 6.0_BETA2
>Organization:
>Environment:
System: NetBSD 6.0_BETA2 i386
Architecture: i386, but affects all ports which can have SATA disks
Machine: i386
>Description:
Modern AFT SATA disks have physical 4KB (and logical 512B) sectors,
but current newfs(8) and sysinst(8) choose 16KB/2KB by default
even for >1TB partitions, so it could cause awful performance.

>How-To-Repeat:
Install NetBSD 6.0_BETA2 into AFT disks and create partitions
using default settings.

There is a report that unmounting targetroot after installation
including pkgsrc tree takes more than 30 minutes.

>Fix:
Currently there is no way to detect AFT disks on userland,
but using 32KB/4KB default values for larger disks
is still much better than current 16KB/2KB default,
and I guess there is few bad side effect on larger fragments
on such large partitions even on traditional 512B sector disks.

The following patch makes newfs(8) and sysinst(8)
to use 32KB/4KB for >= 128 GB partitions.
(128 GB is also used by fdisk(8) to choose 1MB alignment)


Index: sbin/newfs/newfs.c
===================================================================
RCS file: /cvsroot/src/sbin/newfs/newfs.c,v
retrieving revision 1.110
diff -u -p -r1.110 newfs.c
--- sbin/newfs/newfs.c	13 Feb 2012 12:59:56 -0000	1.110
+++ sbin/newfs/newfs.c	24 Jun 2012 19:47:24 -0000
@@ -157,14 +157,17 @@ const char lmsg[] = "%s: can't read disk
  */
 /*
  * For file systems smaller than SMALL_FSSIZE we use the S_DFL_* defaults,
- * otherwise if less than MEDIUM_FSSIZE use M_DFL_*, otherwise use
- * L_DFL_*.
+ * otherwise if less than MEDIUM_FSSIZE use M_DFL_*,
+ * otherwise if less than LARGE_FSSIZE use L_DFL_*,
+ * otherwise use LL_DFL_* especially for modern AFT disks.
  */
 #define	SMALL_FSSIZE	(20*1024*2)
 #define	S_DFL_FRAGSIZE	512
 #define	MEDIUM_FSSIZE	(1000*1024*2)
 #define	M_DFL_FRAGSIZE	1024
+#define	LARGE_FSSIZE	(128*1024*1024*2)
 #define	L_DFL_FRAGSIZE	2048
+#define	LL_DFL_FRAGSIZE	4096
 #define	DFL_FRAG_BLK	8

 /* Apple requires the fragment size to be at least APPLEUFS_DIRBLKSIZ
@@ -624,8 +627,10 @@ main(int argc, char *argv[])
 					fsize = S_DFL_FRAGSIZE;
 				else if (fssize < MEDIUM_FSSIZE)
 					fsize = M_DFL_FRAGSIZE;
-				else
+				else if (fssize < LARGE_FSSIZE)
 					fsize = L_DFL_FRAGSIZE;
+				else
+					fsize = LL_DFL_FRAGSIZE;
 				if (fsize < sectorsize)
 					fsize = sectorsize;
 			}
Index: sbin/newfs/newfs.8
===================================================================
RCS file: /cvsroot/src/sbin/newfs/newfs.8,v
retrieving revision 1.82
diff -u -p -r1.82 newfs.8
--- sbin/newfs/newfs.8	14 May 2011 19:46:10 -0000	1.82
+++ sbin/newfs/newfs.8	24 Jun 2012 19:47:24 -0000
@@ -111,10 +111,12 @@ The default size depends upon the size o
 .Ar block-size
 .It \*[Lt] 20 MB
 4 KB
-.It \*[Lt] 1024 MB
+.It \*[Lt] 1000 MB
 8 KB
-.It \*[Gt]= 1024 MB
+.It \*[Lt] 128 GB
 16 KB
+.It \*[Gt]= 128 GB
+32 KB
 .El
 .It Fl d Ar maxbsize
 Set the maximum extent size to
@@ -151,10 +153,12 @@ The default size depends upon the size o
 .Ar frag-size
 .It \*[Lt] 20 MB
 0.5 KB
-.It \*[Lt] 1024 MB
+.It \*[Lt] 1000 MB
 1 KB
-.It \*[Gt]= 1024 MB
+.It \*[Lt] 128 GB
 2 KB
+.It \*[Gt]= 128 GB
+4 KB
 .El
 .It Fl G
 Treat garbage parameters as non-fatal.
@@ -182,10 +186,12 @@ bytes of data space:
 .Ar bytes-per-inode
 .It \*[Lt] 20 MB
 2 KB
-.It \*[Lt] 1024 MB
+.It \*[Lt] 1000 MB
 4 KB
-.It \*[Gt]= 1024 MB
+.It \*[Lt] 128 GB
 8 KB
+.It \*[Gt]= 128 GB
+16 KB
 .El
 .It Fl m Ar free-space
 The percentage of space reserved from normal users; the minimum free
Index: distrib/utils/sysinst/label.c
===================================================================
RCS file: /cvsroot/src/distrib/utils/sysinst/label.c,v
retrieving revision 1.61
diff -u -p -r1.61 label.c
--- distrib/utils/sysinst/label.c	5 Jan 2012 22:18:36 -0000	1.61
+++ distrib/utils/sysinst/label.c	24 Jun 2012 19:47:24 -0000
@@ -213,8 +213,23 @@ set_ptype(partinfo *p, int fstype, int f
 	p->pi_fstype = fstype;
 	if (fstype == FS_BSDFFS || fstype == FS_BSDLFS) {
 		p->pi_frag = 8;
-		/* match newfs defaults for fragments size (2k if >= 1024MB) */
-		p->pi_fsize = p->pi_size > 1024*1024*1024 / 512 ? 2048 : 1024;
+		/*
+		 * match newfs defaults for fragments size:
+		 * fs size	frag size
+		 * < 20 MB	0.5 KB
+		 * < 1000 MB	1 KB
+		 * < 128 GB	2 KB
+		 * >= 128 GB	4 KB
+		 */
+	 	/* note pi_size is uint32_t so we have to avoid overflow */
+		if (p->pi_size < (20 * 1024 * (1024 / 512)))
+			p->pi_fsize = 512;
+		else if (p->pi_size < (1000 * 1024 * (1024 / 512)))
+			p->pi_fsize = 1024;
+		else if (p->pi_size < (128 * 1024 * 1024 * (1024 / 512)))
+			p->pi_fsize = 2048;
+		else
+			p->pi_fsize = 4096;
 	} else {
 		/* zero - fields not used */
 		p->pi_frag = 0;

>Release-Note:

>Audit-Trail:
From: "Izumi Tsutsui" <tsutsui@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46629 CVS commit: src
Date: Sat, 30 Jun 2012 15:34:02 +0000

 Module Name:	src
 Committed By:	tsutsui
 Date:		Sat Jun 30 15:34:02 UTC 2012

 Modified Files:
 	src/distrib/utils/sysinst: label.c
 	src/sbin/newfs: newfs.8 newfs.c

 Log Message:
 Use 32KB/4KB for default block/fragment size on >= 128 GB partitions
 for modern AFT disks.  No particular comments against PR install/46629.


 To generate a diff of this commit:
 cvs rdiff -u -r1.61 -r1.62 src/distrib/utils/sysinst/label.c
 cvs rdiff -u -r1.82 -r1.83 src/sbin/newfs/newfs.8
 cvs rdiff -u -r1.110 -r1.111 src/sbin/newfs/newfs.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->pending-pullups
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Mon, 02 Jul 2012 01:27:13 +0900
State-Changed-Why:
pullup-6 #396


From: "Jeff Rizzo" <riz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46629 CVS commit: [netbsd-6] src
Date: Thu, 5 Jul 2012 17:38:27 +0000

 Module Name:	src
 Committed By:	riz
 Date:		Thu Jul  5 17:38:27 UTC 2012

 Modified Files:
 	src/distrib/utils/sysinst [netbsd-6]: label.c
 	src/sbin/newfs [netbsd-6]: newfs.8 newfs.c

 Log Message:
 Pull up following revision(s) (requested by tsutsui in ticket #396):
 	sbin/newfs/newfs.8: revision 1.83
 	sbin/newfs/newfs.c: revision 1.111
 	distrib/utils/sysinst/label.c: revision 1.62
 Use 32KB/4KB for default block/fragment size on >= 128 GB partitions
 for modern AFT disks.  No particular comments against PR install/46629.


 To generate a diff of this commit:
 cvs rdiff -u -r1.61 -r1.61.2.1 src/distrib/utils/sysinst/label.c
 cvs rdiff -u -r1.82 -r1.82.6.1 src/sbin/newfs/newfs.8
 cvs rdiff -u -r1.110 -r1.110.2.1 src/sbin/newfs/newfs.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Sat, 27 Oct 2012 01:51:22 +0900
State-Changed-Why:
already in 6.0


>Unformatted:

Home	PR Database Search