NetBSD Problem Report #37696

From stix@stix.id.au  Sat Jan  5 04:22:15 2008
Return-Path: <stix@stix.id.au>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by narn.NetBSD.org (Postfix) with ESMTP id B86E763B852
	for <gnats-bugs@gnats.NetBSD.org>; Sat,  5 Jan 2008 04:22:14 +0000 (UTC)
Message-Id: <20080105042211.8684967FA1@stix.id.au>
Date: Sat,  5 Jan 2008 15:22:11 +1100 (EST)
From: stix@stix.id.au
Reply-To: stix@stix.id.au
To: gnats-bugs@NetBSD.org
Subject: msdosfs large reads
X-Send-Pr-Version: 3.95

>Number:         37696
>Category:       kern
>Synopsis:       msdosfs: add large read / readahead support
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    kern-bug-people
>State:          closed
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Sat Jan 05 04:25:00 +0000 2008
>Closed-Date:    Sat Jan 05 19:01:12 +0000 2008
>Last-Modified:  Sat Jan 05 19:01:12 +0000 2008
>Originator:     Paul Ripke
>Release:        NetBSD 4.0_STABLE
>Organization:
>Environment:
System: NetBSD zion.stix.org.au 4.0_STABLE NetBSD 4.0_STABLE (ZION) #3: Tue Jan 1 12:04:06 EST 2008 stix@zion.stix.org.au:/export/netbsd/netbsd-4/obj.i386/export/netbsd/netbsd-4/src/sys/arch/i386/compile/ZION i386

Architecture: i386
Machine: i386
>Description:
	Introduce run-length detection to msdosfs_bmap()
>How-To-Repeat:
>Fix:
Ported a patch from FreeBSD, tested on a 120 GiB 2.5" USB drive with
512 byte sectors and an 80 GiB iPod with 2048 byte sectors. Have not
tested as yet with the recent patch by reinoud to ask UVM to not do
reads while extending files, but both patches together should make
a huge difference.

Patch against netbsd-4 branch:
-----
Index: msdosfs_vnops.c
===================================================================
RCS file: /usr/netbsd/cvsroot/src/sys/fs/msdosfs/msdosfs_vnops.c,v
retrieving revision 1.34.2.1
diff -u -d -r1.34.2.1 msdosfs_vnops.c
--- msdosfs_vnops.c	17 Feb 2007 23:27:44 -0000	1.34.2.1
+++ msdosfs_vnops.c	5 Jan 2008 03:53:46 -0000
@@ -1702,19 +1702,37 @@
 		int *a_runp;
 	} */ *ap = v;
 	struct denode *dep = VTODE(ap->a_vp);
+	int run, maxrun;
+	daddr_t runbn;
 	int status;

 	if (ap->a_vpp != NULL)
 		*ap->a_vpp = dep->de_devvp;
 	if (ap->a_bnp == NULL)
 		return (0);
-	if (ap->a_runp) {
-		/*
-		 * Sequential clusters should be counted here.
-		 */
-		*ap->a_runp = 0;
-	}
 	status = pcbmap(dep, ap->a_bn, ap->a_bnp, 0, 0);
+
+	/*
+	 * From FreeBSD:
+	 * A little kludgy, but we loop calling pcbmap until we
+	 * reach the end of the contiguous piece, or reach MAXPHYS.
+	 * Since it reduces disk I/Os, the "wasted" CPU is put to
+	 * good use (4 to 5 fold sequential read I/O improvement on USB
+	 * drives).
+	 */
+	if (ap->a_runp != NULL) {
+		/* taken from ufs_bmap */
+		maxrun = ulmin(MAXPHYS / dep->de_pmp->pm_bpcluster - 1,
+			       dep->de_pmp->pm_maxcluster - ap->a_bn);
+		for (run = 1; run <= maxrun; run++) {
+			if (pcbmap(dep, ap->a_bn + run, &runbn, NULL, NULL)
+			    != 0 || runbn !=
+			            *ap->a_bnp + de_cn2bn(dep->de_pmp, run))
+				break;
+		}
+		*ap->a_runp = run - 1;
+	}
+
 	/*
 	 * We need to scale *ap->a_bnp by sector_size/DEV_BSIZE
 	 */
-----

Some stats I gathered before and after this patch:

before with 2.5" 120 GB USB enclosure, 512 bytes/sector:

hex:ksh$ dd if=/dev/zero of=/mnt/sd0e/1GiB bs=64k count=16k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 428.989 secs (2502958 bytes/sec)
hex:ksh$ sudo umount /mnt/sd0e && sudo mount /mnt/sd0e
hex:ksh$ dd if=/mnt/sd0e/1GiB of=/dev/null bs=64k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 197.598 secs (5433971 bytes/sec)

before with 80 GB iPod video, 2048 bytes/sector:

hex:ksh$ dd if=/dev/zero of=/mnt/sd0e/1GiB bs=64k count=16k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 1405.737 secs (763828 bytes/sec)
hex:ksh$ sudo umount /mnt/sd0e && sudo mount /mnt/sd0e
hex:ksh$ dd if=/mnt/sd0e/1GiB of=/dev/null bs=64k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 272.871 secs (3934979 bytes/sec)

after with 2.5" 120 GB USB enclosure, 512 bytes/sector:

hex:ksh$ dd if=/dev/zero of=/mnt/sd0e/1GiB bs=64k count=16k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 179.856 secs (5970008 bytes/sec)
hex:ksh$ sudo umount /mnt/sd0e && sudo mount /mnt/sd0e
hex:ksh$ dd if=/mnt/sd0e/1GiB of=/dev/null bs=64k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 41.963 secs (25587823 bytes/sec)

after with 80 GB iPod video, 2048 bytes/sector:

hex:ksh$ dd if=/dev/zero of=/mnt/sd0e/1GiB bs=64k count=16k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 1391.287 secs (771761 bytes/sec)
hex:ksh$ sudo umount /mnt/sd0e && sudo mount /mnt/sd0e
hex:ksh$ dd if=/mnt/sd0e/1GiB of=/dev/null bs=64k
16384+0 records in
16384+0 records out
1073741824 bytes transferred in 180.402 secs (5951939 bytes/sec)

>Release-Note:

>Audit-Trail:
From: Christos Zoulas <christos@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: PR/37696 CVS commit: src/sys/fs/msdosfs
Date: Sat,  5 Jan 2008 18:42:15 +0000 (UTC)

 Module Name:	src
 Committed By:	christos
 Date:		Sat Jan  5 18:42:15 UTC 2008

 Modified Files:
 	src/sys/fs/msdosfs: msdosfs_vnops.c

 Log Message:
 PR/37696: Paul Ripke: add large read / readahead support (Introduce run-length
 detection to msdosfs_bmap(), from FreeBSD)


 To generate a diff of this commit:
 cvs rdiff -r1.46 -r1.47 src/sys/fs/msdosfs/msdosfs_vnops.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: christos@netbsd.org
State-Changed-When: Sat, 05 Jan 2008 14:01:12 -0500
State-Changed-Why:
patch applied, thanks


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.