NetBSD Problem Report #46529

From agrier@poofygoof.com  Sun Jun  3 06:14:36 2012
Return-Path: <agrier@poofygoof.com>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id BF7E563BA27
	for <gnats-bugs@gnats.NetBSD.org>; Sun,  3 Jun 2012 06:14:36 +0000 (UTC)
Message-Id: <20120603061433.B15D716A36@pythagoras.poofy.goof.com>
Date: Sat,  2 Jun 2012 23:14:33 -0700 (PDT)
From: agrier@poofygoof.com
Reply-To: agrier@poofygoof.com
To: gnats-bugs@gnats.NetBSD.org
Subject: ntfs-3g appears to be rate-limited
X-Send-Pr-Version: 3.95

>Number:         46529
>Category:       kern
>Synopsis:       ntfs-3g appears to be rate-limited
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Jun 03 06:15:01 +0000 2012
>Closed-Date:    Thu Sep 03 00:48:23 +0000 2026
>Last-Modified:  Thu Sep 03 00:48:23 +0000 2026
>Originator:     agrier@poofygoof.com
>Release:        NetBSD 5.1_STABLE
>Organization:
  Aaron J. Grier | "Not your ordinary poofy goof." | agrier@poofygoof.com
>Environment:
System: NetBSD pythagoras.poofy.goof.com 5.1_STABLE NetBSD 5.1_STABLE (GENERIC) #0: Wed Jan 12 03:35:24 PST 2011 agrier@fourier.poofy.goof.com:/var/obj/amd64/usr/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
copying files with ntfs-3g appears to be limited to 2.5MB/s.
copying files from the same device under Linux (which also uses ntfs-3g)
yields > 20MB/s.

pythagoras$ pkg_info -e fuse-ntfs  
fuse-ntfs-3g-1.1120

ehci0 at pci0 dev 18 function 2: vendor 0x1002 product 0x4396 (rev. 0x00)
ehci0: interrupting at ioapic0 pin 17
ehci0: dropped intr workaround enabled
ehci0: BIOS has given up ownership
ehci0: EHCI version 1.0
ehci0: companion controllers, 3 ports each: ohci0 ohci1
...
usb2 at ehci0: USB revision 2.0
...
uhub2 at usb2: vendor 0x1002 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
uhub2: 6 ports with 6 removable, self powered
...
umass1 at uhub2 port 4 configuration 1 interface 0
umass1: Seagate FreeAgentDesktop, rev 2.00/0.00, addr 3
umass1: using SCSI over Bulk-Only
scsibus1 at umass1: 2 targets, 1 lun per target
sd1 at scsibus1 target 0 lun 0: <Seagate, FreeAgentDesktop, 100D> disk fixed
sd1: fabricating a geometry
sd1: 465 GB, 476940 cyl, 64 head, 32 sec, 512 bytes/sect x 976773168 sectors
sd1: fabricating a geometry

the drive has power savings mode disabled.

>How-To-Repeat:
install ntfs-3g from pkgsrc.
attach NTFS drive to USB.
ntfs-3g /dev/sd1e /disk/media -o ro,uid=100,gid=100,umask=222
do some rsyncing or dding to get speed statistics
>Fix:
my current workaround is to perform NTFS copies from a Linux machine to my
NetBSD file server over the LAN, which puts a 100mbit limit on the copy
(~9MB/s).  this is slower than I observe natively under Linux, but faster than
accessing the drive when directly connected under NetBSD.

>Release-Note:

>Audit-Trail:
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/46529: ntfs-3g appears to be rate-limited
Date: Sun, 3 Jun 2012 05:01:44 -0400

 On Sun,  3 Jun 2012 06:15:01 +0000 (UTC)
 agrier@poofygoof.com wrote:

 > copying files with ntfs-3g appears to be limited to 2.5MB/s.
 > copying files from the same device under Linux (which also uses ntfs-3g)
 > yields > 20MB/s.

 Does using the raw device work better than the block device (i.e.
 rsd1e rather than sd1e in your case)?

 Also, can you confirm that disk access itself is faster when not
 passing through ntfs-3g?  I.e. using dd if=/dev/rsd1e of=/dev/null
 bs=512 and checking if "systat vmstat" shows a decent speed.

 Thanks,
 -- 
 Matt

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/46529: ntfs-3g appears to be rate-limited
Date: Mon, 4 Jun 2012 03:06:09 +0000

 On Sun, Jun 03, 2012 at 06:15:01AM +0000, agrier@poofygoof.com wrote:
  > >How-To-Repeat:
  > install ntfs-3g from pkgsrc.
  > attach NTFS drive to USB.
  > ntfs-3g /dev/sd1e /disk/media -o ro,uid=100,gid=100,umask=222
  > do some rsyncing or dding to get speed statistics

 How is the raw bandwidth of /dev/rsd1e? If that is awful, there's no
 point worrying about ntfs-3g or fuse.

 Also, I'm not sure if fuse filesystems are supposed to be using the
 block device (sd1e) or the raw device (rsd1e). Using the wrong one is
 likely to cause problems.

 In any case if ntfs-3g on Linux doesn't have the same speed problem,
 chances are it is not a pkgsrc issue but a kernel one, or perhaps a
 refuse/perfuse/whateverfuse one.

 -- 
 David A. Holland
 dholland@netbsd.org

From: "Aaron J. Grier" <agrier@poofygoof.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/46529: ntfs-3g appears to be rate-limited
Date: Mon, 11 Jun 2012 11:44:07 -0700

 (drive relocated to sd0 due to some hot-plugging, but it is the same
 device.)

 netbsd# dd if=/dev/rsd0e of=/dev/null bs=64k count=4000
 4000+0 records in
 4000+0 records out
 262144000 bytes transferred in 8.804 secs (29775556 bytes/sec)

 this is comparable to performance under linux:

 linux# dd if=/dev/sdc1 of=/dev/null count=800000
 800000+0 records in
 800000+0 records out
 409600000 bytes (410 MB) copied, 12.66 s, 32.4 MB/s

 the raw device cannot be mounted by ntfs-3g under NetBSD:

 netbsd# ntfs-3g /dev/rsd0e /mnt -o force,uid=100,gid=100,umask=222
 ntfs_attr_pread: ntfs_pread failed: Invalid argument
 Failed to read NTFS $Bitmap: Invalid argument

 -- 
   Aaron J. Grier | "Not your ordinary poofy goof." | agrier@poofygoof.com

Responsible-Changed-From-To: pkg-manager->kern-bug-people
Responsible-Changed-By: dholland@NetBSD.org
Responsible-Changed-When: Wed, 13 Jun 2012 19:25:55 +0000
Responsible-Changed-Why:
ok, this is a kernel problem until proven otherwise.


State-Changed-From-To: open->feedback
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Tue, 23 Jan 2024 21:55:33 +0000
State-Changed-Why:
Is this still problem with newer fuse-ntfs-3g?


From: "Aaron J. Grier" <agrier@poofygoof.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/46529: ntfs-3g appears to be rate-limited
Date: Tue, 5 May 2026 20:07:41 -0700

 dusting this one off; I technically still have the drive and system
 originally tested, but if the hypothesis is that fuse-ntfs-3g (or
 something in the puffs / fuse pipeline) is the rate limit, the issue
 should also show up in a different configuration.

 test is moved to NetBSD-11 running under KVM VM.

 raw (virtual) disk speed:
 cratylus$ sudo dd if=/dev/ld0e of=/dev/null bs=64k count=40000
 40000+0 records in
 40000+0 records out
 2621440000 bytes transferred in 17.449 secs (150234397 bytes/sec)

 and after bumping up kern.sbmax so librefuse can setsockopt() its
 buffers for 16 * FUSE_BUFSIZE, I get the following from a large file on
 a ntfs-3g fuse mount:

 cratylus$ dd if=bigfile1 of=/dev/null bs=64k
 7187+1 records in
 7187+1 records out
 471026263 bytes transferred in 6.603 secs (71335190 bytes/sec)
 cratylus$ dd if=bigfile2  of=/dev/null bs=64k
 11204+1 records in
 11204+1 records out
 734328832 bytes transferred in 10.439 secs (70344748 bytes/sec)

 I'd say that definitively demonstrates the problem isn't with
 fuse-ntfs-3g or the NetBSD library and kernel side of things for
 NetBSD-11, so this can be closed.

 I'm half wondering if the problem is in the USB stack, or if the buffer
 size for the perfuse unix domain socket was an issue in the past, but if
 I find anything later it's easy enough to submit another bug.

 thanks for renewing the reminder emails from gnats; I'm sure a lot of
 them get bit-bucketed, but they were the push I needed to wrap this one
 up.

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/46529: ntfs-3g appears to be rate-limited
Date: Wed, 6 May 2026 06:03:04 -0000 (UTC)

 gnats-admin@NetBSD.org ("Aaron J. Grier via gnats") writes:

 > raw (virtual) disk speed:
 > cratylus$ sudo dd if=/dev/ld0e of=/dev/null bs=64k count=40000
 > 40000+0 records in
 > 40000+0 records out
 > 2621440000 bytes transferred in 17.449 secs (150234397 bytes/sec)

 That's quite a number for reading from ld0.

 ld0 means, you read from the buffer cache with teensy 2kB I/O requests.

 The bs=64k causes 32 requests of 2kB to be issued in parallel, which
 probably also are executed in parallel for the virtual drive. A regular
 USB drive (umass) doesn't allow this, there you get sequential 2KB I/O.


 > 471026263 bytes transferred in 6.603 secs (71335190 bytes/sec)

 That sounds reasonable. The userland filesystem will also access
 the block device, and adds the filesystem overhead.


 To verify this, run "iostat -kx 1 ld0" while doing the test.

From: "Aaron J. Grier" <agrier@poofygoof.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/46529: ntfs-3g appears to be rate-limited
Date: Wed, 6 May 2026 11:40:05 -0700

 you are absolutely correct, the baseline is all zfs cache behavior on
 the backing side, and doesn't represent actual disk I/O.

 looking on the back-end I see zvol iostat showing ~120MB/s but no
 activity on the disks in the array, and no blinking lights on the drives
 themselves.  since the disk image is on a sparse (and compressed) ZFS
 volume, this has to be expected.

 I disabled my swap wedge and dding from it generated actual disk
 activity, but since the backing store is sparse, as soon as reads moved
 past the small amount of existing swap data, sparse handling kicked in
 and the rest of the wedge data was fulfulled without any disk I/O.

 I don't have an easy way to get actual USB passthrough on the system (it
 lives in a rack and has no easy-access USB ports), so I'm not sure how
 to get a good baseline out of this.

 here's an iostat snippet from the NetBSD side while reading through ntfs-3g:

 device  read KB/t    r/s   time     MB/s write KB/t    w/s   time     MB/s
 ld0          2.00  38431   0.58    75.06       0.00      0   0.58     0.00
 ld0          2.00  34170   0.69    66.74       0.00      0   0.69     0.00
 ld0          2.00  37623   0.70    73.48       0.00      0   0.70     0.00
 ld0          2.00  34487   0.79    67.36       0.00      0   0.79     0.00
 ld0          2.00  35610   0.87    69.55       0.00      0   0.87     0.00
 ld0          2.00  39171   0.70    76.51       0.00      0   0.70     0.00
 ld0          2.00  35614   0.81    69.56       0.00      0   0.81     0.00
 ld0          2.00  38249   0.71    74.70       0.00      0   0.71     0.00
 ld0          2.00  40597   0.52    79.29       0.00      0   0.52     0.00
 ld0          2.00  36121   0.79    70.55       0.00      0   0.79     0.00

 the VM back-end is all cached at this point, but it's not NetBSD's
 buffercache.  

 the original problem was that ntfs-3g was slow, which I'm definitely no longer
 observing.  I'm satisifed now, and feel like I'm beating a dead horse at this
 point, given the original ticket was filed in 2012.

 ready to close this one if you are.

State-Changed-From-To: feedback->closed
State-Changed-By: maya@NetBSD.org
State-Changed-When: Thu, 03 Sep 2026 00:48:23 +0000
State-Changed-Why:
feedback provided & submitter is ready to have this ticket closed. Maybe people would like to dicsuss it more, but if the submitter is satisfied, perhaps we should stop pinging them :)


>Unformatted:

Home	PR Database Search