NetBSD Problem Report #37025
From Ephaeton@gmx.net Tue Sep 25 15:44:24 2007
Return-Path: <Ephaeton@gmx.net>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by narn.NetBSD.org (Postfix) with ESMTP id 726D263B8C2
for <gnats-bugs@gnats.NetBSD.org>; Tue, 25 Sep 2007 15:44:24 +0000 (UTC)
Message-Id: <20070925154420.D818AE32C@circe.entropie.net>
Date: Tue, 25 Sep 2007 17:44:20 +0200 (CEST)
From: Ephaeton@gmx.net
Reply-To: Ephaeton@gmx.net
To: gnats-bugs@NetBSD.org
Subject: mount_smbfs write performance is <censored>
X-Send-Pr-Version: 3.95
>Number: 37025
>Category: kern
>Synopsis: mount_smbfs write performance is <censored>
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Sep 25 15:45:00 +0000 2007
>Closed-Date: Sat Apr 04 16:12:33 +0000 2020
>Last-Modified: Sat Apr 04 16:12:33 +0000 2020
>Originator: Martin S. Weber
>Release: NetBSD 4.0 Beta 2 - mid sep 2007, 6.0_BETA
>Organization:
>Environment:
System: NetBSD rfhinf038 4.0_BETA2 NetBSD 4.0_BETA2 (GENERIC) #0: Mon Sep 17 16:41:59 CEST 2007 root@rfhinf038:/usr/home/netbsd/netbsd-4-obj/sys/arch/i386/compile/GENERIC i386
Architecture: i386
Machine: i386
>Description:
(see also http://mail-index.netbsd.org/netbsd-users/2007/09/21/0001.html)
mount_smbfs write performance sucks. It is outperformed by smbclient(1)
by a factor of 4 to 5. This is not acceptable, as it basically renders
it unusable.
Here are some measurements between two 4.0 b2 nbsd i386 machines on
a local 100mbit lan:
-- samba - writing
# smbclient. This is somewhere near expected.
#
$ smbclient -N -c "cd samba-share;put file.bz2" '\\smb.srv\data'
putting file file.bz2 as \daten\tmp\fritzilein\file.bz2 (5570.2 kb/s) (average 5570.2 kb/s)
$ bs=512
$ while [ $bs -lt 64000 ]; do for block in obs bs; do echo "bs:$bs block:$block";
dd if=file.bz2 of=/somewhere/samba-share/file.bz2 $block=$bs
# results.
# Worse than 1/4 of the smbclient result.
#
bs:512 block:obs 83117097 bytes transferred in 68.297 secs (1216994 bytes/sec)
bs:512 block:bs 83117097 bytes transferred in 68.316 secs (1216656 bytes/sec)
bs:1024 block:obs 83117097 bytes transferred in 44.410 secs (1871585 bytes/sec)
bs:1024 block:bs 83117097 bytes transferred in 44.746 secs (1857531 bytes/sec)
bs:2048 block:obs 83117097 bytes transferred in 44.197 secs (1880604 bytes/sec)
bs:2048 block:bs 83117097 bytes transferred in 47.088 secs (1765143 bytes/sec)
bs:4096 block:obs 83117097 bytes transferred in 46.196 secs (1799227 bytes/sec)
bs:4096 block:bs 83117097 bytes transferred in 44.257 secs (1878055 bytes/sec)
bs:8192 block:obs 83117097 bytes transferred in 48.595 secs (1710404 bytes/sec)
bs:8192 block:bs 83117097 bytes transferred in 50.414 secs (1648690 bytes/sec)
bs:16384 block:obs 83117097 bytes transferred in 42.976 secs (1934035 bytes/sec)
bs:16384 block:bs 83117097 bytes transferred in 47.292 secs (1757529 bytes/sec)
bs:32768 block:obs 83117097 bytes transferred in 48.668 secs (1707838 bytes/sec)
bs:32768 block:bs 83117097 bytes transferred in 48.268 secs (1721991 bytes/sec)
# dd with default settings / buffersizes
#
default 83117097 bytes transferred in 102.354 secs (812055 bytes/sec)
# writing via cp(1) isn't any better. Lookup the time in the one
# megabyte buffersize example above.
#
$ time cp -fv file.bz2 /somewhere/samba-share/file.bz2
file.bz2 -> /somewhere/samba-share/file.bz2
45.90s real 0.01s user 0.28s system
# -- samba - reading
# smbclient. This again is somewhere near expected. (I know for large files it
# goes up to the network max).
#
$ smbclient -N -c "cd samba-share;get file.bz2" '\\smb.srv\data'
getting file \samba-share\file.bz2 of size 83117097 as file.bz2 (6458.9 kb/s) (average 6458.9 kb/s)
# cp(1) surprises me given the results in the write case.
#
$ time cp -fv /somewhere/samba-share/file.bz2 file.bz2
/somewhere/samba-share/file.bz2 -> file.bz2
13.78s real 0.00s user 0.39s system
$ bs=512
$ while [ $bs -lt 64000 ]; do for block in ibs bs; do echo "bs:$bs block:$block";
dd of=file.bz2 if=/somewhere/samba-share/file.bz2 $block=$bs
bs:512 block:ibs 83117097 bytes transferred in 14.680 secs (5661927 bytes/sec)
bs:512 block:bs 83117097 bytes transferred in 13.050 secs (6369126 bytes/sec)
bs:1024 block:ibs 83117097 bytes transferred in 14.686 secs (5659614 bytes/sec)
bs:1024 block:bs 83117097 bytes transferred in 13.068 secs (6360353 bytes/sec)
bs:2048 block:ibs 83117097 bytes transferred in 14.671 secs (5665400 bytes/sec)
bs:2048 block:bs 83117097 bytes transferred in 13.020 secs (6383801 bytes/sec)
bs:4096 block:ibs 83117097 bytes transferred in 14.777 secs (5624761 bytes/sec)
bs:4096 block:bs 83117097 bytes transferred in 13.006 secs (6390673 bytes/sec)
bs:8192 block:ibs 83117097 bytes transferred in 14.696 secs (5655763 bytes/sec)
bs:8192 block:bs 83117097 bytes transferred in 12.993 secs (6397067 bytes/sec)
bs:16384 block:ibs 83117097 bytes transferred in 14.716 secs (5648076 bytes/sec)
bs:16384 block:bs 83117097 bytes transferred in 13.009 secs (6389199 bytes/sec)
bs:32768 block:ibs 83117097 bytes transferred in 14.797 secs (5617158 bytes/sec)
bs:32768 block:bs 83117097 bytes transferred in 13.721 secs (6057655 bytes/sec)
>How-To-Repeat:
Use a nbsd 4.0 samba server (3.0.24), use a nbsd 4.0 client, hang 'em on
a 100mbit lan, try to write an iso image to the samba server and then
control the upcoming waves of aggression and the felt need to kill somebody
near. Then, try to kill (even -9) the process just for fun (not that it'd
work when it's in smbwrq state). Scream some. Then use smbclient.
>Fix:
dunno. But someone prolly should fix that before 4.0
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback
State-Changed-By: tron@NetBSD.org
State-Changed-When: Mon, 12 Oct 2009 11:00:56 +0000
State-Changed-Why:
Can you please re-test with NetBSD-current (5.99.20)? NetBSD-current can
write to my Time Capsule at 11MBs which is as fast as this NAS can go.
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025 (mount_smbfs write performance is <censored>)
Date: Mon, 12 Oct 2009 14:53:58 -0400
On Mon, Oct 12, 2009 at 11:00:58AM +0000, tron@NetBSD.org wrote:
> Synopsis: mount_smbfs write performance is <censored>
>
> State-Changed-From-To: open->feedback
> State-Changed-By: tron@NetBSD.org
> State-Changed-When: Mon, 12 Oct 2009 11:00:56 +0000
> State-Changed-Why:
> Can you please re-test with NetBSD-current (5.99.20)? NetBSD-current can
> write to my Time Capsule at 11MBs which is as fast as this NAS can go.
Hrmpf, -current... ok. This is not an easy thing to do as I have to
replicate a lot of setup that has since gone away (we moved to NFS
instead as this didn't have the performance penalty), but I'll perform
some tests at some places. I should get -current partitions at my machines
though :) I'll try to get back to you soon but it may take me some time.
Aside of that, looking forward to some promising results.
-Martin
From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/37025
Date: Mon, 12 Oct 2009 22:49:24 +0300
I doubt -current is really any faster. The fundamental problem is that
smbfs does not support UBC and does writes in-context (at least based
on my 2min glance at the code). So if you do a lot of small writes,
you end up paying a lot of RTT penalty.
(well, ok, it shouldn't be quite *that* slow as what you report)
You can mitigate this a little by using rump_smbfs, for which puffs
provides UBC in front of smbfs. People (like Matthias ;) have reported
it being several times faster than a kernel mount for small I/O units.
However, be warned that mount_smbfs does not correctly provide a stable
vnode from a lookup. Combined with UBC, this might cause some cache
screwage in a multiuser environment. (It is also probably quite easy
to fix, but I didn't look into the SMB protocol.)
From: Matthias Scheler <tron@zhadum.org.uk>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025
Date: Tue, 13 Oct 2009 07:52:09 +0100
On Mon, Oct 12, 2009 at 07:50:04PM +0000, Antti Kantee wrote:
> You can mitigate this a little by using rump_smbfs, for which puffs
> provides UBC in front of smbfs. People (like Matthias ;) have reported
> it being several times faster than a kernel mount for small I/O units.
The last result was actually with the in-kernel file-system, but with
a large blocksize.
Kind regards
--
Matthias Scheler http://zhadum.org.uk/
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@gnats.netbsd.org
Cc:
Subject: Re: kern/37025
Date: Fri, 5 Feb 2010 00:43:58 -0500
Just fyi: I started doing some measurements w/ 5.0_STABLE as my baseline,
only problem is that transfers halt at times and I have to do an "ls"
in the mounted samba share to get them to continue. I believe there's
another PR open detailling that. Anyways, took me a while to figure that
out and I have adjusted my measurement method to accommodate for that.
Now I gotta get a statically linked -current in place so I can compare.
Well, I haven't forgotten about it, just things go slooooow at times.
Regards,
-Martin
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/37025: mount_smbfs write performance is <censored>
Date: Mon, 1 Mar 2010 03:03:19 +0000
(sent to gnats instead of gnats-bugs)
------
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: NetBSD Problem Report DB Administrator <gnats@NetBSD.org>
Subject: kern/37025
Date: Sat, 27 Feb 2010 00:17:03 -0500
Fwiw, I started my first measurements finally. As I've noted before, there's
some unrelated problems I'm facing, which should be fixed this way or another.
I probably should open separate PRs for them:
- a process in smbwrq often cannot be killed (even with -9) before the WHOLE
writing is done.
- sometimes transfers stall. I have to cd in the dir in question (hmm or somewhere
on the mounted share?) and/or do an ls in that directory to get the transfer
to pick up pace again.
First results show that (not surprisingly I guess (even though I reported
this BEFORE 4.0 :p)) NetBSD-5.0's kernel-backed mount_smbfs performs
old-school: putting packets on horse back, exchanging horses at a couple of
stations and losing some of them. Performance peaks at about 1.5MiB/s, but
at times is as low as 600KiB/s - smbclient works at the network maximum.
Another observation (via xosview's NET display) is that smbclient uses a lot
less sent packets when reading from the share in contrast to mount_smbfs.
Some earlier observation in this PR, that more RTT suffering comes into play
with the native mount, seems to be backed up by this observation. I still
have some things to do here, especially measure -current, and probably other
means of getting a file onto the samba share; measure the RTT to see if you
can relate it to the results; probably some netstat stats for mount_smbfs
vs. smbclient vs. -current (if that is performing any better, which I doubt:
why should it now, after a few months, have been improved, if it hasn't for
ALL THESE YEARS before).
Stay tuned...
-Martin
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@gnats.netbsd.org
Cc:
Subject: Re: kern/37025
Date: Mon, 1 Mar 2010 17:27:46 -0500
Alright, it seems I prematurely jumped to conclusions here. I left
the computer alone when it was still running tests with low
blocksizes; with higher blocksizes the image changes.
The following are results for reading and writing ~130MB of "random"
data (a release.tar containing a set of i386 binary sets) to the
samba share (read from or written to a local wapbl ffs). The server
is something native windows -- the network a 100MBit. I don't have
any control whatsoever over the server in case you wanna ask
questions, although I might find one thing or another out. The
client machine was a NetBSD 5.0_STABLE (GENERIC) #4: Tue Jan 5
13:58:42 EST 2010
Good news first: you'll see that mount_smbfs performs (nearly) en
par with smbclient, if the blocksize is just high enough.
writing:
block \ block type & attempt
size \ obs#1 bs#1 obs#2 bs#2 obs#3 bs#3
--------\-----------------------------------------------
1/2 kiB | 694kB 800kB 866kB 910kB 935kB 1021kB
1 kiB | 1272kB 1306kB 1553kB 280kB 1493kB 444kB
2 kiB | 2666kB 2760kB 2952kB 2972kB 2719kB 2877kB
4 kiB | 3760kB 3668kB 4593kB 4599kB 4647kB 4093kB
8 kiB | 4892kB 5040kB 6350kB 6014kB 6232kB 6200kB
16 kiB | 6675kB 7112kB 7913kB 7970kB 7850kB 7944kB
32 kiB | 8338kB 8459kB 8977kB 9189kB 9035kB 9148kB
64 kiB | 8664kB 8753kB 9203kB 9252kB 9275kB 9336kB
--------+-----------------------------------------------
default | 1022kB (at about the time of bs#3 measurement)
smbcl | #1: 8595kB #2: 9198kB #3: 9264kB
reading:
block \ block type & attempt
size \ ibs#1 bs#1 ibs#2 bs#2 ibs#3 bs#3
--------\-----------------------------------------------
1/2 kiB | 759kB 736kB 944kB 944kB 1049kB 328kB
1 kiB | 1319kB 1389kB 1567kB 1813kB 1532kB 1537kB
2 kiB | 2281kB 2083kB 2952kB 3114kB 2874kB 3007kB
4 kiB | 3285kB 3655kB 4487kB 4798kB 3609kB 4503kB
8 kiB | 4890kB 5196kB 5870kB 6468kB 5974kB 6358kB
16 kiB | 6286kB 7178kB 7268kB 7914kB 7415kB 7955kB
32 kiB | 7907kB 8611kB 8513kB 9495kB 8504kB 9621kB
64 kiB | 7968kB 8843kB 8584kB 9536kB 9275kB 9336kB
--------+-----------------------------------------------
default | 1070kB (at about the time of bs#3 measurement)
smbcl | #1: 9253kB #2: 9845kB #3: 9896kB
For the bad news:
- I had to keep some while :; do ls ; sleep 5; done running to have
access to the share every now and then, without it, the transfers
will stall pretty quickly.
- Even with that, sometimes the ls itself was stuck (in tstile)
while the dd was stuck in smbwrq (at least the one case where I
thought of checking dd's state). Things wouldn't move without me
doing ANOTHER ls on the samba share. Then, both the ls would
immediately do its job as would the dd pick up pace again (as seen
on xosview's NET traffic display).
- I didn't notice all of these "stuck" situations at once, that's
where the worst results in above tables come from : reading (1/2
kiB, bs#3), writing (1 kiB, bs#3)
- I'm not satisfied yet with the results and will do further
measurements (cp specifically although I doubt it will differ
from the dd-default case at least for writing)
For completeness: following outlines the measurement method:
==========
#! /bin/ksh
date
bs=512
while [ $bs -lt 66000 ]
do
echo "dd writing"
for block in obs bs
do
echo -n "bs:$bs block:$block "
dd if=release.tar of=$SMBSHARE/release.tar $block=$bs
done
echo "dd reading"
for block in ibs bs
do
echo -n "bs:$bs block:$block "
dd if=$SMBSHARE/release.tar of=release.tar $block=$bs
done
bs=$(($bs * 2))
done
echo "smbclient reading"
smbclient -c "cd $SMBHOME/$SMBSHARE; get release.tar" \
-I $SMBSRV -W $WRK -U $USR //$USR@$SMB/$SHR $password
echo "smbclient writing"
smbclient -c "cd $SMBHOME/$SMBSHARE; put release.tar" \
-I $SMBSRV -W $WRK -U $USR //$USR@$SMB/$SHR $password
date
==========
If you see a systematic/fundamental flaw in the above, please
let me know.
Regards,
-Martin
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025
Date: Tue, 2 Mar 2010 19:31:19 -0500
Ok, further measurements under 5.0 show that obviously cp and tar write
to the share with a big enough blocksize to just be "a bit" (~10%) behind
smbclient. So at least in these cases the performance no longer is to
be censored in comparison to smbclient - still noticably slower though.
Next measurement will actually feature -current, finally :)
Btw, the file server I'm writing to is a BlueArc Titan...
Regards,
-Martin
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025 (mount_smbfs write performance is <censored>)
Date: Mon, 22 Mar 2010 21:51:40 -0400
Alright, I've measured -current, too. The image is basically the same as
with -5: cp, mv etc. work fast while dd with varying blocksizes build up
as the other measurements show. So smbclient still outperforms NetBSD's
kernel supported smbfs mount - just a bit for cp etc., and seriously for
dd with non-ridiculous blocksizes.
Now where to go from here? I still believe there's a problem here,
although the situation seriously improved in contrast to the eons
ago when I first filed this PR.
Also the problem with random hangs of the mount persists. I haven't
seen this happening with smbclient yet, only with a mount_smbfs-mounted
share.
-Martin
-current results:
dd writing
bs:512 block:obs 259020+0 records in
259020+0 records out
132618240 bytes transferred in 171.851 secs (771704 bytes/sec)
bs:512 block:bs 259020+0 records in
259020+0 records out
132618240 bytes transferred in 118.349 secs (1120569 bytes/sec)
dd reading
bs:512 block:ibs 259020+0 records in
259020+0 records out
132618240 bytes transferred in 130.862 secs (1013420 bytes/sec)
bs:512 block:bs 259020+0 records in
259020+0 records out
132618240 bytes transferred in 120.308 secs (1102322 bytes/sec)
dd writing
bs:1024 block:obs 259020+0 records in
129510+0 records out
132618240 bytes transferred in 74.345 secs (1783821 bytes/sec)
bs:1024 block:bs 129510+0 records in
129510+0 records out
132618240 bytes transferred in 71.654 secs (1850814 bytes/sec)
dd reading
bs:1024 block:ibs 129510+0 records in
259020+0 records out
132618240 bytes transferred in 60.762 secs (2182585 bytes/sec)
bs:1024 block:bs 129510+0 records in
129510+0 records out
132618240 bytes transferred in 61.042 secs (2172573 bytes/sec)
dd writing
bs:2048 block:obs 259020+0 records in
64755+0 records out
132618240 bytes transferred in 40.570 secs (3268874 bytes/sec)
bs:2048 block:bs 64755+0 records in
64755+0 records out
132618240 bytes transferred in 40.334 secs (3288001 bytes/sec)
dd reading
bs:2048 block:ibs 64755+0 records in
259020+0 records out
132618240 bytes transferred in 41.401 secs (3203261 bytes/sec)
bs:2048 block:bs 64755+0 records in
64755+0 records out
132618240 bytes transferred in 39.635 secs (3345988 bytes/sec)
dd writing
bs:4096 block:obs 259020+0 records in
32377+1 records out
132618240 bytes transferred in 25.979 secs (5104824 bytes/sec)
bs:4096 block:bs 32377+1 records in
32377+1 records out
132618240 bytes transferred in 26.142 secs (5072995 bytes/sec)
dd reading
bs:4096 block:ibs 32377+1 records in
259020+0 records out
132618240 bytes transferred in 26.150 secs (5071443 bytes/sec)
bs:4096 block:bs 32377+1 records in
32377+1 records out
132618240 bytes transferred in 24.839 secs (5339113 bytes/sec)
dd writing
bs:8192 block:obs 259020+0 records in
16188+1 records out
132618240 bytes transferred in 19.214 secs (6902167 bytes/sec)
bs:8192 block:bs 16188+1 records in
16188+1 records out
132618240 bytes transferred in 18.975 secs (6989103 bytes/sec)
dd reading
bs:8192 block:ibs 16188+1 records in
259020+0 records out
132618240 bytes transferred in 21.275 secs (6233524 bytes/sec)
bs:8192 block:bs 16188+1 records in
16188+1 records out
132618240 bytes transferred in 20.035 secs (6619328 bytes/sec)
dd writing
bs:16384 block:obs 259020+0 records in
8094+1 records out
132618240 bytes transferred in 15.925 secs (8327675 bytes/sec)
bs:16384 block:bs 8094+1 records in
8094+1 records out
132618240 bytes transferred in 15.859 secs (8362333 bytes/sec)
dd reading
bs:16384 block:ibs 8094+1 records in
259020+0 records out
132618240 bytes transferred in 17.842 secs (7432924 bytes/sec)
bs:16384 block:bs 8094+1 records in
8094+1 records out
132618240 bytes transferred in 16.072 secs (8251508 bytes/sec)
dd writing
bs:32768 block:obs 259020+0 records in
4047+1 records out
132618240 bytes transferred in 14.317 secs (9262990 bytes/sec)
bs:32768 block:bs 4047+1 records in
4047+1 records out
132618240 bytes transferred in 14.157 secs (9367679 bytes/sec)
dd reading
bs:32768 block:ibs 4047+1 records in
259020+0 records out
132618240 bytes transferred in 15.469 secs (8573161 bytes/sec)
bs:32768 block:bs 4047+1 records in
4047+1 records out
132618240 bytes transferred in 13.937 secs (9515551 bytes/sec)
dd writing
bs:65536 block:obs 259020+0 records in
2023+1 records out
132618240 bytes transferred in 14.156 secs (9368341 bytes/sec)
bs:65536 block:bs 2023+1 records in
2023+1 records out
132618240 bytes transferred in 13.853 secs (9573250 bytes/sec)
dd reading
bs:65536 block:ibs 2023+1 records in
259020+0 records out
132618240 bytes transferred in 986.283 secs (134462 bytes/sec)
bs:65536 block:bs 2023+1 records in
2023+1 records out
132618240 bytes transferred in 13.984 secs (9483569 bytes/sec)
smbclient reading
Domain=[$DOMAIN] OS=[BlueArc Titan 6.1.1684.20] Server=[BlueArc Titan 6.1.1684.20]
getting file ...\release.tar of size 132618240 as release.tar (10033.3 kb/s) (average 10033.3 kb/s)
smbclient writing
Domain=[$DOMAIN] OS=[BlueArc Titan 6.1.1684.20] Server=[BlueArc Titan 6.1.1684.20]
putting file release.tar as ...\release.tar (9291.9 kb/s) (average 9291.9 kb/s)
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Ephaeton@gmx.net
Subject: Re: kern/37025 (mount_smbfs write performance is <censored>)
Date: Tue, 23 Mar 2010 17:29:04 +0000
On Tue, Mar 23, 2010 at 01:55:02AM +0000, Martin S. Weber wrote:
> Now where to go from here? I still believe there's a problem here,
> although the situation seriously improved in contrast to the eons
> ago when I first filed this PR.
Kernel profiling or fine-grained timing to figure out where the time
is going? It is likely to be a pain.
> Also the problem with random hangs of the mount persists. I haven't
> seen this happening with smbclient yet, only with a mount_smbfs-mounted
> share.
Figuring out whether this is network-related (e.g. stalls due to lost
packets, protocol botches, etc.) or vfs-related (lock
contention/deadlock/etc.) might also be helpful. After all, the
performance problem might be small instances of the same issue...
ps -alx should be enough to diagnose this; look for something that
isn't "tstile". ("tstile" just means "waiting for something else").
--
David A. Holland
dholland@netbsd.org
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025 (mount_smbfs write performance is <censored>)
Date: Tue, 23 Mar 2010 15:29:37 -0400
On Tue, Mar 23, 2010 at 05:30:04PM +0000, David Holland wrote:
> The following reply was made to PR kern/37025; it has been noted by GNATS.
>
> From: David Holland <dholland-bugs@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc: Ephaeton@gmx.net
> Subject: Re: kern/37025 (mount_smbfs write performance is <censored>)
> Date: Tue, 23 Mar 2010 17:29:04 +0000
>
> On Tue, Mar 23, 2010 at 01:55:02AM +0000, Martin S. Weber wrote:
> > Now where to go from here? I still believe there's a problem here,
> > although the situation seriously improved in contrast to the eons
> > ago when I first filed this PR.
>
> Kernel profiling or fine-grained timing to figure out where the time
> is going? It is likely to be a pain.
Hell yeah :) I'll also need a pointer on how to do that correctly please.
>
> > Also the problem with random hangs of the mount persists. I haven't
> > seen this happening with smbclient yet, only with a mount_smbfs-mounted
> > share.
>
> Figuring out whether this is network-related (e.g. stalls due to lost
> packets, protocol botches, etc.) or vfs-related (lock
> contention/deadlock/etc.) might also be helpful. After all, the
> performance problem might be small instances of the same issue...
>
> ps -alx should be enough to diagnose this; look for something that
> isn't "tstile". ("tstile" just means "waiting for something else").
I've written in an earlier reply to this PR that the stuck processes
are sitting in smbwrq. I don't think it's network related, because they
will sit there stalled until I do an ls in another shell on the share.
Then, immediately (!), the writing/reading continues.
Please let me know how I can furtherly try to diagnose a stalled
transfer for locking problems or other vfs problems (pointer to howtos
should do it).
Also note that sometimes processes in smbwrq are unkillable (even with -9).
Regards & TIA,
-Martin
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Ephaeton@gmx.net
Subject: Re: kern/37025 (mount_smbfs write performance is <censored>)
Date: Tue, 23 Mar 2010 19:52:30 +0000
On Tue, Mar 23, 2010 at 07:30:08PM +0000, Martin S. Weber wrote:
>>> Now where to go from here? I still believe there's a problem here,
>>> although the situation seriously improved in contrast to the eons
>>> ago when I first filed this PR.
>>
>> Kernel profiling or fine-grained timing to figure out where the time
>> is going? It is likely to be a pain.
>
> Hell yeah :) I'll also need a pointer on how to do that correctly please.
I mostly don't know; I avoid that kind of thing as much as possible :-/
There's some way to do kernel profiling (in the strict sense of
getting output for gmon); another way is to explicitly instrument
interesting paths with time calls at strategic points. dtrace may be
able to do this for you nowadays... or at least some of it.
> I've written in an earlier reply to this PR that the stuck processes
> are sitting in smbwrq. I don't think it's network related, because they
> will sit there stalled until I do an ls in another shell on the share.
Ah, I didn't see that. This means it's probably not either a protocol
issue or a vfs-level issue but an ordinary synchronization bug in the
smbfs code.
> Then, immediately (!), the writing/reading continues.
> Please let me know how I can furtherly try to diagnose a stalled
> transfer for locking problems or other vfs problems (pointer to howtos
> should do it).
That's not so much a (!) -- what it means is that the wakeup() call or
equivalent condvar operation that should have been done was not done.
When something else happens, the stuck guy is then also awakened.
The way to find this one is to examine how the smbwrq object is used
and look for code paths with missing wakeups or race conditions
between going to sleep and being awakened. I will take a look at it
sometime myself (but my queue is extremely long...)
If there are multiple things going by the wchan name "smbwrq", hacking
the names in your kernel to disambiguate them is a good step too.
--
David A. Holland
dholland@netbsd.org
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: NetBSD Problem Report DB Administrator <gnats@NetBSD.org>
Cc:
Subject: Re: kern/37025
Date: Thu, 12 Apr 2012 17:49:18 -0400
For the record: I let the script that has been attached to this PR already loose on the
fileserver out of curiosity to see what NetBSD 6.0_BETA would do. Well, it runs out of
bufspace. This happened before somewhen between 5.0 and a later 5.x release (I had only
been tracking releases here) -- see kern/44092.
So, yeah, kern/44092 still is open on NetBSD 6.0_BETA, and as long as it is, I won't
run more tests for THIS PR.
Regards, -Martin
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025
Date: Sun, 30 Dec 2012 07:40:45 +0000
On Thu, Apr 12, 2012 at 09:00:08PM +0000, Martin S. Weber wrote:
> For the record: I let the script that has been attached to this PR
> already loose on the fileserver out of curiosity to see what
> NetBSD 6.0_BETA would do. Well, it runs out of bufspace. This
> happened before somewhen between 5.0 and a later 5.x release (I
> had only been tracking releases here) -- see kern/44092.
>
> So, yeah, kern/44092 still is open on NetBSD 6.0_BETA, and as long
> as it is, I won't run more tests for THIS PR.
hi, a candidate fix for 44092 was checked in back in May -- (1) any
luck with it? and (2) any further data on this one?
--
David A. Holland
dholland@netbsd.org
From: "Martin S. Weber" <Ephaeton@gmx.net>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/37025
Date: Sun, 30 Dec 2012 09:56:45 -0500
On Sun, Dec 30, 2012 at 07:45:14AM +0000, David Holland wrote:
> The following reply was made to PR kern/37025; it has been noted by GNATS.
>
> From: David Holland <dholland-bugs@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: kern/37025
> Date: Sun, 30 Dec 2012 07:40:45 +0000
>
> On Thu, Apr 12, 2012 at 09:00:08PM +0000, Martin S. Weber wrote:
> > For the record: I let the script that has been attached to this PR
> > already loose on the fileserver out of curiosity to see what
> > NetBSD 6.0_BETA would do. Well, it runs out of bufspace. This
> > happened before somewhen between 5.0 and a later 5.x release (I
> > had only been tracking releases here) -- see kern/44092.
> >
> > So, yeah, kern/44092 still is open on NetBSD 6.0_BETA, and as long
> > as it is, I won't run more tests for THIS PR.
>
> hi, a candidate fix for 44092 was checked in back in May -- (1) any
> luck with it? and (2) any further data on this one?
(1) dunno, I switched to NFS mounts quite a while ago because I needed
something that just works (tm) and luckily there was some nfs server
to be found in the network.
(2) I am not as often as I used to be next to the machine physically
and it no longer is accessible from the outside world. So it'll be
a while (mid to end january) before I really can test this again.
Regards,
-Martin
State-Changed-From-To: feedback->open
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sat, 20 May 2017 19:30:58 +0000
State-Changed-Why:
while the last change to this PR was in 2012, it does not appear that it
should currently be in feedback.
also there's no particular reason to think it's been fixed, since I can't
remember anyone working on smbfs in the intervening years.
State-Changed-From-To: open->closed
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Sat, 04 Apr 2020 16:12:33 +0000
State-Changed-Why:
Obsolete, SMBFS/nsmb was removed from the tree. Thanks for report.
>Unformatted:
(Contact us)
$NetBSD: gnats-precook-prs,v 1.4 2018/12/21 14:20:20 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home