NetBSD Problem Report #42175

From tron@zhadum.org.uk  Mon Oct 12 11:04:19 2009
Return-Path: <tron@zhadum.org.uk>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 9449763B8B6
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 12 Oct 2009 11:04:19 +0000 (UTC)
Message-Id: <20091012110416.96760F8EBD@lyssa.zhadum.org.uk>
Date: Mon, 12 Oct 2009 12:04:16 +0100 (BST)
From: tron@zhadum.org.uk
Reply-To: tron@zhadum.org.uk
To: gnats-bugs@gnats.NetBSD.org
Subject: smbfs doesn't support files large than 4GB
X-Send-Pr-Version: 3.95

>Number:         42175
>Category:       kern
>Synopsis:       smbfs doesn't support files large than 4GB
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    tron
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Oct 12 11:05:00 +0000 2009
>Closed-Date:    Tue Oct 27 20:51:58 +0000 2009
>Last-Modified:  Tue Oct 27 20:51:58 +0000 2009
>Originator:     tron@zhadum.org.uk
>Release:        NetBSD 5.99.20
>Organization:
Matthias Scheler                                  http://zhadum.org.uk/
>Environment:
System: NetBSD lyssa.zhadum.org.uk 5.99.20 NetBSD 5.99.20 (LYSSA) #0: Mon Oct 12 09:54:46 BST 2009 tron@lyssa.zhadum.org.uk:/src/sys/compile/LYSSA i386
Architecture: i386
Machine: i386
>Description:
To test NetBSD-current's SMBFS I tried to write a large file to my
Apple Time Capsule. It worked fine until the file got larger
than 4GB:

tron@lyssa:/mnt#dd if=/dev/zero bs=1024k count=5120 of=test.img
dd: test.img: File too large
4097+0 records in
4096+0 records out
4294967296 bytes transferred in 367.010 secs (11702589 bytes/sec)

I don't think this is a problem with the Time Capsule (which is *not*
using Samba) as I can upload and download files larger than 10GB
with Samba's "smbclient" without problems.

>How-To-Repeat:
mount -t smbfs ... /mnt
dd if=/dev/zero bs=1024k count=5120 of=test.img

>Fix:
Not known.

>Release-Note:

>Audit-Trail:
From: Nicolas Joly <njoly@pasteur.fr>
To: gnats-bugs@NetBSD.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: kern/42175: smbfs doesn't support files large than 4GB
Date: Wed, 14 Oct 2009 21:26:29 +0200

 On Mon, Oct 12, 2009 at 11:05:00AM +0000, tron@zhadum.org.uk wrote:
 > >Number:         42175
 > >Category:       kern
 > >Synopsis:       smbfs doesn't support files large than 4GB
 [...]
 > >Description:
 > To test NetBSD-current's SMBFS I tried to write a large file to my
 > Apple Time Capsule. It worked fine until the file got larger
 > than 4GB:

 I wanted to have a look, but your test does not fail for me. This is a
 smbfs mount from a NetApp filer.

 njoly@lanfeust [silo/njoly]> uname -a
 NetBSD lanfeust.sis.pasteur.fr 5.99.20 NetBSD 5.99.20 (LANFEUST) #3: Wed Oct 14 15:15:48 CEST 2009  njoly@lanfeust.sis.pasteur.fr:/local/src/NetBSD/obj.amd64/sys/arch/amd64/compile/LANFEUST amd64
 njoly@lanfeust [silo/njoly]> df -h .
 Filesystem            Size       Used      Avail %Cap Mounted on
 //NJOLY@SILO/SIS      1.3T       1.2T        40G  96% /mount/silo
 njoly@lanfeust [silo/njoly]> dd if=/dev/zero bs=1024k count=5120 of=test.img
 5120+0 records in
 5120+0 records out
 5368709120 bytes transferred in 621.933 secs (8632294 bytes/sec)

 njoly@hal [/mnt/njoly]> uname -a
 NetBSD hal.sis.pasteur.fr 5.99.20 NetBSD 5.99.20 (HAL) #0: Tue Oct 13 19:39:35 CEST 2009  njoly@hal.sis.pasteur.fr:/local/src/NetBSD/obj.i386/sys/arch/i386/compile/HAL i386
 njoly@hal [/mnt/njoly]> df -h .
 Filesystem            Size       Used      Avail %Cap Mounted on
 //NJOLY@SILO/SIS      1.3T       1.2T        40G  96% /mnt
 njoly@hal [/mnt/njoly]> dd if=/dev/zero bs=1024k count=5120 of=test.img
 5120+0 records in
 5120+0 records out
 5368709120 bytes transferred in 660.953 secs (8122679 bytes/sec)

 -- 
 Nicolas Joly

 Biological Software and Databanks.
 Institut Pasteur, Paris.

From: Matthias Scheler <tron@zhadum.org.uk>
To: Nicolas Joly <njoly@pasteur.fr>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/42175: smbfs doesn't support files large than 4GB
Date: Wed, 14 Oct 2009 21:37:55 +0100

 On Wed, Oct 14, 2009 at 07:30:04PM +0000, Nicolas Joly wrote:
 >  njoly@lanfeust [silo/njoly]> uname -a
 >  NetBSD lanfeust.sis.pasteur.fr 5.99.20 NetBSD 5.99.20 (LANFEUST) #3: Wed Oct 14 15:15:48 CEST 2009  njoly@lanfeust.sis.pasteur.fr:/local/src/NetBSD/obj.amd64/sys/arch/amd64/compile/LANFEUST amd64
 >  njoly@lanfeust [silo/njoly]> df -h .
 >  Filesystem            Size       Used      Avail %Cap Mounted on
 >  //NJOLY@SILO/SIS      1.3T       1.2T        40G  96% /mount/silo
 >  njoly@lanfeust [silo/njoly]> dd if=/dev/zero bs=1024k count=5120 of=test.img
 >  5120+0 records in
 >  5120+0 records out
 >  5368709120 bytes transferred in 621.933 secs (8632294 bytes/sec)

 Hmmmmm, I'm using NetBSD/i386. But that shouldn't make a difference.

 	Kind regards

 -- 
 Matthias Scheler                                  http://zhadum.org.uk/

From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/42175
Date: Thu, 15 Oct 2009 00:09:34 +0300

 a quick dip into the code:

         int wx = (SMB_CAPS(SSTOVC(ssp)) & SMB_CAP_LARGE_WRITEX);
 ...
         if (wx)
             error = smb_smb_writex(ssp, fid, &len, &resid, uio, scred);
         else
             error = smb_smb_write(ssp, fid, &len, &resid, uio, scred);

 and write contains:
         /* Cannot write at/beyond 4G */
         if (uio->uio_offset >= (1LL << 32))
                 return (EFBIG);

 So either the check is wrong (there's SMB_CAP_LARGE_FILES too), or it's
 not getting detected properly.  Either way, should only be a minute or
 two of debugging (with rump_smbfs, of course ;) or specreading.

State-Changed-From-To: open->closed
State-Changed-By: tron@NetBSD.org
State-Changed-When: Fri, 16 Oct 2009 16:25:41 +0000
State-Changed-Why:
This seems to be a bug in the CIFS server in the Apple Time Capsule.
It advertises the following capabilities:
- SMB_CAP_UNICODE
- SMB_CAP_LARGE_FILES
- SMB_CAP_NT_SMBS
- SMB_CAP_RPC_REMOTE_APIS
- SMB_CAP_STATUS32
- SMB_CAP_INFOLEVEL_PASSTHRU

It does however not advertise SMB_CAP_LARGE_READX and SMB_CAP_LARGE_WRITEX
which is the only way to write with 64-bit offsets as far as I can tell.


Responsible-Changed-From-To: kern-bug-people->tron
Responsible-Changed-By: tron@NetBSD.org
Responsible-Changed-When: Fri, 16 Oct 2009 20:53:44 +0000
Responsible-Changed-Why:
I'll try to handle this PR.


State-Changed-From-To: closed->open
State-Changed-By: tron@NetBSD.org
State-Changed-When: Fri, 16 Oct 2009 20:53:44 +0000
State-Changed-Why:
The CIFS technical reference, revision 1.0, says on page 49:
CAP_LARGE_FILES (0x0008) means "The server supports large files with
64 bit offsets"

The feature check in NetBSD's smbfs is wrong.


From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42175 CVS commit: src/sys/netsmb
Date: Sun, 18 Oct 2009 22:53:36 +0000

 Module Name:	src
 Committed By:	tron
 Date:		Sun Oct 18 22:53:36 UTC 2009

 Modified Files:
 	src/sys/netsmb: smb_smb.c

 Log Message:
 Fix detection of SMB capabilities according to the CIFS spec:
 1.) SMB_CAP_LARGE_FILES advertises support for 64-bit file offsets.
 2.) SMB_CAP_LARGE_READX and SMB_CAP_LARGE_WRITEX advertise support for
     large reads and writes (larger than 64KB).
 The code previously only used SMB_CAP_LARGE_READX and SMB_CAP_LARGE_WRITEX
 which is not correct and doesn't work for the Apple Time Capsule which
 only supports SMB_CAP_LARGE_FILES. With these changes SMBFS can copy a
 5GB to a Time Capsule and read it back without problems.

 Thanks a lot to Allen Briggs for pointing out the broke assumptions
 and explaining the CIFS spec to me. This fixes PR kern/42175.


 To generate a diff of this commit:
 cvs rdiff -u -r1.30 -r1.31 src/sys/netsmb/smb_smb.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->analyzed
State-Changed-By: tron@NetBSD.org
State-Changed-When: Sun, 18 Oct 2009 22:59:23 +0000
State-Changed-Why:
The bug should be fixed now. The change should be pulled up after
further testing.


From: Matthias Scheler <tron@NetBSD.org>
To: NetBSD GNATS <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: kern/42175 (smbfs doesn't support files large than 4GB)
Date: Mon, 19 Oct 2009 13:35:39 +0100

 	Hello,

 this commit is required as well:

 ----- Forwarded message from Matthias Scheler <tron@netbsd.org> -----

 Module Name:	src
 Committed By:	tron
 Date:		Sun Oct 18 23:20:31 UTC 2009

 Modified Files:
 	src/sys/netsmb: smb_smb.c

 Log Message:
 Fix cut & paste error spotted by Nicolas Joly.


 To generate a diff of this commit:
 cvs rdiff -u -r1.31 -r1.32 src/sys/netsmb/smb_smb.c

 ----- End forwarded message -----

 	Kind regards

 -- 
 Matthias Scheler                                  http://zhadum.org.uk/

State-Changed-From-To: analyzed->pending-pullups
State-Changed-By: tron@NetBSD.org
State-Changed-When: Mon, 19 Oct 2009 14:37:03 +0000
State-Changed-Why:
The bug is fixed, I've requested a pullup into the "netbsd-5" branch.


From: Manuel Bouyer <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42175 CVS commit: [netbsd-5] src/sys/netsmb
Date: Tue, 27 Oct 2009 20:31:15 +0000

 Module Name:	src
 Committed By:	bouyer
 Date:		Tue Oct 27 20:31:15 UTC 2009

 Modified Files:
 	src/sys/netsmb [netbsd-5]: smb_smb.c

 Log Message:
 Pull up following revision(s) (requested by tron in ticket #1104):
 	sys/netsmb/smb_smb.c: revision 1.31
 	sys/netsmb/smb_smb.c: revision 1.32
 Fix detection of SMB capabilities according to the CIFS spec:
 1.) SMB_CAP_LARGE_FILES advertises support for 64-bit file offsets.
 2.) SMB_CAP_LARGE_READX and SMB_CAP_LARGE_WRITEX advertise support for
     large reads and writes (larger than 64KB).
 The code previously only used SMB_CAP_LARGE_READX and SMB_CAP_LARGE_WRITEX
 which is not correct and doesn't work for the Apple Time Capsule which
 only supports SMB_CAP_LARGE_FILES. With these changes SMBFS can copy a
 5GB to a Time Capsule and read it back without problems.
 Thanks a lot to Allen Briggs for pointing out the broke assumptions
 and explaining the CIFS spec to me. This fixes PR kern/42175.
 Fix cut & paste error spotted by Nicolas Joly.


 To generate a diff of this commit:
 cvs rdiff -u -r1.29 -r1.29.6.1 src/sys/netsmb/smb_smb.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: tron@NetBSD.org
State-Changed-When: Tue, 27 Oct 2009 20:51:58 +0000
State-Changed-Why:
The fix has been pulled up.


>Unformatted:

Home	PR Database Search