NetBSD Problem Report #40229

From www@NetBSD.org  Fri Dec 19 17:50:24 2008
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by narn.NetBSD.org (Postfix) with ESMTP id 592E063BB12
	for <gnats-bugs@gnats.netbsd.org>; Fri, 19 Dec 2008 17:50:24 +0000 (UTC)
Message-Id: <20081219175024.1E4A363B8BD@narn.NetBSD.org>
Date: Fri, 19 Dec 2008 17:50:24 +0000 (UTC)
From: jan+nbsd@v6shell.org
Reply-To: jan+nbsd@v6shell.org
To: gnats-bugs@NetBSD.org
Subject: NetBSD pkgsrc/devel/subversion-base - NFS-mounted repository failures
X-Send-Pr-Version: www-1.0

>Number:         40229
>Category:       pkg
>Synopsis:       NetBSD pkgsrc/devel/subversion-base - NFS-mounted repository failures
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Dec 19 17:55:00 +0000 2008
>Closed-Date:    Sun Apr 08 05:33:19 +0000 2012
>Last-Modified:  Sun Apr 08 05:33:19 +0000 2012
>Originator:     J.A. Neitzel
>Release:        NetBSD 4.0.1_PATCH
>Organization:
not applicable
>Environment:
NetBSD haven.local 4.0.1_PATCH NetBSD 4.0.1_PATCH (GENERIC) #0: Mon Dec  8 01:39:56 UTC 2008  jneitzel@serenity.local:/usr/obj/sys/arch/amd64/compile/GENERIC amd64
>Description:
Operations on NFS-mounted subversion repositories fail.  This causes
subversion to be unreliable (and unusable) in such a context.  This
problematic behavior is new since subversion-1.5.x; subversion-1.4.x
worked as expected w/ NFS-mounted subversion repositories.

NOTE:
1) subversion repositories on locally-mounted FFS filesystems seem to
work as expected.

2) The only workaround is to not serve subversion repositories via NFS,
but this may not be a valid workaround for all users.

3) This behavior has also been observed on NetBSD 2.1.0_STABLE alpha,
but it has apparently never been reported.
>How-To-Repeat:
In the following description...  Even though subversion allowed me
to import an unversioned tree, it did it incorrectly.  Upon checking
out a working copy of the imported tree, only 3 of the 10 files
(all of which were non-empty) contained the pre-imported content. 
The other 7 were now empty files.

In addition, many of the files in the .svn directory were empty 
files.  This in turn made the working copy unusable.

On an NFS client, do the following (replace /home/jneitzel w/ an
appropriate value as needed):

% mkdir ~/rep ; chmod 2775 ~/rep ; svnadmin create ~/rep

% svn mkdir file:///home/jneitzel/rep/stuff ; rm -f svn-commit.tmp
svn: Can't read directory '/home/jneitzel/rep/db/transactions/0-1.txn':\
        Partial results are valid but processing is incomplete
svn: Your commit message was left in a temporary file:
svn:    'svn-commit.tmp'

% svn ls -v file:///home/jneitzel/rep
      1 jneitzel              Dec 19 01:32 ./
      1 jneitzel              Dec 19 01:32 stuff/

% svn log file:///home/jneitzel/rep/stuff
------------------------------------------------------------------------
r1 | jneitzel | 2008-12-19 01:32:08 +0000 (Fri, 19 Dec 2008) | 2 lines

Add a root directory for the stuff project.

------------------------------------------------------------------------

% svn import stuff file:///home/jneitzel/rep/stuff/trunk               
Adding         stuff/0                
Adding         stuff/1                
Adding         stuff/2                
Adding         stuff/3                
svn: Can't get entries of non-directory
svn: Your commit message was left in a temporary file:                 
svn:    'svn-commit.tmp'              

% svn import --force stuff file:///home/jneitzel/rep/stuff/trunk       
Adding         stuff/0                
Adding         stuff/1                
Adding         stuff/2                
Adding         stuff/3                
Adding         stuff/4                
Adding         stuff/5                
Adding         stuff/6                
Adding         stuff/7                
Adding         stuff/8                
Adding         stuff/9                
svn: Can't read directory '/home/jneitzel/rep/db/transactions/1-3.txn':\
        Partial results are valid but processing is incomplete         
svn: Your commit message was left in a temporary file:                 
svn:    'svn-commit.2.tmp'            

% svn ls -v file:///home/jneitzel/rep/stuff/trunk                      
      2 jneitzel              Dec 19 05:40 ./                          
      2 jneitzel            0 Dec 19 05:40 0                           
      2 jneitzel            4 Dec 19 05:40 1                           
      2 jneitzel            4 Dec 19 05:40 2                           
      2 jneitzel            6 Dec 19 05:40 3                           
      2 jneitzel            5 Dec 19 05:40 4                           
      2 jneitzel            5 Dec 19 05:40 5                           
      2 jneitzel            4 Dec 19 05:40 6                           
      2 jneitzel            6 Dec 19 05:40 7                           
      2 jneitzel            6 Dec 19 05:40 8                           
      2 jneitzel            5 Dec 19 05:40 9                           

% svn log file:///home/jneitzel/rep/stuff/trunk                        
------------------------------------------------------------------------
r2 | jneitzel | 2008-12-19 05:40:10 +0000 (Fri, 19 Dec 2008) | 2 lines 

Initial import of stuff trunk.        

------------------------------------------------------------------------

...
Otherwise, try to use the repository as usual.
Watch it fail.
>Fix:
Unknown, but changes to the following file between 1.4.x and 1.5.x
may be partly to blame:

work/subversion-1.5.4/subversion/libsvn_subr/io.c

...
The comments and code for svn_io_remove_dir2() may help to determine
what should be done?

>Release-Note:

>Audit-Trail:
From: Eric Gillespie <epg@NetBSD.org>
To: gnats-bugs@NetBSD.org, jan+nbsd@v6shell.org
Cc: 
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted repository failures
Date: Fri, 19 Dec 2008 13:23:46 -0800

 This is not a pkgsrc issue; take it to
 users@subversion.tigris.org .

 Until recently, I had nearly 400 users committing to NFS-backed
 repositories via multiple servers, using 1.5 on the servers and
 1.3, 1.4, 1.5, and even SVNKit on the clients.  I think you may
 be having some local trouble.  I see that you're using file:///
 access; I can't discourage that enough, and I doubt you'll hear
 anything different from the users list...

 -- 
 Eric Gillespie <*> epg@NetBSD.org

From: "J.A. Neitzel" <jan+nbsd@v6shell.org>
To: pkgsrc-bugs@NetBSD.org, pkg-manager@NetBSD.org, gnats-bugs@NetBSD.org,
        gnats-admin@NetBSD.org
Cc: 
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted
 repository failures
Date: Fri, 19 Dec 2008 22:04:09 +0000

 >  This is not a pkgsrc issue; take it to
 >  users@subversion.tigris.org .

 ACK

 >  Until recently, I had nearly 400 users committing to NFS-backed
 >  repositories via multiple servers, using 1.5 on the servers and
 >  1.3, 1.4, 1.5, and even SVNKit on the clients.  I think you may
 >  be having some local trouble.  I see that you're using file:///
 >  access; I can't discourage that enough, and I doubt you'll hear
 >  anything different from the users list...

 ACK; thank you.
 -- 
 J.A. Neitzel
 V6 Thompson Shell Port - http://v6shell.org/

Responsible-Changed-From-To: pkg-manager->epg
Responsible-Changed-By: obache@NetBSD.org
Responsible-Changed-When: Mon, 29 Dec 2008 08:34:48 +0000
Responsible-Changed-Why:
Over to maintainer.


From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted
	repository failures
Date: Sat, 21 Feb 2009 20:25:04 +0000

 blah, forgot gnats.

    ------

 From: David Holland <dholland-pbugs@netbsd.org>
 To: Eric Gillespie <epg@NetBSD.org>
 Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org,
 	jan+nbsd@v6shell.org
 Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted
 	repository failures
 Date: Sat, 21 Feb 2009 20:24:02 +0000

 On Fri, Dec 19, 2008 at 09:25:01PM +0000, Eric Gillespie wrote:
  >  Until recently, I had nearly 400 users committing to NFS-backed
  >  repositories via multiple servers, using 1.5 on the servers and
  >  1.3, 1.4, 1.5, and even SVNKit on the clients.  I think you may
  >  be having some local trouble.  I see that you're using file:///
  >  access; I can't discourage that enough, and I doubt you'll hear
  >  anything different from the users list...

 For what it's worth, I've just seen the same thing a couple times
 using subversion-base-1.5.5 on NetBSD 5.99.7.

 -- 
 David A. Holland
 dholland@netbsd.org

From: Mark Davies <mark@ecs.vuw.ac.nz>
To: gnats-bugs@netbsd.org
Cc: David Holland <dholland-pbugs@netbsd.org>
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted repository failures
Date: Wed, 25 Feb 2009 02:12:45 +1300

 On Sunday 22 February 2009 David Holland wrote:
 > On Fri, Dec 19, 2008 at 09:25:01PM +0000, Eric Gillespie wrote:
 > >  Until recently, I had nearly 400 users committing to NFS-backed
 > >  repositories via multiple servers, using 1.5 on the servers and
 > >  1.3, 1.4, 1.5, and even SVNKit on the clients.  I think you may
 > >  be having some local trouble.  I see that you're using file:///
 > >  access; I can't discourage that enough, and I doubt you'll hear
 > >  anything different from the users list...

 > For what it's worth, I've just seen the same thing a couple times
 > using subversion-base-1.5.5 on NetBSD 5.99.7.

 We were seeing this.  I've tracked it down to the following:

 In subversion/libsvn_subr/io.c (in svn_io_remove_dir2()) there is code to 
 work around an issue described in this comment

 /*
  Mac OS X has a bug where if you're readding the contents of a
  directory via readdir in a loop, and you remove one of the entries in
  the directory and the directory has 338 or more files in it you will
  skip over some of the entries in the directory.  Needless to say,
  this causes problems if you are using this kind of loop inside a
  function that is recursively deleting a directory, because when you
  get around to removing the directory it will still have something in
  it.

  Similar problem has been observed on FreeBSD.

  See http://subversion.tigris.org/issues/show_bug.cgi?id=1896 for more
  discussion and an initial solution.

  To work around the problem, we do a rewinddir after we delete all files
  and see if there's anything left. We repeat the steps untill there's
  nothing left to delete.

  This workaround causes issues on Windows where delete's are asynchronous,
  however, so we never rewind if we're on Windows (the delete says it is
  complete, we rewind, we see the same file and try to delete it again,
  we fail.
 */



 On NetBSD on an NFS filesystem, when it does the rewinddir and "see if 
 there's anything left" it sees files remaining (due to NFS dir caching?) 
 so tries to delete them again and fails as they are already deleted.

 I'm currently working around the problem with this patch

 --- subversion/libsvn_subr/io.c.orig    2008-10-09 08:28:59.000000000 +1300
 +++ subversion/libsvn_subr/io.c
 @@ -1839,7 +1839,7 @@ svn_io_remove_dir2(const char *path, svn
              {
                const char *fullpath, *entry_utf8;

 -#ifndef WIN32
 +#if 0 /* ndef WIN32 */
                need_rewind = TRUE;
  #endif

 cheers
 mark

From: Mark Davies <mark@ecs.vuw.ac.nz>
To: Eric Gillespie <epg@netbsd.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted repository failures
Date: Thu, 26 Feb 2009 13:04:27 +1300

 On Thu, 26 Feb 2009, you wrote:
 > This patch is obviously not acceptable for pkgsrc.  Given that
 > the issue that led to the rewinddir hack in the first place has
 > been seen on FreeBSD, it's not even acceptable to conditionalize
 > based on __NETBSD__ or whatever.

 Obviously, thats why I said it was a work around not a patch.

 cheers
 mark

From: Eric Gillespie <epg@NetBSD.org>
To: gnats-bugs@NetBSD.org, Mark Davies <mark@ecs.vuw.ac.nz>
Cc: 
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted repository failures
Date: Wed, 25 Feb 2009 15:23:01 -0800

 Mark Davies <mark@ecs.vuw.ac.nz> writes:

 >  --- subversion/libsvn_subr/io.c.orig    2008-10-09 08:28:59.000000000 +1300
 >  +++ subversion/libsvn_subr/io.c
 >  @@ -1839,7 +1839,7 @@ svn_io_remove_dir2(const char *path, svn
 >               {
 >                 const char *fullpath, *entry_utf8;
 >  
 >  -#ifndef WIN32
 >  +#if 0 /* ndef WIN32 */
 >                 need_rewind = TRUE;
 >   #endif

 This patch is obviously not acceptable for pkgsrc.  Given that
 the issue that led to the rewinddir hack in the first place has
 been seen on FreeBSD, it's not even acceptable to conditionalize
 based on __NETBSD__ or whatever.

 At any rate, this is not a pkgsrc issue.  Take it to
 dev@subversion.tigris.org .

 -- 
 Eric Gillespie <*> epg@NetBSD.org

State-Changed-From-To: open->closed
State-Changed-By: epg@NetBSD.org
State-Changed-When: Thu, 26 Feb 2009 03:40:26 -0500
State-Changed-Why:
not a pkgsrc issue


From: Edgar =?iso-8859-1?B?RnXf?= <ef@math.uni-bonn.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted
	repository failures
Date: Thu, 24 Sep 2009 12:05:59 +0200

 > At any rate, this is not a pkgsrc issue.
 > Take it to dev@subversion.tigris.org.
 Done that:
 http://subversion.tigris.org/ds/viewMessage.do?dsMessageId=2392271&dsForumId=462
 and
 http://subversion.tigris.org/ds/viewMessage.do?dsMessageId=2398903&dsForumId=462

 I also filed an Issue (#3501) with them:
 http://subversion.tigris.org/issues/show_bug.cgi?id=3501

From: Edgar =?iso-8859-1?B?RnXf?= <ef@math.uni-bonn.de>
To: gnats-bugs@NetBSD.org
Cc: Mark Davies <mark@ecs.vuw.ac.nz>
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted
 repository failures
Date: Wed, 13 Jan 2010 16:43:28 +0100

 > I also filed an Issue (#3501) with them:
 > http://subversion.tigris.org/issues/show_bug.cgi?id=3501
 Finally, after raising this topic on dev@subversion for the third time, Peter Samuelson came up with a fix that works for me:
 http://mail-archives.apache.org/mod_mbox/subversion-dev/201001.mbox/%3c20100107065500.GE7154@p12n.org%3e

 Issue 3501 is still open, but hopefully, said patch or something equivalent will show up in the next Subversion release.

State-Changed-From-To: closed->suspended
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Wed, 13 Jan 2010 19:27:49 +0000
State-Changed-Why:
problem is not fixed, awaiting upstream action.


From: Edgar =?iso-8859-1?B?RnXf?= <ef@math.uni-bonn.de>
To: gnats-bugs@NetBSD.org
Cc: Mark Davies <mark@ecs.vuw.ac.nz>
Subject: Re: pkg/40229: NetBSD pkgsrc/devel/subversion-base - NFS-mounted
 repository failures
Date: Thu, 14 Jan 2010 15:21:35 +0100

 > Issue 3501 is still open, but hopefully, said patch or something equivalent will show up in the next Subversion release.
 A mostly equivalent fix has been committed to Subversion as r898963.

From: YAMAMOTO Takashi <yamt@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/40229 CVS commit: src
Date: Sun, 26 Sep 2010 02:27:00 +0000

 Module Name:	src
 Committed By:	yamt
 Date:		Sun Sep 26 02:27:00 UTC 2010

 Modified Files:
 	src/include: dirent.h
 	src/lib/libc/gen: Makefile.inc closedir.c dirent_private.h opendir.c
 	    rewinddir.c
 Added Files:
 	src/lib/libc/gen: initdir.c

 Log Message:
 fix rewinddir on nfs.  fix PR/42879 (and probably PR/40229.)


 To generate a diff of this commit:
 cvs rdiff -u -r1.33 -r1.34 src/include/dirent.h
 cvs rdiff -u -r1.171 -r1.172 src/lib/libc/gen/Makefile.inc
 cvs rdiff -u -r1.15 -r1.16 src/lib/libc/gen/closedir.c
 cvs rdiff -u -r1.3 -r1.4 src/lib/libc/gen/dirent_private.h
 cvs rdiff -u -r0 -r1.1 src/lib/libc/gen/initdir.c
 cvs rdiff -u -r1.36 -r1.37 src/lib/libc/gen/opendir.c
 cvs rdiff -u -r1.12 -r1.13 src/lib/libc/gen/rewinddir.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: suspended->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 26 Sep 2010 23:33:14 +0000
State-Changed-Why:
Are you (or is anyone else) able to test svn on nfs on -current with yamt's
fixes?


Responsible-Changed-From-To: epg->pkg-manager
Responsible-Changed-By: wiz@NetBSD.org
Responsible-Changed-When: Mon, 28 Feb 2011 15:06:15 +0000
Responsible-Changed-Why:
Reset responsible field for retired developer.


State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 08 Apr 2012 05:33:19 +0000
State-Changed-Why:
Feedback timeout. Assume the nfs fixes work; if svn is still broken
it's upstream's problem.


>Unformatted:

Home	PR Database Search