NetBSD Problem Report #42961

From dholland@eecs.harvard.edu  Thu Mar 11 19:14:05 2010
Return-Path: <dholland@eecs.harvard.edu>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 269ED63B873
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 11 Mar 2010 19:14:05 +0000 (UTC)
Message-Id: <20100311191313.86A9AFA4C@tanaqui.eecs.harvard.edu>
Date: Thu, 11 Mar 2010 14:13:13 -0500 (EST)
From: dholland@eecs.harvard.edu
Reply-To: dholland@eecs.harvard.edu
To: gnats-bugs@gnats.NetBSD.org
Subject: cvs near-silently throws away local mods
X-Send-Pr-Version: 3.95

>Number:         42961
>Category:       bin
>Synopsis:       cvs near-silently throws away local mods
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 11 19:15:00 +0000 2010
>Last-Modified:  Sat Jan 25 18:45:01 +0000 2020
>Originator:     David A. Holland
>Release:        NetBSD 4.99.72 (20081004)
>Organization:
>Environment:
System: NetBSD valkyrie 4.99.72 NetBSD 4.99.72 (VALKYRIE) #32: Sat Oct 4 12:25:22 EDT 2008 root@valkyrie:/usr/src/sys/arch/amd64/compile/VALKYRIE amd64
Architecture: x86_64
Machine: amd64
>Description:

I had a largish patch to ufs_wapbl.c in my "miscellaneous" HEAD source
tree (not one of the important patches fortunately, it was a pile of
comments documenting the refcount flow in rename) and on cvs update
just now the following happened:

   valkyrie% cvs -q update -dP
   P lfs/lfs_vfsops.c
   P ufs/ufs_lookup.c
   P ufs/ufs_wapbl.c
   cvs update: checksum failure after patch to ufs/ufs_wapbl.c; will refetch
   cvs client: refetching unpatchable files
   cvs update: warning: ufs/ufs_wapbl.c was lost
   U ufs/ufs_wapbl.c
   valkyrie% 

and now:
   valkyrie% cvs diff ufs/ufs_wapbl.c
   valkyrie% 

Note that it did not even attempt to merge.

It also did not generate a .#ufs_wapbl.c.1.7 file containing the prior
modified version. (I do have a .#ufs_wapbl.c.1.6 file from the
previous merge some time back, but if the changes had been more recent
I wouldn't have.)

I have seen this problem twice before in recent months (the checksum
failure message) but this is the first time it happened someplace I
knew for sure what was there beforehand.

The tree where this happened is updated regularly. Note also that
while this machine's install is outdated it is as far as I know the
same cvs as in -5.

This is a pretty serious problem, as I and a lot of people keep random
uncommitted/local changes in source trees and rely on CVS not to
randomly lose them like this.

Worse, in a large update most likely the message would go by without
anyone noticing it; I tend to grep update logs for '^[CM]' and some
people just do find . -name '*.rej' to see if anything failed.

>How-To-Repeat:

I have no idea what might trigger this.

>Fix:

Shoot CVS?

>Audit-Trail:
From: David Holland <dholland@eecs.harvard.edu>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Thu, 11 Mar 2010 14:42:12 -0500

 Oh right, for reference for folks from outside NetBSD who come across
 this:

    valkyrie% cvs --version

    Concurrent Versions System (CVS) 1.11.22 (client/server)

    [...]

 There are some NetBSD-specific patches and it's possible that one of
 them causes this behavior. But it doesn't seem too likely.

 -- 
    - David A. Holland / dholland@eecs.harvard.edu

From: "Greg A. Woods" <woods@planix.ca>
To: NetBSD GNATS <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Thu, 11 Mar 2010 16:33:02 -0500

 --pgp-sign-Multipart_Thu_Mar_11_16:33:02_2010-1
 Content-Type: text/plain; charset=US-ASCII
 Content-Transfer-Encoding: quoted-printable

 At Thu, 11 Mar 2010 19:15:01 +0000 (UTC), dholland@eecs.harvard.edu wrote:
 Subject: bin/42961: cvs near-silently throws away local mods
 >=20
 > This is a pretty serious problem, as I and a lot of people keep random
 > uncommitted/local changes in source trees and rely on CVS not to
 > randomly lose them like this.

 In fact this is in effect the only way I use CVS with NetBSD.

 All my local changes are kept in local working directories where I
 frequently run "cvs -q update -dP" to get upstream changes.

 As far as I can remember though I've never seen the error you mention.

 One major difference is, in this case with how I use NetBSD CVS at
 least, I use a local copy of the whole repository, so there's never any
 network connection involved.  I.e. I don't think my usage case ever
 touches the same code path that causes you the error, so I suppose it's
 not surprising that I've never seen a similar error.

 Anyway, there are at least two possibilities for your error:

 1. it could be due to this bug:

 	http://savannah.nongnu.org/bugs/?17032

    (basically a problem with keyword expansion not resulting in the same
    text on both the server and client side)

 2. could the timestamps in your working directory have been wrong?

 	http://www.mail-archive.com/info-cvs@nongnu.org/msg03179.html

    this same issue is further discussed in this very recent bug report:

 	http://savannah.nongnu.org/bugs/?29058

 Interestingly the same error appeared in an automated daily update
 status message:

     http://mail-index.netbsd.org/current-users/2000/10/12/0008.html

 There shouldn't have been any changes in those working directories, so
 that might have been an example of the first possibility.

 I must say one of the reasons I only use the rsync'ed copy of the
 repository (other than the obvious advantages of speed and availability)
 is for the very reason that the logic and handling in the client/server
 mode in CVS is far less trustworthy than the local mode.  I've just
 never really trusted the client/server code very much, despite having
 worked on it for quite a long time as a maintainer.

 --=20
 						Greg A. Woods
 						Planix, Inc.

 <woods@planix.com>       +1 416 218 0099        http://www.planix.com/

 --pgp-sign-Multipart_Thu_Mar_11_16:33:02_2010-1
 Content-Type: application/pgp-signature
 Content-Transfer-Encoding: 7bit

 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.9 (NetBSD)

 iD8DBQBLmWGOZn1xt3i/9H8RArx6AKD+7rzm69Dv87kcC5GtqTq9JGnIxACg2isL
 asoZDXmS+RB++kd4znNo2AE=
 =n9wx
 -----END PGP SIGNATURE-----

 --pgp-sign-Multipart_Thu_Mar_11_16:33:02_2010-1--

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org, dholland@eecs.harvard.edu
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Fri, 12 Mar 2010 21:46:23 +0100

 On Thu, Mar 11, 2010 at 09:35:02PM +0000, Greg A. Woods wrote:
 >  Subject: bin/42961: cvs near-silently throws away local mods
 >  >=20
 >  > This is a pretty serious problem, as I and a lot of people keep random
 >  > uncommitted/local changes in source trees and rely on CVS not to
 >  > randomly lose them like this.
 >  
 >  In fact this is in effect the only way I use CVS with NetBSD.
 >  
 >  All my local changes are kept in local working directories where I
 >  frequently run "cvs -q update -dP" to get upstream changes.
 >  
 >  As far as I can remember though I've never seen the error you mention.

 I've never seen this either; although I frequenly update trees
 with local patches

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Fri, 12 Mar 2010 22:55:20 -0500

 On Thu, 11 Mar 2010 21:35:02 +0000 (UTC)
 "Greg A. Woods" <woods@planix.ca> wrote:

 >  I must say one of the reasons I only use the rsync'ed copy of the
 >  repository (other than the obvious advantages of speed and availability)
 >  is for the very reason that the logic and handling in the client/server
 >  mode in CVS is far less trustworthy than the local mode.  I've just
 >  never really trusted the client/server code very much, despite having
 >  worked on it for quite a long time as a maintainer.

 I've never seen that problem either, but I also use a local repository
 copy updated via cvsup, via NFS rather than using ssh or pserver.
 I agree that it's a serious bug, however.
 -- 
 Matt

From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Tue, 16 Mar 2010 17:36:44 -0400

 On Mar 11,  7:15pm, dholland@eecs.harvard.edu (dholland@eecs.harvard.edu) wrote:
 -- Subject: bin/42961: cvs near-silently throws away local mods

 | Worse, in a large update most likely the message would go by without
 | anyone noticing it; I tend to grep update logs for '^[CM]' and some
 | people just do find . -name '*.rej' to see if anything failed.

 The original file is kept in .#<filename>:<revision>. CVS does this
 (I think) when it determines that the file does not match the original
 at all. I think it is a bug that it does not say what it did, and perhaps
 it should exit with a non-zero code... Anyway, your changes are not lost :-)

 christos

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Tue, 16 Mar 2010 22:02:22 +0000

 On Tue, Mar 16, 2010 at 09:40:05PM +0000, Christos Zoulas wrote:
  >  | Worse, in a large update most likely the message would go by without
  >  | anyone noticing it; I tend to grep update logs for '^[CM]' and some
  >  | people just do find . -name '*.rej' to see if anything failed.
  >  
  >  The original file is kept in .#<filename>:<revision>. CVS does this
  >  (I think) when it determines that the file does not match the original
  >  at all.

 It didn't this time.

 My changes aren't lost, but only because of pre-existing .#* files
 from earlier merges.

 What worries me is how many times this may have happened without me
 noticing at all, and how many changes may have been lost entirely...

 -- 
 David A. Holland
 dholland@netbsd.org

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Mon, 24 Dec 2012 04:15:13 +0000

 On Thu, Mar 11, 2010 at 07:15:01PM +0000, dholland@eecs.harvard.edu wrote:
  >    valkyrie% cvs -q update -dP
  >    P lfs/lfs_vfsops.c
  >    P ufs/ufs_lookup.c
  >    P ufs/ufs_wapbl.c
  >    cvs update: checksum failure after patch to ufs/ufs_wapbl.c; will refetch
  >    cvs client: refetching unpatchable files
  >    cvs update: warning: ufs/ufs_wapbl.c was lost
  >    U ufs/ufs_wapbl.c
  >    valkyrie% 
  > 
  > and now:
  >    valkyrie% cvs diff ufs/ufs_wapbl.c
  >    valkyrie% 
  > 
  > Note that it did not even attempt to merge.

 I just had this happen again, after a couple years without seeing it:

 macaran% cvs -q update -dP
 P files/make_pkgsrc.mak
 cvs update: checksum failure after patch to files/make_pkgsrc.mak;
 will refetch
 cvs client: refetching unpatchable files
 cvs update: warning: files/make_pkgsrc.mak was lost
 U files/make_pkgsrc.mak
 macaran% cvs diff files/make_pkgsrc.mak 
 macaran% 

 In this case I can report exactly what happened to provoke the
 situation, although I haven't tried reproducing it:

 (1) edit a file on host A (macaran), test-compile
 (2) apply the same edit by hand on host B, make a one-character typo
 (3) fail to notice the typo, commit (to pkgsrc, so remote cvs on all hosts)
 (4) update on macaran
 (5) get a merge conflict
 (6) examine the merge conflict, notice the typo
 (7) fix the typo on host B
 (8) commit on host B
 (9) update on macaran again, with the above result.

 This time the changes that got lost were not important (being just the
 merge conflict with the typo) but it did lose them; there is no .#
 file from the event above, only one from step (4).

 The version of CVS on macaran is:

 Concurrent Versions System (CVS) 1.12.13 (client/server)
 with CVSACL Patch 1.2.5 (cvsacl.sourceforge.net)

 and it's running 6.99.11 from October 15.

 -- 
 David A. Holland
 dholland@netbsd.org

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Mon, 21 Jul 2014 06:18:36 +0000

 On Mon, Dec 24, 2012 at 04:20:05AM +0000, David Holland wrote:
  >  I just had this happen again, after a couple years without seeing it:

 ...and again today.

 I think it may be connected in some fashion to files with local
 changes that still contain merge conflicts from a previous update.

 -- 
 David A. Holland
 dholland@netbsd.org

From: "David H. Gutteridge" <david@gutteridge.ca>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/42961: cvs near-silently throws away local mods
Date: Sat, 25 Jan 2020 13:40:30 -0500

 I hit this issue the other day. In my case, what I lost wasn't very
 important or aggravating to lose, but I realized I have various changes
 across two trees that I haven't committed (or abandoned), so I guess I
 learned something there.

 Anyway, I mention this mostly because:

 On Mon, 21 Jul 2014 at 06:18:36 +0000, David Holland wrote:
 >I think it may be connected in some fashion to files with local
 >changes that still contain merge conflicts from a previous update.

 I'm positive that was the case for me.

 (I guess this topic may be moot soon, anyway.)

 Dave

Home	PR Database Search