NetBSD Problem Report #24457

Received: (qmail 8577 invoked by uid 605); 17 Feb 2004 14:34:55 -0000
Message-Id: <200402171434.i1HEYENl004309@apb.cequrux.com>
Date: Tue, 17 Feb 2004 16:34:14 +0200 (SAST)
From: apb@cequrux.com
Sender: gnats-bugs-owner@NetBSD.org
Reply-To: apb@cequrux.com
To: gnats-bugs@gnats.NetBSD.org
Subject: metalog contains incorrect information for hard links
X-Send-Pr-Version: 3.95

>Number:         24457
>Category:       toolchain
>Synopsis:       metalog contains incorrect information for hard links
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    joerg
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Feb 17 14:35:00 +0000 2004
>Closed-Date:    
>Last-Modified:  Sat Jun 06 22:15:13 +0000 2009
>Originator:     Alan Barrett
>Release:        NetBSD 1.6ZK
>Organization:
Not much
>Environment:
NetBSD 1.6ZK
Architecture: i386
Machine: i386
>Description:
Hard-linked files can end up with incorrect information in
${DESTDIR}/METALOG during a build; this incorrect information
also appears in /etc/mtree/set.* after an installation.

>How-To-Repeat:
$ build.sh -U [more options] release

$ tar -z -tvf ${RELEASEDIR}/${MACHINE}/binary/sets/base.tgz | grep 'sys*stat'
-r-xr-sr-x  1 root     kmem       94915 Feb 16 17:23 ./usr/bin/systat
-r-xr-xr-x  2 root     wheel          0 Feb 16 17:23 ./usr/bin/sysstat == ./usr/bin/systat
$ mkdir ${TEMPROOT}
$ tar -z -xpf ${RELEASEDIR}/${MACHINE}/binary/sets/base.tgz -C ${TEMPROOT}
$ ( cd ${TEMPROOT}/usr/bin ; ls -l sys*tat )
-r-xr-sr-x  2 root  kmem  94915 Feb 16 17:23 sysstat
-r-xr-sr-x  2 root  kmem  94915 Feb 16 17:23 systat
$ # note that sysstat is a hard link to systat

$ mtree -p ${TEMPROOT} -f ${TEMPROOT}/etc/mtree/base.set -e | grep -A 2 sysstat
usr/bin/sysstat:
        gid (0, 2)
	permissions (0555, 02555)
$ # note that mtree thinks sysstat should have had a different group and mode

$ grep 'bin/sys*stat' ${DESTDIR}/METALOG
./usr/bin/systat type=file mode=02555 uname=root gname=kmem time=1076945030.477119 size=94915 sha1=c4c7d795fbce718cdd955a312491259c8abbedce
./usr/bin/sysstat type=file mode=0555 size=94915 sha1=c4c7d795fbce718cdd955a312491259c8abbedce
$ grep 'bin/sys*stat' ${TEMPROOT}/etc/mtree/set.base
./usr/bin/systat type=file uname=root gname=kmem mode=02555 size=94915 time=1076945030.477119 sha1=c4c7d795fbce718cdd955a312491259c8abbedce
./usr/bin/sysstat type=file mode=0555 size=94915 sha1=c4c7d795fbce718cdd955a312491259c8abbedce
$ # note the incorrect modes for sysstat, but correct modes for systat

>Fix:
I don't see an easy fix.  When a build is performed in unprivileged mode
using a metalog, bsd.links.mk doesn't know the correct uname/gname/mode,
and install(1) can't just guess it by stat(2)ing the target (because the
target will have different permissions as a result of the unprivileged
build).

The correct uname/gname/mode for the about-to-be-installed link (e.g.
usr/bin/sysstat) should already be recorded in the metalog's entry
for the previously-installed file (e.g. usr/bin/systat), so I suppose
bsd.links.mk could grep for it and pass it to install(1).

Or the mtree(1) syntax (and the metalog) could be extended to explicitly
record information like "sysstat should be a hard link to systat".

>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: install-manager->joerg
Responsible-Changed-By: joerg@NetBSD.org
Responsible-Changed-When: Wed, 16 Apr 2008 18:22:54 +0000
Responsible-Changed-Why:
I'll work on it.


From: Alan Barrett <apb@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/24457 CVS commit: src
Date: Fri, 10 Apr 2009 16:16:12 +0000

 Module Name:	src
 Committed By:	apb
 Date:		Fri Apr 10 16:16:12 UTC 2009

 Modified Files:
 	src/distrib/common: parselist.awk
 	src/share/dict: Makefile
 	src/share/mk: bsd.README bsd.hostprog.mk bsd.kmodule.mk bsd.lib.mk
 	    bsd.links.mk bsd.man.mk bsd.prog.mk
 	src/share/zoneinfo: Makefile

 Log Message:
 Add LINKSOWN, LINKSGRP, and LINKSMODE variables for use by bsd.links.mk
 when installing hard links.  They have no effect except when using a
 metalog, in which case the information is added to the metalog.  In
 the future, these variables may be replaced by a method for explicitly
 recording hard links in a metadata log.

 Also change a few things that called ${INSTALL_LINK} without going
 through bsd.links.mk.

 Reviewed by perry and joerg.  This should fix PR 24457 and PR 41155.


 To generate a diff of this commit:
 cvs rdiff -u -r1.15 -r1.16 src/distrib/common/parselist.awk
 cvs rdiff -u -r1.16 -r1.17 src/share/dict/Makefile
 cvs rdiff -u -r1.248 -r1.249 src/share/mk/bsd.README
 cvs rdiff -u -r1.54 -r1.55 src/share/mk/bsd.hostprog.mk
 cvs rdiff -u -r1.18 -r1.19 src/share/mk/bsd.kmodule.mk
 cvs rdiff -u -r1.297 -r1.298 src/share/mk/bsd.lib.mk
 cvs rdiff -u -r1.33 -r1.34 src/share/mk/bsd.links.mk
 cvs rdiff -u -r1.99 -r1.100 src/share/mk/bsd.man.mk
 cvs rdiff -u -r1.240 -r1.241 src/share/mk/bsd.prog.mk
 cvs rdiff -u -r1.42 -r1.43 src/share/zoneinfo/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 18 May 2009 07:30:58 +0000
State-Changed-Why:
Fixed?


From: Alan Barrett <apb@cequrux.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: toolchain/24457 (metalog contains incorrect information for
	hard links)
Date: Tue, 19 May 2009 11:10:08 +0300

 On Mon, 18 May 2009, dholland@NetBSD.org wrote:
 > Synopsis: metalog contains incorrect information for hard links
 > Fixed?

  From one point of view, it's fixed (but perhaps it needs a pullup to
 netbsd-5).  From another point of view, a proper fix would require
 extending the mtree format, and that has not been done.

 --apb (Alan Barrett)

State-Changed-From-To: feedback->open
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 25 May 2009 18:31:46 +0000
State-Changed-Why:
Not fully fixed yet.


From: Manuel Bouyer <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/24457 CVS commit: [netbsd-5] src
Date: Sat, 6 Jun 2009 22:10:13 +0000

 Module Name:	src
 Committed By:	bouyer
 Date:		Sat Jun  6 22:10:13 UTC 2009

 Modified Files:
 	src/distrib/common [netbsd-5]: parselist.awk
 	src/distrib/sets [netbsd-5]: maketars
 	src/share/dict [netbsd-5]: Makefile
 	src/share/mk [netbsd-5]: bsd.README bsd.hostprog.mk bsd.kmodule.mk
 	    bsd.lib.mk bsd.links.mk bsd.man.mk bsd.prog.mk
 	src/share/zoneinfo [netbsd-5]: Makefile
 	src/usr.bin/xinstall [netbsd-5]: xinstall.c

 Log Message:
 Pull up following revision(s) (requested by snj in ticket #790):
 	share/mk/bsd.lib.mk: revision 1.298
 	share/mk/bsd.man.mk: revision 1.100
 	share/mk/bsd.hostprog.mk: revision 1.55
 	distrib/sets/maketars: revision 1.66
 	share/zoneinfo/Makefile: revision 1.43
 	share/mk/bsd.kmodule.mk: revision 1.19
 	usr.bin/xinstall/xinstall.c: revisions 1.106 - 1.108 via patch
 	share/mk/bsd.prog.mk: revision 1.241
 	share/dict/Makefile: revision 1.17
 	share/mk/bsd.README: revision 1.249
 	distrib/common/parselist.awk: revision 1.16
 	share/mk/bsd.links.mk: revision 1.34
 Explicitly sort entries on preparing set files from METALOG.
 METALOG could have different order due to install(1) race
 on parallel builds, and mtree(8) doesn't sort files.
 Should fix inconsistent shared sets among builds as seen in
 /pub/NetBSD-daily/netbsd-5/200904010000Z/shared/ and
 /pub/NetBSD-daily/netbsd-5/200904010002Z/shared/ dirs.
 Okay'ed by snj@.
 Changes for installing with a metalog:
 * When installing hard links and using a metalog, if -o, -g, -m, or -f
   args were explicitly specified on the command line, then believe them,
   but do not implicitly believe uname/gname/mode/flags from the file
   system.
 * Output fields in the same order used by mtree.
 Fix -Wshadow -Wcast-qual issues
 Re-calculate size and digestresult after stripping, to ensure that
 correct values appear in the metalog.
 Add LINKSOWN, LINKSGRP, and LINKSMODE variables for use by bsd.links.mk
 when installing hard links.  They have no effect except when using a
 metalog, in which case the information is added to the metalog.  In
 the future, these variables may be replaced by a method for explicitly
 recording hard links in a metadata log.
 Also change a few things that called ${INSTALL_LINK} without going
 through bsd.links.mk.
 Reviewed by perry and joerg.  This should fix PR 24457 and PR 41155.


 To generate a diff of this commit:
 cvs rdiff -u -r1.15 -r1.15.4.1 src/distrib/common/parselist.awk
 cvs rdiff -u -r1.64.6.2 -r1.64.6.3 src/distrib/sets/maketars
 cvs rdiff -u -r1.16 -r1.16.38.1 src/share/dict/Makefile
 cvs rdiff -u -r1.236.2.7 -r1.236.2.8 src/share/mk/bsd.README
 cvs rdiff -u -r1.53 -r1.53.2.1 src/share/mk/bsd.hostprog.mk
 cvs rdiff -u -r1.13.2.1 -r1.13.2.2 src/share/mk/bsd.kmodule.mk
 cvs rdiff -u -r1.289.2.2 -r1.289.2.3 src/share/mk/bsd.lib.mk
 cvs rdiff -u -r1.32 -r1.32.24.1 src/share/mk/bsd.links.mk
 cvs rdiff -u -r1.98 -r1.98.2.1 src/share/mk/bsd.man.mk
 cvs rdiff -u -r1.239.2.1 -r1.239.2.2 src/share/mk/bsd.prog.mk
 cvs rdiff -u -r1.42 -r1.42.12.1 src/share/zoneinfo/Makefile
 cvs rdiff -u -r1.103 -r1.103.4.1 src/usr.bin/xinstall/xinstall.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Manuel Bouyer <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/24457 CVS commit: [netbsd-5-0] src
Date: Sat, 6 Jun 2009 22:10:50 +0000

 Module Name:	src
 Committed By:	bouyer
 Date:		Sat Jun  6 22:10:50 UTC 2009

 Modified Files:
 	src/distrib/common [netbsd-5-0]: parselist.awk
 	src/distrib/sets [netbsd-5-0]: maketars
 	src/share/dict [netbsd-5-0]: Makefile
 	src/share/mk [netbsd-5-0]: bsd.README bsd.hostprog.mk bsd.kmodule.mk
 	    bsd.lib.mk bsd.links.mk bsd.man.mk bsd.prog.mk
 	src/share/zoneinfo [netbsd-5-0]: Makefile
 	src/usr.bin/xinstall [netbsd-5-0]: xinstall.c

 Log Message:
 Pull up following revision(s) (requested by snj in ticket #790):
 	share/mk/bsd.lib.mk: revision 1.298
 	share/mk/bsd.man.mk: revision 1.100
 	share/mk/bsd.hostprog.mk: revision 1.55
 	distrib/sets/maketars: revision 1.66
 	share/zoneinfo/Makefile: revision 1.43
 	share/mk/bsd.kmodule.mk: revision 1.19
 	usr.bin/xinstall/xinstall.c: revisions 1.106 - 1.108 via patch
 	share/mk/bsd.prog.mk: revision 1.241
 	share/dict/Makefile: revision 1.17
 	share/mk/bsd.README: revision 1.249
 	distrib/common/parselist.awk: revision 1.16
 	share/mk/bsd.links.mk: revision 1.34
 Explicitly sort entries on preparing set files from METALOG.
 METALOG could have different order due to install(1) race
 on parallel builds, and mtree(8) doesn't sort files.
 Should fix inconsistent shared sets among builds as seen in
 /pub/NetBSD-daily/netbsd-5/200904010000Z/shared/ and
 /pub/NetBSD-daily/netbsd-5/200904010002Z/shared/ dirs.
 Okay'ed by snj@.
 Changes for installing with a metalog:
 * When installing hard links and using a metalog, if -o, -g, -m, or -f
   args were explicitly specified on the command line, then believe them,
   but do not implicitly believe uname/gname/mode/flags from the file
   system.
 * Output fields in the same order used by mtree.
 Fix -Wshadow -Wcast-qual issues
 Re-calculate size and digestresult after stripping, to ensure that
 correct values appear in the metalog.
 Add LINKSOWN, LINKSGRP, and LINKSMODE variables for use by bsd.links.mk
 when installing hard links.  They have no effect except when using a
 metalog, in which case the information is added to the metalog.  In
 the future, these variables may be replaced by a method for explicitly
 recording hard links in a metadata log.
 Also change a few things that called ${INSTALL_LINK} without going
 through bsd.links.mk.
 Reviewed by perry and joerg.  This should fix PR 24457 and PR 41155.


 To generate a diff of this commit:
 cvs rdiff -u -r1.15 -r1.15.8.1 src/distrib/common/parselist.awk
 cvs rdiff -u -r1.64.6.2 -r1.64.6.2.2.1 src/distrib/sets/maketars
 cvs rdiff -u -r1.16 -r1.16.42.1 src/share/dict/Makefile
 cvs rdiff -u -r1.236.2.7 -r1.236.2.7.2.1 src/share/mk/bsd.README
 cvs rdiff -u -r1.53 -r1.53.4.1 src/share/mk/bsd.hostprog.mk
 cvs rdiff -u -r1.13.2.1 -r1.13.2.1.2.1 src/share/mk/bsd.kmodule.mk
 cvs rdiff -u -r1.289.2.2 -r1.289.2.2.2.1 src/share/mk/bsd.lib.mk
 cvs rdiff -u -r1.32 -r1.32.28.1 src/share/mk/bsd.links.mk
 cvs rdiff -u -r1.98 -r1.98.6.1 src/share/mk/bsd.man.mk
 cvs rdiff -u -r1.239.2.1 -r1.239.2.1.2.1 src/share/mk/bsd.prog.mk
 cvs rdiff -u -r1.42 -r1.42.16.1 src/share/zoneinfo/Makefile
 cvs rdiff -u -r1.103 -r1.103.6.1 src/usr.bin/xinstall/xinstall.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

Home	PR Database Search