NetBSD Problem Report #55370
From triaxx@netbsd.org Fri Jun 12 09:59:14 2020
Return-Path: <triaxx@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 352EC1A9219
for <gnats-bugs@gnats.NetBSD.org>; Fri, 12 Jun 2020 09:59:14 +0000 (UTC)
Message-Id: <20200612095913.085B01A9242@mollari.NetBSD.org>
Date: Fri, 12 Jun 2020 09:59:12 +0000 (UTC)
From: triaxx@netbsd.org
Reply-To: triaxx@netbsd.org
To: gnats-bugs@NetBSD.org
Subject: Invalid modification/connection time with mount_null(8)
X-Send-Pr-Version: 3.95
>Number: 55370
>Category: kern
>Synopsis: Invalid modification/connection time with mount_null(8)
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: chs
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Fri Jun 12 10:00:00 +0000 2020
>Closed-Date: Mon Apr 26 18:45:54 +0000 2021
>Last-Modified: Mon Apr 26 18:45:54 +0000 2021
>Originator: Frederic Fauberteau
>Release: NetBSD 9.99.64
>Organization:
>Environment:
System: NetBSD hydralisk.triaxx.org 9.99.64 NetBSD 9.99.64 (GENERIC) #0: Wed Jun 3 07:06:18 UTC 2020 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
Some files seem to have inconsistent modification time when they are created on null filesystem. The problem has been identified due to configuration/building errors for some pkgsrc packages built using pbulk in sandbox with PKGSRCDIR and WRKOBJDIR mounted as null filesystem.
Here is the example of fonts/unifont:
Part of src/Makefile:
--------------------------------------------------
LOCALBINDIR = ../bin
CPROGS = unibmp2hex unicoverage unidup unibdf2hex unibmpbump \
unifont1per unifontpic unigencircles unigenwidth \
unihex2bmp unihexgen unipagecount
all: $(CPROGS) bin
bin: bin-stamp
bin-stamp: $(CPROGS)
install -m0755 -d $(LOCALBINDIR)
install -m0755 $(CPROGS) $(LOCALBINDIR)
\rm -f $(CPROGS)
touch $@
--------------------------------------------------
I consider two working directories for WRKOBJDIR: /tmp/work and /tmp/null as a binding of /tmp/work using mount_null(8).
$ mkdir /tmp/work /tmp/null
# mount -t null /tmp/work /tmp/null
First, I configure fonts/unifont to /tmp/work using the pkgsrc framework and I manually run make to inspect the times of some files.
$ bmake WRKOBJDIR=/tmp/work configure
$ cd /tmp/work/fonts/unifont/work/unifont-13.0.02
$ /usr/pkg/bin/gmake
$ $ stat -f "st_atime=%a st_mtime=%m %N" src/bin-stamp bin/unibmp2hex
st_atime=1591954072 st_mtime=1591954072 src/bin-stamp
st_atime=1591954072 st_mtime=1591954072 bin/unibmp2hex
Using the debug option of gmake, I can inspect the prerequisites of target when I try to run 'make install':
$ /usr/pkg/bin/gmake -d > make.dump 2>&1
Prerequisite '../bin/unibmp2hex' is older than target 'bin-stamp'.
Then, I configure fonts/unifont to /tmp/null (after cleaning the previous configuration) and I reproduce the same process.
$ bmake WRKOBJDIR=/tmp/null configure
$ cd /tmp/null/fonts/unifont/work/unifont-13.0.02
$ ktrace -di /usr/pkg/bin/gmake
$ stat -f "st_atime=%a st_mtime=%m %N" src/bin-stamp bin/unibmp2hex
st_atime=1591954536 st_mtime=1591954536 src/bin-stamp
st_atime=1591954536 st_mtime=1591954541 bin/unibmp2hex
Now the value of st_mtime differs from the one of st_atime. It results in an unexpected behavior during the prerequisite analysis when running 'make install':
$ /usr/pkg/bin/gmake -d > make.dump 2>&1
Prerequisite '../bin/unibmp2hex' is newer than target 'bin-stamp'.
When I inspect the trace produced by make (kdump -T | less), I can notice that the time of the last syscall that handles unibmp2hex is a time 1591954536:
20569 20569 rm 1591954536.785814687 CALL unlink(0x7f7fff63ddbe)
20569 20569 rm 1591954536.785817137 NAMI "unibmp2hex"
The modification time of 1591954541 for bin/unibmp2hex seems not to be due to make since the last line of the trace is prior:
15647 15647 gmake 1591954537.052526140 CALL exit(0)
>How-To-Repeat:
The problem is reproducible.
>Fix:
>Release-Note:
>Audit-Trail:
From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Tue, 21 Jul 2020 10:59:54 +0200
On Fri, Jun 12, 2020 at 10:00:00AM +0000, triaxx@netbsd.org wrote:
> >Number: 55370
> >Category: kern
> >Synopsis: Invalid modification/connection time with mount_null(8)
This breaks the build of the lang/gcc* packages with null mounts because of:
Comparing stages 2 and 3
warning: gcc/cc1obj-checksum.o differs
warning: gcc/cc1objplus-checksum.o differs
Bootstrap comparison failure!
gcc/build/version.o differs
gcc/version.o differs
...
(timestamps)
so a fix would be appreciated.
Thomas
From: coypu@sdf.org
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Tue, 21 Jul 2020 18:54:30 +0000
time wise, this fits with ad@'s namei changes in May 26.
this is a revert to test this hypothesis, for anyone who can easily
reproduce this problem: http://coypu.sdf.org/namei
From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: maya@NetBSD.org
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Thu, 30 Jul 2020 16:20:04 +0200
On Tue, Jul 21, 2020 at 06:55:01PM +0000, coypu@sdf.org wrote:
> time wise, this fits with ad@'s namei changes in May 26.
> this is a revert to test this hypothesis, for anyone who can easily
> reproduce this problem: http://coypu.sdf.org/namei
I tried today's current with this patch reverse-applied, but I still
see the same issue building gcc8 in a sandbox with null mounts:
ving directory '/scratch/lang/gcc8/work/build'
Comparing stages 2 and 3
warning: gcc/cc1obj-checksum.o differs
warning: gcc/cc1objplus-checksum.o differs
Bootstrap comparison failure!
gcc/version.o differs
gcc/crtbegin.o differs
gcc/crtend.o differs
gcc/crtbeginS.o differs
gcc/crtendS.o differs
gcc/crtbeginT.o differs
gmp/mpf/init.o differs
gmp/mpf/get_dfl_prec.o differs
gmp/mpf/int_p.o differs
gmp/mpf/fits_uint.o differs
gmp/mpf/fits_ushort.o differs
gmp/mpn/fib_table.o differs
gmp/mpn/com.o differs
gmp/mpn/diveby3.o differs
gmp/mpn/cmp.o differs
gmp/mpq/clear.o differs
gmp/mpz/clear.o differs
gmp/mpz/fits_ushort.o differs
gmp/mpz/get_d.o differs
gmp/mpz/iset_d.o differs
gmp/mpz/iset_ui.o differs
gmp/mpz/perfpow.o differs
gmp/mpz/size.o differs
gmp/rand/randclr.o differs
gmp/rand/randsd.o differs
gmp/invalid.o differs
gmp/mp_set_fns.o differs
intl/dcngettext.o differs
intl/ngettext.o differs
intl/osdep.o differs
isl/isl_deprecated.o differs
isl/isl_ffs.o differs
isl/isl_version.o differs
isl/polyhedron_detect_equalities.o differs
libbacktrace/.libs/state.o differs
libiberty/pic/lrealpath.o differs
libiberty/pic/stack-limit.o differs
libiberty/pic/xasprintf.o differs
libiberty/pic/xatexit.o differs
libiberty/fdmatch.o differs
libiberty/fopen_unlocked.o differs
libiberty/getruntime.o differs
libiberty/hex.o differs
libiberty/lrealpath.o differs
libiberty/timeval-utils.o differs
libiberty/xatexit.o differs
libiberty/xmemdup.o differs
libiberty/xstrdup.o differs
libiberty/xstrerror.o differs
libiberty/mempcpy.o differs
mpc/src/add_ui.o differs
mpc/src/cmp.o differs
mpc/src/conj.o differs
mpc/src/cosh.o differs
mpc/src/cos.o differs
mpc/src/div_fr.o differs
mpc/src/fr_sub.o differs
mpc/src/get_version.o differs
mpc/src/get_prec.o differs
mpc/src/init3.o differs
mpc/src/mul_si.o differs
mpc/src/real.o differs
mpc/src/sin.o differs
mpc/src/sub.o differs
mpc/src/tanh.o differs
mpc/src/ui_div.o differs
mpfr/src/isinf.o differs
mpfr/src/isnan.o differs
mpfr/src/print_rnd_mode.o differs
mpfr/src/set_inf.o differs
mpfr/src/version.o differs
mpfr/src/mp_clz_tab.o differs
mpfr/src/iszero.o differs
mpfr/src/logging.o differs
mpfr/src/get_patches.o differs
mpfr/src/isregular.o differs
mpfr/src/mpfr-mini-gmp.o differs
x86_64--netbsd/libgcc/sfp-exceptions.o differs
x86_64--netbsd/libgcc/negtf2.o differs
x86_64--netbsd/libgcc/floatunsitf.o differs
x86_64--netbsd/libgcc/enable-execute-stack.o differs
x86_64--netbsd/libgcc/sfp-exceptions_s.o differs
x86_64--netbsd/libgcc/negtf2_s.o differs
x86_64--netbsd/libgcc/enable-execute-stack_s.o differs
x86_64--netbsd/libgcc/crtbegin.o differs
x86_64--netbsd/libgcc/crtend.o differs
x86_64--netbsd/libgcc/crtendS.o differs
x86_64--netbsd/libgcc/crtbeginS.o differs
x86_64--netbsd/libgcc/crtbeginT.o differs
x86_64--netbsd/libgomp/.libs/mutex.o differs
x86_64--netbsd/libgomp/.libs/ptrlock.o differs
x86_64--netbsd/libgomp/ptrlock.o differs
x86_64--netbsd/libstdc++-v3/libsupc++/atexit_arm.o differs
x86_64--netbsd/libstdc++-v3/src/c++98/list-aux-2.o differs
x86_64--netbsd/libstdc++-v3/src/.libs/compatibility-debug_list-2.o differs
x86_64--netbsd/libstdc++-v3/src/compatibility-debug_list-2.o differs
gmake[2]: *** [Makefile:26302: compare] Error 1
gmake[2]: Leaving directory '/scratch/lang/gcc8/work/build'
gmake[1]: *** [Makefile:26282: stage3-bubble] Error 2
gmake[1]: Leaving directory '/scratch/lang/gcc8/work/build'
gmake: *** [Makefile:949: all] Error 2
*** Error code 2
Stop.
In case someone's interested, here's such a sandbox:
tmpfs on /sandboxes/client4 type tmpfs (local)
ptyfs on /sandboxes/client4/dev/ptyfs type ptyfs (local)
procfs on /sandboxes/client4/proc type procfs (local)
/bin on /sandboxes/client4/bin type null (read-only, local)
/sbin on /sandboxes/client4/sbin type null (read-only, local)
/lib on /sandboxes/client4/lib type null (read-only, local)
/libexec on /sandboxes/client4/libexec type null (read-only, local)
/usr/bin on /sandboxes/client4/usr/bin type null (read-only, local)
/usr/games on /sandboxes/client4/usr/games type null (read-only, local)
/usr/include on /sandboxes/client4/usr/include type null (read-only, local)
/usr/lib on /sandboxes/client4/usr/lib type null (read-only, local)
/usr/libdata on /sandboxes/client4/usr/libdata type null (read-only, local)
/usr/libexec on /sandboxes/client4/usr/libexec type null (read-only, local)
/usr/share on /sandboxes/client4/usr/share type null (read-only, local)
/usr/sbin on /sandboxes/client4/usr/sbin type null (read-only, local)
/usr/X11R7 on /sandboxes/client4/usr/X11R7 type null (read-only, local)
/var/mail on /sandboxes/client4/var/mail type null (read-only, local)
/usr/src on /sandboxes/client4/usr/src type null (read-only, local)
/usr/pkgsrc on /sandboxes/client4/usr/pkgsrc type null (local)
/usr/xsrc on /sandboxes/client4/usr/xsrc type null (read-only, local)
/packages/9.99.69 on /sandboxes/client4/packages type null (local)
/distfiles on /sandboxes/client4/distfiles type null (local)
From: Patrick Welche <prlw1@cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Fri, 14 Aug 2020 10:32:25 +0100
Reached this bug via
http://mail-index.netbsd.org/current-users/2020/06/18/msg038924.html
having just got
gmake[2]: Entering directory '/tmp/pkgsrc/lang/gcc8/work.x86_64/build'
gmake[3]: Entering directory '/tmp/pkgsrc/lang/gcc8/work.x86_64/build'
rm -f stage_current
gmake[3]: Leaving directory '/tmp/pkgsrc/lang/gcc8/work.x86_64/build'
Comparing stages 2 and 3
warning: gcc/cc1obj-checksum.o differs
warning: gcc/cc1objplus-checksum.o differs
Bootstrap comparison failure!
gcc/crtbegin.o differs
gcc/crtend.o differs
etc., with 9.99.70/amd64, -current pkgsrc. The surprise is: no null
mounts involved, just make on the command line in a zfs
ZFS /usr/pkgsrc/lang/gcc8 and tmpfs /tmp.
From: Patrick Welche <prlw1@cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Wed, 7 Oct 2020 16:43:42 +0100
See the same with gcc9 (with its MAKE_JOBS_SAFE=no), 9.99.73/amd64.
gmake[3]: Leaving directory '/tmp/pkgsrc/lang/gcc9/work.x86_64/build'
Comparing stages 2 and 3
warning: gcc/cc1obj-checksum.o differs
warning: gcc/cc1objplus-checksum.o differs
Bootstrap comparison failure!
gcc/graphite-isl-ast-to-gimple.o differs
gcc/crtbegin.o differs
...
(zfs pkgsrc, tmpfs workdir)
From: Patrick Welche <prlw1@cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Thu, 8 Oct 2020 14:09:26 +0100
On Wed, Oct 07, 2020 at 03:45:01PM +0000, Patrick Welche wrote:
> (zfs pkgsrc, tmpfs workdir)
Changing /usr/pkgsrc from ZFS to FFS didn't change anything, so reproducible
without NULLFS nor anything else out of the oridinary.
From: Patrick Welche <prlw1@cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55370: Invalid modification/connection time with
mount_null(8)
Date: Mon, 14 Dec 2020 15:48:34 +0000
The problem I was seeing has been fixed by chs@ in
https://src.fossil.netbsd.org/info/4c02f505d012befb
Disable use of UBC_FAULTBUSY in tmpfs_write() for now
thanks
Responsible-Changed-From-To: kern-bug-people->chs
Responsible-Changed-By: chs@NetBSD.org
Responsible-Changed-When: Mon, 26 Apr 2021 18:45:54 +0000
Responsible-Changed-Why:
State-Changed-From-To: open->closed
State-Changed-By: chs@NetBSD.org
State-Changed-When: Mon, 26 Apr 2021 18:45:54 +0000
State-Changed-Why:
no longer reproducible, appears to be fixed now.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home