NetBSD Problem Report #57314

From www@netbsd.org  Thu Mar 30 20:50:41 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 3F40B1A9239
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 30 Mar 2023 20:50:41 +0000 (UTC)
Message-Id: <20230330205039.AB6871A923C@mollari.NetBSD.org>
Date: Thu, 30 Mar 2023 20:50:39 +0000 (UTC)
From: jspath55@gmail.com
Reply-To: jspath55@gmail.com
To: gnats-bugs@NetBSD.org
Subject: ATF unit tests fail on 3 of 7 cases in program lib/libc/c063/t_utimensat on evbarm/Rpi 02W
X-Send-Pr-Version: www-1.0

>Number:         57314
>Category:       lib
>Synopsis:       ATF unit tests fail on 3 of 7 cases in program lib/libc/c063/t_utimensat on evbarm/Rpi 02W
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          analyzed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 30 20:55:00 +0000 2023
>Closed-Date:    
>Last-Modified:  Mon Apr 03 13:35:01 +0000 2023
>Originator:     Jim Spath
>Release:        10.0_BETA
>Organization:
>Environment:
NetBSD arm32 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Fri Jan 13 19:15:32 UTC 2023  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC evbarm
>Description:
One of the ATF test programs fails on 3 of 7 cases, only on a Raspberry Pi Zero 2W (GENERIC evbarm kernel). The same program fully succeeds every case, on other architectures including GENERIC64 evbarm / aarch64 / Pi 4.

Pi02W program executed:
$ file /usr/tests/lib/libc/c063/t_utimensat
/usr/tests/lib/libc/c063/t_utimensat: ELF 32-bit LSB pie executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /usr/libexec/ld.elf_so, for NetBSD 10.0, compiled for: earmv7hf, not stripped

(same program on GENERIC64)
$ file /usr/tests/lib/libc/c063/t_utimensat
/usr/tests/lib/libc/c063/t_utimensat: ELF 64-bit LSB pie executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /usr/libexec/ld.elf_so, for NetBSD 10.0, not stripped


Failures:
utimensat_fd
utimensat_fdcwd
utimensat_fdlink

The failing systems run on microSD media, while other systems are on SSD or USB storage.


>How-To-Repeat:
Two test results follow:

$  atf-run lib/libc/c063/t_utimensat 

(1)

Content-Type: application/X-atf-tps; version="3"

info: atf.version, Automated Testing Framework 0.20 (atf-0.20)
info: tests.root, /usr/tests
info: time.start, Tue Mar 28 02:38:33 UTC 2023
info: uname.sysname, NetBSD
info: uname.nodename, arm32
info: uname.release, 10.0_BETA
info: uname.version, NetBSD 10.0_BETA (GENERIC) #0: Fri Jan 13 19:15:32 UTC 2023 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC
info: uname.machine, evbarm
info: env, PWD=/usr/tests
info: env, _=/usr/bin/atf-run
info: env, PATH=/usr/bin:/bin:/usr/pkg/bin:/usr/local/bin:/usr/X11R7/bin:/sbin:/usr/sbin
info: env, TERM=xterm
info: env, SHELL=/bin/sh
info: env, SHLVL=1
tps-count: 1
tp-start: 1679971113.847015, lib/libc/c063/t_utimensat, 7
tc-start: 1679971113.847090, utimensat_fd
tc-end: 1679971113.937139, utimensat_fd, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:78: st.st_atimespec.tv_sec == tptr[0].tv_sec not met
tc-start: 1679971113.975402, utimensat_fdcwd
tc-end: 1679971114.61099, utimensat_fdcwd, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:103: st.st_atimespec.tv_sec == tptr[0].tv_sec not met
tc-start: 1679971114.103105, utimensat_fdcwderr
tc-end: 1679971114.179727, utimensat_fdcwderr, passed
tc-start: 1679971114.222949, utimensat_fderr1
tc-end: 1679971114.299049, utimensat_fderr1, passed
tc-start: 1679971114.338284, utimensat_fderr2
tc-end: 1679971114.427645, utimensat_fderr2, passed
tc-start: 1679971114.468194, utimensat_fderr3
tc-end: 1679971114.559644, utimensat_fderr3, passed
tc-start: 1679971114.602969, utimensat_fdlink
tc-end: 1679971114.693602, utimensat_fdlink, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:195: st.st_atimespec.tv_sec == tptr[0].tv_sec not met
tp-end: 1679971114.734524, lib/libc/c063/t_utimensat
info: time.end, Tue Mar 28 02:38:34 UTC 2023

(2)

Thu Mar 30 20:32:27 UTC 2023
Content-Type: application/X-atf-tps; version="3"

info: atf.version, Automated Testing Framework 0.20 (atf-0.20)
info: tests.root, /usr/tests
info: time.start, Thu Mar 30 20:32:27 UTC 2023
info: uname.sysname, NetBSD
info: uname.nodename, arm32
info: uname.release, 10.0_BETA
info: uname.version, NetBSD 10.0_BETA (GENERIC) #0: Fri Jan 13 19:15:32 UTC 2023 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC
info: uname.machine, evbarm
info: env, PWD=/usr/tests
info: env, _=/usr/bin/atf-run
info: env, PATH=/usr/bin:/bin:/usr/pkg/bin:/usr/local/bin:/usr/X11R7/bin:/sbin:/usr/sbin
info: env, SHELL=/bin/sh
info: env, SHLVL=1
tps-count: 1
tp-start: 1680208348.150417, lib/libc/c063/t_utimensat, 7
tc-start: 1680208348.150519, utimensat_fd
tc-end: 1680208348.254754, utimensat_fd, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:78: st.st_atimespec.tv_sec == tptr[0].tv_sec not met
tc-start: 1680208348.306778, utimensat_fdcwd
tc-end: 1680208348.406359, utimensat_fdcwd, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:103: st.st_atimespec.tv_sec == tptr[0].tv_sec not met
tc-start: 1680208348.456770, utimensat_fdcwderr
tc-end: 1680208348.542213, utimensat_fdcwderr, passed
tc-start: 1680208348.625978, utimensat_fderr1
tc-end: 1680208348.870874, utimensat_fderr1, passed
tc-start: 1680208349.5955, utimensat_fderr2
tc-end: 1680208349.279219, utimensat_fderr2, passed
tc-start: 1680208349.415803, utimensat_fderr3
tc-end: 1680208349.470019, utimensat_fderr3, passed
tc-start: 1680208349.497313, utimensat_fdlink
tc-end: 1680208349.552037, utimensat_fdlink, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:195: st.st_atimespec.tv_sec == tptr[0].tv_sec not met
tp-end: 1680208349.579470, lib/libc/c063/t_utimensat
info: time.end, Thu Mar 30 20:32:29 UTC 2023


>Fix:
I have been unable to glean any root cause, nor why these specific 3 cases fail. I assume there is a common denominator.


>Release-Note:

>Audit-Trail:
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program lib/libc/c063/t_utimensat on evbarm/Rpi 02W
Date: Fri, 31 Mar 2023 08:33:38 -0000 (UTC)

 jspath55@gmail.com writes:

 >tc-end: 1679971113.937139, utimensat_fd, failed, /usr/src/tests/lib/libc/c063/t_utimensat.c:78: st.st_atimespec.tv_sec == tptr[0].tv_sec not met

 >The failing systems run on microSD media, while other systems are on SSD or USB storage.


 These errors occur if the filesystem used for testing cannot handle
 timestamps with even 1-second resolution, e.g.: msdosfs.

 Do you run the test with /tmp being a FAT filesystem or do you
 redirect the test to use a FAT filesystem?

From: Jim Spath <jspath55@gmail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program
 lib/libc/c063/t_utimensat on evbarm/Rpi 02W
Date: Fri, 31 Mar 2023 10:24:44 -0400

 Michael - thank you for the reply. It sounds like this result might be
 expected though the underlying conditions may not be obvious.

 The /tmp filesystem is on the SD card for these systems. I followed
 these steps for the install:

 http://mail-index.netbsd.org/port-arm/2022/02/14/msg007592.html

 >  /tmp being a FAT filesystem or do you redirect the test to use a FAT fil=
 esystem?

 Not that I am aware. The "disk" layout looks like this:

 $ df -m /tmp
 Filesystem   1M-blocks      Used     Avail %Cap Mounted on
 /dev/ld0a        58604      5215     50458   9% /

 $ df -m
 Filesystem   1M-blocks      Used     Avail %Cap Mounted on
 /dev/ld0a        58604      5215     50458   9% /
 /dev/ld0e           79        73         5  92% /boot
 ptyfs                0         0         0 100% /dev/pts
 procfs               0         0         0 100% /proc
 tmpfs              112         0       111   0% /var/shm

 $ cat /etc/fstab
 # NetBSD /etc/fstab
 # See /usr/share/examples/fstab/ for more examples.
 ROOT.a          /               ffs     rw,noatime      1 1
 ROOT.e          /boot           msdos   rw      1 1
 ptyfs           /dev/pts        ptyfs   rw
 procfs          /proc           procfs  rw
 tmpfs           /var/shm        tmpfs   rw,-m1777,-sram%25

 # disklabel -r ld0
 # /dev/rld0:
 type: SCSI
 disk: STORAGE DEVICE
 label: fictitious
 flags: removable
 bytes/sector: 512
 sectors/track: 32
 tracks/cylinder: 64
 sectors/cylinder: 2048
 cylinders: 1217
 total sectors: 122167296
 rpm: 3600
 interleave: 1
 trackskew: 0
 cylinderskew: 0
 headswitch: 0           # microseconds
 track-to-track seek: 0  # microseconds
 drivedata: 0

 8 partitions:
 #        size    offset     fstype [fsize bsize cpg/sgs]
  a: 121970688    196608     4.2BSD      0     0     0  # (Cyl.     96 -  59=
 651)
  c: 122167296         0     unused      0     0        # (Cyl.      0 -  59=
 651)
  d: 122167296         0     unused      0     0        # (Cyl.      0 -  59=
 651)
  e:    163840     32768      MSDOS                     # (Cyl.     16 -    =
  95)

 dmesg shows:
 [     1.840447] ld0: 59652 MB, 7604 cyl, 255 head, 63 sec, 512
 bytes/sect x 122167296 sectors
 [     1.850447] ld0: 4-bit width, High-Speed/SDR25, 50.000 MHz

 I can try with TMPDIR set elsewhere, or other permutations that might
 shed light.

 Jim

 On Fri, Mar 31, 2023 at 4:35=E2=80=AFAM Michael van Elst <mlelstv@serpens.d=
 e> wrote:
 >
 > The following reply was made to PR lib/57314; it has been noted by GNATS.
 >
 > From: mlelstv@serpens.de (Michael van Elst)
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program li=
 b/libc/c063/t_utimensat on evbarm/Rpi 02W
 > Date: Fri, 31 Mar 2023 08:33:38 -0000 (UTC)
 >
 >  jspath55@gmail.com writes:
 >
 >  >tc-end: 1679971113.937139, utimensat_fd, failed, /usr/src/tests/lib/lib=
 c/c063/t_utimensat.c:78: st.st_atimespec.tv_sec =3D=3D tptr[0].tv_sec not m=
 et
 >
 >  >The failing systems run on microSD media, while other systems are on SS=
 D or USB storage.
 >
 >
 >  These errors occur if the filesystem used for testing cannot handle
 >  timestamps with even 1-second resolution, e.g.: msdosfs.
 >
 >  Do you run the test with /tmp being a FAT filesystem or do you
 >  redirect the test to use a FAT filesystem?
 >

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program lib/libc/c063/t_utimensat on evbarm/Rpi 02W
Date: Fri, 31 Mar 2023 14:56:07 -0000 (UTC)

 jspath55@gmail.com (Jim Spath) writes:

 > 
 > >  /tmp being a FAT filesystem or do you redirect the test to use a FAT fil=
 > esystem?
 > 
 > Not that I am aware. The "disk" layout looks like this:
 > 
 > $ df -m /tmp
 > Filesystem   1M-blocks      Used     Avail %Cap Mounted on
 > /dev/ld0a        58604      5215     50458   9% /

 > $ cat /etc/fstab
 > # NetBSD /etc/fstab
 > # See /usr/share/examples/fstab/ for more examples.
 > ROOT.a          /               ffs     rw,noatime      1 1
 > ROOT.e          /boot           msdos   rw      1 1
 > ptyfs           /dev/pts        ptyfs   rw
 > procfs          /proc           procfs  rw
 > tmpfs           /var/shm        tmpfs   rw,-m1777,-sram%25


 Doesn't look like FAT (msdos) and the armv7.img uses ffs1.
 But to be sure, please run the mount command.


 > I can try with TMPDIR set elsewhere, or other permutations that might
 > shed light.

 atf uses ATF_WORKDIR and defaults to /tmp.


From: Jim Spath <jspath55@gmail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program
 lib/libc/c063/t_utimensat on evbarm/Rpi 02W
Date: Fri, 31 Mar 2023 11:56:09 -0400

 (apologies if this double posts; gnats and gmail are like chalk and cheese)

 Michael wrote:

 > please run the mount command.

 $ mount
 /dev/ld0a on / type ffs (noatime, local)
 /dev/ld0e on /boot type msdos (local)
 ptyfs on /dev/pts type ptyfs (local)
 procfs on /proc type procfs (local)
 tmpfs on /var/shm type tmpfs (local)

 > atf uses ATF_WORKDIR and defaults to /tmp

 All 7 test cases succeeded when I set the ATF_WORKDIR to an
 NFS-mounted directory.

 The time man page shows noatime means "Never update the access time
 field for files" so that sounds like the probable cause.

 Jim



 On Fri, Mar 31, 2023 at 11:00=E2=80=AFAM Michael van Elst <mlelstv@serpens.=
 de> wrote:
 >
 > The following reply was made to PR lib/57314; it has been noted by GNATS.
 >
 > From: mlelstv@serpens.de (Michael van Elst)
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program li=
 b/libc/c063/t_utimensat on evbarm/Rpi 02W
 > Date: Fri, 31 Mar 2023 14:56:07 -0000 (UTC)
 >
 >  jspath55@gmail.com (Jim Spath) writes:
 >
 >  >
 >  > >  /tmp being a FAT filesystem or do you redirect the test to use a FA=
 T fil=3D
 >  > esystem?
 >  >
 >  > Not that I am aware. The "disk" layout looks like this:
 >  >
 >  > $ df -m /tmp
 >  > Filesystem   1M-blocks      Used     Avail %Cap Mounted on
 >  > /dev/ld0a        58604      5215     50458   9% /
 >
 >  > $ cat /etc/fstab
 >  > # NetBSD /etc/fstab
 >  > # See /usr/share/examples/fstab/ for more examples.
 >  > ROOT.a          /               ffs     rw,noatime      1 1
 >  > ROOT.e          /boot           msdos   rw      1 1
 >  > ptyfs           /dev/pts        ptyfs   rw
 >  > procfs          /proc           procfs  rw
 >  > tmpfs           /var/shm        tmpfs   rw,-m1777,-sram%25
 >
 >
 >  Doesn't look like FAT (msdos) and the armv7.img uses ffs1.
 >  But to be sure, please run the mount command.
 >
 >
 >  > I can try with TMPDIR set elsewhere, or other permutations that might
 >  > shed light.
 >
 >  atf uses ATF_WORKDIR and defaults to /tmp.
 >
 >

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program
 lib/libc/c063/t_utimensat on evbarm/Rpi 02W
Date: Fri, 31 Mar 2023 18:14:07 +0200

 On Fri, Mar 31, 2023 at 04:00:05PM +0000, Jim Spath wrote:
 >  All 7 test cases succeeded when I set the ATF_WORKDIR to an
 >  NFS-mounted directory.

 You do not want to run atf test with a workdir on an SD card or similar.
 Best practise is to create a tmpfs for /tmp (of at least 32MB size).

 Martin

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314: ATF unit tests fail on 3 of 7 cases in program lib/libc/c063/t_utimensat on evbarm/Rpi 02W
Date: Fri, 31 Mar 2023 16:31:02 -0000 (UTC)

 jspath55@gmail.com (Jim Spath) writes:

 > The time man page shows noatime means "Never update the access time
 > field for files" so that sounds like the probable cause.

 Indeed....


State-Changed-From-To: open->analyzed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sat, 01 Apr 2023 22:26:49 +0000
State-Changed-Why:
Config problem; but it seems like we ought to improve things to
make this hole harder to fall into.

It seems reasonable for the timestamp tests to check for noatime
and skip the atime cases. Maybe they should also check file system
type.


From: Jim Spath <jspath55@gmail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/57314 (ATF unit tests fail on 3 of 7 cases in program
 lib/libc/c063/t_utimensat on evbarm/Rpi 02W)
Date: Mon, 3 Apr 2023 09:33:01 -0400

 Thank you for moving this ticket to "analyzed". While the subject test
 cases are relatively minor, I've learned a lot about the Automated
 Test Framework meanwhile. I admit not even knowing it existed a few
 weeks ago despite being a NetBSD feature for years. For consideration,
 a few suggestions:

 1) Failing test cases on file systems with "noatime"
 Add a filesystem check and skip tests that would fail with noatime
 set. (As Dave Holland mentions, fs type also)

 2) Documentation
 a) atf-run(1) add NOTE section describing SD-card class issue.
 b) add "See also atf-config" (where ATF_WORKDIR is described)
 c) update https://wiki.netbsd.org/tutorials/atf/
 Include atf-config in the Introduction where atf-check etc are.
 Add details on atf-workdir in the FAQ section "Do I need to remove
 temporary files?"
 Mention ATF_WORKDIR: " atf uses ATF_WORKDIR and defaults to /tmp."
 Add this comment from Martin Husemann:
 "Best practise is to create a tmpfs for /tmp (of at least 32MB size)."

 3) Future-proofing
 Beyond the scope of this PR and beyond NetBSD 10 BETA plans, the next
 atf version (0.21) is in PkgSrc.
 I tried to build it and am not done yet.
 Unclear if that suite could co-exist with base system 0.20, or if it
 would override. A conflicting change is the removal of ATF_WORKDIR.
 Others might assume, as I did, that TMPDIR affects atf.

 http://web.mit.edu/freebsd/head/contrib/atf/NEWS
 > Changes in version 0.21
 > Released on October 23rd, 2014.
 > * Removed ATF_WORKDIR.  The only remaining consumers have been converted to use the standard TMPDIR environment variable.  As a benefit, and because
   Kyua forces the TMPDIR to live within the test case's work
 directory, any stale files left behind by ATF will be automatically
 cleaned up.

 a) mention 0.21 in the atf man page as a planned future upgrade
 b) Hint about test harnesses that define tmp locations


 Jim

>Unformatted:

Home	PR Database Search