NetBSD Problem Report #57463

From lukem@thistledown.com.au  Sat Jun 10 19:52:31 2023
Return-Path: <lukem@thistledown.com.au>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id D487F1A923D
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 10 Jun 2023 19:52:31 +0000 (UTC)
Message-Id: <20230610195228.BD07A5A71@thoreau.thistledown.com.au>
Date: Sun, 11 Jun 2023 05:52:28 +1000 (AEST)
From: lukem@netbsd.org
Reply-To: lukem@netbsd.org
To: gnats-bugs@NetBSD.org
Subject: issues with build to DESTDIR on case insenstive file system
X-Send-Pr-Version: 3.95

>Number:         57463
>Category:       toolchain
>Synopsis:       issues with build to DESTDIR on case insenstive file system
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    toolchain-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Jun 10 19:55:00 +0000 2023
>Originator:     Luke Mewburn
>Release:        NetBSD-current as at 2023-07-10
>Organization:
NetBSD
>Environment:
>Description:

tl;dr: building on case insenstive file systems will result in
sets will missing files and/or files with incorrect contents
because of duplicate case-insensitive file names, mostly
manual pages.


Various files installed into DESTDIR that have conflicting names
if DESTDIR is on a case-insensitve file system won't
be available; only one of the duplicates will exist.

This isn't obvious at first, since during "make install" portion of the
build the conflicting-if-case-insensitve filenames just get overwritten
without an error, and the sets appear to be generated fine.

I first observed this when running on a macOS host
	% ./build.sh -D DESTDIR -E install=ROOTDIR
where these are both on case insensitive file systems,
which failed with:
	pax: Cannot link file ROOTDIR/./usr/share/man/html9/FREE.html to itself
	pax: Cannot link file ROOTDIR/./usr/share/man/html9/ISSET.html to itself


Checking the METALOG for case-insensitive matches of "9/free" using
	% grep -i 9/free DESTDIR/METALOG

On a NetBSD host:
	./usr/share/man/html9/FREE.html type=file uname=root gname=wheel mode=0444 size=12225 sha256=871393d86f3f8fa1f94fe44458e1daf2cfbb06bacdbc974a882c073a25d06558
	./usr/share/man/html9/free.html type=file uname=root gname=wheel mode=0444 size=12225 sha256=871393d86f3f8fa1f94fe44458e1daf2cfbb06bacdbc974a882c073a25d06558
	./usr/share/man/man9/FREE.9 type=file uname=root gname=wheel mode=0444 size=6806 sha256=f23c599362b8a57aaf568ded6e9633f86eefcd5b057daa239c1b949d9c74e171
	./usr/share/man/man9/free.9 type=file uname=root gname=wheel mode=0444 size=6806 sha256=f23c599362b8a57aaf568ded6e9633f86eefcd5b057daa239c1b949d9c74e171

On a case-insenstive macOS host:
	./usr/share/man/html9/FREE.html type=file uname=root gname=wheel mode=0444 size=12225 sha256=871393d86f3f8fa1f94fe44458e1daf2cfbb06bacdbc974a882c073a25d06558
	./usr/share/man/html9/free.html type=file uname=root gname=wheel mode=0444 size=12225 sha256=871393d86f3f8fa1f94fe44458e1daf2cfbb06bacdbc974a882c073a25d06558
	./usr/share/man/man9/free.9 type=file uname=root gname=wheel mode=0444 size=6806 sha256=f23c599362b8a57aaf568ded6e9633f86eefcd5b057daa239c1b949d9c74e171

Notice usr/share/man/man9/FREE.9 is missing?


Checking the output of the comp.tar.xz set built on case insensitive
macOS host versus a NetBSD host showed that some of the files were
stored differently too.

On a NetBSD host:
	% tar -Jtvf SETDIR/comp.tar.xz | grep -i 9/free | wc -l
	18

On a macOS host:
	% tar -Jtvf SETDIR/comp.tar.xz | grep -i 9/free | wc -l
	8


A method to determine all the potential case-insensitive filename
conflicts is to build to a DESTDIR on a NetBSD host and run:
	% cd DESTDIR
	% find . -type f | tr A-Z a-z | sort | uniq -d

E.g.,:
	./usr/share/man/html2/_exit.html
	./usr/share/man/html3/gcq_init.html
	./usr/share/man/html3/gcq_init_head.html
	./usr/share/man/html4/dbcool.html
	./usr/share/man/html4/i386/pcibios.html
	./usr/share/man/html8/makedev.html
	./usr/share/man/html8/makedev.local.html
	./usr/share/man/html9/cardbus_conf_read.html
	./usr/share/man/html9/cardbus_conf_write.html
	./usr/share/man/html9/cardbus_free_tag.html
	./usr/share/man/html9/cardbus_function_disable.html
	./usr/share/man/html9/cardbus_function_enable.html
	./usr/share/man/html9/cardbus_make_tag.html
	./usr/share/man/html9/cardbus_mapreg_map.html
	./usr/share/man/html9/cardbus_mapreg_unmap.html
	./usr/share/man/html9/delay.html
	./usr/share/man/html9/free.html
	./usr/share/man/html9/isset.html
	./usr/share/man/html9/knote.html
	./usr/share/man/html9/m_prepend.html
	./usr/share/man/html9/malloc.html
	./usr/share/man/man2/_exit.2
	./usr/share/man/man3/gcq_init.3
	./usr/share/man/man3/gcq_init_head.3
	./usr/share/man/man4/dbcool.4
	./usr/share/man/man4/i386/pcibios.4
	./usr/share/man/man8/makedev.8
	./usr/share/man/man8/makedev.local.8
	./usr/share/man/man9/cardbus_conf_read.9
	./usr/share/man/man9/cardbus_conf_write.9
	./usr/share/man/man9/cardbus_free_tag.9
	./usr/share/man/man9/cardbus_function_disable.9
	./usr/share/man/man9/cardbus_function_enable.9
	./usr/share/man/man9/cardbus_make_tag.9
	./usr/share/man/man9/cardbus_mapreg_map.9
	./usr/share/man/man9/cardbus_mapreg_unmap.9
	./usr/share/man/man9/delay.9
	./usr/share/man/man9/free.9
	./usr/share/man/man9/isset.9
	./usr/share/man/man9/knote.9
	./usr/share/man/man9/m_prepend.9
	./usr/share/man/man9/malloc.9

>How-To-Repeat:
Build releases and sets on a case insenstive file system.
Carefuly compare the differences in the generated sets.

>Fix:
To be determined.

It's long been a goal of build.sh to support building on systems
with case insensitive file systems, and our CVS source was adapted
many years ago to allow source checkouts on that.

Home	PR Database Search