NetBSD Problem Report #57265

From www@netbsd.org  Thu Mar  9 16:05:12 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 3DF351A9239
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  9 Mar 2023 16:05:12 +0000 (UTC)
Message-Id: <20230309160510.3E55C1A923A@mollari.NetBSD.org>
Date: Thu,  9 Mar 2023 16:05:10 +0000 (UTC)
From: jbglaw@lug-owl.de
Reply-To: jbglaw@lug-owl.de
To: gnats-bugs@NetBSD.org
Subject: Cross-build broken from Linux amd64 to NetBSD amd64
X-Send-Pr-Version: www-1.0

>Number:         57265
>Category:       port-amd64
>Synopsis:       Cross-build broken from Linux amd64 to NetBSD amd64
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-amd64-maintainer
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 09 16:10:00 +0000 2023
>Closed-Date:    Wed Mar 15 11:38:18 +0000 2023
>Last-Modified:  Wed Mar 15 11:38:18 +0000 2023
>Originator:     Jan-Benedict Glaw
>Release:        NetBSD src TRUNK
>Organization:
>Environment:
Linux lili 5.16.0-4-amd64 #1 SMP PREEMPT Debian 5.16.12-1 (2022-03-08) x86_64 GNU/Linux
>Description:
There is an interesting interaction of local host libs vs. newly build
NetBSD target libs. When attempting to cross-build NetBSD amd64/x86_64
from a Linux amd64 box, build breaks as zlib is built.

The symptom is that when nbmandoc is called (worked endless times
before), it won't find "libc.so.12" as soon as libz.so got build.  So
my guess is that the GNU libc ld.so tries to start nbmandoc,
encounters the freshly build *target* libz and tries to link it, which
may pull in target libc.

This cross-compilation used to always fail for me, it though had a
phase (where a lot was updates) where it worked.  But that was a
short-lifed exception.

A build log for a recent build attempt (even with -N 4) can be found
at http://toolchain.lug-owl.de/laminar/log/netbsd-amd64-x86_64/11 .


[bld rls 2023-03-09 11:51:40] #     build  libz/libz.so.1.0
[bld rls 2023-03-09 11:51:40] rm -f libz.so.1.0
[bld rls 2023-03-09 11:51:40] + rm -f libz.so.1.0
[bld rls 2023-03-09 11:51:40] /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/bin/x86_64--netbsd-gcc  -shared -Wl,-soname,libz.so.1  -Wl,--warn-shared-textrel -Wl,-Map=libz.so.1.map   --sysroot=/var/lib/laminar/run/netbsd-amd64-x86_64/11/dest-amd64-x86_64 -Wl,--warn-shared-textrel -Wl,-z,relro  -o libz.so.1.0.tmp  -Wl,-rpath,/lib  -L=/lib -Wl,-x  -Wl,--whole-archive libz_pic.a  -Wl,--no-whole-archive  
[bld rls 2023-03-09 11:51:40] + /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/bin/x86_64--netbsd-gcc -shared -Wl,-soname,libz.so.1 -Wl,--warn-shared-textrel -Wl,-Map=libz.so.1.map --sysroot=/var/lib/laminar/run/netbsd-amd64-x86_64/11/dest-amd64-x86_64 -Wl,--warn-shared-textrel -Wl,-z,relro -o libz.so.1.0.tmp -Wl,-rpath,/lib -L=/lib -Wl,-x -Wl,--whole-archive libz_pic.a -Wl,--no-whole-archive
[bld rls 2023-03-09 11:51:40] mv -f libz.so.1.0.tmp libz.so.1.0
[bld rls 2023-03-09 11:51:40] + mv -f libz.so.1.0.tmp libz.so.1.0
[bld rls 2023-03-09 11:51:40] ln -sf libz.so.1.0 libz.so.1.tmp
[bld rls 2023-03-09 11:51:40] + ln -sf libz.so.1.0 libz.so.1.tmp
[bld rls 2023-03-09 11:51:40] mv -f libz.so.1.tmp libz.so.1
[bld rls 2023-03-09 11:51:40] + mv -f libz.so.1.tmp libz.so.1
[bld rls 2023-03-09 11:51:40] ln -sf libz.so.1.0 libz.so.tmp
[bld rls 2023-03-09 11:51:40] + ln -sf libz.so.1.0 libz.so.tmp
[bld rls 2023-03-09 11:51:40] mv -f libz.so.tmp libz.so
[bld rls 2023-03-09 11:51:40] + mv -f libz.so.tmp libz.so
[bld rls 2023-03-09 11:51:40] echo '#  ' " format " libz/zlib.html3
[bld rls 2023-03-09 11:51:40] + echo #    format  libz/zlib.html3
[bld rls 2023-03-09 11:51:40] #    format  libz/zlib.html3
[bld rls 2023-03-09 11:51:40] if test "" != "yes"; then  /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/bin/nbmandoc -Thtml -Oman=../html%S/%N.html,style=../style.css  /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/lib/libz/zlib.3 > zlib.html3.tmp &&  mv -f zlib.html3.tmp zlib.html3;  else  GROFF_ENCODING=  GROFF_BIN_PATH=/var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/lib/groff  GROFF_FONT_PATH=/var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/share/groff/site-font:/var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/share/groff/font  GROFF_TMAC_PATH=/var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/share/groff/site-tmac:/var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/share/groff/tmac /var/lib/laminar/run/
 netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/bin/nbgroff -dpaper=letter -Tlatin1 -mdoc2html /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/lib/libz/zlib.3   > zlib.html3.tmp && mv -f zlib.html3.tmp zlib.html3;  fi
[bld rls 2023-03-09 11:51:40] + test  != yes
[bld rls 2023-03-09 11:51:40] + /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/bin/nbmandoc -Thtml -Oman=../html%S/%N.html,style=../style.css /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/lib/libz/zlib.3
[bld rls 2023-03-09 11:51:40] /var/lib/laminar/run/netbsd-amd64-x86_64/11/NetBSD-src/obj/tooldir.Linux-5.16.0-4-amd64-x86_64/bin/nbmandoc: error while loading shared libraries: libc.so.12: cannot open shared object file: No such file or directory

>How-To-Repeat:
On a amd64 Linux box:

./build.sh -N 4 -P -U -m amd64 -a x86_64 -E -D /var/lib/laminar/run/netbsd-amd64-x86_64/11/dest-amd64-x86_64 -R /var/lib/laminar/run/netbsd-amd64-x86_64/11/release-amd64-x86_64 tools

./build.sh -N 4 -P -U -u -m amd64 -a x86_64 -E -D /var/lib/laminar/run/netbsd-amd64-x86_64/11/dest-amd64-x86_64 -R /var/lib/laminar/run/netbsd-amd64-x86_64/11/release-amd64-x86_64 release
>Fix:
I *guess* a LD_LIBRARY_PATH with '.' or the $DESTDIR/lib or something
similar sneaked in.

>Release-Note:

>Audit-Trail:

From: Taylor R Campbell <riastradh@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: port-amd64-maintainer@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: port-amd64/57265: Cross-build broken from Linux amd64 to NetBSD amd64
Date: Thu, 9 Mar 2023 17:16:15 +0000

 > Date: Thu,  9 Mar 2023 16:10:01 +0000 (UTC)
 > From: jbglaw@lug-owl.de
 > 
 > I *guess* a LD_LIBRARY_PATH with '.' or the $DESTDIR/lib or
 > something similar sneaked in.

 Nothing in the build uses LD_LIBRARY_PATH,[*] and it is unlikely for
 nbmandoc to have been inadvertently linked against a libc that doesn't
 exist yet.  But from your build log, I see:

 ++ export LD_LIBRARY_PATH=/usr/lib/gcc-snapshot/lib:
 ++ LD_LIBRARY_PATH=/usr/lib/gcc-snapshot/lib:

 Is that in your build environment?  What happens if you remove it?

 [*] Exceptions: bsd.test.mk, not relevant here, only in native builds;
     and libexec/httpd/lua/Makefile, for a development-only test
     target, not used automatically as far as I know, and not likely
     relevant to mandoc or libc issues.

From: Valery Ushakov <uwe@stderr.spb.ru>
To: gnats-bugs@netbsd.org
Cc: Taylor R Campbell <riastradh@NetBSD.org>
Subject: Re: port-amd64/57265: Cross-build broken from Linux amd64 to NetBSD
 amd64
Date: Thu, 9 Mar 2023 20:49:22 +0300

 On Thu, Mar 09, 2023 at 17:16:15 +0000, Taylor R Campbell wrote:

 > Nothing in the build uses LD_LIBRARY_PATH,[*] and it is unlikely for
 > nbmandoc to have been inadvertently linked against a libc that doesn't
 > exist yet.  But from your build log, I see:

 It's not the nbmandoc, it must be the freshly built libz.so that gets
 pulled in, and then that netbsd libz.so depends on (netbsd)
 libc.so.12, which cannot be found.

 -uwe

From: Jan-Benedict Glaw <jbglaw@lug-owl.de>
To: gnats-bugs@netbsd.org
Cc: port-amd64-maintainer@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: port-amd64/57265: Cross-build broken from Linux amd64 to NetBSD
 amd64
Date: Thu, 9 Mar 2023 19:07:06 +0100

 --bai2ihk4tmftpslt
 Content-Type: text/plain; charset=utf-8
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable

 On Thu, 2023-03-09 17:20:01 +0000, Taylor R Campbell <riastradh@NetBSD.org>=
  wrote:
 > From: jbglaw@lug-owl.de
 > >=20
 > > I *guess* a LD_LIBRARY_PATH with '.' or the $DESTDIR/lib or
 > > something similar sneaked in.
 > =20
 >  Nothing in the build uses LD_LIBRARY_PATH,[*] and it is unlikely for
 >  nbmandoc to have been inadvertently linked against a libc that doesn't
 >  exist yet.  But from your build log, I see:

 As I wrote, nbmandoc is called several times beforehand and all these
 invocations were running perfectly fine. "libc.so.6" is the one on a
 GNU/Linux system.  It starts with libz being build (and nbmandoc links
 with -lz).

 >  ++ export LD_LIBRARY_PATH=3D/usr/lib/gcc-snapshot/lib:
 >  ++ LD_LIBRARY_PATH=3D/usr/lib/gcc-snapshot/lib:
 > =20
 >  Is that in your build environment?  What happens if you remove it?

 It is!  That's from a little to choose between different compilers
 (ie. a recent GCC snapshot, the "usual" (older) system GCC, some CLANG
 versions.  But that LD_LIBRARY_PATH should in no way enable ld.so to
 pull in some libz (that's a guess) from the destdir, should it?

   Just started a build using the sytem compiler (not setting
 anything). I wonder (read: hope!) that the empty path component
 doesn't wrongly resolve to "." ...  (If the next build succeeds, my
 next test would be LD_LIBRARY_PATH=3D"/does_not_exist:"...)

 >  [*] Exceptions: bsd.test.mk, not relevant here, only in native builds;
 >      and libexec/httpd/lua/Makefile, for a development-only test
 >      target, not used automatically as far as I know, and not likely
 >      relevant to mandoc or libc issues.

 As I guess, it's just mandoc showing the symptom: It's linked against
 -lz and works before libz is build, but as the library emerges, it
 fails by not finding libc.so.12 (which is NetBSD libc.)

 MfG, JBG

 --=20

 --bai2ihk4tmftpslt
 Content-Type: application/pgp-signature; name="signature.asc"

 -----BEGIN PGP SIGNATURE-----

 iF0EABECAB0WIQQlDTvPcScNjKREqWEdvV51g5nhuwUCZAogSAAKCRAdvV51g5nh
 uyLiAJ4l06yLnXeYPp72i2rmA8GITDzepACffbfk7jCwVDnlLYdmX+nBAAeQDtM=
 =FWTy
 -----END PGP SIGNATURE-----

 --bai2ihk4tmftpslt--

From: Jan-Benedict Glaw <jbglaw@lug-owl.de>
To: gnats-bugs@netbsd.org
Cc: port-amd64-maintainer@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: port-amd64/57265: Cross-build broken from Linux amd64 to NetBSD
 amd64
Date: Thu, 9 Mar 2023 20:47:44 +0100

 --ljj7eos5y5qd7glk
 Content-Type: text/plain; charset=utf-8
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable

 On Thu, 2023-03-09 17:50:02 +0000, Valery Ushakov <uwe@stderr.spb.ru> wrote:
 > On Thu, Mar 09, 2023 at 17:16:15 +0000, Taylor R Campbell wrote:
 >=20
 > > Nothing in the build uses LD_LIBRARY_PATH,[*] and it is unlikely for
 > > nbmandoc to have been inadvertently linked against a libc that doesn't
 > > exist yet.  But from your build log, I see:
 >=20
 > It's not the nbmandoc, it must be the freshly built libz.so that gets
 > pulled in, and then that netbsd libz.so depends on (netbsd)
 > libc.so.12, which cannot be found.

 That's actually a good point: As NetBSD libz is found, but not NetBSD
 libc, it's probably _not_ picking it up from $destdir/lib, but from
 $PWD.  Just realized that, as it seems, the freshly libz isn't yet
 installed to $destdir, and if $destdir/lib were in LD_LIBRARY_PATH,
 that would have been found as well.

 MfG, JBG

 --=20

 --ljj7eos5y5qd7glk
 Content-Type: application/pgp-signature; name="signature.asc"

 -----BEGIN PGP SIGNATURE-----

 iF0EABECAB0WIQQlDTvPcScNjKREqWEdvV51g5nhuwUCZAo33QAKCRAdvV51g5nh
 uzMsAJ9E8PbhF7HZTEDZNspGpeStCcx7jACgjJtMCkqt94rTY0jfLuaMEySbxgg=
 =3B37
 -----END PGP SIGNATURE-----

 --ljj7eos5y5qd7glk--

From: Taylor R Campbell <riastradh@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-amd64/57265: Cross-build broken from Linux amd64 to NetBSD amd64
Date: Wed, 15 Mar 2023 11:30:44 +0000

 This is a multi-part message in MIME format.
 --=_yEQegd6KE+J0RR5NTU8CY9iUdk+bjG6u

 gnats-bugs@ got dropped from cc list

 --=_yEQegd6KE+J0RR5NTU8CY9iUdk+bjG6u
 Content-Type: message/rfc822
 Content-Disposition: inline

 Date: Fri, 10 Mar 2023 12:21:22 +0100
 From: Jan-Benedict Glaw <jbglaw@lug-owl.de>
 To: port-amd64-maintainer@netbsd.org, gnats-admin@netbsd.org,
 	netbsd-bugs@netbsd.org
 Subject: Re: port-amd64/57265: Cross-build broken from Linux amd64 to NetBSD
 	amd64
 Message-ID: <20230310112122.bznozwomag5kywz4@lug-owl.de>
 In-Reply-To: <20230309181002.C7FC21A923A@mollari.NetBSD.org>
 References: <pr-port-amd64-57265@gnats.netbsd.org>
 	<20230309160510.3E55C1A923A@mollari.NetBSD.org>
 	<20230309181002.C7FC21A923A@mollari.NetBSD.org>
 MIME-Version: 1.0
 Content-Type: multipart/signed; micalg=pgp-sha1;
 	protocol="application/pgp-signature"; boundary="cnot5engxscipb6p"
 Content-Disposition: inline


 --cnot5engxscipb6p
 Content-Type: text/plain; charset=utf-8
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable

 Hi!

 On Thu, 2023-03-09 18:10:02 +0000, Jan-Benedict Glaw <jbglaw@lug-owl.de> wr=
 ote:
 >  >  ++ export LD_LIBRARY_PATH=3D3D/usr/lib/gcc-snapshot/lib:
 >  >  ++ LD_LIBRARY_PATH=3D3D/usr/lib/gcc-snapshot/lib:
 >  > =3D20
 >  >  Is that in your build environment?  What happens if you remove it?
 > =20
 >  It is!  That's from a little to choose between different compilers
 >  (ie. a recent GCC snapshot, the "usual" (older) system GCC, some CLANG
 >  versions.  But that LD_LIBRARY_PATH should in no way enable ld.so to
 >  pull in some libz (that's a guess) from the destdir, should it?
 > =20
 >    Just started a build using the sytem compiler (not setting
 >  anything). I wonder (read: hope!) that the empty path component
 >  doesn't wrongly resolve to "." ...  (If the next build succeeds, my
 >  next test would be LD_LIBRARY_PATH=3D"/does_not_exist:"...)

 Turns out: `bash` has the documented feature of treating an empty path
 component in `$PATH` as the current directory---and GNU libc's `ld.so`
 does the same for `$LD_LIBRARY_PATH`. So ... this ticket can be
 closed, not having an trailing colon is the fix here, it's _not_ a
 NetBSD issue at all. I'm sorry!  OTOH, I'll continue to inquire the
 Debian Security team if this is desired behavior or a CVE-worthy bug.

   I guess that there are quite some place where people assign $PATH
 and $LD_LIBRARY_PATH like this:

     export LD_LIBRARY_PATH=3D"/my/lib:${LD_LIBRARY_PATH}"

 which triggers this issue, at least for GNU ld.so .  The more correct
 assignment would be

     export LD_LIBRARY_PATH=3D"/my/lib{LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

 which is more clumpsy, but doesn't leave you deadly wounded...

   It seems NetBSD's ld.elf_so does do the same:

 197 void
 198 _rtld_add_paths(const char *execname, Search_Path **path_p, const char =
 *pathstr)
 199 {
  :
 205         if (pathstr[0] =3D=3D ':') {
 206                 /*
 207                  * Leading colon means append to current path
 208                  */
 209                 while ((*path_p) !=3D NULL)
 210                         path_p =3D &(*path_p)->sp_next;
 211                 pathstr++;
 212         }

 The man page does not mention this somewhat surprising behavior as
 well:

      LD_LIBRARY_PATH  A colon separated list of directories, overriding the
                       default search path for shared libraries.


 Thanks,
   Jan-Benedict

 --=20

 --cnot5engxscipb6p
 Content-Type: application/pgp-signature; name="signature.asc"

 -----BEGIN PGP SIGNATURE-----

 iF0EABECAB0WIQQlDTvPcScNjKREqWEdvV51g5nhuwUCZAsSsAAKCRAdvV51g5nh
 u58pAJoD23Og8U4xUMHVzHr3LbRphQDBzQCffy8iwZIozFKBZvTTO20CgUmBGeQ=
 =PXqx
 -----END PGP SIGNATURE-----

 --cnot5engxscipb6p--

 --=_yEQegd6KE+J0RR5NTU8CY9iUdk+bjG6u--

State-Changed-From-To: open->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Wed, 15 Mar 2023 11:38:18 +0000
State-Changed-Why:
problem in submitter's build environment


>Unformatted:

Home	PR Database Search