NetBSD Problem Report #45510

From www@NetBSD.org  Sat Oct 22 15:25:39 2011
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id A483E63D58E
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 22 Oct 2011 15:25:39 +0000 (UTC)
Message-Id: <20111022152538.ABE9E63B89A@www.NetBSD.org>
Date: Sat, 22 Oct 2011 15:25:38 +0000 (UTC)
From: matthias.rampke@googlemail.com
Reply-To: matthias.rampke@googlemail.com
To: gnats-bugs@NetBSD.org
Subject: OS X bootstrap broken due to liblzma from /usr/local leaking into libarchive
X-Send-Pr-Version: www-1.0

>Number:         45510
>Category:       pkg
>Synopsis:       OS X bootstrap broken due to liblzma from /usr/local leaking into libarchive
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Oct 22 15:30:00 +0000 2011
>Closed-Date:    Mon Dec 11 01:26:56 +0000 2023
>Last-Modified:  Mon Dec 11 01:26:56 +0000 2023
>Originator:     Matthias Rampke
>Release:        Mac OS X Lion
>Organization:
>Environment:
Darwin mb.local 11.2.0 Darwin Kernel Version 11.2.0: Tue Aug  9 20:54:00 PDT 2011; root:xnu-1699.24.8~1/RELEASE_X86_64 x86_64
>Description:
If xz from Homebrew is installed in /usr/local, bootstrapping with the default options is broken because libarchive finds a "usable" lzma.h, but since Homebrew is 64-bit by default, pkgsrc 32-bit, the corresponding symbols are not there for i386. Therefore linking pkg_create fails due to unresolved symbols in libarchive.a:


gcc -isystem /usr/include -DHAVE_CONFIG_H -D_LARGEFILE_SOURCE -D_LARGE_FILES -D_FILE_OFFSET_BITS=64 -DDEF_UMASK=0022 -I/Users/m
atthias/src/pkgsrc/bootstrap/work/wrk/pkgtools/pkg_install/work/libarchive/libarchive -I/Users/matthias/src/pkgsrc/bootstrap/wo
rk/wrk/pkgtools/pkg_install/work/libfetch -DHAVE_NBCOMPAT_H=1 -I/Users/matthias/src/pkgsrc/bootstrap/work/wrk/pkgtools/pkg_inst
all/work/libnbcompat -no-cpp-precomp -isystem /usr/include -I/usr/include -I. -I. -I../lib -pipe -O2 -I/usr/include -c build.c
gcc -isystem /usr/include -L/Users/matthias/src/pkgsrc/bootstrap/work/wrk/pkgtools/pkg_install/work/libarchive/.libs -Wl,-searc
h_paths_first -L/Users/matthias/src/pkgsrc/bootstrap/work/wrk/pkgtools/pkg_install/work/libfetch -L/Users/matthias/src/pkgsrc/b
ootstrap/work/wrk/pkgtools/pkg_install/work/libnbcompat -L/usr/lib -L/usr/pkg/lib -L../lib -o pkg_create main.o perform.o pl.o 
util.o build.o -linstall -lfetch -larchive -lbz2 -lz -lnbcompat -lcrypto -lcrypto -lcrypto -lcrypto -lnbcompat
Undefined symbols for architecture i386:
  "_lzma_end", referenced from:
      _archive_compressor_xz_finish in libarchive.a(archive_write_set_compression_xz.o)
  "_lzma_lzma_preset", referenced from:
      _archive_compressor_xz_init in libarchive.a(archive_write_set_compression_xz.o)
  "_lzma_stream_encoder", referenced from:
      _archive_compressor_xz_init in libarchive.a(archive_write_set_compression_xz.o)
  "_lzma_alone_encoder", referenced from:
      _archive_compressor_xz_init in libarchive.a(archive_write_set_compression_xz.o)
  "_lzma_code", referenced from:
      _drive_compressor in libarchive.a(archive_write_set_compression_xz.o)
  "_lzma_memusage", referenced from:
      _drive_compressor in libarchive.a(archive_write_set_compression_xz.o)
ld: symbol(s) not found for architecture i386
collect2: ld returned 1 exit status



This would probably also happen on other operating systems, but becomes obvious on OS X especially because of multi-arch and multiple package managers.

>How-To-Repeat:
bootstrap on Mac OS X with 64-bit-only xz (e.g. from Homebrew) in /usr/local


>Fix:
Workaround: brew unlink xz && bootstrap && brew link xz

non-general fix:

diff --git a/pkgtools/pkg_install/Makefile b/pkgtools/pkg_install/Makefile
index 5436787..527373a 100644
--- a/pkgtools/pkg_install/Makefile
+++ b/pkgtools/pkg_install/Makefile
@@ -170,7 +170,7 @@ pre-configure:
                ${CONFIG_SHELL} ${CONFIG_SHELL_FLAGS}                   \
                ./configure --disable-shared --disable-bsdtar           \
                --disable-bsdcpio --without-expat --without-xml2        \
-               --disable-dependency-tracking
+               --disable-dependency-tracking --without-lzma
        cd ${WRKDIR}/libarchive && ${BUILD_MAKE_CMD}
 .endif
        cd ${WRKDIR}/libfetch && ${SETENV} ${MAKE_ENV} ${BSD_MAKE_ENV}  \


but the problem will keep popping up in other places. Not sure how to fix that.

>Release-Note:

>Audit-Trail:
From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from /usr/local
 leaking into libarchive
Date: Sat, 22 Oct 2011 08:44:14 -0700

 On Sat, Oct 22, 2011 at 03:30:00PM +0000, matthias.rampke@googlemail.com wrote:
 > >Description:
 > If xz from Homebrew is installed in /usr/local, bootstrapping with the
 > default options is broken because libarchive finds a "usable" lzma.h,
 > but since Homebrew is 64-bit by default, pkgsrc 32-bit,
 > the corresponding symbols are not there for i386. Therefore linking
 > pkg_create fails due to unresolved symbols in libarchive.a:

 I don't consider this an issue of pkgsrc. It's not our problem if
 incomplete versions of useful dependencies are around in the default GCC
 path. There isn't anything short of disabling XZ support completely,
 which is creating a major regression. No idea if the fat binary support
 can be disabled...

 Joerg

From: Matthias Rampke <matthias.rampke@googlemail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from
 /usr/local leaking into libarchive
Date: Sat, 22 Oct 2011 17:55:38 +0200

 It is pkgsrc that overrides the default machine architecture =E2=80=93 an=
 d then expects every installed library to provide legacy code.

From: Matthias Rampke <matthias.rampke@googlemail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from
 /usr/local leaking into libarchive
Date: Sat, 22 Oct 2011 18:28:28 +0200

 I just checked, bootstrap --abi 64 works with xz linked. So that is probably the best (only) solution in the long run.

 IMHO pkgsrc should be 64 bit by default, at least on 10.7+, now that OS X Lion is 64-bit only.

 Best, matthiasr

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from /usr/local
 leaking into libarchive
Date: Sat, 5 Nov 2011 15:13:48 +0000

 On Sat, Oct 22, 2011 at 03:45:03PM +0000, Joerg Sonnenberger wrote:
  >  I don't consider this an issue of pkgsrc. It's not our problem if
  >  incomplete versions of useful dependencies are around in the default GCC
  >  path. There isn't anything short of disabling XZ support completely,
  >  which is creating a major regression. No idea if the fat binary support
  >  can be disabled...

 Nonsense. The entire point of buildlink and wrappers is to avoid
 running into trouble if random stuff appears in unexpected places.

 -- 
 David A. Holland
 dholland@netbsd.org

From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from /usr/local
 leaking into libarchive
Date: Sat, 5 Nov 2011 16:38:12 +0100

 On Sat, Nov 05, 2011 at 03:15:11PM +0000, David Holland wrote:
 > The following reply was made to PR pkg/45510; it has been noted by GNATS.
 > 
 > From: David Holland <dholland-pbugs@netbsd.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from /usr/local
 >  leaking into libarchive
 > Date: Sat, 5 Nov 2011 15:13:48 +0000
 > 
 >  On Sat, Oct 22, 2011 at 03:45:03PM +0000, Joerg Sonnenberger wrote:
 >   >  I don't consider this an issue of pkgsrc. It's not our problem if
 >   >  incomplete versions of useful dependencies are around in the default GCC
 >   >  path. There isn't anything short of disabling XZ support completely,
 >   >  which is creating a major regression. No idea if the fat binary support
 >   >  can be disabled...
 >  
 >  Nonsense. The entire point of buildlink and wrappers is to avoid
 >  running into trouble if random stuff appears in unexpected places.

 buildlink and wrappers can't work around broken system include paths.

 Joerg

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45510: OS X bootstrap broken due to liblzma from /usr/local
 leaking into libarchive
Date: Sat, 5 Nov 2011 15:57:02 +0000

 On Sat, Nov 05, 2011 at 03:40:04PM +0000, Joerg Sonnenberger wrote:
  >  >  Nonsense. The entire point of buildlink and wrappers is to avoid
  >  >  running into trouble if random stuff appears in unexpected places.
  >  
  >  buildlink and wrappers can't work around broken system include paths.

 That depends; certainly for some platforms it's feasible and arguably
 desirable to add -nostdinc -isystem /usr/include and thereby exclude
 /usr/local.

 In any event it seems like this particular problem could be worked
 around by improving the configure test.

 -- 
 David A. Holland
 dholland@netbsd.org

State-Changed-From-To: open->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Mon, 11 Dec 2023 01:26:56 +0000
State-Changed-Why:
bootstrap has defaulted to ABI=64 on amd64 Macs since 2015 now.


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.47 2022/09/11 19:34:41 kim Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2023 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.