NetBSD Problem Report #55400

From www@netbsd.org  Fri Jun 19 14:43:21 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id C87201A9217
	for <gnats-bugs@gnats.NetBSD.org>; Fri, 19 Jun 2020 14:43:21 +0000 (UTC)
Message-Id: <20200619144320.812A71A921F@mollari.NetBSD.org>
Date: Fri, 19 Jun 2020 14:43:20 +0000 (UTC)
From: venture37@geeklan.co.uk
Reply-To: venture37@geeklan.co.uk
To: gnats-bugs@NetBSD.org
Subject: bootstrap broken on FreeBSD following libarchive update
X-Send-Pr-Version: www-1.0

>Number:         55400
>Category:       pkg
>Synopsis:       bootstrap broken on FreeBSD following libarchive update
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    freebsd-pkg-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jun 19 14:45:00 +0000 2020
>Closed-Date:    Tue Oct 13 01:57:11 +0000 2020
>Last-Modified:  Wed Oct 28 16:55:01 +0000 2020
>Originator:     Sevan Janiyan
>Release:        pkgsrc-current
>Organization:
>Environment:
FreeBSD 13.0-CURRENT #0 r362034: Thu Jun 11 04:10:59 BST 2020 amd64
>Description:
Attempting to bootstrap pkgsrc on FreeBSD results in linking errors related to libarchive

cc -L/tmp/work-pbulk/libnbcompat -L../lib -o pkg_create main.o perform.o pl.o util.o build.o -linstall /tmp/work-pbulk/libarc
hive/.libs/libarchive.a  -lnbcompat                                                                                         
ld: error: undefined symbol: _libmd_MD5Init                                                                                 
>>> referenced by archive_digest.c:138 (libarchive/archive_digest.c:138)                                                    
>>>               archive_digest.o:(__archive_md5init) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a             

ld: error: undefined symbol: _libmd_MD5Update                                                                               
>>> referenced by archive_digest.c:146 (libarchive/archive_digest.c:146)                                                    
>>>               archive_digest.o:(__archive_md5update) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a           

ld: error: undefined symbol: _libmd_MD5Final                                                                                
>>> referenced by archive_digest.c:153 (libarchive/archive_digest.c:153)                                                    
>>>               archive_digest.o:(__archive_md5final) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a            

ld: error: undefined symbol: _libmd_RIPEMD160_Init                                                                          
>>> referenced by archive_digest.c:353 (libarchive/archive_digest.c:353)                                                    
>>>               archive_digest.o:(__archive_ripemd160init) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a     

ld: error: undefined symbol: _libmd_RIPEMD160_Final
>>> referenced by archive_digest.c:368 (libarchive/archive_digest.c:368)
>>>               archive_digest.o:(__archive_ripemd160final) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA1_Init
>>> referenced by archive_digest.c:519 (libarchive/archive_digest.c:519)
>>>               archive_digest.o:(__archive_sha1init) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA1_Update
>>> referenced by archive_digest.c:527 (libarchive/archive_digest.c:527)
>>>               archive_digest.o:(__archive_sha1update) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA1_Final
>>> referenced by archive_digest.c:534 (libarchive/archive_digest.c:534)
>>>               archive_digest.o:(__archive_sha1final) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA256_Init
>>> referenced by archive_digest.c:782 (libarchive/archive_digest.c:782)
>>>               archive_digest.o:(__archive_sha256init) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA256_Update
>>> referenced by archive_digest.c:790 (libarchive/archive_digest.c:790)
>>>               archive_digest.o:(__archive_sha256update) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a
ld: error: undefined symbol: _libmd_SHA256_Final
>>> referenced by archive_digest.c:797 (libarchive/archive_digest.c:797)
>>>               archive_digest.o:(__archive_sha256final) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a


ld: error: undefined symbol: _libmd_SHA512_Init
>>> referenced by archive_digest.c:1276 (libarchive/archive_digest.c:1276)
>>>               archive_digest.o:(__archive_sha512init) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA512_Update
>>> referenced by archive_digest.c:1284 (libarchive/archive_digest.c:1284)
>>>               archive_digest.o:(__archive_sha512update) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a

ld: error: undefined symbol: _libmd_SHA512_Final
>>> referenced by archive_digest.c:1291 (libarchive/archive_digest.c:1291)
>>>               archive_digest.o:(__archive_sha512final) in archive /tmp/work-pbulk/libarchive/.libs/libarchive.a
cc: error: linker command failed with exit code 1 (use -v to see invocation)
*** [pkg_create] Error code 1

bmake[1]: stopped in /tmp/work-pbulk/pkg_install/create    
>How-To-Repeat:
attempt to bootstrap on FreeBSD/amd64
cd pkgsrc/bootstrap
./bootstrap
>Fix:

>Release-Note:

>Audit-Trail:
From: Sevan Janiyan <venture37@geeklan.co.uk>
To: "gnats-bugs@netbsd.org" <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: pkg/55400: bootstrap broken on FreeBSD following libarchive
 update
Date: Fri, 19 Jun 2020 16:38:24 +0100

 On 19/06/2020 15:45, venture37@geeklan.co.uk wrote:
 >> Environment:
 > FreeBSD 13.0-CURRENT #0 r362034: Thu Jun 11 04:10:59 BST 2020 amd64

 Confirmed on 12.1-RELEASE as well.

 The linker needs to told to use libmd (-lmd) now?


 Sevan

From: Sevan Janiyan <venture37@geeklan.co.uk>
To: "gnats-bugs@netbsd.org" <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: pkg/55400: bootstrap broken on FreeBSD following libarchive
 update
Date: Fri, 19 Jun 2020 23:34:17 +0100

 On 19/06/2020 16:38, Sevan Janiyan wrote:
 > The linker needs to told to use libmd (-lmd) now?

 bodged it forward by adding -lmd to pkg_create's Makefile.in and
 libarchive's to CFLAGS and LDFLAGS.

 Things still come tumbling down due to configure issues, can't find
 sha2.h and it goes down hill from there (missing definition).


 The API in libarchive has changed also changed and
 pkgtools/pkg_install/files/lib/vulnerabilities-file.c is calling
 functions that do not exist now

 From macos 10.15 (same warnings also on FreeBSD):

 --- vulnerabilities-file.o ---
 cc -isystem /usr/include -DHAVE_CONFIG_H
 -DDEF_LOG_DIR=\"/tmp/pkg/pkgdb\"
 -I/opt/pkgsrc/bootstrap/work/libnbcompat
 -I/opt/pkgsrc/bootstrap/work/libarchive/libarchive -I. -I.
 -DSYSCONFDIR=\"/tmp/pkg/etc\" -DBOOTSTRAP -g -O2 -c vulnerabilities-file.c
 vulnerabilities-file.c:83:22: warning: implicit declaration of function
 'archive_read_new' is invalid in C99 [-Wimplicit-function-declaration]
         struct archive *a = archive_read_new();
                             ^
 vulnerabilities-file.c:83:18: warning: incompatible integer to pointer
 conversion initializing 'struct archive *' with an expression of type
 'int' [-Wint-conversion]
         struct archive *a = archive_read_new();
                         ^   ~~~~~~~~~~~~~~~~~~
 vulnerabilities-file.c:87:2: warning: implicit declaration of function
 'archive_read_support_filter_gzip' is invalid in C99
 [-Wimplicit-function-declaration]
         archive_read_support_filter_gzip(a);
         ^
 vulnerabilities-file.c:88:2: warning: implicit declaration of function
 'archive_read_support_filter_bzip2' is invalid in C99
 [-Wimplicit-function-declaration]
         archive_read_support_filter_bzip2(a);
         ^
 vulnerabilities-file.c:89:2: warning: implicit declaration of function
 'archive_read_support_filter_xz' is invalid in C99
 [-Wimplicit-function-declaration]
         archive_read_support_filter_xz(a);
         ^
 vulnerabilities-file.c:90:2: warning: implicit declaration of function
 'archive_read_support_format_raw' is invalid in C99
 [-Wimplicit-function-declaration]
         archive_read_support_format_raw(a);
         ^

From: Sevan Janiyan <venture37@geeklan.co.uk>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/55400: bootstrap broken on FreeBSD following libarchive
 update
Date: Fri, 19 Jun 2020 23:38:56 +0100

 On 19/06/2020 23:35, Sevan Janiyan wrote:
 > The API in libarchive has changed also changed and
 >  pkgtools/pkg_install/files/lib/vulnerabilities-file.c is calling
 >  functions that do not exist now
 >  
 >  From macos 10.15 (same warnings also on FreeBSD):
 >  
 >  --- vulnerabilities-file.o ---
 >  cc -isystem /usr/include -DHAVE_CONFIG_H
 >  -DDEF_LOG_DIR=\"/tmp/pkg/pkgdb\"
 >  -I/opt/pkgsrc/bootstrap/work/libnbcompat
 >  -I/opt/pkgsrc/bootstrap/work/libarchive/libarchive -I. -I.
 >  -DSYSCONFDIR=\"/tmp/pkg/etc\" -DBOOTSTRAP -g -O2 -c vulnerabilities-file.c
 >  vulnerabilities-file.c:83:22: warning: implicit declaration of function
 >  'archive_read_new' is invalid in C99 [-Wimplicit-function-declaration]
 >          struct archive *a = archive_read_new();
 >                              ^
 >  vulnerabilities-file.c:83:18: warning: incompatible integer to pointer
 >  conversion initializing 'struct archive *' with an expression of type
 >  'int' [-Wint-conversion]
 >          struct archive *a = archive_read_new();
 >                          ^   ~~~~~~~~~~~~~~~~~~
 >  vulnerabilities-file.c:87:2: warning: implicit declaration of function
 >  'archive_read_support_filter_gzip' is invalid in C99
 >  [-Wimplicit-function-declaration]
 >          archive_read_support_filter_gzip(a);
 >          ^
 >  vulnerabilities-file.c:88:2: warning: implicit declaration of function
 >  'archive_read_support_filter_bzip2' is invalid in C99
 >  [-Wimplicit-function-declaration]
 >          archive_read_support_filter_bzip2(a);
 >          ^
 >  vulnerabilities-file.c:89:2: warning: implicit declaration of function
 >  'archive_read_support_filter_xz' is invalid in C99
 >  [-Wimplicit-function-declaration]
 >          archive_read_support_filter_xz(a);
 >          ^
 >  vulnerabilities-file.c:90:2: warning: implicit declaration of function
 >  'archive_read_support_format_raw' is invalid in C99
 >  [-Wimplicit-function-declaration]
 >          archive_read_support_format_raw(a);
 >          ^
 >  

 Actually, this issue is there when building with on libarchive-3.4.0.


 Sevan

Responsible-Changed-From-To: pkg-manager->joerg
Responsible-Changed-By: leot@NetBSD.org
Responsible-Changed-When: Wed, 24 Jun 2020 11:48:05 +0000
Responsible-Changed-Why:
Jörg, can you please give it a look?
(over to MAINTAINER)


Responsible-Changed-From-To: joerg->nia
Responsible-Changed-By: joerg@NetBSD.org
Responsible-Changed-When: Wed, 24 Jun 2020 12:02:43 +0000
Responsible-Changed-Why:
I didn't do an update without talking to the MAINTAINER.


Responsible-Changed-From-To: nia->freebsd-pkg-people
Responsible-Changed-By: nia@NetBSD.org
Responsible-Changed-When: Wed, 24 Jun 2020 14:01:49 +0000
Responsible-Changed-Why:


State-Changed-From-To: open->closed
State-Changed-By: maya@NetBSD.org
State-Changed-When: Tue, 13 Oct 2020 01:57:11 +0000
State-Changed-Why:
I believe this problem was fixed by sjmulder in https://github.com/NetBSD/pkgsrc/commit/83e5e3c468180584f0fd0c1b48d907fb1953e464
Let me know if you are still having problems.


From: "Maya Rashish" <maya@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55400 CVS commit: pkgsrc
Date: Wed, 28 Oct 2020 16:52:43 +0000

 Module Name:	pkgsrc
 Committed By:	maya
 Date:		Wed Oct 28 16:52:43 UTC 2020

 Modified Files:
 	pkgsrc/bootstrap: bootstrap
 	pkgsrc/pkgtools/pkg_install/files/add: Makefile.in
 	pkgsrc/pkgtools/pkg_install/files/create: Makefile.in

 Log Message:
 Bootstrap: Fix bootstrap on FreeBSD 12.1, likely broken by an issue in lld.

 It looks like lld doesn't want to statically link a libarchive without
 resolving all of the symbols, even if only a few symbols are used.

 In order to resolve all of the symbols, we need to also link with -lmd.
 One generic way to do so is inspect Libs.private in the pkgconfig file.

 While pkgsrc is likely not at fault here, having a dysfunctional bootstrap
 is bad. We should check again in the future to see if this can be removed.

 Actually fixes PR pkg/55400.


 To generate a diff of this commit:
 cvs rdiff -u -r1.294 -r1.295 pkgsrc/bootstrap/bootstrap
 cvs rdiff -u -r1.33 -r1.34 pkgsrc/pkgtools/pkg_install/files/add/Makefile.in
 cvs rdiff -u -r1.29 -r1.30 \
     pkgsrc/pkgtools/pkg_install/files/create/Makefile.in

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

Home	PR Database Search