NetBSD Problem Report #58829

From www@netbsd.org  Mon Nov 18 02:32:18 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits)
	 client-signature RSA-PSS (2048 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id B64031A9238
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 18 Nov 2024 02:32:18 +0000 (UTC)
Message-Id: <20241118023217.8E7671A923B@mollari.NetBSD.org>
Date: Mon, 18 Nov 2024 02:32:17 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: libarchive exports excess symbols
X-Send-Pr-Version: www-1.0

>Number:         58829
>Category:       lib
>Synopsis:       libarchive exports excess symbols
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          needs-pullups
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Nov 18 02:35:00 +0000 2024
>Closed-Date:    
>Last-Modified:  Tue Nov 26 14:22:43 +0000 2024
>Originator:     Taylor R Campbell
>Release:        current, 10, 9, ...
>Organization:
The NetBSD Archivexcess
>Environment:
>Description:
libarchive is currently built without either

(a) -fvisibility=hidden and -D__LIBARCHIVE_ENABLE_VISIBILITY, or
(b) a symbol map / version script.

Consequently, it exposes a lot of symbols it shouldn't, like:

PPMD8_kExpEscape
Ppmd8_Alloc
...
__archive_check_child
__archive_check_magic
...
_archive_entry_acl_from_text_l
_archive_entry_acl_text_l
...
blake2s
blake2s_final
...
pack_find
pack_native
>How-To-Repeat:
1. nm
2. link an application against libarchive and some other library like the blake2 reference implementation
>Fix:
Yes, please!

1. Either
   (a) use -fvisibility=hidden -D__LIBARCHIVE_ENABLE_VISIBILITY, or
   (b) create a symbol map,
   or both.
2. Add a .expsym file to verify it so we notice when things change.

Unfortunately, this means removing some symbols from a shared library, like the blake2s ones which clearly don't belong in libarchive's exports.  This is generally prohibited on release branches.  But...I suspect in this case there is likely more harm in exporting the symbols than in ceasing to export them -- as is, there are likely applications that get confused by symbol collisions, manifesting in random confusing crashes; by ceasing to export the symbols we catch those mistakes earlier.

>Release-Note:

>Audit-Trail:
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58829 CVS commit: src/external/bsd/libarchive/lib/libarchive
Date: Thu, 21 Nov 2024 18:17:40 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Thu Nov 21 18:17:40 UTC 2024

 Modified Files:
 	src/external/bsd/libarchive/lib/libarchive: Makefile

 Log Message:
 libarchive: Export only public symbols.

 1. Build with -fvisibility=hidden -D__LIBARCHIVE_ENABLE_VISIBILITY,
    which is upstream's supported way to export only public symbols.

    => Verified that this only suppresses symbols which are _not_
       declared in libarchive's public header files.

 2. Link with -Wl,--exclude-libs,libzstd so that statically linked
    libzstd symbols aren't exported.

    => Verified that this only suppresses libzstd symbols which are not
       part of libarchive at all.

 This deletes exported symbols from a shared library -- but only those
 that should never have been exported to begin with, and that can't be
 used by normal libarchive applications without going out of their way
 to get at internal symbols.  In fact, some of these symbols, like
 blake2s_init, are likely to collide with other libraries or
 applications.

 So while this technically violates the letter of shlib major rules
 and releng pullup rules by deleting exported symbols, I'm not bumping
 and I intend to pull this up to 10 (and maybe 9) -- because I suspect
 it is more likely that libarchive's private symbols will conflict
 with other uses in real applications than that real applications will
 rely on libarchive's private symbols.

 PR lib/58829: libarchive exports excess symbols


 To generate a diff of this commit:
 cvs rdiff -u -r1.19 -r1.20 \
     src/external/bsd/libarchive/lib/libarchive/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58829 CVS commit: src/external/bsd/libarchive/lib/libarchive
Date: Thu, 21 Nov 2024 18:17:54 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Thu Nov 21 18:17:54 UTC 2024

 Added Files:
 	src/external/bsd/libarchive/lib/libarchive: archive.expsym

 Log Message:
 libarchive: Add an expected symbols list.

 Not for pullup unless we pull up all the *.expsym business to
 bsd.lib.mk too.

 PR lib/58829: libarchive exports excess symbols


 To generate a diff of this commit:
 cvs rdiff -u -r0 -r1.1 \
     src/external/bsd/libarchive/lib/libarchive/archive.expsym

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58829 CVS commit: src/tests/lib/libarchive
Date: Thu, 21 Nov 2024 20:14:07 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Thu Nov 21 20:14:07 UTC 2024

 Modified Files:
 	src/tests/lib/libarchive: Makefile

 Log Message:
 tests/lib/libarchive: Link against libarchive statically.

 The tests reach into libarchive guts, so they don't work with only
 the public symbols, which are all that libarchive.so exports now.

 PR lib/58829: libarchive exports excess symbols


 To generate a diff of this commit:
 cvs rdiff -u -r1.15 -r1.16 src/tests/lib/libarchive/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58829 CVS commit: [netbsd-10] src
Date: Tue, 26 Nov 2024 09:44:37 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Tue Nov 26 09:44:36 UTC 2024

 Modified Files:
 	src/external/bsd/libarchive/lib/libarchive [netbsd-10]: Makefile
 	src/tests/lib/libarchive [netbsd-10]: Makefile

 Log Message:
 Pull up following revision(s) (requested by riastradh in ticket #1017):

 	external/bsd/libarchive/lib/libarchive/Makefile: revision 1.20
 	tests/lib/libarchive/Makefile: revision 1.16

 libarchive: Export only public symbols.

 1. Build with -fvisibility=hidden -D__LIBARCHIVE_ENABLE_VISIBILITY,
    which is upstream's supported way to export only public symbols.
    => Verified that this only suppresses symbols which are _not_
       declared in libarchive's public header files.

 2. Link with -Wl,--exclude-libs,libzstd so that statically linked
    libzstd symbols aren't exported.
    => Verified that this only suppresses libzstd symbols which are not
       part of libarchive at all.

 This deletes exported symbols from a shared library -- but only those
 that should never have been exported to begin with, and that can't be
 used by normal libarchive applications without going out of their way
 to get at internal symbols.  In fact, some of these symbols, like
 blake2s_init, are likely to collide with other libraries or
 applications.

 So while this technically violates the letter of shlib major rules
 and releng pullup rules by deleting exported symbols, I'm not bumping
 and I intend to pull this up to 10 (and maybe 9) -- because I suspect
 it is more likely that libarchive's private symbols will conflict
 with other uses in real applications than that real applications will
 rely on libarchive's private symbols.
 PR lib/58829: libarchive exports excess symbols

 tests/lib/libarchive: Link against libarchive statically.
 The tests reach into libarchive guts, so they don't work with only
 the public symbols, which are all that libarchive.so exports now.

 PR lib/58829: libarchive exports excess symbols


 To generate a diff of this commit:
 cvs rdiff -u -r1.14.2.3 -r1.14.2.4 \
     src/external/bsd/libarchive/lib/libarchive/Makefile
 cvs rdiff -u -r1.5.6.3 -r1.5.6.4 src/tests/lib/libarchive/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->needs-pullups
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Tue, 26 Nov 2024 14:22:43 +0000
State-Changed-Why:
pulled up to 10, needs pullup-9 after testing


>Unformatted:

Home	PR Database Search