NetBSD Problem Report #57581

From www@netbsd.org  Sun Aug 13 12:01:32 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 50A561A9238
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 13 Aug 2023 12:01:32 +0000 (UTC)
Message-Id: <20230813120130.BC4171A923A@mollari.NetBSD.org>
Date: Sun, 13 Aug 2023 12:01:30 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: set lists should have a way to obsolete shlibs in DESTDIR but not in postinstall
X-Send-Pr-Version: www-1.0

>Number:         57581
>Category:       misc
>Synopsis:       set lists should have a way to obsolete shlibs in DESTDIR but not in postinstall
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Aug 13 12:05:00 +0000 2023
>Last-Modified:  Tue Sep 05 17:30:02 +0000 2023
>Originator:     Taylor R Campbell
>Release:        current
>Organization:
The NetBSD Found in DESTDIR but missing from flist 
>Environment:
>Description:
The set list `obsolete' tag marks a file that used to be in a NetBSD installation and should no longer be.  On incremental build, such files are deleted from DESTDIR.  On postinstall, such files are deleted from the installation.

Public shared libraries are different.  Installations may have programs built in, e.g., pkgsrc that were linked against the old version of a public shared library, found by ld via its libfoo.so symlink and recorded in the executable as a libfoo.so.N soname.  When we bump a shlib major, we want to change the libfoo.so symlink to point to libfoo.so.(N+1) but leave the old libfoo.so.N around so programs built against it continue to work when you upgrade NetBSD.

Currently, the mechanism we use in the set lists is to just delete the entries for libfoo.so.N and libfoo.so.N.M, as documented at the top of base/shl.mi:

https://nxr.netbsd.org/xref/src/distrib/sets/lists/base/shl.mi?r=1.961

But the current mechanism causes incremental builds to fail in checkflist, which requires manual intervention.
>How-To-Repeat:
run an incremental build before and after a shlib major bump, e.g. the openssl3 update
>Fix:
Yes, please!

Perhaps we could create a shlibobsolete tag that causes the file to be deleted from the DESTDIR at build-time, but not added to the obsolete lists -- that way postinstall won't delete the shlibs from installations when you update them.

>Audit-Trail:
From: Valery Ushakov <uwe@stderr.spb.ru>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: misc/57581: set lists should have a way to obsolete shlibs in
 DESTDIR but not in postinstall
Date: Mon, 14 Aug 2023 01:48:57 +0300

 > Currently, the mechanism we use in the set lists is to just delete
 > the entries for libfoo.so.N and libfoo.so.N.M, as documented at the
 > top of base/shl.mi:

 postinstall uses an awk script to obsolete old shlibs.  That script
 has a variable AllLibs that controls whether old majors are deleted.
 Currently that variable cannot be set via postinstall options.
 Perhaps postinstall should be taught a new flag and that flag is used
 when DESTDIR is cleaned as part of the build?

 -uwe

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57581 CVS commit: src/distrib/sets/lists
Date: Mon, 4 Sep 2023 19:07:59 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Mon Sep  4 19:07:59 UTC 2023

 Modified Files:
 	src/distrib/sets/lists/base: shl.mi
 	src/distrib/sets/lists/debug: shl.mi
 	src/distrib/sets/lists/xbase: shl.mi
 	src/distrib/sets/lists/xdebug: shl.mi

 Log Message:
 lists: Remove bogus libfoo.so.N and libfoo.so.N.M obsolete entries.

 These must stay around so applications linked against them will still
 work after upgrade, even if libfoo.so now points to libfoo.so.(N+1)
 or libfoo.so.N.(M+1).

 Exceptions:

 - I'm willing to believe the rump modules have a different story so I
   left those obsolete entries alone.

 - libuv.so was never supposed to be exposed publicly anyway and never
   went out in a release.  (Maybe this information should be recorded
   somewhere?)

 - Same is probably true of lib{gmp,mpc,mpfr}.so, not sure of the
   history.  Maybe libg2c.so too, no idea what that is.

 - libisns.so was moved from /usr/lib to /lib, so it's legitimate for
   the debug data to live there too now.  (XXX Maybe we should have a
   separate marker for this.)

 - Libraries under /usr/tests are not used by normal applications, so
   they can safely be deleted when obsoleted.

 Note: The libfoo.so symlink for a library that has been deleted
 altogether, not just upgraded, can be obsoleted.  Loadable modules
 that applications aren't linked against can be obsoleted, even if
 some of them like npf ext_*.so or pam_*.so are formally versioned
 (for reasons unclear to me).

 Note: This means that incremental builds may complain about these
 .so.N and .so.N.M files in destdir (PR misc/57581), but it's much
 worse for an upgrade to break working applications.


 To generate a diff of this commit:
 cvs rdiff -u -r1.968 -r1.969 src/distrib/sets/lists/base/shl.mi
 cvs rdiff -u -r1.328 -r1.329 src/distrib/sets/lists/debug/shl.mi
 cvs rdiff -u -r1.102 -r1.103 src/distrib/sets/lists/xbase/shl.mi
 cvs rdiff -u -r1.68 -r1.69 src/distrib/sets/lists/xdebug/shl.mi

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57581 CVS commit: [netbsd-10] src/distrib/sets/lists
Date: Tue, 5 Sep 2023 17:26:25 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Tue Sep  5 17:26:24 UTC 2023

 Modified Files:
 	src/distrib/sets/lists/base [netbsd-10]: shl.mi
 	src/distrib/sets/lists/debug [netbsd-10]: shl.mi
 	src/distrib/sets/lists/xbase [netbsd-10]: shl.mi
 	src/distrib/sets/lists/xdebug [netbsd-10]: shl.mi

 Log Message:
 Pull up following revision(s) (requested by riastradh in ticket #348):

 	distrib/sets/lists/xbase/shl.mi: revision 1.103 (patch)
 	distrib/sets/lists/debug/shl.mi: revision 1.329 (patch)
 	distrib/sets/lists/xdebug/shl.mi: revision 1.69 (patch)
 	distrib/sets/lists/base/shl.mi: revision 1.969 (patch)

 lists: Remove bogus libfoo.so.N and libfoo.so.N.M obsolete entries.

 These must stay around so applications linked against them will still
 work after upgrade, even if libfoo.so now points to libfoo.so.(N+1)
 or libfoo.so.N.(M+1).

 Exceptions:
 - I'm willing to believe the rump modules have a different story so I
   left those obsolete entries alone.
 - libuv.so was never supposed to be exposed publicly anyway and never
   went out in a release.  (Maybe this information should be recorded
   somewhere?)
 - Same is probably true of lib{gmp,mpc,mpfr}.so, not sure of the
   history.  Maybe libg2c.so too, no idea what that is.
 - libisns.so was moved from /usr/lib to /lib, so it's legitimate for
   the debug data to live there too now.  (XXX Maybe we should have a
   separate marker for this.)
 - Libraries under /usr/tests are not used by normal applications, so
   they can safely be deleted when obsoleted.

 Note: The libfoo.so symlink for a library that has been deleted
 altogether, not just upgraded, can be obsoleted.  Loadable modules
 that applications aren't linked against can be obsoleted, even if
 some of them like npf ext_*.so or pam_*.so are formally versioned
 (for reasons unclear to me).

 Note: This means that incremental builds may complain about these
 .so.N and .so.N.M files in destdir (PR misc/57581), but it's much
 worse for an upgrade to break working applications.


 To generate a diff of this commit:
 cvs rdiff -u -r1.942.2.8 -r1.942.2.9 src/distrib/sets/lists/base/shl.mi
 cvs rdiff -u -r1.298.2.9 -r1.298.2.10 src/distrib/sets/lists/debug/shl.mi
 cvs rdiff -u -r1.102 -r1.102.2.1 src/distrib/sets/lists/xbase/shl.mi
 cvs rdiff -u -r1.68 -r1.68.2.1 src/distrib/sets/lists/xdebug/shl.mi

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Home	PR Database Search