NetBSD Problem Report #59406

From www@netbsd.org  Thu May  8 12:48:33 2025
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 9119E1A923D
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  8 May 2025 12:48:33 +0000 (UTC)
Message-Id: <20250508124832.8A1891A923E@mollari.NetBSD.org>
Date: Thu,  8 May 2025 12:48:32 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: pbulk: upload should omit restricted packages from pkg_summary
X-Send-Pr-Version: www-1.0

>Number:         59406
>Category:       pkg
>Synopsis:       pbulk: upload should omit restricted packages from pkg_summary
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu May 08 12:50:00 +0000 2025
>Originator:     Taylor R Campbell
>Release:        current
>Organization:
The NO_NETBSD_ON_FOUNDATION
>Environment:
>Description:
pbulk will:

1. build _all_ packages (assuming SKIP_LICENSE_CHECK=yes),
2. record NO_BIN_ON_FTP notes in presolve,
3. generate a pkg_summary for _all_ built packages with pkg_info -X, and
4. upload that pkg_summary, together with the non-restricted ones based on presolve (determined by compute-packages.awk).

The resulting uploaded pkg_summary will have entries for binary packages that weren't uploaded (if there are any NO_BIN_ON_FTP packages).

Later, a cron job messes with the pkg_summary, via the (sometimes mangled) .pkgcache it also uses to generate readmes, so that it doesn't list the restricted packages.  (But sometimes it just lists mangled results, as in PR pkg/59365: binary package repo wrong, https://gnats.netbsd.org/59365)

This is bonkers.  What we upload, and publish at known URLs, should be a consistent snapshot, and should not be mangled in place by a poltergeist provoked by cron.
>How-To-Repeat:
infrastructure inspection
>Fix:
Teach pbulk to limit pkg_summary it uploads to the packages it's uploading too.

Perhaps we should just run compute-packages.awk (via ${packages_script}) in the build script, rather than in the upload script.

Home	PR Database Search