NetBSD Problem Report #59195

From www@netbsd.org  Tue Mar 18 00:03:50 2025
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id F40831A9239
	for <gnats-bugs@gnats.NetBSD.org>; Tue, 18 Mar 2025 00:03:49 +0000 (UTC)
Message-Id: <20250318000348.A63311A923C@mollari.NetBSD.org>
Date: Tue, 18 Mar 2025 00:03:48 +0000 (UTC)
From: logix@foobar.franken.de
Reply-To: logix@foobar.franken.de
To: gnats-bugs@NetBSD.org
Subject: Hashes missing in gzimg directories in FTP tree
X-Send-Pr-Version: www-1.0

>Number:         59195
>Category:       install
>Synopsis:       Hashes missing in gzimg directories in FTP tree
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    install-manager
>State:          pending-pullups
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Mar 18 00:05:00 +0000 2025
>Closed-Date:    
>Last-Modified:  Thu Apr 03 02:20:02 +0000 2025
>Originator:     Harold Gutch
>Release:        head, 10, 9, ...
>Organization:
The Nohash Foundation
>Environment:
>Description:
The "gzimg" directories in the FTP tree at https://nycdn.netbsd.org/pub/NetBSD-daily/*/*/*/binary/gzimg/ (for example https://nycdn.netbsd.org/pub/NetBSD-daily/HEAD/latest/evbarm-aarch64/binary/gzimg/ ) don't contain SHA512 and MD5 hash files.
>How-To-Repeat:
ftp -o - https://nycdn.netbsd.org/pub/NetBSD-daily/HEAD/latest/evbarm-aarch64/binary/gzimg/
>Fix:
Unsure, src/etc/Makefile runs ${MAKESUMS} for all sorts of targets/directories, but not for INSTALLATION_DIRS which (among other things) contains binary/gzimg in src/etc/etc.evb{arm,mips,ppc}/Makefile.inc .

>Release-Note:

>Audit-Trail:
From: Harold Gutch <logix@foobar.franken.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/59195: Hashes missing in gzimg directories in FTP tree
Date: Thu, 20 Mar 2025 10:27:09 +0100

 --3hAdtgBjtgL7p0NQ
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline

 Only did some rudimentary testing, but this seems to do the job.

 --3hAdtgBjtgL7p0NQ
 Content-Type: text/x-diff; charset=us-ascii
 Content-Disposition: attachment; filename="gzimg_hashes.patch"

 --- src/distrib/utils/embedded/mkimage.orig	2025-03-17 10:15:42.882887666 +0100
 +++ src/distrib/utils/embedded/mkimage	2025-03-20 10:14:04.467803455 +0100
 @@ -415,4 +415,8 @@
  	image="${image}.gz"
  fi

 +cd "${IMAGEDIR}"
 +${CKSUM} -a MD5 "$(basename "${image}")" > MD5
 +${CKSUM} -a SHA512 "$(basename "${image}")" > SHA512
 +
  echo ${bar} Image is ${image} ${bar}
 --- src/etc/etc.evbarm/Makefile.inc.orig	2025-03-17 10:15:55.623898568 +0100
 +++ src/etc/etc.evbarm/Makefile.inc	2025-03-20 09:39:15.468388551 +0100
 @@ -95,6 +95,8 @@
  	MACHINE=${MACHINE} \
  	MKDTB=${MKDTB} \
  	MKPOSTFIX=${MKPOSTFIX} \
 +	CKSUM=${CKSUM} \
 +	IMAGEDIR=${IMAGE.dir} \
  	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
  	    -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} \
  	    ${IMAGE.dir}/${.TARGET:S/smp_//}.img
 --- src/etc/etc.evbmips/Makefile.inc.orig	2025-03-17 10:16:09.826910721 +0100
 +++ src/etc/etc.evbmips/Makefile.inc	2025-03-20 09:39:31.865405784 +0100
 @@ -70,6 +70,8 @@
  	MACHINE=${MACHINE} \
  	MKDTB=${MKDTB} \
  	MKPOSTFIX=${MKPOSTFIX} \
 +	CKSUM=${CKSUM} \
 +	IMAGEDIR=${IMAGE.dir} \
  	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
              -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} \
              ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
 --- src/etc/etc.evbppc/Makefile.inc.orig	2025-03-17 10:16:24.527923303 +0100
 +++ src/etc/etc.evbppc/Makefile.inc	2025-03-20 09:54:03.747337797 +0100
 @@ -58,6 +58,8 @@
  	MACHINE=${MACHINE} \
  	MKDTB=${MKDTB} \
  	MKPOSTFIX=${MKPOSTFIX} \
 +	CKSUM=${CKSUM} \
 +	IMAGEDIR=${IMAGE.dir} \
  	${HOST_SH} -x ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
              -S ${NETBSDSRCDIR} -B ${IMAGEENDIAN} ${MKI_OPTS.${.TARGET}} \
              ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz
 --- src/etc/etc.riscv/Makefile.inc.orig	2025-03-17 10:16:40.871937301 +0100
 +++ src/etc/etc.riscv/Makefile.inc	2025-03-20 09:39:49.245424050 +0100
 @@ -34,6 +34,8 @@
  	MACHINE=${MACHINE} \
  	MKDTB=${MKDTB} \
  	MKPOSTFIX=${MKPOSTFIX} \
 +	CKSUM=${CKSUM} \
 +	IMAGEDIR=${IMAGE.dir} \
  	${HOST_SH} ${MKIMAGE} -h ${.TARGET:S/smp_//} -D ${DESTDIR} \
              -S ${NETBSDSRCDIR} ${MKI_OPTS.${.TARGET}} \
              ${IMAGE.dir}/${.TARGET:S/smp_//}.img.gz

 --3hAdtgBjtgL7p0NQ--

State-Changed-From-To: open->pending-pullups
State-Changed-By: hgutch@NetBSD.org
State-Changed-When: Sat, 22 Mar 2025 10:36:00 +0000
State-Changed-Why:
Addressed in https://mail-index.netbsd.org/source-changes/2025/03/22/msg156023.html

Waiting for one successful HEAD run before requesting pullups.


From: "David H. Gutteridge" <david@gutteridge.ca>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/59195: Hashes missing in gzimg directories in FTP tree
Date: Wed, 02 Apr 2025 22:16:17 -0400

 I've noticed a couple of issues with this. First, if there's more than
 one file to generate a checksum for, the final one is basically missing
 in the output. That is, all but two characters of the line are lost.
 E.g., from
 https://nycdn.netbsd.org/pub/NetBSD-daily/HEAD/202504020520Z/evbarm-aarch64=
 /binary/gzimg/SHA512
 where there are the files arm64.img.gz and arm64mbr.img.gz we have:

 SHA512 (arm64.img) =3D
 5b1a2ce7ce6a5401f8fc2563953d12351143e24a3f7706f09bb1ef084b2ec50338f1f455
 bbf43dea7c069de92960e7e4d86e1dbb8a8fcad2e67230ac1127fe97
 f3

 (The "f3" is the last two characters of the SHA512 checksum for
 arm64mbr.img.)

 Since the latter file is missing, if one runs cksum on MD5 or SHA512,
 it isn't actually checked at all. (Leading to a false sense of success,
 potentially, if one doesn't use -w with cksum.)

 Separately, the current output kind of violates the POLA, since the
 files themselves are gzipped, but the checksums are generated on the
 raw images. This is confusing, since it's not the practice used for
 similar checksum files provided by TNF, which are calculated against
 the gzipped content.

 Dave

>Unformatted:

Home	PR Database Search