NetBSD Problem Report #58223

From www@netbsd.org  Sat May  4 08:49:46 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 8B49D1A923B
	for <gnats-bugs@gnats.NetBSD.org>; Sat,  4 May 2024 08:49:46 +0000 (UTC)
Message-Id: <20240504084945.60ED61A923C@mollari.NetBSD.org>
Date: Sat,  4 May 2024 08:49:45 +0000 (UTC)
From: rvp@SDF.ORG
Reply-To: rvp@SDF.ORG
To: gnats-bugs@NetBSD.org
Subject: gzip: multiple issues
X-Send-Pr-Version: www-1.0

>Number:         58223
>Category:       bin
>Synopsis:       gzip: multiple issues
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    christos
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat May 04 08:50:00 +0000 2024
>Closed-Date:    Mon Oct 14 18:23:14 +0000 2024
>Last-Modified:  Mon Oct 14 18:23:14 +0000 2024
>Originator:     RVP
>Release:        NetBSD-10.99.10
>Organization:
>Environment:
NetBSD-10.99.10 amd64
>Description:
gzip in the current -HEAD has multiple issues:

1. With .xz and .lz files, the -t flag decompresses the file to stdout
   instead of just testing the file. 

2. It fails to decode certain .lz files compressed at some levels:
   lzip -[24789] games.tar
>How-To-Repeat:
Get any of the recent games.tar.xz tarball of -HEAD.

For 1)
$ gzip -t games.tar.{xz,lz}

For 2)
$ for l in {0..9}; do lzip -${l}c games.tar > games.tar.$l.lz; done
$ for f in games.tar.*.lz; do gzip -t $f >/dev/null; done
gzip: games.tar.2.lz: uncompress failed
gzip: games.tar.4.lz: uncompress failed
gzip: games.tar.7.lz: uncompress failed
gzip: games.tar.8.lz: uncompress failed
gzip: games.tar.9.lz: uncompress failed
$ 

>Fix:
diff -urN gzip.orig/unlz.c gzip/unlz.c
--- gzip.orig/unlz.c	2023-06-10 04:45:25.000000000 +0000
+++ gzip/unlz.c	2024-05-04 08:00:32.963583086 +0000
@@ -295,7 +295,7 @@

 	size_t size = (size_t)offs;
 	lz_crc_update(&lz->crc, lz->obuf + lz->spos, size);
-	if (fwrite(lz->obuf + lz->spos, 1, size, lz->fout) != size)
+	if (!tflag && fwrite(lz->obuf + lz->spos, 1, size, lz->fout) != size)
 		return -1;

 	lz->wrapped = lz->pos >= lz->dict_size;
@@ -602,7 +602,7 @@
 lz_get_dict_size(unsigned char c)
 {
 	unsigned dict_size = 1 << (c & 0x1f);
-	dict_size -= (dict_size >> 2) * ( (c >> 5) & 0x7);
+	dict_size -= (dict_size >> 4) * ( (c >> 5) & 0x7);
 	if (dict_size < MIN_DICTIONARY_SIZE || dict_size > MAX_DICTIONARY_SIZE)
 		return 0;
 	return dict_size;
diff -urN gzip.orig/unxz.c gzip/unxz.c
--- gzip.orig/unxz.c	2018-10-06 16:36:45.000000000 +0000
+++ gzip/unxz.c	2024-05-04 07:40:02.518116365 +0000
@@ -99,7 +99,7 @@
 		if (strm.avail_out == 0 || ret != LZMA_OK) {
 			const size_t write_size = sizeof(obuf) - strm.avail_out;

-			if (write(o, obuf, write_size) != (ssize_t)write_size)
+			if (!tflag && write(o, obuf, write_size) != (ssize_t)write_size)
 				maybe_err("write failed");

 			strm.next_out = obuf;

>Release-Note:

>Audit-Trail:
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58223 CVS commit: src/usr.bin/gzip
Date: Sat, 4 May 2024 09:17:04 -0400

 Module Name:	src
 Committed By:	christos
 Date:		Sat May  4 13:17:03 UTC 2024

 Modified Files:
 	src/usr.bin/gzip: unlz.c unxz.c

 Log Message:
 PR/58223: RVP: Don't write to stdout when testing.


 To generate a diff of this commit:
 cvs rdiff -u -r1.8 -r1.9 src/usr.bin/gzip/unlz.c src/usr.bin/gzip/unxz.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Taylor R Campbell <riastradh@NetBSD.org>
To: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Cc: rvp@SDF.ORG
Subject: Re: bin/58223: gzip: multiple issues
Date: Mon, 1 Jul 2024 01:14:10 +0000

 Misaddressed commit should have been appended to this PR:

 https://mail-index.netbsd.org/source-changes/2024/05/04/msg151128.html

 Module Name:    src
 Committed By:   christos
 Date:           Sat May  4 13:18:06 UTC 2024

 Modified Files:
         src/usr.bin/gzip: unlz.c

 Log Message:
 PR/58233: RVP: Fix lzip dictionary size calculation


 To generate a diff of this commit:
 cvs rdiff -u -r1.9 -r1.10 src/usr.bin/gzip/unlz.c

State-Changed-From-To: open->needs-pullups
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Tue, 23 Jul 2024 21:19:06 +0000
State-Changed-Why:
fixed in HEAD, probably needs pullup-10 and pullup-9


Responsible-Changed-From-To: bin-bug-people->christos
Responsible-Changed-By: riastradh@NetBSD.org
Responsible-Changed-When: Fri, 11 Oct 2024 15:01:04 +0000
Responsible-Changed-Why:
christos: Since you committed the fix, can I trouble you to add a test
and submit pullup requests for the test and fix?


From: Christos Zoulas <christos@zoulas.com>
To: gnats-bugs@netbsd.org
Cc: christos@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org,
 "riastradh@netbsd.org" <riastradh@NetBSD.org>,
 "rvp@sdf.org" <rvp@SDF.ORG>
Subject: Re: bin/58223 (gzip: multiple issues)
Date: Fri, 11 Oct 2024 13:00:12 -0400

 --Apple-Mail=_58C93EF2-9977-46AB-AAF3-77F8CF70B673
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/plain;
 	charset=utf-8

 Sure, I've put it on my list of things to do.

 christos

 > On Oct 11, 2024, at 11:01=E2=80=AFAM, riastradh@netbsd.org =
 <riastradh@NetBSD.org> wrote:
 >=20
 > Synopsis: gzip: multiple issues
 >=20
 > Responsible-Changed-From-To: bin-bug-people->christos
 > Responsible-Changed-By: riastradh@NetBSD.org
 > Responsible-Changed-When: Fri, 11 Oct 2024 15:01:04 +0000
 > Responsible-Changed-Why:
 > christos: Since you committed the fix, can I trouble you to add a test
 > and submit pullup requests for the test and fix?
 >=20
 >=20


 --Apple-Mail=_58C93EF2-9977-46AB-AAF3-77F8CF70B673
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCZwlZnAAKCRBxESqxbLM7
 OqkOAJ9dMcX7BaTKN/BJudx6o9u7giE/8ACfTcgH/oOKYIpAwgLyteBxMk5teeg=
 =xDvF
 -----END PGP SIGNATURE-----

 --Apple-Mail=_58C93EF2-9977-46AB-AAF3-77F8CF70B673--

From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58223 CVS commit: src/tests/usr.bin/gzip
Date: Mon, 14 Oct 2024 13:51:39 -0400

 Module Name:	src
 Committed By:	christos
 Date:		Mon Oct 14 17:51:39 UTC 2024

 Modified Files:
 	src/tests/usr.bin/gzip: t_gzip.sh

 Log Message:
 Add a test for PR/58223


 To generate a diff of this commit:
 cvs rdiff -u -r1.1 -r1.2 src/tests/usr.bin/gzip/t_gzip.sh

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Christos Zoulas <christos@zoulas.com>
To: gnats-bugs@netbsd.org
Cc: christos@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org,
 "rvp@sdf.org" <rvp@SDF.ORG>
Subject: Re: bin/58223 (gzip: multiple issues)
Date: Mon, 14 Oct 2024 13:57:17 -0400

 --Apple-Mail=_7EA0C45F-CF0F-4980-AE43-C39DEC9E7F86
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain;
 	charset=us-ascii

 Done!

 christos


 --Apple-Mail=_7EA0C45F-CF0F-4980-AE43-C39DEC9E7F86
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCZw1bfQAKCRBxESqxbLM7
 OtQ7AKDzuyImp+3tvaeTx9mUJjOPjZoZaQCeJi4OUhFlaWji28U83LtNcOJ4Mzg=
 =XnpJ
 -----END PGP SIGNATURE-----

 --Apple-Mail=_7EA0C45F-CF0F-4980-AE43-C39DEC9E7F86--

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58223 CVS commit: [netbsd-10] src
Date: Mon, 14 Oct 2024 18:11:36 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Mon Oct 14 18:11:36 UTC 2024

 Modified Files:
 	src/tests/usr.bin/gzip [netbsd-10]: t_gzip.sh
 	src/usr.bin/gzip [netbsd-10]: unlz.c unxz.c

 Log Message:
 Pull up following revision(s) (requested by christos in ticket #979):

 	usr.bin/gzip/unlz.c: revision 1.10
 	usr.bin/gzip/unlz.c: revision 1.9
 	usr.bin/gzip/unxz.c: revision 1.9
 	tests/usr.bin/gzip/t_gzip.sh: revision 1.2

 PR/58223: RVP: Don't write to stdout when testing.

 PR/58233: RVP: Fix lzip dictionary size calculation

 Add a test for PR/58223


 To generate a diff of this commit:
 cvs rdiff -u -r1.1 -r1.1.44.1 src/tests/usr.bin/gzip/t_gzip.sh
 cvs rdiff -u -r1.7.2.1 -r1.7.2.2 src/usr.bin/gzip/unlz.c
 cvs rdiff -u -r1.8 -r1.8.10.1 src/usr.bin/gzip/unxz.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58223 CVS commit: [netbsd-9] src
Date: Mon, 14 Oct 2024 18:13:37 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Mon Oct 14 18:13:36 UTC 2024

 Modified Files:
 	src/tests/usr.bin/gzip [netbsd-9]: t_gzip.sh
 	src/usr.bin/gzip [netbsd-9]: unlz.c unxz.c

 Log Message:
 Pull up following revision(s) (requested by christos in ticket #1913):

 	usr.bin/gzip/unlz.c: revision 1.10
 	usr.bin/gzip/unlz.c: revision 1.9
 	usr.bin/gzip/unxz.c: revision 1.9
 	tests/usr.bin/gzip/t_gzip.sh: revision 1.2

 PR/58223: RVP: Don't write to stdout when testing.

 PR/58233: RVP: Fix lzip dictionary size calculation

 Add a test for PR/58223


 To generate a diff of this commit:
 cvs rdiff -u -r1.1 -r1.1.36.1 src/tests/usr.bin/gzip/t_gzip.sh
 cvs rdiff -u -r1.6 -r1.6.6.1 src/usr.bin/gzip/unlz.c
 cvs rdiff -u -r1.8 -r1.8.2.1 src/usr.bin/gzip/unxz.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: needs-pullups->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Mon, 14 Oct 2024 18:23:14 +0000
State-Changed-Why:
fixed in HEAD, pulled up to 9 and 10
pullup-10 #979 https://releng.netbsd.org/cgi-bin/req-10.cgi?show=979
pullup-9 #1913 https://releng.netbsd.org/cgi-bin/req-9.cgi?show=1913


>Unformatted:

Home	PR Database Search