NetBSD Problem Report #55751

From tsutsui@ceres.dti.ne.jp  Sat Oct 24 18:30:31 2020
Return-Path: <tsutsui@ceres.dti.ne.jp>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 7AE681A9254
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 24 Oct 2020 18:30:31 +0000 (UTC)
Message-Id: <202010241830.09OIUNLA012196@ceres.dti.ne.jp>
Date: Sun, 25 Oct 2020 03:30:23 +0900 (JST)
From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Reply-To: tsutsui@ceres.dti.ne.jp
To: gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: alpha should use gz install sets rather than xz
X-Send-Pr-Version: 3.95

>Number:         55751
>Category:       install
>Synopsis:       alpha should use gz install sets rather than xz
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    install-manager
>State:          closed
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Sat Oct 24 18:35:00 +0000 2020
>Closed-Date:    Thu Mar 04 16:10:11 +0000 2021
>Last-Modified:  Thu Mar 04 16:10:11 +0000 2021
>Originator:     Izumi Tsutsui
>Release:        NetBSD 9.1 and -current
>Organization:
>Environment:
System: NetBSD 9.1 and -current
Architecture: alpha
Machine: alpha
>Description:
NetBSD/alpha has been changed to "USE_XZ_SETS=yes" as other 64 bit ports:
 http://cvsweb.netbsd.org/bsdweb.cgi/src/share/mk/bsd.own.mk#rev1.1077

But at least on my DEC 3000/300 (21064 150MHz) there are several problems:

- gzip(1) gets "out of swap" during extracting base.tar.xz on sysinst.
  My DEC 3000 has 64MB but it is not enough for "xz -9", as noted in
  the xz(1) man page:
>>  For example, decompressing a file created with xz -9 currently requires
>>  65 MiB of memory.

- Even if sysinst is configured to use swap partition, extracting
  base.tar.xz and comp.tar.xz is really slow on my DEC 3000.
  (~100kB/sec; similar speed to extrace gzip sets on 68040 25MHz)

- Less benefits. I guess USE_XZ_SETS was introduce to make
  iso images small enough (to fit in 800MB CD size?), but current
  alpha's iso with USE_XZ_SETS=no is much smaller than that.
  (~300MB, and xz one is ~180MB; not sure this is reasonable though)

>How-To-Repeat:
See above. 

>Fix:
Just revert to set "USE_XZ_SET=no" for alpha?
Or relax the compress level of xz from -9?

---
Izumi Tsutsui

>Release-Note:

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/55751: alpha should use gz install sets rather than xz
Date: Sat, 24 Oct 2020 21:27:23 +0200

 On Sat, Oct 24, 2020 at 06:35:00PM +0000, Izumi Tsutsui wrote:
 > Just revert to set "USE_XZ_SET=no" for alpha?

 Yes, that is the correct fix. It is effectively only needed to cut
 down ISO sizes for amd64 and sparc64 below CD size (but I am not sure
 it really matters for amd64 as all machines likely have a DVD drive
 or none at all).

 Back then I just switched alpha as a 64bit platform too, no deeper
 background - so let's just move it back to .tgz.

 Please feel free to commit the obvious change (and request pullup).

 Martin

From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: martin@duskware.de
Cc: gnats-bugs@netbsd.org, tsutsui@ceres.dti.ne.jp
Subject: Re: install/55751: alpha should use gz install sets rather than xz
Date: Sun, 25 Oct 2020 06:40:45 +0900

 >  It is effectively only needed to cut
 >  down ISO sizes for amd64 and sparc64 below CD size

 I guess the size issue was caused by 32-bit compat binaries
 (especially 32 bit debug sets), not just 64 bit binary sizes.

 Note there are several ports that have >800MB isos:
 NetBSD-9.1-evbarm-earm.iso      1.9G	 
 NetBSD-9.1-evbarm-earmhf.iso    1.0G	 
 NetBSD-9.1-evbarm-earmv7hf.iso  971M	 
 NetBSD-9.1-sun2.iso             805M	 

 Maybe
 - evbarm has too many kernels
 - sun2 has no shlib support (but no sysinst anyway)

 ---
 Izumi Tsutsui

From: Martin Husemann <martin@duskware.de>
To: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Cc: gnats-bugs@netbsd.org
Subject: Re: install/55751: alpha should use gz install sets rather than xz
Date: Sun, 25 Oct 2020 07:42:47 +0100

 On Sun, Oct 25, 2020 at 06:40:45AM +0900, Izumi Tsutsui wrote:
 > >  It is effectively only needed to cut
 > >  down ISO sizes for amd64 and sparc64 below CD size
 > 
 > I guess the size issue was caused by 32-bit compat binaries
 > (especially 32 bit debug sets), not just 64 bit binary sizes.

 Yes, mostly 32bit compat sets, but we do not put debug sets on install
 ISOs anyway (which is a shame).

 > Note there are several ports that have >800MB isos:
 > NetBSD-9.1-evbarm-earm.iso      1.9G	 

 I am not sure how usefull that evbarm ISO is (and for what machine it
 would be used).

 > Maybe
 > - evbarm has too many kernels

 Definitively, but it got better in -current.

 > - sun2 has no shlib support (but no sysinst anyway)

 sysinst is not the problem, getting actualy usefull boot media for whatever
 kind of install is the issue.

 Sun 2 install instructions do not even mention the ISO (and I can not remember
 using it when I did an installation in TME).

 This all came up with a bug report for sparc64 (where many machines
 shipped with SCSI CD rom drives, so no DVD would be usefull for
 booting).

 Martin

From: "Izumi Tsutsui" <tsutsui@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55751 CVS commit: src/share/mk
Date: Sun, 25 Oct 2020 09:06:24 +0000

 Module Name:	src
 Committed By:	tsutsui
 Date:		Sun Oct 25 09:06:24 UTC 2020

 Modified Files:
 	src/share/mk: bsd.own.mk

 Log Message:
 Switch alpha to USE_XZ_SETS=no, i.e. using gzip for installation sets.

 Ok'ed by martin@ in PR/55751.  Maybe worth to pullup to netbsd-9.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1228 -r1.1229 src/share/mk/bsd.own.mk

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55751 CVS commit: [netbsd-9] src/share/mk
Date: Thu, 4 Feb 2021 17:12:11 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Thu Feb  4 17:12:10 UTC 2021

 Modified Files:
 	src/share/mk [netbsd-9]: bsd.own.mk

 Log Message:
 Pull up following revision(s) (requested by tsutsui in ticket #1193):

 	share/mk/bsd.own.mk: revision 1.1229

 Switch alpha to USE_XZ_SETS=no, i.e. using gzip for installation sets.

 Ok'ed by martin@ in PR/55751.  Maybe worth to pullup to netbsd-9.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1149.2.4 -r1.1149.2.5 src/share/mk/bsd.own.mk

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Thu, 04 Mar 2021 16:10:11 +0000
State-Changed-Why:
Committed and pulled up.


>Unformatted:

Home	PR Database Search