NetBSD Problem Report #54844

From paul@whooppee.com  Wed Jan  8 13:39:24 2020
Return-Path: <paul@whooppee.com>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 9BC4B7A154
	for <gnats-bugs@gnats.NetBSD.org>; Wed,  8 Jan 2020 13:39:24 +0000 (UTC)
Message-Id: <20200108133852.7434A30F2C3@speedy.whooppee.com>
Date: Wed,  8 Jan 2020 05:38:52 -0800 (PST)
From: paul@whooppee.com
Reply-To: paul@whooppee.com
To: gnats-bugs@NetBSD.org
Subject: ``build.sh install-image'' fails with -V USE_PIGZGZIP=yes
X-Send-Pr-Version: 3.95

>Number:         54844
>Category:       install
>Synopsis:       ``build.sh install-image'' fails with -V USE_PIGZGZIP=yes
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    install-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Jan 08 13:40:00 +0000 2020
>Closed-Date:    
>Last-Modified:  Sat Feb 18 17:25:01 +0000 2023
>Originator:     Paul Goyette
>Release:        NetBSD 9.99.32
>Organization:
+--------------------+--------------------------+-----------------------+
| Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
| (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
+--------------------+--------------------------+-----------------------+
>Environment:


System: NetBSD speedy.whooppee.com 9.99.32 NetBSD 9.99.32 (SPEEDY 2020-01-01 13:34:35 UTC) #0: Wed Jan 1 16:22:29 UTC 2020 paul@speedy.whooppee.com:/build/netbsd-local/obj/amd64/sys/arch/amd64/compile/SPEEDY amd64
Architecture: x86_64
Machine: amd64
>Description:
``build.sh install-image'' fails with -V USE_PIGZGZIP=yes
===> build.sh command:    ./build.sh -T /build/netbsd-local/tools/x86_64/amd64 -D /build/netbsd-local/dest/amd64 -O /build/netbsd-local/obj/amd64 -R /build/netbsd-local/release -V RELEASEMACHINEDIR=amd64 -V MKDEBUG=yes -V MKKDEBUG=yes -V USE_PIGZGZIP=yes -U -x -N0 -m amd64 -j10 install-image
===> build.sh started:    Wed Jan  8 01:13:17 UTC 2020
===> NetBSD version:      9.99.34
===> MACHINE:             amd64
===> MACHINE_ARCH:        x86_64
===> Build platform:      NetBSD 9.99.32 amd64
===> HOST_SH:             /bin/sh
===> MAKECONF file:       /etc/mk.conf
===> TOOLDIR path:        /build/netbsd-local/tools/x86_64/amd64
===> DESTDIR path:        /build/netbsd-local/dest/amd64
===> RELEASEDIR path:     /build/netbsd-local/release
===> Updated makewrapper: /build/netbsd-local/tools/x86_64/amd64/bin/nbmake-amd64
install-image ===> etc
install_image ===> /build/netbsd-local/src_ro/distrib	(with: INSTIMG_RELEASEDIR=/build/netbsd-local/release/images)
install_image ===> /build/netbsd-local/src_ro/distrib/notes
install_image ===> /build/netbsd-local/src_ro/distrib/utils
install_image ===> /build/netbsd-local/src_ro/distrib/amd64
install_image ===> /build/netbsd-local/src_ro/distrib/amd64/installimage
creating MBR labels...
1+0 records in
1+0 records out
512 bytes transferred in 0.001 secs (512000 bytes/sec)
Making partition 0 active.
2048+0 records in
2048+0 records out
1048576 bytes transferred in 0.031 secs (33825032 bytes/sec)
Preparing /etc/fstab ...
Extracting kern-GENERIC.tgz ...
Extracting modules.tgz ...
Extracting base.tgz ...
Extracting etc.tgz ...
Copying secondary boot...
Setting rc_configured=YES in /etc/rc.conf ...
Copying extra dirs...
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/.
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/kern-GENERIC_KASLR.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/kern-XEN3_DOM0.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/kern-GENERIC.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/kern-XEN3_DOMU.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/tests.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/xbase.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/rescue.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/games.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/xdebug.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/MD5
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/SHA512
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/comp.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/etc.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/modules.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/misc.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/man.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/xserver.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/debug.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/xfont.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/text.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/xetc.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/base.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/sets/xcomp.tgz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-XEN3_DOMU.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-XEN3_DOM0.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-GENERIC.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-GENERIC_KASLR.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-INSTALL_XEN3_DOMU.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-INSTALL.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/MD5
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/SHA512
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-INSTALL.symbols.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./binary/kernel/netbsd-GENERIC.symbols.gz
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation
./installation/cdrom >> <empty string>
./installation/cdrom/boot.iso >> <empty string>
./installation/cdrom/MD5 >> <empty string>
./installation/cdrom/SHA512 >> <empty string>
./installation/cdrom/boot-com.iso >> <empty string>
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/misc
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/misc/MD5
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/misc/SHA512
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/misc/pxeboot_ia32.bin
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/miniroot
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/miniroot/MD5
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/miniroot/SHA512
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/miniroot/miniroot.kmod
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./installation/miniroot/cgdroot.kmod
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./INSTALL.html
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./INSTALL.ps
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./INSTALL.txt
/build/netbsd-local/obj/amd64/distrib/amd64/installimage/work/amd64/./INSTALL.more
Copying extra files...
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/src_ro/distrib/amd64/installimage/etc.ttys work/etc/ttys
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/src_ro/distrib/amd64/installimage/etc.rc work/etc/rc
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/src_ro/distrib/amd64/installimage/install.sh work/.
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/obj/amd64/distrib/amd64/installimage/boot.cfg work/.
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/obj/amd64/usr.sbin/sysinst/arch/amd64/sysinstmsgs.de work/.
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/obj/amd64/usr.sbin/sysinst/arch/amd64/sysinstmsgs.es work/.
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/obj/amd64/usr.sbin/sysinst/arch/amd64/sysinstmsgs.fr work/.
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/obj/amd64/usr.sbin/sysinst/arch/amd64/sysinstmsgs.pl work/.
/build/netbsd-local/tools/x86_64/amd64/bin/x86_64--netbsd-install -c -m 0644 /build/netbsd-local/obj/amd64/usr.sbin/sysinst/arch/amd64/sysinst work/.
Preparing spec files for makefs...
Creating rootfs...
nbmakefs: `work' size of 1994883072 is larger than the maxsize of 1519386624.
*** Failed target:  imgroot.fs
*** Failed command: /build/netbsd-local/tools/x86_64/amd64/bin/nbmakefs -M 1519386624 -m 1519386624 -B 1234 -F work.spec -N work/etc -o bsize=16384,fsize=2048,density=8192 work.rootfs work
*** Error code 1
Stop.
nbmake[4]: stopped in /build/netbsd-local/src_ro/distrib/amd64/installimage
--- install_image ---
*** [install_image] Error code 1

nbmake[3]: stopped in /build/netbsd-local/src_ro/distrib/amd64
1 error

nbmake[3]: stopped in /build/netbsd-local/src_ro/distrib/amd64

*** Failed target:  install_image-amd64
*** Failed command: _makedirtarget() { dir="$1"; shift; target="$1"; shift; case "${dir}" in /*) this="${dir}/"; real="${dir}" ;; .) this="/build/netbsd-local/src_ro/distrib/"; real="/build/netbsd-local/src_ro/distrib" ;; *) this="/build/netbsd-local/src_ro/distrib/${dir}/"; real="/build/netbsd-local/src_ro/distrib/${dir}" ;; esac; show=${this:-.}; echo "${target} ===> ${show%/}${1:+ (with: $@)}"; cd "${real}" && /build/netbsd-local/tools/x86_64/amd64/bin/nbmake _THISDIR_="${this}" "$@" ${target}; }; _makedirtarget amd64 install_image
*** Error code 2
Stop.
nbmake[2]: stopped in /build/netbsd-local/src_ro/distrib
--- install-image ---
*** [install-image] Error code 1

nbmake[1]: stopped in /build/netbsd-local/src_ro/etc
1 error

nbmake[1]: stopped in /build/netbsd-local/src_ro/etc
--- install-image ---
*** [install-image] Error code 2

nbmake: stopped in /build/netbsd-local/src_ro

nbmake: stopped in /build/netbsd-local/src_ro
>How-To-Repeat:
See above

>Fix:
Unknown


>Release-Note:

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes
Date: Wed, 8 Jan 2020 15:05:37 +0100

 Not sure if we should support non standard options to this level, but
 you could try something like below.

 Martin

 Index: Makefile
 ===================================================================
 RCS file: /cvsroot/src/distrib/amd64/installimage/Makefile,v
 retrieving revision 1.10
 diff -u -r1.10 Makefile
 --- Makefile	15 Dec 2018 18:03:16 -0000	1.10
 +++ Makefile	8 Jan 2020 14:04:06 -0000
 @@ -4,7 +4,11 @@

  INSTIMGBASE=	NetBSD-${DISTRIBVER}-amd64-install	# gives ${IMGBASE}.img

 +.if ${USE_XZ_SETS:Uno} == "yes"
  INSTIMAGEMB?=	1450			# for all installation binaries
 +.else
 +INSTIMAGEMB?=	1950			# for all installation binaries
 +.endif

  PRIMARY_BOOT=		bootxx_ffsv1
  SECONDARY_BOOT=		boot

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes (fwd)
Date: Wed, 8 Jan 2020 06:59:18 -0800 (PST)

 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

 ---------- Forwarded message ----------
 Date: Wed, 8 Jan 2020 06:58:30 -0800 (PST)
 From: Paul Goyette <paul@whooppee.com>
 To: Martin Husemann <martin@duskware.de>
 Subject: Re: install/54844: ``build.sh install-image'' fails with -V
      USE_PIGZGZIP=yes

 I'm not so sure this is just a size issue.  At one point I actually
 looked at the contents of the work directory, and found _another_
 work directory a couple levels down.  The lower-level work directory
 seemed to be the correct size.

 Perhaps there are some conflicting command-line args that work for
 xz but not for pigz?  (Guesing: Perhaps related to sym-links?)


 On Wed, 8 Jan 2020, Martin Husemann wrote:

 > The following reply was made to PR install/54844; it has been noted by GNATS.
 > 
 > From: Martin Husemann <martin@duskware.de>
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 > USE_PIGZGZIP=yes
 > Date: Wed, 8 Jan 2020 15:05:37 +0100
 > 
 > Not sure if we should support non standard options to this level, but
 > you could try something like below.
 > 
 > Martin
 > 
 > Index: Makefile
 > ===================================================================
 > RCS file: /cvsroot/src/distrib/amd64/installimage/Makefile,v
 > retrieving revision 1.10
 > diff -u -r1.10 Makefile
 > --- Makefile	15 Dec 2018 18:03:16 -0000	1.10
 > +++ Makefile	8 Jan 2020 14:04:06 -0000
 > @@ -4,7 +4,11 @@
 >
 >  INSTIMGBASE=	NetBSD-${DISTRIBVER}-amd64-install	# gives ${IMGBASE}.img
 > 
 > +.if ${USE_XZ_SETS:Uno} == "yes"
 >  INSTIMAGEMB?=	1450			# for all installation binaries
 > +.else
 > +INSTIMAGEMB?=	1950			# for all installation binaries
 > +.endif
 >
 >  PRIMARY_BOOT=		bootxx_ffsv1
 >  SECONDARY_BOOT=		boot
 > 
 > 
 > !DSPAM:5e15e2f6268761828113096!
 > 
 >

 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: martin@duskware.de
Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes
Date: Wed, 8 Jan 2020 08:42:50 -0800 (PST)

 On Wed, 8 Jan 2020, Paul Goyette wrote:

 > I'm not so sure this is just a size issue.  At one point I actually
 > looked at the contents of the work directory, and found _another_
 > work directory a couple levels down.  The lower-level work directory
 > seemed to be the correct size.
 >
 > Perhaps there are some conflicting command-line args that work for
 > xz but not for pigz?  (Guesing: Perhaps related to sym-links?)

 Well, I just checked again, and I don't see the "nested" work directory.

 So perhapas setting the size as you suggest is appropriate.


 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@netbsd.org, install-manager@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, paul@whooppee.com
Cc: 
Subject: Re: install/54844: ``build.sh install-image'' fails with -V USE_PIGZGZIP=yes
Date: Wed, 8 Jan 2020 12:37:03 -0500

 On Jan 8,  4:45pm, paul@whooppee.com (Paul Goyette) wrote:
 -- Subject: Re: install/54844: ``build.sh install-image'' fails with -V USE_P

 | The following reply was made to PR install/54844; it has been noted by GNATS.
 | 
 | From: Paul Goyette <paul@whooppee.com>
 | To: gnats-bugs@netbsd.org
 | Cc: martin@duskware.de
 | Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 |  USE_PIGZGZIP=yes
 | Date: Wed, 8 Jan 2020 08:42:50 -0800 (PST)
 | 
 |  On Wed, 8 Jan 2020, Paul Goyette wrote:
 |  
 |  > I'm not so sure this is just a size issue.  At one point I actually
 |  > looked at the contents of the work directory, and found _another_
 |  > work directory a couple levels down.  The lower-level work directory
 |  > seemed to be the correct size.
 |  >
 |  > Perhaps there are some conflicting command-line args that work for
 |  > xz but not for pigz?  (Guesing: Perhaps related to sym-links?)
 |  
 |  Well, I just checked again, and I don't see the "nested" work directory.
 |  
 |  So perhapas setting the size as you suggest is appropriate.

 But why is it that much bigger?

 christos

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: martin@duskware.de
Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes
Date: Wed, 8 Jan 2020 12:07:51 -0800 (PST)

 On Wed, 8 Jan 2020, Paul Goyette wrote:

 > On Wed, 8 Jan 2020, Paul Goyette wrote:
 >
 >> I'm not so sure this is just a size issue.  At one point I actually
 >> looked at the contents of the work directory, and found _another_
 >> work directory a couple levels down.  The lower-level work directory
 >> seemed to be the correct size.
 >> 
 >> Perhaps there are some conflicting command-line args that work for
 >> xz but not for pigz?  (Guesing: Perhaps related to sym-links?)
 >
 > Well, I just checked again, and I don't see the "nested" work directory.
 >
 > So perhaps setting the size as you suggest is appropriate.

 Actually, the patch as-is doesn't work.  It seems that USE_XZ_SETS
 is still defined - for some reason USE_PIGZGZIP didn't unset it (at
 least, not for the install-image target).




 +--------------------+--------------------------+-----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:     |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com     |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org   |
 +--------------------+--------------------------+-----------------------+

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes
Date: Mon, 18 Apr 2022 12:20:35 -0700 (PDT)

 OK, I've been looking a little deper into this.  As christos@ asked
 some time ago

  	But why is it that much bigger?

 The answer seems to be that pigz(1) is not nearly as good as xz(1)
 at compressing NetBSD distribution sets.  Here is a comparison of
 sizes of some distribution sets:

  			pigz	 xz
  			----	----
  	base.tgz	 77M	 44M
  	comp.tgz	160M	 69M
  	debug.tgz	1.2G	688M

 Since the distribution sets themselves are included in the install
 image, the image's filesystem needs to be a lot bigger.

 So, since (1) there's no real bug here, (2) using pigz is not a
 support option, and (3) the problem can be worked around by
 including ``-V INSTIMAGEMB=<size-needed>'' on the build.sh command
 line, I'm going to close this PR.



 +--------------------+--------------------------+----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:    |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com    |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org  |
 | & Network Engineer |                          | pgoyette99@gmail.com |
 +--------------------+--------------------------+----------------------+

State-Changed-From-To: open->closed
State-Changed-By: pgoyette@NetBSD.org
State-Changed-When: Mon, 18 Apr 2022 19:27:55 +0000
State-Changed-Why:
Not a bug, involves unsupported option, and workaround exists.

XXX It would be nice, however, if TOOL_XZ could be built to include
multi-CPU / thread support.  :-)


From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: install-manager@netbsd.org, netbsd-bugs@netbsd.org,
	gnats-admin@netbsd.org, pgoyette@NetBSD.org, paul@whooppee.com
Subject: Re: install/54844 (``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes)
Date: Tue, 19 Apr 2022 06:30:29 +0200

 On Mon, Apr 18, 2022 at 07:27:56PM +0000, pgoyette@NetBSD.org wrote:
 > Not a bug, involves unsupported option, and workaround exists.

 I wouldn't mind making the image size depend on USE_XZ_SETS, the problem
 here is that in many places there is a pair of knobs that need to be
 tuned togethere, one in a Makefile (easy to use USE_XZ_SETS), and one in
 a config(8) file - and for the latter we have no easy conditional.

 If someone would enhance config(8) input language so we could override
 some of the defines in there on the command line, we could solve this
 properly.

 Martin

From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/54844: ``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes
Date: Tue, 19 Apr 2022 14:35:32 -0700 (PDT)

 The following patches provided by tnn@ will enable posxi-thread mode
 when building TOOL_XZ.  The final patch adds the ``-T0'' option when
 invoking TOOL_XZ to build the distribution sets.  Together, these
 changes reduce my time-to-build from 2 hours to 90 minutes, a 25%
 reduction.

 It would be nice if someone with more make-foo than I could get
 this a bit prettier and controllable with a USE_XZ_TOOL_THREADS
 knob (or similar).


 Index: external/public-domain/xz/bin/xz/Makefile
 ===================================================================
 RCS file: /cvsroot/src/external/public-domain/xz/bin/xz/Makefile,v
 retrieving revision 1.6
 diff -p -u -r1.6 Makefile
 --- external/public-domain/xz/bin/xz/Makefile   12 Apr 2021 02:54:08 -0000 1.6
 +++ external/public-domain/xz/bin/xz/Makefile   19 Apr 2022 18:03:58 -0000
 @@ -44,7 +44,7 @@ FILESNAME_${XZSRCDIR}/po/${lang}.gmo= xz
   .if defined(HOSTPROG)
   HOST_CPPFLAGS+=	${CPPFLAGS:N-Wp,-iremap,*}
   XZLIBDIR!=	cd ${NETBSDSRCDIR}/tools/xz-lib && ${PRINTOBJDIR}
 -LDADD+=	-L${XZLIBDIR} -llzma
 +LDADD+=	-L${XZLIBDIR} -llzma -lpthread
   DPADD+=	${XZLIBDIR}/liblzma.a
   .else
   DPADD+=	${LIBLZMA} ${LIBINTL} ${LIBPTHREAD}
 Index: external/public-domain/xz/lib/Makefile
 ===================================================================
 RCS file: /cvsroot/src/external/public-domain/xz/lib/Makefile,v
 retrieving revision 1.10
 diff -p -u -r1.10 Makefile
 --- external/public-domain/xz/lib/Makefile      25 Sep 2018 05:42:08 -0000 1.10
 +++ external/public-domain/xz/lib/Makefile      19 Apr 2022 18:03:58 -0000
 @@ -57,9 +57,7 @@ SRCS+=        common.c block_util.c easy_preset
   	index_decoder.c index_hash.c stream_buffer_decoder.c \
   	stream_decoder.c stream_flags_decoder.c vli_decoder.c

 -.if !defined(HOSTLIB)
   SRCS+=	stream_encoder_mt.c
 -.endif

   .PATH: ${XZSRCDIR}/src/liblzma/delta
   SRCS+= delta_common.c delta_encoder.c delta_decoder.c
 Index: tools/xz-include/Makefile
 ===================================================================
 RCS file: /cvsroot/src/tools/xz-include/Makefile,v
 retrieving revision 1.4
 diff -p -u -r1.4 Makefile
 --- tools/xz-include/Makefile   18 Sep 2021 01:47:11 -0000      1.4
 +++ tools/xz-include/Makefile   19 Apr 2022 18:03:59 -0000
 @@ -8,7 +8,7 @@
   #
   .include "Makefile.inc"

 -CONFIGURE_ARGS+=	--enable-threads=no --disable-nls
 +CONFIGURE_ARGS+=	--enable-threads=posix --disable-nls
   .if ${MAKEVERBOSE} == 0
   CONFIGURE_ARGS+=	--silent
   .endif
 Index: distrib/sets/Makefile
 ===================================================================
 RCS file: /cvsroot/src/distrib/sets/Makefile,v
 retrieving revision 1.107
 diff -u -p -r1.107 Makefile
 --- Makefile    10 Mar 2019 11:04:08 -0000      1.107
 +++ Makefile    19 Apr 2022 21:28:32 -0000
 @@ -18,6 +18,7 @@ all:
   COMPRESS_PROGRAM=${"${USE_XZ_SETS:Uno}"!="no":?${TOOL_XZ}:${TOOL_GZIP}}
   TAR_SUFF=${"${USE_XZ_SETS:Uno}"!="no":?tar.xz:tgz}
   XZ_OPT= 	-9
 +XZ_OPT+= 	-T0
   GZIP_FLAGS=	${GZIP_N_FLAG}

   SETSENV=	DESTDIR=${DESTDIR:Q} \



 +--------------------+--------------------------+----------------------+
 | Paul Goyette       | PGP Key fingerprint:     | E-mail addresses:    |
 | (Retired)          | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com    |
 | Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org  |
 | & Network Engineer |                          | pgoyette99@gmail.com |
 +--------------------+--------------------------+----------------------+

State-Changed-From-To: closed->open
State-Changed-By: pgoyette@NetBSD.org
State-Changed-When: Sat, 18 Feb 2023 17:16:50 +0000
State-Changed-Why:
Per discussion on IRC, this should be looked at as an issue with
USE_XZ_SETS, which (when building install-image target) does not
get undefined when USE_PIGZGZIP is enabled.


From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/54844 (``build.sh install-image'' fails with -V
 USE_PIGZGZIP=yes)
Date: Sat, 18 Feb 2023 18:22:37 +0100

 Am Sat, Feb 18, 2023 at 05:16:51PM +0000 schrieb pgoyette@NetBSD.org:
 > Synopsis: ``build.sh install-image'' fails with -V USE_PIGZGZIP=yes
 > 
 > State-Changed-From-To: closed->open
 > State-Changed-By: pgoyette@NetBSD.org
 > State-Changed-When: Sat, 18 Feb 2023 17:16:50 +0000
 > State-Changed-Why:
 > Per discussion on IRC, this should be looked at as an issue with
 > USE_XZ_SETS, which (when building install-image target) does not
 > get undefined when USE_PIGZGZIP is enabled.

 No, that's not what I said. USE_XZ_SETS should be what decides the
 compression method (xz vs gzip). USE_PIGZGZIP decides what
 implementation of gzip is used for tools purposes. 

 Joerg

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.47 2022/09/11 19:34:41 kim Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2023 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.