NetBSD Problem Report #51084

From paul@whooppee.com  Thu Apr 21 22:41:26 2016
Return-Path: <paul@whooppee.com>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id B2EEF7A46A
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 21 Apr 2016 22:41:26 +0000 (UTC)
Message-Id: <20160421224053.5449816E5F@pokey.whooppee.com>
Date: Fri, 22 Apr 2016 06:40:53 +0800 (PHT)
From: paul@whooppee.com
Reply-To: paul@whooppee.com
To: gnats-bugs@NetBSD.org
Subject: 'build.sh release' fails with MAKEVERBOSE >= 3
X-Send-Pr-Version: 3.95

>Number:         51084
>Category:       toolchain
>Synopsis:       'build.sh release' fails with MAKEVERBOSE >= 3
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lukem
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Apr 21 22:45:00 +0000 2016
>Closed-Date:    Sat Jun 10 07:17:24 +0000 2023
>Last-Modified:  Sat Jun 10 07:17:24 +0000 2023
>Originator:     Paul Goyette
>Release:        NetBSD 7.99.26
>Organization:
+------------------+--------------------------+------------------------+
| Paul Goyette     | PGP Key fingerprint:     | E-mail addresses:      |
| (Retired)        | FA29 0E3B 35AF E8AE 6651 | paul at whooppee.com   |
| Kernel Developer | 0786 F758 55DE 53BA 7731 | pgoyette at netbsd.org |
+------------------+--------------------------+------------------------+
>Environment:


System: NetBSD pokey.whooppee.com 7.99.26 NetBSD 7.99.26 (POKEY 2016-02-16 03:01:55) #0: Tue Feb 16 12:33:45 PHT 2016 paul@pokey.whooppee.com:/build/netbsd-local/obj/amd64/sys/arch/amd64/compile/POKEY amd64
Architecture: x86_64
Machine: amd64
>Description:
	(This PR relates to the NetBSD build environment.  If "install"
	is an incorrect category, please feel free to update!)

	If you invoke 'build.sh release' with MAKEVERBOSE set to a value
	of 3 or more, the build system sets MAKEFLAGS to "-dl" or "-dx"
	to force make(1) to display the shell commands in each target's
	build script.

	This causes the output of the arch_to_cpu() function in file
	src/distrib/sets/sets.subr to include the 'echo' command, as
	well as its output, and the echo command is then included in
	the value of ${MACHINE_CPU}.  This, in turn, causes the awk
	script in function list_set_files() to fail.

	/build/netbsd-local/tools/x86_64/amd64/bin/nbawk: non-terminated string echo x86_6... at source line 70
	 context is
	                                        wanted["machine_cpu=" "echo x86_64


>How-To-Repeat:
	Just invoke 

		# cd $srcdir
		# ./build.sh -V MAKEVERBOSE=3 release

>Fix:
	While almost certainly not a correct fix, the following patch to
	sets.subr demonstrates the source/cause of the problem:

Index: sets.subr
===================================================================
RCS file: /cvsroot/src/distrib/sets/sets.subr,v
retrieving revision 1.174
diff -u -p -r1.174 sets.subr
--- sets.subr   21 Feb 2016 04:16:36 -0000      1.174
+++ sets.subr   21 Apr 2016 08:21:30 -0000
@@ -648,6 +648,11 @@ print_set_lists()
 #
 arch_to_cpu()
 {
+       # set +x        # unfortunately, if this is needed, it won't help!
+       unset MAKEVERBOSE
+       unset MAKEFLAGS
+       exec 2> /dev/null       # just in case
+
 	MACHINE_ARCH=${1} ${MAKE} -B -f- all <<EOMAKE
 .include <bsd.own.mk>
 all:



>Release-Note:

>Audit-Trail:
From: dieter roelants <dieter.NetBSD@pandora.be>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: install/51084: 'build.sh release' fails with MAKEVERBOSE >= 3
Date: Fri, 22 Apr 2016 18:56:08 +0200

 On Thu, 21 Apr 2016 22:45:00 +0000 (UTC)
 paul@whooppee.com wrote:

 > 	While almost certainly not a correct fix, the following patch to
 > 	sets.subr demonstrates the source/cause of the problem:
 > 
 > Index: sets.subr
 > ===================================================================
 > RCS file: /cvsroot/src/distrib/sets/sets.subr,v
 > retrieving revision 1.174
 > diff -u -p -r1.174 sets.subr
 > --- sets.subr   21 Feb 2016 04:16:36 -0000      1.174
 > +++ sets.subr   21 Apr 2016 08:21:30 -0000
 > @@ -648,6 +648,11 @@ print_set_lists()
 >  #
 >  arch_to_cpu()
 >  {
 > +       # set +x        # unfortunately, if this is needed, it won't help!
 > +       unset MAKEVERBOSE
 > +       unset MAKEFLAGS
 > +       exec 2> /dev/null       # just in case
 > +
 >  	MACHINE_ARCH=${1} ${MAKE} -B -f- all <<EOMAKE
 >  .include <bsd.own.mk>
 >  all:

 Is it incorrect because you unset the vars for the rest of the script
 or because you lose a tiny bit of debug info? If the former, what about:
 	MACHINE_ARCH=${1} ${MAKE} -B -f- all <<EOMAKE | fgrep -v 'echo '
 ? (or `| tail -n 1`, or run the whole function body in a subshell)

 kind regards
 dieter

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: install/51084: 'build.sh release' fails with MAKEVERBOSE >= 3
Date: Sat, 23 Apr 2016 02:18:48 +0700

     Date:        Fri, 22 Apr 2016 17:00:00 +0000 (UTC)
     From:        dieter roelants <dieter.NetBSD@pandora.be>
     Message-ID:  <20160422170000.F0B357AA95@mollari.NetBSD.org>

   |  Is it incorrect because you unset the vars for the rest of the script
   |  or because you lose a tiny bit of debug info?

 I think Paul meant that it is (probably) not the best, because there must
 be a better way...

 There's no problem about unsetting the vars, this function is only ever
 used (and only makes sense to be used) in a subshell, ie: as

 	MACHINE_CPU=$(arch_to_cpu ${MACHINE_ARCH})

 so the changes affect only the shell that runs the function, and not the
 parent.

 I doubt anyone much cares about the debug info from running this script either
 (if one needs to debug the setting of MACHINE_CPU I doubt that the way to
 do it would be to run a full release build with a high MAKEVERBOSE setting
 and then troll through the output...)

 I believe, the "fix" Paul gave works just fine - but it is so crude, and ugly,
 as well as being belt, braces, suspenders, and safety-pins (ie: just
 do everything that might influence the result, without caring which change
 actually makes things correct, or if several of them, individually, would
 be sufficient.)

 The objective of the PR is to get someone who actually understands what is
 going on with the build system to take a look at this, and fix it in an
 elegant way, rather than this crude hack...

 Incidentally, the "install" category of this PR isn't really right,
 but nor is "toolchain", or for that matter, anything else - a whole new
 gnats category might be warranted for build issues)

 kre

State-Changed-From-To: open->feedback
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Thu, 18 May 2023 15:31:51 +0000
State-Changed-Why:

The mechanism to determine MACHINE_CPU (etc) in distrib/sets/sets.subr
was changed to using ${MAKE} .. -V '${MACHINE_CPU}'
by uwe@ on 2018-01-10.

Has this fixed the issue for you?


Responsible-Changed-From-To: install-manager->lukem
Responsible-Changed-By: lukem@NetBSD.org
Responsible-Changed-When: Sat, 20 May 2023 08:38:46 +0000
Responsible-Changed-Why:


State-Changed-From-To: feedback->closed
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Sat, 10 Jun 2023 07:17:24 +0000
State-Changed-Why:
I built a -current release with MAKEVERBOSE=4 without problem.
(Was very verbose; the build output was 60 MiB of text)
This looks fixed to me


>Unformatted:

Home	PR Database Search