NetBSD Problem Report #42704

From kre@munnari.OZ.AU  Sat Jan 30 11:19:01 2010
Return-Path: <kre@munnari.OZ.AU>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 2371263B886
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 30 Jan 2010 11:19:01 +0000 (UTC)
Message-Id: <201001301118.o0UBIpKD002548@jade.coe.psu.ac.th>
Date: Sat, 30 Jan 2010 18:18:51 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@gnats.NetBSD.org
Subject: Adding support for bmake as a pkgsrc tool (ala gmake)
X-Send-Pr-Version: 3.95

>Number:         42704
>Category:       pkg
>Synopsis:       Adding support for bmake as a pkgsrc tool (ala gmake)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    pkg-manager
>State:          closed
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Sat Jan 30 11:20:00 +0000 2010
>Closed-Date:    Mon Dec 29 09:02:49 +0000 2014
>Last-Modified:  Sun May 01 10:10:01 +0000 2016
>Originator:     Robert Elz
>Release:        NetBSD 4.0 / i386   (pkgsrc current 2010-01-30)
>Organization:
	Prince of Songkla University
>Environment:
System: NetBSD jade.coe.psu.ac.th 4.0_STABLE NetBSD 4.0_STABLE (JADE-1.696-20080517) #9: Fri May 23 18:55:13 ICT 2008 kre@jade.coe.psu.ac.th:/usr/obj/4/kernels/JADE i386
Architecture: i386
Machine: i386
>Description:
	There are more and more packages (gnome derived ones mostly)
	that are triggering the bug in NetBSD 4's make program related
	to long input lines (usually found in a .deps/*.po type file)

	Since NetBSD 4 is still supposed to be supported by pkgsrc,
	and since it seems unlikely that there will ever be a new NetBSD 4
	release to incorporate a fixed version of make, all the packages
	that encounter this problem need to use a pkgsrc supplied
	version of make rather than the system version (when building
	in a NetBSD 4 environment).

	Until now, the only alternative has been gmake, and while gmake
	and bmake are largely compatible, "largely" should probably be
	in quotes - it seems to me that if a package builds using the
	standard NetBSD make on NetBSD 5 (or later) it should use as close
	to that as possible on NetBSD 4.

	Since we have devel/bmake already in pkgsrc, I thought I'd see how
	easy it would be to make that available as a build tool for
	pkgsrc (as gmake is, for packages that have gmake extensions in
	their Makefiles).   This turns out to be easy...

	What's more, it seems to work...

>How-To-Repeat:
	I'm not sur ethis section makes much sense in a PR that's a
	change request, but to see an example of a package that has
	the problem today, try time/gtodo-applet - attempt to build
	that on NetBSD 4, and expect to see ...

===> Installing for gtodo-applet-0.1nb18
=> Generating pre-install file lists
Making install in src
[1]   Segmentation fault (core dumped) (cd ${subdir} &&...
*** Error code 1

Stop.
make: stopped in /pkg_comp/obj/pkgsrc/time/gtodo-applet/4x/gtodo-applet-0.1

	the core file dumped is from make, and sure enough, there's a
	BIG .deps/xxx.po file (I forget its name, it isn't important here)

	After the following patch is applied, adding

		USE_TOOLS+= bmake

	to gtodo-applet's Makefile allows that package to build successfully.

	Quite possibly using gmake would as well - since these are mostly
	gnome applications that encounter the bug, it would be surprising if
	gmake wouldn't work ... neevrtheless, the NetBSD 4 make bug could be
	experienced anywhere, so I submit that being able to use bmake as
	a tool is a useful function to add.

>Fix:
	The following patch (applied in pkgsrc/mk) will add support
	for adding bmake as a tool.

Index: tools/defaults.mk
===================================================================
RCS file: /cvsroot/NetBSD/pkgsrc/mk/tools/defaults.mk,v
retrieving revision 1.57
diff -u -r1.57 defaults.mk
--- tools/defaults.mk	8 Jul 2009 12:51:16 -0000	1.57
+++ tools/defaults.mk	30 Jan 2010 10:48:11 -0000
@@ -55,6 +55,7 @@
 _TOOLS_VARNAME.basename=	BASENAME
 _TOOLS_VARNAME.bash=		BASH
 _TOOLS_VARNAME.bison-yacc=	YACC
+_TOOLS_VARNAME.bmake=		BMAKE
 _TOOLS_VARNAME.byacc=		YACC
 _TOOLS_VARNAME.bzcat=		BZCAT
 _TOOLS_VARNAME.bzip2=		BZIP2
Index: tools/make.mk
===================================================================
RCS file: /cvsroot/NetBSD/pkgsrc/mk/tools/make.mk,v
retrieving revision 1.13
diff -u -r1.13 make.mk
--- tools/make.mk	17 Jul 2005 21:36:24 -0000	1.13
+++ tools/make.mk	30 Jan 2010 10:54:12 -0000
@@ -49,6 +49,8 @@
 # Define MAKE_PROGRAM to be the "make" used to build the package.
 .if !empty(_USE_TOOLS:Mgmake)
 MAKE_PROGRAM=		${GMAKE}
+.elif !empty(_USE_TOOLS:Mbmake)
+MAKE_PROGRAM=		${BMAKE}
 .elif !empty(_USE_TOOLS:Mimake) && defined(IMAKE_MAKE)
 MAKE_PROGRAM=		${IMAKE_MAKE}
 .else
Index: tools/replace.mk
===================================================================
RCS file: /cvsroot/NetBSD/pkgsrc/mk/tools/replace.mk,v
retrieving revision 1.223
diff -u -r1.223 replace.mk
--- tools/replace.mk	20 Jan 2010 14:47:30 -0000	1.223
+++ tools/replace.mk	30 Jan 2010 10:46:06 -0000
@@ -258,6 +258,17 @@
 TOOLS_VALUE_GNU.bison-yacc=	${TOOLS_CMDLINE.bison-yacc}
 .endif

+.if !defined(TOOLS_IGNORE.bmake) && !empty(_USE_TOOLS:Mbmake)
+.  if !empty(PKGPATH:Mdevel/bmake)
+MAKEFLAGS+=			TOOLS_IGNORE.bmake=
+.  elif !empty(_TOOLS_USE_PKGSRC.bmake:M[yY][eE][sS])
+TOOLS_DEPENDS.bmake?=		bmake>=20090909:../../devel/bmake
+TOOLS_CREATE+=			bmake
+TOOLS_FIND_PREFIX+=		TOOLS_PREFIX.bmake=bmake
+TOOLS_PATH.bmake=		${TOOLS_PREFIX.bmake}/bin/bmake
+.  endif
+.endif
+
 .if !defined(TOOLS_IGNORE.bsdtar) && !empty(_USE_TOOLS:Mbsdtar)
 .  if !empty(PKGPATH:Marchivers/bsdtar)
 MAKEFLAGS+=			TOOLS_IGNORE.bsdtar=

>Release-Note:

>Audit-Trail:
From: "OBATA Akio" <obache@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/42704: Adding support for bmake as a pkgsrc tool (ala gmake)
Date: Sat, 30 Jan 2010 20:31:47 +0900

 How about install devel/bmake and use "bmake" to build packages instead
 of builtin "make" on NetBSD-4?

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704: Adding support for bmake as a pkgsrc tool (ala gmake)
Date: Sat, 30 Jan 2010 19:08:42 +0700

     Date:        Sat, 30 Jan 2010 11:35:01 +0000 (UTC)
     From:        "OBATA Akio" <obache@NetBSD.org>
     Message-ID:  <20100130113501.D56D863BBFC@www.NetBSD.org>

   |  How about install devel/bmake and use "bmake" to build packages instead
   |  of builtin "make" on NetBSD-4?

 If you mean making bmake be the default make to use for NetBSD 4 systems
 (so it just always gets installed there very early, much like the latest
 version of pkgtools/pkg_install does) then that sounds like an excellent
 idea - it would certainly avoid someone needing to trip over each and every
 package that triggers the bug, then add patches to them all (ie: that would
 make PR 42705 just vanish into the dust).

 The only downside I see is that this would add some overhead to NetBSD 4
 package builders - so far I think I've come across about 8 or 10 packages
 that cause make core dumps (and not all of those are verified to be this
 problem, some are still at the state of "something dumped core - needs
 investigating") of the 8000 or so packages that exist (ie: unless you happen
 to want to build one of the few, the standard system make is adequate.)

 I guess I lean towards "screw overhead, make it correct" personally, but
 I'm not sure everyone does (then again, how many NetBSD 4 package builders
 remain these days?)

 I'd still suggest that having bmake available as a build tool is a good
 idea, aside from dealing with the bug in NetBSD 4's make, this would also
 allow pkgsrc developers to make use of the latest additions to bmake, which
 are currently more or less off limits - as anything recently added to make
 is certainly not going to exist in old, but still supported, systems.
 For this usage though, more support, beyond my pkgsrc abilities, is probably
 needed, so that bmake only gets installed on systems that need the specific'
 feature (like testing what version is installed on the base system, and
 installing devel/bmake only if the base is too old for the feature used by
 a particular package.)

 kre

From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704: Adding support for bmake as a pkgsrc tool (ala
 gmake)
Date: Wed, 3 Feb 2010 00:54:56 +0100

 On Sat, Jan 30, 2010 at 11:20:00AM +0000, kre@munnari.OZ.AU wrote:
 > 	Since NetBSD 4 is still supposed to be supported by pkgsrc,
 > 	and since it seems unlikely that there will ever be a new NetBSD 4
 > 	release to incorporate a fixed version of make, all the packages
 > 	that encounter this problem need to use a pkgsrc supplied
 > 	version of make rather than the system version (when building
 > 	in a NetBSD 4 environment).

 Frankly, I would just mark them as broken with the system make.
 If someone bothers enough, he/she can always bootstrap to get the modern
 make. I explicitly do not want to have another special case for this.

 Joerg

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704: Adding support for bmake as a pkgsrc tool (ala gmake)
Date: Fri, 05 Feb 2010 06:32:08 +0700

     Date:        Tue,  2 Feb 2010 23:55:05 +0000 (UTC)
     From:        Joerg Sonnenberger <joerg@britannica.bec.de>
     Message-ID:  <20100202235505.230D663C462@www.NetBSD.org>

   |  Frankly, I would just mark them as broken with the system make.

 That's certainly a possibility, I wouldn't mind marking half
 of gnome as broken (with the percentage probably growing over time).

   |  If someone bothers enough, he/she can always bootstrap to get the modern
   |  make.

 For me, at least, that's practical iff it is automated as part of pkgsrc.
 I tear down and rebuild the entire sandbox (starting from the release set
 files) for every package I build - extract the release, pkg_add the
 dependencies, then build, make a package, and remove it all again.
 "bootstrap" works only if it is automated (the way a new required set
 of pkgtools is automated, for example.)   If there's some automated way to
 do it (whether or not that means running an additional command), then I
 could deal with it (but I don't want to build version dependent dependencies
 into my build scripts, so I don't want to just always "pkg_add bmake-*" as
 [art of the setup process, as on anything other than NetBSD 4, that's
 either already done, or not required.

   | I explicitly do not want to have another special case for this.

 For just dealing with the NetBSD 4 (and earlier, but that doesn't matter)
 make bug, and the bizarre gnome packages with depend lists beyond
 comprehension, I'd sympathise.

 But I think allowing bmake as an alternative make is more useful than that.

 At the minute, when a few feature is added to make, /usr/src gets to use
 it 5 minutes later.  pkgsrc probably has to wait 5 years (two full
 release cycles, at least) - before then you cannot assume the system make
 in a supported system has the new feature.   That's very hard to do, especially
 for new developers who have no idea that this particular feature is new
 (or within 2 releases new).   That's the primary advantage of allowing
 devel/bmake as a supported tool I think, and given just how small the
 change is, I'd suggest it is one well worth making.

 kre

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704: Adding support for bmake as a pkgsrc tool (ala
	gmake)
Date: Mon, 15 Feb 2010 21:35:14 +0000

 On Thu, Feb 04, 2010 at 11:35:01PM +0000, Robert Elz wrote:
  > At the minute, when a few feature is added to make, /usr/src gets
  > to use it 5 minutes later.  pkgsrc probably has to wait 5 years
  > (two full release cycles, at least) - before then you cannot assume
  > the system make in a supported system has the new feature.  That's
  > very hard to do, especially for new developers who have no idea
  > that this particular feature is new (or within 2 releases new).
  > That's the primary advantage of allowing devel/bmake as a supported
  > tool I think, and given just how small the change is, I'd suggest
  > it is one well worth making.

 I'd agree, FWIW.

 Although maybe USE_TOOLS isn't the right framework, since what we're
 really after is a conditional dependency based on whether the system
 make is new enough for the package.

 Maybe "new enough" can be construed globally though...

 -- 
 David A. Holland
 dholland@netbsd.org

State-Changed-From-To: open->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 29 Dec 2014 09:02:49 +0000
State-Changed-Why:
netbsd-4 is EOL for some time and so this issue is no longer relevant.


From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704 (Adding support for bmake as a pkgsrc tool (ala gmake))
Date: Mon, 09 Mar 2015 02:10:34 +0700

     Date:        Mon, 29 Dec 2014 09:02:49 +0000 (UTC)
     From:        dholland@NetBSD.org
     Message-ID:  <20141229090250.9156BA65D6@mollari.NetBSD.org>

   | netbsd-4 is EOL for some time and so this issue is no longer relevant.

 I disagree with this one - the particular issue that led to the proposed
 change may no longer be relevant, but the underlying problem will always
 be present.

 Exactly the same issue could apply today for NetBSD 5, and even perhaps
 for NetBSD 6 - (b)make keeps being upgraded &/or fixed, a developer with
 a new (and/or fixed) version is naturally going to want to use it fully
 (that's why new features are added, and how bugs are found so they can be
 fixed).

 Outlawing that for pkgsrc packages would be absurd - but since enhancements
 cannot be retroactively added to a released version (at best they can be,
 sometimes, added into the next point release) as long as the older release
 is to be supported by pkgsrc, that would be the effect - unless, as we do
 for everything else (X libraries, etc, gcc, libtool, iconv, even the pkg tools)
 we allow pkgsrc versions to replace the version that comes native with the
 system.   The one glaring exception to this is (b)make.

 Since it is so trivial to allow it, I see no reason not to do so.
 I suspect that the only reason that we don't run across problems with
 this more often, is that very fex pkgsrc packages are actually developed
 on NetBSD.  That's sad - but making it harder cannot help.

 kre

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704 (Adding support for bmake as a pkgsrc tool (ala gmake))
Date: Sun, 1 May 2016 04:36:40 +0000

 On Sun, Mar 08, 2015 at 07:15:00PM +0000, Robert Elz wrote:
  >    | netbsd-4 is EOL for some time and so this issue is no longer relevant.
  >  
  >  I disagree with this one - the particular issue that led to the proposed
  >  change may no longer be relevant, but the underlying problem will always
  >  be present.
  >  
  >  Exactly the same issue could apply today for NetBSD 5, and even perhaps
  >  for NetBSD 6 - (b)make keeps being upgraded &/or fixed, a developer with
  >  a new (and/or fixed) version is naturally going to want to use it fully
  >  (that's why new features are added, and how bugs are found so they can be
  >  fixed).

 Right, which is why the recommended procedure if using an EOL version
 of NetBSD is to bootstrap. This gives you the latest pkgsrc bmake.

 -- 
 David A. Holland
 dholland@netbsd.org

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/42704 (Adding support for bmake as a pkgsrc tool (ala gmake))
Date: Sun, 01 May 2016 17:07:50 +0700

     Date:        Sun,  1 May 2016 04:40:00 +0000 (UTC)
     From:        David Holland <dholland-pbugs@netbsd.org>
     Message-ID:  <20160501044000.DD94B7AA95@mollari.NetBSD.org>

   |  Right, which is why the recommended procedure if using an EOL version
   |  of NetBSD is to bootstrap. This gives you the latest pkgsrc bmake.

 When the PR was submitted, NetBSD 4 was not EOL, and when the comment
 was added, NetBSD 5 wasn't ... and even today, NetBSD 6 is not.

 The issue is when packages use some feature just added to NetBSD's make
 last week - which obviously is not available in any released version.
 For that, they need the very latest version of Make.

 Adding support for using NetBSD's make from pkgsrc(just as there is support
 for using gmake) is trivial to do, and I see no reason not to allow it.

 kre

>Unformatted:

Home	PR Database Search