NetBSD Problem Report #53561

From gson@gson.org  Thu Aug 30 09:02:34 2018
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 4405F7A19F
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 30 Aug 2018 09:02:34 +0000 (UTC)
Message-Id: <20180830090226.F10E898A35B@guava.gson.org>
Date: Thu, 30 Aug 2018 12:02:26 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: Build log contains gmake debug output
X-Send-Pr-Version: 3.95

>Number:         53561
>Category:       misc
>Synopsis:       Build log contains gmake debug output
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    misc-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 30 09:05:00 +0000 2018
>Closed-Date:    Fri Nov 15 07:42:07 +0000 2019
>Last-Modified:  Fri Nov 15 07:42:07 +0000 2019
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current, source date >= 2016.04.03.06.13.28
>Organization:

>Environment:
System: NetBSD
Architecture: x86_64
Machine: amd64
>Description:

On August 16, the i386 testbed sent out a build failure notification
which failed to include the compiler error messages, probably because
the heuristic for locating them was confused by several thousand lines
of GNU Make debug output in the log output from build.sh.  Some of it
ended up in the report:

        Still updating file `symtab.o'.
        Considering target file `traditional.o'.
        File `traditional.o' was considered already.
       Finished prerequisites of target file `libcpp.a'.
      The prerequisites of `libcpp.a' are being made.
     Finished prerequisites of target file `all'.
    The prerequisites of `all' are being made.
    Live child 0x7f7ff7b2a480 (init.o) PID 17958 
    Live child 0x7f7ff7b2a660 (symtab.o) PID 18497 
    Live child 0x7f7ff7b2a570 (macro.o) PID 14922 
    Live child 0x7f7ff7b2a4d0 (lex.o) PID 3561 
    Live child 0x7f7ff7b2a3e0 (files.o) PID 13987 
    Live child 0x7f7ff7b2a390 (expr.o) PID 9834 
    Live child 0x7f7ff7b2a2a0 (directives.o) PID 2536 
    Live child 0x7f7ff7b2a200 (charset.o) PID 2228 

IMO, this does not belong in the build log and should be suppressed.

I'm having trouble figuring out exactly where it is is printed, but
bisection shows that it appeared on source date 2016.04.03.06.13.28
with mrg's commit of src/share/mk/bsd.own.mk 1.917, which switched
amd64 and i386 to GCC 5.3.

>How-To-Repeat:

>Fix:

>Release-Note:

>Audit-Trail:
From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@netbsd.org, gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org, gson@gson.org
Cc: 
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 19:58:43 +1100

 martin informs me that this is still happening.  i can't
 reproduce it myself today, but i've seen it occasionally
 in my many builds in the last few years.

 i had a look at the implementation, and after discussion
 with dholland we're wondering if this is a make(1) bug
 in the handling of a particular expression:

    GMAKE_J_ARGS?=  ${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}

 which comes from tools/Makefile.gnuhost not properly
 removing the "-d" that build.sh adds here:

    MAKEFLAGS="-de -m ${TOP}/share/mk ${MAKEFLAGS}"

 the expression *should* only be capable of keeping the
 '-jN' or '-j N' parts in MAKEFLAGS, removing any other
 strings.

 i'm guessing above.  i wonder if something in the i386
 builds is tripping this.. but i have no idea what yet.


 .mrg.

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 10:46:40 +0100

 Have you ever been able to reproduce this on any other host or outside
 of bracket? I wonder if there is something host specific that triggers
 it (like a hidden config file or a bug in libc that causes gmake to
 misbehave or something). It seems I can not reproduce it localy.

 Martin

From: Andreas Gustafsson <gson@gson.org>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 17:58:35 +0200

 Martin Husemann wrote:
 >  Have you ever been able to reproduce this on any other host or outside
 >  of bracket? I wonder if there is something host specific that triggers
 >  it (like a hidden config file or a bug in libc that causes gmake to
 >  misbehave or something). It seems I can not reproduce it localy.

 I looked at four different bracket installations, including one
 hosted on Linux, and it happen on all of them.  For example:

 lyta.netbsd.org /bracket $ grep -c 'Live child 0x' */test/*/build.log
 evbarm-aarch64/test/2019.11.08.14.30.17/build.log:464
 evbarm-aarch64/test/2019.11.10.06.47.30/build.log:395
 evbarm-aarch64/test/2019.11.11.04.04.29/build.log:536
 evbarm-aarch64/test/2019.11.12.05.13.29/build.log:471
 evbarm-aarch64/test/2019.11.13.02.54.59/build.log:359
 evbarm-earmv7hf/test/2019.11.06.23.17.37/build.log:531
 evbarm-earmv7hf/test/2019.11.08.00.35.16/build.log:419
 [etc]

 gumbo.gson.org /bracket $ grep -c 'Live child 0x' */test/*/build.log
 amd64/test/2018.03.16.08.48.34/build.log:452
 hpcmips/test/2019.06.26.12.30.13/build.log:450
 [etc]

 The build log from a build I did manually outside of bracket did not
 contain any 'Live child' messages, even though the logs from builds
 done under bracket on the same machine do, so it looks like the bug
 is somehow triggered by bracket.

 Note that the line identified by mrg,

    GMAKE_J_ARGS?=  ${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}

 is already known to have a bug that is triggered by doing the build in
 a directory called "saint-john" but not one called "saint-mary" (I kid
 you not, see PR 54456), so I would not be surprised if it has other
 equally bizarre bugs triggered by equally incidental circumstances.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Martin Husemann <martin@duskware.de>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 17:11:32 +0100

 On Wed, Nov 13, 2019 at 05:58:35PM +0200, Andreas Gustafsson wrote:
 > Note that the line identified by mrg,
 > 
 >    GMAKE_J_ARGS?=  ${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}
 > 
 > is already known to have a bug that is triggered by doing the build in
 > a directory called "saint-john" but not one called "saint-mary" (I kid
 > you not, see PR 54456), so I would not be surprised if it has other
 > equally bizarre bugs triggered by equally incidental circumstances.

 Yeah, -j triggering the match, but how does the directory get into
 MAKEFLAGS, and what triggers it in the bracket case?

 Can you try modifying the next line to echo some debug data, like:

 GMAKE_J_ARGS?=  ${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}
 BUILD_COMMAND=  echo "MAKEFLAGS: ${MAKEFLAGS}; echo "GMAKE_J_ARGS: ${GMAKE_J_ARGS}; /usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE$

 and grep for that in a bracket log?

 Martin

From: Andreas Gustafsson <gson@gson.org>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org,
    matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 18:25:50 +0200

 Martin Husemann wrote:
 > Yeah, -j triggering the match, but how does the directory get into
 > MAKEFLAGS, and what triggers it in the bracket case?

 PR 54456 shows the contents of MAKEFLAGS, which contains three copies of
 the build directory name, by way of -m, _SRC_TOP_, and _SRC_TOP_OBJ_:

    -d e -m /tmp/bracket/build/2019.08.08.14.00.32-amd64-job-000014/src/share/mk -j 12 -J 15,16 .MAKE.LEVEL.ENV=MAKELEVEL HOST_OSTYPE=NetBSD-8.1-amd64 MKOBJDIRS=yes _SRC_TOP_=/tmp/bracket/build/2019.08.08.14.00.32-amd64-job-000014/src _SRC_TOP_OBJ_=/tmp/bracket/build/2019.08.08.14.00.32-amd64-job-000014/obj _THISDIR_=tools/

 > Can you try modifying the next line to echo some debug data, like:
 > 
 > GMAKE_J_ARGS?=  ${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}
 > BUILD_COMMAND=  echo "MAKEFLAGS: ${MAKEFLAGS}; echo "GMAKE_J_ARGS: ${GMAKE_J_ARGS}; /usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE$
 > 
 > and grep for that in a bracket log?

 I can't parse the final "${GMAKE$", please (re)send me the exact
 BUILD_COMMAND you would like me to use.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Martin Husemann <martin@duskware.de>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 17:37:18 +0100

 On Wed, Nov 13, 2019 at 06:25:50PM +0200, Andreas Gustafsson wrote:
 > Martin Husemann wrote:
 > > Yeah, -j triggering the match, but how does the directory get into
 > > MAKEFLAGS, and what triggers it in the bracket case?
 > 
 > PR 54456 shows the contents of MAKEFLAGS, which contains three copies of
 > the build directory name, by way of -m, _SRC_TOP_, and _SRC_TOP_OBJ_:
 > 
 >    -d e -m /tmp/bracket/build/2019.08.08.14.00.32-amd64-job-000014/src/share/mk -j 12 -J 15,16 .MAKE.LEVEL.ENV=MAKELEVEL HOST_OSTYPE=NetBSD-8.1-amd64 MKOBJDIRS=yes _SRC_TOP_=/tmp/bracket/build/2019.08.08.14.00.32-amd64-job-000014/src _SRC_TOP_OBJ_=/tmp/bracket/build/2019.08.08.14.00.32-amd64-job-000014/obj _THISDIR_=tools/
 > 
 > > Can you try modifying the next line to echo some debug data, like:
 > > 
 > > GMAKE_J_ARGS?=  ${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}
 > > BUILD_COMMAND=  echo "MAKEFLAGS: ${MAKEFLAGS}; echo "GMAKE_J_ARGS: ${GMAKE_J_ARGS}; /usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE$
 > > 
 > > and grep for that in a bracket log?
 > 
 > I can't parse the final "${GMAKE$", please (re)send me the exact
 > BUILD_COMMAND you would like me to use.

 Ooops, copy + pasto...


 Index: Makefile.gnuhost
 ===================================================================
 RCS file: /cvsroot/src/tools/Makefile.gnuhost,v
 retrieving revision 1.51
 diff -u -r1.51 Makefile.gnuhost
 --- Makefile.gnuhost	22 Oct 2018 13:19:42 -0000	1.51
 +++ Makefile.gnuhost	13 Nov 2019 16:30:57 -0000
 @@ -96,7 +96,7 @@
  # sub-gmake's get them, otherwise tools/gcc tries to build libgcc and
  # fails.  it also uses "env -i" to entirely clear out MAKEFLAGS.
  GMAKE_J_ARGS?=	${MAKEFLAGS:[*]:M*-j*:C/.*(-j ?[0-9]*).*/\1/W}
 -BUILD_COMMAND=	/usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE_J_ARGS} -e ${MAKE_ARGS}
 +BUILD_COMMAND=	echo "MAKEFLAGS: ${MAKEFLAGS}; echo "GMAKE_J_ARGS: ${GMAKE_J_ARGS}; /usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE_J_ARGS} -e ${MAKE_ARGS}

  .endif

 Martin

From: Andreas Gustafsson <gson@gson.org>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 19:37:56 +0200

 Earlier today, I wrote:
 > The build log from a build I did manually outside of bracket did not
 > contain any 'Live child' messages, even though the logs from builds
 > done under bracket on the same machine do, so it looks like the bug
 > is somehow triggered by bracket.

 Actually, that manual build was from a "build.sh tools kernel=FOO",
 not a "build.sh release".  I looked at a log from a manual release
 build on the same machine, and it does have the "Live child" messages,
 so apparently bracket is not required after all, but a release build is.

 The build showing the "Live child" messages was done in
 /ssd/prod/8.0/src, and the log begins with:

 ===> build.sh command:    build.sh -j 12 -D /ssd/prod/8.0/amd64/destdir -R /ssd/prod/8.0/amd64/release -T /ssd/prod/8.0/amd64/tools -O /ssd/prod/8.0/amd64/obj -m amd64 -U -x release
 ===> build.sh started:    Wed Sep 12 18:15:50 EEST 2018
 ===> NetBSD version:      8.0
 ===> MACHINE:             amd64
 ===> MACHINE_ARCH:        x86_64
 ===> Build platform:      NetBSD 8.0 amd64
 ===> HOST_SH:             /bin/sh
 ===> No $TOOLDIR/bin/nbmake, needs building.
 ===> Bootstrapping nbmake

 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 21:13:51 +0200

 > -BUILD_COMMAND=	/usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE_J_ARGS} -e ${MAKE_ARGS}
 > +BUILD_COMMAND=	echo "MAKEFLAGS: ${MAKEFLAGS}; echo "GMAKE_J_ARGS: ${GMAKE_J_ARGS}; /usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE_J_ARGS} -e ${MAKE_ARGS}

 The grep output had lines too long to safely email, so it's now at

   http://www.gson.org/netbsd/bugs/53561/makeflags.txt

 The directory also holds the full build log as build.log, but it
 may get deleted at some point as it's rather large.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Martin Husemann <martin@duskware.de>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 20:46:44 +0100

 On Wed, Nov 13, 2019 at 09:13:51PM +0200, Andreas Gustafsson wrote:
 > > -BUILD_COMMAND=	/usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE_J_ARGS} -e ${MAKE_ARGS}
 > > +BUILD_COMMAND=	echo "MAKEFLAGS: ${MAKEFLAGS}; echo "GMAKE_J_ARGS: ${GMAKE_J_ARGS}; /usr/bin/env -i ${BUILD_ENV} ${MAKE_ARGS} ${TOOL_GMAKE} ${GMAKE_J_ARGS} -e ${MAKE_ARGS}
 > 
 > The grep output had lines too long to safely email, so it's now at
 > 
 >   http://www.gson.org/netbsd/bugs/53561/makeflags.txt

 So GMAKE_J_ARGS is just "-j 12" and MAKEFLAGS includes -d (as expected).
 Since we drop (at least that is the idea) MAKEFLAGS before running gmake,
 this is stil ... mysterious.

 Martin

From: Andreas Gustafsson <gson@gson.org>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org,
    matthew green <mrg@eterna.com.au>
Subject: Re: misc/53561: Build log contains gmake debug output
Date: Wed, 13 Nov 2019 22:05:11 +0200

 Martin Husemann wrote:
 > So GMAKE_J_ARGS is just "-j 12" and MAKEFLAGS includes -d (as expected).
 > Since we drop (at least that is the idea) MAKEFLAGS before running gmake,
 > this is stil ... mysterious.

 Are you sure MAKEFLAGS is dropped both when building gcc as a tool and
 as a non-tool?
 -- 
 Andreas Gustafsson, gson@gson.org

From: matthew green <mrg@eterna.com.au>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org, Martin Husemann <martin@duskware.de>
Subject: re: misc/53561: Build log contains gmake debug output
Date: Thu, 14 Nov 2019 19:25:00 +1100

 i think i found it.  i noticed your big build.log shows the error
 only in the native libcpp build phase, not tools at all.  the fix
 is to copy the "env -i" idiom from tools/Makefile.gnuhost.


 .mrg.


 external/gpl3/gcc/usr.bin/host-libcpp/Makefile has:

 libcpp/libcpp.a:
         [ ! -d libcpp ] && mkdir libcpp || true
         (cd libcpp; \
                 CC=3D${HOST_CC:Q} CXX=3D${HOST_CXX:Q}  \
                 CFLAGS=3D${HOST_CFLAGS:Q} \
                 MAKE=3D${TOOL_GMAKE:Q} \
                 CONFIG_SHELL=3D${HOST_SH:Q} \
                 ${HOST_SH} ${DIST}/libcpp/configure -v am_cv_func_iconv=3D=
 no \
                 && CC=3D${HOST_CC:Q} CXX=3D${HOST_CXX:Q} CFLAGS=3D${HOST_C=
 FLAGS:Q} ${TOOL_GMAKE})

From: "matthew green" <mrg@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53561 CVS commit: src/external/gpl3/gcc/usr.bin/host-libcpp
Date: Thu, 14 Nov 2019 09:27:09 +0000

 Module Name:	src
 Committed By:	mrg
 Date:		Thu Nov 14 09:27:09 UTC 2019

 Modified Files:
 	src/external/gpl3/gcc/usr.bin/host-libcpp: Makefile

 Log Message:
 duplicate a some of Makefile.gnuhost.

 should fix PR#53561.


 To generate a diff of this commit:
 cvs rdiff -u -r1.3 -r1.4 src/external/gpl3/gcc/usr.bin/host-libcpp/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: gson@NetBSD.org
State-Changed-When: Fri, 15 Nov 2019 07:42:07 +0000
State-Changed-Why:
Fixed by mrg in src/external/gpl3/gcc/usr.bin/host-libcpp/Makefile 1.4, thanks!


>Unformatted:

Home	PR Database Search