NetBSD Problem Report #58771

From wiz@exadelic.gatalith.at  Tue Oct 22 18:27:33 2024
Return-Path: <wiz@exadelic.gatalith.at>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits)
	 client-signature RSA-PSS (2048 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 2F9B21A923D
	for <gnats-bugs@gnats.NetBSD.org>; Tue, 22 Oct 2024 18:27:33 +0000 (UTC)
Message-Id: <20241022182725.C61FE2EBBAD8@exadelic.gatalith.at>
Date: Tue, 22 Oct 2024 20:27:24 +0200 (CEST)
From: Thomas Klausner <wiz@NetBSD.org>
Reply-To: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Subject: missing dependency breaks parallel build
X-Send-Pr-Version: 3.95

>Number:         58771
>Category:       toolchain
>Synopsis:       missing dependency breaks parallel build
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    toolchain-manager
>State:          needs-pullups
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Oct 22 18:30:00 +0000 2024
>Closed-Date:    
>Last-Modified:  Fri Nov 29 04:11:46 +0000 2024
>Originator:     Thomas Klausner
>Release:        NetBSD 10.99.12
>Organization:

>Environment:


Architecture: x86_64
Machine: amd64
>Description:
Sometimes, parallel builds fail because of:

--- /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h ---
*** Failed target: /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h
*** Failed commands:
        @cmp -s ${.ALLSRC} ${.TARGET} > /dev/null 2>&1 ||  (${_MKSHMSG_INSTALL} ${.TARGET};  ${_MKSHECHO} "${INSTALL_FILE} -o ${BINOWN} -g ${BINGRP}  -m ${NONBINMODE} ${.ALLSRC} ${.TARGET}" &&  ${INSTALL_FILE} -o ${BINOWN} -g ${BINGRP}  -m ${NONBINMODE} ${.ALLSRC} ${.TARGET})
        => @cmp -s krb5_asn1.h /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h > /dev/null 2>&1 ||  (echo '#  ' "install " /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h;  echo "/usr/obj/tools.gcc/bin/x86_64--netbsd-install  -N /usr/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h" &&  /usr/obj/tools.gcc/bin/x86_64--netbsd-install  -N /usr/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h)
*** [/usr/obj/amd64.gcc.20240411/usr/include/krb5/krb5_asn1.h] Error code 1
nbmake[4]: stopped in /usr/src/crypto/external/bsd/heimdal/lib/libasn1

Reading 'make -n includes' in src/crypto/external/bsd/heimdal/lib/libasn1, I see:

/usr/obj/tools/bin/nbasn1_compile                        --one-code-file                 --option-file=/disk/storage-202004/archive/foreign/src/crypto/external/bsd/heimdal/dist/lib/asn1/krb5.opt       /disk/storage-202004/archive/foreign/src/crypto/external/bsd/heimdal/dist/lib/asn1/krb5.asn1 krb5_asn1
/usr/obj/tools/bin/nbsed -E                                              -e 's,#include <(.*)_asn1\.h>,#include <krb5/\1_asn1.h>,'       2> /dev/null < krb5_asn1.hx > krb5_asn1.h
...
cmp -s krb5_asn1.h /usr/include/krb5/krb5_asn1.h > /dev/null 2>&1 ||  (echo '#  ' "install " /usr/include/krb5/krb5_asn1.h;  echo "/usr/obj/tools/bin/x86_64--netbsd-install  -N /disk/storage-202004/archive/foreign/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/include/krb5/krb5_asn1.h" &&  /usr/obj/tools/bin/x86_64--netbsd-install  -N /disk/storage-202004/archive/foreign/src/etc -c  -r -o root -g wheel  -m 444 krb5_asn1.h /usr/include/krb5/krb5_asn1.h)


I tried reading src/crypto/external/bsd/heimdal/Makefile.rules.inc but
I couldn't find the dependency that states that krb5_asn1.h is a file
that is generated by the (sed after the) nbasn1_compile command.

AFAICT, this is the block that's generating the header file:

.if exists(${src:.asn1=.opt})
${src:.asn1=.ts}: ${src} ${src:.asn1=.opt} ${TOOL_ASN1_COMPILE}
        @touch $@
        ${TOOL_ASN1_COMPILE}                    \
                ${ASN1_OPTS.${src}}             \
                --option-file=${.ALLSRC:[2]}    \
                ${.ALLSRC:[1]} ${src:.asn1=_asn1}
        @${TOOL_SED} -E                                                 \
            -e 's,#include <(.*)_asn1\.h>,#include <krb5/\1_asn1.h>,'   \
            2> /dev/null < ${src:.asn1=_asn1.hx} > ${src:.asn1=_asn1.h}
        @cmp -s ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}    \
            2> /dev/null ||                                             \
            cp ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}
.else
${src:.asn1=.ts}: ${src} ${TOOL_ASN1_COMPILE}
        @touch $@
        ${TOOL_ASN1_COMPILE}                    \
                ${ASN1_OPTS.${src}}             \
                ${.ALLSRC:[1]} ${src:.asn1=_asn1}
        @${TOOL_SED} -E                                                 \
            -e 's,#include <(.*)_asn1\.h>,#include <krb5/\1_asn1.h>,'   \
            2> /dev/null < ${src:.asn1=_asn1.hx} > ${src:.asn1=_asn1.h}
        @cmp -s ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}    \
            2> /dev/null ||                                             \
            cp ${src:.asn1=_asn1-priv.hx} ${src:.asn1=_asn1-priv.h}
.endif

and it only has the .ts file on the left hand side of the dependency.

>How-To-Repeat:
Run build.sh -j 16, be unlucky.
>Fix:
not provided.

>Release-Note:

>Audit-Trail:
From: "Luke Mewburn" <lukem@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58771 CVS commit: src/crypto/external/bsd/heimdal
Date: Sat, 23 Nov 2024 07:39:05 +0000

 Module Name:	src
 Committed By:	lukem
 Date:		Sat Nov 23 07:39:05 UTC 2024

 Modified Files:
 	src/crypto/external/bsd/heimdal: Makefile.rules.inc

 Log Message:
 heimdal: fix parallel make includes

 Split out the rules that generate SRC_asn1.h and SRC_asn1-priv.h
 from SRC.asn1 into their own explicit targets that create those files
 from the intermediate SRC.ts, which may help with parallel builds.
 Add MAKEVERBOSE messages to improve debugging.

 PR toolchain/58771: missing dependency breaks parallel build


 To generate a diff of this commit:
 cvs rdiff -u -r1.10 -r1.11 src/crypto/external/bsd/heimdal/Makefile.rules.inc

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Sat, 23 Nov 2024 07:42:58 +0000
State-Changed-Why:
I've committed a fix that may help, and I couldn't get a make -j 20
build to fail with it.
The previous Makefile did have a dependency on SRC.ts by SRC_asn1.h
in a separate rule, but I'm not sure why that didn't work for you.
I just split up the rules so that the creation of the subsidiary
.h and -priv.h files have their own targets now.
I also added MAKEVERBOSE support to make (heh) it more obvious.

Does this help?


State-Changed-From-To: feedback->closed
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Sat, 23 Nov 2024 08:36:12 +0000
State-Changed-Why:
I trust you :) I'll file a new bug report if I see a similar problem ever again
(it only happened once every 5 builds or so)


State-Changed-From-To: closed->needs-pullups
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Fri, 29 Nov 2024 04:11:46 +0000
State-Changed-Why:
Pullups to netbsd-10 and netbsd-9 to be considered if
this build failure is experienced on those branches.


>Unformatted:

Home	PR Database Search