NetBSD Problem Report #57431

From lukem@thistledown.com.au  Wed May 24 06:55:07 2023
Return-Path: <lukem@thistledown.com.au>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id EC3C11A9238
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 24 May 2023 06:55:06 +0000 (UTC)
Message-Id: <20230524065500.7E06C6A76@thoreau.thistledown.com.au>
Date: Wed, 24 May 2023 16:55:00 +1000 (AEST)
From: lukem@netbsd.org
Reply-To: lukem@netbsd.org
To: gnats-bugs@NetBSD.org
Subject: tool m4 can't regen tools/compat/configure
X-Send-Pr-Version: 3.95

>Number:         57431
>Category:       toolchain
>Synopsis:       tool m4 can't regen tools/compat/configure
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lukem
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed May 24 07:00:01 +0000 2023
>Closed-Date:    Sun May 28 15:41:31 +0000 2023
>Last-Modified:  Sun May 28 15:41:31 +0000 2023
>Originator:     Luke Mewburn
>Release:        NetBSD 9.99.101
>Organization:
NetBSD
>Environment:
System: NetBSD thoreau 9.99.101 NetBSD 9.99.101 (THOREAU.git) #58: Sun Oct 23 22:10:19 AEDT 2022 simonb@thoreau:THOREAU amd64
Architecture: x86_64
Machine: amd64

>Description:
It's not possible to run "$TOOLDIR/bin/make -C tools/compat regen"
with $TOOLDIR/bin/nbm4 as the m4 implementation for tools/autoconf;
nbm4 complains about a regular expression used within autoconf.
GNU M4 works fine.

This makes it tricky to regen tools/compat after modifying
tools/compat/configure.ac for other issues.

The tools/compat regen (before mine with this workaround) was on 2021-02-25.
Possibly relevant changes:
- usr.bin/m4:
	I couldn't see any obvious changes in usr.bin/m4
	(besides documentation) after 2020-06-27.
- lib/libc/regex on netbsd-9 (for the host $TOOLDIR/bin/nbm4)
	Last commit on netbsd-9 was 2019-02-07

>How-To-Repeat:
lukem@thoreau 380> /home/lukem/obj/amd64/tools/bin/nbmake-amd64 MAKEVERBOSE=2 -C tools/compat regen
#     regen  configure
cd /home/lukem/src/netbsd/src/tools/compat && /home/lukem/obj/amd64/tools/bin/nbautoconf
nbm4: configure.ac at line 22: regular expression error: repetition-operator operand invalid for: `\\|`|\$\(|\${|@'
nbautom4te: /home/lukem/obj/amd64/tools/bin/nbm4 failed with exit status: 1

*** Failed target:  regen
*** Failed command: cd /home/lukem/src/netbsd/src/tools/compat && /home/lukem/obj/amd64/tools/bin/nbautoconf

>Fix:
If I modify tools/autoconf/Makefile to use GNU m4 as gm4 (from pkgsrc):
	#CONFIGURE_ENV+=        M4=${TOOL_M4:Q}" -g -D__gnu__"
	CONFIGURE_ENV+= M4=gm4
and install that version of autoconf, then I can do the regen.

I may add a (temporary) conditional override to tools/autoconf/Makefile
to allow similar overrides without requiring editing that Makefile.

>Release-Note:

>Audit-Trail:
From: Luke Mewburn <luke@mewburn.net>
To: gnats-bugs@netbsd.org
Cc: toolchain-manager@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: toolchain/57431: tool m4 can't regen tools/compat/configure
Date: Thu, 25 May 2023 07:05:43 +1000

 On 23-05-24 07:00, Luke Mewburn wrote:
   | >Number:         57431
   | >Category:       toolchain
   | >Synopsis:       tool m4 can't regen tools/compat/configure
   |
   | It's not possible to run "$TOOLDIR/bin/make -C tools/compat regen"
   | with $TOOLDIR/bin/nbm4 as the m4 implementation for tools/autoconf;
   | nbm4 complains about a regular expression used within autoconf.
   | GNU M4 works fine.
   | 
   | This makes it tricky to regen tools/compat after modifying
   | tools/compat/configure.ac for other issues.

 I analysed this further - see discussion on source-changed-d:
 - https://mail-index.netbsd.org/source-changes-d/2023/05/24/msg013956.html
 - https://mail-index.netbsd.org/source-changes-d/2023/05/24/msg013957.html


 In short, we don't need Christos' rev 1.2 change to
   external/gpl3/autoconf/dist/lib/autoconf/general.m4
 and should revert that.

 What we need to do is change unescaped GNU m4 regexp and patsubst
 '{' and '}' to '\{' and '\}', because GNU m4 regexp is imp

 GNU m4 regexp is implemented using gnulib Emacs-mode regexp
 which are documented in
   https://www.gnu.org/software/gnulib/manual/html_node/emacs-regular-expression-syntax.html

 Specifically, { and } have no special meaning for GNU m4;
 these are not interval characters, and \{ and \} are not
 the equivalent (unlike POSIX BREs).


 I have experimented with converting our m4's -g mode (GNU emulation)
 to escape { and } in regexes (in the twiddle()) function and
 this seems to work. Further testing is required.

Responsible-Changed-From-To: toolchain-manager->lukem
Responsible-Changed-By: lukem@NetBSD.org
Responsible-Changed-When: Wed, 24 May 2023 21:50:13 +0000
Responsible-Changed-Why:


State-Changed-From-To: open->analyzed
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Wed, 24 May 2023 21:50:13 +0000
State-Changed-Why:


From: "Luke Mewburn" <lukem@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57431 CVS commit: src/usr.bin/m4
Date: Wed, 24 May 2023 21:58:19 +0000

 Module Name:	src
 Committed By:	lukem
 Date:		Wed May 24 21:58:19 UTC 2023

 Modified Files:
 	src/usr.bin/m4: gnum4.c

 Log Message:
 m4 -g: { and } aren't supported in patsubst() or regexp()

 When running in -g (GNU m4) emulation, patsubst() and regexp()
 use the GNU m4 emacs-like regexes as implemented by gnulib,
 which don't support {..} intervals.
 When converting a GNU m4 regex to a POSIX ERE, escape raw { and }.

 Autoconf relies on the GNU m4 regex behaviour.

 See:
 - https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Redefined-M4-Macros.html
 - https://www.gnu.org/software/gnulib/manual/html_node/emacs-regular-expression-syntax.html

 This fixes the tools/compat/configure regen.

 PR toolchain/57431


 To generate a diff of this commit:
 cvs rdiff -u -r1.11 -r1.12 src/usr.bin/m4/gnum4.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57431 CVS commit: [netbsd-10] src/usr.bin/m4
Date: Sun, 28 May 2023 10:03:54 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Sun May 28 10:03:54 UTC 2023

 Modified Files:
 	src/usr.bin/m4 [netbsd-10]: gnum4.c

 Log Message:
 Pull up following revision(s) (requested by lukem in ticket #180):

 	usr.bin/m4/gnum4.c: revision 1.12

 m4 -g: { and } aren't supported in patsubst() or regexp()

 When running in -g (GNU m4) emulation, patsubst() and regexp()
 use the GNU m4 emacs-like regexes as implemented by gnulib,
 which don't support {..} intervals.

 When converting a GNU m4 regex to a POSIX ERE, escape raw { and }.
 Autoconf relies on the GNU m4 regex behaviour.

 See:
 - https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Redefined-M4-Macros.html
 - https://www.gnu.org/software/gnulib/manual/html_node/emacs-regular-expression-syntax.html

 This fixes the tools/compat/configure regen.

 PR toolchain/57431


 To generate a diff of this commit:
 cvs rdiff -u -r1.11 -r1.11.2.1 src/usr.bin/m4/gnum4.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: analyzed->closed
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Sun, 28 May 2023 15:41:31 +0000
State-Changed-Why:
fixed in -current, pulled up to netbsd-10.


>Unformatted:

Home	PR Database Search