NetBSD Problem Report #51142
From www@NetBSD.org Sun May 15 17:03:02 2016
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 0A6CD7AAB4
for <gnats-bugs@gnats.NetBSD.org>; Sun, 15 May 2016 17:03:02 +0000 (UTC)
Message-Id: <20160515170300.C814A7AAB5@mollari.NetBSD.org>
Date: Sun, 15 May 2016 17:03:00 +0000 (UTC)
From: richard@NetBSD.org
Reply-To: richard@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: pdksh has occasional issues with empty 'for' loops
X-Send-Pr-Version: www-1.0
>Number: 51142
>Category: pkg
>Synopsis: pdksh has occasional issues with empty 'for' loops
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: pkg-manager
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sun May 15 17:05:00 +0000 2016
>Closed-Date: Tue Jul 05 08:48:20 +0000 2016
>Last-Modified: Tue Jul 05 08:48:20 +0000 2016
>Originator: Richard PALO
>Release: current
>Organization:
>Environment:
SunOS omnis 5.11 omnios-master-5409e8f i86pc i386 i86pc
>Description:
the following configure snippet always fails on my platform when using pdksh as my configure shell (noticed with evince3):
# ================ yelp-tools stuff ============= #
YELP_LC_MEDIA_LINKS=true
YELP_LC_DIST=true
for yelpopt in ; do
case $yelpopt in
lc-media-links) YELP_LC_MEDIA_LINKS=true ;;
no-lc-media-links) YELP_LC_MEDIA_LINKS= ;;
lc-dist) YELP_LC_DIST=true ;;
no-lc-dist) YELP_LC_DIST= ;;
*) as_fn_error $? "Unrecognized YELP_HELP_INIT option $yelpopt\"" "$LINENO" 5 ;;
esac
done;
at configure time this gives (for example):
>configure: error: Unrecognized YELP_HELP_INIT option dummy"
>*** Error code 1
'dummy' seems to only be used via 'set dummy blahbla; ac_word=$2'
type of things.
Seems to be cases where something in pdksh isn't reinitialised
correctly, as manually running the snippet in isolation doesn't
show the problem.
bash and ksh93 work fine too
>How-To-Repeat:
see above, but a quick work-around patch that seems to work is:
diff --git a/print/evince3/patches/patch-configure b/print/evince3/patches/patch-configure
new file mode 100644
index 0000000..f81f857
--- /dev/null
+++ b/print/evince3/patches/patch-configure
@@ -0,0 +1,21 @@
+$NetBSD$
+
+deal with YELP_HELP_INIT invoked with no options
+
+--- configure.orig 2016-03-21 16:28:17.000000000 +0000
++++ configure
+@@ -22382,12 +22382,13 @@ fi
+ YELP_LC_MEDIA_LINKS=true
+ YELP_LC_DIST=true
+
+-for yelpopt in ; do
++for yelpopt in z-end; do
+ case $yelpopt in
+ lc-media-links) YELP_LC_MEDIA_LINKS=true ;;
+ no-lc-media-links) YELP_LC_MEDIA_LINKS= ;;
+ lc-dist) YELP_LC_DIST=true ;;
+ no-lc-dist) YELP_LC_DIST= ;;
++ z-end) ;;
+ *) as_fn_error $? "Unrecognized YELP_HELP_INIT option $yelpopt\"" "$LINENO" 5 ;;
+ esac
+ done;
(I also have a patch for yelp.m4 in wip/yelp-tools)
>Fix:
>Release-Note:
>Audit-Trail:
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Sun, 15 May 2016 19:28:04 +0200
On Sun, May 15, 2016 at 05:05:00PM +0000, richard@NetBSD.org wrote:
> >Synopsis: pdksh has occasional issues with empty 'for' loops
That's not a shell bug, but a "feature" of the standard. A common
pattern found e.g. in mk/ is to explicit append '' (empty string) and
short cut with [ -n "$i" ] || break
Joerg
From: Robert Elz <kre@munnari.OZ.AU>
To: Joerg Sonnenberger <joerg@bec.de>
Cc: gnats-bugs@NetBSD.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Mon, 16 May 2016 08:01:11 +0700
Date: Sun, 15 May 2016 19:28:04 +0200
From: Joerg Sonnenberger <joerg@bec.de>
Message-ID: <20160515172804.GA11874@britannica.bec.de>
| On Sun, May 15, 2016 at 05:05:00PM +0000, richard@NetBSD.org wrote:
| > >Synopsis: pdksh has occasional issues with empty 'for' loops
|
| That's not a shell bug, but a "feature" of the standard.
I believe you're thinking of "for i; do" rather than "for i in; do"
It certainly looks like a bug in pdksh to me (it isn't as if it does not
have many) - certainly if that were reported against /bin/sh I would be
fixing it.
I notice the PR says that bash and ksh93 work (as they should) but says
nothing about /bin/sh. Is there a reason for that? (In general, unless you
want it for interactive use reasons, I'd always suggest /bin/sh on NetBSD
over /bin/ksh)
kre
From: Richard PALO <richard@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Mon, 16 May 2016 06:52:36 +0200
Le 15/05/16 19:30, Joerg Sonnenberger a écrit :
> That's not a shell bug, but a "feature" of the standard. A common
> pattern found e.g. in mk/ is to explicit append '' (empty string) and
> short cut with [ -n "$i" ] || break
>
> Joerg
Indeed!
http://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html#tag_02_14
does say:
> The format for the for loop is as follows:
>
> for name [ in [word ... ]]do
> compound-list
>
>
> done
>
> First, the list of words following in shall be expanded to generate a list of items.
> Then, the variable name shall be set to each item, in turn, and the compound-list
> executed each time. If no items result from the expansion, the compound-list shall
> not be executed. Omitting:
>
> in word ...
>
> shall be equivalent to:
>
> in "$@"
>
and using "set dummy; shift" prior to the 'for' loop gets over the yelp problem too.
So it looks as if pdksh inadvertently performs the "$@" substitution even if the "in" remains.
Testing again bash and ksh93, ommiting the "in" gives 'dummy' as with pdksh so it does
look like pdksh is the culprit here.
cheers,
--
Richard PALO
From: Richard PALO <richard@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Mon, 16 May 2016 07:00:47 +0200
Le 16/05/16 03:05, Robert Elz a écrit :
> I believe you're thinking of "for i; do" rather than "for i in; do"
>
Guess I should have refreshed first, after coming to the same conclusion...
> It certainly looks like a bug in pdksh to me (it isn't as if it does not
> have many) - certainly if that were reported against /bin/sh I would be
> fixing it.
>
> I notice the PR says that bash and ksh93 work (as they should) but says
> nothing about /bin/sh. Is there a reason for that? (In general, unless you
> want it for interactive use reasons, I'd always suggest /bin/sh on NetBSD
> over /bin/ksh)
>
> kre
I'm running on SunOS 5.11 where native /bin/sh is the same ksh93, but my ksh93
test was with pkgsrc ksh93...
(PM: bootstrap --full gives pdksh as the default pkgsrc shell; which I use)
cheers
--
Richard PALO
From: Richard PALO <richard.palo@free.fr>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Mon, 16 May 2016 09:10:17 +0200
Le 16/05/16 06:55, Richard PALO a écrit :
> Testing again bash and ksh93, ommiting the "in" gives 'dummy' as with pdksh so it does
> look like pdksh is the culprit here.
>
Although by no means specialist in shell processing, this did seem to come down to around
the following in the wordlist() function found in syn.c
Prior to this patch, the result was the same whether "in" was specified with a null set of options or not.
> diff --git a/shells/pdksh/files/syn.c b/shells/pdksh/files/syn.c
> index d3bce2e..3b19d0a 100644
> --- a/shells/pdksh/files/syn.c
> +++ b/shells/pdksh/files/syn.c
> @@ -609,13 +609,8 @@ wordlist()
> XPput(args, yylval.cp);
> if (c != '\n' && c != ';')
> syntaxerr((char *) 0);
> - if (XPsize(args) == 0) {
> - XPfree(args);
> - return NULL;
> - } else {
> - XPput(args, NULL);
> - return (char **) XPclose(args);
> - }
> + XPput(args, NULL);
> + return (char **) XPclose(args);
> }
>
> /*
before:
> richard@omnis:/home/richard/src/pkgsrc/shells/pdksh$ /opt/local/bin/pdksh
> $ set dummy
> $ for i in ; do echo $i; done
> dummy
> $ for i ; do echo $i; done
> dummy
after:
> richard@omnis:/tmp/pkgsrc/shells/pdksh/work/.destdir/opt/local/bin$ ./pdksh
> $ set dummy
> $ for i in ; do echo $i; done
> $ for i ; do echo $i; done
> dummy
does this look reasonable?
--
Richard PALO
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org,
richard@NetBSD.org
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Mon, 16 May 2016 10:58:26 +0200
On Mon, May 16, 2016 at 04:55:01AM +0000, Richard PALO wrote:
> The following reply was made to PR pkg/51142; it has been noted by GNATS.
>
> From: Richard PALO <richard@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
> Date: Mon, 16 May 2016 06:52:36 +0200
>
> Le 15/05/16 19:30, Joerg Sonnenberger a écrit :
> > That's not a shell bug, but a "feature" of the standard. A common
> > pattern found e.g. in mk/ is to explicit append '' (empty string) and
> > short cut with [ -n "$i" ] || break
> >
> > Joerg
>
> Indeed!
> http://pubs.opengroup.org/onlinepubs/009604499/utilities/xcu_chap02.html#tag_02_14
> does say:
> > The format for the for loop is as follows:
> >
> > for name [ in [word ... ]]do
> > compound-list
> >
> >
> > done
Actually, you are looking at a too-new version. Try SUSv2 for example:
http://pubs.opengroup.org/onlinepubs/7908799/xcu/chap2.html#tag_001_009_004_002
for name [ in word ... ]
do
compound-list
done
...and that has been interpreted historically by some shells (this one
included) to mean that after the "in" something must be found. The BNF
grammar is similar problematic. I'm not really objecting to fixing this
is pdksh, but be careful to call it a bug.
Joerg
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Mon, 16 May 2016 17:13:46 +0700
Date: Mon, 16 May 2016 07:10:01 +0000 (UTC)
From: Richard PALO <richard.palo=40free.fr>
Message-ID: <20160516071001.CEF3F7AAB4=40mollari.NetBSD.org>
=7C does this look reasonable?
I can't comment on the patch, as I don't know the code, but the
result looks to be what is wanted.
joerg=40bec.de said:
=7C Actually, you are looking at a too-new version.=20
The problem with a lot of the older shell specs is that they were
wildly wrong (did not say what they really intended). Whether
an implementation which adapted to what those seemed to require can
be said to have bugs or not is just semantics - no-one is assigning
blame here (or shouldn't be) just trying to get everything operating
as it should. And whether that =22word...=22 was actually intended to m=
ean
one more more words, or zero or more words doesn't really matter any
more - anything that believed the 1 or more interpretation must now
know that was incorrect, and should be fixed.
kre
From: Richard PALO <richard@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Thu, 19 May 2016 13:43:30 +0200
I'm concerned about the extent of the fix wherein the code is working on *both* 'for' and 'select'.
Judging by http://linux.die.net/man/1/pdksh it appears that 'select' shouldn't necessarily
support the 'in ;' sort of usage.
Could I get any suggestions, confirmation (or infirmation) of this to determine if there needs to be a
a bit of adapting for 'select' case.
Other than this mention, outside of a bulk build (which I can't do since a while until
I resync my tree's 'current' pbulk with 'joyent' bits) the patch seems be working well.
--
Richard PALO
From: Richard PALO <richard@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Tue, 31 May 2016 17:49:06 +0200
Le 19/05/16 13:45, Richard PALO a écrit :
>
> I'm concerned about the extent of the fix wherein the code is working on *both* 'for' and 'select'.
>
Adding as well the following patch seems to get 'select' to a similarly working situation:
> diff --git a/shells/pdksh/files/exec.c b/shells/pdksh/files/exec.c
> index 734d484..14e216b 100644
> --- a/shells/pdksh/files/exec.c
> +++ b/shells/pdksh/files/exec.c
> @@ -334,7 +334,7 @@ execute(t, flags)
> }
> #ifdef KSH
> else { /* TSELECT */
> - for (;;) {
> + while (*ap != NULL) {
> if (!(cp = do_selectargs(ap, is_first))) {
> rv = 1;
> break;
before:
> richard@omnis:/home/richard/src/pkgsrc/shells/pdksh$ /opt/local/bin/pdksh.orig
> $ set dummy
> $ select i in ; do echo $i; break; done
> 1) dummy
> #? 1
> dummy
> $ select i ; do echo $i; break; done
> 1) dummy
> #? 1
> dummy
after:
> richard@omnis:/home/richard/src/pkgsrc/shells/pdksh$ /tmp/pkgsrc/shells/pdksh/work/.destdir/opt/local/bin/pdksh
> $ set dummy
> $ select i in ; do echo $i; break; done
> $ echo $?
> 0
> $ select i ; do echo $i; break; done
> 1) dummy
> #? 1
> dummy
> $ echo $?
> 0
Request some 'okays' to commit
--
Richard PALO
From: Richard PALO <richard@netbsd.org>
To: gnats-bugs@NetBSD.org, =?UTF-8?B?SsO2cmcgU29ubmVuYmVyZ2Vy?=
<joerg@netbsd.org>, Robert Elz <kre@munnari.OZ.AU>
Cc:
Subject: Re: pkg/51142: pdksh has occasional issues with empty 'for' loops
Date: Wed, 01 Jun 2016 08:32:50 +0200
I put a copy of the complete patchset proposed here:
> http://www.netbsd.org/~richard/pdksh.patch
as jörg indicated, this seems not to be a bug per se but, but as already indicated in the source,
this is for ast-ksh compatibility as encountered.
btw, seems in bash, as opposed to ast-ksh, the select clause without the 'in ...' gives a syntax error.
I don't believe this has any impact, strictly speaking, on pkgsrc as a build shell.
cheers
--
Richard PALO
From: "Richard PALO" <richard@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/51142 CVS commit: pkgsrc/shells/pdksh
Date: Sun, 5 Jun 2016 04:44:56 +0000
Module Name: pkgsrc
Committed By: richard
Date: Sun Jun 5 04:44:56 UTC 2016
Modified Files:
pkgsrc/shells/pdksh: Makefile
pkgsrc/shells/pdksh/files: exec.c syn.c
Log Message:
PR/51142: address for/select loop compatibility with ksh93 in pdksh
To generate a diff of this commit:
cvs rdiff -u -r1.23 -r1.24 pkgsrc/shells/pdksh/Makefile
cvs rdiff -u -r1.3 -r1.4 pkgsrc/shells/pdksh/files/exec.c \
pkgsrc/shells/pdksh/files/syn.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Tue, 05 Jul 2016 01:40:06 +0000
State-Changed-Why:
Is this fixed?
From: Richard PALO <richard@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/51142 (pdksh has occasional issues with empty 'for' loops)
Date: Tue, 05 Jul 2016 08:24:45 +0200
Le 05/07/16 03:40, dholland@NetBSD.org a écrit :
> Synopsis: pdksh has occasional issues with empty 'for' loops
>
> State-Changed-From-To: open->feedback
> State-Changed-By: dholland@NetBSD.org
> State-Changed-When: Tue, 05 Jul 2016 01:40:06 +0000
> State-Changed-Why:
> Is this fixed?
>
>
>
>
I have not received any [negative] feedback since the patch
committed. It can probably be closed, at least for now.
--
Richard PALO
State-Changed-From-To: feedback->closed
State-Changed-By: bsiegert@NetBSD.org
State-Changed-When: Tue, 05 Jul 2016 08:48:20 +0000
State-Changed-Why:
Submitter says fixed.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.