NetBSD Problem Report #52090
From kre@munnari.OZ.AU Sun Mar 19 03:25:33 2017
Return-Path: <kre@munnari.OZ.AU>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 6E2E47A2B7
for <gnats-bugs@www.NetBSD.org>; Sun, 19 Mar 2017 03:25:33 +0000 (UTC)
Message-Id: <201703190325.v2J3PLcY021813@andromeda.noi.kre.to>
Date: Sun, 19 Mar 2017 10:25:21 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@www.NetBSD.org
Subject: /bin/sh does not expand $* (or $@) when unquoted correctly
X-Send-Pr-Version: 3.95
>Number: 52090
>Category: bin
>Synopsis: /bin/sh does not expand $* (or $@) when unquoted correctly
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: kre
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sun Mar 19 03:30:00 +0000 2017
>Closed-Date: Thu Mar 23 11:25:57 +0000 2017
>Last-Modified: Thu Mar 23 11:25:57 +0000 2017
>Originator: Robert Elz
>Release: NetBSD 7.99.65 (irrelevant, applies to all versions)
>Organization:
>Environment:
System: NetBSD andromeda.noi.kre.to 7.99.43 NetBSD 7.99.43 (VBOX64-1.2-20161202) #21: Sat Dec 3 00:43:51 ICT 2016 kre@magnolia.noi.kre.to:/usr/obj/current/kernels/amd64/VBOX64 amd64
Architecture: x86_64
Machine: amd64
>Description:
The rules for expanding an unquoted $* (or $@) are that they
produce 1 word for each set positional parameter.
From Posix sh standard, section 2.5.2
74866 * Expands to the positional parameters, starting from one,
initially producing one field for
74867 each positional parameter that is set.
(the margin numbers are the line numbers from the current version,
I wrapped the first line for this e-mail)
There are, of course, more details, but those aren't relevant here.
After those fields are produced, word splitting is (usually)
performed, on each of them in turn, possibly generating more
fields.
[aside: "field' is the term used for what we would thing of as a word,
or perhaps an arg, but is more precisely defined, and those other terms
are used with different meanings.]
What the NetBSD sh does is produce a single field, with the
positional parameters separated by IFS[0] (the first char of $IFS)
When word splitting is done, the field is split at each IFS[0]
character (and other IFS characters if they occur) so each
inserted IFS in the single field produced causes a new field
to appear in the final output.
It appears as if everything is just fine implementing it this
way (and it more or less is) -- except when IFS=''
Then the way the sh currently does it is simply to run all the
positional parameters together into a single string. Since there
are no chars in IFS, when word splitting is attempted nothing
happens. The result is a single field, where there should be
more one for each positional parameter.
>How-To-Repeat:
An easy way to see this is
set a b c d ; # $# is 4, and $1 .. $4 are set
IFS= ; no chars in IFS
set $* ; set the args to whatever $* expands to
echo $# ; should produce 4
On NetBSD the final echo writes "1" rather than "4" as it should.
There were 4 positional parameters, therefore $* should produce 4
fields, each of which should be set back into the positional params
1 .. 4. Instead the NetBSD sh produces a single param "abcd"
Note that if $* were quoted ("$*") what we do now is correct, even
if it doesn't happen internally in quite the way envisaged.
>Fix:
Not yet...
>Release-Note:
>Audit-Trail:
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/52090 CVS commit: src/tests/bin/sh
Date: Sun, 19 Mar 2017 20:29:30 +0000
Module Name: src
Committed By: kre
Date: Sun Mar 19 20:29:30 UTC 2017
Modified Files:
src/tests/bin/sh: t_expand.sh
Log Message:
PR bin/52090
Add 5 new test cases to test various ways that $* can be expanded.
Currently 3 of the 5 are marked as "expected to fail" because of the
bug in this PR.
To generate a diff of this commit:
cvs rdiff -u -r1.9 -r1.10 src/tests/bin/sh/t_expand.sh
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Responsible-Changed-From-To: bin-bug-people->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Sun, 19 Mar 2017 22:33:04 +0000
Responsible-Changed-Why:
I am looking into this PR
State-Changed-From-To: open->analyzed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sun, 19 Mar 2017 22:34:51 +0000
State-Changed-Why:
I think I know what is happening, and why - and have a fix that looks
like it might work, but it is going to need a bunch of testing
(at the very least, using the fixed shell as /bin/sh for a full build)
before I will consider comitting it.
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/52090 CVS commit: src/bin/sh
Date: Mon, 20 Mar 2017 11:48:02 +0000
Module Name: src
Committed By: kre
Date: Mon Mar 20 11:48:01 UTC 2017
Modified Files:
src/bin/sh: expand.c
Log Message:
PR bin/52090 - fix expansion of unquoted $*
To generate a diff of this commit:
cvs rdiff -u -r1.103 -r1.104 src/bin/sh/expand.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/52090 CVS commit: src/tests/bin/sh
Date: Mon, 20 Mar 2017 11:48:41 +0000
Module Name: src
Committed By: kre
Date: Mon Mar 20 11:48:41 UTC 2017
Modified Files:
src/tests/bin/sh: t_expand.sh
Log Message:
PR bin/52090 - the $* tests are no longer expected to fail.
To generate a diff of this commit:
cvs rdiff -u -r1.10 -r1.11 src/tests/bin/sh/t_expand.sh
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: analyzed->closed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Thu, 23 Mar 2017 11:25:57 +0000
State-Changed-Why:
Problem fixed
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.