NetBSD Problem Report #52090

From kre@munnari.OZ.AU  Sun Mar 19 03:25:33 2017
Return-Path: <kre@munnari.OZ.AU>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 6E2E47A2B7
	for <gnats-bugs@www.NetBSD.org>; Sun, 19 Mar 2017 03:25:33 +0000 (UTC)
Message-Id: <201703190325.v2J3PLcY021813@andromeda.noi.kre.to>
Date: Sun, 19 Mar 2017 10:25:21 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@www.NetBSD.org
Subject: /bin/sh does not expand $* (or $@) when unquoted correctly
X-Send-Pr-Version: 3.95

>Number:         52090
>Category:       bin
>Synopsis:       /bin/sh does not expand $* (or $@) when unquoted correctly
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kre
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Mar 19 03:30:00 +0000 2017
>Closed-Date:    Thu Mar 23 11:25:57 +0000 2017
>Last-Modified:  Thu Mar 23 11:25:57 +0000 2017
>Originator:     Robert Elz
>Release:        NetBSD 7.99.65 (irrelevant, applies to all versions)
>Organization:
>Environment:
System: NetBSD andromeda.noi.kre.to 7.99.43 NetBSD 7.99.43 (VBOX64-1.2-20161202) #21: Sat Dec 3 00:43:51 ICT 2016 kre@magnolia.noi.kre.to:/usr/obj/current/kernels/amd64/VBOX64 amd64
Architecture: x86_64
Machine: amd64
>Description:
	The rules for expanding an unquoted $* (or $@) are that they
	produce 1 word for each set positional parameter.

	From Posix sh standard, section 2.5.2

74866    *	Expands to the positional parameters, starting from one,
		initially producing one field for
74867   	each positional parameter that is set.

	(the margin numbers are the line numbers from the current version,
	I wrapped the first line for this e-mail)

	There are, of course, more details, but those aren't relevant here.

	After those fields are produced, word splitting is (usually)
	performed, on each of them in turn, possibly generating more
	fields.

[aside: "field' is the term used for what we would thing of as a word,
or perhaps an arg, but is more precisely defined, and those other terms
are used with different meanings.]

	What the NetBSD sh does is produce a single field, with the
	positional parameters separated by IFS[0] (the first char of $IFS)

	When word splitting is done, the field is split at each IFS[0]
	character (and other IFS characters if they occur) so each
	inserted IFS in the single field produced causes a new field
	to appear in the final output.

	It appears as if everything is just fine implementing it this
	way (and it more or less is) -- except when IFS=''

	Then the way the sh currently does it is simply to run all the
	positional parameters together into a single string.  Since there
	are no chars in IFS, when word splitting is attempted nothing
	happens.  The result is a single field, where there should be
	more one for each positional parameter.

>How-To-Repeat:
	An easy way to see this is

		set a b c d		; # $# is 4, and $1 .. $4 are set
		IFS=			; no chars in IFS
		set $*			; set the args to whatever $* expands to
		echo $#			; should produce 4

	On NetBSD the final echo writes "1" rather than "4" as it should.
	There were 4 positional parameters, therefore $* should produce 4
	fields, each of which should be set back into the positional params
	1 .. 4.   Instead the NetBSD sh produces a single param "abcd"

	Note that if $* were quoted ("$*") what we do now is correct, even
	if it doesn't happen internally in quite the way envisaged.

>Fix:
	Not yet...

>Release-Note:

>Audit-Trail:
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/52090 CVS commit: src/tests/bin/sh
Date: Sun, 19 Mar 2017 20:29:30 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Sun Mar 19 20:29:30 UTC 2017

 Modified Files:
 	src/tests/bin/sh: t_expand.sh

 Log Message:
 PR bin/52090

 Add 5 new test cases to test various ways that $* can be expanded.
 Currently 3 of the 5 are marked as "expected to fail" because of the
 bug in this PR.


 To generate a diff of this commit:
 cvs rdiff -u -r1.9 -r1.10 src/tests/bin/sh/t_expand.sh

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: bin-bug-people->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Sun, 19 Mar 2017 22:33:04 +0000
Responsible-Changed-Why:
I am looking into this PR


State-Changed-From-To: open->analyzed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sun, 19 Mar 2017 22:34:51 +0000
State-Changed-Why:
I think I know what is happening, and why - and have a fix that looks
like it might work, but it is going to need a bunch of testing
(at the very least, using the fixed shell as /bin/sh for a full build)
before I will consider comitting it.


From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/52090 CVS commit: src/bin/sh
Date: Mon, 20 Mar 2017 11:48:02 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Mon Mar 20 11:48:01 UTC 2017

 Modified Files:
 	src/bin/sh: expand.c

 Log Message:
 PR bin/52090 - fix expansion of unquoted $*


 To generate a diff of this commit:
 cvs rdiff -u -r1.103 -r1.104 src/bin/sh/expand.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/52090 CVS commit: src/tests/bin/sh
Date: Mon, 20 Mar 2017 11:48:41 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Mon Mar 20 11:48:41 UTC 2017

 Modified Files:
 	src/tests/bin/sh: t_expand.sh

 Log Message:
 PR bin/52090 - the $* tests are no longer expected to fail.


 To generate a diff of this commit:
 cvs rdiff -u -r1.10 -r1.11 src/tests/bin/sh/t_expand.sh

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: analyzed->closed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Thu, 23 Mar 2017 11:25:57 +0000
State-Changed-Why:
Problem fixed


>Unformatted:

Home	PR Database Search