NetBSD Problem Report #55526

From www@netbsd.org  Fri Jul 31 03:19:31 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 3A3FE1A9217
	for <gnats-bugs@gnats.NetBSD.org>; Fri, 31 Jul 2020 03:19:31 +0000 (UTC)
Message-Id: <20200731031930.372581A9218@mollari.NetBSD.org>
Date: Fri, 31 Jul 2020 03:19:30 +0000 (UTC)
From: uwe@stderr.spb.ru
Reply-To: uwe@stderr.spb.ru
To: gnats-bugs@NetBSD.org
Subject: sh: "command foo" removes function "foo"
X-Send-Pr-Version: www-1.0

>Number:         55526
>Category:       bin
>Synopsis:       sh: "command foo" removes function "foo"
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kre
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jul 31 03:20:00 +0000 2020
>Closed-Date:    Mon Dec 07 21:26:42 +0000 2020
>Last-Modified:  Mon Dec 07 21:26:42 +0000 2020
>Originator:     Valery Ushakov
>Release:        NetBSD 8-STABLE
>Organization:
>Environment:
NetBSD pony 8.1_STABLE NetBSD 8.1_STABLE (GENERIC) #0: Sat Jun 15 07:30:17 MSK 2019  uwe@sampo:/home/uwe/work/netbsd/build8/obj/macppc/sys/arch/macppc/compile/GENERIC macppc  
>Description:
If there's a shell function "foo" with the same name as a command
in your path, running "command foo" seems to clobber the function
of the same name.
>How-To-Repeat:
$ command -V ls
ls is /bin/ls
$ ls
file.sh
$ ls() { command ls -aCF "$@"; }
$ ls
./        ../       file.sh*
$ ls
file.sh
$ command -V ls
ls is a tracked alias for /bin/ls
$ ls() { command ls -aCF "$@"; }
$ command -V ls
ls is a shell function
$ command -V ls
ls is a shell function
$ command ls
file.sh
$ ls
file.sh
$ 

>Fix:

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: bin-bug-people->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Fri, 31 Jul 2020 05:51:19 +0000
Responsible-Changed-Why:
I am looking into this PR


From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/55526: sh: "command foo" removes function "foo"
Date: Fri, 31 Jul 2020 21:43:26 +0700

 This looks to be a "day 1" bug in sh, it was fixed where
 command is applied to a builtin by dsl@ back in 2003, but
 this case, for an external command wasn't noticed, and
 has been there apparently forever (it is certainly in the
 NetBSD 6 sh, I still have that one around to test).

 I wondered how I never encountered this problem, as I
 very often redefine "cd" in ways which eventually get
 to execute "command cd" - but then I saw that the two
 cases run through different paths in the code, and this
 one was long fixed.

 I have a fix I will commit after my test build and ATF
 run completes, but the effect of this, is that while it
 will be correct, it won't be efficient.   The hash table
 which in sh s used to hold functions, builtins, just
 about everything, can only have one entry for a name, it
 is either a function (or builtin) or it is the results
 of a PATH search from a previous execution of the command.
 They cannot both be present - the bug is in adding the
 path found (/bin/ls in the example) to the has table, in
 the "ls" slot (whereupon the function gets lost), and the
 short term fix will be just not to do that (which means
 that when one runs "command ls" and there is a function "ls"
 the results of the PATH search to find /bin/ls will never
 be cached (if there is no function, it still will be, it
 is that which will inhibit the addition, not the use of
 "command").

 I'll look and see if there's a reasonable way to handle both
 later (to have both the function and the hashed path).

 kre

 ps: note that this combined "do everything" hash table is one
 of the features of ash derived shells which make them relatively
 fast - there is just one lookup of a command, no "is this a function,
 no, then is it a builtin, no, then is it in the hash table, no, then
 do a path search" just "is it in the hash table, if no, then search
 path and add it" with a "don't find functions" flag for use with the
 "command" builtin.

State-Changed-From-To: open->analyzed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Fri, 31 Jul 2020 15:42:08 +0000
State-Changed-Why:
Problem understood, fix being tested.


State-Changed-From-To: analyzed->needs-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sat, 01 Aug 2020 17:52:10 +0000
State-Changed-Why:
Needs pullups to -8 and -9 (but let it percolate in HEAD first).


From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55526 CVS commit: src/bin/sh
Date: Sat, 1 Aug 2020 17:51:18 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Sat Aug  1 17:51:18 UTC 2020

 Modified Files:
 	src/bin/sh: exec.c

 Log Message:
 PR bin/55526

 Fix a bug that has existed since the "command" command was added in
 2003.   "command foo" would cause the definition of a function "foo"
 to be lost (not freed, simply discarded) if "foo" is (in addition to
 being a function) a filesystem command.   The case where "foo" is
 a builtin was handled.

 For now, when a function exists with the same name as a filesystem
 command, the latter can never appear in the command hash table, and
 when used (which can only be via "command foo", just "foo" finds
 the function) will always result in a full PATH search.

 XXX pullup everything (from NetBSD 2.0 onwards).   (really -8 and -9)


 To generate a diff of this commit:
 cvs rdiff -u -r1.53 -r1.54 src/bin/sh/exec.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: needs-pullups->pending-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Wed, 26 Aug 2020 22:07:06 +0000
State-Changed-Why:
[pullup-9 #1064] 

nb: reset this to needs-pullups when done, as pullup to -8 is
yet to be requested.


From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55526 CVS commit: [netbsd-9] src/bin/sh
Date: Thu, 27 Aug 2020 09:15:39 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Thu Aug 27 09:15:39 UTC 2020

 Modified Files:
 	src/bin/sh [netbsd-9]: exec.c

 Log Message:
 Pull up following revision(s) (requested by kre in ticket #1064):

 	bin/sh/exec.c: revision 1.54

 PR bin/55526

 Fix a bug that has existed since the "command" command was added in
 2003.   "command foo" would cause the definition of a function "foo"
 to be lost (not freed, simply discarded) if "foo" is (in addition to
 being a function) a filesystem command.   The case where "foo" is
 a builtin was handled.

 For now, when a function exists with the same name as a filesystem
 command, the latter can never appear in the command hash table, and
 when used (which can only be via "command foo", just "foo" finds
 the function) will always result in a full PATH search.

 XXX pullup everything (from NetBSD 2.0 onwards).   (really -8 and -9)


 To generate a diff of this commit:
 cvs rdiff -u -r1.53 -r1.53.2.1 src/bin/sh/exec.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->needs-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Thu, 27 Aug 2020 14:58:37 +0000
State-Changed-Why:
Pullup to -9 complete, now waiting on requesting one for -8


State-Changed-From-To: needs-pullups->pending-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Mon, 07 Dec 2020 00:15:49 +0000
State-Changed-Why:
[pullup-8 #1629] requested.


From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55526 CVS commit: [netbsd-8] src/bin/sh
Date: Mon, 7 Dec 2020 19:39:09 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Mon Dec  7 19:39:09 UTC 2020

 Modified Files:
 	src/bin/sh [netbsd-8]: exec.c

 Log Message:
 Pull up following revision(s) (requested by kre in ticket #1629):

 	bin/sh/exec.c: revision 1.54

 PR bin/55526

 Fix a bug that has existed since the "command" command was added in
 2003.   "command foo" would cause the definition of a function "foo"
 to be lost (not freed, simply discarded) if "foo" is (in addition to
 being a function) a filesystem command.   The case where "foo" is
 a builtin was handled.

 For now, when a function exists with the same name as a filesystem
 command, the latter can never appear in the command hash table, and
 when used (which can only be via "command foo", just "foo" finds
 the function) will always result in a full PATH search.

 XXX pullup everything (from NetBSD 2.0 onwards).   (really -8 and -9)


 To generate a diff of this commit:
 cvs rdiff -u -r1.47.2.4 -r1.47.2.5 src/bin/sh/exec.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Mon, 07 Dec 2020 21:26:42 +0000
State-Changed-Why:
Pullups completed


>Unformatted:

Home	PR Database Search