NetBSD Problem Report #52641

From kre@munnari.OZ.AU  Mon Oct 23 06:03:34 2017
Return-Path: <kre@munnari.OZ.AU>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id E361C7A1CB
	for <gnats-bugs@www.NetBSD.org>; Mon, 23 Oct 2017 06:03:34 +0000 (UTC)
Message-Id: <201710230603.v9N6353I000391@andromeda.noi.kre.to>
Date: Mon, 23 Oct 2017 13:03:05 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@www.NetBSD.org
Subject: /bin/sh "jobs" command incorrectly does an implicit "wait" on completed jobs
X-Send-Pr-Version: 3.95

>Number:         52641
>Category:       bin
>Synopsis:       /bin/sh "jobs" command incorrectly does an implicit "wait" on completed jobs
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kre
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Oct 23 06:05:00 +0000 2017
>Closed-Date:    Fri Nov 17 18:33:07 +0000 2017
>Last-Modified:  Fri Nov 17 18:33:07 +0000 2017
>Originator:     Robert Elz
>Release:        NetBSD 8.99.1  (essentially all of them)
>Organization:
>Environment:
System: NetBSD andromeda.noi.kre.to 8.99.1 NetBSD 8.99.1 (VBOX64-1.3-20170812) #39: Sat Aug 12 15:25:04 ICT 2017 kre@magnolia.noi.kre.to:/usr/obj/current/kernels/amd64/VBOX64 amd64
Architecture: x86_64
Machine: amd64
>Description:
	When the "jobs" command is performed, /bin/sh discards the
	status of any job it displays which is completed.

	That's (arguably) acceptable for interactive shells, which is
	where "jobs" is most commonly used (though probably technically
	incorrect - but it mirrors the behaviour of interactive shells only
	ever displaying the status of a completed job once) but is simply
	wrong for scripts.

>How-To-Repeat:
	If you had a shell where PR bin/52640 is fixed already, and this
	PR is not, you'd run the following script ...

#! /bin/sh

for f in 1 2 3 4 5 6
do
	(sleep $f && exit $f) & eval P$f=$!
done

jobs -l

sleep 10

jobs -l

wait $P1 $P3 $P5
echo Status $?

jobs -l

wait $P6 $P4
echo Status $?

jobs -l

wait $P2
echo Status $?

jobs -l

	And observe that the "Status" reports all say 127, and that there
	is no more output from the "jobs" command after the first one
	after the "sleep 10" which shows all of the background jobs as "Done".

	The Status messages should be "5" "4" and "2"

	Whether "jobs" should display the unwaited for jobs, or only those
	not previously displayed as "done" is unclear, and not the same
	in other shells - bash and ksh display completed jobs only once,
	FreeBSD's sh keeps showing them in the jobs command until they're
	waited on.   dash has the same bug that our /bin/sh has.

	Of course, none of you will ever actually observe this, as
	this problem will be fixed in the same commit as the one that
	fixes bin/52640

>Fix:
	Arrange for non-interactive shells to leave the jobs table alone
	when performing a "jobs" command - we will continue to let "jobs"
	in interactive shells act just like "wait" as that's what it is
	commonly used for (a kind of polling wait.)

	Fix coming soon...

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: bin-bug-people->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Mon, 23 Oct 2017 06:06:04 +0000
Responsible-Changed-Why:
Another of mine


From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/52641 CVS commit: src/bin/sh
Date: Mon, 23 Oct 2017 10:52:07 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Mon Oct 23 10:52:07 UTC 2017

 Modified Files:
 	src/bin/sh: jobs.c

 Log Message:
 PR bin/52640  PR bin/52641

 Don't delete jobs from the jobs table merely because they finished,
 if they are not the job we are waiting upon.   (bin/52640 part 1)

 In a sub-shell environment, don't allow wait to find jobs from the
 parent shell that had already exited (before the sub-shell was
 created) and return status for them as if they are our children.
 (bin/52640 part 2)

 Don't have the "jobs" command also be an implicit "wait" command
 in non-interactive shells.  (bin/52641)

 Use WCONTINUED (when it exists) so we can report on stopped jobs that
 "mysteriously" move back to running state without the user issuing
 a "bg" command (eg: kill -CONT <pid>)   Previously they would keep
 being reported as stopped until they exited.

 When a job is detected as having changed status just as we're
 issuing a "jobs" command (i.e.: the change occurred between the last
 prompt and the jobs command being entered) don't report it twice,
 once from the status change, and then again in the jobs command
 output.   Once is enough (keep the jobs output, suppress the other).

 Apply some sanity to the way jobs_invalid is processed - ignore it
 in getjob() instead of just ignoring it most of the time there, and
 instead always check it before calling getjob() in situations where
 we can handle only children of the current shell.  This allows the
 (totally broken) save/clear/restore of jobs_invalid in jobscmd() to
 be done away with (previously an error while in the clear state would
 have left jobs_invalid incorrectly cleared - shouldn't have mattered
 since jobs_invalid => subshell => error causes exit, but better to be safe).

 Add/improve the DEBUG more tracing.

 XXX pullup -8


 To generate a diff of this commit:
 cvs rdiff -u -r1.90 -r1.91 src/bin/sh/jobs.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->needs-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Mon, 23 Oct 2017 10:57:06 +0000
State-Changed-Why:
waiting on some testing before requesting puttup to -8


State-Changed-From-To: needs-pullups->pending-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sun, 29 Oct 2017 16:02:41 +0000
State-Changed-Why:
Ticket 337 for netbsd-8


From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/52641 CVS commit: [netbsd-8] src/bin/sh
Date: Fri, 17 Nov 2017 14:56:52 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Fri Nov 17 14:56:52 UTC 2017

 Modified Files:
 	src/bin/sh [netbsd-8]: jobs.c

 Log Message:
 Pull up following revision(s) (requested by kre in ticket #337):
 	bin/sh/jobs.c: revision 1.91 (patch)

 PR bin/52640  PR bin/52641

 Don't delete jobs from the jobs table merely because they finished,
 if they are not the job we are waiting upon.   (bin/52640 part 1)

 In a sub-shell environment, don't allow wait to find jobs from the
 parent shell that had already exited (before the sub-shell was
 created) and return status for them as if they are our children.
 (bin/52640 part 2)

 Don't have the "jobs" command also be an implicit "wait" command
 in non-interactive shells.  (bin/52641)

 Use WCONTINUED (when it exists) so we can report on stopped jobs that
 "mysteriously" move back to running state without the user issuing
 a "bg" command (eg: kill -CONT <pid>)   Previously they would keep
 being reported as stopped until they exited.
 When a job is detected as having changed status just as we're
 issuing a "jobs" command (i.e.: the change occurred between the last
 prompt and the jobs command being entered) don't report it twice,
 once from the status change, and then again in the jobs command
 output.   Once is enough (keep the jobs output, suppress the other).

 Apply some sanity to the way jobs_invalid is processed - ignore it
 in getjob() instead of just ignoring it most of the time there, and
 instead always check it before calling getjob() in situations where
 we can handle only children of the current shell.  This allows the
 (totally broken) save/clear/restore of jobs_invalid in jobscmd() to
 be done away with (previously an error while in the clear state would
 have left jobs_invalid incorrectly cleared - shouldn't have mattered
 since jobs_invalid => subshell => error causes exit, but better to be safe).

 Add/improve the DEBUG more tracing.


 To generate a diff of this commit:
 cvs rdiff -u -r1.85.2.1 -r1.85.2.2 src/bin/sh/jobs.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Fri, 17 Nov 2017 18:33:07 +0000
State-Changed-Why:
Pullups completed


>Unformatted:

Home	PR Database Search