NetBSD Problem Report #46021
From jklowden@schemamania.org Tue Feb 14 04:46:24 2012
Return-Path: <jklowden@schemamania.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
by www.NetBSD.org (Postfix) with ESMTP id 6685263B86B
for <gnats-bugs@gnats.NetBSD.org>; Tue, 14 Feb 2012 04:46:24 +0000 (UTC)
Message-Id: <20120214025940.4EC0A30E20B1@oak.schemamania.org>
Date: Mon, 13 Feb 2012 21:59:40 -0500 (EST)
From: jklowden@schemamania.org
Reply-To: jklowden@schemamania.org
To: gnats-bugs@gnats.NetBSD.org
Subject: xargs should keep still
X-Send-Pr-Version: 3.95
>Number: 46021
>Category: bin
>Synopsis: xargs should keep still
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: bin-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Feb 14 04:50:00 +0000 2012
>Closed-Date: Sun Jan 06 23:34:05 +0000 2013
>Last-Modified: Sun Jan 06 23:34:05 +0000 2013
>Originator: jklowden@schemamania.org
>Release: NetBSD 5.0.2
>Organization:
>Environment:
System: NetBSD oak.schemamania.org 5.0.2 NetBSD 5.0.2 (GENERIC) #0: Sat Feb 6 13:44:19 UTC 2010 builds@b8.netbsd.org:/home/builds/ab/netbsd-5-0-2-RELEASE/amd64/201002061851Z-obj/home/builds/ab/netbsd-5-0-2-RELEASE/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
xargs writes error messages to standard error if the utility
it's executing exits with an error or on a signal. E.g.:
$ find /usr | xargs ls | head -1
/usr/X11R7/bin/X
xargs: ls terminated by SIGPIPE
As the above example shows, the message is unwelcome and unnecessary.
If the *utility* needs to produce a message, it can, as can the shell
(or other caller). But xargs is mere plumbing in the works, made
neccessary only by the accident that e.g. ls(1) doesn't read from
standard input.
Note also that in most cases the shell captures (in $?, say) the
exit status only of the last command in a pipeline. As such, the
a shell script using a pipeline as above would (correctly) deem
the pipeline successful. If the author of the script wishes to
suppress the superfluous message, he can of course redirect
standard error, but in so doing he'll supress genuine messages from
ls.
$ echo nonesuch /usr/* | xargs ls | head -3
ls: nonesuch: No such file or directory
/usr/X11R6:
bin
include
xargs: ls terminated by SIGPIPE
$ echo nonesuch /usr/* | xargs ls 2>/dev/null | head -3
/usr/X11R6:
bin
include
It is illustrative that although find and ls in the first example
were terminated by SIGPIPE, only xargs complained. Most netbsd
utilities do not write to standard error for SIGPIPE. That is by
design. The reader of a pipe cannot signal the writer that it has
what it needs except by closing the pipe. Therefore pipe closure
is expected and not an error condition. I have phrased this
observation as
"Failure to read is not an error".
In the case of xargs, the crime is more eggregious. xargs has no
business remarking on the exit status of its client, period. It
does not matter what the signal is. In a perfect world xargs would
disappear. In ours, it should hold its tongue.
>How-To-Repeat:
See above.
>Fix:
Index: xargs.c
===================================================================
RCS file: /cvsroot/src/usr.bin/xargs/xargs.c,v
retrieving revision 1.20
diff -u -r1.20 xargs.c
--- xargs.c 17 Dec 2010 11:32:57 -0000 1.20
+++ xargs.c 14 Feb 2012 02:21:05 -0000
@@ -593,19 +593,11 @@
*/
if (WIFEXITED(status)) {
if (WEXITSTATUS (status) == 255) {
- warnx ("%s exited with status 255", name);
exit(124);
} else if (WEXITSTATUS (status) != 0) {
rval = 123;
}
} else if (WIFSIGNALED (status)) {
- if (WTERMSIG(status) < NSIG) {
- warnx("%s terminated by SIG%s", name,
- sys_signame[WTERMSIG(status)]);
- } else {
- warnx("%s terminated by signal %d", name,
- WTERMSIG(status));
- }
exit(125);
}
}
>Release-Note:
>Audit-Trail:
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: bin/46021: xargs should keep still
Date: Tue, 14 Feb 2012 07:20:15 +0000
On Tue, Feb 14, 2012 at 04:50:00AM +0000, jklowden@schemamania.org wrote:
> xargs writes error messages to standard error if the utility
> it's executing exits with an error or on a signal. E.g.:
>
> $ find /usr | xargs ls | head -1
> /usr/X11R7/bin/X
> xargs: ls terminated by SIGPIPE
>
> As the above example shows, the message is unwelcome and unnecessary.
> [...]
I don't agree with your reasoning. Let's consider a similar and not
uncommon case:
First, create a program that always exits with a signal:
% ./die
Segmentation fault
%
Now let's place it in a pipeline:
% find /usr/include | ./die
Segmentation fault
% find /usr/include | ./die | cat
Segmentation fault
% find /usr/include | xargs ./die
xargs: ./die terminated by SIGSEGV
% find /usr/include | xargs ./die | cat
xargs: ./die terminated by SIGSEGV
%
Looks to me like the behavior is uniform across all four cases. xargs
doesn't print the same message, but it's equivalent and printed in the
same circumstances.
With sh your argument is even weaker, as sh prints more:
$ find /usr/include | ./die
[1] Broken pipe find /usr/include |
Segmentation fault ./die
$ find /usr/include | ./die | cat
[1] Broken pipe find /usr/include |
Segmentation fault ./die |
Done cat
$ find /usr/include | xargs ./die
xargs: ./die terminated by SIGSEGV
$ find /usr/include | xargs ./die | cat
xargs: ./die terminated by SIGSEGV
$
zsh is similar to sh.
Maybe SIGPIPE should be handled differently from other signals, but
the change you've proposed is definitely wrong.
--
David A. Holland
dholland@netbsd.org
From: jklowden@schemamania.org
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: bin/46021: xargs should keep still
Date: Tue, 14 Feb 2012 11:59:17 -0500
On Tue, Feb 14, 2012 at 07:25:02AM +0000, David Holland wrote:
> The following reply was made to PR bin/46021; it has been noted by GNATS.
>
> From: David Holland <dholland-bugs@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: bin/46021: xargs should keep still
> Date: Tue, 14 Feb 2012 07:20:15 +0000
>
> On Tue, Feb 14, 2012 at 04:50:00AM +0000, jklowden@schemamania.org wrote:
> > xargs writes error messages to standard error if the utility
> > it's executing exits with an error or on a signal. E.g.:
> >
> > $ find /usr | xargs ls | head -1
> > /usr/X11R7/bin/X
> > xargs: ls terminated by SIGPIPE
> >
> > As the above example shows, the message is unwelcome and unnecessary.
> > [...]
>
> I don't agree with your reasoning. Let's consider a similar and not
> uncommon case:
>
> First, create a program that always exits with a signal:
I have a better answer, see below, but I have to admit being amused at beginning
with a "not uncommon" case by writing it. :-)
> % find /usr/include | ./die
> Segmentation fault
> % find /usr/include | ./die | cat
> Segmentation fault
> % find /usr/include | xargs ./die
> xargs: ./die terminated by SIGSEGV
> % find /usr/include | xargs ./die | cat
> xargs: ./die terminated by SIGSEGV
> %
>
> xargs
> doesn't print the same message, but it's equivalent and printed in the
> same circumstances.
No, definitely not. xargs interpreted the signal and printed a message.
It swallowed the signal, preventing the shell from seeing it. In
the last example, xargs littered the terminal via standard error, but cat
returned normally (or would, if it ever got started).
Instead, if you feel it's important to mimic a pipe as precisely as possible
(I don't. I just don't want to see xargs messages, ever.) then xargs
should propagate the signal. If it did that, all four of your examples would
produce the same "Segmentation fault" message, and mine would produce none at
all.
I am prepared to modify my patch in that way, but it will affect how
xargs returns when its child is signalled.
--jkl
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: bin/46021: xargs should keep still
Date: Sun, 22 Apr 2012 20:02:37 +0000
On Tue, Feb 14, 2012 at 05:00:07PM +0000, jklowden@schemamania.org wrote:
> > % find /usr/include | ./die
> > Segmentation fault
> > % find /usr/include | ./die | cat
> > Segmentation fault
> > % find /usr/include | xargs ./die
> > xargs: ./die terminated by SIGSEGV
> > % find /usr/include | xargs ./die | cat
> > xargs: ./die terminated by SIGSEGV
> > %
> >
> > xargs
> > doesn't print the same message, but it's equivalent and printed in the
> > same circumstances.
>
> No, definitely not. xargs interpreted the signal and printed a message.
> It swallowed the signal, preventing the shell from seeing it.
So your objection is that what you'd like to see is:
% find /usr/include | ./die
Segmentation fault
% find /usr/include | ./die | cat
Segmentation fault
% find /usr/include | xargs ./die
Segmentation fault
% find /usr/include | xargs ./die | cat
Segmentation fault
%
?
But your patch would produce
% find /usr/include | ./die
Segmentation fault
% find /usr/include | ./die | cat
Segmentation fault
% find /usr/include | xargs ./die
% find /usr/include | xargs ./die | cat
%
and this really doesn't make much sense.
> In the last example, xargs littered the terminal via standard
> error, but cat returned normally (or would, if it ever got
> started).
And cat also returns normally in the second case, without xargs. I
don't see what's different.
I don't understand either what you're complaining about or what you're
trying to accomplish.
--
David A. Holland
dholland@netbsd.org
State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Thu, 03 Jan 2013 23:35:41 +0000
State-Changed-Why:
Please clarify... or perhaps this should just be closed.
From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dholland@NetBSD.org,
jklowden@schemamania.org
Subject: re: bin/46021 (xargs should keep still)
Date: Fri, 04 Jan 2013 16:41:10 +1100
reading the original description i think that what's happening
is something slightly different.
xargs isn't dying because it can't *read* anymore, it's dying
because it can't *write* anymore, because eg, "head -3" exited,
and then ls gets SIGPIPE when it tries to write more.
i agree with dh - this isn't a bug.
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/46021 (xargs should keep still)
Date: Sun, 6 Jan 2013 23:20:14 +0000
Please sent PR traffic to gnats-bugs.
------
From: "James K. Lowden" <jklowden@schemamania.org>
To: matthew green <mrg@eterna.com.au>
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dholland@NetBSD.org
Subject: Re: bin/46021 (xargs should keep still)
Date: Sat, 5 Jan 2013 12:27:58 -0500
On Fri, 04 Jan 2013 16:41:10 +1100
matthew green <mrg@eterna.com.au> wrote:
> xargs isn't dying because it can't *read* anymore, it's dying
> because it can't *write* anymore, because eg, "head -3" exited,
> and then ls gets SIGPIPE when it tries to write more.
Correct.
> i agree with dh - this isn't a bug.
Well, it's certainly not a feature. :-)
I assert no command-line utility should ever report a broken pipe error
when writing to standard output. Why?
1. It's common for a reader, e.g. head(1), not to read everything.
2. There is no mechanism, other than closing the pipe, for the reader
to notify the writer that it's done.
Ergo, closing the pipe is *normal*, not an error.
Although I've never seen it articulated clearly, there is a long, long
unix tradition of ignoring SIGPIPE on standard output, from arp(1) to
zcat(1), not to mention ls(1) and cat(1). The few that I've found that
don't include xargs, pax, svn, and IIRC git.
In case the reader is concerned that a genuine error may be missed,
bear in mind the utility is free to return a nonzero status and the
shell remains free to interpret it. Since time immemorial most shells
I know of have treated the last element in the pipeline as the arbiter
of the pipeline's success. I don't see any better choice.
In the specific instance of xargs, the case is only more obvious
because xargs is really only plumbing, no more significant on the
command line than "|" or "``". It's just an accident of history that
xargs exists as a utility instead of being part of the shell syntax.
David left it to me to supply a patch, which I never did. As I
understood it at the time, he was skeptical but understood my reasoning
and was willing to apply a patch of acceptable quality.
I agree with David that feedback is desirable. We should have consensus
on this issue before engaging the compiler.
--jkl
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/46021 (xargs should keep still)
Date: Sun, 6 Jan 2013 23:28:18 +0000
On Sat, Jan 05, 2013 at 12:27:58PM -0500, James K. Lowden wrote:
> David left it to me to supply a patch, which I never did. As I
> understood it at the time, he was skeptical but understood my reasoning
> and was willing to apply a patch of acceptable quality.
I was skeptical that there's a problem; what I wrote was:
: I don't understand either what you're complaining about or what
: you're trying to accomplish.
--
David A. Holland
dholland@netbsd.org
State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 06 Jan 2013 23:34:05 +0000
State-Changed-Why:
Feedback received; submitter needs to figure out exactly what he's trying to
accomplish, and gather consensus, which is best done on a mailing list rather
than in the bug database.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.