NetBSD Problem Report #46021

From jklowden@schemamania.org  Tue Feb 14 04:46:24 2012
Return-Path: <jklowden@schemamania.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 6685263B86B
	for <gnats-bugs@gnats.NetBSD.org>; Tue, 14 Feb 2012 04:46:24 +0000 (UTC)
Message-Id: <20120214025940.4EC0A30E20B1@oak.schemamania.org>
Date: Mon, 13 Feb 2012 21:59:40 -0500 (EST)
From: jklowden@schemamania.org
Reply-To: jklowden@schemamania.org
To: gnats-bugs@gnats.NetBSD.org
Subject: xargs should keep still
X-Send-Pr-Version: 3.95

>Number:         46021
>Category:       bin
>Synopsis:       xargs should keep still
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    bin-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Feb 14 04:50:00 +0000 2012
>Closed-Date:    Sun Jan 06 23:34:05 +0000 2013
>Last-Modified:  Sun Jan 06 23:34:05 +0000 2013
>Originator:     jklowden@schemamania.org
>Release:        NetBSD 5.0.2
>Organization:

>Environment:


System: NetBSD oak.schemamania.org 5.0.2 NetBSD 5.0.2 (GENERIC) #0: Sat Feb 6 13:44:19 UTC 2010 builds@b8.netbsd.org:/home/builds/ab/netbsd-5-0-2-RELEASE/amd64/201002061851Z-obj/home/builds/ab/netbsd-5-0-2-RELEASE/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
	xargs writes error messages to standard error if the utility 
	it's executing exits with an error or on a signal.  E.g.:

	$ find /usr | xargs ls | head -1
	/usr/X11R7/bin/X
	xargs: ls terminated by SIGPIPE

	As the above example shows, the message is unwelcome and unnecessary.  
	If the *utility* needs to produce a message, it can, as can the shell
	(or other caller).  But xargs is mere plumbing in the works, made 
	neccessary only by the accident that e.g. ls(1) doesn't read from
	standard input.  

	Note also that in most cases the shell captures (in $?, say) the 
	exit status only of the last command in a pipeline.  As such, the
	a shell script using a pipeline as above would (correctly) deem
	the pipeline successful.  If the author of the script wishes to
	suppress the superfluous message, he can of course redirect
	standard error, but in so doing he'll supress genuine messages from
	ls.  

	$ echo nonesuch /usr/* | xargs ls | head -3
	ls: nonesuch: No such file or directory
	/usr/X11R6:
	bin
	include
	xargs: ls terminated by SIGPIPE

	$ echo nonesuch /usr/* | xargs ls 2>/dev/null | head -3
	/usr/X11R6:
	bin
	include

	It is illustrative that although find and ls in the first example
	were terminated by SIGPIPE, only xargs complained.  Most netbsd
	utilities do not write to standard error for SIGPIPE.  That is by
	design.  The reader of a pipe cannot signal the writer that it has
	what it needs except by closing the pipe.  Therefore pipe closure
	is expected and not an error condition.  I have phrased this
	observation as 

		"Failure to read is not an error".  

	In the case of xargs, the crime is more eggregious.  xargs has no
	business remarking on the exit status of its client, period.  It
	does not matter what the signal is.  In a perfect world xargs would
	disappear.  In ours, it should hold its tongue.  

>How-To-Repeat:
	See above. 
>Fix:

Index: xargs.c
===================================================================
RCS file: /cvsroot/src/usr.bin/xargs/xargs.c,v
retrieving revision 1.20
diff -u -r1.20 xargs.c
--- xargs.c	17 Dec 2010 11:32:57 -0000	1.20
+++ xargs.c	14 Feb 2012 02:21:05 -0000
@@ -593,19 +593,11 @@
 		 */
 		if (WIFEXITED(status)) {
 			if (WEXITSTATUS (status) == 255) {
-				warnx ("%s exited with status 255", name);
 				exit(124);
 			} else if (WEXITSTATUS (status) != 0) {
 				rval = 123;
 			}
 		} else if (WIFSIGNALED (status)) {
-			if (WTERMSIG(status) < NSIG) {
-				warnx("%s terminated by SIG%s", name,
-				    sys_signame[WTERMSIG(status)]);
-			} else {
-				warnx("%s terminated by signal %d", name,
-				    WTERMSIG(status));
-			}
 			exit(125);
 		}
 	}

>Release-Note:

>Audit-Trail:
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/46021: xargs should keep still
Date: Tue, 14 Feb 2012 07:20:15 +0000

 On Tue, Feb 14, 2012 at 04:50:00AM +0000, jklowden@schemamania.org wrote:
  > 	xargs writes error messages to standard error if the utility 
  > 	it's executing exits with an error or on a signal.  E.g.:
  > 	
  > 	$ find /usr | xargs ls | head -1
  > 	/usr/X11R7/bin/X
  > 	xargs: ls terminated by SIGPIPE
  > 	
  > 	As the above example shows, the message is unwelcome and unnecessary.  
  > [...]

 I don't agree with your reasoning. Let's consider a similar and not
 uncommon case:

 First, create a program that always exits with a signal:

 	% ./die
 	Segmentation fault
 	% 

 Now let's place it in a pipeline:

 	% find /usr/include | ./die
 	Segmentation fault
 	% find /usr/include | ./die | cat
 	Segmentation fault 
 	% find /usr/include | xargs ./die 
 	xargs: ./die terminated by SIGSEGV
 	% find /usr/include | xargs ./die | cat
 	xargs: ./die terminated by SIGSEGV
 	% 

 Looks to me like the behavior is uniform across all four cases. xargs
 doesn't print the same message, but it's equivalent and printed in the
 same circumstances.

 With sh your argument is even weaker, as sh prints more:

 	$ find /usr/include | ./die
 	[1]   Broken pipe             find /usr/include |
 	      Segmentation fault      ./die
 	$ find /usr/include | ./die | cat
 	[1]   Broken pipe             find /usr/include |
 	      Segmentation fault      ./die |
 	      Done                    cat
 	$ find /usr/include | xargs ./die
 	xargs: ./die terminated by SIGSEGV
 	$ find /usr/include | xargs ./die | cat
 	xargs: ./die terminated by SIGSEGV
 	$ 

 zsh is similar to sh.

 Maybe SIGPIPE should be handled differently from other signals, but
 the change you've proposed is definitely wrong.

 -- 
 David A. Holland
 dholland@netbsd.org

From: jklowden@schemamania.org
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/46021: xargs should keep still
Date: Tue, 14 Feb 2012 11:59:17 -0500

 On Tue, Feb 14, 2012 at 07:25:02AM +0000, David Holland wrote:
 > The following reply was made to PR bin/46021; it has been noted by GNATS.
 > 
 > From: David Holland <dholland-bugs@netbsd.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: bin/46021: xargs should keep still
 > Date: Tue, 14 Feb 2012 07:20:15 +0000
 > 
 >  On Tue, Feb 14, 2012 at 04:50:00AM +0000, jklowden@schemamania.org wrote:
 >   > 	xargs writes error messages to standard error if the utility 
 >   > 	it's executing exits with an error or on a signal.  E.g.:
 >   > 	
 >   > 	$ find /usr | xargs ls | head -1
 >   > 	/usr/X11R7/bin/X
 >   > 	xargs: ls terminated by SIGPIPE
 >   > 	
 >   > 	As the above example shows, the message is unwelcome and unnecessary.  
 >   > [...]
 >  
 >  I don't agree with your reasoning. Let's consider a similar and not
 >  uncommon case:
 >  
 >  First, create a program that always exits with a signal:

 I have a better answer, see below, but I have to admit being amused at beginning
 with a "not uncommon" case by writing it.  :-)

 >  	% find /usr/include | ./die
 >  	Segmentation fault
 >  	% find /usr/include | ./die | cat
 >  	Segmentation fault 
 >  	% find /usr/include | xargs ./die 
 >  	xargs: ./die terminated by SIGSEGV
 >  	% find /usr/include | xargs ./die | cat
 >  	xargs: ./die terminated by SIGSEGV
 >  	% 
 >  
 >  xargs
 >  doesn't print the same message, but it's equivalent and printed in the
 >  same circumstances.

 No, definitely not.  xargs interpreted the signal and printed a message. 
 It swallowed the signal, preventing the shell from seeing it.  In
 the last example, xargs littered the terminal via standard error, but cat
 returned normally (or would, if it ever got started).  

 Instead, if you feel it's important to mimic a pipe as precisely as possible 
 (I don't.  I just don't want to see xargs messages, ever.) then xargs 
 should propagate the signal.  If it did that, all four of your examples would
 produce the same "Segmentation fault" message, and mine would produce none at
 all.  

 I am prepared to modify my patch in that way, but it will affect how 
 xargs returns when its child is signalled.  

 --jkl

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/46021: xargs should keep still
Date: Sun, 22 Apr 2012 20:02:37 +0000

 On Tue, Feb 14, 2012 at 05:00:07PM +0000, jklowden@schemamania.org wrote:
  >  >  	% find /usr/include | ./die
  >  >  	Segmentation fault
  >  >  	% find /usr/include | ./die | cat
  >  >  	Segmentation fault
  >  >  	% find /usr/include | xargs ./die 
  >  >  	xargs: ./die terminated by SIGSEGV
  >  >  	% find /usr/include | xargs ./die | cat
  >  >  	xargs: ./die terminated by SIGSEGV
  >  >  	% 
  >  >  
  >  >  xargs
  >  >  doesn't print the same message, but it's equivalent and printed in the
  >  >  same circumstances.
  >  
  >  No, definitely not.  xargs interpreted the signal and printed a message. 
  >  It swallowed the signal, preventing the shell from seeing it.

 So your objection is that what you'd like to see is:

 	% find /usr/include | ./die
 	Segmentation fault
 	% find /usr/include | ./die | cat
 	Segmentation fault
 	% find /usr/include | xargs ./die 
 	Segmentation fault
 	% find /usr/include | xargs ./die | cat
 	Segmentation fault
 	% 

 ?

 But your patch would produce

 	% find /usr/include | ./die
 	Segmentation fault
 	% find /usr/include | ./die | cat
 	Segmentation fault
 	% find /usr/include | xargs ./die 
 	% find /usr/include | xargs ./die | cat
 	% 

 and this really doesn't make much sense.

  > In the last example, xargs littered the terminal via standard
  > error, but cat returned normally (or would, if it ever got
  > started).

 And cat also returns normally in the second case, without xargs. I
 don't see what's different.

 I don't understand either what you're complaining about or what you're
 trying to accomplish.

 -- 
 David A. Holland
 dholland@netbsd.org

State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Thu, 03 Jan 2013 23:35:41 +0000
State-Changed-Why:
Please clarify... or perhaps this should just be closed.


From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dholland@NetBSD.org,
    jklowden@schemamania.org
Subject: re: bin/46021 (xargs should keep still)
Date: Fri, 04 Jan 2013 16:41:10 +1100

 reading the original description i think that what's happening
 is something slightly different.

 xargs isn't dying because it can't *read* anymore, it's dying
 because it can't *write* anymore, because eg, "head -3" exited,
 and then ls gets SIGPIPE when it tries to write more.


 i agree with dh - this isn't a bug.

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/46021 (xargs should keep still)
Date: Sun, 6 Jan 2013 23:20:14 +0000

 Please sent PR traffic to gnats-bugs.

    ------

 From: "James K. Lowden" <jklowden@schemamania.org>
 To: matthew green <mrg@eterna.com.au>
 Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dholland@NetBSD.org
 Subject: Re: bin/46021 (xargs should keep still)
 Date: Sat, 5 Jan 2013 12:27:58 -0500

 On Fri, 04 Jan 2013 16:41:10 +1100
 matthew green <mrg@eterna.com.au> wrote:

 > xargs isn't dying because it can't *read* anymore, it's dying
 > because it can't *write* anymore, because eg, "head -3" exited,
 > and then ls gets SIGPIPE when it tries to write more.

 Correct.  

 > i agree with dh - this isn't a bug.

 Well, it's certainly not a feature.  :-)  

 I assert no command-line utility should ever report a broken pipe error
 when writing to standard output.  Why?  

 1.  It's common for a reader, e.g. head(1), not to read everything.  

 2.  There is no mechanism, other than closing the pipe, for the reader
 to notify the writer that it's done.  

 Ergo, closing the pipe is *normal*, not an error.  

 Although I've never seen it articulated clearly, there is a long, long
 unix tradition of ignoring SIGPIPE on standard output, from arp(1) to
 zcat(1), not to mention ls(1) and cat(1).  The few that I've found that
 don't include xargs, pax, svn, and IIRC git. 

 In case the reader is concerned that a genuine error may be missed,
 bear in mind the utility is free to return a nonzero status and the
 shell remains free to interpret it.  Since time immemorial most shells
 I know of have treated the last element in the pipeline as the arbiter
 of the pipeline's success.  I don't see any better choice.  

 In the specific instance of xargs, the case is only more obvious
 because xargs is really only plumbing, no more significant on the
 command line than "|" or "``".  It's just an accident of history that
 xargs exists as a utility instead of being part of the shell syntax.  

 David left it to me to supply a patch, which I never did.  As I
 understood it at the time, he was skeptical but understood my reasoning
 and was willing to apply a patch of acceptable quality.  

 I agree with David that feedback is desirable.  We should have consensus
 on this issue before engaging the compiler.  

 --jkl

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/46021 (xargs should keep still)
Date: Sun, 6 Jan 2013 23:28:18 +0000

 On Sat, Jan 05, 2013 at 12:27:58PM -0500, James K. Lowden wrote:
  >  David left it to me to supply a patch, which I never did.  As I
  >  understood it at the time, he was skeptical but understood my reasoning
  >  and was willing to apply a patch of acceptable quality.  

 I was skeptical that there's a problem; what I wrote was:

  : I don't understand either what you're complaining about or what
  : you're trying to accomplish.

 -- 
 David A. Holland
 dholland@netbsd.org

State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 06 Jan 2013 23:34:05 +0000
State-Changed-Why:
Feedback received; submitter needs to figure out exactly what he's trying to
accomplish, and gather consensus, which is best done on a mailing list rather
than in the bug database.


>Unformatted:

Home	PR Database Search