NetBSD Problem Report #14087

Received: (qmail 14937 invoked from network); 28 Sep 2001 12:08:52 -0000
Message-Id: <20010928120548.6AFEE780C@mail.vaasje.org>
Date: Fri, 28 Sep 2001 14:05:48 +0200 (MEST)
From: fvdl@wasabisystems.com
Sender: fvdl@vaasje.org
Reply-To: fvdl@wasabisystems.com
To: gnats-bugs@gnats.netbsd.org
Subject: scp failure with NEW_PIPE
X-Send-Pr-Version: 3.95

>Number:         14087
>Category:       kern
>Synopsis:       scp failure with NEW_PIPE
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    jdolecek
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Sep 28 12:09:00 +0000 2001
>Closed-Date:    Sat Sep 29 13:49:10 +0000 2001
>Last-Modified:  Sat Sep 29 13:49:10 +0000 2001
>Originator:     Frank van der Linden
>Release:        NetBSD-current, 27 sept 2001
>Organization:
Noone, nowhere, nonono.
>Environment:
NetBSD k2 1.5Y NetBSD 1.5Y (K1) #0: Fri Sep 28 13:17:18 CEST 2001     fvdl@k2:/work/trees/current/sys/arch/i386/compile/K1 i386


>Description:
	Using scp to copy a file over a slowish network fails. The file
	in question is:

-rwxr-xr-x  1 fvdl  wheel  2247389 Sep 28 12:27 netbsd.gz

	The progress bar gets to around 80%, at which point it dies with
	'connection closed by remote host'. At the other end, the file
	then seems to be complete (the size looks ok), but an md5 check
	gives a different checksum.

	Things a re ok when I use a kernel from the same sources, compile
	without NEW_PIPE.

	What seems to happen is that for the NEW_PIPE case, the ssh forked
	by scp exits early somehow, causing a SIGPIPE.

	Ktrace output available on request.
>How-To-Repeat:
	scp netbsd.gz cvs.netbsd.org:
>Fix:
	unknown
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: kern-bug-people->jdolecek 
Responsible-Changed-By: jdolecek 
Responsible-Changed-When: Fri Sep 28 09:37:21 PDT 2001 
Responsible-Changed-Why:  
I'm looking at this. 

From: Jaromír Dolecek <jdolecek@netbsd.org>
To: Frank van der Linden <fvdl@wasabisystems.com>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/14087 scp failure with NEW_PIPE (Was: Re: pipe_direct and ERESTART
 (possibly other errors))
Date: Sat, 29 Sep 2001 13:01:39 +0200 (CEST)

 Hi Frank,

 Thanks for pointing out the dofilewrite() handling, I haven't realized
 that yet :)

 Frank van der Linden wrote:
 > However, it doesn't think so, because uio_resid wasn't updated
 > when the pipe code returned with ERESTART. So it looks like
 > pipe_direct_write must update resid in these cases.

 Indeed. It seems it's interrupted and _part_ of the loaned buffer
 was already read. The code in pipe_direct_write() only handles
 EOF in such cases. Even that was a bit incorrect.
 I've changed the pipe_direct_write() code to handle this for all
 errors, and return error to caller (i.e. pipe_write()), so that they
 recognize it and return. I'm not decided whether or not should I handle
 EPIPE specially; I think that the EPIPE/SIGPIPE should be probably
 returned to the process even if the write partially succeeded, i.e. I now think
 the former behaviour of NEW_PIPE was incorrect.

 I'm currently writing a test program exploiting the original
 bug, so that I'd be able to test locally.

 If you would like to test the patch I have so far, I'm appending
 it to this e-mail. It should work AFAICS, though I have not tested
 it myself yet.

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org>      http://www.ics.muni.cz/~dolecek/
 NetBSD - just plain best OS! -=*=- Got spare MCA cards or docs? Hand me them!

From: Jaromír Dolecek <jdolecek@netbsd.org>
To: fvdl@netbsd.org
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/14087: scp failure with NEW_PIPE
Date: Sat, 29 Sep 2001 14:24:24 +0200 (CEST)

 --ELM710439572-8405-0_
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain; charset=US-ASCII

 Hi Frank,
 I've verified the patch fixes the problem with my test program.
 I'm appending both the test program and the patch for GNATS reference.
 Let me know if the patch also fixes behaviour with scp, please.
 I'll add the test program as pipe regression test too after some cleanup.

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org>      http://www.ics.muni.cz/~dolecek/
 NetBSD - just plain best OS! -=*=- Got spare MCA cards or docs? Hand me them!

 --ELM710439572-8405-0_
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain; charset=ISO-8859-2
 Content-Disposition: attachment; filename=pipetest.c

 #include <sys/types.h>
 #include <unistd.h>
 #include <stdio.h>
 #include <stdlib.h>
 #include <poll.h>
 #include <sched.h>
 #include <signal.h>
 #include <sys/wait.h>

 pid_t pid;

 void
 sighand(int sig)
 {
 	if (sig == SIGALRM)
 		kill(pid, SIGINFO);
 }

 int
 main()
 {
 	int pp[2], st;
 	ssize_t sz, todo, done;
 	char *f;

 	todo = 2 * 1024 * 1024;
 	f = (char *) malloc(todo);

 	pipe(pp);

 	switch((pid = fork())) {
 	case 0: /* child */
 		close(pp[1]);

 		signal(SIGINFO, sighand);

 		done = read(pp[0], f, 128 * 1024);

 		pause();

 		while((sz = read(pp[0], f, 1024 * 1024)) > 0)
 			done += sz;

 		printf("done %d, should be %d\n", done, todo);
 		exit(0);

 		break;

 	case -1: /* error */
 		perror("fork");
 		_exit(1);
 		/* NOTREACHED */

 	default:
 		signal(SIGALRM, sighand);
 		close(pp[0]);

 		alarm(2);

 		while(todo > 0 && ((sz = write(pp[1], f, todo)) > 0))
 			todo -= sz;

 		close(pp[1]);
 		waitpid(pid, &st, 0);
 		break;
 	}

 	return (0);
 }

 --ELM710439572-8405-0_
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain; charset=ISO-8859-2
 Content-Disposition: attachment; filename=qw

 Index: kern/sys_pipe.c
 ===================================================================
 RCS file: /cvsroot/syssrc/sys/kern/sys_pipe.c,v
 retrieving revision 1.14
 diff -u -p -r1.14 sys_pipe.c
 --- kern/sys_pipe.c	2001/09/25 19:01:21	1.14
 +++ kern/sys_pipe.c	2001/09/29 12:22:26
 @@ -1086,23 +1086,22 @@ retry:
  		pipe_loan_free(wpipe);
  	pipeunlock(wpipe);

 -	if (error == EPIPE) {
 +	if (error) {
  		pipeselwakeup(wpipe, wpipe);

  		/*
 -		 * If anything was read from what we offered, return success
 -		 * and short write. We return EOF on next write(2).
 +		 * If nothing was read from what we offered, return error
 +		 * streight on. Otherwise update uio resid first. Caller
 +		 * would deal with the error condition - possibly finish
 +		 * the write(2) call returning short write.
  		 */
 -		if (wpipe->pipe_map.cnt < bcnt) {
 -			bcnt -= wpipe->pipe_map.cnt;
 -			error = 0;
 +		if (wpipe->pipe_map.cnt == bcnt) {
 +   error:
 +			wakeup(wpipe);
 +			return (error);
  		}
 -	}

 -	if (error) {
 -   error:
 -		wakeup(wpipe);
 -		return (error);
 +		bcnt -= wpipe->pipe_map.cnt;
  	}

  	uio->uio_resid  -= bcnt;
 @@ -1114,7 +1113,7 @@ retry:
  		uio->uio_iovcnt--;
  	}

 -	return (0);
 +	return (error);
  }
  #endif /* !PIPE_NODIRECT */
  #endif /* NetBSD */

 --ELM710439572-8405-0_--
State-Changed-From-To: open->closed 
State-Changed-By: jdolecek 
State-Changed-When: Sat Sep 29 06:48:40 PDT 2001 
State-Changed-Why:  
Fixed, thanks for help. 
>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.