NetBSD Problem Report #13333

Received: (qmail 21142 invoked from network); 28 Jun 2001 16:21:23 -0000
Message-Id: <200106281623.f5SGNfB00291@cordouan.antioche.eu.org>
Date: Thu, 28 Jun 2001 18:23:41 +0200 (MEST)
From: bouyer@antioche.lip6.fr (Manuel Bouyer)
Reply-To: bouyer@antioche.lip6.fr (Manuel Bouyer)
To: gnats-bugs@gnats.netbsd.org
Subject: bw_pipe hangs on vax with NEW_PIPE
X-Send-Pr-Version: 3.95

>Number:         13333
>Category:       port-vax
>Synopsis:       bw_pipe hangs on vax with NEW_PIPE
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    jdolecek
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jun 28 16:22:00 +0000 2001
>Closed-Date:    Sun Mar 24 19:09:21 +0000 2002
>Last-Modified:  Sun Mar 24 19:09:21 +0000 2002
>Originator:     Manuel Bouyer
>Release:        1.5W as of 2 days ago.
>Organization:
none	
>Environment:

System: NetBSD cordouan.antioche.eu.org 1.5U NetBSD 1.5U (CORDOUAN) #3: Tue Apr 24 08:15:07 MEST 2001 root@cordouan.antioche.eu.org:/home/sources-vax/src/tmp/sys/arch/vax/compile/CORDOUAN vax

MicroVAX 3300, 12Mb RAM, diskless

>Description:
	While benchmarking the NEW_PIPE code, the bw_pipe benchmark (from
	the hbench-1.0 suite) hung for transfersize of 262144 or 524288.
	Smaller transfer sizes (I tried 4096, 8192, 16384, 32768, 65536,
	131072) works fine. The old pipe code also works fine for all
	transfer sizes (4k -> 512k).
	The hung processes looks like (ps -axlw):
331 175  103   4  10  0  536  312 wait     S+   p0 0:01.43 /usr/pkg/bin/hbench/netbsd1.5-vax/bw_pipe 10 524288 
331 176  175  20  -6  0  536  132 pipdwt   S+   p0 0:01.42 /usr/pkg/bin/hbench/netbsd1.5-vax/bw_pipe 10 524288 


>How-To-Repeat:
	install hbench 1.0 from pub/NetBSD/packages/1.5/vax/All/hbench-1.0.tgz
	Run /usr/pkg/bin/hbench/netbsd1.5-vax/bw_pipe 10 262144
>Fix:
	Workaround: remove "options NEW_PIPE" and recompile kernel.
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: port-vax-maintainer->jdolecek 
Responsible-Changed-By: jdolecek 
Responsible-Changed-When: Thu Jun 28 11:56:43 PDT 2001 
Responsible-Changed-Why:  
Unless it's vax-specific, this is my responsibility. 

From: Jaromír Dolecek <jdolecek@NetBSD.org>
To: bouyer@antioche.lip6.fr
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-vax/13333: bw_pipe hangs on vax with NEW_PIPE
Date: Thu, 28 Jun 2001 23:16:27 +0200 (CEST)

 --ELM993762987-410-0_
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain; charset=US-ASCII

 Hi Manuel,
 can you please try out if appended patch helps? This makes
 the newpipe code less clever and return short write in some cases, however
 it should make hbench return reliably.

 The problem is that hbench expects some kind of behaviour. Even
 through I'm not convinced that the thing it expects is really right,
 some other applications may do same expectation. Thus, the correct
 fix is to make less assumptions in newpipe code :-/

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org>      http://www.ics.muni.cz/~dolecek/
 NetBSD - just plain best OS! -=*=- Got spare MCA cards or docs? Hand me them!

 --ELM993762987-410-0_
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain; charset=ISO-8859-2
 Content-Disposition: attachment; filename=newpipe.diff

 Index: kern/sys_pipe.c
 ===================================================================
 RCS file: /cvsroot/syssrc/sys/kern/sys_pipe.c,v
 retrieving revision 1.4
 diff -u -p -r1.4 sys_pipe.c
 --- kern/sys_pipe.c	2001/06/21 18:59:51	1.4
 +++ kern/sys_pipe.c	2001/06/28 20:59:15
 @@ -630,11 +630,6 @@ pipe_read(fp, offset, uio, cred, flags)
  			if (rpipe->pipe_map.cnt == 0) {
  				rpipe->pipe_state &= ~PIPE_DIRECTW;
  				wakeup(rpipe);
 -#ifdef __NetBSD__
 -				if (uio->uio_resid > 0 &&
 -				    (rpipe->pipe_state & PIPE_MOREW))
 -					goto waitformore;
 -#endif /* NetBSD */
  			}
  #endif
  		} else {
 @@ -667,9 +662,6 @@ pipe_read(fp, offset, uio, cred, flags)
  				break;
  			}

 -#if defined(__NetBSD__) && !defined(PIPE_NODIRECT)
 -		waitformore:
 -#endif
  			/*
  			 * Unlock the pipe buffer for our remaining processing.
  			 * We will either break out with an error or we will
 @@ -997,11 +989,11 @@ pipe_direct_write(wpipe, uio)
  	struct pipe *wpipe;
  	struct uio *uio;
  {
 -	int error, i, npages, j;
 +	int error, npages, j;
  	struct vm_page **res;
  	vaddr_t bbase, kva, base, bend;
  	vsize_t blen, bcnt;
 -	voff_t boff, bpos;
 +	voff_t bpos;
  	struct vm_map *wmap = &uio->uio_procp->p_vmspace->vm_map;
  retry:
  	while (wpipe->pipe_state & PIPE_DIRECTW) {
 @@ -1037,115 +1029,90 @@ retry:
  	}

  	/*
 -	 * For each iovec:
 -	 * 1. Loan the pages to kernel.
 -	 * 2. Set up pipe structures.
 -	 * 3. Wait until consumer reads it all or exits.
 +	 * Handle first iovec, first PIPE_CHUNK_SIZE bytes. Expect caller
 +	 * to deal with short write.
 +	 *
 +	 * Note: need to handle buffers not aligned to PAGE_SIZE.
  	 */
 -	boff = 0;
 -	for(i=0; i < uio->uio_iovcnt; ) {
 -		/*
 -		 * Note: need to handle buffers not aligned to PAGE_SIZE.
 -		 */
 -		bbase = (vaddr_t)uio->uio_iov[i].iov_base;
 -		base = trunc_page(bbase + boff);
 -		bend = round_page(bbase + uio->uio_iov[i].iov_len);
 -		blen = bend - base;
 -
 -		if (boff == 0)
 -			bpos = bbase % PAGE_SIZE;
 -		else
 -			bpos = 0;
 -
 -		if (blen > PIPE_DIRECT_CHUNK) {
 -			blen = PIPE_DIRECT_CHUNK;
 -			boff += PIPE_DIRECT_CHUNK;
 -			bend = base + blen;
 -			bcnt = PIPE_DIRECT_CHUNK - bpos;
 -			wpipe->pipe_state |= PIPE_MOREW;
 -		} else {
 -			if (boff == 0)
 -				bcnt = uio->uio_iov[i].iov_len;
 -			else
 -				bcnt = ((bbase % PAGE_SIZE) +
 -				    uio->uio_iov[i].iov_len) %PIPE_DIRECT_CHUNK;
 -			boff = 0;
 -			i++;
 -			wpipe->pipe_state &= ~PIPE_MOREW;
 -		}
 +	bbase = (vaddr_t)uio->uio_iov[0].iov_base;
 +	base = trunc_page(bbase);
 +	bend = round_page(bbase + uio->uio_iov[0].iov_len);
 +	blen = bend - base;
 +	bpos = bbase - base;
 +
 +	if (blen > PIPE_DIRECT_CHUNK) {
 +		blen = PIPE_DIRECT_CHUNK;
 +		bend = base + blen;
 +		bcnt = PIPE_DIRECT_CHUNK - bpos;
 +	} else
 +		bcnt = uio->uio_iov[0].iov_len;

 -		npages = blen / PAGE_SIZE;
 +	npages = blen / PAGE_SIZE;

 -		/*
 -		 * Free the old kva if we need more pages than we have
 -		 * allocated.
 -		 */
 -		if (wpipe->pipe_map.kva
 -		    && npages > wpipe->pipe_map.npages)
 -			pipe_loan_free(wpipe);
 -
 -		/* Allocate new kva. */
 -		if (!wpipe->pipe_map.kva) {
 -			if ((error = pipe_loan_alloc(wpipe,
 -					npages, blen)))
 -				goto error;
 -		}
 -		
 -		/* Loan the write buffer memory from writer process */
 -		res = wpipe->pipe_map.ms;
 -		error = uvm_loan(wmap, base, blen,
 -				(void **) res, UVM_LOAN_TOPAGE);
 -		if (error)
 -			goto cleanup;
 -		
 -		/* Enter the loaned pages to kva */
 -		kva = wpipe->pipe_map.kva;
 -		for(j=0; j < npages; j++, kva += PAGE_SIZE)
 -			pmap_enter(pmap_kernel(), kva, res[j]->phys_addr,
 -				VM_PROT_READ, 0);
 -
 -		wpipe->pipe_map.pos = bpos;
 -		wpipe->pipe_map.cnt = bcnt;
 -		wpipe->pipe_state |= PIPE_DIRECTW;
 -
 -		error = 0;
 -		while (!error && (wpipe->pipe_state & PIPE_DIRECTW)) {
 -			if (wpipe->pipe_state & PIPE_EOF) {
 -				error = EPIPE;
 -				break;
 -			}
 -			if (wpipe->pipe_state & PIPE_WANTR) {
 -				wpipe->pipe_state &= ~PIPE_WANTR;
 -				wakeup(wpipe);
 -			}
 -			pipeselwakeup(wpipe, wpipe);
 -			error = tsleep(wpipe, PRIBIO | PCATCH, "pipdwt", 0);
 -		}
 +	/*
 +	 * Free the old kva if we need more pages than we have
 +	 * allocated.
 +	 */
 +	if (wpipe->pipe_map.kva && npages > wpipe->pipe_map.npages)
 +		pipe_loan_free(wpipe);

 -	cleanup:
 -		pipelock(wpipe,0);
 -		if (amountpipekva > maxpipekva)
 -			pipe_loan_free(wpipe);
 -		uvm_unloanpage(res, npages);
 -		pipeunlock(wpipe);
 -		if (error) {
 -	error:
 -			/* XXX update uio ? */
 -			if (error == EPIPE)
 -				pipeselwakeup(wpipe, wpipe);
 +	/* Allocate new kva. */
 +	if (!wpipe->pipe_map.kva) {
 +		if ((error = pipe_loan_alloc(wpipe, npages, blen)))
 +			goto error;
 +	}
 +	
 +	/* Loan the write buffer memory from writer process */
 +	res = wpipe->pipe_map.ms;
 +	error = uvm_loan(wmap, base, blen, (void **) res, UVM_LOAN_TOPAGE);
 +	if (error)
 +		goto cleanup;
 +	
 +	/* Enter the loaned pages to kva */
 +	kva = wpipe->pipe_map.kva;
 +	for(j=0; j < npages; j++, kva += PAGE_SIZE)
 +		pmap_enter(pmap_kernel(), kva, res[j]->phys_addr,
 +			VM_PROT_READ, 0);

 -			wpipe->pipe_state &= ~PIPE_MOREW;
 -			goto error1;
 +	wpipe->pipe_map.pos = bpos;
 +	wpipe->pipe_map.cnt = bcnt;
 +	wpipe->pipe_state |= PIPE_DIRECTW;
 +
 +	error = 0;
 +	while (!error && (wpipe->pipe_state & PIPE_DIRECTW)) {
 +		if (wpipe->pipe_state & PIPE_EOF) {
 +			error = EPIPE;
 +			break;
 +		}
 +		if (wpipe->pipe_state & PIPE_WANTR) {
 +			wpipe->pipe_state &= ~PIPE_WANTR;
 +			wakeup(wpipe);
  		}
 -				
 -		uio->uio_offset += bcnt;
 -		uio->uio_resid  -= bcnt;
 +		pipeselwakeup(wpipe, wpipe);
 +		error = tsleep(wpipe, PRIBIO | PCATCH, "pipdwt", 0);
 +	}

 -	} /* for */
 +    cleanup:
 +	pipelock(wpipe,0);
 +	if (error || amountpipekva > maxpipekva)
 +		pipe_loan_free(wpipe);
 +	uvm_unloanpage(res, npages);
 +	pipeunlock(wpipe);
 +	if (error) {
 +    error:
 +		/* XXX update uio ? */
 +		if (error == EPIPE)
 +			pipeselwakeup(wpipe, wpipe);

 -	return (error);
 +		goto error1;
 +	}
 +			
 +	uio->uio_offset += bcnt;
 +	uio->uio_resid  -= bcnt;

 -error1:
 +	return (0);
 +
 +   error1:
  	wakeup(wpipe);
  	return (error);
  }
 @@ -1241,9 +1208,13 @@ pipe_write(fp, offset, uio, cred, flags)
  		    (fp->f_flag & FNONBLOCK) == 0 &&
  		    (wpipe->pipe_map.kva || (amountpipekva < limitpipekva))) {
  			error = pipe_direct_write(wpipe, uio);
 -			if (error)
 -				break;
 -			continue;
 +
 +			/*
 +			 * We either errorred, wrote whole buffer, or
 +			 * wrote part of buffer. In either case, we want
 +			 * to return now.
 +			 */
 +			break;
  		}
  #endif /* PIPE_NODIRECT */


 --ELM993762987-410-0_--
State-Changed-From-To: open->feedback 
State-Changed-By: jdolecek 
State-Changed-When: Fri Jun 29 02:50:57 PDT 2001 
State-Changed-Why:  
Please let me know if the patch helped. 

From: Jaromír Dolecek <jdolecek@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:  
Subject: Re: port-vax/13333: bw_pipe hangs on vax with NEW_PIPE
Date: Wed, 4 Jul 2001 12:31:17 +0200 (CEST)

 Hi Manuel,
 could you please let me know if -current (with rev. 1.5 of kern/sys_pipe.c)
 still has the problem?

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org>      http://www.ics.muni.cz/~dolecek/
 NetBSD - just plain best OS! -=*=- Got spare MCA cards or docs? Hand me them!

From: Jaromír Dolecek <jdolecek@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:  
Subject: Re: port-vax/13333: bw_pipe hangs on vax with NEW_PIPE
Date: Thu, 23 Aug 2001 22:14:02 +0200 (CEST)

 Hi,
 do you still see this bug?

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org>      http://www.ics.muni.cz/~dolecek/
 NetBSD - just plain best OS! -=*=- Got spare MCA cards or docs? Hand me them!
State-Changed-From-To: feedback->closed 
State-Changed-By: jdolecek 
State-Changed-When: Tue Sep 25 12:04:01 PDT 2001 
State-Changed-Why:  
Presumed fixed. If you feel it's incorrect, contact me to reopen 
the PR. Thanks for report. 
State-Changed-From-To: closed->feedback 
State-Changed-By: jdolecek 
State-Changed-When: Tue Sep 25 14:00:13 PDT 2001 
State-Changed-Why:  
Awaiting confirmation this is really fixed (or not). 

From: Jaromír Dolecek <jdolecek@netbsd.org>
To: Manuel Bouyer <bouyer@antioche.eu.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: port-vax/13333
Date: Tue, 25 Sep 2001 22:59:54 +0200 (CEST)

 OK, thanks. I'd set the PR back to feedback then.

 Jaromir

 Manuel Bouyer wrote:
 > Well, my vax is in bad shape rigth now ... I'll try again when I can get it
 > past single-user.
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org>      http://www.ics.muni.cz/~dolecek/
 NetBSD - just plain best OS! -=*=- Got spare MCA cards or docs? Hand me them!
State-Changed-From-To: feedback->closed 
State-Changed-By: jdolecek 
State-Changed-When: Sat Nov 3 03:08:12 PST 2001 
State-Changed-Why:  
Presumed fixed, closed due to lack of feedback. If you find NEW_PIPE 
nonworking on vax, let me know and I'd reopen this PR. Thanks. 
State-Changed-From-To: closed->feedback 
State-Changed-By: jdolecek 
State-Changed-When: Sat Nov 17 13:47:31 PST 2001 
State-Changed-Why:  
OK, Manuel wants to test this, but can't boot his vax multiuser 
due to insufficient VM (Ragge is supposed to be looking on this). 
So reopen this, until reported fixed. 

From: Jaromir Dolecek <jdolecek@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc:  
Subject: Re: port-vax/13333: bw_pipe hangs on vax with NEW_PIPE
Date: Mon, 24 Dec 2001 00:19:19 +0100 (CET)

 Hi Manuel,
 got your vax into shape to test this again yet?

 Jaromir

From: Manuel Bouyer <bouyer@antioche.lip6.fr>
To: gnats-bugs@gnats.netbsd.org
Cc: jdolecek@NetBSD.org
Subject: Re: port-vax/13333
Date: Thu, 10 Jan 2002 16:10:12 +0100

 Hi,
 I tried it again, and it still hangs. But it may be related to KVM resource
 shortage I'm otherwise getting on the machine: with more than 10-15
 processes running I can't start new processes or open new files ...

 --
 Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
 --

From: Jaromir Dolecek <jdolecek@NetBSD.org>
To: Manuel Bouyer <bouyer@antioche.lip6.fr>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-vax/13333
Date: Thu, 10 Jan 2002 16:17:49 +0100 (CET)

 Does it hang with hbench compiled from pkgsrc after 2001/09/22,
 particularily one with patch-al patch to src/bw_pipe.c?
 There was a bug in hbench related to this.

 Jaromir

 Manuel Bouyer wrote:
 > Hi,
 > I tried it again, and it still hangs. But it may be related to KVM resource
 > shortage I'm otherwise getting on the machine: with more than 10-15
 > processes running I can't start new processes or open new files ...
 > 
 > --
 > Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
 > --
 > 


 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org> http://www.NetBSD.org/Ports/i386/ps2.html
 -=  Those who would give up liberty for a little temporary safety deserve  =-
 -=  neither liberty nor safety, and will lose both.  -- Benjamin Franklin  =-

From: Manuel Bouyer <bouyer@antioche.lip6.fr>
To: Jaromir Dolecek <jdolecek@NetBSD.org>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: port-vax/13333
Date: Thu, 10 Jan 2002 19:07:43 +0100

 On Thu, Jan 10, 2002 at 04:17:49PM +0100, Jaromir Dolecek wrote:
 > Does it hang with hbench compiled from pkgsrc after 2001/09/22,
 > particularily one with patch-al patch to src/bw_pipe.c?
 > There was a bug in hbench related to this.

 I tried with a just-built hbench, and it still hang.

 --
 Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
 --

From: Manuel Bouyer <bouyer@antioche.lip6.fr>
To: gnats-bugs@gnats.netbsd.org
Cc:  
Subject: Re: port-vax/13333
Date: Tue, 5 Mar 2002 16:49:38 +0100

 Hi,
 about the bw_pipe hangs on vax:
 With recent ragge's fixes I've been able to get a working kernel on my
 vax again. This improved things for bw_pipe: now the 256K test passes, but
 the 512k still hangs. However now it's clear that it hangs on KVM resources
 shortage:
 331 16334 16241  12  -6   0  536  296 piperd   IW   p1 0:00.00 /usr/pkg/bin/hbench/netbsd1.5V-vax/bw_pipe 10 524288 
 331 16335 16334  17 -18   0  536  140 vallocwa DW   p1 0:00.00 /usr/pkg/bin/hbench/netbsd1.5V-vax/bw_pipe 10 524288 

 vallocwa is in uvm_km.c.

 --
 Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
 --

From: Jaromir Dolecek <jdolecek@netbsd.org>
To: Manuel Bouyer <bouyer@antioche.lip6.fr>
Cc: gnats-bugs@netbsd.org
Subject: Re: port-vax/13333
Date: Tue, 5 Mar 2002 20:43:41 +0100 (CET)

 Manuel,
 can you try to add e.g.

 options	PIPE_DIRECT_CHUNK=65536

 into your vax kernel config, remove sys_pipe.o and recompile?

 The default throttle value is 1MB, perhaps this is too much for
 vax.

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org> http://www.NetBSD.org/Ports/i386/ps2.html
 -=- We should be mindful of the potential goal, but as the tantric    -=-
 -=- Buddhist masters say, ``You may notice during meditation that you -=-
 -=- sometimes levitate or glow.   Do not let this distract you.''     -=-

From: Jaromir Dolecek <jdolecek@NetBSD.org>
To: Christos Zoulas <christos@zoulas.com>
Cc: Manuel Bouyer <bouyer@antioche.lip6.fr>, gnats-bugs@NetBSD.org
Subject: Re: port-vax/13333
Date: Tue, 5 Mar 2002 20:56:58 +0100 (CET)

 Christos Zoulas wrote:
 > We probably need to add some resource throttling to the new pipe stuff.

 There is (see the e-mail I sent Manuel). The default of
 PIPE_DIRECT_CHUNK should be probably put to somewhere around 64 or 128KB
 for vax. I wonder if arm26 wouldn't have same kva shortage problem.

 Jaromir
 -- 
 Jaromir Dolecek <jdolecek@NetBSD.org> http://www.NetBSD.org/Ports/i386/ps2.html
 -=- We should be mindful of the potential goal, but as the tantric    -=-
 -=- Buddhist masters say, ``You may notice during meditation that you -=-
 -=- sometimes levitate or glow.   Do not let this distract you.''     -=-

From: Manuel Bouyer <bouyer@antioche.lip6.fr>
To: Jaromir Dolecek <jdolecek@netbsd.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: port-vax/13333
Date: Wed, 6 Mar 2002 12:19:32 +0100

 On Tue, Mar 05, 2002 at 08:43:41PM +0100, Jaromir Dolecek wrote:
 > Manuel,
 > can you try to add e.g.
 > 
 > options	PIPE_DIRECT_CHUNK=65536
 > 
 > into your vax kernel config, remove sys_pipe.o and recompile?
 > 
 > The default throttle value is 1MB, perhaps this is too much for
 > vax.

 Yes, with 64k it works fine for me. Let me know if you want to test
 other value, but as changing the value in the config file requieres 40mn
 to give a kernel, I prefer test only usefull values :)

 --
 Manuel Bouyer, LIP6, Universite Paris VI.           Manuel.Bouyer@lip6.fr
 --
State-Changed-From-To: feedback->closed 
State-Changed-By: jdolecek 
State-Changed-When: Sun Mar 24 11:08:41 PST 2002 
State-Changed-Why:  
Fixed, with rev. 1.49 of sys/arch/vax/include/param.h. 
Thanks for report and testing. 
>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.