NetBSD Problem Report #15364

Received: (qmail 24774 invoked from network); 25 Jan 2002 22:19:52 -0000
Message-Id: <20020125221918.7051.qmail@h187n2fls4o804.telia.com>
Date: 25 Jan 2002 22:19:18 -0000
From: Tomas Svensson <tsn@gbdev.net>
Reply-To: Tomas Svensson <tsn@gbdev.net>
To: gnats-bugs@gnats.netbsd.org
Subject: panic: softdep_pageiodone
X-Send-Pr-Version: 3.95

>Number:         15364
>Category:       kern
>Synopsis:       panic: softdep_pageiodone
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    chs
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jan 25 22:20:01 +0000 2002
>Closed-Date:    Sat Oct 01 15:20:48 +0000 2005
>Last-Modified:  Wed May 13 13:08:24 +0000 2015
>Originator:     
>Release:        NetBSD 1.5ZA
>Organization:
>Environment:
		NetBSD 1.5ZA i386, kernel made from CVS sources January 14.
>Description:

panic: softdep_pageiodone: resid < 0, vp 0xc6ad59d0 lbn 0x8 pcbp 0x6add410
Stopped in pid 7 (aiodoned) at cpu_Debugger+0x4:     leave

db> trace

cpu_Debugger(c6ad59d0,10000,0,3,29a) at cpu_Debugger+0x4
panic(c0227bc0,c6ad59d0,8,c6add410,0) at panic+0xad
softdep_pageiodone(c04620d0,c0396200,c032d1f4,c0282720) at softdep_pageiodone+0x147
uvm_aio_aiodone(c04620d0,c0462548,ffffffff,c01bb496,c686fac8) at uvm_aio_aiodone+0x77
uvm_aiodone_daemon(c686fac8) at uvm_aiodone_daemon+0x98

db> reboot
syncing disks... panic: softdep_lock: locking against myself

>How-To-Repeat:
	Was transferring one file to the host, one file from it simultaneously 
	over 100baseTX LAN.
>Fix:
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: chs 
State-Changed-When: Sun May 5 10:03:52 PDT 2002 
State-Changed-Why:  
fixed with rev. 1.49 of ffs_vnops.c. 


Responsible-Changed-From-To: kern-bug-people->chs 
Responsible-Changed-By: chs 
Responsible-Changed-When: Sun May 5 10:03:52 PDT 2002 
Responsible-Changed-Why:  
my bug. 
State-Changed-From-To: closed->open
State-Changed-By: yamt@netbsd.org
State-Changed-When: Wed, 24 Aug 2005 08:08:41 +0000
State-Changed-Why:
i saw this panic today.


From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: gnats-bugs@NetBSD.org
Cc: chs@netbsd.org
Subject: Re: kern/15364
Date: Thu, 25 Aug 2005 11:44:49 +0900

 > Synopsis: panic: softdep_pageiodone

 here's a simple test code to trigger the bug.

 	% cc a.c
 	% ./a.out > testfile

 YAMAMOTO Takashi


 #include <sys/mman.h>

 #include <string.h>
 #include <unistd.h>

 #define	MAXPHYS	65536
 #define	BLKSIZ	16384
 #define	FRAGSIZ	2048
 #define	PAGE_SIZE 4096

 /* assuming FRAGSIZ < PAGE_SIZE < BLKSIZ < MAXPHYS */

 int
 main()
 {
 	char buf[BLKSIZ];
 	char *cp;

 	memset(buf, 0xaa, sizeof(buf));
 	pwrite(1, buf, 1, MAXPHYS - 1);
 	cp = mmap(0, MAXPHYS*16, PROT_WRITE, MAP_FILE|MAP_SHARED, 1, 0);

 	memset(cp, 0xbb, BLKSIZ);
 	fsync(1);

 	memset(cp + PAGE_SIZE, 0xee, MAXPHYS - PAGE_SIZE);
 	/* create a page dep, b_resid == PAGE_SIZE + FRAGSIZE */
 	pwrite(1, buf, PAGE_SIZE + FRAGSIZ, MAXPHYS);

 	/* write out the page at the offset MAXPHYS */
 	msync(cp + PAGE_SIZE, 1, MS_SYNC);
 	/* now b_resid == FRAGSIZ */

 	/* re-dirty and write the page, b_resid goes to negative. */
 	pwrite(1, buf, PAGE_SIZE + FRAGSIZ, MAXPHYS);
 	fsync(1);
 }

From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Sun, 28 Aug 2005 08:00:27 -0700

 --ew6BAiZeqk4r7MaW
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline

 ah, it's the write clustering that's causing the problem.
 the attached patch forces the putpages write clustering to obey
 the same block-alignment rule that the getpages path uses, so that
 pages involved in a softdep dependency will always be written together.
 do you see any problem with this?

 -Chuck

 --ew6BAiZeqk4r7MaW
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: attachment; filename="diff.softdep"

 Index: src/sys/sys/mount.h
 ===================================================================
 RCS file: /cvsroot/src/sys/sys/mount.h,v
 retrieving revision 1.129
 diff -u -p -r1.129 mount.h
 --- src/sys/sys/mount.h	29 May 2005 21:19:41 -0000	1.129
 +++ src/sys/sys/mount.h	28 Aug 2005 14:55:26 -0000
 @@ -126,6 +126,7 @@ struct mount {
  	int		mnt_iflag;		/* internal flags */
  	int		mnt_fs_bshift;		/* offset shift for lblkno */
  	int		mnt_dev_bshift;		/* shift for device sectors */
 +	int		mnt_clusteralign;	/* cluster alignment needed */
  	struct statvfs	mnt_stat;		/* cache of filesystem stats */
  	void		*mnt_data;		/* private data */
  	int		mnt_wcnt;		/* count of vfs_busy waiters */
 Index: src/sys/kern/vfs_syscalls.c
 ===================================================================
 RCS file: /cvsroot/src/sys/kern/vfs_syscalls.c,v
 retrieving revision 1.229
 diff -u -p -r1.229 vfs_syscalls.c
 --- src/sys/kern/vfs_syscalls.c	19 Aug 2005 12:30:02 -0000	1.229
 +++ src/sys/kern/vfs_syscalls.c	28 Aug 2005 14:55:28 -0000
 @@ -314,6 +314,7 @@ sys_mount(struct lwp *l, void *v, regist
  	mp->mnt_stat.f_owner = p->p_ucred->cr_uid;
  	mp->mnt_unmounter = NULL;
  	mp->mnt_leaf = mp;
 +	mp->mnt_clusteralign = PAGE_SIZE;

  	/*
  	 * The underlying file system may refuse the mount for
 Index: src/sys/ufs/ffs/ffs_vfsops.c
 ===================================================================
 RCS file: /cvsroot/src/sys/ufs/ffs/ffs_vfsops.c,v
 retrieving revision 1.169
 diff -u -p -r1.169 ffs_vfsops.c
 --- src/sys/ufs/ffs/ffs_vfsops.c	23 Aug 2005 08:05:13 -0000	1.169
 +++ src/sys/ufs/ffs/ffs_vfsops.c	28 Aug 2005 14:55:29 -0000
 @@ -450,6 +450,9 @@ ffs_mount(struct mount *mp, const char *
  		}
  		(void) ffs_cgupdate(ump, MNT_WAIT);
  	}
 +	if (mp->mnt_flag & MNT_SOFTDEP) {
 +		mp->mnt_clusteralign = MAX(PAGE_SIZE, fs->fs_bsize);
 +	}
  	return (error);

  fail:
 Index: src/sys/miscfs/genfs/genfs_vnops.c
 ===================================================================
 RCS file: /cvsroot/src/sys/miscfs/genfs/genfs_vnops.c,v
 retrieving revision 1.104
 diff -u -p -r1.104 genfs_vnops.c
 --- src/sys/miscfs/genfs/genfs_vnops.c	26 Jul 2005 08:06:29 -0000	1.104
 +++ src/sys/miscfs/genfs/genfs_vnops.c	28 Aug 2005 14:55:30 -0000
 @@ -1095,24 +1095,21 @@ genfs_putpages(void *v)
  		voff_t a_offhi;
  		int a_flags;
  	} */ *ap = v;
 +	const int maxpages = MAXPHYS >> PAGE_SHIFT;
  	struct vnode *vp = ap->a_vp;
  	struct uvm_object *uobj = &vp->v_uobj;
  	struct simplelock *slock = &uobj->vmobjlock;
 +	struct lwp *l = curlwp ? curlwp : &lwp0;
 +	struct genfs_node *gp = VTOG(vp);
 +	struct vm_page *pgs[maxpages], *pg, *nextpg, *tpg, curmp, endmp;
  	off_t startoff = ap->a_offlo;
  	off_t endoff = ap->a_offhi;
 -	off_t off;
 +	off_t off, aofflo, aoffhi;
  	int flags = ap->a_flags;
 -	/* Even for strange MAXPHYS, the shift rounds down to a page */
 -	const int maxpages = MAXPHYS >> PAGE_SHIFT;
 -	int i, s, error, npages, nback;
 -	int freeflag;
 -	struct vm_page *pgs[maxpages], *pg, *nextpg, *tpg, curmp, endmp;
 +	int i, s, error, npages, nbytes, nback, freeflag, dirtygen, amask;
  	boolean_t wasclean, by_list, needs_clean, yld;
  	boolean_t async = (flags & PGO_SYNCIO) == 0;
  	boolean_t pagedaemon = curproc == uvm.pagedaemon_proc;
 -	struct lwp *l = curlwp ? curlwp : &lwp0;
 -	struct genfs_node *gp = VTOG(vp);
 -	int dirtygen;
  	boolean_t modified = FALSE;
  	boolean_t cleanall;

 @@ -1183,6 +1180,11 @@ genfs_putpages(void *v)
  	    startoff == 0 && endoff == trunc_page(LLONG_MAX) &&
  	    (vp->v_flag & VONWORKLST) != 0;
  	dirtygen = gp->g_dirtygen;
 +	if (vp->v_type == VREG) {
 +		amask = vp->v_mount->mnt_clusteralign - 1;
 +	} else {
 +		amask = PAGE_MASK;
 +	}
  	freeflag = pagedaemon ? PG_PAGEOUT : PG_RELEASED;
  	curmp.uobject = uobj;
  	curmp.offset = (voff_t)-1;
 @@ -1326,11 +1328,16 @@ genfs_putpages(void *v)
  			UVM_PAGE_OWN(pg, "genfs_putpages");

  			/*
 -			 * first look backward.
 +			 * first look backward, making sure the range
 +			 * is aligned as needed.
  			 */

 -			npages = MIN(maxpages >> 1, off >> PAGE_SHIFT);
 +			nbytes = MIN(maxpages << (PAGE_SHIFT - 1), off);
 +			aofflo = (off - PAGE_SIZE - nbytes + amask) & ~amask;
 +			aoffhi = MAX(off - PAGE_SIZE, 0);
 +			npages = MIN(aoffhi - aofflo, off) >> PAGE_SHIFT;
  			nback = npages;
 +
  			uvn_findpages(uobj, off - PAGE_SIZE, &nback, &pgs[0],
  			    UFP_NOWAIT|UFP_NOALLOC|UFP_DIRTYONLY|UFP_BACKWARD);
  			if (nback) {
 @@ -1352,10 +1359,14 @@ genfs_putpages(void *v)

  			/*
  			 * then look forward to fill in the remaining space in
 -			 * the array of pages.
 +			 * the array of pages, again aligning the range.
  			 */

 -			npages = maxpages - nback - 1;
 +			nbytes = (maxpages - nback - 1) << PAGE_SHIFT;
 +			aofflo = off + PAGE_SIZE;
 +			aoffhi = (aofflo + nbytes) & ~amask;
 +			npages = (aoffhi - aofflo) >> PAGE_SHIFT;
 +
  			uvn_findpages(uobj, off + PAGE_SIZE, &npages,
  			    &pgs[nback + 1],
  			    UFP_NOWAIT|UFP_NOALLOC|UFP_DIRTYONLY);

 --ew6BAiZeqk4r7MaW--

From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Mon, 29 Aug 2005 09:18:36 +0900

 > ah, it's the write clustering that's causing the problem.
 > the attached patch forces the putpages write clustering to obey
 > the same block-alignment rule that the getpages path uses, so that
 > pages involved in a softdep dependency will always be written together.
 > do you see any problem with this?
 > 
 > -Chuck

 i don't think that block-alignment is really needed here.  (unlike lfs.)
 isn't it better to have a bitmap in softdep "pcbp" to track which
 pages in a block have been written out?

 YAMAMOTO Takashi

From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Mon, 29 Aug 2005 10:01:34 -0700

 On Mon, Aug 29, 2005 at 09:18:36AM +0900, YAMAMOTO Takashi wrote:
 > > ah, it's the write clustering that's causing the problem.
 > > the attached patch forces the putpages write clustering to obey
 > > the same block-alignment rule that the getpages path uses, so that
 > > pages involved in a softdep dependency will always be written together.
 > > do you see any problem with this?
 > > 
 > > -Chuck
 > 
 > i don't think that block-alignment is really needed here.  (unlike lfs.)
 > isn't it better to have a bitmap in softdep "pcbp" to track which
 > pages in a block have been written out?

 well, I was thinking it would be good to have the property that a given block
 is written in one I/O, since that's what the softdep code was designed for,
 and it'd be safer to maintain the assumptions that the softdep code makes.
 but you're probably right that it's not really required.  would you like to
 implement your bitmap idea?

 -Chuck

From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Tue, 30 Aug 2005 10:53:22 +0900

 > well, I was thinking it would be good to have the property that a given block
 > is written in one I/O, since that's what the softdep code was designed for,
 > and it'd be safer to maintain the assumptions that the softdep code makes.
 > but you're probably right that it's not really required.

 while block-alignment might have good aspects,
 it can confuse pagedaemon and increase number of i/o
 if both of active and inactive pages are in a block.
 i'm not sure if it affects real-world performance, tho.

 > would you like to
 > implement your bitmap idea?

 i will.

 YAMAMOTO Takashi

From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Tue, 30 Aug 2005 09:25:34 -0700

 On Tue, Aug 30, 2005 at 10:53:22AM +0900, YAMAMOTO Takashi wrote:
 > > well, I was thinking it would be good to have the property that a given block
 > > is written in one I/O, since that's what the softdep code was designed for,
 > > and it'd be safer to maintain the assumptions that the softdep code makes.
 > > but you're probably right that it's not really required.
 > 
 > while block-alignment might have good aspects,
 > it can confuse pagedaemon and increase number of i/o
 > if both of active and inactive pages are in a block.

 could you give some details on how the pagedaemon would be confused by this?
 the pagedaemon will do the write-clustering regardless of what range is
 considered for clustering, so how does the block-alignment make things worse?


 > i'm not sure if it affects real-world performance, tho.

 I doubt it would make any difference.  in the common case of sequential writes,
 the ranges sent to VOP_PUTPAGES (by VOP_WRITE) are block-aligned anyway.
 only for random unaligned writes that happen to be close to MAXPHYS in size
 would we create more I/O requests than before.

 -Chuck

From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Wed, 31 Aug 2005 05:29:45 +0900

 > could you give some details on how the pagedaemon would be confused by this?

 pagedaemon frees the whole block when one of the corresponding pages
 happens to be inactive.

 YAMAMOTO Takashi

From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Wed, 31 Aug 2005 05:30:58 +0900

 --NextPart-20050831053047-0408200
 Content-Type: Text/Plain; charset=us-ascii

 > > would you like to
 > > implement your bitmap idea?
 > 
 > i will.

 here is.

 YAMAMOTO Takashi

 --NextPart-20050831053047-0408200
 Content-Type: Text/Plain; charset=us-ascii
 Content-Disposition: attachment; filename="a.diff"

 Index: ffs_softdep.c
 ===================================================================
 --- ffs_softdep.c	(revision 1330)
 +++ ffs_softdep.c	(working copy)
 @@ -197,6 +197,9 @@ void softdep_pageiodone __P((struct buf 
  void softdep_flush_vnode __P((struct vnode *, daddr_t));
  static void softdep_trackbufs(struct inode *, int, boolean_t);

 +#define	PCBP_BITMAP(off, size) \
 +	(((1 << howmany((size), PAGE_SIZE)) - 1) << ((off) >> PAGE_SHIFT))
 +
  /*
   * Exported softdep operations.
   */
 @@ -5804,9 +5807,13 @@ softdep_setup_pagecache(ip, lbn, size)
  		LIST_INSERT_HEAD(&pcbphashhead[PCBPHASH(vp, lbn)], bp, b_hash);
  		LIST_INSERT_HEAD(&ip->i_pcbufhd, bp, b_vnbufs);
  	}
 -	bp->b_bcount = bp->b_resid = size;
 +	bp->b_bcount = size;
 +	KASSERT(size <= PAGE_SIZE * sizeof(bp->b_resid) * CHAR_BIT);
 +	bp->b_resid = PCBP_BITMAP(0, size);
  	UVMHIST_LOG(ubchist, "vp = %p, lbn = %ld, "
 -	    "bp = %p, bcount = resid = %ld", vp, lbn, bp, size);
 +	    "bp = %p, bcount = %ld", vp, lbn, bp, size);
 +	UVMHIST_LOG(ubchist, "b_resid = %ld",
 +	    bp->b_resid, 0, 0, 0);
  	return bp;
  }

 @@ -5933,7 +5940,7 @@ softdep_pageiodone1(bp)
  	struct worklist *wk;
  	daddr_t lbn;
  	voff_t off;
 -	long iosize = bp->b_bcount;
 +	int iosize = bp->b_bcount;
  	int size, asize, bshift, bsize;
  	int i;
  	UVMHIST_FUNC("softdep_pageiodone"); UVMHIST_CALLED(ubchist);
 @@ -5946,12 +5953,14 @@ softdep_pageiodone1(bp)
  	for (i = 0; i < npages; i++) {
  		pg = uvm_pageratop((vaddr_t)bp->b_data + (i << PAGE_SHIFT));
  		if (pg == NULL) {
 -			continue;
 +			panic("%s: no page", __func__);
  		}

  		for (off = pg->offset;
  		     off < pg->offset + PAGE_SIZE;
  		     off += bsize) {
 +			int pgmask;
 +
  			size = MIN(asize, iosize);
  			iosize -= size;
  			lbn = off >> bshift;
 @@ -5968,14 +5977,14 @@ softdep_pageiodone1(bp)
  			UVMHIST_LOG(ubchist,
  			    "pcbp %p iosize %ld, size %d, asize %d",
  			    pcbp, iosize, size, asize);
 -			pcbp->b_resid -= size;
 -			if (pcbp->b_resid < 0) {
 -				panic("softdep_pageiodone: "
 -				    "resid < 0, vp %p lbn 0x%" PRIx64 " pcbp %p"
 -				    " iosize %ld, size %d, asize %d, bsize %d",
 -				    vp, lbn, pcbp, iosize, size, asize, bsize);
 +			pgmask = PCBP_BITMAP(off & (bsize - 1), size);
 +			if ((~pcbp->b_resid & pgmask) != 0) {
 +				UVMHIST_LOG(ubchist,
 +				    "multiple write resid %lx, pgmask %lx",
 +				    pcbp->b_resid, pgmask, 0, 0);
  			}
 -			if (pcbp->b_resid > 0) {
 +			pcbp->b_resid &= ~pgmask;
 +			if (pcbp->b_resid != 0) {
  				continue;
  			}

 Index: ffs_vnops.c
 ===================================================================
 --- ffs_vnops.c	(revision 1276)
 +++ ffs_vnops.c	(working copy)
 @@ -116,7 +116,7 @@ const struct vnodeopv_entry_desc ffs_vno
  	{ &vop_update_desc, ffs_update },		/* update */
  	{ &vop_bwrite_desc, vn_bwrite },		/* bwrite */
  	{ &vop_getpages_desc, ffs_getpages },		/* getpages */
 -	{ &vop_putpages_desc, ffs_putpages },		/* putpages */
 +	{ &vop_putpages_desc, genfs_putpages },		/* putpages */
  	{ NULL, NULL }
  };
  const struct vnodeopv_desc ffs_vnodeop_opv_desc =
 @@ -533,58 +533,6 @@ ffs_getpages(void *v)
  		return EINVAL;
  	}
  	return genfs_getpages(v);
 -}
 -
 -int
 -ffs_putpages(void *v)
 -{
 -	struct vop_putpages_args /* {
 -		struct vnode *a_vp;
 -		voff_t a_offlo;
 -		voff_t a_offhi;
 -		int a_flags;
 -	} */ *ap = v;
 -	struct vnode *vp = ap->a_vp;
 -	struct uvm_object *uobj = &vp->v_uobj;
 -	struct inode *ip = VTOI(vp);
 -	struct fs *fs = ip->i_fs;
 -	struct vm_page *pg;
 -	off_t off;
 -
 -	if (!DOINGSOFTDEP(vp) || (ap->a_flags & PGO_CLEANIT) == 0) {
 -		return genfs_putpages(v);
 -	}
 -
 -	/*
 -	 * for softdep files, force the pages in a block to be written together.
 -	 * if we're the pagedaemon and we would have to wait for other pages,
 -	 * just fail the request.  the pagedaemon will pick a different page.
 -	 */
 -
 -	ap->a_offlo &= ~fs->fs_qbmask;
 -	ap->a_offhi = blkroundup(fs, ap->a_offhi);
 -	if (curproc == uvm.pagedaemon_proc) {
 -		for (off = ap->a_offlo; off < ap->a_offhi; off += PAGE_SIZE) {
 -			pg = uvm_pagelookup(uobj, off);
 -
 -			/*
 -			 * we only have missing pages here because the
 -			 * calculation of offhi above doesn't account for
 -			 * fragments.  so once we see one missing page,
 -			 * the rest should be missing as well, but we'll
 -			 * check for the rest just to be paranoid.
 -			 */
 -
 -			if (pg == NULL) {
 -				continue;
 -			}
 -			if (pg->flags & PG_BUSY) {
 -				simple_unlock(&uobj->vmobjlock);
 -				return EBUSY;
 -			}
 -		}
 -	}
 -	return genfs_putpages(v);
  }

  /*
 Index: ffs_extern.h
 ===================================================================
 --- ffs_extern.h	(revision 1266)
 +++ ffs_extern.h	(working copy)
 @@ -153,7 +153,6 @@ int	ffs_write(void *);
  int	ffs_fsync(void *);
  int	ffs_reclaim(void *);
  int	ffs_getpages(void *);
 -int	ffs_putpages(void *);
  void	ffs_gop_size(struct vnode *, off_t, off_t *, int);

  #ifdef SYSCTL_SETUP_PROTO

 --NextPart-20050831053047-0408200--

From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Fri, 2 Sep 2005 09:08:54 -0700

 On Wed, Aug 31, 2005 at 05:29:45AM +0900, YAMAMOTO Takashi wrote:
 > > could you give some details on how the pagedaemon would be confused by this?
 > 
 > pagedaemon frees the whole block when one of the corresponding pages
 > happens to be inactive.

 could you explain how this happens?
 here's the code that sets PG_RELEASE or PG_PAGEOUT on a page.
 the marked lines skip setting those flags on pages in the cluster that
 are outside the original request.  this should prevent the problem you're
 describing, right?


                 /*
                  * apply FREE or DEACTIVATE options if requested.
                  */

                 if (flags & (PGO_DEACTIVATE|PGO_FREE)) {
                         uvm_lock_pageq();
                 }
                 for (i = 0; i < npages; i++) {
                         tpg = pgs[i];
                         KASSERT(tpg->uobject == uobj);
                         if (by_list && tpg == TAILQ_NEXT(pg, listq))
                                 pg = tpg;
 -->                     if (tpg->offset < startoff || tpg->offset >= endoff)
 -->                             continue;
                         if (flags & PGO_DEACTIVATE &&
                             (tpg->pqflags & PQ_INACTIVE) == 0 &&
                             tpg->wire_count == 0) {
                                 (void) pmap_clear_reference(tpg);
                                 uvm_pagedeactivate(tpg);
                         } else if (flags & PGO_FREE) {
                                 pmap_page_protect(tpg, VM_PROT_NONE);
                                 if (tpg->flags & PG_BUSY) {
                                         tpg->flags |= freeflag;
                                         if (pagedaemon) {
                                                 uvmexp.paging++;
                                                 uvm_pagedequeue(tpg);
                                         }
                                 } else {

                                         /*
                                          * ``page is not busy''
                                          * implies that npages is 1
                                          * and needs_clean is false.
                                          */

                                         nextpg = TAILQ_NEXT(tpg, listq);
                                         uvm_pagefree(tpg);
                                         if (pagedaemon)
                                                 uvmexp.pdfreed++;
                                 }
                         }
                 }


 -Chuck

From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Sun, 04 Sep 2005 18:47:15 +0900

 > On Wed, Aug 31, 2005 at 05:29:45AM +0900, YAMAMOTO Takashi wrote:
 > > > could you give some details on how the pagedaemon would be confused by this?
 > > 
 > > pagedaemon frees the whole block when one of the corresponding pages
 > > happens to be inactive.
 > 
 > could you explain how this happens?
 > here's the code that sets PG_RELEASE or PG_PAGEOUT on a page.
 > the marked lines skip setting those flags on pages in the cluster that
 > are outside the original request.  this should prevent the problem you're
 > describing, right?

 > -->                     if (tpg->offset < startoff || tpg->offset >= endoff)
 > -->                             continue;

 for normal filesystems, right.
 however, for softupdate, no.
 as ffs_putpages expands the request, the "original request" here
 is already expanded.
 i have seen a system on which uvmexp.pdfreed is larger
 than uvmexp.pdscans.

 YAMAMOTO Takashi

From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Wed, 7 Sep 2005 09:27:24 -0700

 On Sun, Sep 04, 2005 at 06:47:15PM +0900, YAMAMOTO Takashi wrote:
 > > On Wed, Aug 31, 2005 at 05:29:45AM +0900, YAMAMOTO Takashi wrote:
 > > > > could you give some details on how the pagedaemon would be confused by this?
 > > > 
 > > > pagedaemon frees the whole block when one of the corresponding pages
 > > > happens to be inactive.
 > > 
 > > could you explain how this happens?
 > > here's the code that sets PG_RELEASE or PG_PAGEOUT on a page.
 > > the marked lines skip setting those flags on pages in the cluster that
 > > are outside the original request.  this should prevent the problem you're
 > > describing, right?
 > 
 > > -->                     if (tpg->offset < startoff || tpg->offset >= endoff)
 > > -->                             continue;
 > 
 > for normal filesystems, right.
 > however, for softupdate, no.
 > as ffs_putpages expands the request, the "original request" here
 > is already expanded.
 > i have seen a system on which uvmexp.pdfreed is larger
 > than uvmexp.pdscans.

 ah, that makes sense.

 the patch that you sent earlier looks good, I'd just change the
 "multiple write resid" check into a KASSERT instead of UVMHIST_LOG.
 same for the "no page" panic.

 thanks,
 -Chuck

From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Thu, 08 Sep 2005 05:24:51 +0900

 > the patch that you sent earlier looks good, I'd just change the
 > "multiple write resid" check into a KASSERT instead of UVMHIST_LOG.
 > same for the "no page" panic.

 "multiple write" here is normal if the page is re-dirtied.
 it's the reason why we need a bitmap rather than a byte-count.

 YAMAMOTO Takashi

From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Thu, 8 Sep 2005 10:20:25 -0700

 On Thu, Sep 08, 2005 at 05:24:51AM +0900, YAMAMOTO Takashi wrote:
 > > the patch that you sent earlier looks good, I'd just change the
 > > "multiple write resid" check into a KASSERT instead of UVMHIST_LOG.
 > > same for the "no page" panic.
 > 
 > "multiple write" here is normal if the page is re-dirtied.
 > it's the reason why we need a bitmap rather than a byte-count.

 oh... yea, for some reason I was thinking you were setting them again if
 the page was dirtied again... nevermind.

 I'd be good if you added a comment describing exactly how you're using
 the b_resid field, since it's different that all other uses, and even
 those of us familiar with the code get confused if we're not paying
 attention.  :-)

 -Chuck

From: YAMAMOTO Takashi <yamt@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/15364 CVS commit: src/sys/ufs/ffs
Date: Fri,  9 Sep 2005 15:00:39 +0000 (UTC)

 Module Name:	src
 Committed By:	yamt
 Date:		Fri Sep  9 15:00:39 UTC 2005

 Modified Files:
 	src/sys/ufs/ffs: ffs_extern.h ffs_vnops.c

 Log Message:
 revert the code to expand putpage requests to block boundary.
 because:
 	- it was incomplete in some cases.
 	- it can confuse pagedaemon.
 see PR/15364 for details.


 To generate a diff of this commit:
 cvs rdiff -r1.44 -r1.45 src/sys/ufs/ffs/ffs_extern.h
 cvs rdiff -r1.74 -r1.75 src/sys/ufs/ffs/ffs_vnops.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: YAMAMOTO Takashi <yamt@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/15364 CVS commit: src/sys/ufs/ffs
Date: Fri,  9 Sep 2005 15:04:08 +0000 (UTC)

 Module Name:	src
 Committed By:	yamt
 Date:		Fri Sep  9 15:04:08 UTC 2005

 Modified Files:
 	src/sys/ufs/ffs: ffs_softdep.c

 Log Message:
 - for pagecache dependency, track which page in the block
   has been written or not individually by (ab)using b_resid
   in pcbp as a bitmap.
 - add a comment to explain why it's needed.

 PR/15364.  reviewed by Chuck Silvers.


 To generate a diff of this commit:
 cvs rdiff -r1.69 -r1.70 src/sys/ufs/ffs/ffs_softdep.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: chs@netbsd.org
State-Changed-When: Sat, 01 Oct 2005 15:20:48 +0000
State-Changed-Why:
fixed again and better by yamt


From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/15364 CVS commit: [netbsd-3] src/sys/ufs/ffs
Date: Fri, 21 Oct 2005 11:25:56 +0000 (UTC)

 Module Name:	src
 Committed By:	tron
 Date:		Fri Oct 21 11:25:56 UTC 2005

 Modified Files:
 	src/sys/ufs/ffs [netbsd-3]: ffs_extern.h ffs_vnops.c

 Log Message:
 Pull up following revision(s) (requested by yamt in ticket #845):
 	sys/ufs/ffs/ffs_extern.h: revision 1.45 via patch
 	sys/ufs/ffs/ffs_vnops.c: revision 1.75 via patch
 revert the code to expand putpage requests to block boundary.
 because:
 	- it was incomplete in some cases.
 	- it can confuse pagedaemon.
 see PR/15364 for details.


 To generate a diff of this commit:
 cvs rdiff -r1.42 -r1.42.2.1 src/sys/ufs/ffs/ffs_extern.h
 cvs rdiff -r1.69.2.1 -r1.69.2.2 src/sys/ufs/ffs/ffs_vnops.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/15364 CVS commit: [netbsd-3] src/sys/ufs/ffs
Date: Fri, 21 Oct 2005 11:27:00 +0000 (UTC)

 Module Name:	src
 Committed By:	tron
 Date:		Fri Oct 21 11:27:00 UTC 2005

 Modified Files:
 	src/sys/ufs/ffs [netbsd-3]: ffs_softdep.c

 Log Message:
 Pull up following revision(s) (requested by yamt in ticket #845):
 	sys/ufs/ffs/ffs_softdep.c: revision 1.70 via patch
 - for pagecache dependency, track which page in the block
   has been written or not individually by (ab)using b_resid
   in pcbp as a bitmap.
 - add a comment to explain why it's needed.
 PR/15364.  reviewed by Chuck Silvers.


 To generate a diff of this commit:
 cvs rdiff -r1.63.2.1 -r1.63.2.2 src/sys/ufs/ffs/ffs_softdep.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.