NetBSD Problem Report #15364
Received: (qmail 24774 invoked from network); 25 Jan 2002 22:19:52 -0000
Message-Id: <20020125221918.7051.qmail@h187n2fls4o804.telia.com>
Date: 25 Jan 2002 22:19:18 -0000
From: Tomas Svensson <tsn@gbdev.net>
Reply-To: Tomas Svensson <tsn@gbdev.net>
To: gnats-bugs@gnats.netbsd.org
Subject: panic: softdep_pageiodone
X-Send-Pr-Version: 3.95
>Number: 15364
>Category: kern
>Synopsis: panic: softdep_pageiodone
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: chs
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Fri Jan 25 22:20:01 +0000 2002
>Closed-Date: Sat Oct 01 15:20:48 +0000 2005
>Last-Modified: Wed May 13 13:08:24 +0000 2015
>Originator:
>Release: NetBSD 1.5ZA
>Organization:
>Environment:
NetBSD 1.5ZA i386, kernel made from CVS sources January 14.
>Description:
panic: softdep_pageiodone: resid < 0, vp 0xc6ad59d0 lbn 0x8 pcbp 0x6add410
Stopped in pid 7 (aiodoned) at cpu_Debugger+0x4: leave
db> trace
cpu_Debugger(c6ad59d0,10000,0,3,29a) at cpu_Debugger+0x4
panic(c0227bc0,c6ad59d0,8,c6add410,0) at panic+0xad
softdep_pageiodone(c04620d0,c0396200,c032d1f4,c0282720) at softdep_pageiodone+0x147
uvm_aio_aiodone(c04620d0,c0462548,ffffffff,c01bb496,c686fac8) at uvm_aio_aiodone+0x77
uvm_aiodone_daemon(c686fac8) at uvm_aiodone_daemon+0x98
db> reboot
syncing disks... panic: softdep_lock: locking against myself
>How-To-Repeat:
Was transferring one file to the host, one file from it simultaneously
over 100baseTX LAN.
>Fix:
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed
State-Changed-By: chs
State-Changed-When: Sun May 5 10:03:52 PDT 2002
State-Changed-Why:
fixed with rev. 1.49 of ffs_vnops.c.
Responsible-Changed-From-To: kern-bug-people->chs
Responsible-Changed-By: chs
Responsible-Changed-When: Sun May 5 10:03:52 PDT 2002
Responsible-Changed-Why:
my bug.
State-Changed-From-To: closed->open
State-Changed-By: yamt@netbsd.org
State-Changed-When: Wed, 24 Aug 2005 08:08:41 +0000
State-Changed-Why:
i saw this panic today.
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: gnats-bugs@NetBSD.org
Cc: chs@netbsd.org
Subject: Re: kern/15364
Date: Thu, 25 Aug 2005 11:44:49 +0900
> Synopsis: panic: softdep_pageiodone
here's a simple test code to trigger the bug.
% cc a.c
% ./a.out > testfile
YAMAMOTO Takashi
#include <sys/mman.h>
#include <string.h>
#include <unistd.h>
#define MAXPHYS 65536
#define BLKSIZ 16384
#define FRAGSIZ 2048
#define PAGE_SIZE 4096
/* assuming FRAGSIZ < PAGE_SIZE < BLKSIZ < MAXPHYS */
int
main()
{
char buf[BLKSIZ];
char *cp;
memset(buf, 0xaa, sizeof(buf));
pwrite(1, buf, 1, MAXPHYS - 1);
cp = mmap(0, MAXPHYS*16, PROT_WRITE, MAP_FILE|MAP_SHARED, 1, 0);
memset(cp, 0xbb, BLKSIZ);
fsync(1);
memset(cp + PAGE_SIZE, 0xee, MAXPHYS - PAGE_SIZE);
/* create a page dep, b_resid == PAGE_SIZE + FRAGSIZE */
pwrite(1, buf, PAGE_SIZE + FRAGSIZ, MAXPHYS);
/* write out the page at the offset MAXPHYS */
msync(cp + PAGE_SIZE, 1, MS_SYNC);
/* now b_resid == FRAGSIZ */
/* re-dirty and write the page, b_resid goes to negative. */
pwrite(1, buf, PAGE_SIZE + FRAGSIZ, MAXPHYS);
fsync(1);
}
From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Sun, 28 Aug 2005 08:00:27 -0700
--ew6BAiZeqk4r7MaW
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
ah, it's the write clustering that's causing the problem.
the attached patch forces the putpages write clustering to obey
the same block-alignment rule that the getpages path uses, so that
pages involved in a softdep dependency will always be written together.
do you see any problem with this?
-Chuck
--ew6BAiZeqk4r7MaW
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="diff.softdep"
Index: src/sys/sys/mount.h
===================================================================
RCS file: /cvsroot/src/sys/sys/mount.h,v
retrieving revision 1.129
diff -u -p -r1.129 mount.h
--- src/sys/sys/mount.h 29 May 2005 21:19:41 -0000 1.129
+++ src/sys/sys/mount.h 28 Aug 2005 14:55:26 -0000
@@ -126,6 +126,7 @@ struct mount {
int mnt_iflag; /* internal flags */
int mnt_fs_bshift; /* offset shift for lblkno */
int mnt_dev_bshift; /* shift for device sectors */
+ int mnt_clusteralign; /* cluster alignment needed */
struct statvfs mnt_stat; /* cache of filesystem stats */
void *mnt_data; /* private data */
int mnt_wcnt; /* count of vfs_busy waiters */
Index: src/sys/kern/vfs_syscalls.c
===================================================================
RCS file: /cvsroot/src/sys/kern/vfs_syscalls.c,v
retrieving revision 1.229
diff -u -p -r1.229 vfs_syscalls.c
--- src/sys/kern/vfs_syscalls.c 19 Aug 2005 12:30:02 -0000 1.229
+++ src/sys/kern/vfs_syscalls.c 28 Aug 2005 14:55:28 -0000
@@ -314,6 +314,7 @@ sys_mount(struct lwp *l, void *v, regist
mp->mnt_stat.f_owner = p->p_ucred->cr_uid;
mp->mnt_unmounter = NULL;
mp->mnt_leaf = mp;
+ mp->mnt_clusteralign = PAGE_SIZE;
/*
* The underlying file system may refuse the mount for
Index: src/sys/ufs/ffs/ffs_vfsops.c
===================================================================
RCS file: /cvsroot/src/sys/ufs/ffs/ffs_vfsops.c,v
retrieving revision 1.169
diff -u -p -r1.169 ffs_vfsops.c
--- src/sys/ufs/ffs/ffs_vfsops.c 23 Aug 2005 08:05:13 -0000 1.169
+++ src/sys/ufs/ffs/ffs_vfsops.c 28 Aug 2005 14:55:29 -0000
@@ -450,6 +450,9 @@ ffs_mount(struct mount *mp, const char *
}
(void) ffs_cgupdate(ump, MNT_WAIT);
}
+ if (mp->mnt_flag & MNT_SOFTDEP) {
+ mp->mnt_clusteralign = MAX(PAGE_SIZE, fs->fs_bsize);
+ }
return (error);
fail:
Index: src/sys/miscfs/genfs/genfs_vnops.c
===================================================================
RCS file: /cvsroot/src/sys/miscfs/genfs/genfs_vnops.c,v
retrieving revision 1.104
diff -u -p -r1.104 genfs_vnops.c
--- src/sys/miscfs/genfs/genfs_vnops.c 26 Jul 2005 08:06:29 -0000 1.104
+++ src/sys/miscfs/genfs/genfs_vnops.c 28 Aug 2005 14:55:30 -0000
@@ -1095,24 +1095,21 @@ genfs_putpages(void *v)
voff_t a_offhi;
int a_flags;
} */ *ap = v;
+ const int maxpages = MAXPHYS >> PAGE_SHIFT;
struct vnode *vp = ap->a_vp;
struct uvm_object *uobj = &vp->v_uobj;
struct simplelock *slock = &uobj->vmobjlock;
+ struct lwp *l = curlwp ? curlwp : &lwp0;
+ struct genfs_node *gp = VTOG(vp);
+ struct vm_page *pgs[maxpages], *pg, *nextpg, *tpg, curmp, endmp;
off_t startoff = ap->a_offlo;
off_t endoff = ap->a_offhi;
- off_t off;
+ off_t off, aofflo, aoffhi;
int flags = ap->a_flags;
- /* Even for strange MAXPHYS, the shift rounds down to a page */
- const int maxpages = MAXPHYS >> PAGE_SHIFT;
- int i, s, error, npages, nback;
- int freeflag;
- struct vm_page *pgs[maxpages], *pg, *nextpg, *tpg, curmp, endmp;
+ int i, s, error, npages, nbytes, nback, freeflag, dirtygen, amask;
boolean_t wasclean, by_list, needs_clean, yld;
boolean_t async = (flags & PGO_SYNCIO) == 0;
boolean_t pagedaemon = curproc == uvm.pagedaemon_proc;
- struct lwp *l = curlwp ? curlwp : &lwp0;
- struct genfs_node *gp = VTOG(vp);
- int dirtygen;
boolean_t modified = FALSE;
boolean_t cleanall;
@@ -1183,6 +1180,11 @@ genfs_putpages(void *v)
startoff == 0 && endoff == trunc_page(LLONG_MAX) &&
(vp->v_flag & VONWORKLST) != 0;
dirtygen = gp->g_dirtygen;
+ if (vp->v_type == VREG) {
+ amask = vp->v_mount->mnt_clusteralign - 1;
+ } else {
+ amask = PAGE_MASK;
+ }
freeflag = pagedaemon ? PG_PAGEOUT : PG_RELEASED;
curmp.uobject = uobj;
curmp.offset = (voff_t)-1;
@@ -1326,11 +1328,16 @@ genfs_putpages(void *v)
UVM_PAGE_OWN(pg, "genfs_putpages");
/*
- * first look backward.
+ * first look backward, making sure the range
+ * is aligned as needed.
*/
- npages = MIN(maxpages >> 1, off >> PAGE_SHIFT);
+ nbytes = MIN(maxpages << (PAGE_SHIFT - 1), off);
+ aofflo = (off - PAGE_SIZE - nbytes + amask) & ~amask;
+ aoffhi = MAX(off - PAGE_SIZE, 0);
+ npages = MIN(aoffhi - aofflo, off) >> PAGE_SHIFT;
nback = npages;
+
uvn_findpages(uobj, off - PAGE_SIZE, &nback, &pgs[0],
UFP_NOWAIT|UFP_NOALLOC|UFP_DIRTYONLY|UFP_BACKWARD);
if (nback) {
@@ -1352,10 +1359,14 @@ genfs_putpages(void *v)
/*
* then look forward to fill in the remaining space in
- * the array of pages.
+ * the array of pages, again aligning the range.
*/
- npages = maxpages - nback - 1;
+ nbytes = (maxpages - nback - 1) << PAGE_SHIFT;
+ aofflo = off + PAGE_SIZE;
+ aoffhi = (aofflo + nbytes) & ~amask;
+ npages = (aoffhi - aofflo) >> PAGE_SHIFT;
+
uvn_findpages(uobj, off + PAGE_SIZE, &npages,
&pgs[nback + 1],
UFP_NOWAIT|UFP_NOALLOC|UFP_DIRTYONLY);
--ew6BAiZeqk4r7MaW--
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Mon, 29 Aug 2005 09:18:36 +0900
> ah, it's the write clustering that's causing the problem.
> the attached patch forces the putpages write clustering to obey
> the same block-alignment rule that the getpages path uses, so that
> pages involved in a softdep dependency will always be written together.
> do you see any problem with this?
>
> -Chuck
i don't think that block-alignment is really needed here. (unlike lfs.)
isn't it better to have a bitmap in softdep "pcbp" to track which
pages in a block have been written out?
YAMAMOTO Takashi
From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Mon, 29 Aug 2005 10:01:34 -0700
On Mon, Aug 29, 2005 at 09:18:36AM +0900, YAMAMOTO Takashi wrote:
> > ah, it's the write clustering that's causing the problem.
> > the attached patch forces the putpages write clustering to obey
> > the same block-alignment rule that the getpages path uses, so that
> > pages involved in a softdep dependency will always be written together.
> > do you see any problem with this?
> >
> > -Chuck
>
> i don't think that block-alignment is really needed here. (unlike lfs.)
> isn't it better to have a bitmap in softdep "pcbp" to track which
> pages in a block have been written out?
well, I was thinking it would be good to have the property that a given block
is written in one I/O, since that's what the softdep code was designed for,
and it'd be safer to maintain the assumptions that the softdep code makes.
but you're probably right that it's not really required. would you like to
implement your bitmap idea?
-Chuck
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Tue, 30 Aug 2005 10:53:22 +0900
> well, I was thinking it would be good to have the property that a given block
> is written in one I/O, since that's what the softdep code was designed for,
> and it'd be safer to maintain the assumptions that the softdep code makes.
> but you're probably right that it's not really required.
while block-alignment might have good aspects,
it can confuse pagedaemon and increase number of i/o
if both of active and inactive pages are in a block.
i'm not sure if it affects real-world performance, tho.
> would you like to
> implement your bitmap idea?
i will.
YAMAMOTO Takashi
From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Tue, 30 Aug 2005 09:25:34 -0700
On Tue, Aug 30, 2005 at 10:53:22AM +0900, YAMAMOTO Takashi wrote:
> > well, I was thinking it would be good to have the property that a given block
> > is written in one I/O, since that's what the softdep code was designed for,
> > and it'd be safer to maintain the assumptions that the softdep code makes.
> > but you're probably right that it's not really required.
>
> while block-alignment might have good aspects,
> it can confuse pagedaemon and increase number of i/o
> if both of active and inactive pages are in a block.
could you give some details on how the pagedaemon would be confused by this?
the pagedaemon will do the write-clustering regardless of what range is
considered for clustering, so how does the block-alignment make things worse?
> i'm not sure if it affects real-world performance, tho.
I doubt it would make any difference. in the common case of sequential writes,
the ranges sent to VOP_PUTPAGES (by VOP_WRITE) are block-aligned anyway.
only for random unaligned writes that happen to be close to MAXPHYS in size
would we create more I/O requests than before.
-Chuck
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Wed, 31 Aug 2005 05:29:45 +0900
> could you give some details on how the pagedaemon would be confused by this?
pagedaemon frees the whole block when one of the corresponding pages
happens to be inactive.
YAMAMOTO Takashi
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Wed, 31 Aug 2005 05:30:58 +0900
--NextPart-20050831053047-0408200
Content-Type: Text/Plain; charset=us-ascii
> > would you like to
> > implement your bitmap idea?
>
> i will.
here is.
YAMAMOTO Takashi
--NextPart-20050831053047-0408200
Content-Type: Text/Plain; charset=us-ascii
Content-Disposition: attachment; filename="a.diff"
Index: ffs_softdep.c
===================================================================
--- ffs_softdep.c (revision 1330)
+++ ffs_softdep.c (working copy)
@@ -197,6 +197,9 @@ void softdep_pageiodone __P((struct buf
void softdep_flush_vnode __P((struct vnode *, daddr_t));
static void softdep_trackbufs(struct inode *, int, boolean_t);
+#define PCBP_BITMAP(off, size) \
+ (((1 << howmany((size), PAGE_SIZE)) - 1) << ((off) >> PAGE_SHIFT))
+
/*
* Exported softdep operations.
*/
@@ -5804,9 +5807,13 @@ softdep_setup_pagecache(ip, lbn, size)
LIST_INSERT_HEAD(&pcbphashhead[PCBPHASH(vp, lbn)], bp, b_hash);
LIST_INSERT_HEAD(&ip->i_pcbufhd, bp, b_vnbufs);
}
- bp->b_bcount = bp->b_resid = size;
+ bp->b_bcount = size;
+ KASSERT(size <= PAGE_SIZE * sizeof(bp->b_resid) * CHAR_BIT);
+ bp->b_resid = PCBP_BITMAP(0, size);
UVMHIST_LOG(ubchist, "vp = %p, lbn = %ld, "
- "bp = %p, bcount = resid = %ld", vp, lbn, bp, size);
+ "bp = %p, bcount = %ld", vp, lbn, bp, size);
+ UVMHIST_LOG(ubchist, "b_resid = %ld",
+ bp->b_resid, 0, 0, 0);
return bp;
}
@@ -5933,7 +5940,7 @@ softdep_pageiodone1(bp)
struct worklist *wk;
daddr_t lbn;
voff_t off;
- long iosize = bp->b_bcount;
+ int iosize = bp->b_bcount;
int size, asize, bshift, bsize;
int i;
UVMHIST_FUNC("softdep_pageiodone"); UVMHIST_CALLED(ubchist);
@@ -5946,12 +5953,14 @@ softdep_pageiodone1(bp)
for (i = 0; i < npages; i++) {
pg = uvm_pageratop((vaddr_t)bp->b_data + (i << PAGE_SHIFT));
if (pg == NULL) {
- continue;
+ panic("%s: no page", __func__);
}
for (off = pg->offset;
off < pg->offset + PAGE_SIZE;
off += bsize) {
+ int pgmask;
+
size = MIN(asize, iosize);
iosize -= size;
lbn = off >> bshift;
@@ -5968,14 +5977,14 @@ softdep_pageiodone1(bp)
UVMHIST_LOG(ubchist,
"pcbp %p iosize %ld, size %d, asize %d",
pcbp, iosize, size, asize);
- pcbp->b_resid -= size;
- if (pcbp->b_resid < 0) {
- panic("softdep_pageiodone: "
- "resid < 0, vp %p lbn 0x%" PRIx64 " pcbp %p"
- " iosize %ld, size %d, asize %d, bsize %d",
- vp, lbn, pcbp, iosize, size, asize, bsize);
+ pgmask = PCBP_BITMAP(off & (bsize - 1), size);
+ if ((~pcbp->b_resid & pgmask) != 0) {
+ UVMHIST_LOG(ubchist,
+ "multiple write resid %lx, pgmask %lx",
+ pcbp->b_resid, pgmask, 0, 0);
}
- if (pcbp->b_resid > 0) {
+ pcbp->b_resid &= ~pgmask;
+ if (pcbp->b_resid != 0) {
continue;
}
Index: ffs_vnops.c
===================================================================
--- ffs_vnops.c (revision 1276)
+++ ffs_vnops.c (working copy)
@@ -116,7 +116,7 @@ const struct vnodeopv_entry_desc ffs_vno
{ &vop_update_desc, ffs_update }, /* update */
{ &vop_bwrite_desc, vn_bwrite }, /* bwrite */
{ &vop_getpages_desc, ffs_getpages }, /* getpages */
- { &vop_putpages_desc, ffs_putpages }, /* putpages */
+ { &vop_putpages_desc, genfs_putpages }, /* putpages */
{ NULL, NULL }
};
const struct vnodeopv_desc ffs_vnodeop_opv_desc =
@@ -533,58 +533,6 @@ ffs_getpages(void *v)
return EINVAL;
}
return genfs_getpages(v);
-}
-
-int
-ffs_putpages(void *v)
-{
- struct vop_putpages_args /* {
- struct vnode *a_vp;
- voff_t a_offlo;
- voff_t a_offhi;
- int a_flags;
- } */ *ap = v;
- struct vnode *vp = ap->a_vp;
- struct uvm_object *uobj = &vp->v_uobj;
- struct inode *ip = VTOI(vp);
- struct fs *fs = ip->i_fs;
- struct vm_page *pg;
- off_t off;
-
- if (!DOINGSOFTDEP(vp) || (ap->a_flags & PGO_CLEANIT) == 0) {
- return genfs_putpages(v);
- }
-
- /*
- * for softdep files, force the pages in a block to be written together.
- * if we're the pagedaemon and we would have to wait for other pages,
- * just fail the request. the pagedaemon will pick a different page.
- */
-
- ap->a_offlo &= ~fs->fs_qbmask;
- ap->a_offhi = blkroundup(fs, ap->a_offhi);
- if (curproc == uvm.pagedaemon_proc) {
- for (off = ap->a_offlo; off < ap->a_offhi; off += PAGE_SIZE) {
- pg = uvm_pagelookup(uobj, off);
-
- /*
- * we only have missing pages here because the
- * calculation of offhi above doesn't account for
- * fragments. so once we see one missing page,
- * the rest should be missing as well, but we'll
- * check for the rest just to be paranoid.
- */
-
- if (pg == NULL) {
- continue;
- }
- if (pg->flags & PG_BUSY) {
- simple_unlock(&uobj->vmobjlock);
- return EBUSY;
- }
- }
- }
- return genfs_putpages(v);
}
/*
Index: ffs_extern.h
===================================================================
--- ffs_extern.h (revision 1266)
+++ ffs_extern.h (working copy)
@@ -153,7 +153,6 @@ int ffs_write(void *);
int ffs_fsync(void *);
int ffs_reclaim(void *);
int ffs_getpages(void *);
-int ffs_putpages(void *);
void ffs_gop_size(struct vnode *, off_t, off_t *, int);
#ifdef SYSCTL_SETUP_PROTO
--NextPart-20050831053047-0408200--
From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Fri, 2 Sep 2005 09:08:54 -0700
On Wed, Aug 31, 2005 at 05:29:45AM +0900, YAMAMOTO Takashi wrote:
> > could you give some details on how the pagedaemon would be confused by this?
>
> pagedaemon frees the whole block when one of the corresponding pages
> happens to be inactive.
could you explain how this happens?
here's the code that sets PG_RELEASE or PG_PAGEOUT on a page.
the marked lines skip setting those flags on pages in the cluster that
are outside the original request. this should prevent the problem you're
describing, right?
/*
* apply FREE or DEACTIVATE options if requested.
*/
if (flags & (PGO_DEACTIVATE|PGO_FREE)) {
uvm_lock_pageq();
}
for (i = 0; i < npages; i++) {
tpg = pgs[i];
KASSERT(tpg->uobject == uobj);
if (by_list && tpg == TAILQ_NEXT(pg, listq))
pg = tpg;
--> if (tpg->offset < startoff || tpg->offset >= endoff)
--> continue;
if (flags & PGO_DEACTIVATE &&
(tpg->pqflags & PQ_INACTIVE) == 0 &&
tpg->wire_count == 0) {
(void) pmap_clear_reference(tpg);
uvm_pagedeactivate(tpg);
} else if (flags & PGO_FREE) {
pmap_page_protect(tpg, VM_PROT_NONE);
if (tpg->flags & PG_BUSY) {
tpg->flags |= freeflag;
if (pagedaemon) {
uvmexp.paging++;
uvm_pagedequeue(tpg);
}
} else {
/*
* ``page is not busy''
* implies that npages is 1
* and needs_clean is false.
*/
nextpg = TAILQ_NEXT(tpg, listq);
uvm_pagefree(tpg);
if (pagedaemon)
uvmexp.pdfreed++;
}
}
}
-Chuck
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Sun, 04 Sep 2005 18:47:15 +0900
> On Wed, Aug 31, 2005 at 05:29:45AM +0900, YAMAMOTO Takashi wrote:
> > > could you give some details on how the pagedaemon would be confused by this?
> >
> > pagedaemon frees the whole block when one of the corresponding pages
> > happens to be inactive.
>
> could you explain how this happens?
> here's the code that sets PG_RELEASE or PG_PAGEOUT on a page.
> the marked lines skip setting those flags on pages in the cluster that
> are outside the original request. this should prevent the problem you're
> describing, right?
> --> if (tpg->offset < startoff || tpg->offset >= endoff)
> --> continue;
for normal filesystems, right.
however, for softupdate, no.
as ffs_putpages expands the request, the "original request" here
is already expanded.
i have seen a system on which uvmexp.pdfreed is larger
than uvmexp.pdscans.
YAMAMOTO Takashi
From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Wed, 7 Sep 2005 09:27:24 -0700
On Sun, Sep 04, 2005 at 06:47:15PM +0900, YAMAMOTO Takashi wrote:
> > On Wed, Aug 31, 2005 at 05:29:45AM +0900, YAMAMOTO Takashi wrote:
> > > > could you give some details on how the pagedaemon would be confused by this?
> > >
> > > pagedaemon frees the whole block when one of the corresponding pages
> > > happens to be inactive.
> >
> > could you explain how this happens?
> > here's the code that sets PG_RELEASE or PG_PAGEOUT on a page.
> > the marked lines skip setting those flags on pages in the cluster that
> > are outside the original request. this should prevent the problem you're
> > describing, right?
>
> > --> if (tpg->offset < startoff || tpg->offset >= endoff)
> > --> continue;
>
> for normal filesystems, right.
> however, for softupdate, no.
> as ffs_putpages expands the request, the "original request" here
> is already expanded.
> i have seen a system on which uvmexp.pdfreed is larger
> than uvmexp.pdscans.
ah, that makes sense.
the patch that you sent earlier looks good, I'd just change the
"multiple write resid" check into a KASSERT instead of UVMHIST_LOG.
same for the "no page" panic.
thanks,
-Chuck
From: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
To: chuq@chuq.com
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Thu, 08 Sep 2005 05:24:51 +0900
> the patch that you sent earlier looks good, I'd just change the
> "multiple write resid" check into a KASSERT instead of UVMHIST_LOG.
> same for the "no page" panic.
"multiple write" here is normal if the page is re-dirtied.
it's the reason why we need a bitmap rather than a byte-count.
YAMAMOTO Takashi
From: Chuck Silvers <chuq@chuq.com>
To: YAMAMOTO Takashi <yamt@mwd.biglobe.ne.jp>
Cc: gnats-bugs@NetBSD.org
Subject: Re: kern/15364
Date: Thu, 8 Sep 2005 10:20:25 -0700
On Thu, Sep 08, 2005 at 05:24:51AM +0900, YAMAMOTO Takashi wrote:
> > the patch that you sent earlier looks good, I'd just change the
> > "multiple write resid" check into a KASSERT instead of UVMHIST_LOG.
> > same for the "no page" panic.
>
> "multiple write" here is normal if the page is re-dirtied.
> it's the reason why we need a bitmap rather than a byte-count.
oh... yea, for some reason I was thinking you were setting them again if
the page was dirtied again... nevermind.
I'd be good if you added a comment describing exactly how you're using
the b_resid field, since it's different that all other uses, and even
those of us familiar with the code get confused if we're not paying
attention. :-)
-Chuck
From: YAMAMOTO Takashi <yamt@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/15364 CVS commit: src/sys/ufs/ffs
Date: Fri, 9 Sep 2005 15:00:39 +0000 (UTC)
Module Name: src
Committed By: yamt
Date: Fri Sep 9 15:00:39 UTC 2005
Modified Files:
src/sys/ufs/ffs: ffs_extern.h ffs_vnops.c
Log Message:
revert the code to expand putpage requests to block boundary.
because:
- it was incomplete in some cases.
- it can confuse pagedaemon.
see PR/15364 for details.
To generate a diff of this commit:
cvs rdiff -r1.44 -r1.45 src/sys/ufs/ffs/ffs_extern.h
cvs rdiff -r1.74 -r1.75 src/sys/ufs/ffs/ffs_vnops.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: YAMAMOTO Takashi <yamt@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/15364 CVS commit: src/sys/ufs/ffs
Date: Fri, 9 Sep 2005 15:04:08 +0000 (UTC)
Module Name: src
Committed By: yamt
Date: Fri Sep 9 15:04:08 UTC 2005
Modified Files:
src/sys/ufs/ffs: ffs_softdep.c
Log Message:
- for pagecache dependency, track which page in the block
has been written or not individually by (ab)using b_resid
in pcbp as a bitmap.
- add a comment to explain why it's needed.
PR/15364. reviewed by Chuck Silvers.
To generate a diff of this commit:
cvs rdiff -r1.69 -r1.70 src/sys/ufs/ffs/ffs_softdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->closed
State-Changed-By: chs@netbsd.org
State-Changed-When: Sat, 01 Oct 2005 15:20:48 +0000
State-Changed-Why:
fixed again and better by yamt
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/15364 CVS commit: [netbsd-3] src/sys/ufs/ffs
Date: Fri, 21 Oct 2005 11:25:56 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Fri Oct 21 11:25:56 UTC 2005
Modified Files:
src/sys/ufs/ffs [netbsd-3]: ffs_extern.h ffs_vnops.c
Log Message:
Pull up following revision(s) (requested by yamt in ticket #845):
sys/ufs/ffs/ffs_extern.h: revision 1.45 via patch
sys/ufs/ffs/ffs_vnops.c: revision 1.75 via patch
revert the code to expand putpage requests to block boundary.
because:
- it was incomplete in some cases.
- it can confuse pagedaemon.
see PR/15364 for details.
To generate a diff of this commit:
cvs rdiff -r1.42 -r1.42.2.1 src/sys/ufs/ffs/ffs_extern.h
cvs rdiff -r1.69.2.1 -r1.69.2.2 src/sys/ufs/ffs/ffs_vnops.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/15364 CVS commit: [netbsd-3] src/sys/ufs/ffs
Date: Fri, 21 Oct 2005 11:27:00 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Fri Oct 21 11:27:00 UTC 2005
Modified Files:
src/sys/ufs/ffs [netbsd-3]: ffs_softdep.c
Log Message:
Pull up following revision(s) (requested by yamt in ticket #845):
sys/ufs/ffs/ffs_softdep.c: revision 1.70 via patch
- for pagecache dependency, track which page in the block
has been written or not individually by (ab)using b_resid
in pcbp as a bitmap.
- add a comment to explain why it's needed.
PR/15364. reviewed by Chuck Silvers.
To generate a diff of this commit:
cvs rdiff -r1.63.2.1 -r1.63.2.2 src/sys/ufs/ffs/ffs_softdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.