NetBSD Problem Report #20191
Received: (qmail 8013 invoked by uid 605); 4 Feb 2003 03:46:18 -0000
Message-Id: <200302040346.h143kBi09384@stix.homeunix.net>
Date: Tue, 4 Feb 2003 14:46:11 +1100 (EST)
From: Paul Ripke <stix@stix.homeunix.net>
Sender: gnats-bugs-owner@netbsd.org
Reply-To: stixpjr@ozemail.com.au
To: gnats-bugs@gnats.netbsd.org
Subject: RAIDframe RAID5 + softdep can deplete kernel memory
X-Send-Pr-Version: 3.95
>Number: 20191
>Category: kern
>Synopsis: RAIDframe RAID5 + softdep can deplete kernel memory
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: oster
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Feb 04 03:47:00 +0000 2003
>Closed-Date: Fri Apr 09 23:13:27 +0000 2004
>Last-Modified: Sun Apr 11 11:20:38 +0000 2004
>Originator: Paul Ripke
>Release: NetBSD 1.6M (-current 2003-01-29)
>Organization:
>Environment:
System: NetBSD stix-pc.stix.org.au 1.6M NetBSD 1.6M (STIX-PC) #37: Fri Jan 31 14:28:33 EST 2003 stix@stix-amd.stix.org.au:/usr/src/sys/arch/i386/compile/STIX-PC i386
Architecture: i386
Machine: i386
RAM: 128MB
>Description:
When testing RAIDframe and newfs settings across a three disk raid
level 5, using three 1GB vnd devices on three destinct physical disks,
I have repeatedly hung the kernel. "show uvmexp" and gdb on cores show
uvmexp.free less than 5, often only 1.
Hang also reliably occurs when testing a degraded 3-disk RAID5 set, set up
on two 30GB wd devices.
I am yet to see the hang on a mirror or stripe.
From a dump taken from an earlier kernel:
(gdb) print uvmexp
$1 = {pagesize = 4096, pagemask = 4095, pageshift = 12, npages = 31381, free = 4, active = 16949, inactive = 8493,
paging = 180, wired = 686, ncolors = 32, colormask = 31, zeropages = 0, reserve_pagedaemon = 1, reserve_kernel = 5,
anonpages = 12784, filepages = 13198, execpages = 2683, freemin = 64, freetarg = 85, inactarg = 8480,
wiredmax = 10460, anonmin = 25, execmin = 12, filemin = 25, anonminpct = 10, execminpct = 5, fileminpct = 10,
anonmax = 204, execmax = 76, filemax = 128, anonmaxpct = 80, execmaxpct = 30, filemaxpct = 50, nswapdev = 1,
swpages = 262079, swpginuse = 11241, swpgonly = 9386, nswget = 2294, nanon = 291415, nanonneeded = 291415,
nfreeanon = 272082, faults = 1641588, traps = 2781446, intrs = 16918234, swtch = 7362904, softs = 14611102,
syscalls = 9382809, pageins = 2182, swapins = 373, swapouts = 439, pgswapin = 0, pgswapout = 11526, forks = 6379,
forks_ppwait = 1204, forks_sharevm = 1254, pga_zerohit = 0, pga_zeromiss = 478622, zeroaborts = 0,
colorhit = 9403421, colormiss = 66632, fltnoram = 3, fltnoanon = 0, fltpgwait = 6, fltpgrele = 0, fltrelck = 6860,
fltrelckok = 6860, fltanget = 998920, fltanretry = 2188, fltamcopy = 72135, fltnamap = 129168, fltnomap = 738405,
fltlget = 191039, fltget = 4670, flt_anon = 849972, flt_acow = 57721, flt_obj = 167057, flt_prcopy = 23982,
flt_przero = 442479, pdwoke = 2668, pdrevs = 2347, pdswout = 2347, pdfreed = 154, pdscans = 1744940,
pdanscan = 11501, pdobscan = 588268, pdreact = 310694, pdbusy = 5380, pdpageouts = 748, pdpending = 748,
pddeact = 1897808, pdreanon = 646097, pdrefile = 2340, pdreexec = 180660}
(gdb) xps
proc pid flag st wchan comm
0xcb6997c0 6377 4006 4 0xc04023d0 pax (uvn_fp1)
0xcb893244 6349 20204 3 0xc07460d4 raidio0 (raidiow)
0xcb7fc998 6348 20204 3 0xc04023d0 raid0 (km_getwait2)
0xcb5a53e4 474 4006 4 0xc04023d0 top (flt_noram5)
0xcb516b5c 451 105 4 0xc04023d0 screen-3.9.11 (flt_noram1)
0xcb51601c 288 4 4 0xc04023d0 cron (flt_noram3)
...
I have seen the pagedaemon in "emergva" on many occasions.
>How-To-Repeat:
RAID5 sets tested ranged from 8-64 sect/SU, fifo from 1-100.
newfs usually "-i 32k -f 4k -b 32k", but other parameters have been tried.
mount must be "-o softdep" for the problem to occur.
With standard kernel, system will hang part way through unpacking
gcc-2.95.2.tar.gz.
With the following kernel parameters changed, the script below hung
on the 60th extract of the gcc tarball.
uvmexp.reserve_kernel = 128
uvmexp.freemin = 192
uvmexp.freetarg = 256
#!/bin/ksh
cd /mnt
i=0
while [ $i -le 100 ]; do
echo $i
pax -rvzf /tmp/gcc-2.95.2.tar.gz 2>&1 | tail -1
mv gcc-2.95.2 z$i
let i++
done
>Fix:
Workaround: mount the filesystem without softdep.
>Release-Note:
>Audit-Trail:
From: Paul Ripke <stixpjr@ozemail.com.au>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/20191
Date: Fri, 21 Feb 2003 09:02:54 +1100
Correction: I've just had my first hang writing to a non-softdep
raid5 filesystem, while attempting to migrate another filesystem
to raid5, from a single-user boot. Swap was mounted, and the only
writable filesystems were /tmp and the destination.
dump -0f - /dev/wd1a | restore -xvf -
--
Paul Ripke
From: Paul Ripke <stixpjr@ozemail.com.au>
To: gnats-bugs@netbsd.org
Cc: Greg Oster <oster@cs.usask.ca>
Subject: Re: kern/20191 - RAIDframe RAID5
Date: Thu, 15 May 2003 10:17:36 +1000
After some more research: the problem occurs when the page daemon
pages out to a RAID5 volume in extreme low memory situations. The
destination may be a swap device, or it may be a normal RAID5
filesystem where the page daemon is flushing dirty file pages to
free memory.
RAIDframe expects to be able to allocate fairly large chunks of
memory, especially to complete partial stripe writes (read old
data, old parity, recalculate and write back out). These chunks
(I believe) are usually the stripe unit size (often 16KB).
Additionally, RAIDframe normally keeps 6 dags in flight per raid
device, so memory allocated can be of the order of, say
6 x 48KB = 288KB. The kernel reserves free pages for its own use
(uvmexp.reserve_kernel = 5), but this is hardly sufficient for
RAIDframe. The following patch makes for a rock-solid system in
my case, but isn't the ideal solution. Preferably, RAIDframe needs
to keep pre-allocated buffers for use in low memory situations,
similar to what, I believe, Vinum does.
Index: dev/raidframe/rf_netbsdkintf.c
===================================================================
RCS file: /cvsroot/src/sys/dev/raidframe/rf_netbsdkintf.c,v
retrieving revision 1.157
diff -u -d -b -w -r1.157 rf_netbsdkintf.c
--- dev/raidframe/rf_netbsdkintf.c 2003/04/10 03:36:02 1.157
+++ dev/raidframe/rf_netbsdkintf.c 2003/05/14 22:57:45
@@ -133,6 +133,8 @@
#include <sys/user.h>
#include <sys/reboot.h>
+#include <uvm/uvm.h>
+
#include <dev/raidframe/raidframevar.h>
#include <dev/raidframe/raidframeio.h>
#include "raid.h"
@@ -1696,7 +1698,15 @@
}
/* Check to see if we're at the limit... */
- while (raidPtr->openings > 0) {
+ /*
+ * XXX: we check to see if the system is running low on memory,
and
+ * limit activity if it is. Should prevent RAIDframe from
consuming
+ * all available memory, and deadlocking the system. Shouldn't
be
+ * done here, and like this, though. P.Ripke
+ */
+ while (raidPtr->openings > 0 &&
+ !(raidPtr->openings < RAIDOUTSTANDING - 1 &&
+ uvmexp.free < uvmexp.freetarg)) {
RF_UNLOCK_MUTEX(raidPtr->mutex);
/* get the next item, if any, from the queue */
Index: uvm/uvm_page.c
===================================================================
RCS file: /cvsroot/src/sys/uvm/uvm_page.c,v
retrieving revision 1.86
diff -u -d -b -w -r1.86 uvm_page.c
--- uvm/uvm_page.c 2003/04/22 14:28:15 1.86
+++ uvm/uvm_page.c 2003/05/14 22:57:47
@@ -385,7 +385,7 @@
*/
uvmexp.reserve_pagedaemon = 1;
- uvmexp.reserve_kernel = 5;
+ uvmexp.reserve_kernel = 32;
uvmexp.anonminpct = 10;
uvmexp.fileminpct = 10;
uvmexp.execminpct = 5;
Cheers,
--
Paul Ripke
Responsible-Changed-From-To: kern-bug-people->oster
Responsible-Changed-By: oster
Responsible-Changed-When: Fri Jun 20 15:08:08 UTC 2003
Responsible-Changed-Why:
Another RAIDframe bug for me.
State-Changed-From-To: open->feedback
State-Changed-By: tls
State-Changed-When: Thu Apr 1 17:58:42 UTC 2004
State-Changed-Why:
I believe recent changes have fixed, or at least greatly reduced the
likelihood of, this condition. Can you confirm?
From: Greg Oster <oster@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: src/sys/dev/raidframe
Date: Fri, 9 Apr 2004 23:10:17 +0000 (UTC)
Module Name: src
Committed By: oster
Date: Fri Apr 9 23:10:17 UTC 2004
Modified Files:
src/sys/dev/raidframe: rf_aselect.c rf_dag.h rf_dagdegwr.c rf_dagffwr.c
rf_dagutils.c rf_dagutils.h rf_desc.h rf_driver.c rf_netbsd.h
rf_raid.h
Log Message:
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.19 -r1.20 src/sys/dev/raidframe/rf_aselect.c \
src/sys/dev/raidframe/rf_netbsd.h
cvs rdiff -r1.15 -r1.16 src/sys/dev/raidframe/rf_dag.h
cvs rdiff -r1.23 -r1.24 src/sys/dev/raidframe/rf_dagdegwr.c
cvs rdiff -r1.26 -r1.27 src/sys/dev/raidframe/rf_dagffwr.c
cvs rdiff -r1.43 -r1.44 src/sys/dev/raidframe/rf_dagutils.c
cvs rdiff -r1.16 -r1.17 src/sys/dev/raidframe/rf_dagutils.h
cvs rdiff -r1.12 -r1.13 src/sys/dev/raidframe/rf_desc.h
cvs rdiff -r1.98 -r1.99 src/sys/dev/raidframe/rf_driver.c
cvs rdiff -r1.32 -r1.33 src/sys/dev/raidframe/rf_raid.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: feedback->closed
State-Changed-By: oster
State-Changed-When: Fri Apr 9 23:12:28 UTC 2004
State-Changed-Why:
The last huge change I've made should have squished this one for good.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:03 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:03 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_dag.h
Log Message:
Pull up revision 1.16 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.15 -r1.15.2.1 src/sys/dev/raidframe/rf_dag.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:08 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:08 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_dagutils.h
Log Message:
Pull up revision 1.17 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.16 -r1.16.2.1 src/sys/dev/raidframe/rf_dagutils.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:16 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:16 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_dagutils.c
Log Message:
Pull up revision 1.44 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.43 -r1.43.2.1 src/sys/dev/raidframe/rf_dagutils.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:25 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:25 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_dagffwr.c
Log Message:
Pull up revision 1.27 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.26 -r1.26.2.1 src/sys/dev/raidframe/rf_dagffwr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:32 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:32 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_aselect.c rf_netbsd.h
Log Message:
Pull up revision 1.20 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.19 -r1.19.2.1 src/sys/dev/raidframe/rf_aselect.c \
src/sys/dev/raidframe/rf_netbsd.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:37 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:37 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_desc.h
Log Message:
Pull up revision 1.13 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.12 -r1.12.2.1 src/sys/dev/raidframe/rf_desc.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:44 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:44 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_driver.c
Log Message:
Pull up revision 1.99 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.98 -r1.98.2.1 src/sys/dev/raidframe/rf_driver.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:51 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:50 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_raid.h
Log Message:
Pull up revision 1.33 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.32 -r1.32.2.1 src/sys/dev/raidframe/rf_raid.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: pr/20191 CVS commit: [netbsd-2-0] src/sys/dev/raidframe
Date: Sun, 11 Apr 2004 11:19:58 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sun Apr 11 11:19:58 UTC 2004
Modified Files:
src/sys/dev/raidframe [netbsd-2-0]: rf_dagdegwr.c
Log Message:
Pull up revision 1.24 (requested by oster in ticket #123):
These changes complete the effective removal of malloc() from all
write paths within RAIDframe. They also resolve the "panics with
RAID 5 sets with more than 3 components" issue which was present
(briefly) in the commits which were previously supposed to address
the malloc() issue.
With this new code the 5-component RAID 5 set panics are now gone.
It is also now also possible to swap to RAID 5.
The changes made are:
1) Introduce rf_AllocStripeBuffer() and rf_FreeStripeBuffer() to
allocate/free one stripe's worth of space. rf_AllocStripeBuffer() is
used in rf_MapUnaccessedPortionOfStripe() where it is not sufficient to
allocate memory using just rf_AllocBuffer(). rf_FreeStripeBuffer() is
called from rf_FreeRaidAccDesc(), well after the DAG is finished.
2) Add a set of emergency "stripe buffers" to struct RF_Raid_s.
Arrange for their initialization in rf_Configure(). In low-memory
situations these buffers will be returned by rf_AllocStripeBuffer()
and re-populated by rf_FreeStripeBuffer().
3) Move RF_VoidPointerListElem_t *iobufs from the dagHeader into
into struct RF_RaidAccessDesc_s. This is more consistent with the
original code, and will not result in items being freed "too early".
4) Add a RF_RaidAccessDesc_t *desc to RF_DagHeader_s so that we have a
way to find desc->iobufs.
5) Arrange for desc in the DagHeader to be initialized in InitHdrNode().
6) Don't cleanup iobufs in rf_FreeDAG() -- the freeing is now delayed
until rf_FreeRaidAccDesc() (which is how the original code handled the
allocList, and for which there seem to be some subtle, undocumented
assumptions).
7) Rename rf_AllocBuffer2() to be rf_AllocBuffer() and remove the
former rf_AllocBuffer(). Fix all callers of rf_AllocBuffer().
(This was how it was *supposed* to be after the last time these
changes were made, before they were backed out).
8) Remove RF_IOBufHeader and all references to it.
9) Remove desc->cleanupList and all references to it.
Fixes PR#20191
To generate a diff of this commit:
cvs rdiff -r1.23 -r1.23.2.1 src/sys/dev/raidframe/rf_dagdegwr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.