NetBSD Problem Report #38019
From woods@once.weird.com Wed Feb 13 17:39:26 2008
Return-Path: <woods@once.weird.com>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by narn.NetBSD.org (Postfix) with ESMTP id ADBD663B293
for <gnats-bugs@gnats.NetBSD.org>; Wed, 13 Feb 2008 17:39:26 +0000 (UTC)
Message-Id: <m1JPLZp-0018L7C@once.weird.com>
Date: Wed, 13 Feb 2008 12:39:21 -0500 (EST)
From: "Greg A. Woods" <woods@planix.com>
Sender: "Greg A. Woods" <woods@once.weird.com>
Reply-To: "Greg A. Woods" <woods@planix.com>
To: gnats-bugs@gnats.NetBSD.org
Subject: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
X-Send-Pr-Version: 3.95
>Number: 38019
>Category: kern
>Synopsis: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Feb 13 17:40:01 +0000 2008
>Closed-Date: Sun Dec 16 17:44:40 +0000 2012
>Last-Modified: Sun Dec 16 17:44:40 +0000 2012
>Originator: Greg A. Woods
>Release: NetBSD 4.0_STABLE 2008/02/10
>Organization:
Planix, Inc.; Toronto, Ontario; Canada
>Environment:
System: NetBSD 4.0_STABLE GENERIC.MP
Architecture: i386
Machine: i386
>Description:
I've been experiencing regular hangs and unkillable processes on
my Dell PE2650 running NetBSD-4.0_STABLE GENERIC.MP.
The most regular problem is triggered during the big "find" runs
invoked by /etc/daily et al.
The system managed to make it through its nightly cron jobs
without hanging last night and I managed to use it (lightly) for
several hours this morning before problems began to appear. I
had been investigating problems with the apcupsd package and had
unpacked it and built it once or twice, then suddenly the
"extract" phase hung, but the make process was interruptable, so
I tried it again, with the same result. Soon I discovered the
gzcat and tar processes from both attempts were still present,
and they were unkillable.
Interestingly it doesn't seem to be access to the file gzcat is
reading which causes problems. Clearly the second run of
"digest" didn't hang, and manual access with 'cat' and 'dd'
works without hanging too.
When the nightly "find" is one that locks it seems all access to
the same device (and filesystem?) soon causes all (useful)
processes to lock up.
The common denomiator with the oft-nightly hangs and this
currently more minor hang is that some of the stuck processes
are in "vmmapva" and they are unkillable.
This may, or may not, be related to the same causes of the
problem my PR#37993.
One difference between this machine and others I know are
running very similar kernels is that this machine's local
filesystems are all accessed via the built-in Dell PERC/3Di
controller and the aac(4) (and ld(4)) drivers. The aac(4)
driver is known to be rather buggy, the worst problem of which
is that it seems to miss some interrupts, but a regular job
reading a block from the raw /dev/ld0d device seems to wake it
up enough to keep things running normally. See the 'sh' process
running 'dd' in a loop in the info below.
I'll try to capture a crash dump from the system as I reboot it.
I'll also try to find and back-port some of the aac(4) fixes
that were discussed some time ago when I first noted that the
driver is buggy and is als way out of date w.r.t. its original
sources in FreeBSD.
12:07 [2291] # df
Filesystem 512-blocks Used Avail %Cap Mounted on
/dev/ld0a 4032824 2294492 1536692 59% /
/dev/ld0e 10089944 2127724 7457724 22% /var
/dev/ld0f 16130008 10325496 4998012 67% /usr/pkg
/dev/ld0g 231522520 131320032 88626364 59% /rest
mfs:413 2024220 142308 1780704 7% /tmp
kernfs 2 2 0 100% /kern
most:/var/spool/ftp/pub/mirror 8749688 1896624 6415576 22% /var/package-distfiles
12:08 [2292] # mount
/dev/ld0a on / type ffs (local)
/dev/ld0e on /var type ffs (nosuid, nodev, NFS exported, local)
/dev/ld0f on /usr/pkg type ffs (nodev, soft dependencies, local)
/dev/ld0g on /rest type ffs (nosuid, nodev, NFS exported, local)
mfs:413 on /tmp type mfs (synchronous, nosuid, nodev, local)
kernfs on /kern type kernfs (local)
most:/var/spool/ftp/pub/mirror on /var/package-distfiles type nfs (nosuid, nodev)
12:08 [2293] # ps -la
UID PID PPID CPU PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND
1000 12225 10492 0 3 0 604 4 ttyin IWs+ ttyp0 0:00.48 -ksh
1000 3359 3354 0 18 0 568 4 pause IWs ttyp1 0:00.09 -ksh
0 11549 3359 0 3 0 604 588 ttyin I+ ttyp1 0:00.12 ksh
1000 11156 10187 91 3 0 572 584 ttyin Is+ ttyp2 0:00.28 -ksh
1000 10696 10694 0 3 0 580 592 ttyin Is+ ttyp3 0:00.20 -ksh
0 7610 17451 0 -18 0 424 368 vmmapva D ttyp4 0:00.74 /usr/bin/gzcat /var/package-distfiles//apcupsd-3.
0 10067 27688 0 28 0 364 324 - R+ ttyp4 0:00.00 ps -la
1000 12435 29241 1746 18 0 568 4 pause IWs ttyp4 0:00.12 -ksh
0 15005 27796 0 -18 0 424 368 vmmapva D ttyp4 0:00.83 /usr/bin/gzcat /var/package-distfiles//apcupsd-3.
0 15626 27796 0 2 0 424 360 pipecl D ttyp4 0:00.01 /bin/tar -xf -
0 17451 1 0 10 0 472 436 wait I ttyp4 0:00.00 (sh)
0 17645 17451 0 2 0 424 360 pipecl D ttyp4 0:00.00 /bin/tar -xf -
0 27688 12435 0 18 0 684 704 pause S ttyp4 0:00.62 ksh
0 27796 1 0 10 0 472 436 wait I ttyp4 0:00.00 (sh)
1000 532 2485 1258 18 0 568 4 pause IWs ttyp5 0:00.13 -ksh
1000 7340 532 0 2 0 1512 1072 select I+ ttyp5 0:01.18 slogin whome.planix.com (ssh2)
1000 22241 12282 0 3 0 564 560 ttyin Is+ ttyp6 0:00.31 -ksh
1000 8897 6756 0 18 0 568 572 pause Is ttyp7 0:00.12 -ksh
0 27674 8897 0 3 0 604 624 ttyin I+ ttyp7 0:00.08 ksh
0 350 1 0 10 0 456 364 wait S tty00- 1:37.04 sh -c while : ; do dd if=/dev/rld0d of=/dev/null
0 2098 1 0 3 0 256 4 ttyin IWs+ tty00 0:00.07 /usr/libexec/getty default constty
0 1582 1 1348 3 0 256 4 ttyin IWs+ ttyE0 0:00.07 /usr/libexec/getty Pc ttyE0
0 1511 1 0 3 0 436 4 ttyin IWs+ ttyE1 0:00.16 -ksh
0 2103 1 1348 3 0 256 4 ttyin IWs+ ttyE2 0:00.28 /usr/libexec/getty Pc ttyE2
0 2066 1 1348 3 0 256 4 ttyin IWs+ ttyE3 0:00.08 /usr/libexec/getty Pc ttyE3
0 2041 1 1348 3 0 256 4 ttyin IWs+ ttyE4 0:00.08 /usr/libexec/getty Pc ttyE4
0 1980 1 1348 3 0 256 4 ttyin IWs+ ttyE5 0:00.08 /usr/libexec/getty Pc ttyE5
0 1981 1 1348 3 0 256 4 ttyin IWs+ ttyE6 0:00.15 /usr/libexec/getty Pc ttyE6
12:09 [2294] # top
12:10 [2295] # fstat -p 7610
USER CMD PID FD MOUNT INUM MODE SZ|DV R/W
root gzcat 7610 wd /rest 9070540 drwxr-xr-x 512 r
root gzcat 7610 0 / 162324 crw------- ttyp4 rw
root gzcat 7610 1* pipe 0xd7b282bc -> 0xd7b284b0 w
root gzcat 7610 2 / 162324 crw------- ttyp4 rw
root gzcat 7610 3 /var/package-distfiles 121 -rw-r--r-- 4356614 r
root gzcat 7610 5 - - none -
12:16 [2296] # fstat -p 15005
USER CMD PID FD MOUNT INUM MODE SZ|DV R/W
root gzcat 15005 wd /rest 9070539 drwxr-xr-x 0 r
root gzcat 15005 0 / 162324 crw------- ttyp4 rw
root gzcat 15005 1* pipe 0xd7b28258 -> 0xd7b28064 w
root gzcat 15005 2 / 162324 crw------- ttyp4 rw
root gzcat 15005 3 /var/package-distfiles 121 -rw-r--r-- 4356614 r
root gzcat 15005 5 - - none -
12:16 [2297] # kill -9 15626 7610 15005 17645
12:18 [2298] # kill -9 15626 7610 15005 17645
12:18 [2399] # kill -9 15626 7610 15005 17645
12:18 [2300] # fstat -p 15626
USER CMD PID FD MOUNT INUM MODE SZ|DV R/W
root tar 15626 wd /rest 9070539 drwxr-xr-x 0 r
root tar 15626 1 / 162324 crw------- ttyp4 rw
root tar 15626 2 / 162324 crw------- ttyp4 rw
root tar 15626 3 / 160798 crw-rw-rw- tty rw
root tar 15626 4 /rest 9070539 drwxr-xr-x 0 r
root tar 15626 5 - - none -
12:20 [2301] # fstat -p 17645
USER CMD PID FD MOUNT INUM MODE SZ|DV R/W
root tar 17645 wd /rest 9070540 drwxr-xr-x 512 r
root tar 17645 1 / 162324 crw------- ttyp4 rw
root tar 17645 2 / 162324 crw------- ttyp4 rw
root tar 17645 3 / 160798 crw-rw-rw- tty rw
root tar 17645 4 /rest 9070540 drwxr-xr-x 512 r
root tar 17645 5 - - none -
12:18 [2302] #
>How-To-Repeat:
>Fix:
>Release-Note:
>Audit-Trail:
From: "Greg A. Woods; Planix, Inc." <woods@planix.ca>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
Date: Thu, 14 Feb 2008 16:10:21 -0500
I got another crash dump today after the machine had begun to get
wedged, this time triggered by hitting <CTRL-T> on the console:
Please let me know if these crash dumps could be of any use!
kernel: supervisor trap page fault, code=0
Stopped at netbsd:ttyinfo+0xfb: cmpb $0,0x3b4(%eax)
db{0}> trace
ttyinfo(cea66e70,1,0,a03726f,ce9dfeb8) at netbsd:ttyinfo+0xfb
ttyinput_wlock(5b062dd8,44623,ce9dff0c,61,c52a8600) at
netbsd:ttyinput_wlock+0xb
ttyinput(14,cea66e70,4,0,c09c00b8) at netbsd:ttyinput+0x53
comsoft(c52a8600,c0100da1,c0a4e280,0,c0100da1) at netbsd:comsoft+0x108
softintr_dispatch(2,ce9d0010,30,10,80010010) at
netbsd:softintr_dispatch+0xb6
Xsoftserial() at netbsd:Xsoftserial+0x2e
--- interrupt ---
netbsd:cpu_switch+0xab:
db{0}> reboot 0x104
dumping to dev 19,1 offset 10579195
dump 3839 3838 3837 3836 3835 3834 3833 3832 3831 3830 3829 3828 3827
3826 3825
--
Greg A. Woods; Planix, Inc.
<woods@planix.ca>
From: "Greg A. Woods; Planix, Inc." <woods@planix.ca>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
Date: Fri, 22 Feb 2008 16:15:37 -0500
Well in the way these things sometimes go, my machine has been
relatively stable since booting a kernel built with LOCKDEBUG.
Curiously though there are occasional "pauses", and some of the
former problems with processes getting stuck in vmmapva are now
eventually getting unstuck, apparently all by themselves (though
sometimes coincidentally with larger amounts of disk I/O occurring).
Also curiously I began to see the words "write", and sometimes
"rstart" and "rend" appear on the console. Finally after much
searching I found printf()s containing these very poor clues in sys/
arch/x86/x86/ipmi.c. I don't know if they're related to the hangs
and other weirdness, or not. The following patch might make them a
little less obtuse, or at least a little easier to find in the sources:
Index: sys/arch/x86/x86/ipmi.c
===================================================================
RCS file: /cvs/master/m-NetBSD/main/src/sys/arch/x86/x86/ipmi.c,v
retrieving revision 1.4.4.2
diff -u -r1.4.4.2 ipmi.c
--- sys/arch/x86/x86/ipmi.c 19 Dec 2007 19:38:52 -0000 1.4.4.2
+++ sys/arch/x86/x86/ipmi.c 21 Feb 2008 02:53:11 -0000
@@ -527,13 +527,13 @@
int v;
/* Wait for expected flag bits */
- v = bmc_io_wait(sc, _SMIC_FLAG_REG, mask, val, "smicwait");
+ v = bmc_io_wait(sc, _SMIC_FLAG_REG, mask, val, "smicwait"); /
* XXX should that be passing lbl */
if (v < 0)
return (-1);
/* Return current status */
v = bmc_read(sc, _SMIC_CTRL_REG);
- dbg_printf(99, "smic_wait = %.2x\n", v);
+ dbg_printf(99, "smic_wait = %.2x %s\n", v, lbl);
return (v);
}
@@ -581,11 +581,11 @@
int sts, idx;
sts = smic_write_cmd_data(sc, SMS_CC_START_TRANSFER, &data[0]);
- ErrStat(sts != SMS_SC_WRITE_START, "wstart");
+ ErrStat(sts != SMS_SC_WRITE_START, "smic_sendmsg: wstart\n");
for (idx = 1; idx < len - 1; idx++) {
sts = smic_write_cmd_data(sc, SMS_CC_NEXT_TRANSFER,
&data[idx]);
- ErrStat(sts != SMS_SC_WRITE_NEXT, "write");
+ ErrStat(sts != SMS_SC_WRITE_NEXT, "smic_sendmsg: write
\n");
}
sts = smic_write_cmd_data(sc, SMS_CC_END_TRANSFER, &data[idx]);
if (sts != SMS_SC_WRITE_END) {
@@ -607,14 +607,14 @@
return (-1);
sts = smic_write_cmd_data(sc, SMS_CC_START_RECEIVE, NULL);
- ErrStat(sts != SMS_SC_READ_START, "rstart");
+ ErrStat(sts != SMS_SC_READ_START, "smic_recvmsg: rstart\n");
for (idx = 0;; ) {
sts = smic_read_data(sc, &data[idx++]);
if (sts != SMS_SC_READ_START && sts !=
SMS_SC_READ_NEXT)
break;
smic_write_cmd_data(sc, SMS_CC_NEXT_RECEIVE, NULL);
}
- ErrStat(sts != SMS_SC_READ_END, "rend");
+ ErrStat(sts != SMS_SC_READ_END, "smic_recvmsg: rend\n");
*len = idx;
--
Greg A. Woods; Planix, Inc.
<woods@planix.ca>
From: "Greg A. Woods" <woods@planix.com>
To: NetBSD GNATS <gnats-bugs@NetBSD.org>
Cc: NetBSD Kernel Technical Discussion List <tech-kern@NetBSD.org>, NetBSD/i386 Discussion List <port-i386@NetBSD.org>
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
Date: Fri, 04 Apr 2008 15:16:31 -0400
--pgp-sign-Multipart_Fri_Apr__4_15:15:53_2008-1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable
I think I may have been mistaking two different problems on my Dell
server. I think my ongoing problems with lockups may actually be due to
possible starvation of kernel virtual space on large-memory (i386)
systems, while other crashes (PR# 38246, for example) are possibly
locking and/or SA related.
I was going though an old 3.0 kernel config for a client machine,
preparing to build it on the netbsd-4 and noted that I had applied the
3.0 LAST_MINUTE hack to adjust KERNBASE_LOCORE as that machine has 4GB
of RAM. Remembering this, noting that my PE2650 also has 4GB of RAM,
and given that the last few hangs always show processes stuck in
vmmapva, I've begun to wonder if maybe this hack might still be
necessary in netbsd-4 too.
I'm rebuilding my netbsd-4 and wrstuden-fixsa kernels now with the
following in hopes it helps my machine make it safely through the
nightly cron jobs.
I would welcome any comments on this issue, especially suggestions as to
whether or not I might be on the right track here....
# Machines configured with more than 2 GB of physical memory may
# experience hangs due to running out of kernel virtual space, which
# is needed for kernel data structures such as file system metadata
# buffers. Various kernel subsystems (including the file system
# metadata cache) automatically adjust their memory usage based on the
# amount of physical memory in the system, but this method
# over-allocates the available kernel virtual space (1 GB) when the
# amount of physical memory greatly exceeds the amount of kernel
# virtual space.
#=20
# The only reliable workaround currently known for this problem is to
# increase the amount of virtual space available to the kernel by
# adding the following line to the kernel config file.
#=20
# This will increase the kernel virtual space to 2 GB, but this has
# the drawback that the application virtual space is decreased from 3
# GB to 2 GB (since the 32-bit address space supported by the hardware
# must be shared between the kernel and application on this
# architecture).
#
#options KERNBASE=3D"0xc0000000UL" # start of kernel virtual space
options KERNBASE=3D"0x80000000UL"
--=20
Greg A. Woods
Planix, Inc.
<woods@planix.com> +1 416 489-5852 x122 http://www.planix.com/
--pgp-sign-Multipart_Fri_Apr__4_15:15:53_2008-1
Content-Type: application/pgp-signature
Content-Transfer-Encoding: 7bit
-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
MessageID: NSl8FXdY2IO6In60dddHziKnfh5LoK2c
iQA/AwUBR/Z+j2Z9cbd4v/R/EQJYxwCfflIQnfX2wogWSzX3f4nRjmXvZO8AoI1K
e6Ou066f/O0d3moXgmr5DVxL
=FkAm
-----END PGP SIGNATURE-----
--pgp-sign-Multipart_Fri_Apr__4_15:15:53_2008-1--
From: "Greg A. Woods" <woods@planix.com>
To: NetBSD GNATS <gnats-bugs@NetBSD.org>
Cc: NetBSD Kernel Technical Discussion List <tech-kern@NetBSD.org>,
NetBSD/i386 Discussion List <port-i386@NetBSD.org>,
YAMAMOTO Takashi <yamt@NetBSD.org>
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
Date: Sat, 05 Apr 2008 12:56:13 -0400
At Fri, 04 Apr 2008 15:16:31 -0400, Me-planix.com wrote:
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
>
> I was going though an old 3.0 kernel config for a client machine,
> preparing to build it on the netbsd-4 and noted that I had applied the
> 3.0 LAST_MINUTE hack to adjust KERNBASE_LOCORE as that machine has 4GB
> of RAM. Remembering this, noting that my PE2650 also has 4GB of RAM,
> and given that the last few hangs always show processes stuck in
> vmmapva, I've begun to wonder if maybe this hack might still be
> necessary in netbsd-4 too.
Is it possible that the following changes may correct the possible
lockups caused by KVA exhaustion on large-RAM machines without requiring
hacks to KERNBASE and/or manual manipulation of the various tunable
limits such as vm.bufmem_hiwater? If so I would like to request that
these changes (and any others which are pre- or co-requisite) be pulled
up to netbsd-4. If desired I can pre-test the pull-up(s) on my machine.
FYI, on my machine right now, after it has once again survived a near
miraculous three days and nights without locking up while running the
wrstuden-fixsa kernel (without any change to KERNBASE), I observe the
following:
12:01 [2051] # sysctl vm
vm.loadavg: 6.21 3.31 3.56
vm.nkmempages = 32768
vm.maxslp = 20
vm.uspace = 12288
vm.idlezero = 0
vm.anonmin = 40
vm.filemin = 5
vm.execmin = 40
vm.anonmax = 80
vm.filemax = 50
vm.execmax = 50
vm.inactivepct = 33
vm.bufcache = 15
vm.bufmem = 333012992
vm.bufmem_lowater = 75486720
vm.bufmem_hiwater = 603893760
----------------------------
revision 1.193
date: 2008/03/25 23:21:43; author: yamt; state: Exp; lines: +9 -3
- for some ports, especially for ones without pmap_growkernel,
buf_memcalc is used by bootstrap as well. fix NULL dereference for them.
- limit kva usage for each cache to 20% of vm_map. XXX a bit arbitrary.
- add a comment.
----------------------------
revision 1.192
date: 2008/03/23 10:39:52; author: yamt; state: Exp; lines: +9 -9
when calculating some cache sizes, consider the amount of available kva.
PR/33185.
----------------------------
Index: sys/kern/vfs_bio.c
===================================================================
RCS file: /cvs/master/m-NetBSD/main/src/sys/kern/vfs_bio.c,v
retrieving revision 1.191
retrieving revision 1.193
diff -u -r1.191 -r1.193
--- sys/kern/vfs_bio.c 23 Mar 2008 10:33:15 -0000 1.191
+++ sys/kern/vfs_bio.c 25 Mar 2008 23:21:43 -0000 1.193
@@ -1,4 +1,4 @@
-/* $NetBSD: vfs_bio.c,v 1.191 2008/03/23 10:33:15 yamt Exp $ */
+/* $NetBSD: vfs_bio.c,v 1.193 2008/03/25 23:21:43 yamt Exp $ */
/*-
* Copyright (c) 2007 The NetBSD Foundation, Inc.
@@ -114,7 +114,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: vfs_bio.c,v 1.191 2008/03/23 10:33:15 yamt Exp $");
+__KERNEL_RCSID(0, "$NetBSD: vfs_bio.c,v 1.193 2008/03/25 23:21:43 yamt Exp $");
#include "fs_ffs.h"
#include "opt_bufcache.h"
@@ -388,6 +388,10 @@
}
}
+/*
+ * note that for some ports this is used by pmap bootstrap code to
+ * determine kva size.
+ */
u_long
buf_memcalc(void)
{
@@ -413,7 +417,9 @@
printf("forcing bufcache %d -> 95", bufcache);
bufcache = 95;
}
- n = physmem / 100 * bufcache;
+ n = calc_cache_size(buf_map, bufcache,
+ (buf_map != kernel_map) ? 100 : BUFCACHE_VA_MAXPCT)
+ / PAGE_SIZE;
}
n <<= PAGE_SHIFT;
@@ -437,12 +443,6 @@
mutex_init(&buffer_lock, MUTEX_DEFAULT, IPL_NONE);
cv_init(&needbuffer_cv, "needbuf");
- /*
- * Initialize buffer cache memory parameters.
- */
- bufmem = 0;
- buf_setwm();
-
if (bufmem_valimit != 0) {
vaddr_t minaddr = 0, maxaddr;
buf_map = uvm_km_suballoc(kernel_map, &minaddr, &maxaddr,
@@ -452,6 +452,12 @@
} else
buf_map = kernel_map;
+ /*
+ * Initialize buffer cache memory parameters.
+ */
+ bufmem = 0;
+ buf_setwm();
+
/* On "small" machines use small pool page sizes where possible */
use_std = (physmem < atop(16*1024*1024));
--
Greg A. Woods
Planix, Inc.
<woods@planix.com> +1 416 489-5852 x122 http://www.planix.com/
From: "Greg A. Woods" <woods@planix.com>
To: NetBSD GNATS <gnats-bugs@NetBSD.org>
Cc: NetBSD Kernel Bug People <kern-bug-people@netbsd.org>,
NetBSD GNATS Administrator <gnats-admin@NetBSD.org>
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
Date: Fri, 18 Apr 2008 18:51:35 -0400
At Sat, 5 Apr 2008 17:00:05 +0000 (UTC), Me-planix.com wrote:
Subject: Re: kern/38019: some kind of undetected deadlock slowly kills NetBSD-4.0_STABLE GENERIC.MP
>
> Is it possible that the following changes may correct the possible
> lockups caused by KVA exhaustion on large-RAM machines without requiring
> hacks to KERNBASE and/or manual manipulation of the various tunable
> limits such as vm.bufmem_hiwater? If so I would like to request that
> these changes (and any others which are pre- or co-requisite) be pulled
> up to netbsd-4. If desired I can pre-test the pull-up(s) on my machine.
I still haven't heard from anyone about whether or not the following
changes are the only ones necessary to pull this fix up in its entirety
to the netbsd-4 branch....
> ----------------------------
> revision 1.193
> date: 2008/03/25 23:21:43; author: yamt; state: Exp; lines: +9 -3
> - for some ports, especially for ones without pmap_growkernel,
> buf_memcalc is used by bootstrap as well. fix NULL dereference for them.
> - limit kva usage for each cache to 20% of vm_map. XXX a bit arbitrary.
> - add a comment.
> ----------------------------
> revision 1.192
> date: 2008/03/23 10:39:52; author: yamt; state: Exp; lines: +9 -9
> when calculating some cache sizes, consider the amount of available kva.
> PR/33185.
> ----------------------------
My system has now been running a wrstuden-fixsa kernel with an
outstanding uptime of 16 days. Today for the first time some gzcat
processes _temporarily_ got stuck in the vmmapva state in the same way
as I originally reported when opening this PR.
At that time sysctl vm.bufmem was showing over 300 MB allocated, and
vmstat -m was showing a utilization of just over 82%.
However somehow things got unstuck and now I see vm.bufmem and the
"utilization" reported by vmstat -m both steadily decline over a period
of many minutes.
Note in particular the "Fail" column in the "Memory resource pool
statistics" of the current "vmstat -m" output below. Those numbers for
buf*k increased while the gzcat processes where stuck but haven't
increased since.
Two things about all of this stand out to me:
1. Each time that I've seen the problem happen while I've been using the
system there was some involvement of NFS mounts (and though the
nightly cron jobs do their best to avoid accessing NFS filesystems,
there's no doubt they do at least some NFS operations).
2. Sometimes the problem clears itself up within a few minutes.
Sometimes it doesn't even after many hours. However since running
the wrstuden-fixsa kernel the system has only locked up "permanently"
once and this past 16 days uptime is the longest it has enjoyed in
many months.
I think there's probably some combination of problems causing the
lockup. One is perhaps the over-utilization of kva, and the other is
that sometimes whatever needs to run to free available kva doesn't get a
chance. I don't know if the wrstuden-fixsa changes are responsible, or
if it's just a pure co-incidence. However if the past patterns of
things getting slowly wedged repeat themselves (especially with the
nightly cron jobs) then that might suggest the wrstuden-fixsa changes
have no effect and that it's simply a matter of kva sometimes running
out so completely that even whatever threads could run to free some
space are unable to run. In any case I think it would be very wise to
do something better to assure that bufmem and other cache allocations
don't jeopardize the system under the default configuration.
$ /sbin/sysctl vm
vm.loadavg: 1.85 1.77 1.15
vm.nkmempages = 32768
vm.maxslp = 20
vm.uspace = 12288
vm.idlezero = 0
vm.anonmin = 40
vm.filemin = 5
vm.execmin = 40
vm.anonmax = 80
vm.filemax = 50
vm.execmax = 50
vm.inactivepct = 33
vm.bufcache = 15
vm.bufmem = 199446528
vm.bufmem_lowater = 75486720
vm.bufmem_hiwater = 603893760
$ vmstat -m
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 1147 133 687612 1280 0
32 355 157 25733710 640 0
64 1255 2201 4308715 320 2311066
128 1559 9 25537123 160 0
256 105 71 13693963 80 0
512 78 50 86889 40 17320
1024 1163 5 43242 20 0
2048 289 9 12761311 10 0
4096 41 9 200943141 5 119140350
8192 7 3 333 5 0
16384 8 0 467 5 0
32768 4 0 13 5 0
65536 1 0 1 5 0
262144 6 0 6 5 0
524288 10 0 10 5 0
Memory usage type by bucket size
Size Type(s)
16 acpi, USB, namecache, soname, shm, pipe, sysctldata, routetbl, temp,
pcb, devbuf, ktrace, exec, file desc, VM swap, UVM aobj, kernfs mount,
xform
32 acpi, USB, ifmedia, ether_multi, ifaddr, soname, blist, fragtbl,
routetbl, temp, pcb, devbuf, ktrace, exec, kevent, VM swap, UVM aobj,
MFS node, aacbuf, RAIDframe, tcpcongctl, xform
64 pci_link, acpi, USB, vnodes, soname, pipe, proc, routetbl, temp, pcb,
DMA map, devbuf, ktrace, UVM aobj, NFS diroff, NFS req
128 prop dictionary, pci_link, acpi, USB device, USB, ifaddr, soname,
ttys, sysctlnode, Export Host, routetbl, temp, pcb, devbuf, ktrace,
kevent, VM swap, UVM aobj
256 acpi, ifaddr, vnodes, sysctldata, proc, UFS mount, routetbl, temp,
DMA map, devbuf, ktrace, exec, kevent, ioctlops, file desc, UVM aobj,
newblk, NFS daemon, NFS diroff, nfs_export
512 acpi, vnodes, proc, temp, pcb, devbuf, ktrace, ioctlops, file desc,
VM page, NFS srvsock, NFS daemon, aacbuf, crypto
1024 acpi, USB, ttys, sysctlnode, proc, temp, DMA map, devbuf, exec,
kevent, file desc, file, VM map, UVM aobj, NFS mount, NQNFS Lease
2048 USB device, ifaddr, sysctlnode, proc, in_multi, UFS mount, temp,
DMA map, devbuf, ktrace, exec, file desc
4096 mount, sysctlnode, proc, UFS mount, temp, devbuf, ktrace, file desc,
RAIDframe
8192 acpi, sysctlnode, temp, devbuf, file desc
16384 vmem, sysctlnode, UFS mount, temp, devbuf
32768 temp, devbuf, aacbuf
65536 temp
262144 UFS mount, devbuf, pagedep
524288 namecache, blist, UFS mount, UFS quota, inodedep, NFS node,
MSDOSFS mount, ISOFS mount
Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
prop dictionary 3 1K 1K 78644K 3 0 0 128
pci_link 3 1K 1K 78644K 3 0 0 64,128
acpi 2119 226K 229K 78644K 5978 0 0 16,32,64,128,256,512,1024,8192
USB device 4 7K 7K 78644K 4 0 0 128,2048
USB 13 1K 2K 78644K 14 0 0 16,32,64,128,1024
ifmedia 16 1K 1K 78644K 16 0 0 32
ether_multi 2 1K 1K 78644K 2 0 0 32
ifaddr 10 4K 4K 78644K 10 0 0 32,128,256,2048
mount 7 28K 28K 78644K 8 0 0 4096
vnodes 75 11K 146K 78644K 778495 0 0 64,256,512
namecache 4 3073K 3073K 78644K 4 0 0 16,524288
soname 6 1K 1K 78644K 94377 0 0 16,32,64,128
ttys 85 56K 56K 78644K 85 0 0 128,1024
shm 0 0K 1K 78644K 24 0 0 16
pipe 0 0K 1K 78644K 20202 0 0 16,64
vmem 4 56K 56K 78644K 4 0 0 16384
blist 2 1097K 1097K 78644K 2 0 0 32,524288
sysctldata 1 1K 1K 78644K 3 0 0 16,256
sysctlnode 88 124K 124K 78644K 111 0 0 128,1024,2048,4096,8192,16384
proc 10 6K 8K 78644K 14 0 0 64,256,512,1024,2048,4096
in_multi 1 2K 2K 78644K 1 0 0 2048
Export Host 2 1K 1K 78644K 2 0 0 128
UFS mount 20 2092K 2092K 78644K 21 0 0 256,2048,4096,16384,262144,524288
fragtbl 0 0K 1K 78644K 310 0 0 32
routetbl 79 5K 5K 78644K 745 0 0 16,32,64,128,256
temp 22 66K 120K 78644K 255943798 0 0 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
pcb 45 6K 7K 78644K 94799 0 0 16,32,64,128,512
DMA map 1678 1562K 1562K 78644K 1678 0 0 64,256,1024,2048
devbuf 1520 452K 452K 78644K 791912 0 0 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,262144
ktrace 0 0K 31K 78644K 482 0 0 16,32,64,128,256,512,2048,4096
exec 1 1K 17K 78644K 25514567 0 0 16,32,256,1024,2048
kevent 9 4K 6K 78644K 154887 0 0 32,128,256,1024
ioctlops 0 0K 1K 78644K 39 0 0 256,512
file desc 12 3K 33K 78644K 392919 0 0 16,256,512,1024,2048,4096,8192
file 0 0K 1K 78644K 4 0 0 1024
VM swap 3 1K 1K 78644K 3 0 0 16,32,128
VM page 1 1K 1K 78644K 1 0 0 512
VM map 4 4K 4K 78644K 4 0 0 1024
UVM aobj 2 2K 2K 78644K 26 0 0 16,32,64,128,256,1024
UFS quota 1 1024K 1024K 78644K 1 0 0 524288
MFS node 1 1K 1K 78644K 1 0 0 32
newblk 1 1K 1K 78644K 1 0 0 256
inodedep 1 1024K 1024K 78644K 1 0 0 524288
pagedep 1 256K 256K 78644K 1 0 0 262144
NFS mount 1 1K 1K 78644K 1 0 0 1024
NFS srvsock 1 1K 1K 78644K 1 0 0 512
NFS daemon 5 3K 3K 78644K 5 0 0 256,512
NFS diroff 9 1K 1K 78644K 28 0 0 64,256
NFS req 0 0K 1K 78644K 667 0 0 64
NQNFS Lease 1 1K 1K 78644K 1 0 0 1024
NFS node 1 1024K 1024K 78644K 1 0 0 524288
nfs_export 6 2K 2K 78644K 6 0 0 256
kernfs mount 1 1K 1K 78644K 1 0 0 16
MSDOSFS mount 1 512K 512K 78644K 1 0 0 524288
ISOFS mount 1 1024K 1024K 78644K 1 0 0 524288
aacbuf 132 53K 53K 78644K 132 0 0 32,512,32768
RAIDframe 10 37K 37K 78644K 10 0 0 32,4096
tcpcongctl 2 1K 1K 78644K 2 0 0 32
crypto 1 1K 1K 78644K 1 0 0 512
xform 0 0K 1K 78644K 128 0 0 16,32
Memory totals: In Use Free Requests
13841K 272K 283796548
Memory resource pool statistics
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
extent 20 255 0 251 1 0 1 1 0 inf 0
nfsrvdescpl 180 48663 0 48663 707 707 0 1 0 inf 0
nfsreqcachepl 36 64 0 0 1 0 1 1 0 inf 0
pnbufpl 1024 39005 0 38999 8341 8339 2 4 0 inf 0
sigapl 2052 12786185 0 12786100 4588 4503 85 268 0 inf 0
mclpl 2048 964 0 661 356 188 168 196 16 512 6
mbpl 256 1940 0 1597 33 5 28 31 1 inf 0
buf16k 16384 1042215 490710 1032983 23242 19333 3909 5270 1 1 0
buf8k 8192 55871 156066 55812 2491 2483 8 23 1 1 0
buf4k 4096 138168 148124 137993 2527 2516 11 23 1 1 0
buf2k 2048 1646417 21540 1633857 8421 7536 885 2024 1 1 0
buf1k 1024 7 0 7 1 0 1 1 1 1 1
kmem-4 4 202527 0 202182 32 28 4 8 0 inf 0
kmem-8 8 19483 0 19407 13 11 2 2 0 inf 0
kmem-12 12 10135 0 10126 5 4 1 3 0 inf 0
kmem-16 16 17109 0 17088 19 18 1 2 0 inf 0
kmem-20 20 91076 0 91073 11 10 1 4 0 inf 0
kmem-24 24 93193 0 93181 146 145 1 29 0 inf 0
kmem-28 28 92166 0 92154 39 38 1 6 0 inf 0
kmem-32 32 11355 0 11349 96 95 1 10 0 inf 0
kmem-36 36 9281 0 9278 41 40 1 6 0 inf 0
kmem-40 40 8669 0 8666 31 30 1 6 0 inf 0
kmem-44 44 8745 0 8745 189 188 1 12 0 inf 1
kmem-48 48 8957 0 8944 190 188 2 9 0 inf 0
kmem-56 56 7580 0 7553 152 147 5 20 0 inf 0
kmem-64 64 22170 0 21775 467 409 58 120 0 inf 0
kmem-72 72 5582 0 5552 88 83 5 9 0 inf 0
kmem-84 84 5698 0 5689 188 186 2 7 0 inf 0
kmem-100 100 4875 0 4872 59 56 3 6 0 inf 0
kmem-128 128 4486 0 4447 143 133 10 13 0 inf 0
propdict 112 63 0 0 2 0 2 2 0 inf 0
pdict32 64 3 0 0 1 0 1 1 0 inf 0
pdict16 48 4 0 0 1 0 1 1 0 inf 0
propnmbr 36 5 0 0 1 0 1 1 0 inf 0
ataspl 56 8 0 8 1 1 0 1 0 inf 0
rtentpl 136 387 0 344 2 0 2 2 0 inf 0
vnodepl 236 11140446 0 10893458 20073 3 20070 20070 0 inf 0
lockfpl 52 348185 0 348161 1 0 1 1 0 inf 0
ncachepl 84 392608731 0 392591797 241699 240915 784 6661 0 inf 7
biopl 148 35858714 0 35858713 132 118 14 17 0 inf 13
bufpl 148 1025552 0 1003526 3621 1373 2248 2635 0 inf 1
sockpl 216 110913 0 110801 43 36 7 17 0 inf 0
ttypl 288 17 0 0 2 0 2 2 0 inf 0
shmmp 12 24 0 24 1 1 0 1 0 inf 0
pipepl 124 495934 0 495881 69 67 2 6 0 inf 0
vmembtpl 28 15344 0 12089 39 1 38 38 0 inf 0
ptimerspl 140 395 0 371 1 0 1 1 0 inf 0
ptimerpl 124 396 0 371 1 0 1 1 0 inf 0
ksiginfo 44 23165357 0 23165357 1 0 1 1 0 inf 1
siginfo 128 88 0 88 44 44 0 1 0 inf 0
savppl 80 1 0 0 1 0 1 1 0 inf 0
sastackpl 24 16 0 0 1 0 1 1 0 inf 0
saupcpl 1612 518063 0 518062 1 0 1 1 0 inf 0
sadatapl 64 1 0 0 1 0 1 1 0 inf 0
sessionpl 56 6046 0 5996 1 0 1 1 0 inf 0
rusgepl 72 12786101 0 12786101 30586 30585 1 1 0 inf 1
pstatspl 224 12786205 0 12786101 151 144 7 16 0 inf 0
plimitpl 200 6462 0 6441 34 32 2 2 0 inf 0
pgrppl 16 91855 0 91795 1 0 1 1 0 inf 0
procpl 492 12786205 0 12786101 411 395 16 36 0 inf 0
lwppl 172 12786208 0 12786101 133 127 6 13 0 inf 0
ktepl 56 481 0 481 2 2 0 2 0 inf 0
knotepl 72 871 0 839 1 0 1 1 0 inf 0
kqueuepl 284 34 0 31 1 0 1 1 0 inf 0
fdescpl 308 12786185 0 12786100 200 192 8 21 0 inf 0
cwdipl 40 12786185 0 12786100 73 71 2 3 0 inf 0
filepl 84 201266242 0 201265947 425 418 7 11 0 inf 0
kauthcredpl 124 546686 0 546655 12 11 1 2 0 inf 0
rndsample 528 1676876 0 1676871 1 0 1 1 0 inf 0
ractx 24 13982452 0 13865253 3163 1277 1886 1999 0 inf 7
vmmpepl 88 204611349 0 204610029 603 570 33 59 0 inf 3
vmsppl 336 12786184 0 12786099 227 218 9 23 0 inf 0
aobjpl 80 25 0 24 1 0 1 1 0 inf 0
anonpl 40 1629774 0 1515697 4884 3692 1192 2013 0 inf 0
amappl 72 97109274 0 97108389 142 123 19 31 0 inf 0
ufsdirpl 264 11035341 0 11035341 3951 3950 1 1 0 inf 1
dino1pl 128 284952247 0 284705279 11282 277 11005 11006 0 inf 0
ffsinopl 240 284952247 0 284705279 22432 1127 21305 21324 0 inf 1
dirrempl 44 360 0 360 5 5 0 2 0 inf 0
mkdirpl 32 10 0 10 3 3 0 1 0 inf 0
diraddpl 40 366 0 366 5 5 0 2 0 inf 0
freefilepl 40 206 0 206 4 4 0 1 0 inf 0
freeblkspl 176 162 0 162 10 10 0 4 0 inf 0
allocindirpl 64 1534 0 1534 26 26 0 12 0 inf 0
indirdeppl 32 25 0 25 3 3 0 1 0 inf 0
allocdirectpl 80 509 0 509 11 11 0 5 0 inf 0
bmsafemappl 36 17 0 17 3 3 0 1 0 inf 0
newblkpl 36 2043 0 2043 5 5 0 1 0 inf 0
inodedeppl 92 410 0 410 13 13 0 5 0 inf 0
pagedeppl 72 43 0 43 4 4 0 1 0 inf 0
sdpcpool 148 2013 0 2013 11 11 0 5 0 inf 0
nfsvapl 112 79 0 77 1 0 1 1 0 inf 0
nfsnodepl 388 79 0 77 2 1 1 2 0 inf 0
tcpcbpl 392 6037 0 5984 69 63 6 24 0 inf 0
sackholepl 20 22 0 22 4 4 0 1 0 inf 0
synpl 172 3096 0 3096 9 9 0 1 0 inf 0
tcpipqepl 44 6785 0 6785 14 13 1 1 0 inf 1
ipqepl 44 4181 0 4181 4 4 0 1 0 inf 0
inmltpl 32 2 0 0 1 0 1 1 0 inf 0
inpcbpl 108 16124 0 16047 18 15 3 8 0 inf 0
kvakmem 4096 375 0 269 5 0 5 5 0 inf 0
kvakernel 4096 393802 0 335000 2080 52 2028 2049 0 inf 0
pdppl 4096 36177 0 36089 35716 35628 88 269 0 inf 0
pmappl 100 12786184 0 12786099 108 105 3 7 0 inf 0
pcgpool 140 360542 0 360501 778 770 8 220 0 inf 0
phpool-128 164 32 0 28 1 0 1 1 0 inf 0
phpool-64 100 18 0 15 1 0 1 1 0 inf 0
phpool-32 68 2300 0 263 36 0 36 36 0 inf 0
phpool-16 52 836 0 820 1 0 1 1 0 inf 0
phpool-8 44 2096 0 2004 2 0 2 2 0 inf 0
phpool-4 40 1433 0 1402 1 0 1 1 0 inf 0
phpool-0 36 81131 0 76030 57 1 56 57 0 inf 0
In use 333593K, total allocated 569462K; utilization 58.6%
After about 20 minutes of much lighter activity things read as:
$ /sbin/sysctl vm
vm.loadavg: 0.04 0.55 0.87
vm.nkmempages = 32768
vm.maxslp = 20
vm.uspace = 12288
vm.idlezero = 0
vm.anonmin = 40
vm.filemin = 5
vm.execmin = 40
vm.anonmax = 80
vm.filemax = 50
vm.execmax = 50
vm.inactivepct = 33
vm.bufcache = 15
vm.bufmem = 179273728
vm.bufmem_lowater = 75486720
vm.bufmem_hiwater = 603893760
$ vmstat -m
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 1147 133 688516 1280 0
32 355 157 25769331 640 0
64 1251 2205 4314672 320 2313441
128 1559 9 25573073 160 0
256 105 71 13712180 80 0
512 78 50 86889 40 17320
1024 1163 5 43251 20 0
2048 289 9 12779142 10 0
4096 41 9 202547117 5 120744322
8192 7 3 333 5 0
16384 8 0 479 5 0
32768 4 0 13 5 0
65536 1 0 1 5 0
262144 6 0 6 5 0
524288 10 0 10 5 0
Memory usage type by bucket size
Size Type(s)
16 acpi, USB, namecache, soname, shm, pipe, sysctldata, routetbl, temp,
pcb, devbuf, ktrace, exec, file desc, VM swap, UVM aobj, kernfs mount,
xform
32 acpi, USB, ifmedia, ether_multi, ifaddr, soname, blist, fragtbl,
routetbl, temp, pcb, devbuf, ktrace, exec, kevent, VM swap, UVM aobj,
MFS node, aacbuf, RAIDframe, tcpcongctl, xform
64 pci_link, acpi, USB, vnodes, soname, pipe, proc, routetbl, temp, pcb,
DMA map, devbuf, ktrace, UVM aobj, NFS diroff, NFS req
128 prop dictionary, pci_link, acpi, USB device, USB, ifaddr, soname,
ttys, sysctlnode, Export Host, routetbl, temp, pcb, devbuf, ktrace,
kevent, VM swap, UVM aobj
256 acpi, ifaddr, vnodes, sysctldata, proc, UFS mount, routetbl, temp,
DMA map, devbuf, ktrace, exec, kevent, ioctlops, file desc, UVM aobj,
newblk, NFS daemon, NFS diroff, nfs_export
512 acpi, vnodes, proc, temp, pcb, devbuf, ktrace, ioctlops, file desc,
VM page, NFS srvsock, NFS daemon, aacbuf, crypto
1024 acpi, USB, ttys, sysctlnode, proc, temp, DMA map, devbuf, exec,
kevent, file desc, file, VM map, UVM aobj, NFS mount, NQNFS Lease
2048 USB device, ifaddr, sysctlnode, proc, in_multi, UFS mount, temp,
DMA map, devbuf, ktrace, exec, file desc
4096 mount, sysctlnode, proc, UFS mount, temp, devbuf, ktrace, file desc,
RAIDframe
8192 acpi, sysctlnode, temp, devbuf, file desc
16384 vmem, sysctlnode, UFS mount, temp, devbuf
32768 temp, devbuf, aacbuf
65536 temp
262144 UFS mount, devbuf, pagedep
524288 namecache, blist, UFS mount, UFS quota, inodedep, NFS node,
MSDOSFS mount, ISOFS mount
Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
prop dictionary 3 1K 1K 78644K 3 0 0 128
pci_link 3 1K 1K 78644K 3 0 0 64,128
acpi 2119 226K 229K 78644K 5978 0 0 16,32,64,128,256,512,1024,8192
USB device 4 7K 7K 78644K 4 0 0 128,2048
USB 13 1K 2K 78644K 14 0 0 16,32,64,128,1024
ifmedia 16 1K 1K 78644K 16 0 0 32
ether_multi 2 1K 1K 78644K 2 0 0 32
ifaddr 10 4K 4K 78644K 10 0 0 32,128,256,2048
mount 7 28K 28K 78644K 8 0 0 4096
vnodes 73 11K 146K 78644K 783925 0 0 64,256,512
namecache 4 3073K 3073K 78644K 4 0 0 16,524288
soname 6 1K 1K 78644K 94527 0 0 16,32,64,128
ttys 85 56K 56K 78644K 85 0 0 128,1024
shm 0 0K 1K 78644K 24 0 0 16
pipe 0 0K 1K 78644K 20213 0 0 16,64
vmem 4 56K 56K 78644K 4 0 0 16384
blist 2 1097K 1097K 78644K 2 0 0 32,524288
sysctldata 1 1K 1K 78644K 3 0 0 16,256
sysctlnode 88 124K 124K 78644K 111 0 0 128,1024,2048,4096,8192,16384
proc 10 6K 8K 78644K 14 0 0 64,256,512,1024,2048,4096
in_multi 1 2K 2K 78644K 1 0 0 2048
Export Host 2 1K 1K 78644K 2 0 0 128
UFS mount 20 2092K 2092K 78644K 21 0 0 256,2048,4096,16384,262144,524288
fragtbl 0 0K 1K 78644K 310 0 0 32
routetbl 79 5K 5K 78644K 747 0 0 16,32,64,128,256
temp 22 66K 120K 78644K 257619550 0 0 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536
pcb 43 6K 7K 78644K 94949 0 0 16,32,64,128,512
DMA map 1678 1562K 1562K 78644K 1678 0 0 64,256,1024,2048
devbuf 1520 452K 452K 78644K 793129 0 0 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,262144
ktrace 0 0K 31K 78644K 482 0 0 16,32,64,128,256,512,2048,4096
exec 1 1K 17K 78644K 25550206 0 0 16,32,256,1024,2048
kevent 9 4K 6K 78644K 154998 0 0 32,128,256,1024
ioctlops 0 0K 1K 78644K 39 0 0 256,512
file desc 12 3K 33K 78644K 392922 0 0 16,256,512,1024,2048,4096,8192
file 0 0K 1K 78644K 4 0 0 1024
VM swap 3 1K 1K 78644K 3 0 0 16,32,128
VM page 1 1K 1K 78644K 1 0 0 512
VM map 4 4K 4K 78644K 4 0 0 1024
UVM aobj 2 2K 2K 78644K 26 0 0 16,32,64,128,256,1024
UFS quota 1 1024K 1024K 78644K 1 0 0 524288
MFS node 1 1K 1K 78644K 1 0 0 32
newblk 1 1K 1K 78644K 1 0 0 256
inodedep 1 1024K 1024K 78644K 1 0 0 524288
pagedep 1 256K 256K 78644K 1 0 0 262144
NFS mount 1 1K 1K 78644K 1 0 0 1024
NFS srvsock 1 1K 1K 78644K 1 0 0 512
NFS daemon 5 3K 3K 78644K 5 0 0 256,512
NFS diroff 9 1K 1K 78644K 28 0 0 64,256
NFS req 0 0K 1K 78644K 667 0 0 64
NQNFS Lease 1 1K 1K 78644K 1 0 0 1024
NFS node 1 1024K 1024K 78644K 1 0 0 524288
nfs_export 6 2K 2K 78644K 6 0 0 256
kernfs mount 1 1K 1K 78644K 1 0 0 16
MSDOSFS mount 1 512K 512K 78644K 1 0 0 524288
ISOFS mount 1 1024K 1024K 78644K 1 0 0 524288
aacbuf 132 53K 53K 78644K 132 0 0 32,512,32768
RAIDframe 10 37K 37K 78644K 10 0 0 32,4096
tcpcongctl 2 1K 1K 78644K 2 0 0 32
crypto 1 1K 1K 78644K 1 0 0 512
xform 0 0K 1K 78644K 128 0 0 16,32
Memory totals: In Use Free Requests
13841K 272K 285515013
Memory resource pool statistics
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg Maxpg Idle
extent 20 255 0 251 1 0 1 1 0 inf 0
nfsrvdescpl 180 48757 0 48757 713 712 1 1 0 inf 1
nfsreqcachepl 36 64 0 0 1 0 1 1 0 inf 0
pnbufpl 1024 39067 0 39062 8342 8340 2 4 0 inf 0
sigapl 2052 12804008 0 12803930 4613 4529 84 268 0 inf 6
mclpl 2048 1128 0 866 397 250 147 196 16 512 4
mbpl 256 2316 0 2000 33 5 28 31 1 inf 0
buf16k 16384 1049466 490710 1041878 23466 21568 1898 5270 1 1 1
buf8k 8192 56149 156066 56125 2500 2491 9 23 1 1 0
buf4k 4096 138796 148124 138720 2537 2522 15 23 1 1 0
buf2k 2048 1702230 21540 1675646 9856 8134 1722 2024 1 1 0
buf1k 1024 7 0 7 1 0 1 1 1 1 1
kmem-4 4 202800 0 202458 32 28 4 8 0 inf 0
kmem-8 8 19522 0 19446 13 11 2 2 0 inf 0
kmem-12 12 10168 0 10159 5 4 1 3 0 inf 0
kmem-16 16 17139 0 17115 19 18 1 2 0 inf 0
kmem-20 20 91145 0 91133 11 10 1 4 0 inf 0
kmem-24 24 93253 0 93238 146 145 1 29 0 inf 0
kmem-28 28 92228 0 92216 39 38 1 6 0 inf 0
kmem-32 32 11398 0 11389 97 96 1 10 0 inf 0
kmem-36 36 9311 0 9305 41 40 1 6 0 inf 0
kmem-40 40 8702 0 8696 31 30 1 6 0 inf 0
kmem-44 44 8778 0 8775 191 190 1 12 0 inf 0
kmem-48 48 9005 0 8990 192 190 2 9 0 inf 0
kmem-56 56 7610 0 7580 152 147 5 20 0 inf 0
kmem-64 64 22276 0 21878 467 409 58 120 0 inf 0
kmem-72 72 5612 0 5582 89 84 5 9 0 inf 0
kmem-84 84 5729 0 5723 190 188 2 7 0 inf 1
kmem-100 100 4906 0 4897 59 56 3 6 0 inf 0
kmem-128 128 4508 0 4468 143 133 10 13 0 inf 0
propdict 112 63 0 0 2 0 2 2 0 inf 0
pdict32 64 3 0 0 1 0 1 1 0 inf 0
pdict16 48 4 0 0 1 0 1 1 0 inf 0
propnmbr 36 5 0 0 1 0 1 1 0 inf 0
ataspl 56 8 0 8 1 1 0 1 0 inf 0
rtentpl 136 388 0 345 2 0 2 2 0 inf 0
vnodepl 236 11373632 0 11081476 20073 4 20069 20070 0 inf 0
lockfpl 52 348335 0 348311 1 0 1 1 0 inf 0
ncachepl 84 395218759 0 395209526 245567 245255 312 6661 0 inf 74
biopl 148 36602913 0 36602912 149 148 1 17 0 inf 0
bufpl 148 1081586 0 1047314 3926 1598 2328 2635 0 inf 14
sockpl 216 111080 0 110973 44 37 7 17 0 inf 1
ttypl 288 17 0 0 2 0 2 2 0 inf 0
shmmp 12 24 0 24 1 1 0 1 0 inf 0
pipepl 124 496104 0 496058 69 67 2 6 0 inf 0
vmembtpl 28 15455 0 12361 39 1 38 38 0 inf 0
ptimerspl 140 399 0 375 1 0 1 1 0 inf 0
ptimerpl 124 400 0 375 1 0 1 1 0 inf 0
ksiginfo 44 23214838 0 23214838 1 0 1 1 0 inf 1
siginfo 128 88 0 88 44 44 0 1 0 inf 0
savppl 80 1 0 0 1 0 1 1 0 inf 0
sastackpl 24 16 0 0 1 0 1 1 0 inf 0
saupcpl 1612 518803 0 518802 1 0 1 1 0 inf 0
sadatapl 64 1 0 0 1 0 1 1 0 inf 0
sessionpl 56 6065 0 6017 1 0 1 1 0 inf 0
rusgepl 72 12803933 0 12803933 30586 30585 1 1 0 inf 1
pstatspl 224 12804030 0 12803933 151 144 7 16 0 inf 0
plimitpl 200 6472 0 6451 34 32 2 2 0 inf 0
pgrppl 16 91900 0 91842 1 0 1 1 0 inf 0
procpl 492 12804030 0 12803933 411 396 15 36 0 inf 0
lwppl 172 12804033 0 12803933 133 127 6 13 0 inf 0
ktepl 56 481 0 481 2 2 0 2 0 inf 0
knotepl 72 872 0 840 1 0 1 1 0 inf 0
kqueuepl 284 34 0 31 1 0 1 1 0 inf 0
fdescpl 308 12804010 0 12803932 200 192 8 21 0 inf 0
cwdipl 40 12804010 0 12803932 73 72 1 3 0 inf 0
filepl 84 203293706 0 203293427 425 418 7 11 0 inf 0
kauthcredpl 124 547368 0 547338 15 13 2 2 0 inf 1
rndsample 528 1700406 0 1700401 1 0 1 1 0 inf 0
ractx 24 14246866 0 14022068 4401 3070 1331 1999 0 inf 0
vmmpepl 88 206251463 0 206250258 614 584 30 59 0 inf 0
vmsppl 336 12804009 0 12803931 227 218 9 23 0 inf 0
aobjpl 80 25 0 24 1 0 1 1 0 inf 0
anonpl 40 1632093 0 1522873 4884 3692 1192 2013 0 inf 1
amappl 72 97944402 0 97943595 142 123 19 31 0 inf 0
ufsdirpl 264 11223534 0 11223534 3951 3950 1 1 0 inf 1
dino1pl 128 286884063 0 286591926 13160 3736 9424 11006 0 inf 0
ffsinopl 240 286884063 0 286591926 26126 7867 18259 21324 0 inf 0
dirrempl 44 360 0 360 5 5 0 2 0 inf 0
mkdirpl 32 10 0 10 3 3 0 1 0 inf 0
diraddpl 40 366 0 366 5 5 0 2 0 inf 0
freefilepl 40 206 0 206 4 4 0 1 0 inf 0
freeblkspl 176 162 0 162 10 10 0 4 0 inf 0
allocindirpl 64 1534 0 1534 26 26 0 12 0 inf 0
indirdeppl 32 25 0 25 3 3 0 1 0 inf 0
allocdirectpl 80 509 0 509 11 11 0 5 0 inf 0
bmsafemappl 36 17 0 17 3 3 0 1 0 inf 0
newblkpl 36 2043 0 2043 5 5 0 1 0 inf 0
inodedeppl 92 410 0 410 13 13 0 5 0 inf 0
pagedeppl 72 43 0 43 4 4 0 1 0 inf 0
sdpcpool 148 2013 0 2013 11 11 0 5 0 inf 0
nfsvapl 112 79 0 78 1 0 1 1 0 inf 0
nfsnodepl 388 79 0 78 2 1 1 2 0 inf 0
tcpcbpl 392 6043 0 5993 70 64 6 24 0 inf 1
sackholepl 20 22 0 22 4 4 0 1 0 inf 0
synpl 172 3098 0 3098 10 9 1 1 0 inf 1
tcpipqepl 44 6840 0 6840 19 18 1 1 0 inf 1
ipqepl 44 4181 0 4181 4 4 0 1 0 inf 0
inmltpl 32 2 0 0 1 0 1 1 0 inf 0
inpcbpl 108 16141 0 16067 20 17 3 8 0 inf 1
kvakmem 4096 402 0 308 5 0 5 5 0 inf 0
kvakernel 4096 404844 0 351621 2080 52 2028 2049 0 inf 6
pdppl 4096 36221 0 36136 35729 35641 88 269 0 inf 3
pmappl 100 12804009 0 12803931 108 105 3 7 0 inf 0
pcgpool 140 361332 0 361221 786 778 8 220 0 inf 0
phpool-128 164 32 0 28 1 0 1 1 0 inf 0
phpool-64 100 18 0 15 1 0 1 1 0 inf 0
phpool-32 68 2300 0 263 36 0 36 36 0 inf 0
phpool-16 52 841 0 825 1 0 1 1 0 inf 0
phpool-8 44 2099 0 2007 2 0 2 2 0 inf 0
phpool-4 40 1433 0 1402 1 0 1 1 0 inf 0
phpool-0 36 82864 0 78951 57 1 56 57 0 inf 0
In use 364591K, total allocated 494562K; utilization 73.7%
--
Greg A. Woods
Planix, Inc.
<woods@planix.com> +1 416 489-5852 x122 http://www.planix.com/
From: "David A. Holland" <dholland@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/38019 CVS commit: src/sys/arch/x86/x86
Date: Sun, 22 Feb 2009 07:14:46 +0000 (UTC)
Module Name: src
Committed By: dholland
Date: Sun Feb 22 07:14:46 UTC 2009
Modified Files:
src/sys/arch/x86/x86: ipmi.c
Log Message:
Improve some cryptic warning messages; from a patch attached to PR 38019
by Greg A. Woods, with a couple adjustments. Compile-tested only, but should
not be able to break anything.
To generate a diff of this commit:
cvs rdiff -r1.28 -r1.29 src/sys/arch/x86/x86/ipmi.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 16 Dec 2012 17:44:40 +0000
State-Changed-Why:
Problem affected netbsd-4 only, and netbsd-4 is now EOL.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home