NetBSD Problem Report #41215
From salo@Xtrmntr.org Tue Apr 14 21:45:16 2009
Return-Path: <salo@Xtrmntr.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by www.NetBSD.org (Postfix) with ESMTP id C94E463B8A5
for <gnats-bugs@gnats.NetBSD.org>; Tue, 14 Apr 2009 21:45:15 +0000 (UTC)
Message-Id: <20090414214513.8BD6748B9@otaku.Xtrmntr.org>
Date: Tue, 14 Apr 2009 23:45:13 +0200 (CEST)
From: Lubomir Sedlacik <salo@Xtrmntr.org>
Reply-To: salo@Xtrmntr.org
To: gnats-bugs@gnats.NetBSD.org
Subject: cannot load miniroot as kmod, kernel heap full
X-Send-Pr-Version: 3.95
>Number: 41215
>Category: kern
>Synopsis: cannot load miniroot as kmod, kernel heap full
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Apr 14 21:50:00 +0000 2009
>Last-Modified: Mon Feb 28 17:30:03 +0000 2011
>Originator: Lubomir Sedlacik
>Release: NetBSD 5.99.10
>Organization:
>Environment:
System: NetBSD 5.99.10
Architecture: i386
Machine: i386
>Description:
I got the custom miniroot.kmod built pretty easily but it's failing at boot
with full kernel heap.
(capured in qemu, since it's very hard to stop before it scrolls away on real
hardware)
(qemu) >> VESA VBE Version 2.0 4096 k
>> NetBSD/x86 BIOS Boot, Revision 5.3 (from NetBSD 5.99.10)
>> Memory: 639/130048 k
1. Boot normally
2. Boot single user
3. Disable ACPI
4. Disable ACPI and SMP
5. Drop to boot prompt
Choose an option; RETURN for default; SPACE to stop countdown.
Option 1 will be chosen in 0
8509704+406516+538152 [462032+449388]=0x9e473c
Loading /miniroot.kmod heap full (0x4ae64+32768)
exit
exit
exit
exit
exit
exit
exit
exit
exit
exit
exit
[...]
The boot.cfg is default with added line:
load=/miniroot.kmod
I've tried GENERIC kernel with both my custom and the default provided
miniroot.kmod from the NetBSD-daily snapshot, both fail in the same fashion.
I found that it was already reported last month:
http://mail-index.netbsd.org/current-users/2008/12/17/msg006699.html
>How-To-Repeat:
# dd if=/dev/zero of=/tmp/disk.img bs=1m count=16
# vnconfig vnd0 /tmp/disk.img
# disklabel vnd0
[...]
# size offset fstype [fsize bsize cpg/sgs]
a: 32705 63 4.2BSD 0 0 0 # (Cyl. 0*- 15)
d: 32768 0 unused 0 0 # (Cyl. 0 - 15)
# fdisk -u vnd0
[...]
# fdisk -a vnd0
[...]
# fdisk vnd0
[...]
Partition table:
0: NetBSD (sysid 169)
start 63, size 32705 (16 MB, Cyls 0/1/32-15), Active
# installboot -v -o console=com0 /dev/rvnd0a .../usr/mdec/bootxx_ffsv1
# newfs_ffs /dev/rvnd0a
# mount /dev/vnd0a /mnt
# cp .../netbsd /mnt
# cp .../usr/mdec/boot /mnt
# cp -r .../stand /mnt
# cp .../miniroot.kmod /mnt
# cp .../boot.cfg /mnt
# sync
# qemu -hda /dev/vnd0d -boot c -nographic
>Fix:
>Audit-Trail:
From: Andrew Doran <ad@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Wed, 15 Apr 2009 08:12:42 +0000
Hi Lubomir,
Can you try increasing the heap size in the boot loader?
IIRC it is in CPPFLAGS in one of the Makefiles under i386/stand/boot.
From: Lubomir Sedlacik <salo@Xtrmntr.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Thu, 16 Apr 2009 01:20:52 +0200
On Wed, Apr 15, 2009 at 08:15:05AM +0000, Andrew Doran wrote:
> Can you try increasing the heap size in the boot loader?
> IIRC it is in CPPFLAGS in one of the Makefiles under i386/stand/boot.
Tried that, got a bit further but still no dice:
Index: Makefile.boot
===================================================================
RCS file: /cvsroot/src/sys/arch/i386/stand/boot/Makefile.boot,v
retrieving revision 1.40
diff -u -p -r1.40 Makefile.boot
--- Makefile.boot 30 Mar 2009 09:22:52 -0000 1.40
+++ Makefile.boot 15 Apr 2009 23:08:43 -0000
@@ -81,7 +81,7 @@ CPPFLAGS+= -DEPIA_HACK
# The biosboot code is linked to 'virtual' address of zero and is
# loaded at physical address 0x10000.
# XXX The heap values should be determined from _end.
-SAMISCCPPFLAGS+= -DHEAP_START=0x30000 -DHEAP_LIMIT=0x50000
+SAMISCCPPFLAGS+= -DHEAP_START=0x30000 -DHEAP_LIMIT=0x60000
SAMISCMAKEFLAGS+= SA_USE_CREAD=yes # Read compressed kernels
SAMISCMAKEFLAGS+= SA_INCLUDE_NET=no # Netboot via TFTP, NFS
---
8509704+406516+538152 [462032+449388]=0x9e473c
Loading /miniroot.kmod
Loading ffs
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
[...]
boot device: wd0
root on md0a dumps on md0b
root file system type: ffs
exec /sbin/init: error 8
init: trying /sbin/oinit
exec /sbin/oinit: error 2
init: trying /sbin/init.bak
exec /sbin/init.bak: error 2
init path (default /sbin/init):
---
Given that errno 8 is "Exec format error", I tried to force load exec_elf32:
8509704+406516+538152 [462032+449388]=0x9e473c
Loading /miniroot.kmod
Loading exec_elf32
Loading ffs
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
[...]
attimer0: attached to pcppi0
uvm_fault(0xc09db680, 0, 1) -> 0xe
fatal page fault in supervisor mode
trap type 6 code 0 eip c069d5b5 cs 8 eflags 206 cr2 14 ilevel 0
kernel: supervisor trap page fault, code=0
Stopped in pid 0.1 (system) at netbsd:_atomic_inc_32+0x5: incl 0(%edx)
db{0}> t
_atomic_inc_32(c0e1fbc4,ca420d80,0,c090a90f,73,c0e1fcb4,0,32,c0976480,ca42af00) at netbsd:_atomic_inc_32+0x5
vn_open(c0e1fbc4,1,0,ca41faa4,0,c09c1be4,c0e1fbe8,c044fa2c,c0976480,cb0a2c00) at netbsd:vn_open+0x80
kobj_load_file(ca41faac,c0e1fcac,c09bc020,1,140cd80,0,1e1fc70,c0e1fc6c,c09c1be4, 0) at netbsd:kobj_load_file+0x94
module_do_load(0,ca41f968,0,1,0,0,e1fce8,c04429b8,c0de7b68,c0de7b42) at netbsd:module_do_load+0x2c5
module_do_load(0,0,0,0,0,e1d000,c0e1fd38,c03d41a7,0,c09c3300) at netbsd:module_do_load+0x435
module_init_class(0,c09c3300,c0e1fd38,c03d4134,0,0,0,0,0,0) at netbsd:module_init_class+0xa2
main(0,c01002cd,0,0,0,0,0,0,0,0) at netbsd:main+0x227
---
Not sure if the heap is too big now or whether there is something else
going wrong. E.g., 0x55000 was enough to load miniroot and ffs but not
enough for miniroot, ffs and exec_elf32.
--
-- Lubomir Sedlacik <salo@{NetBSD,Xtrmntr,silcnet}.org> --
From: David Laight <david@l8s.co.uk>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Thu, 16 Apr 2009 07:41:13 +0100
On Tue, Apr 14, 2009 at 09:50:00PM +0000, Lubomir Sedlacik wrote:
> >Number: 41215
> >Synopsis: cannot load miniroot as kmod, kernel heap full
...
> I got the custom miniroot.kmod built pretty easily but it's failing at boot
> with full kernel heap.
...
> 8509704+406516+538152 [462032+449388]=0x9e473c
> Loading /miniroot.kmod heap full (0x4ae64+32768)
> exit
What is the filesystem block size ?
Possibly the file code is leaking a large buffer.
(It hadn't used to matter since only 1 file was ever processed)
David
--
David Laight: david@l8s.co.uk
From: dieter roelants <dieter.NetBSD@pandora.be>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Thu, 16 Apr 2009 15:33:20 +0200
On Tue, 14 Apr 2009 21:50:00 +0000 (UTC)
Lubomir Sedlacik <salo@Xtrmntr.org> wrote:
> I got the custom miniroot.kmod built pretty easily but it's failing at boot
> with full kernel heap.
I don't know if it's the same cause, but booting Xen fails for me with
the same error. But see below.
> (capured in qemu, since it's very hard to stop before it scrolls away on real
> hardware)
My way of working around it (I can't use qemu, obviously), was to set
the vesa mode to 1280x1024 because it scrolls slow enough to make me
able to read it. To increase the time the error was on-screen, I filled
the display first with a `vesa list`, a `help` and finally a `ls`. And
guess what... when I run ls from the boot prompt, it succeeds loading
the netbsd and xen kernel afterwards (I tried this several times). I
have no clue why this is so and if it's a workaround in your case as
well.
dieter
From: Lubomir Sedlacik <salo@Xtrmntr.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Fri, 17 Apr 2009 00:49:00 +0200
On Thu, Apr 16, 2009 at 06:40:03AM +0000, David Laight wrote:
> What is the filesystem block size ?
> Possibly the file code is leaking a large buffer.
> (It hadn't used to matter since only 1 file was ever processed)
4096 bytes, according to dumpfs.
--
-- Lubomir Sedlacik <salo@{NetBSD,Xtrmntr,silcnet}.org> --
From: David Laight <david@l8s.co.uk>
To: gnats-bugs@NetBSD.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, salo@Xtrmntr.org
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Sat, 18 Apr 2009 15:03:00 +0100
On Thu, Apr 16, 2009 at 10:50:04PM +0000, Lubomir Sedlacik wrote:
> The following reply was made to PR kern/41215; it has been noted by GNATS.
>
> From: Lubomir Sedlacik <salo@Xtrmntr.org>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
> Date: Fri, 17 Apr 2009 00:49:00 +0200
>
> On Thu, Apr 16, 2009 at 06:40:03AM +0000, David Laight wrote:
> > What is the filesystem block size ?
> > Possibly the file code is leaking a large buffer.
> > (It hadn't used to matter since only 1 file was ever processed)
>
> 4096 bytes, according to dumpfs.
That sounds like the fragment size - giving a block size of 32k,
rather than the more likely defaults of 1k and 8k.
The standalons fs code allocates block sized buffers for allocation
maps and (IIRC inodes), so I'd guess it isn't freeing them.
It ought to be possibly to only need disk block sized buffers - which
is also necessary in order to load /boot from a fs with large blocks.
But that requires a full understanding of the fs layout which isn't
obvious from the current #defines.
David
--
David Laight: david@l8s.co.uk
From: David Laight <david@l8s.co.uk>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Mon, 20 Apr 2009 07:41:30 +0100
On Thu, Apr 16, 2009 at 06:40:03AM +0000, David Laight wrote:
>
> Possibly the file code is leaking a large buffer.
Just a thought, does the standalone code even implement free() ?
A very trivial version might be useful here!
David
--
David Laight: david@l8s.co.uk
From: =?ISO-8859-1?Q?Gr=E9goire_Sutre?= <gregoire.sutre@gmail.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Wed, 10 Mar 2010 10:37:11 +0100
Hi,
[ This is a copy of a message I sent to port-i386. ]
With -current boot(8) on i386, loading a (large) compressed module, such
as a kernel or an installation miniroot.kmod, leads to an error heap
full. In qemu:
>> NetBSD/x86 BIOS Boot, Revision 5.4 (from NetBSD 5.99.24)
>> Memory: 639/130048 k
> load /miniroot.kmod
> boot /netbsd-GENERIC.gz
8982376+493364+539784 [476288+461807]=0xa73910
WARNING: couldn't open ffs (/stand/i386/5.99.24/modules/ffs/ffs.kmod)
Loading /miniroot.kmod heap full (0x48f9c+32768)
exit
exit
[many more exit]
This is likely related to PRs #41215 and #41524, and to [1]. The
following patch seems to work, but it's a wild guess:
diff -u -r1.42 Makefile.boot
--- sys/arch/i386/stand/boot/Makefile.boot
+++ sys/arch/i386/stand/boot/Makefile.boot
@@ -81,7 +81,7 @@
# The biosboot code is linked to 'virtual' address of zero and is
# loaded at physical address 0x10000.
# XXX The heap values should be determined from _end.
-SAMISCCPPFLAGS+= -DHEAP_START=0x30000 -DHEAP_LIMIT=0x50000
+SAMISCCPPFLAGS+= -DHEAP_START=0x28000 -DHEAP_LIMIT=0x50000
SAMISCMAKEFLAGS+= SA_USE_CREAD=yes # Read compressed kernels
SAMISCMAKEFLAGS+= SA_INCLUDE_NET=no # Netboot via TFTP, NFS
According to the CVS logs, the value of HEAP_START was bumped from
0x20000 to 0x30000 a year ago since the size of /boot was getting
dangerously close to the limit (HEAP_START - 0x10000). But the
downside is that the heap size was reduced from 0x30000 to 0x20000, and
the latter is apparently not enough (for compressed modules). Using
0x28000 for HEAP_START leaves enough space for /boot, and enough room
for the heap in my tests.
I don't know what are the right values here, and why the HEAP_LIMIT is
set to 0x50000. Some help would be welcome :-)
Thanks,
Grégoire
p.s. In my tests, the problem does not occur if the modules are
uncompressed.
[1] http://mail-index.netbsd.org/port-xen/2009/10/29/msg005516.html
From: =?ISO-8859-1?Q?Gr=E9goire_Sutre?= <gregoire.sutre@gmail.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/41215: cannot load miniroot as kmod, kernel heap full
Date: Mon, 28 Feb 2011 18:27:23 +0100
For the record, the problem I experienced does not occur anymore with
a recent /boot (5.99.47). This is likely due to:
> Module Name: src
> Committed By: jakllsch
> Date: Mon Dec 20 00:55:10 UTC 2010
>
> Modified Files:
> src/sys/arch/i386/stand/boot: Makefile.boot
>
> Log Message:
> Move the bottom of the heap off the top of the stack. Also, increase the heap
> size by 64KiB. These changes allows gzipped Xen with gzipped Dom0 kernels
> to successfully boot. I also suspect this will cure the amd64 CD boot issue
> without disabling ext2fs support.
Grégoire
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.