NetBSD Problem Report #43438
From mrg@eterna.com.au Wed Jun 9 02:54:20 2010
Return-Path: <mrg@eterna.com.au>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by www.NetBSD.org (Postfix) with ESMTP id C06BB63B908
for <gnats-bugs@gnats.NetBSD.org>; Wed, 9 Jun 2010 02:54:20 +0000 (UTC)
Message-Id: <20100609025418.90A3C3752D@splode.eterna.com.au>
Date: Wed, 9 Jun 2010 12:54:18 +1000 (EST)
From: mrg@eterna.com.au
Reply-To: mrg@eterna.com.au
To: gnats-bugs@gnats.NetBSD.org
Subject: x86 module map size is limited
X-Send-Pr-Version: 3.95
>Number: 43438
>Category: port-amd64
>Synopsis: x86 module map size is limited
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: port-amd64-maintainer
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Jun 09 02:55:00 +0000 2010
>Closed-Date: Fri Jul 28 16:37:38 +0000 2017
>Last-Modified: Fri Jul 28 16:37:38 +0000 2017
>Originator: matthew green
>Release: NetBSD 5.99.30
>Organization:
people's front against (bozotic) www (softwar foundation)
>Environment:
>Description:
trying to load the "solaris" and "zfs" kmodules into an amd64
GENERIC kernel results in ENOMEM being returned by the zfs load
attempt. i tracked this down to the x86 module_map being
sized at 20MB - size of kernel + rodata. in my case, the module
map was only left about 3MB or space, and it could not satisfy
the modload attempts.
>How-To-Repeat:
boot large amd64 kernel (GENERIC + debugging seems sufficient,
plus *VERBOSE too) and try to modload zfs.
>Fix:
i assume that there is some x86 specific setup that limits this
size to 20MB but i'm not sure what the fix is.
i worked around the issue by removing about 100 drivers i wasn't
using from my kernel and freeing up enough space in the module map.
>Release-Note:
>Audit-Trail:
From: Nicolas Joly <njoly@pasteur.fr>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-amd64/43438: x86 module map size is limited
Date: Wed, 9 Jun 2010 14:14:23 +0200
--3V7upXqbjpZ4EhLz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Wed, Jun 09, 2010 at 02:55:00AM +0000, mrg@eterna.com.au wrote:
> >Number: 43438
> >Category: port-amd64
> >Synopsis: x86 module map size is limited
[...]
> trying to load the "solaris" and "zfs" kmodules into an amd64
> GENERIC kernel results in ENOMEM being returned by the zfs load
> attempt. i tracked this down to the x86 module_map being
> sized at 20MB - size of kernel + rodata. in my case, the module
> map was only left about 3MB or space, and it could not satisfy
> the modload attempts.
I hit the same problem quite some time ago while playing with
compat_linux modules ... At least for me, the problem is that modules
for a MKDEBUG=yes,COPTS=-g build are gigantic. The attached patch,
which remove debugging symbols from modules to be installed (but keeps
a .gdb copy handy just in case ...) solved the problem.
Unstripped:
njoly@lanfeust [dest/amd64]> du -sh stand/amd64/5.99.30
42M stand/amd64/5.99.30
Stripped:
njoly@lanfeust [dest/amd64]> du -sh stand/amd64/5.99.30
6.5M stand/amd64/5.99.30
--
Nicolas Joly
Biological Software and Databanks.
Institut Pasteur, Paris.
--3V7upXqbjpZ4EhLz
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="netbsd-stripkmod.diff"
Index: share/mk/bsd.kmodule.mk
===================================================================
RCS file: /cvsroot/src/share/mk/bsd.kmodule.mk,v
retrieving revision 1.24
diff -u -p -r1.24 bsd.kmodule.mk
--- share/mk/bsd.kmodule.mk 18 Jan 2010 23:39:07 -0000 1.24
+++ share/mk/bsd.kmodule.mk 9 Jun 2010 11:54:48 -0000
@@ -48,6 +48,14 @@ KMODSCRIPT= ${DESTDIR}/usr/libdata/ldscr
OBJS+= ${SRCS:N*.h:N*.sh:R:S/$/.o/g}
PROG?= ${KMOD}.kmod
+.if ${MKDEBUG} != "no"
+KMODDEBUG= mv ${PROG} ${PROG}.gdb && \
+ ${STRIP} --strip-debug -o ${PROG} ${PROG}.gdb
+CLEANFILES+= ${PROG}.gdb
+.else
+KMODDEBUG= true
+.endif
+
##### Build rules
realall: ${PROG}
@@ -57,6 +65,7 @@ ${PROG}: ${OBJS} ${DPADD}
${_MKTARGET_LINK}
${CC} ${LDFLAGS} -nostdlib -r -Wl,-T,${KMODSCRIPT},-d \
-o ${.TARGET} ${OBJS}
+ ${KMODDEBUG}
##### Install rules
.if !target(kmodinstall)
@@ -88,7 +97,7 @@ ${_PROG}: .MADE # no build at instal
done
${INSTALL_DIR} ${KMODULEDIR}
${INSTALL_FILE} -o ${KMODULEOWN} -g ${KMODULEGRP} -m ${KMODULEMODE} \
- ${.ALLSRC} ${.TARGET}
+ ${STRIPFLAG} ${.ALLSRC} ${.TARGET}
kmodinstall:: ${_PROG}
.PHONY: kmodinstall
--3V7upXqbjpZ4EhLz--
From: David Laight <david@l8s.co.uk>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-amd64/43438: x86 module map size is limited
Date: Wed, 9 Jun 2010 21:02:18 +0100
On Wed, Jun 09, 2010 at 12:15:04PM +0000, Nicolas Joly wrote:
>
> I hit the same problem quite some time ago while playing with
> compat_linux modules ... At least for me, the problem is that modules
> for a MKDEBUG=yes,COPTS=-g build are gigantic...
Do they also have the .ehframe section?
We ought to be removing that (gcc option) for all amd64 kernel bits.
David
--
David Laight: david@l8s.co.uk
From: Nicolas Joly <njoly@pasteur.fr>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-amd64/43438: x86 module map size is limited
Date: Wed, 9 Jun 2010 22:30:04 +0200
--RnlQjJ0d97Da+TV1
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
On Wed, Jun 09, 2010 at 08:05:03PM +0000, David Laight wrote:
> The following reply was made to PR port-amd64/43438; it has been noted by GNATS.
>
> From: David Laight <david@l8s.co.uk>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: port-amd64/43438: x86 module map size is limited
> Date: Wed, 9 Jun 2010 21:02:18 +0100
>
> On Wed, Jun 09, 2010 at 12:15:04PM +0000, Nicolas Joly wrote:
> >
> > I hit the same problem quite some time ago while playing with
> > compat_linux modules ... At least for me, the problem is that modules
> > for a MKDEBUG=yes,COPTS=-g build are gigantic...
>
> Do they also have the .ehframe section?
Yes.
2 .eh_frame 000115a8 0000000000000000 0000000000000000 000b5988 2**3
CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
> We ought to be removing that (gcc option) for all amd64 kernel bits.
Ok. The new patch save a few more byte by discarding this section in
modules.
njoly@petaure [~]> ll /stand/amd64/5.99.30/modules/zfs/zfs.kmod
-r--r--r-- 1 root wheel 1461032 Jun 8 09:34 /stand/amd64/5.99.30/modules/zfs/zfs.kmod
njoly@petaure [~]> ll /stand/amd64/5.99.30/modules/zfs/zfs.kmod
-r--r--r-- 1 root wheel 1338720 Jun 9 22:22 /stand/amd64/5.99.30/modules/zfs/zfs.kmod
--
Nicolas Joly
Biological Software and Databanks.
Institut Pasteur, Paris.
--RnlQjJ0d97Da+TV1
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename="netbsd-stripkmod.diff"
Index: share/mk/bsd.kmodule.mk
===================================================================
RCS file: /cvsroot/src/share/mk/bsd.kmodule.mk,v
retrieving revision 1.24
diff -u -p -r1.24 bsd.kmodule.mk
--- share/mk/bsd.kmodule.mk 18 Jan 2010 23:39:07 -0000 1.24
+++ share/mk/bsd.kmodule.mk 9 Jun 2010 20:23:19 -0000
@@ -48,6 +48,15 @@ KMODSCRIPT= ${DESTDIR}/usr/libdata/ldscr
OBJS+= ${SRCS:N*.h:N*.sh:R:S/$/.o/g}
PROG?= ${KMOD}.kmod
+.if ${MKDEBUG} != "no"
+KMODDEBUG= mv ${PROG} ${PROG}.gdb && \
+ ${STRIP} --strip-debug --remove-section=.eh_frame \
+ -o ${PROG} ${PROG}.gdb
+CLEANFILES+= ${PROG}.gdb
+.else
+KMODDEBUG= true
+.endif
+
##### Build rules
realall: ${PROG}
@@ -57,6 +66,7 @@ ${PROG}: ${OBJS} ${DPADD}
${_MKTARGET_LINK}
${CC} ${LDFLAGS} -nostdlib -r -Wl,-T,${KMODSCRIPT},-d \
-o ${.TARGET} ${OBJS}
+ ${KMODDEBUG}
##### Install rules
.if !target(kmodinstall)
@@ -88,7 +98,7 @@ ${_PROG}: .MADE # no build at instal
done
${INSTALL_DIR} ${KMODULEDIR}
${INSTALL_FILE} -o ${KMODULEOWN} -g ${KMODULEGRP} -m ${KMODULEMODE} \
- ${.ALLSRC} ${.TARGET}
+ ${STRIPFLAG} ${.ALLSRC} ${.TARGET}
kmodinstall:: ${_PROG}
.PHONY: kmodinstall
--RnlQjJ0d97Da+TV1--
From: David Laight <david@l8s.co.uk>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-amd64/43438: x86 module map size is limited
Date: Thu, 10 Jun 2010 19:37:48 +0100
On Wed, Jun 09, 2010 at 08:35:02PM +0000, Nicolas Joly wrote:
> >
> > Do they also have the .ehframe section?
>
> Yes.
>
> 2 .eh_frame 000115a8 0000000000000000 0000000000000000 000b5988 2**3
> CONTENTS, ALLOC, LOAD, RELOC, READONLY, DATA
>
> > We ought to be removing that (gcc option) for all amd64 kernel bits.
>
> Ok. The new patch save a few more byte by discarding this section in
> modules.
...
> +.if ${MKDEBUG} != "no"
> +KMODDEBUG= mv ${PROG} ${PROG}.gdb && \
> + ${STRIP} --strip-debug --remove-section=.eh_frame \
> + -o ${PROG} ${PROG}.gdb
I'm sure I remember there being symbols in the .eh_frame section that
get referenced from elsewhere.
This means (at least for userspace) you can't use objcopy to remove the
section, but must tell gcc to not generate it.
David
--
David Laight: david@l8s.co.uk
From: Nicolas Joly <njoly@pasteur.fr>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-amd64/43438: x86 module map size is limited
Date: Fri, 11 Jun 2010 18:10:18 +0200
On Thu, Jun 10, 2010 at 06:40:05PM +0000, David Laight wrote:
> ...
> > +.if ${MKDEBUG} != "no"
> > +KMODDEBUG= mv ${PROG} ${PROG}.gdb && \
> > + ${STRIP} --strip-debug --remove-section=.eh_frame \
> > + -o ${PROG} ${PROG}.gdb
>
> I'm sure I remember there being symbols in the .eh_frame section that
> get referenced from elsewhere.
> This means (at least for userspace) you can't use objcopy to remove the
> section, but must tell gcc to not generate it.
I had a quick look to gcc, but nothing obvious :(
--
Nicolas Joly
Biological Software and Databanks.
Institut Pasteur, Paris.
From: "Maxime Villard" <maxv@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/43438 CVS commit: src/sys/arch/amd64/amd64
Date: Sat, 2 Jul 2016 07:22:10 +0000
Module Name: src
Committed By: maxv
Date: Sat Jul 2 07:22:10 UTC 2016
Modified Files:
src/sys/arch/amd64/amd64: machdep.c
Log Message:
Explain why we should use kernel_map instead of module_map, and why we
can't.
We should probably add some GCC flags in the modules makefiles to make
sure the relocations generated are not 32bit.
Related to PR/43438.
To generate a diff of this commit:
cvs rdiff -u -r1.219 -r1.220 src/sys/arch/amd64/amd64/machdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->closed
State-Changed-By: maxv@NetBSD.org
State-Changed-When: Fri, 28 Jul 2017 16:37:38 +0000
State-Changed-Why:
it's not a bug, it's just that the map has a maximum size because
of amd64 reasons. In the future, NKL2_KIMG_ENTRIES can be increased
if needed.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home