NetBSD Problem Report #1439
From gnats Tue Sep 5 13:29:39 1995
Received: from pain.lcs.mit.edu (pain.lcs.mit.edu [128.52.46.239]) by sun-lamp.pc.cs.cmu.edu (8.6.11/8.6.10) with ESMTP id NAA01713 for <gnats-bugs@sun-lamp.cs.berkeley.edu>; Tue, 5 Sep 1995 13:29:29 -0400
Message-Id: <199509051633.SAA11804@honey.idt.unit.no>
Date: Tue, 5 Sep 1995 18:33:58 +0200
From: jarle@idt.unit.no
Reply-To: jarle@idt.unit.no
To: gnats-bugs@netbsd.org
Subject: stacksize rlimit value not multiple of pagesize causes execve to fail.
X-Send-Pr-Version: 3.2
>Number: 1439
>Category: kern
>Synopsis: stacksize rlimit value not multiple of pagesize causes execve to fail.
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Sep 05 13:35:00 +0000 1995
>Closed-Date: Fri Dec 04 13:31:02 +0000 1998
>Last-Modified: Fri Dec 04 13:31:49 +0000 1998
>Originator:
>Release: 1.0
>Organization:
Free Hardware Foundation, UnLtd.
>Environment:
System: NetBSD honey.idt.unit.no 1.0 NetBSD 1.0 (HONEY) #3: Mon May 8 00:07:00 MET DST 1995 jarle@honey.idt.unit.no:/local/NetBSD/src/sys/arch/i386/compile/HONEY i386
>Description:
If the value of p->p_rlimit[RLIMIT_STACK] is set to a value that is not a
multiple of the pagesize, a process' children may (will?) fail when trying
to do an execve. As far as I can see the vm subsystem will react
negatively to such a setting when trying to set up the new program's
stack. (execve() -> vmcmd_map_zero() -> vm->allocate() -> ??).
vm_allocate() will return an KERN_NO_SPACE error code.
>How-To-Repeat:
Run csh and do a 'limit stacksize 512'. Due to the way csh parses this
input it will actually set the stacksize to 0x80200, i.e. 512k + 512
bytes. All attempts at starting new programs will now fail with an IOT
trap. Doing a 'limit stacksize 512k' will set the stacksize to 0x80000,
i.e. 512k exactly, and starting other programs will work again. This
behavior was observed with gdb.
>Fix:
There are at least three ways to fix this:
* To make sure that the stacksize is always rounded (or truncated) to a
proper pagealigned value wherever it's being used.
* Round/truncate the stacksize to a pagealigned value where the rlimit
is being set. This second fix is the `quickest', but I'm not really sure
which one is the most correct one.... Among other things it will make
`limit stacksize 512' set the stacksize to 516k.
* Make the exec- and vm-subsystem deal with non-aligned address and size
values properly.
-jarle
Second fix:
*** kern_resource.c.orig Tue Sep 5 12:37:55 1995
--- kern_resource.c Tue Sep 5 18:10:11 1995
***************
*** 254,257 ****
--- 254,258 ----
case RLIMIT_STACK:
+ limp->rlim_cur = round_page(limp->rlim_cur);
if (limp->rlim_cur > maxsmap)
limp->rlim_cur = maxsmap;
--
trigraph ??!??! die
-- erik@naggum.no
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed
State-Changed-By: bouyer
State-Changed-When: Fri Dec 4 05:31:02 PST 1998
State-Changed-Why:
The proper checks should be here now, I can't reproduce this any more.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.