NetBSD Problem Report #1439

From gnats  Tue Sep  5 13:29:39 1995
Received: from pain.lcs.mit.edu (pain.lcs.mit.edu [128.52.46.239]) by sun-lamp.pc.cs.cmu.edu (8.6.11/8.6.10) with ESMTP id NAA01713 for <gnats-bugs@sun-lamp.cs.berkeley.edu>; Tue, 5 Sep 1995 13:29:29 -0400
Message-Id: <199509051633.SAA11804@honey.idt.unit.no>
Date: Tue, 5 Sep 1995 18:33:58 +0200
From: jarle@idt.unit.no
Reply-To: jarle@idt.unit.no
To: gnats-bugs@netbsd.org
Subject: stacksize rlimit value not multiple of pagesize causes execve to fail.
X-Send-Pr-Version: 3.2

>Number:         1439
>Category:       kern
>Synopsis:       stacksize rlimit value not multiple of pagesize causes execve to fail.
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Sep 05 13:35:00 +0000 1995
>Closed-Date:    Fri Dec 04 13:31:02 +0000 1998
>Last-Modified:  Fri Dec 04 13:31:49 +0000 1998
>Originator:     
>Release:        1.0
>Organization:
Free Hardware Foundation, UnLtd.
>Environment:

System: NetBSD honey.idt.unit.no 1.0 NetBSD 1.0 (HONEY) #3: Mon May 8 00:07:00 MET DST 1995 jarle@honey.idt.unit.no:/local/NetBSD/src/sys/arch/i386/compile/HONEY i386


>Description:
If the value of p->p_rlimit[RLIMIT_STACK] is set to a value that is not a
multiple of the pagesize, a process' children may (will?) fail when trying
to do an execve.  As far as I can see the vm subsystem will react
negatively to such a setting when trying to set up the new program's
stack. (execve() -> vmcmd_map_zero() -> vm->allocate() -> ??).
vm_allocate() will return an KERN_NO_SPACE error code.


>How-To-Repeat:
Run csh and do a 'limit stacksize 512'.  Due to the way csh parses this
input it will actually set the stacksize to 0x80200, i.e. 512k + 512
bytes.  All attempts at starting new programs will now fail with an IOT
trap.  Doing a 'limit stacksize 512k' will set the stacksize to 0x80000,
i.e. 512k exactly, and starting other programs will work again.  This
behavior was observed with gdb.

>Fix:
There are at least three ways to fix this:  
* To make sure that the stacksize is always rounded (or truncated) to a
proper pagealigned value wherever it's being used. 
* Round/truncate the stacksize to a pagealigned value where the rlimit
is being set.  This second fix is the `quickest', but I'm not really sure
which one is the most correct one.... Among other things it will make
`limit stacksize 512' set the stacksize to 516k.
* Make the exec- and vm-subsystem deal with non-aligned address and size
values properly.
					-jarle

Second fix:
*** kern_resource.c.orig        Tue Sep  5 12:37:55 1995
--- kern_resource.c     Tue Sep  5 18:10:11 1995
***************
*** 254,257 ****
--- 254,258 ----

        case RLIMIT_STACK:
+               limp->rlim_cur = round_page(limp->rlim_cur);
                if (limp->rlim_cur > maxsmap)
                        limp->rlim_cur = maxsmap;


-- 
trigraph ??!??! die
					-- erik@naggum.no
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: bouyer 
State-Changed-When: Fri Dec 4 05:31:02 PST 1998 
State-Changed-Why:  
The proper checks should be here now, I can't reproduce this any more. 
>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.