NetBSD Problem Report #8681
Received: (qmail 19832 invoked from network); 25 Oct 1999 14:48:04 -0000
Message-Id: <199910251447.QAA20630@vader.runit.sintef.no>
Date: Mon, 25 Oct 1999 16:47:31 +0200 (MEST)
From: Havard.Eidnes@runit.sintef.no
Reply-To: Havard.Eidnes@runit.sintef.no
To: gnats-bugs@gnats.netbsd.org
Subject: grep may bomb out with "memory exhausted"
X-Send-Pr-Version: 3.95
>Number: 8681
>Category: bin
>Synopsis: grep may bomb out with "memory exhausted"
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: bin-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Mon Oct 25 07:50:59 +0000 1999
>Closed-Date:
>Last-Modified: Wed Apr 27 21:25:02 +0000 2011
>Originator: Havard Eidnes <he@uninett.no>
>Release: 1.4.1
>Organization:
RUNIT AS
>Environment:
NetBSD vever.runit.sintef.no 1.4.1 NetBSD 1.4.1 (VEVER) #5: Wed Oct 20 19:42:00 MEST 1999 he@vever.runit.sintef.no:/usr/src/sys/arch/i386/compile/VEVER i386
>Description:
It appears that grep/egrep/fgrep use mmap() to read files but
never releases the memory segments with munmap(), causing an ever
increasing data segment (?) / virtual size, eventually bumping into
resource limits.
>How-To-Repeat:
In my /sys/arch/i386/compile tree I have a couple of 30+MB
netbsd.gdb files lying around. Then, doing:
cd /sys/arch/i386/
find . -type f | xargs grep -n constab /dev/null
grep: memory exhausted
grep: memory exhausted
From following grep's progress with 'top', it appears that it
gets an ever increasing virtual size. Inspection of the source code
would seem to indicate that it is mmap()ing the files to be searched,
and never doing munmap() on the same regions when they have been
searched, thus causing an ever increasing virtual size until they bump
into the resource limit set for datasize (in my case 128MB, identical
to physical memory on this box).
(I thought that mmap()ed segments weren't counted against resource
limits, but apparently they are... (now?))
>Fix:
Two alternatives:
o Turn off HAVE_MMAP in /usr/src/gnu/usr.bin/grep/config.h
o Find a better solution (doesn't immediately look obvious)
>Release-Note:
>Audit-Trail:
From: Simon Burge <simonb@NetBSD.ORG>
To: Havard.Eidnes@runit.sintef.no
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Tue, 26 Oct 1999 09:23:13 +1000
Havard.Eidnes@runit.sintef.no wrote:
> >Number: 8681
> >Category: bin
> >Synopsis: grep may bomb out with "memory exhausted"
>
> ...
>
> >Description:
> It appears that grep/egrep/fgrep use mmap() to read files but
> never releases the memory segments with munmap(), causing an ever
> increasing data segment (?) / virtual size, eventually bumping into
> resource limits.
>
> ...
>
> >Fix:
> Two alternatives:
> o Turn off HAVE_MMAP in /usr/src/gnu/usr.bin/grep/config.h
> o Find a better solution (doesn't immediately look obvious)
With both this and bin/8461, seems like a very strong case to turn off
HAVE_MMAP. Any objections?
Simon.
From: Havard.Eidnes@runit.sintef.no
To: simonb@NetBSD.ORG
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Tue, 26 Oct 1999 01:43:52 +0200
> With both this and bin/8461, seems like a very strong case to turn off
> HAVE_MMAP. Any objections?
Gah, even doing that appears not to be sufficient to cure the
problem:
vever# cd /sys/arch/i386
vever# find . -type f | xargs egrep XXBLAHXX
egrep: memory exhausted
egrep: memory exhausted
vever# find . -type f -size +4000 -ls
122221 6400 -rwxr-xr-x 1 root wheel 3264606 Oct 25 14:24 ./compile/GENERIC/netbsd
800446 6752 -rwxr-xr-x 1 root wheel 3445282 Jul 15 16:57 ./compile/INSTALL/netbsd
42399 4960 -rwxr-xr-x 1 root wheel 2524794 Jul 15 17:02 ./compile/INSTALL_TINY/netbsd
929336 43792 -rwxr-xr-x 1 root wheel 22395758 Oct 25 19:31 ./compile/SERVER/netbsd.gdb
532948 6320 -rwxr-xr-x 1 root wheel 3223473 Oct 25 11:05 ./compile/SERVER_1/netbsd
532947 72608 -rwxr-xr-x 1 root wheel 37136160 Oct 25 11:05 ./compile/SERVER_1/netbsd.gdb
457225 44928 -rwxr-xr-x 1 root wheel 22976155 Oct 25 18:46 ./compile/SERVER_2/netbsd.gdb
56770 43504 -rwxr-xr-x 1 root wheel 22241975 Oct 25 14:41 ./compile/RYPE/netbsd.gdb
vever# limit
cputime unlimited
filesize unlimited
datasize 131072 kbytes
stacksize 2048 kbytes
coredumpsize unlimited
memoryuse 57984 kbytes
memorylocked 19328 kbytes
maxproc 532
openfiles 64
vever#
- Havard
From: Simon Burge <simonb@NetBSD.ORG>
To: Havard.Eidnes@runit.sintef.no
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Tue, 26 Oct 1999 11:21:03 +1000
Havard.Eidnes@runit.sintef.no wrote:
> > With both this and bin/8461, seems like a very strong case to turn off
> > HAVE_MMAP. Any objections?
>
> Gah, even doing that appears not to be sufficient to cure the
> problem:
It seems that under some circumstances that grep wants to save a very
large chunk of the current file in a buffer, and that five times that
buffer is allocated when the buffers are expanded. So when it tries to
look in:
532947 72608 -rwxr-xr-x 1 root wheel 37136160 Oct 25 11:05 ./compile/SERVER_1/netbsd.gdb
it'll _could_ try to allocate up to 175MB... A simple reproduction is:
% dd if=/dev/zero of=/tmp/foo bs=1m count=8
% limit data 8192
% grep foo /tmp/foo
grep: memory exhausted
The following patch limits the size of the save buffer to 2MB, which
means up to 10MB will be allocated. I'm really not sure what the
implications of this are though - simple tests still seem to work ok.
I'll send something to the BUG bugs list and see what they say about
this.
Simon.
--
Index: grep.c
===================================================================
RCS file: /cvsroot/gnusrc/gnu/dist/grep/src/grep.c,v
retrieving revision 1.4
diff -p -u -r1.4 grep.c
--- grep.c 1999/04/06 16:54:55 1.4
+++ grep.c 1999/10/26 00:55:55
@@ -334,6 +334,9 @@ fillbuf (save, stats)
caddr_t maddr;
#endif
+ /* limit save area to 2MB so that memory doesn't overflow on large files */
+ if (save > 2 * 1024 * 1024)
+ save = 2 * 1024 * 1024;
if (save > bufsalloc)
{
char *nubuffer;
From: Simon Burge <simonb@NetBSD.ORG>
To: Havard.Eidnes@runit.sintef.no
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Tue, 26 Oct 1999 11:47:15 +1000
Simon Burge wrote:
> Havard.Eidnes@runit.sintef.no wrote:
>
> > > With both this and bin/8461, seems like a very strong case to turn off
> > > HAVE_MMAP. Any objections?
> >
> > Gah, even doing that appears not to be sufficient to cure the
> > problem:
>
> It seems that under some circumstances that grep wants to save a very
> large chunk of the current file in a buffer, and that five times that
> buffer is allocated when the buffers are expanded.
Looking at this closer, it's as simple as looking in the current buffer
for a newline. If the buffer is all zeros (as some of your kernel may
have, and the test file I used did), then it wont find a newline and
tries to save the all of the current buffer and so on until memory runs
out. At this stage, I think the buffer size limiting patch is probably
the right thing...
Simon.
From: David Brownlee <abs@mono.org>
To: Simon Burge <simonb@netbsd.org>
Cc: Havard.Eidnes@runit.sintef.no, gnats-bugs@gnats.netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Tue, 26 Oct 1999 12:00:56 +0100 (BST)
Maybe it could call getrlimit and base the size of the limited
buffer on that (1/8th?)
David/absolute
On Tue, 26 Oct 1999, Simon Burge wrote:
> Havard.Eidnes@runit.sintef.no wrote:
>
> > > With both this and bin/8461, seems like a very strong case to turn off
> > > HAVE_MMAP. Any objections?
> >
> > Gah, even doing that appears not to be sufficient to cure the
> > problem:
>
> It seems that under some circumstances that grep wants to save a very
> large chunk of the current file in a buffer, and that five times that
> buffer is allocated when the buffers are expanded. So when it tries to
> look in:
>
> 532947 72608 -rwxr-xr-x 1 root wheel 37136160 Oct 25 11:05 ./compile/SERVER_1/netbsd.gdb
>
> it'll _could_ try to allocate up to 175MB... A simple reproduction is:
>
> % dd if=/dev/zero of=/tmp/foo bs=1m count=8
> % limit data 8192
> % grep foo /tmp/foo
> grep: memory exhausted
>
> The following patch limits the size of the save buffer to 2MB, which
> means up to 10MB will be allocated. I'm really not sure what the
> implications of this are though - simple tests still seem to work ok.
>
> I'll send something to the BUG bugs list and see what they say about
> this.
>
> Simon.
> --
> Index: grep.c
> ===================================================================
> RCS file: /cvsroot/gnusrc/gnu/dist/grep/src/grep.c,v
> retrieving revision 1.4
> diff -p -u -r1.4 grep.c
> --- grep.c 1999/04/06 16:54:55 1.4
> +++ grep.c 1999/10/26 00:55:55
> @@ -334,6 +334,9 @@ fillbuf (save, stats)
> caddr_t maddr;
> #endif
>
> + /* limit save area to 2MB so that memory doesn't overflow on large files */
> + if (save > 2 * 1024 * 1024)
> + save = 2 * 1024 * 1024;
> if (save > bufsalloc)
> {
> char *nubuffer;
>
From: Todd Whitesel <toddpw@best.com>
To: abs@mono.org (David Brownlee)
Cc: simonb@netbsd.org, Havard.Eidnes@runit.sintef.no,
gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Wed, 27 Oct 1999 00:34:07 -0700 (PDT)
> Maybe it could call getrlimit and base the size of the limited
> buffer on that (1/8th?)
Ugh. These are all cheesy heuristics.
It would be much better if there was a simple system-specific enhancement
requiring HAVE_MMAP, where it picks an address way out in the process space
somewhere and starts mmap-ing the growable buffer there. When it needs to
grow the buffer, it just mmap's some more memory on the end of it.
This is just using sparse addressing to get the benefits of segments...
Admittedly I didn't follow this entire thread, so I'm assuming grep really
needs all that memory, and we should let it mmap away until it hits an rlimit.
Todd Whitesel
toddpw @ best.com
From: David Brownlee <abs@mono.org>
To: Todd Whitesel <toddpw@best.com>
Cc: simonb@netbsd.org, Havard.Eidnes@runit.sintef.no,
gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Thu, 28 Oct 1999 00:47:56 +0100 (BST)
On Wed, 27 Oct 1999, Todd Whitesel wrote:
> > Maybe it could call getrlimit and base the size of the limited
> > buffer on that (1/8th?)
>
> Ugh. These are all cheesy heuristics.
>
> It would be much better if there was a simple system-specific enhancement
> requiring HAVE_MMAP, where it picks an address way out in the process space
> somewhere and starts mmap-ing the growable buffer there. When it needs to
> grow the buffer, it just mmap's some more memory on the end of it.
>
> This is just using sparse addressing to get the benefits of segments...
>
> Admittedly I didn't follow this entire thread, so I'm assuming grep really
> needs all that memory, and we should let it mmap away until it hits an rlimit.
>
As I understand it this is only used for grep as memory to store
a single newline when searching through files.
Arguably any file with several megabytes of data without a newline
is pathalogical (in a text sense), though a grep failing to find
a string in a binary file because it was split across an arbitrary
block boundary is a bad thing...
David/absolute
From: Simon Burge <simonb@NetBSD.ORG>
To: David Brownlee <abs@mono.org>
Cc: Todd Whitesel <toddpw@best.com>, simonb@NetBSD.ORG,
Havard.Eidnes@runit.sintef.no, gnats-bugs@gnats.netbsd.org,
netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Thu, 28 Oct 1999 10:39:38 +1000
David Brownlee wrote:
> On Wed, 27 Oct 1999, Todd Whitesel wrote:
>
> As I understand it this is only used for grep as memory to store
> a single newline when searching through files.
This is correct.
> Arguably any file with several megabytes of data without a newline
> is pathalogical (in a text sense), though a grep failing to find
> a string in a binary file because it was split across an arbitrary
> block boundary is a bad thing...
Unfortunately, some debugging kernels seem to have this behaviour. From
my reading of the code, it's only if the pattern spans the buffer that
you'll get a problem. With my suggested patch, how many regex's span
2MB plus at least that again? If it does find a match, then you'll
"only" get 2MB + plus up to the current buffer (of 10MB) of the line,
not the complete line :-)
This is what the maintainer of grep has to say:
GNU grep needs to buffer a line of input in main memory. If your
input lines are too long to fit into your main memory, then the right
thing to do is to get more memory. See the section `Memory Usage' in
the GNU coding standards for more.
Personally, I'd be happy to put the size limit in and put a note in the
man page stating "you may have problems if you are searching for a regex
that is greater than 4MB, or if you want to see a line greater than 2MB
long that has a match". This doesn't seem unreasonable to me...
Simon.
From: woods@most.weird.com (Greg A. Woods)
To: gnats-bugs@gnats.netbsd.org,
netbsd-bugs@netbsd.org
Cc: Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Wed, 27 Oct 1999 23:15:11 -0400 (EDT)
[ On Thursday, October 28, 1999 at 10:39:38 (+1000), Simon Burge wrote: ]
> Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
>
> This is what the maintainer of grep has to say:
>
> GNU grep needs to buffer a line of input in main memory. If your
> input lines are too long to fit into your main memory, then the right
> thing to do is to get more memory. See the section `Memory Usage' in
> the GNU coding standards for more.
>
> Personally, I'd be happy to put the size limit in and put a note in the
> man page stating "you may have problems if you are searching for a regex
> that is greater than 4MB, or if you want to see a line greater than 2MB
> long that has a match". This doesn't seem unreasonable to me...
Personally I'd say the GNU Grep maintainer(s) has(have) a brain blockage
if that's his(their) attitude! ;-)
Why not simply fail the match for the partial line, optionally print a
warning message to stderr, flush the input buffer (perhaps giving back
as much of it as it had grown beyond the statistical line size), and
start reading again pretending that there had been a newline at the
point where available memory ran out? This might not do very nice
things to the system, but it should make 'grep' continue to work under
adverse conditions....
Assuming the problem is with an RE that does span the majority of a line
up to the newline then perhaps the documentation should mention that the
'out-of-memory' warning indicates the user has chosen a sub-optimal RE
for the data he or she is searching and it might be prudent to try
re-writing the RE so that it didn't try to span an entire line (i.e.
didn't anchor at "$").
--
Greg A. Woods
+1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
From: "R. C. Dowdeswell" <elric@mabelode.imrryr.org>
To: Simon Burge <simonb@netbsd.org>
Cc: David Brownlee <abs@mono.org>, Todd Whitesel <toddpw@best.com>,
Havard.Eidnes@runit.sintef.no, gnats-bugs@gnats.netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Wed, 27 Oct 1999 22:32:20 -0700
On 941071178 seconds since the Beginning of the UNIX epoch
Simon Burge wrote:
>
>This is what the maintainer of grep has to say:
>
> GNU grep needs to buffer a line of input in main memory. If your
> input lines are too long to fit into your main memory, then the right
> thing to do is to get more memory. See the section `Memory Usage' in
> the GNU coding standards for more.
But be that as it may, why is grep using all of that memory, I
wonder. Why not mmap(2) the file MAP_SHARED and PROT_READ. It's
not like grep is going to modify the pages, and shared pages
shouldn't be counted towards memory usage as they are only backed
on disk. Perhaps this gives odd characteristics if someone else
modifies the file, but those'll already exist with read(2) and
write(2).
Even if you just do read/write
$ grep foo big_file
grep: mem exhausted or some such.
and got the message myself. This is a pretty simple state machine,
I can't see any reason that with read and write this would need to
use any memory at all -- just save the position of the beginning
of the line and run the line through the state machine, if it
matches and your buffer is too small, seek to the start of the line
and output from there to the end. Sure that could be slow, but...
Perhaps for some of the more complicated expressions with back
references might be hard to do that way. I haven't given it enough
thought.
In short, I don't think that this should be a problem. Wasn't there
someone writing a BSD grep to replace GNU grep? What came of that?
Does that have this limitation?
== Roland Dowdeswell http://www.Imrryr.ORG/~elric/ ==
== The Unofficial NetBSD Web Pages http://www.Imrryr.ORG/NetBSD/ ==
== The NetBSD Project http://www.NetBSD.ORG/ ==
From: Havard.Eidnes@runit.sintef.no
To: simonb@NetBSD.ORG
Cc: abs@mono.org, toddpw@best.com, gnats-bugs@gnats.netbsd.org,
netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Thu, 28 Oct 1999 09:51:39 +0200
> This is what the maintainer of grep has to say:
>
> GNU grep needs to buffer a line of input in main memory. If
> your input lines are too long to fit into your main memory,
> then the right thing to do is to get more memory. See the
> section `Memory Usage' in the GNU coding standards for more.
Hm, so folks stuck with traditional systems should try to buy more
memory, and if that can't be done (the hp300 can AFAIK only hold
64MB, I suppose other NetBSD platforms have more restrictions).
The end result is that on these systems grep will bomb out and not
search subsequent files if one of these "problem" files are among
the files being searched. I don't think that's acceptable, and I
find the above response just a little bit too snotty for my tastes
(I'm sure you agree).
> Personally, I'd be happy to put the size limit in and put a note
> in the man page stating "you may have problems if you are
> searching for a regex that is greater than 4MB, or if you want to
> see a line greater than 2MB long that has a match". This doesn't
> seem unreasonable to me...
Right. Except the formulation should probably be "you may have
problems if you are searching with a pattern which may end up with a
match larger than 4MB, or if you want to see a line greater than 2MB
long that has a match" (?).
- H=E5vard
From: Richard Earnshaw <rearnsha@arm.com>
To: "R. C. Dowdeswell" <elric@imrryr.org>
Cc: Simon Burge <simonb@netbsd.org>, David Brownlee <abs@mono.org>,
Todd Whitesel <toddpw@best.com>, Havard.Eidnes@runit.sintef.no,
gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Thu, 28 Oct 1999 11:04:23 +0100
elric@mabelode.imrryr.org said:
> Why not mmap(2) the file MAP_SHARED and PROT_READ. It's not like grep
> is going to modify the pages, and shared pages shouldn't be counted
> towards memory usage as they are only backed on disk.
I haven't looked at the code, but it's probably to avoid having to
special-case <stdin>.
R.
From: "R. C. Dowdeswell" <elric@mabelode.imrryr.org>
To: richard.earnshaw@arm.com
Cc: Simon Burge <simonb@netbsd.org>, David Brownlee <abs@mono.org>,
Todd Whitesel <toddpw@best.com>, Havard.Eidnes@runit.sintef.no,
gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Thu, 28 Oct 1999 03:28:55 -0700
On 941105063 seconds since the Beginning of the UNIX epoch
Richard Earnshaw wrote:
>
>
>elric@mabelode.imrryr.org said:
>> Why not mmap(2) the file MAP_SHARED and PROT_READ. It's not like grep
>> is going to modify the pages, and shared pages shouldn't be counted
>> towards memory usage as they are only backed on disk.
>
>I haven't looked at the code, but it's probably to avoid having to
>special-case <stdin>.
They already use HAVE_MMAP, so they must already special case stdin.
But that does bring up an interesting point, that the read(2)/write(2)
method must be fixed regardless of whether one could play games
with how mmap(2) is invoked.
I do still think that a solution should be found, not a workaround.
I ran the experiment on my system which seemed okay, but then I
have a fair bit of memory. I'd hate to see how grep thrashes a
system with little memory if you accidentally grep the `wrong'
file. I really don't think that grep should need to eat that much
memory to efficiently deal with files that have no newline.
== Roland Dowdeswell http://www.Imrryr.ORG/~elric/ ==
== The Unofficial NetBSD Web Pages http://www.Imrryr.ORG/NetBSD/ ==
== The NetBSD Project http://www.NetBSD.ORG/ ==
From: root@ihack.net (Charles M. Hannum)
To: Simon Burge <simonb@netbsd.org>
Cc: David Brownlee <abs@mono.org>, Todd Whitesel <toddpw@best.com>,
Havard.Eidnes@runit.sintef.no, gnats-bugs@gnats.netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: 28 Oct 1999 07:20:38 -0400
Simon Burge <simonb@netbsd.org> writes:
> Personally, I'd be happy to put the size limit in and put a note in the
> man page stating "you may have problems if you are searching for a regex
> that is greater than 4MB, or if you want to see a line greater than 2MB
> long that has a match". This doesn't seem unreasonable to me...
It is completely unreasonable to introduce an arbitrary limit like
this and make the failure mode `silently do the wrong thing'. If
you're going to intentionally make grep(1) do something other than
what it's documented and expected to do, then it should at least fail
with a message explaining what happened and how to work around it.
From: Todd Whitesel <toddpw@best.com>
To: simonb@NetBSD.ORG (Simon Burge)
Cc: abs@mono.org, toddpw@best.com, simonb@NetBSD.ORG,
Havard.Eidnes@runit.sintef.no, gnats-bugs@gnats.netbsd.org,
netbsd-bugs@NetBSD.ORG
Subject: Re: bin/8681: grep may bomb out with "memory exhausted"
Date: Fri, 29 Oct 1999 01:20:43 -0700 (PDT)
> Unfortunately, some debugging kernels seem to have this behaviour.
It isn't just debugging kernels. Any large program with debugging will
trigger this. The real issue here is grepping in debug binaries.
One of the things -g adds to your program is a section containing a huge
block of null-terminated strings. The "real" debug info section is made
up of mostly fixed-length structs, which do not embed strings but instead
use relocated offsets pointing into the strings section. This is how the
debugger prints out your file/function/variable names even when they are
hundreds of characters long (like with C++ templates).
The strings section does not contain any of your program's literal string
constants, so there's not much chance of a newline ever appearing there.
Todd Whitesel
toddpw @ best.com
State-Changed-From-To: open->closed
State-Changed-By: jmmv@NetBSD.org
State-Changed-When: Wed, 27 Apr 2011 21:13:08 +0000
State-Changed-Why:
I've been trying to reproduce this and I haven't been able to. I think this
can be closed, but if I'm wrong please let me know and I'll reopen.
State-Changed-From-To: closed->open
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Wed, 27 Apr 2011 21:20:46 +0000
State-Changed-Why:
I can:
% dd if=/dev/zero bs=102400 count=1024 of=biggy
1024+0 records in
1024+0 records out
104857600 bytes transferred in 1.101 secs (95238510 bytes/sec)
% echo foo >> biggy
% grep foo biggy
Binary file biggy matches
So far so good. Now:
% limit memoryuse 1024k
% limit vmemoryuse 1024k
% limit stacksize 1024k
% limit datasize 1024k
% grep foo biggy
Cannot map anonymous memoryCannot allocate memory: Cannot allocate memory
Exit 1
From: Julio Merino <jmmv@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dholland@netbsd.org,
Havard.Eidnes@runit.sintef.no
Subject: Re: bin/8681 (grep may bomb out with "memory exhausted")
Date: Wed, 27 Apr 2011 22:23:17 +0100
On Wed, Apr 27, 2011 at 10:20 PM, <dholland@netbsd.org> wrote:
> So far so good. Now:
> % limit memoryuse 1024k
> % limit vmemoryuse 1024k
> % limit stacksize 1024k
> % limit datasize 1024k
> % grep foo biggy
> Cannot map anonymous memoryCannot allocate memory: Cannot allocate memory
> Exit 1
Ah, cool. I was trying with the test case in the PR and that didn't
work, but your commands do indeed show the problem.
I'll pay the price by adding a xfail test for this :-P
--
Julio Merino / @jmmv
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/8681 (grep may bomb out with "memory exhausted")
Date: Wed, 27 Apr 2011 21:24:12 +0000
On Wed, Apr 27, 2011 at 09:20:48PM +0000, dholland@NetBSD.org wrote:
> I can:
> [...]
1024k isn't enough to do anything, so that doesn't actually count.
however, with limits of 102400k:
% grep foo /usr/share/dict/words | wc -l
241
% grep foo biggy
grep: memory exhausted
Exit 1
%
--
David A. Holland
dholland@netbsd.org
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home