NetBSD Problem Report #42183

From njoly@lanfeust.sis.pasteur.fr  Wed Oct 14 13:07:26 2009
Return-Path: <njoly@lanfeust.sis.pasteur.fr>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id B280B63B877
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 14 Oct 2009 13:07:26 +0000 (UTC)
Message-Id: <20091014130723.7BFC1DC9B9@lanfeust.sis.pasteur.fr>
Date: Wed, 14 Oct 2009 15:07:23 +0200 (CEST)
From: njoly@pasteur.fr
Reply-To: njoly@pasteur.fr
To: gnats-bugs@gnats.NetBSD.org
Subject: sed(1) memory leak
X-Send-Pr-Version: 3.95

>Number:         42183
>Category:       bin
>Synopsis:       sed(1) memory leak
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 14 13:10:00 +0000 2009
>Closed-Date:    Wed Oct 14 20:57:39 +0000 2009
>Last-Modified:  Wed Oct 14 22:00:04 +0000 2009
>Originator:     Nicolas Joly
>Release:        NetBSD 5.99.20
>Organization:
Intitut Pasteur
>Environment:
System: NetBSD lanfeust.sis.pasteur.fr 5.99.20 NetBSD 5.99.20 (LANFEUST) #2: Tue Oct 13 20:49:48 CEST 2009 njoly@lanfeust.sis.pasteur.fr:/local/src/NetBSD/obj.amd64/sys/arch/amd64/compile/LANFEUST amd64
Architecture: x86_64
Machine: amd64
>Description:
While processing a 41GB file with sed, i noticed that the process was eating
memory up top 4GB when i stoppped it.

A further analysis show that it leaks 128 bytes for each processed line.

njoly@lanfeust [~]> printf "" | mleaks sed '' >/dev/null
sed: found 3 unallocated objects (56 bytes)
        from sed:err+0x109 ... 16 bytes
        from sed:err+0x109 ... 16 bytes
        from sed:err+0x109 ... 24 bytes
njoly@lanfeust [~]> printf "foo\n" | mleaks sed '' >/dev/null 
sed: found 5 unallocated objects (1213 bytes)
        from sed:err+0x109 ... 1029 bytes
        from libc.so.12:__getdelim+0x7f ... 128 bytes
        from sed:err+0x109 ... 16 bytes
        from sed:err+0x109 ... 16 bytes
        from sed:err+0x109 ... 24 bytes
njoly@lanfeust [~]> printf "foo\nfoo\n" | mleaks sed '' >/dev/null
sed: found 6 unallocated objects (1341 bytes)
        from libc.so.12:__getdelim+0x7f ... 128 bytes
        from sed:err+0x109 ... 1029 bytes
        from libc.so.12:__getdelim+0x7f ... 128 bytes
        from sed:err+0x109 ... 16 bytes
        from sed:err+0x109 ... 16 bytes
        from sed:err+0x109 ... 24 bytes
And so on ...

The leaks comes from the fgetstr() call in sed sources :

#0  __getdelim (buf=0x7f7fffffda30, buflen=0x7f7fffffda28, sep=10, 
    fp=0x7f7ffdb13cc0) at /local/src/NetBSD/src/lib/libc/stdio/getdelim.c:58
#1  0x00007f7ffd8a663b in __fgetstr (fp=0x7f7ffdb13cc0, lenp=0x7f7fffffda58, 
    sep=10) at /local/src/NetBSD/src/lib/libc/stdio/fgetstr.c:60
#2  0x00000000004029ea in mf_fgets (sp=0x607c20, spflag=REPLACE)
    at /local/src/NetBSD/src/usr.bin/sed/main.c:336
#3  0x0000000000403721 in process ()
    at /local/src/NetBSD/src/usr.bin/sed/process.c:137
#4  0x0000000000403071 in main (argc=3, argv=0x7f7fffffdb38)
    at /local/src/NetBSD/src/usr.bin/sed/main.c:197

>How-To-Repeat:
Try to run sed on a very big file ... and watch the process memory increase.
>Fix:
n/a

>Release-Note:

>Audit-Trail:
From: Roy Marples <roy@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42183 CVS commit: src/lib/libc/stdio
Date: Wed, 14 Oct 2009 20:54:51 +0000

 Module Name:	src
 Committed By:	roy
 Date:		Wed Oct 14 20:54:51 UTC 2009

 Modified Files:
 	src/lib/libc/stdio: fgetstr.c

 Log Message:
 Store the allocated buffer against FILE, plugging a memory leak.
 Fixes PR bin/42183.


 To generate a diff of this commit:
 cvs rdiff -u -r1.6 -r1.7 src/lib/libc/stdio/fgetstr.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: roy@NetBSD.org
State-Changed-When: Wed, 14 Oct 2009 20:57:39 +0000
State-Changed-Why:
Fixed by commit


From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: PR/42183 CVS commit: src/lib/libc/stdio
Date: Wed, 14 Oct 2009 17:58:15 -0400

 On Oct 14,  8:55pm, roy@netbsd.org (Roy Marples) wrote:
 -- Subject: PR/42183 CVS commit: src/lib/libc/stdio

 | The following reply was made to PR bin/42183; it has been noted by GNATS.
 | 
 | From: Roy Marples <roy@netbsd.org>
 | To: gnats-bugs@gnats.NetBSD.org
 | Cc: 
 | Subject: PR/42183 CVS commit: src/lib/libc/stdio
 | Date: Wed, 14 Oct 2009 20:54:51 +0000
 | 
 |  Module Name:	src
 |  Committed By:	roy
 |  Date:		Wed Oct 14 20:54:51 UTC 2009
 |  
 |  Modified Files:
 |  	src/lib/libc/stdio: fgetstr.c
 |  
 |  Log Message:
 |  Store the allocated buffer against FILE, plugging a memory leak.
 |  Fixes PR bin/42183.
 |  
 |  
 |  To generate a diff of this commit:
 |  cvs rdiff -u -r1.6 -r1.7 src/lib/libc/stdio/fgetstr.c
 |  
 |  Please note that diffs are not public domain; they are subject to the
 |  copyright notices on the relevant files.

 This change still leaks on error. assume that you end up realloc'ing
 in __getdelim, but in the end you still return an error... Then p
 points to the new buffer and base to the old one. You should either
 free(p) on error or pass the right pointer in.

 christos

>Unformatted:

Home	PR Database Search