NetBSD Problem Report #42876

From njoly@lanfeust.sis.pasteur.fr  Tue Feb 23 14:40:50 2010
Return-Path: <njoly@lanfeust.sis.pasteur.fr>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 985C163C49F
	for <gnats-bugs@gnats.NetBSD.org>; Tue, 23 Feb 2010 14:40:50 +0000 (UTC)
Message-Id: <20100223144047.5E75EDC9B9@lanfeust.sis.pasteur.fr>
Date: Tue, 23 Feb 2010 15:40:47 +0100 (CET)
From: njoly@pasteur.fr
Reply-To: njoly@pasteur.fr
To: gnats-bugs@gnats.NetBSD.org
Subject: realloc crash with threads
X-Send-Pr-Version: 3.95

>Number:         42876
>Category:       lib
>Synopsis:       realloc crash with threads
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Feb 23 14:45:00 +0000 2010
>Closed-Date:    Mon Jun 07 05:05:44 +0000 2010
>Last-Modified:  Mon Jun 07 05:05:44 +0000 2010
>Originator:     Nicolas Joly
>Release:        NetBSD 5.99.24
>Organization:
Institut Pasteur
>Environment:
System: NetBSD lanfeust.sis.pasteur.fr 5.99.24 NetBSD 5.99.24 (LANFEUST) #6: Tue Feb 23 13:24:16 CET 2010 njoly@lanfeust.sis.pasteur.fr:/local/src/NetBSD/obj.amd64/sys/arch/amd64/compile/LANFEUST amd64
Architecture: x86_64
Machine: amd64
>Description:
There seems to be a race when using concurrent realloc call with multiple
threads. The following testcase, which makes 4 threads call realloc with
random values, crash with a NULL pointer dereference.

njoly@lanfeust [netbsd/threads]> cat thread_realloc.c
#include <err.h>
#include <pthread.h>
#include <stdlib.h>
#include <unistd.h>

#define THR_NUM 4

static int quit = 0;

void *thr_func(void *arg) {
  int val;
  size_t len;
  void *buf, *new;

  srand(time(NULL));

  buf = new = NULL;
  while (quit != 1) {
    val = rand() % 10; len = val * 1024 * 1024;
    new = realloc(buf, len);
    if (len && new == NULL) { break; }
    buf = new;
  }
  free(buf);

  return NULL; }

int main() {
  int res, i;
  pthread_t thr[THR_NUM];

  for (i = 0; i < THR_NUM; i++) {
    res = pthread_create(&thr[i], NULL, thr_func, NULL);
    if (res != 0)
      errx(1, "pthread_create failed");
  }

  sleep(10);
  quit = 1;

  for (i = 0; i < THR_NUM; i++) {
    res = pthread_join(thr[i], NULL);
    if (res != 0)
      errx(1, "pthread_join failed");
  }

  return 0; }
njoly@lanfeust [netbsd/threads]> cc -pthread -g -Wall -Werror   -o thread_realloc thread_realloc.c
njoly@lanfeust [netbsd/threads]> ./thread_realloc 
zsh: segmentation fault (core dumped)  ./thread_realloc
njoly@lanfeust [netbsd/threads]> gdb thread_realloc thread_realloc.core 
GNU gdb 6.5
[...]
Core was generated by `thread_realloc'.
Program terminated with signal 11, Segmentation fault.
#0  realloc (ptr=0x7f7ffaa00000, size=9437184)
    at /local/src/NetBSD/src/lib/libc/stdlib/jemalloc.c:2872
2872                            node->size = newcsize;
(gdb) p node->size
Cannot access memory at address 0x28
(gdb) p node
$1 = (chunk_node_t *) 0x0
(gdb) bt
#0  realloc (ptr=0x7f7ffaa00000, size=9437184)
    at /local/src/NetBSD/src/lib/libc/stdlib/jemalloc.c:2872
#1  0x0000000000400b66 in thr_func (arg=0x0) at thread_realloc.c:21
#2  0x00007f7ffd80a660 in pthread__create_tramp (cookie=<value optimized out>)
    at /local/src/NetBSD/src/lib/libpthread/pthread.c:470
#3  0x00007f7ffd46e4e0 in ___lwp_park50 () from /usr/lib/libc.so.12
Cannot access memory at address 0x7f7ffd400000
(gdb) info threads
  5 process 72227  0x00007f7ffd43672a in _sys___nanosleep50 ()
   from /usr/lib/libc.so.12
  4 process 203299  pthread__mutex_pause ()
    at /local/src/NetBSD/src/lib/libpthread/pthread_mutex.c:171
  3 process 268835  0x00007f7ffd46e4ca in ___lwp_park50 ()
   from /usr/lib/libc.so.12
  2 process 334371  0x00007f7ffd46e4ca in ___lwp_park50 ()
   from /usr/lib/libc.so.12
* 1 process 137763  realloc (ptr=0x7f7ffaa00000, size=9437184)
    at /local/src/NetBSD/src/lib/libc/stdlib/jemalloc.c:2872

>How-To-Repeat:
Run the provided testcase.
>Fix:
n/a

>Release-Note:

>Audit-Trail:
From: enami tsugutomo <enami@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42876 CVS commit: src/lib/libc/stdlib
Date: Thu, 4 Mar 2010 22:48:31 +0000

 Module Name:	src
 Committed By:	enami
 Date:		Thu Mar  4 22:48:31 UTC 2010

 Modified Files:
 	src/lib/libc/stdlib: jemalloc.c

 Log Message:
 Fix race condition on reallocation of huge category.

 We need to remove the old region before mremap() since if it relesae the
 old region, other thread may map it for the same huge category allocation
 and insert it to the tree before we acquire a lock after mremap().

 Fixes PR/42876.


 To generate a diff of this commit:
 cvs rdiff -u -r1.20 -r1.21 src/lib/libc/stdlib/jemalloc.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Jeff Rizzo <riz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42876 CVS commit: [netbsd-5] src/lib/libc/stdlib
Date: Sat, 13 Mar 2010 00:53:33 +0000

 Module Name:	src
 Committed By:	riz
 Date:		Sat Mar 13 00:53:33 UTC 2010

 Modified Files:
 	src/lib/libc/stdlib [netbsd-5]: jemalloc.c

 Log Message:
 Pull up following revision(s) (requested by enami in ticket #1327):
 	lib/libc/stdlib/jemalloc.c: revision 1.21
 Fix race condition on reallocation of huge category.
 We need to remove the old region before mremap() since if it relesae the
 old region, other thread may map it for the same huge category allocation
 and insert it to the tree before we acquire a lock after mremap().
 Fixes PR/42876.


 To generate a diff of this commit:
 cvs rdiff -u -r1.19 -r1.19.4.1 src/lib/libc/stdlib/jemalloc.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 07 Jun 2010 04:37:47 +0000
State-Changed-Why:
Fixed?


From: Nicolas Joly <njoly@pasteur.fr>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/42876 (realloc crash with threads)
Date: Mon, 7 Jun 2010 07:04:12 +0200

 On Mon, Jun 07, 2010 at 04:37:49AM +0000, dholland@NetBSD.org wrote:
 > Synopsis: realloc crash with threads
 > 
 > State-Changed-From-To: open->feedback
 > State-Changed-By: dholland@NetBSD.org
 > State-Changed-When: Mon, 07 Jun 2010 04:37:47 +0000
 > State-Changed-Why:
 > Fixed?

 Yes. No more problem.

 Thanks.

 -- 
 Nicolas Joly

 Biological Software and Databanks.
 Institut Pasteur, Paris.

State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 07 Jun 2010 05:05:44 +0000
State-Changed-Why:
Confirmed fixed. Thanks for the prompt response :-)


>Unformatted:

Home	PR Database Search