NetBSD Problem Report #46781

From www@NetBSD.org  Tue Aug  7 19:15:02 2012
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 6759E63B882
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  7 Aug 2012 19:15:02 +0000 (UTC)
Message-Id: <20120807191501.7CBDD63B85F@www.NetBSD.org>
Date: Tue,  7 Aug 2012 19:15:01 +0000 (UTC)
From: cfuhrman@panix.com
Reply-To: cfuhrman@panix.com
To: gnats-bugs@NetBSD.org
Subject: Under NetBSD 6.0 bash will crash when LC_ALL=en_US.UTF-8 (and possibly any UTF-8 locale)
X-Send-Pr-Version: www-1.0

>Number:         46781
>Category:       lib
>Synopsis:       Under NetBSD 6.0 bash will crash when LC_ALL=en_US.UTF-8 (and possibly any UTF-8 locale)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    lib-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Aug 07 19:20:00 +0000 2012
>Closed-Date:    Sun Sep 02 03:46:01 +0000 2012
>Last-Modified:  Sun Sep 02 03:46:01 +0000 2012
>Originator:     Christopher M. Fuhrman
>Release:        NetBSD 6.0 BETA2
>Organization:
>Environment:
NetBSD cmf-netbsd.stanford.edu 6.0_BETA2 NetBSD 6.0_BETA2 (GENERIC) #0: Wed Jul 18 11:10:14 PDT 2012  root@cmf-netbsd.stanford.edu:/usr/obj/sys/arch/amd64/compile/GENERIC amd64
>Description:
Setting LC_ALL to en_US.UTF-8 will cause bash to crash with the following output:

-bash-4.2$ export LC_ALL=en_US.UTF-8 

malloc: unknown:0: assertion botched
free: start and end chunk sizes differ
last command: export LC_ALL=en_US.UTF-8
Aborting...Connection to w.x.y.z closed.

I see the same effect using the following sampled locales:

 - en_CA.UTF-8
 - en_GB.UTF-8
 - zh_TW.UTF-8

Note this *DOES NOT* happen under NetBSD 5.1.2.  This is specific to NetBSD 6.0 only.


>How-To-Repeat:
Under NetBSD 6.0_BETA2:

 - Set bash options to static in /etc/mk.conf

   PKG_OPTIONS.bash=static

 - Create a user account using bash for the shell
 - Log in as this user
 - set LC_ALL to en_US.UTF-8 (or any unicode locale)

   export LC_ALL=en_US.UTF-8 

 - watch bash crash


>Fix:
Setting the locale to en_US.ISO8859-1 seems to work okay.

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->lib-bug-people
Responsible-Changed-By: obache@NetBSD.org
Responsible-Changed-When: Wed, 08 Aug 2012 04:19:33 +0000
Responsible-Changed-Why:
problem in base libc.


From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46781: statically compiled bash makes locale loading fail
Date: Wed, 8 Aug 2012 12:24:12 +0200

 I am not yet convinced this is a library bug. Bash replaces malloc() and
 free() with its own versions, and the _rune_read_file() functions fails
 somewhere, calling bash's version of free.

 This function then fails with an assertion:

 #1  0x000000000013d410 in programming_error (
     format=0x2c5df8 "free: start and end chunk sizes differ") at error.c:176
 #2  0x0000000000219494 in xbotch (mem=0x42c008, e=8, 
     s=0x2c5df8 "free: start and end chunk sizes differ", file=0x0, line=0)
     at malloc.c:319
 #3  0x000000000021a7bc in internal_free (mem=0x42c008, file=0x0, line=0, 
     flags=0) at malloc.c:902
 #4  0x000000000021b2f8 in free (mem=0x42c008) at malloc.c:1269
 #5  0x000000000027b284 in _rune_load ()

 This could be the rune code writing out of bounds somewhere - or a bug in
 the malloc/free implementation used in bash. For starters, it does not
 provide proper alignement for a general malloc on all archs:

 #define MALIGN_MASK     7       /* one less than desired alignment */

 so I have not a lot of trust in it. However, fixing that alignement issue,
 the local loading failure perrsists.

 Please someone check why _rune_read_file goes to err: at all, and wether
 anything has been overwritten.

 Why bash needs its own memory allocator at all is beyound me (maybe the
 pkg should completely avoid that?)

 Martin

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46781: statically compiled bash makes locale loading fail
Date: Wed, 8 Aug 2012 12:45:32 +0200

 On closer inspection it looks like indeed the rune loading code writes
 past the end of the allocated block.

 Martin

From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, cfuhrman@panix.com
Cc: 
Subject: Re: lib/46781: statically compiled bash makes locale loading fail
Date: Wed, 8 Aug 2012 07:07:34 -0400

 On Aug 8, 10:50am, martin@duskware.de (Martin Husemann) wrote:
 -- Subject: Re: lib/46781: statically compiled bash makes locale loading fail

 | The following reply was made to PR lib/46781; it has been noted by GNATS.
 | 
 | From: Martin Husemann <martin@duskware.de>
 | To: gnats-bugs@NetBSD.org
 | Cc: 
 | Subject: Re: lib/46781: statically compiled bash makes locale loading fail
 | Date: Wed, 8 Aug 2012 12:45:32 +0200
 | 
 |  On closer inspection it looks like indeed the rune loading code writes
 |  past the end of the allocated block.

 The way the loader is written is antiquated, complicated, and error prone. 
 It will be extremely difficult to add tests to easily prove that the code
 does not have buffer overflow errors on specially crafted input.

 christos

From: SODA Noriyuki <soda@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org,
    gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org,
    cfuhrman@panix.com
Subject: Re: lib/46781: statically compiled bash makes locale loading fail
Date: Wed, 8 Aug 2012 20:13:47 +0900

 nonaka@-san and enami@-san looked at this problem, and found that
 the following patch can be used as a workaround:
 http://gist.github.com/3291695

 And tnozaki@-san is currently working to make real fix...
 -- 
 soda

From: "Takehiko NOZAKI" <tnozaki@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46781 CVS commit: src/lib/libc/locale
Date: Wed, 8 Aug 2012 18:37:26 +0000

 Module Name:	src
 Committed By:	tnozaki
 Date:		Wed Aug  8 18:37:26 UTC 2012

 Modified Files:
 	src/lib/libc/locale: rune.c

 Log Message:
 fix PR lib/46781 statically compiled bash makes locale loading fail.
 analyzed and  patch provided by enami@ nonaka@ obache@ soda@, thanks.
 patch modified by me.


 To generate a diff of this commit:
 cvs rdiff -u -r1.43 -r1.44 src/lib/libc/locale/rune.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: tnozaki@NetBSD.org
State-Changed-When: Wed, 08 Aug 2012 18:49:08 +0000
State-Changed-Why:
fixed in HEAD, thanks.

From: "Jeff Rizzo" <riz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46781 CVS commit: [netbsd-6] src/lib/libc/locale
Date: Mon, 20 Aug 2012 19:27:04 +0000

 Module Name:	src
 Committed By:	riz
 Date:		Mon Aug 20 19:27:04 UTC 2012

 Modified Files:
 	src/lib/libc/locale [netbsd-6]: rune.c

 Log Message:
 Pull up following revision(s) (requested by tnozaki in ticket #518):
 	lib/libc/locale/rune.c: revision 1.44
 	lib/libc/locale/rune.c: revision 1.45
 Restore RCS Id lost in previous.
 fix PR lib/46781 statically compiled bash makes locale loading fail.
 analyzed and  patch provided by enami@ nonaka@ obache@ soda@, thanks.
 patch modified by me.


 To generate a diff of this commit:
 cvs rdiff -u -r1.43 -r1.43.2.1 src/lib/libc/locale/rune.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Christopher M. Fuhrman" <cfuhrman@panix.com>
To: gnats-bugs@gnats.netbsd.org
Cc: 
Subject: Re: lib/46781: statically compiled bash makes locale loading fail
Date: Fri, 31 Aug 2012 14:30:29 -0700 (PDT)

 Howdy,

 As of NetBSD 6.0_RC1, where the fix was pulled-up into the netbsd-6
 branch[1], I am now able to set locale to en_US.UTF-8 using statically
 compiled bash.

 Thanks for the quick response :)


 Footnotes:
 [1]  http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/locale/rune.c?rev=1.43.2.1&content-type=text/x-cvsweb-markup&only_with_tag=netbsd-6

 -- 
 Chris Fuhrman
 cfuhrman@panix.com


State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 02 Sep 2012 03:46:01 +0000
State-Changed-Why:
Confirmed fixed.


>Unformatted:

Home	PR Database Search