NetBSD Problem Report #50316

From dyoung@whitecoralislands.com  Thu Oct  8 20:24:34 2015
Return-Path: <dyoung@whitecoralislands.com>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 5ADBDA5858
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  8 Oct 2015 20:24:34 +0000 (UTC)
Message-Id: <20151008202423.364621BF55B@elmendorf.whitecoralislands.com>
Date: Thu,  8 Oct 2015 15:24:23 -0500 (CDT)
From: dyoung@whitecoralislands.com
Reply-To: dyoung@whitecoralislands.com
To: gnats-bugs@gnats.NetBSD.org
Subject: mergesort(nmemb=0) is undefined
X-Send-Pr-Version: 3.95

>Number:         50316
>Category:       lib
>Synopsis:       mergesort(nmemb=0) is undefined
>Confidential:   no
>Severity:       non-critical
>Priority:       high
>Responsible:    lib-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 08 20:25:00 +0000 2015
>Closed-Date:    Tue Sep 05 01:31:57 +0000 2017
>Last-Modified:  Tue Sep 05 15:00:01 +0000 2017
>Originator:     David Young
>Release:        NetBSD 5.99.56
>Organization:
David Young
dyoung@pobox.com    Urbana, IL    (217) 721-9981
>Environment:
Architecture: i386
Machine: i386
>Description:
	mergesort(nmemb=0) calls setup(n=0) calls insertionsort(n=0).
	In insertionsort(), n is a size_t.  insertionsort() subtracts
	1 from n right away,

	        for (ai = a+size; --n >= 1; ai += size)

        and compares with 1.  Since n is unsigned, it wraps to a
        huge number instead of turning to -1, so insertionsort()
        doesn't quit the for-loop immediately, like it should.
        Instead, it calls the comparison function on a couple of
        fictional array members, leading to a segmentation fault.
>How-To-Repeat:
	Here is a simple program that demonstrates the problem. 

	#include <stdlib.h>

	int
	main(int argc, char **argv)
	{
		mergesort(NULL, 0, sizeof(void *) / 2, NULL);
		return 0;
	}
>Fix:
	Workaround: don't mergesort() an array with 0 members.

        Suggested fix: compare nmemb with 0 at the top of mergesort()
        and if equal, return 0 immediately?

>Release-Note:

>Audit-Trail:
From: "Brian Ginsbach" <ginsbach@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/50316 CVS commit: src/lib/libc/stdlib
Date: Sat, 12 Aug 2017 01:10:04 +0000

 Module Name:	src
 Committed By:	ginsbach
 Date:		Sat Aug 12 01:10:04 UTC 2017

 Modified Files:
 	src/lib/libc/stdlib: merge.c

 Log Message:
 PR lib/50316: Gracefully handle a zero number of members argument.

 Taken from FreeBSD (which fixed this same issue long ago).

 XXX: pullup-8
 XXX: pullup-7
 XXX: pullup-6


 To generate a diff of this commit:
 cvs rdiff -u -r1.14 -r1.15 src/lib/libc/stdlib/merge.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: ginsbach@NetBSD.org
State-Changed-When: Sat, 12 Aug 2017 01:20:37 +0000
State-Changed-Why:
Fix committed. Does this resolve the issue?  (If not obvious since FreeBSD
has had the same fix for over 18 years.)


From: "Soren Jacobsen" <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/50316 CVS commit: [netbsd-8] src/lib/libc/stdlib
Date: Sun, 20 Aug 2017 04:55:40 +0000

 Module Name:	src
 Committed By:	snj
 Date:		Sun Aug 20 04:55:40 UTC 2017

 Modified Files:
 	src/lib/libc/stdlib [netbsd-8]: merge.c

 Log Message:
 Pull up following revision(s) (requested by ginsbach in ticket #214):
 	lib/libc/stdlib/merge.c: revision 1.15
 PR lib/50316: Gracefully handle a zero number of members argument.
 Taken from FreeBSD (which fixed this same issue long ago).


 To generate a diff of this commit:
 cvs rdiff -u -r1.14 -r1.14.24.1 src/lib/libc/stdlib/merge.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: feedback->pending-pullups
State-Changed-By: ginsbach@NetBSD.org
State-Changed-When: Sat, 26 Aug 2017 18:11:43 +0000
State-Changed-Why:
Feedback email undeliverable.  Pullups were requested concurrent with
feedback request.


From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/50316 CVS commit: [netbsd-6] src/lib/libc/stdlib
Date: Thu, 31 Aug 2017 15:05:50 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Thu Aug 31 15:05:50 UTC 2017

 Modified Files:
 	src/lib/libc/stdlib [netbsd-6]: merge.c

 Log Message:
 Pull up following revision(s) (requested by ginsbach in ticket #1495):
 	lib/libc/stdlib/merge.c: revision 1.15
 PR lib/50316: Gracefully handle a zero number of members argument.
 Taken from FreeBSD (which fixed this same issue long ago).
 XXX: pullup-8
 XXX: pullup-7
 XXX: pullup-6


 To generate a diff of this commit:
 cvs rdiff -u -r1.13 -r1.13.6.1 src/lib/libc/stdlib/merge.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Soren Jacobsen" <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/50316 CVS commit: [netbsd-7] src/lib/libc/stdlib
Date: Mon, 4 Sep 2017 06:21:45 +0000

 Module Name:	src
 Committed By:	snj
 Date:		Mon Sep  4 06:21:45 UTC 2017

 Modified Files:
 	src/lib/libc/stdlib [netbsd-7]: merge.c

 Log Message:
 Pull up following revision(s) (requested by ginsbach in ticket #1495):
 	lib/libc/stdlib/merge.c: revision 1.15
 PR lib/50316: Gracefully handle a zero number of members argument.
 Taken from FreeBSD (which fixed this same issue long ago).


 To generate a diff of this commit:
 cvs rdiff -u -r1.14 -r1.14.10.1 src/lib/libc/stdlib/merge.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closeD
State-Changed-By: ginsbach@NetBSD.org
State-Changed-When: Tue, 05 Sep 2017 01:31:57 +0000
State-Changed-Why:
Pullups complete to netbsd-6, netbsd-7, and netbsd-8.


State-Changed-From-To: closeD->closed
State-Changed-By: leot@NetBSD.org
State-Changed-When: Tue, 05 Sep 2017 07:20:44 +0000
State-Changed-Why:
s/closeD/closed/


From: Patrick Welche <prlw1@cam.ac.uk>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/50316 (mergesort(nmemb=0) is undefined)
Date: Tue, 5 Sep 2017 08:30:30 +0100

 On Tue, Sep 05, 2017 at 01:31:58AM +0000, ginsbach@NetBSD.org wrote:
 > Pullups complete to netbsd-6, netbsd-7, and netbsd-7.

 s/netbsd-7/netbsd-8/ ?

From: Brian Ginsbach <ginsbach@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org, dyoung@whitecoralislands.com
Subject: Re: lib/50316 (mergesort(nmemb=0) is undefined)
Date: Tue, 5 Sep 2017 14:55:12 +0000

 On Tue, Sep 05, 2017 at 07:35:01AM +0000, Patrick Welche wrote:
 > The following reply was made to PR lib/50316; it has been noted by GNATS.
 > 
 > From: Patrick Welche <prlw1@cam.ac.uk>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: lib/50316 (mergesort(nmemb=0) is undefined)
 > Date: Tue, 5 Sep 2017 08:30:30 +0100
 > 
 >  On Tue, Sep 05, 2017 at 01:31:58AM +0000, ginsbach@NetBSD.org wrote:
 >  > Pullups complete to netbsd-6, netbsd-7, and netbsd-7.
 >  
 >  s/netbsd-7/netbsd-8/ ?
 >  

 Yes, clearly. I had a terrible case of fat fingers on this one...

>Unformatted:

Home	PR Database Search