NetBSD Problem Report #41893

From mmondor@pulsar-zone.net  Sat Aug 15 21:05:02 2009
Return-Path: <mmondor@pulsar-zone.net>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 812CF63C284
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 15 Aug 2009 21:05:02 +0000 (UTC)
Message-Id: <200908152104.n7FL4cRE004147@ginseng.xisop>
Date: Sat, 15 Aug 2009 17:04:38 -0400 (EDT)
From: Matthew Mondor <mm_lists@pulsar-zone.net>
Reply-To: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@gnats.NetBSD.org
Subject: Should devel/boehm-gc be built with thread support by default?
X-Send-Pr-Version: 3.95


>Number:         41893
>Category:       pkg
>Synopsis:       Should devel/boehm-gc be built with thread support by default?
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    pkg-manager
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Sat Aug 15 21:10:00 +0000 2009
>Last-Modified:  Sat Nov 05 23:30:07 +0000 2011
>Originator:     Matthew Mondor
>Release:        NetBSD 5.0_STABLE
>Organization:
>Environment:
System: NetBSD ginseng.xisop 5.0_STABLE NetBSD 5.0_STABLE (GENERIC_MM) #4: Thu Aug 6 01:01:17 EDT 2009 root@behemoth.xisop:/usr/obj/sys/arch/i386/compile/GENERIC_MM i386
Architecture: i386
Machine: i386
>Description:
	Some packages which can support threads but which are using
	boehm-gc need a boehm-gc also built with threads support.
	This option exists but is disabled by default.


	I've been using ECL with threads built against boehm-gc
	built with threads support, and that same thread-enabled
	boehm-gc is also used here by inkscape and elinks and it
	seems stable (note that these later two do not require a
	thread-enabled boehm-gc however).


	So should boehm-gc be built with thread support by default,
	at least on NetBSD-5+?
>How-To-Repeat:
>Fix:


>Release-Note:


>Audit-Trail:


Responsible-Changed-From-To: pkg-manager->recht
Responsible-Changed-By: asau@NetBSD.org
Responsible-Changed-When: Sat, 15 Aug 2009 21:46:55 +0000
Responsible-Changed-Why:
Assigned to maintainer.


(I propose turning threads on by default for NetBSD, FreeBSD, Linux
and Darwin at least.)




Responsible-Changed-From-To: recht->pkg-manager
Responsible-Changed-By: wiz@NetBSD.org
Responsible-Changed-When: Mon, 28 Feb 2011 15:08:54 +0000
Responsible-Changed-Why:
Reset responsible field for retired developer.




From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/41893: Should devel/boehm-gc be built with thread support
 by default?
Date: Sun, 28 Aug 2011 11:19:40 -0400


 How about this?  recht (the previous maintainer) retired since.

 
 Another possibility would be to have separate boehm-gc and
 boehm-gc-threads package, and have ECL depend on boehm-gc-threads via
 buildlink.

 
 It would also be possible to tell ECL to build and embed its own
 boehm-gc, which is part of the tree, although this duplicates things a
 little in RAM.  Is it a valid concern?

 
 Yet we since force pthread support for everything that uses dlopen(3),
 so why not also enable the current thread option by default on
 netbsd-5+ and Linux? (boehm-gc doesn't use DLFCN(3), though).

 
 Alternatively, would it be possible for the ECL package, if built with
 the threaded option, to also tell its dependency, of not already
 present (boehm-gc) to also build with threads support?  This is not
 perfect though, as boehm-gc might already have been built.  In which
 case, perhaps that building ECL with threads should fail with an error,
 unless using an option to tell it to embed its own?  (or, it actually
 could enable an option automatically in this case to build its own)?

 
 Thanks,
 -- 
 Matt

 
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/41893: Should devel/boehm-gc be built with thread support by
 default?
Date: Sat, 5 Nov 2011 13:52:24 +0000


 On Sun, Aug 28, 2011 at 03:20:03PM +0000, Matthew Mondor wrote:
  >  How about this?  recht (the previous maintainer) retired since.
  >  
  >  Another possibility would be to have separate boehm-gc and
  >  boehm-gc-threads package, and have ECL depend on boehm-gc-threads via
  >  buildlink.
  >
  >  It would also be possible to tell ECL to build and embed its own
  >  boehm-gc, which is part of the tree, although this duplicates things a
  >  little in RAM.  Is it a valid concern?

 
 The primary concern is that if a vulnerability appears in the package,
 someone has to remember where the extra private copies are and mark
 those packages vulnerable too... and then someone has to apply the
 fixes. This is usually a bad idea, so we like to not use included
 private copies of things.

 
  >  Yet we since force pthread support for everything that uses dlopen(3),
  >  so why not also enable the current thread option by default on
  >  netbsd-5+ and Linux? (boehm-gc doesn't use DLFCN(3), though).

 
 Well, the concern I'd have is that thread support may add a lot of
 overhead. Threaded garbage collection is painful enough in general
 that people go to substantial lengths to avoid it on purpose; I don't
 know to what extent this applies to boehm-gc's thread support.

 
 If there's not a significant problem, then we should enable it by
 default, as non-threaded/single-core environments are not the future
 or even at this point the present.

 
 If there is a problem, a separate package is probably the way to go;
 however, I'd go so far as to say the default package should be
 threaded and the other package should be "boehm-gc-unthreaded".

 
 As to whether there's a problem... that's the sort of thing we have
 package maintainers to know about. I have no idea. If you'd be willing
 to look into it, which hopefully by now should require only digging
 around and not actually benchmarking things, we can probably move
 forward on this...

 
  >  Alternatively, would it be possible for the ECL package, if built with
  >  the threaded option, to also tell its dependency, of not already
  >  present (boehm-gc) to also build with threads support?  This is not
  >  perfect though, as boehm-gc might already have been built.  In which
  >  case, perhaps that building ECL with threads should fail with an error,
  >  unless using an option to tell it to embed its own?  (or, it actually
  >  could enable an option automatically in this case to build its own)?

 
 This sort of thing doesn't really work and probably isn't really a
 good idea...

 
 -- 
 David A. Holland
 dholland@netbsd.org

 
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/41893: Should devel/boehm-gc be built with thread support
 by default?
Date: Sat, 5 Nov 2011 17:14:57 -0400


 On Sat,  5 Nov 2011 13:55:02 +0000 (UTC)
 David Holland <dholland-pbugs@netbsd.org> wrote:

 
 >   >  It would also be possible to tell ECL to build and embed its own
 >   >  boehm-gc, which is part of the tree, although this duplicates things a
 >   >  little in RAM.  Is it a valid concern?
 >  
 >  The primary concern is that if a vulnerability appears in the package,
 >  someone has to remember where the extra private copies are and mark
 >  those packages vulnerable too... and then someone has to apply the
 >  fixes. This is usually a bad idea, so we like to not use included
 >  private copies of things.

 
 True, I agree that dynamic linking has an advantage here.

 
 >   >  Yet we since force pthread support for everything that uses dlopen(3),
 >   >  so why not also enable the current thread option by default on
 >   >  netbsd-5+ and Linux? (boehm-gc doesn't use DLFCN(3), though).
 >  
 >  Well, the concern I'd have is that thread support may add a lot of
 >  overhead. Threaded garbage collection is painful enough in general
 >  that people go to substantial lengths to avoid it on purpose; I don't
 >  know to what extent this applies to boehm-gc's thread support.
 >  
 >  If there's not a significant problem, then we should enable it by
 >  default, as non-threaded/single-core environments are not the future
 >  or even at this point the present.
 >  
 >  If there is a problem, a separate package is probably the way to go;
 >  however, I'd go so far as to say the default package should be
 >  threaded and the other package should be "boehm-gc-unthreaded".
 >  
 >  As to whether there's a problem... that's the sort of thing we have
 >  package maintainers to know about. I have no idea. If you'd be willing
 >  to look into it, which hopefully by now should require only digging
 >  around and not actually benchmarking things, we can probably move
 >  forward on this...

 
 I've not checked the code, but according to the tech docs:

 
 According to http://www.hpl.hp.com/personal/Hans_Boehm/gc/gcdescr.html,
 it uses a stop-world mark-and-sweep algorithm, with some support or
 incremental and generational modes for some operating systems (I'm not
 sure if that's true for BSD yet, but no BSD is listed in the OS
 enumeration.

 
 It also supports an experimental parallel-mark, but by default,
 according to http://www.hpl.hp.com/personal/Hans_Boehm/gc/scale.html:

 
     It causes the garbage collector to stop all other threads when it
     needs to see a consistent memory state. It causes the collector to
     acquire a lock around essentially all allocation and garbage
     collection activity. Since a single lock is used for all
     allocation-related activity, only one thread can be allocating or
     collecting at one point. This inherently limits performance of
     multi-threaded applications on multiprocessors.

 
 Personally, I currently am using a boehm-gc built with threads support,
 as I use ECL built with threads support as well.  I can't say that it's
 a problem on my slowest system (a 2793.16 MHz cache-crippled Celeron).
 However, boehm-gc is only used by ECL, w3m and Inkscape on my own
 systems.  I'm not sure that a boehm-gc-unthreaded would even be
 necessary, but who knows...

 
 Thanks,
 -- 
 Matt

 
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/41893: Should devel/boehm-gc be built with thread support
 by default?
Date: Sat, 5 Nov 2011 19:27:15 -0400


 On Sat,  5 Nov 2011 21:15:04 +0000 (UTC)
 Matthew Mondor <mm_lists@pulsar-zone.net> wrote:

 
 >  Personally, I currently am using a boehm-gc built with threads support,
 >  as I use ECL built with threads support as well.  I can't say that it's
 >  a problem on my slowest system (a 2793.16 MHz cache-crippled Celeron).
 >  However, boehm-gc is only used by ECL, w3m and Inkscape on my own
 >  systems.  I'm not sure that a boehm-gc-unthreaded would even be
 >  necessary, but who knows...

 
 I connected to an Ubuntu system and can see that their boehm-gc deb
 package includes thread support in libgc.so too.  However, boehm-gc is
 certainly more tested on Linux (and tweaked for it) than on BSD...
 -- 
 Matt

 
>Unformatted:

 

Home	PR Database Search