NetBSD Problem Report #28761

From kre@jade.coe.psu.ac.th  Thu Dec 23 11:50:12 2004
Return-Path: <kre@jade.coe.psu.ac.th>
Received: from jade.coe.psu.ac.th (unknown [192.150.250.67])
	by narn.netbsd.org (Postfix) with ESMTP id 3CA1D63B400
	for <gnats-bugs@gnats.netbsd.org>; Thu, 23 Dec 2004 11:50:11 +0000 (UTC)
Message-Id: <200412231150.iBNBo9U7015458@jade.coe.psu.ac.th>
Date: Thu, 23 Dec 2004 18:50:09 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@netbsd.org
Subject: make clean CLEANDEPENDS=YES cleans too much...
X-Send-Pr-Version: 3.95

>Number:         28761
>Category:       pkg
>Synopsis:       make clean CLEANDEPENDS=YES cleans too much...
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    pkg-manager
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Thu Dec 23 11:51:00 +0000 2004
>Closed-Date:    
>Last-Modified:  Tue Mar 26 14:45:03 +0000 2013
>Originator:     Robert Elz
>Release:        NetBSD 1.6X   (pkgsrc -current as of date/time of this message)
>Organization:
	Prince of Songkla University
>Environment:
	Irrelevant information folloes...
System: NetBSD jade.coe.psu.ac.th 1.6X NetBSD 1.6X (JADE) #17: Wed Sep 24 20:25:35 ICT 2003 kre@jade.coe.psu.ac.th:/usr/src/real-sys/arch/i386/compile/JADE i386
Architecture: i386
Machine: i386
>Description:
	Doing:
		make clean CLEANDEPENDS=YES
	after building/installing a package ends up running "make clean"
	(make clean-depends really) in more packages than it probably
	should.   (This is a change request, not a bug report..., so
	what it "probably should" is clearly open for discussion)

	As currently implemented, it cleans every dependency of the
	package (in pkgsrc), regardless of whether or not they were
	actually used while building the package.

	It would probably be more useful (faster, and safer) to only
	clean those packages that actually needed to be built in order
	to build the package that is being cleaned.

>How-To-Repeat:
	Install some package that lots of other things depend on (anything
	that something depends upon is enough, but something important and
	low leven is easier to find - like one of the perl versions, or
	libpng, or something like that).   Install it completely.

	Then go to some other package that uses the one that was just
	installed as a dependency (any of the p5-* packages should do
	is perl was what was selected above).   Do
		make clean CLEANDEPENDS=yes
	(not even necessary to make it to start with) and notice that
	it cleans perl (as well as perhaps lots of other stuff).

	This is most annoying if you happen to have the base package
	in a constructed state while looking for some bug, or doing
	some enhancement to the package, when some other package
	just decides to clean it.

	Cleaning all that was actually dirtied while building a package
	should be enough.

>Fix:
	Lots more code in bas.pkg.mk I assume ... (what fun!)
	Most likely need to keep a record (in WRKDIR I guess)
	of which packages were actually built, that is, which were
	not found to already be installed, and then clean just those.

>Release-Note:

>Audit-Trail:
From: Ben Collver <ben@netbsd.org>
To: kre@munnari.OZ.AU
Cc: pkg-bug-handler@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: pkg/28761: make clean CLEANDEPENDS=YES cleans too much...
Date: Wed, 29 Dec 2004 09:42:26 -0800

 Good day Robert,

 I read your change-request and personally disagree with the change.  I
 think that clean-depends should mean exactly that: clean the
 dependencies.  I could imagine pkgsrc providing clean-build-depends
 and clean-buildlink-depends targets, but it sounds like you are asking
 for something more: only clean dependencies if they were actually built
 at the same time as the package that depended on them.  This seems to me
 more likely to cause than cure confusion.

 If you are debugging a base package, it might help to make a copy.  For
 example, make a copy of the perl directory named perl-kre, and work on
 that.  Is this workaround sufficient?

 Thank you,

 Ben

 On Thu, Dec 23, 2004 at 11:51:00AM +0000, kre@munnari.OZ.AU wrote:
 > >Number:         28761
 > >Category:       pkg
 > >Synopsis:       make clean CLEANDEPENDS=YES cleans too much...
 > >Confidential:   no
 > >Severity:       non-critical
 > >Priority:       low
 > >Responsible:    pkg-manager
 > >State:          open
 > >Class:          change-request
 > >Submitter-Id:   net
 > >Arrival-Date:   Thu Dec 23 11:51:00 +0000 2004
 > >Originator:     Robert Elz
 > >Release:        NetBSD 1.6X   (pkgsrc -current as of date/time of this message)
 > >Organization:
 > 	Prince of Songkla University
 > >Environment:
 > 	Irrelevant information folloes...
 > System: NetBSD jade.coe.psu.ac.th 1.6X NetBSD 1.6X (JADE) #17: Wed Sep 24 20:25:35 ICT 2003 kre@jade.coe.psu.ac.th:/usr/src/real-sys/arch/i386/compile/JADE i386
 > Architecture: i386
 > Machine: i386
 > >Description:
 > 	Doing:
 > 		make clean CLEANDEPENDS=YES
 > 	after building/installing a package ends up running "make clean"
 > 	(make clean-depends really) in more packages than it probably
 > 	should.   (This is a change request, not a bug report..., so
 > 	what it "probably should" is clearly open for discussion)
 > 
 > 	As currently implemented, it cleans every dependency of the
 > 	package (in pkgsrc), regardless of whether or not they were
 > 	actually used while building the package.
 > 
 > 	It would probably be more useful (faster, and safer) to only
 > 	clean those packages that actually needed to be built in order
 > 	to build the package that is being cleaned.
 > 
 > >How-To-Repeat:
 > 	Install some package that lots of other things depend on (anything
 > 	that something depends upon is enough, but something important and
 > 	low leven is easier to find - like one of the perl versions, or
 > 	libpng, or something like that).   Install it completely.
 > 
 > 	Then go to some other package that uses the one that was just
 > 	installed as a dependency (any of the p5-* packages should do
 > 	is perl was what was selected above).   Do
 > 		make clean CLEANDEPENDS=yes
 > 	(not even necessary to make it to start with) and notice that
 > 	it cleans perl (as well as perhaps lots of other stuff).
 > 
 > 	This is most annoying if you happen to have the base package
 > 	in a constructed state while looking for some bug, or doing
 > 	some enhancement to the package, when some other package
 > 	just decides to clean it.
 > 
 > 	Cleaning all that was actually dirtied while building a package
 > 	should be enough.
 > 
 > >Fix:
 > 	Lots more code in bas.pkg.mk I assume ... (what fun!)
 > 	Most likely need to keep a record (in WRKDIR I guess)
 > 	of which packages were actually built, that is, which were
 > 	not found to already be installed, and then clean just those.
 > 

State-Changed-From-To: open->feedback
State-Changed-By: wiz@netbsd.org
State-Changed-When: Tue, 04 Jan 2005 14:46:45 +0000
State-Changed-Why:
Feedback requested (cc gnats-bugs@ please).


From: Robert Elz <kre@munnari.OZ.AU>
To: Ben Collver <ben@netbsd.org>
Cc: pkg-bug-handler@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: pkg/28761: make clean CLEANDEPENDS=YES cleans too much... 
Date: Thu, 06 Jan 2005 02:00:20 +0700

     Date:        Wed, 29 Dec 2004 09:42:26 -0800
     From:        Ben Collver <ben@netbsd.org>
     Message-ID:  <20041229174226.GD10617@sanctum.takilma.net>

 Apologies for the tardy reply.

   | I read your change-request and personally disagree with the change.

 As I said initially,...  
 	what it "probably should" is clearly open for discussion

 Obviously there's no necessarily correct behaviour here, it is all
 a matter of what is desirable (and possible).

   | I think that clean-depends should mean exactly that: clean the
   | dependencies.

 Of course, the question is which dependencies should be cleaned.
 The target isn't clean-all-depends after all...  (which of course
 could be added, if my suggestion in this PR were to be adopted).

   | I could imagine pkgsrc providing clean-build-depends
   | and clean-buildlink-depends targets,

 I guess, but ...

   | but it sounds like you are asking for something more:

 Yes.

   | only clean dependencies if they were actually built
   | at the same time as the package that depended on them.

 Only if built because they were required by the package that depended upon
 them (happening to be built at the same time for some other reason wouldn't
 be relevant).

 That is, if the package was entered during building it should be entered
 during cleaning, and not otherwise (for the default clean-depends anyway).

   | This seems to me more likely to cause than cure confusion.

 I'm not sure why that would be.

   | If you are debugging a base package, it might help to make a copy.

 Yes, with hindsight, that kind of thing can help, but this usually strikes
 when least expected - that is, when one is simply not expecting that anything
 else will touch the package (it certainly isn't going to be built again,
 it is already installed, and the "been there done that" dot files in the
 work directory all still exist).

   | For example, make a copy of the perl directory named perl-kre, and work on
   | that.  Is this workaround sufficient?

 No, because I believe that the current behaviour is simply wrong.

 That is, if I dirtied the pond, I should clean it up, if someone else
 did, it is their responsibility, not mine.

 This workaround also doesn't do a thing to speed up builds - the
 clean-depends step can take a while attempting to clean stuff that
 has already been cleaned a hundred times before, and there can be a
 lot of it in some packages (do you have any idea just how many times perl
 got cleaned during a recent build of about 3000 packages I did?   It
 was only built once - or twice, it got upgraded in the middle somewhere).

 kre

State-Changed-From-To: feedback->open
State-Changed-By: wiz@netbsd.org
State-Changed-When: Mon, 14 Mar 2005 14:03:30 +0000
State-Changed-Why:
Feedback supplied.


From: Benny Siegert <bsiegert@gmail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/28761
Date: Tue, 26 Mar 2013 14:31:54 +0100

 Ping.

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/28761
Date: Tue, 26 Mar 2013 21:42:48 +0700

     Date:        Tue, 26 Mar 2013 13:35:03 +0000 (UTC)
     From:        Benny Siegert <bsiegert@gmail.com>
     Message-ID:  <20130326133503.3D5E863E6F8@www.NetBSD.org>

   |  Ping.

 Pong.

 Now the frivolity is out of the way, I still believe that clean-depends
 should really act the way I requested.

 However, I care much less now - when the PR was sent (> 8 years ago)
 I don't think I was using obj directories for pkgsrc, just work (or work.arch
 if that existed yet) inside pkgsrc, and "make clean" was actually
 important.   Now with obj directories, I don't bother with make clean
 any more, "rm -fr /usr/obj/pkg/*" (or whatever subset of that I feel
 inclined to clean) is infinitely faster, and barely any harder to type.

 I also removed all instances of "make clean" from the version of pkg_comp
 I use (I think I made them optional, under an option I do not enable), and
 I also changed the do-bin-install-from-source target in
 pkgsrc/mk/install/bin-install.mk to remove the part that does
 	${RECURSIVE_MAKE} ${MAKEFLAGS} clean

 With those changes, I never see a "make clean" any more, so I no longer
 really care if it is sane or not, so if you really want to close
 this ancient PR, then I guess you can go ahead and do that.

 Incidentally, that "${RECURSIVE_MAKE} ${MAKEFLAGS} clean" was the source
 of the "make clean twice" that I had reported (or mentioned) in  some
 context or other sometime in the past - pkg_comp does "make" then (did anyway)
 "make clean", I have DEPENDS_TARGET=bin-install in (pkg_comp's) mk.conf
 so the make was doing "make bin-install", and that was causing a complete
 clean for every package installed 999% of which were the "bin" variety
 of the "bin-install-from-source" when they're dependencies - that is, I
 would have all the dependencies built as bin-pkgs already, then pkgsrc
 would (quickly) install the dependencies, and (absurdly slowly) do a
 "make clean" (recursively, that is, effectively CLEANDEPENDS=yes) on
 each one of them...   Then once the build had finished, pkg_comp did its
 make clean, including the dependencies, which cleaned all of the binpkgs
 (and their dependencies) all over again, and all for nothing!

 With the change I requested, that would have been relatively fast, as
 nothing was built, nothing would have been cleaned, and I might never have
 noticed the "install then clean" behaviour of pkgsrc.

 kre

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.