NetBSD Problem Report #35340
From www@NetBSD.org Sat Dec 30 04:34:02 2006
Return-Path: <www@NetBSD.org>
Received: by narn.NetBSD.org (Postfix, from userid 31301)
id BCDA963B88A; Sat, 30 Dec 2006 04:34:01 +0000 (UTC)
Message-Id: <20061230043401.BCDA963B88A@narn.NetBSD.org>
Date: Sat, 30 Dec 2006 04:34:01 +0000 (UTC)
From: truist-netbsd-sendpr@truist.com
Reply-To: truist-netbsd-sendpr@truist.com
To: gnats-bugs@NetBSD.org
Subject: pkgsrc OWN_DIRS bug where empty directories are pruned too earnestly
X-Send-Pr-Version: www-1.0
>Number: 35340
>Category: pkg
>Synopsis: pkgsrc OWN_DIRS bug where empty directories are pruned too earnestly
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: dholland
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Dec 30 04:35:00 +0000 2006
>Closed-Date:
>Last-Modified: Sat May 14 21:25:00 +0000 2022
>Originator: Nathan Arthur
>Release: 3.0_STABLE on i386
>Organization:
truist.com
>Environment:
NetBSD home.truist.com 3.0_STABLE NetBSD 3.0_STABLE (TRUIST) #1: Fri Aug 18 08:01:55 EDT 2006 root@home.truist.com:/usr/obj/sys/arch/i386/compile/TRUIST i386
>Description:
I use pkgsrc's bulk build scripts to do bulk rebuilds of all the packages I use on my NetBSD 3.0 machine. I don't build all packages - just the subset that I need. I do the builds in a chroot sandbox (via mk/bulk/mksandbox) and I've discovered a problem where the build of the qmail-qfilter package is deleting the sandbox's /var/tmp directory (which then causes later package builds the fail). Per the email thread below, it seems to be deleted during pkg_delete because of something to do with OWN_DIRS_PERMS. I've dug as far as I can into the scripts to try to understand what's going wrong, and I don't know enough to go further. Amitai (below) asked that I send-pr.
This only seems to happen if /var/tmp is empty at the time that qmail-qfilter is pkg_delete'd, which explains why this doesn't show up for regular package builds. It also doesn't seem to be affecting the regular netbsd bulk rebuilds, presumably because there is something in /var/tmp at the time that this happens.
I can happily give more detail about the process here, but I think all the salient details are above and below. For now, I've worked around this by just putting a file in /var/tmp, but it seems like an issue that should probably be fixed.
-------- Original Message --------
Subject: Re: You broke my bulk build!
Date: Sun, 22 Oct 2006 19:29:40 -0400 (EDT)
From: Amitai Schlair <schmonz@schmonz.com>
To: Nathan Arthur <truist@truist.com>
References: <453BED63.4090205@truist.com>
Nathan Arthur wrote:
> 1) Create the sandbox (which creates an appropriate /var/tmp)
> 2) chroot into it
> 3) Manually build qmail-qfilter, into a package
> 4) Install the qmail-qfilter package
> 5) At this point, /var/tmp is still there, and has an empty qmail-qfilter directory in it.
> 6) pkg_delete qmail-qfilter
> 7) /var/tmp is gone!
Wacky. At a glance, that shouldn't happen. The relevant line of
pkgsrc/mail/qmail-qfilter/Makefile is:
OWN_DIRS_PERMS+= ${QMAIL_QFILTER_TMPDIR} qmaild ${ROOT_GROUP} 700
And the relevant lines in pkgsrc/mk/defaults/mk.conf are:
QMAIL_QFILTER_TMPDIR?= ${VARBASE}/tmp/qmail-qfilter
# The temporary directory for qmail-qfilter.
# Possible: a directory writable only by qmail-qfilter's UID
# Default: ${VARBASE}/tmp/qmail-qfilter
OWN_DIRS is handled somewhere in pkgsrc/mk/install or pkgsrc/mk/pkginstall (I forget which), which are standard shell scripts that get executed by pkg_add and pkg_delete. What's supposed to happen is, package defines OWN_DIRS, it gets created at install time, if it's empty at deinstall time it's removed (there's a related variable that instead instructs the user to remove it). So it sounds like this logic is pruning empty directories a little too earnestly up the filesystem.
>How-To-Repeat:
1) Create the sandbox (which creates an appropriate /var/tmp)
2) chroot into it
3) Manually build qmail-qfilter, into a package
4) Install the qmail-qfilter package
5) At this point, /var/tmp is still there, and has an empty qmail-qfilter directory in it.
6) pkg_delete qmail-qfilter
7) /var/tmp is gone!
>Fix:
Unknown
>Release-Note:
>Audit-Trail:
From: Nathan Arthur <truist@rainskit.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/35340
Date: Wed, 16 Jul 2008 02:15:53 -0400
I can confirm that this problem still exists, as of today, using these
same steps.
Thanks,
Nathan
From: "OBATA Akio" <obache@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/35340
Date: Wed, 16 Jul 2008 15:56:36 +0900
Can you check tail of /var/db/pkg/qmail-qfilter-2.1nb1/+DIRS,
lines start with "# DIR:"?
State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 18 Jan 2009 09:25:34 +0000
State-Changed-Why:
A question was asked.
State-Changed-From-To: feedback->open
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 18 Jan 2009 09:27:12 +0000
State-Changed-Why:
Feedback mail bounced.
Responsible-Changed-From-To: pkg-manager->dholland
Responsible-Changed-By: dholland@NetBSD.org
Responsible-Changed-When: Sun, 09 Aug 2015 22:55:56 +0000
Responsible-Changed-Why:
I can replicate this with a simple test package.
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/35340 (pkgsrc OWN_DIRS bug where empty directories are
pruned too earnestly)
Date: Sun, 9 Aug 2015 23:14:57 +0000
On Sun, Aug 09, 2015 at 10:55:56PM +0000, dholland@NetBSD.org wrote:
> I can replicate this with a simple test package.
The problem is not that the wrong things are listed at the end of
+DIRS; the problem is that +DIRS uses rmdir -p to remove dirs, so that
any empty ancestor directories are pruned regardless of whether pkgsrc
owns them or not.
This could be changed; however, because +DIRS also uses mkdir -p, this
may cause intermediate dirs created by pkgsrc to get left around.
I note that +DIRS sets MKDIR to mkdir -p, and then uses ${MKDIR} -p in
places, suggesting that this may not have been originally intended.
It seems to me that the right fix is to not use either rmdir -p or
mkdir -p, and to fix packages that don't explicitly create ancestor
directories. (That is, if you have
OWN_DIRS_PERMS+= /var/thingy/boojum root wheel 755
OWN_DIRS_PERMS+= /var/thingy/snark root wheel 755
without an explicit entry for /var/thingy, nothing will create
/var/thingy and installation of the binary package will fail.)
This is probably a good thing, at least for OWN_DIRS, as the
permissions on directories created by mkdir -p depend on the umask and
maybe other things and aren't necessarily as repeatable as we'd like.
However, this introduces two other issues: (a) where do we stop (do we
need OWN_DIRS for /var? surely not; but what about e.g. /var/db, which
probably doesn't exist on say Linux?) and (b) does mucking with the
+DIRS script affect other directory creation that we aren't prepared
to deal with or where using mkdir -p is necessary and appropriate? I
don't know...
btw, the replication example is to just add e.g.
OWN_DIRS+= /var/thingy/boojum root wheel 755
to something simple and install/deinstall. Currently creating
/var/thingy beforehand is not necessary, because it will be created by
mkdir -p.
--
David A. Holland
dholland@netbsd.org
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/35340 (pkgsrc OWN_DIRS bug where empty directories are
pruned too earnestly)
Date: Sat, 14 May 2022 21:22:31 +0000
On Sun, Aug 09, 2015 at 11:15:00PM +0000, David Holland wrote:
> On Sun, Aug 09, 2015 at 10:55:56PM +0000, dholland@NetBSD.org wrote:
> > I can replicate this with a simple test package.
>
> The problem is not that the wrong things are listed at the end of
> +DIRS; the problem is that +DIRS uses rmdir -p to remove dirs, so that
> any empty ancestor directories are pruned regardless of whether pkgsrc
> owns them or not.
This still happens, btw.
(Though the package in the original report seems to no longer be
affected, as AFAICT it no longer uses OWN_DIRS.)
--
David A. Holland
dholland@netbsd.org
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.