NetBSD Problem Report #14825

Received: (qmail 8453 invoked from network); 3 Dec 2001 14:53:53 -0000
Message-Id: <200112031453.fB3ErOx29786@kaakeli.acr.fi>
Date: Mon, 3 Dec 2001 16:53:24 +0200 (EET)
From: kivinen@ssh.fi
Reply-To: kivinen@ssh.fi
To: gnats-bugs@gnats.netbsd.org
Subject: /etc/{daily,weekly,montly} should be unified
X-Send-Pr-Version: 3.95

>Number:         14825
>Category:       misc
>Synopsis:       setting of rebuild_locatedb=YES in daily.conf does not do anything
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Mon Dec 03 14:54:00 +0000 2001
>Closed-Date:    
>Last-Modified:  Mon Feb 04 09:46:50 +0000 2002
>Originator:     Tero Kivinen
>Release:        NetBSD 1.5Y
>Organization:
SSH Communications Security
>Environment:
System: NetBSD kaakeli.acr.fi 1.5Y NetBSD 1.5Y (KAAKELI) #47: Wed Oct 31 02:45:43 EET 2001 kivinen@kaakeli.acr.fi:/usr/src/sys/arch/i386/compile/KAAKELI i386
Architecture: i386
Machine: i386
>Description:

	The /etc/{daily,weekly,montly} scripts should be unified so
	that setting for example rebuild_locatedb=YES in daily.conf
	causes the locatedb to be updated daily. I.e all them should
	run the exactly same script, but the configuration parameters
	simply would be different.

>How-To-Repeat:

	Currently rebuild_locatedb=YES is only understood by the
	/etc/weekly. Seting it in /etc/daily.conf or /etc/monthly.conf
	does not affect anything.

>Fix:

	Combine all /etc/{daily,weekly,montly} scripts to one having this
	kind of header (not tested):
----------------------------------------------------------------------
#!/bin/sh -
#
#	$NetBSD: oneinawhile,v 1.8 2000/01/10 17:03:49 ad Exp $
#	from: @(#)oneinawhile	8.1 (Berkeley) 6/9/93
#

export PATH=/bin:/usr/bin:/sbin:/usr/sbin
umask 077

type=`echo $0 | sed 's/.*\///g;'`

if [ -s /etc/$type.conf ]; then
	. /etc/$type.conf
fi

if [ -f /etc/rc.subr ]; then
        . /etc/rc.subr
else
        echo "Can't read /etc/rc.subr; aborting."
        exit 1;
fi

if checkyesno run_only_when_connected_to_ac; then
        if [ `apm -a` == 0 ]; then
                exit 0
        fi
fi

host=`hostname`
date=`date`

echo "To: ${MAILTO:-root}"
echo "Subject: $host $type output for $date"
echo ""

if [ -z "$MAILTO" -o "$USER" != "root" ]; then
	MAILTO=root
fi

----------------------------------------------------------------------

And then all bodies of the /etc/daily, /etc/weekly, and /etc/montly
combined together (removing their headers and footers, and moving some
echo "" and rm -f $TMP $TMP2 etc inside the if checkyesno).

Then the script should have footer like this:
----------------------------------------------------------------------
if [ -f /etc/$type.local ]; then
	echo ""
	echo "Running /etc/$type.local:"
	. /etc/$type.local
fi
----------------------------------------------------------------------

Then remove /etc/{daily,weekly,montly} and create symlinks from them
to /etc/oneinawhile
>Release-Note:
>Audit-Trail:

From: woods@weird.com (Greg A. Woods)
To: kivinen@ssh.fi
Cc: gnats-bugs@gnats.netbsd.org (NetBSD GNATS submissions and followups),
	netbsd-bugs@NetBSD.ORG (NetBSD Bugs and PR posting List)
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon,  3 Dec 2001 12:08:05 -0500 (EST)

 [ On Monday, December 3, 2001 at 16:53:24 (+0200), kivinen@ssh.fi wrote: ]
 > Subject: misc/14825: /etc/{daily,weekly,montly} should be unified
 >
 > 	The /etc/{daily,weekly,montly} scripts should be unified so
 > 	that setting for example rebuild_locatedb=YES in daily.conf
 > 	causes the locatedb to be updated daily. I.e all them should
 > 	run the exactly same script, but the configuration parameters
 > 	simply would be different.

 I do see some merit in that flexibility, but in general I like the
 FreeBSD "periodic(8)" stuff much better.


 In order to implement the kind of flexibility you describe in FreeBSD it
 would simply be a matter of unifying the periodic/* programs into one
 master holding directory and "installing" them into the "daily",
 "weekly", and "monthly" (and whatever periods are defined) with hard
 links (just like is done with /etc/init.d in some other kinds of
 systems).

 The FreeBSD periodic(8) scheme could be modified to use rcorder too,
 instead of the filename based ordering they now use....

 -- 
 							Greg A. Woods

 +1 416 218-0098      VE3TCP      <gwoods@acm.org>     <woods@robohack.ca>
 Planix, Inc. <woods@planix.com>;   Secrets of the Weird <woods@weird.com>

From: Luke Mewburn <lukem@wasabisystems.com>
To: NetBSD GNATS submissions and followups <gnats-bugs@gnats.netbsd.org>,
	NetBSD Bugs and PR posting List <netbsd-bugs@NetBSD.ORG>
Cc: kivinen@ssh.fi, woods@weird.com
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Wed, 5 Dec 2001 00:45:00 +1100

 On Mon, Dec 03, 2001 at 12:08:05PM -0500, Greg A. Woods wrote:
 > In order to implement the kind of flexibility you describe in FreeBSD it
 > would simply be a matter of unifying the periodic/* programs into one
 > master holding directory and "installing" them into the "daily",
 > "weekly", and "monthly" (and whatever periods are defined) with hard
 > links (just like is done with /etc/init.d in some other kinds of
 > systems).

 I'd prefer the one directory and not use (hard) links. We avoided this
 in rc.d, and there's no reason to inflict that upon periodic.d either.
 You could use different rcorder(8) keywords.  Or you could just enable
 stuff in /etc/{daily,weekly,monthly}.conf as necessary.

 Looking at this stuff is a low priority project that I'm partially
 involved in with another developer (who may choose to reveal himself
 if he so desires).


 > The FreeBSD periodic(8) scheme could be modified to use rcorder too,
 > instead of the filename based ordering they now use....

 FreeBSD feature requests are a bit outside the scope of netbsd-bugs ;-)

 Luke.

From: woods@weird.com (Greg A. Woods)
To: Luke Mewburn <lukem@wasabisystems.com>
Cc: NetBSD GNATS submissions and followups <gnats-bugs@gnats.netbsd.org>,
	NetBSD Bugs and PR posting List <netbsd-bugs@NetBSD.ORG>,
	kivinen@ssh.fi
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Wed,  5 Dec 2001 02:33:24 -0500 (EST)

 [ On Wednesday, December 5, 2001 at 00:45:00 (+1100), Luke Mewburn wrote: ]
 > Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
 >
 > I'd prefer the one directory and not use (hard) links. We avoided this
 > in rc.d, and there's no reason to inflict that upon periodic.d either.

 I do very much understand and appreciate the desire to avoid use of hard
 links to store configuration information, though I would like to point
 out that there are many simple ways this technique can be managed in
 such a way that the issues raised by it can be mitigated, if not even
 eliminated.

 > You could use different rcorder(8) keywords.

 Using dependency ordering will not currently work to select which
 arbitrary period might choose to execute a script, though of course
 extra magic could be added to enable a filter to assist in the
 selection.  However adding magic upon magic is only asking for trouble.

 Something like this needs to be concretely visible and terribly obvious
 even on first glance -- it's not a "difficult" problem to conceptualise
 in the way that startup ordering can be.

 >  Or you could just enable
 > stuff in /etc/{daily,weekly,monthly}.conf as necessary.

 Yes, this is probably still the best way....

 > Looking at this stuff is a low priority project that I'm partially
 > involved in with another developer (who may choose to reveal himself
 > if he so desires).

 In that case I'll probably hold off on importing it into my own tree....

 (though the more I think about your suggestion of just sticking with
 rc.conf-style configuration, the more I begin to think that a
 from-scratch implementation which is an only slightly enhanced version
 of our current /etc/rc will be all that's necessary (instead of using
 /etc/rc.d and /etc/rc.conf just select the directory and config file
 based on the period name passed on the command-line).  Perhaps the
 start/stop/etc. parameter handling could be disabled too....

 > > The FreeBSD periodic(8) scheme could be modified to use rcorder too,
 > > instead of the filename based ordering they now use....
 > 
 > FreeBSD feature requests are a bit outside the scope of netbsd-bugs ;-)

 I think you've misunderstood my intent -- I referred only to the
 possibility of using rcorder in NetBSD should FreeBSD's periodic(8)
 scheme be imported into NetBSD.

 -- 
 								Greg A. Woods

 +1 416 218-0098;  <gwoods@acm.org>;  <g.a.woods@ieee.org>;  <woods@robohack.ca>
 Planix, Inc. <woods@planix.com>; VE3TCP; Secrets of the Weird <woods@weird.com>

From: John Hawkinson <jhawk@MIT.EDU> (by way of Erik E. Fair)
To: NetBSD GNATS Problem Report Tracking System <gnats-bugs@gnats.netbsd.org>
Cc:  
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon, 4 Feb 2002 01:44:40 -0800

 | From: "Jeremy C. Reed" <reed@reedmedia.net>

 | > I think they really need to be combined, actually, because

 | John, do you have any specific ideas for this?

 Beyond the above statement, not particularly.
 I'm not sure what you're asking.


 To address Andrew's question, while it might be bad to let long
 discussions run afoul of the GNATS bug, it's worse to have them
 on netbsd-bugs where they won't be found, as opposed to
 having them on a tech- list were folks are likely to search for them.

 | ... But now looking at the daily, weekly, monthly, and security scripts
 | (not current), I only see two uses of find. So maybe it is not a big deal
 | anyways.

 Certainly it is. Those finds are the bulk of the work done by those scripts.

 --jhawk

From: "Jeremy C. Reed" <reed@reedmedia.net> (by way of Erik E. Fair)
To: NetBSD GNATS Problem Report Tracking System <gnats-bugs@gnats.netbsd.org>
Cc:  
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon, 4 Feb 2002 01:44:39 -0800

 On Wed, 5 Dec 2001, John Hawkinson wrote:

 >  I think they really need to be combined, actually, because
 >  some of them run finds on the whole disk, and those should be combined
 >  into one find, not multiple ones.

 John, do you have any specific ideas for this?

 On Wed, 5 Dec 2001, Andrew Brown wrote:

 >  i like that idea also, except that such a solution would probably
 >  require more tools (eg, grep and sort) to support a -0 flag.  some of
 >  the finds are searching for wildly different things as well.  otoh, if
 >  find could be taught, like sed, to be able to spew to different (and
 >  perhaps multiple) output files, then perhaps grep and sort could be
 >  left out of it.

 Or maybe just have find output every file, then use awk to sort it out.

 Recently, I saw postings from A. Crooks and others about using awk to
 parse ls(1) output to generate octal modes. awk could check the find
 output for:

 - core files
 - set uid/gid devices

 ... But now looking at the daily, weekly, monthly, and security scripts
 (not current), I only see two uses of find. So maybe it is not a big deal
 anyways.

     Jeremy C. Reed
     http://www.reedmedia.net/

From: matthew green <mrg@eterna.com.au> (by way of Erik E. Fair)
To: NetBSD GNATS Problem Report Tracking System <gnats-bugs@gnats.netbsd.org>
Cc:  
Subject: re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon, 4 Feb 2002 01:44:40 -0800

 one thing i certainly wish is that i could set 'locate' to rebuild
 every night...



 .mrg.

From: Andrew Brown <atatat@atatdot.net> (by way of Erik E. Fair)
To: NetBSD GNATS Problem Report Tracking System <gnats-bugs@gnats.netbsd.org>
Cc:  
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon, 4 Feb 2002 01:44:39 -0800

 >| [note that i've removed the gnats lists from this discussion.  there's
 >| no reason for this to end up in the bug database.]
 >
 >dunno about that, perhaps you should have added a tech- list.

 i don't think long rambling discussions belong in the bugs database.
 it's already been acknowledged that (a) there's a "problem" and (b)
 that some people have a fix in the works.

 >| * all the jobs from /etc/{daily,weekly,monthly,security} can be little
 >| scripts unto themselves in a directory called, for example,
 >| /etc/periodic.d
 >
 >I think they really need to be combined, actually, because
 >some of them run finds on the whole disk, and those should be combined
 >into one find, not multiple ones.

 i like that idea also, except that such a solution would probably
 require more tools (eg, grep and sort) to support a -0 flag.  some of
 the finds are searching for wildly different things as well.  otoh, if
 find could be taught, like sed, to be able to spew to different (and
 perhaps multiple) output files, then perhaps grep and sort could be
 left out of it.

 -- 
 |-----< "CODE WARRIOR" >-----|
 codewarrior@daemon.org             * "ah!  i see you have the internet
 twofsonet@graffiti.com (Andrew Brown)                that goes *ping*!"
 andrew@crossbar.com       * "information is power -- share the wealth."

From: John Hawkinson <jhawk@MIT.EDU> (by way of Erik E. Fair)
To: NetBSD GNATS Problem Report Tracking System <gnats-bugs@gnats.netbsd.org>
Cc:  
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon, 4 Feb 2002 01:44:38 -0800

 | [note that i've removed the gnats lists from this discussion.  there's
 | no reason for this to end up in the bug database.]

 dunno about that, perhaps you should have added a tech- list.

 | * all the jobs from /etc/{daily,weekly,monthly,security} can be little
 | scripts unto themselves in a directory called, for example,
 | /etc/periodic.d

 I think they really need to be combined, actually, because
 some of them run finds on the whole disk, and those should be combined
 into one find, not multiple ones.

 --jhawk

From: Andrew Brown <atatat@atatdot.net> (by way of Erik E. Fair)
To: NetBSD GNATS Problem Report Tracking System <gnats-bugs@gnats.netbsd.org>
Cc:  
Subject: Re: misc/14825: /etc/{daily,weekly,montly} should be unified
Date: Mon, 4 Feb 2002 01:44:37 -0800

 [note that i've removed the gnats lists from this discussion.  there's
 no reason for this to end up in the bug database.]

 >Using dependency ordering will not currently work to select which
 >arbitrary period might choose to execute a script, though of course
 >extra magic could be added to enable a filter to assist in the
 >selection.  However adding magic upon magic is only asking for trouble.

 i've started this project a few times, and come to the conclusion that

 * all the jobs from /etc/{daily,weekly,monthly,security} can be little
 scripts unto themselves in a directory called, for example,
 /etc/periodic.d

 * you can have {daily,weekly,monthly,security,periodic}.conf files,
 along with the associated separate ones for backwards looking

 * the dependency ordering that rcorder can provide is more than
 sufficient, therefore hard links or symbolic links or any other
 sysv-like organization mechanism is unnecessary

 * moving a "job" from monthly or daily to weekly or monthly (or any
 combination thereof) is trivial, whereas moving jobs in and out of
 security is only slightly less trivial

 * there are a few other things that i must do that are of more
 immediate interest to me

 -- 
 |-----< "CODE WARRIOR" >-----|
 codewarrior@daemon.org             * "ah!  i see you have the internet
 twofsonet@graffiti.com (Andrew Brown)                that goes *ping*!"
 andrew@crossbar.com       * "information is power -- share the wealth."
>Unformatted:

Home	PR Database Search