NetBSD Problem Report #39454
From he@smistad.uninett.no Wed Sep 3 14:29:55 2008
Return-Path: <he@smistad.uninett.no>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by narn.NetBSD.org (Postfix) with ESMTP id D92FB63BC80
for <gnats-bugs@gnats.NetBSD.org>; Wed, 3 Sep 2008 14:29:54 +0000 (UTC)
Message-Id: <20080903142953.4FBBB21DC44@smistad.uninett.no>
Date: Wed, 3 Sep 2008 16:29:53 +0200 (CEST)
From: he@NetBSD.org
Reply-To: he@NetBSD.org
To: gnats-bugs@gnats.NetBSD.org
Subject: UPDATE build of amd64 bootcd fails to pick up new kernel
X-Send-Pr-Version: 3.95
>Number: 39454
>Category: misc
>Synopsis: UPDATE build of amd64 bootcd fails to pick up new kernel
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: misc-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Sep 03 14:30:00 +0000 2008
>Last-Modified: Wed Sep 03 19:50:02 +0000 2008
>Originator: Havard Eidnes
>Release: NetBSD 4.99.72
>Organization:
I try...
>Environment:
System: Target: amd64 4.99.72 -current as of yesterday
Architecture: amd64
Machine: amd64
>Description:
I'm trying to fix bugs preventing the installation of NetBSD
on a HP ProLiant BL460c G1 system, using the amd64 port.
This system typically wants an ISO image to boot from.
I'm trying to use the in-tree methods to replicate the parts
of the build which is creating the boot.iso image, but for
some reason that I can't seem to fathom, I appear to be unable
to make the build work as intended.
I've found distrib/amd64/cdrom/bootcd/, which appears to
describe the build procedure used. However, doing an UPDATE
build using "nbmake-amd64" in distrib/amd64/, the build
creates the new ramdisk and the new netbsd-INSTALL.gz file,
but the build in cdroms/bootcd/ etc. fail to pick up the new
kernel and do not appear to do any work. Also, doing
"nbmake-amd64 release" does not install the new (or old) iso
images. Here's excerpts of the build log:
quattro: {108} /u/build/HEAD/tools/amd64/bin/nbmake-amd64 dependall
dependall ===> ramdisks
dependall ===> ramdisks/ramdisk
`cat.ro' is up to date.
`chmod.ro' is up to date.
`cp.ro' is up to date.
`dd.ro' is up to date.
`df.ro' is up to date.
`ed.ro' is up to date.
`ln.ro' is up to date.
...
depend ===> ramdisks/ramdisk/mbr
`fdisk.ro' is up to date.
`fsck.ro' is up to date.
`fsck_ffs.ro' is up to date.
...
`rtsol.ro' is up to date.
link ramdisk/ramdiskbin
strip ramdiskbin
build ramdisk/work
missing: ./bin (created)
missing: ./dev (created)
...
missing: ./var/spool/lock (created)
create ramdisk/ramdisk.fs
Calculated size of `ramdisk.fs.tmp': 5120000 bytes, 60 inodes
Extent size set to 4096
ramdisk.fs.tmp: 4.9MB (10000 sectors) block size 4096, fragment size 512
using 1 cylinder groups of 4.88MB, 1250 blks, 64 inodes.
super-block backups (for fsck -b #) at:
32,
Populating `ramdisk.fs.tmp'
Image `ramdisk.fs.tmp' complete
dependall ===> instkernel
create instkernel/netbsd-INSTALL (from: /u/build/HEAD/obj/amd64/sys/arch/amd64/compile/INSTALL/netbsd /u/build/HEAD/obj/amd64/distrib/amd64/ramdisks/ramdisk/ramdisk.fs)
got symbols from netbsd-INSTALL.tmp
mapped netbsd-INSTALL.tmp
copying image /u/build/HEAD/obj/amd64/distrib/amd64/ramdisks/ramdisk/ramdisk.fs into netbsd-INSTALL.tmp
done copying image
exiting
create instkernel/netbsd-INSTALL.gz
...
dependall ===> cdroms
dependall ===> cdroms/bootcd
dependall ===> cdroms/bootcd-com
dependall ===> cdroms/installcd
dependall ===> floppies
dependall ===> floppies/bootfloppy
create bootfloppy/netbsd (from: /u/build/HEAD/obj/amd64/distrib/amd64/instkernel/netbsd-INSTALL.gz)
create bootfloppy/boot1.fs (from: boot /u/build/HEAD/src/distrib/amd64/floppies/common/boot.cfg netbsd)
pax: Removing leading / from absolute path names in the archive
Running instboot: /u/build/HEAD/tools/amd64/bin/nbinstallboot -mamd64 floppy.11754.tar /u/build/HEAD/dest/amd64/usr/mdec/bootxx_ustarfs
Free space in last tar block: boot 268, /u/build/HEAD/src/distrib/amd64/floppies/common/boot.cfg 46, netbsd 307,
Writing 5603328 bytes (5472 KB) on 4 disks, free space 270336 bytes (264 KB)
Creating disk 1 to boot1.fs
Creating disk 2 to boot2.fs
Creating disk 3 to boot3.fs
Creating disk 4 to boot4.fs
Final result:
-rw-r--r-- 1 he wheel 1474560 Sep 3 16:18 boot1.fs
-rw-r--r-- 1 he wheel 1474560 Sep 3 16:18 boot2.fs
-rw-r--r-- 1 he wheel 1474560 Sep 3 16:18 boot3.fs
-rw-r--r-- 1 he wheel 1204224 Sep 3 16:18 boot4.fs
You'll note that it dutifully created the new boot floppies,
but alas, no new bootcd iso image.
Doing "nbmake-amd64 prepare" in cdroms/bootcd/ does indeed
copy in the new netbsd INSTALL kernel to the OBJDIR, but
if I thereafter do an "nbmake-amd64 dependall" in the same
directory, nothing is done (no new iso image is created).
I smell a missing dependency tracking allowing UPDATE builds
to work as intended.
I'm also somewhat surprised that there's no top-level target
to build.sh to create the boot images; the iso-image target
from build.sh bundles up the RELEASEDIR contents, and does not
pick up the new INSTALL kernel I just built...
>How-To-Repeat:
See above.
>Fix:
Sorry, this appears to be above what I'm able to figure out.
>Audit-Trail:
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: misc-bug-people@NetBSD.org, gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new kernel
Date: Wed, 3 Sep 2008 16:39:42 +0200
On Wed, Sep 03, 2008 at 02:30:00PM +0000, he@NetBSD.org wrote:
> >Description:
> I'm trying to fix bugs preventing the installation of NetBSD
> on a HP ProLiant BL460c G1 system, using the amd64 port.
>
> This system typically wants an ISO image to boot from.
>
> I'm trying to use the in-tree methods to replicate the parts
> of the build which is creating the boot.iso image, but for
> some reason that I can't seem to fathom, I appear to be unable
> to make the build work as intended.
>
> I've found distrib/amd64/cdrom/bootcd/, which appears to
> describe the build procedure used. However, doing an UPDATE
> build using "nbmake-amd64" in distrib/amd64/, the build
> creates the new ramdisk and the new netbsd-INSTALL.gz file,
> but the build in cdroms/bootcd/ etc. fail to pick up the new
> kernel and do not appear to do any work. Also, doing
> "nbmake-amd64 release" does not install the new (or old) iso
> images. Here's excerpts of the build log:
>
> quattro: {108} /u/build/HEAD/tools/amd64/bin/nbmake-amd64 dependall
I think the right target for this would be "iso_image" and not "dependall".
This is typically done after dependall have been run though the whole tree.
> [...]
> I'm also somewhat surprised that there's no top-level target
> to build.sh to create the boot images; the iso-image target
> from build.sh bundles up the RELEASEDIR contents, and does not
> pick up the new INSTALL kernel I just built...
'build.sh release' will build a release, including binary sets and INSTALL
kernel with embeded ramdisk.
'build.sh iso-image' will take whatever exists in the sets directory and
build a cd image with it. It won't try to update kernels or binaries.
--
Manuel Bouyer, LIP6, Universite Paris VI. Manuel.Bouyer@lip6.fr
NetBSD: 26 ans d'experience feront toujours la difference
--
From: Havard Eidnes <he@NetBSD.org>
To: gnats-bugs@NetBSD.org, bouyer@antioche.eu.org
Cc: misc-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new
kernel
Date: Wed, 03 Sep 2008 20:48:34 +0200 (CEST)
> > quattro: {108} /u/build/HEAD/tools/amd64/bin/nbmake-amd64 dependal=
l
>
> I think the right target for this would be "iso_image" and not "depe=
ndall".
> This is typically done after dependall have been run though the whol=
e tree.
...
> 'build.sh release' will build a release, including binary sets and I=
NSTALL
> kernel with embeded ramdisk.
> 'build.sh iso-image' will take whatever exists in the sets directory=
and
> build a cd image with it. It won't try to update kernels or binaries=
.=
Is there a good reason the distrib/amd64/cdrom/ make target needs to
be different from the floppy targets in this respect? And why does
not "release" dump the built bits into the appropriate RELEASEDIR
subdirectory? I think the Principle Of Least Astonishment has been
seriously violated here.
Regards,
- H=E5vard
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: Havard Eidnes <he@NetBSD.org>
Cc: gnats-bugs@NetBSD.org, misc-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new kernel
Date: Wed, 3 Sep 2008 20:52:21 +0200
On Wed, Sep 03, 2008 at 08:48:34PM +0200, Havard Eidnes wrote:
> > > quattro: {108} /u/build/HEAD/tools/amd64/bin/nbmake-amd64 dependall
> >
> > I think the right target for this would be "iso_image" and not "dependall".
> > This is typically done after dependall have been run though the whole tree.
> ...
> > 'build.sh release' will build a release, including binary sets and INSTALL
> > kernel with embeded ramdisk.
> > 'build.sh iso-image' will take whatever exists in the sets directory and
> > build a cd image with it. It won't try to update kernels or binaries.
>
> Is there a good reason the distrib/amd64/cdrom/ make target needs to
> be different from the floppy targets in this respect?
Yes, because it can't be done at the same time (it has to be run after
release).
> And why does
> not "release" dump the built bits into the appropriate RELEASEDIR
> subdirectory?
I think it does, but maybe not when run from the distrib/ dir.
--
Manuel Bouyer <bouyer@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: Havard Eidnes <he@NetBSD.org>
Cc: gnats-bugs@NetBSD.org, misc-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new kernel
Date: Wed, 3 Sep 2008 21:17:51 +0200
On Wed, Sep 03, 2008 at 08:52:21PM +0200, Manuel Bouyer wrote:
> On Wed, Sep 03, 2008 at 08:48:34PM +0200, Havard Eidnes wrote:
> > > > quattro: {108} /u/build/HEAD/tools/amd64/bin/nbmake-amd64 dependall
> > >
> > > I think the right target for this would be "iso_image" and not "dependall".
> > > This is typically done after dependall have been run though the whole tree.
> > ...
> > > 'build.sh release' will build a release, including binary sets and INSTALL
> > > kernel with embeded ramdisk.
> > > 'build.sh iso-image' will take whatever exists in the sets directory and
> > > build a cd image with it. It won't try to update kernels or binaries.
> >
> > Is there a good reason the distrib/amd64/cdrom/ make target needs to
> > be different from the floppy targets in this respect?
>
> Yes, because it can't be done at the same time (it has to be run after
> release).
>
> > And why does
> > not "release" dump the built bits into the appropriate RELEASEDIR
> > subdirectory?
>
> I think it does, but maybe not when run from the distrib/ dir.
To make is clear; here is what happens:
- 'make distribution' builds from sources, and update binairies in $DESTDIR,
and builds INSTALL kernels/ramdisks/boot media. This may include bootable
iso images without binary sets.
- 'make release' takes whatever is in $DESTDIR and build the release bits
from it. It won't attempts to rebuild outdated binairies from sources.
- 'make iso_image' takes whatever is in $RELEASEDIR and makes a
(possibly bootable) iso image from it.
You have to go though these 3 steps in order to have an updated from sources
iso image with binary sets.
--
Manuel Bouyer <bouyer@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
From: Havard Eidnes <he@NetBSD.org>
To: bouyer@antioche.eu.org
Cc: gnats-bugs@NetBSD.org, misc-bug-people@NetBSD.org,
gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new
kernel
Date: Wed, 03 Sep 2008 21:24:21 +0200 (CEST)
> > Is there a good reason the distrib/amd64/cdrom/ make target needs t=
o
> > be different from the floppy targets in this respect?
>
> Yes, because it can't be done at the same time (it has to be run afte=
r
> release).
Also when doing what's done under "build.sh ... release", which
is to construct an ISO image containing the boot loader and the
"fat" INSTALL kernel? I find that somewhat hard to beleive, and
would still expect "dependall" and "release" to do what they do
elsewhere to avoid violating the Principle of Least Astonishment,
not like now that when CDRELEASE is not set, the actual build
*and* installation is done under "make release", and is hidden
from view (actions are performed even though nothing shows up in
the make log!) when MAKEVERBOSE=3D1... Again, POLA violation.
> > And why does not "release" dump the built bits into the
> > appropriate RELEASEDIR subdirectory?
>
> I think it does, but maybe not when run from the distrib/ dir.
Why would that make a difference?!? If it does, this is yet
another POLA violation.
Regards,
- H=E5vard
From: Havard Eidnes <he@NetBSD.org>
To: gnats-bugs@NetBSD.org, bouyer@antioche.eu.org
Cc: misc-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new
kernel
Date: Wed, 03 Sep 2008 21:34:30 +0200 (CEST)
> > > And why does not "release" dump the built bits into the
> > > appropriate RELEASEDIR subdirectory?
> >
> > I think it does, but maybe not when run from the distrib/ dir.
>
> To make is clear; here is what happens:
>
> - 'make distribution' builds from sources, and update binairies in $=
DESTDIR,
> and builds INSTALL kernels/ramdisks/boot media. This may include b=
ootable
> iso images without binary sets.
> - 'make release' takes whatever is in $DESTDIR and build the release=
bits
> from it. It won't attempts to rebuild outdated binairies from sour=
ces.
> - 'make iso_image' takes whatever is in $RELEASEDIR and makes a
> (possibly bootable) iso image from it.
>
> You have to go though these 3 steps in order to have an updated from=
sources
> iso image with binary sets.
Yes, I'm well aware of that, but that's not what started this
discussion. I was asking for "how do I produce the boot-CD ISO images
with minimal expenditure of resources", and doing a full build does not=
count as "minimal expenditure of resources" in my book.
I wanted to replicate what's done under "build.sh ... release". The IS=
O
images which come out from this typically only contain a bootloader and=
a "fat" INSTALL kernel, i.e. one with an embedded ramdisk with all the
install tools.
Such a method would be needed when repeatedly testing out fixes to the
kernel, and making a new boot-CD ISO image to try out the new INSTALL
kernel.
I think I've now understood that at least the lack of any output under
MAKEVERBOSE=3D1 in distrib/amd64/cdroms/ when doing "make release"
seriously threw me off course, as well as the "do nothing under
dependall, do everything under release" Makefile setup, both of them
serious and as far as I can see completely unneccessary POLA violations=
!
Regards,
- H=E5vard
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: Havard Eidnes <he@NetBSD.org>
Cc: gnats-bugs@NetBSD.org, misc-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new kernel
Date: Wed, 3 Sep 2008 21:42:14 +0200
On Wed, Sep 03, 2008 at 09:24:21PM +0200, Havard Eidnes wrote:
> > > Is there a good reason the distrib/amd64/cdrom/ make target needs to
> > > be different from the floppy targets in this respect?
> >
> > Yes, because it can't be done at the same time (it has to be run after
> > release).
>
> Also when doing what's done under "build.sh ... release", which
> is to construct an ISO image containing the boot loader and the
> "fat" INSTALL kernel?
No, this one is built as part of 'release', I think.
> I find that somewhat hard to beleive, and
> would still expect "dependall" and "release" to do what they do
> elsewhere to avoid violating the Principle of Least Astonishment,
> not like now that when CDRELEASE is not set, the actual build
> *and* installation is done under "make release", and is hidden
> from view (actions are performed even though nothing shows up in
> the make log!) when MAKEVERBOSE=1... Again, POLA violation.
I think they do. I can't see why it wouldn't work for a cdboot.
Reading the makefiles, I think it should always copy the updated binaries;
even if they didn't change. The make targets don't correspond to created
files.
>
> > > And why does not "release" dump the built bits into the
> > > appropriate RELEASEDIR subdirectory?
> >
> > I think it does, but maybe not when run from the distrib/ dir.
>
> Why would that make a difference?!?
Because the bits to do it is in src/etc/ ?
--
Manuel Bouyer <bouyer@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: Havard Eidnes <he@NetBSD.org>
Cc: gnats-bugs@NetBSD.org, misc-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Subject: Re: misc/39454: UPDATE build of amd64 bootcd fails to pick up new kernel
Date: Wed, 3 Sep 2008 21:48:49 +0200
On Wed, Sep 03, 2008 at 09:34:30PM +0200, Havard Eidnes wrote:
> > > > And why does not "release" dump the built bits into the
> > > > appropriate RELEASEDIR subdirectory?
> > >
> > > I think it does, but maybe not when run from the distrib/ dir.
> >
> > To make is clear; here is what happens:
> >
> > - 'make distribution' builds from sources, and update binairies in $DESTDIR,
> > and builds INSTALL kernels/ramdisks/boot media. This may include bootable
> > iso images without binary sets.
> > - 'make release' takes whatever is in $DESTDIR and build the release bits
> > from it. It won't attempts to rebuild outdated binairies from sources.
> > - 'make iso_image' takes whatever is in $RELEASEDIR and makes a
> > (possibly bootable) iso image from it.
> >
> > You have to go though these 3 steps in order to have an updated from sources
> > iso image with binary sets.
>
> Yes, I'm well aware of that, but that's not what started this
> discussion. I was asking for "how do I produce the boot-CD ISO images
> with minimal expenditure of resources", and doing a full build does not
> count as "minimal expenditure of resources" in my book.
>
> I wanted to replicate what's done under "build.sh ... release". The ISO
> images which come out from this typically only contain a bootloader and
> a "fat" INSTALL kernel, i.e. one with an embedded ramdisk with all the
> install tools.
>
> Such a method would be needed when repeatedly testing out fixes to the
> kernel, and making a new boot-CD ISO image to try out the new INSTALL
> kernel.
>
> I think I've now understood that at least the lack of any output under
> MAKEVERBOSE=1 in distrib/amd64/cdroms/ when doing "make release"
> seriously threw me off course, as well as the "do nothing under
> dependall, do everything under release" Makefile setup, both of them
> serious and as far as I can see completely unneccessary POLA violations!
Actually, the "do nothing under dependall, do everything under release" part
can't easily be avoided; because distrib/ isn't self-contained. It depends
on 'dependall' having being run over the whole tree to work.
I'm not sure why MAKEVERBOSE=1 doens't work; the commands do show up in my
make log even without MAKEVERBOSE=1.
--
Manuel Bouyer <bouyer@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.