NetBSD Problem Report #33924

From www@NetBSD.org  Wed Jul  5 22:19:31 2006
Return-Path: <www@NetBSD.org>
Received: by narn.NetBSD.org (Postfix, from userid 31301)
	id 79F9663B863; Wed,  5 Jul 2006 22:19:31 +0000 (UTC)
Message-Id: <20060705221931.79F9663B863@narn.NetBSD.org>
Date: Wed,  5 Jul 2006 22:19:31 +0000 (UTC)
From: tg@mirbsd.org
Reply-To: tg@mirbsd.org
To: gnats-bugs@NetBSD.org
Subject: feature request: makefs(8) should output blocks used by each file
X-Send-Pr-Version: www-1.0

>Number:         33924
>Category:       toolchain
>Synopsis:       feature request: makefs(8) should output blocks used by each file
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    toolchain-manager
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Wed Jul 05 22:20:00 +0000 2006
>Last-Modified:  Sat Oct 25 00:30:02 +0000 2008
>Originator:     Thorsten Glaser
>Release:        code from cvs -rHEAD
>Organization:
MirOS Project
>Environment:
MirBSD odem.66h.42h.de 9 Kv#9s81-stable-20060701 GENERIC#923 i386
>Description:
mkisofs has an option to output the blocks used by
each file during filesystem creation process. This
looks like that:

29 31 workdir/00-README
32 32 /TRANS.TBL
33 33 workdir/etc/boot.cfg
34 54 workdir/v9/i386/boot.iso
55 75 workdir/v9/i386/boot.liv
76 2182 workdir/v9/i386/bsd.rd
2183 2206 workdir/v9/i386/pxeboot

The numbers are: start sector(!) and end sector,
measured in media sectors (here 2048 bytes, on
any other media it would be 512 bytes). These are
sectors, not filesystem blocks.

This _also_ means (and THAT is important for MirOS)
that the file TRANS.TBL starts at offset 32*2048 in
the generated ISO and is (32-32+1)*2048 bytes long.
More importantly, the bootloader starts at 55*2048
and is (75-55+1)*2048 bytes long - that information
is used by our installboot(8). I know NetBSD(R) has
different bootloaders, but in my opinion even you
could use this kind of information somewhere.

While this format is useful for -t cd9660 option of
makefs(8) it does not make much sense for ffs because
files can be split there, as far as I understand that
filesystem. (If I'm wrong here, please ignore the fol-
lowing commentary and just fix makefs(8) to have an
option for output in the above format.)

If a file is split across several filesystem blocks,
say, blocks 8, 9 and 11, and we assume a fs block is
4096 bytes (8 sectors), the file then occupies the
sectors 64-71, 72-79, and 88-95. The optimal output
format would have them already concatenated, so it
would look like this:

path/to/ourfile X 64 79 88 95

The 'X' here is some kind of separator, in case the
filenames contain spaces. It should be user-selectable.
(Or do you guys have a better idea?)

An alternate format, a little harder to parse:

2 64 79 88 95 path/to/ourfile

This might even be the better choice.

I would have attached a patch, but reading the code,
I am unable to write it myself because it looks sort
of messy... (historical ballast and sharing code with
the kernel, probably). I would be glad if someone who
actually knows what he's doing would look into this.

Unrelated footnote: mkisofs 2 has -boot-info-table
option which we need for bootable ISOs. To implement
that would be the next thing. I haven't yet looked
if I can do it myself (I probably can, I think) but
if someone is working on it or wants to work on it,
please do so, because I have an entire operating sy-
stem to take care of ;)

Thanks in advance for any help.

Please Cc: me (tg@mirbsd.org) or our general discus-
sion list (miros-discuss@mirbsd.org) on any followup
if possible. Thanks for your co-operation.
>How-To-Repeat:
this is a feature request
>Fix:
hack in some -v or -T (trace) or whatever option.
Maybe even hide it in -d (debug) if you want.

>Audit-Trail:
From: Luke Mewburn <lukem@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: toolchain-manager@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: toolchain/33924: feature request: makefs(8) should output blocks used by each file
Date: Thu, 6 Jul 2006 09:04:56 +1000

 --UnaWdueM1EBWVRzC
 Content-Type: text/plain; charset=us-ascii
 Content-Disposition: inline
 Content-Transfer-Encoding: quoted-printable

 On Wed, Jul 05, 2006 at 10:20:00PM +0000, tg@mirbsd.org wrote:
   | If a file is split across several filesystem blocks,
   | say, blocks 8, 9 and 11, and we assume a fs block is
   | 4096 bytes (8 sectors), the file then occupies the
   | sectors 64-71, 72-79, and 88-95. The optimal output
   | format would have them already concatenated, so it
   | would look like this:
   |=20
   | path/to/ourfile X 64 79 88 95
   |=20
   | The 'X' here is some kind of separator, in case the
   | filenames contain spaces. It should be user-selectable.
   | (Or do you guys have a better idea?)

 You could just use a space separator, and use the vis(3)
 encoding already used by the mtree format for the filename.
 (This format is used by the "-F specfile" makefs option)

 --UnaWdueM1EBWVRzC
 Content-Type: application/pgp-signature
 Content-Disposition: inline

 -----BEGIN PGP SIGNATURE-----
 Version: GnuPG v1.4.3 (NetBSD)

 iD8DBQFErEWYpBhtmn8zJHIRAvz1AJ94mpy7hCmbMhXJUO+aJsjDGMq0JQCeKG7L
 wE6kUPSAOg4m49oRK8FbDC4=
 =zxsI
 -----END PGP SIGNATURE-----

 --UnaWdueM1EBWVRzC--

From: Thorsten Glaser <tg@mirbsd.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: toolchain/33924: feature request: makefs(8) should output blocks
 used by each file
Date: Wed, 22 Oct 2008 18:44:17 +0000 (UTC)

 Hello,

 please feel free to close this PR. This feature request has been super-
 seded by the existence of http://www.mirbsd.org/man1/getextent_cd9660 =E2=
 =80=93
 source at http://cvs.mirbsd.de/src/usr.sbin/getextent_cd9660/

 Now if you people would please focus on toolchain/33925 which is really
 critical for Live CDs (when using =E2=80=9C-t cd9660=E2=80=9D stops at the =
 assertion in
 makefs/cd9660.c:cd9660_sorted_child_insert(), probably due to too deep
 directory nesting, even using =E2=80=9C-o allow-deep-trees=E2=80=9D did not=
  help, so I
 have to use 4.2FFS for it) where the image size, unlike floppies, has
 not been decided upon prior to image creation.

 TIA,
 //mirabilos
 --=20
 Sometimes they [people] care too much: pretty printers [and syntax highligh=
 -
 ting, d.A.] mechanically produce pretty output that accentuates irrelevant
 detail in the program, which is as sensible as putting all the prepositions
 in English text in bold font.=09-- Rob Pike in "Notes on Programming in C"

From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@NetBSD.org
Cc: tg@mirbsd.org
Subject: Re: toolchain/33924: feature request: makefs(8) should output blocks used by each file
Date: Thu, 23 Oct 2008 16:18:56 +0300

 On Wed Oct 22 2008 at 18:50:03 +0000, Thorsten Glaser wrote:
 > The following reply was made to PR toolchain/33924; it has been noted by GNATS.
 > 
 > From: Thorsten Glaser <tg@mirbsd.de>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: toolchain/33924: feature request: makefs(8) should output blocks
 >  used by each file
 > Date: Wed, 22 Oct 2008 18:44:17 +0000 (UTC)
 > 
 >  Hello,
 >  
 >  please feel free to close this PR. This feature request has been super-
 >  seded by the existence of http://www.mirbsd.org/man1/getextent_cd9660 =E2=
 >  =80=93
 >  source at http://cvs.mirbsd.de/src/usr.sbin/getextent_cd9660/

 On modern NetBSD, such utilities should be written using the rump and ukfs
 libraries so that they work on all file systems and do not unnecessarily
 duplicate file system specific knowledge in the utility.

From: Matthias Scheler <tron@zhadum.org.uk>
To: NetBSD GNATS <gnats-bugs@NetBSD.org>
Cc: Antti Kantee <pooka@cs.hut.fi>
Subject: Re: toolchain/33924: feature request: makefs(8) should output blocks used by each file
Date: Fri, 24 Oct 2008 13:12:42 +0100

 On Thu, Oct 23, 2008 at 01:20:04PM +0000, Antti Kantee wrote:
 >  On modern NetBSD, such utilities should be written using the rump and ukfs
 >  libraries so that they work on all file systems and do not unnecessarily
 >  duplicate file system specific knowledge in the utility.

 "makefs" can be used while cross-building on a platform other than NetBSD.
 AFAIK "rump" cannot be used on other platform.

 	Kind regards

 -- 
 Matthias Scheler                                  http://zhadum.org.uk/

From: Antti Kantee <pooka@cs.hut.fi>
To: Matthias Scheler <tron@zhadum.org.uk>
Cc: NetBSD GNATS <gnats-bugs@NetBSD.org>
Subject: Re: toolchain/33924: feature request: makefs(8) should output blocks used by each file
Date: Sat, 25 Oct 2008 03:27:10 +0300

 On Fri Oct 24 2008 at 13:12:42 +0100, Matthias Scheler wrote:
 > On Thu, Oct 23, 2008 at 01:20:04PM +0000, Antti Kantee wrote:
 > >  On modern NetBSD, such utilities should be written using the rump and ukfs
 > >  libraries so that they work on all file systems and do not unnecessarily
 > >  duplicate file system specific knowledge in the utility.
 > 
 > "makefs" can be used while cross-building on a platform other than NetBSD.
 > AFAIK "rump" cannot be used on other platform.

 Your knowledge is wrong.

 Might not work out-of-the-box, though, as I haven't lately been paying
 much attention to non-NetBSD platforms or tested them, since there are
 currently no users in the tree.  At least the pthread_islocked_np()
 calls are likely to need some massaging.  Then again, compiling without
 DIAGNOSTIC on non-NetBSD might work as a quick fix.

Home	PR Database Search