NetBSD Problem Report #60956

From www@netbsd.org  Mon Nov  2 18:39:19 2026
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits)
	 client-signature RSA-PSS (2048 bits))
	(Client CN "mail.netbsd.org", Issuer "YR1" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 795C81A923B
	for <gnats-bugs@gnats.NetBSD.org>; Mon,  2 Nov 2026 18:39:19 +0000 (UTC)
Message-Id: <20261102183918.2A4CF1A923C@mollari.NetBSD.org>
Date: Mon,  2 Nov 2026 18:39:18 +0000 (UTC)
From: el@horse64.org
Reply-To: el@horse64.org
To: gnats-bugs@NetBSD.org
Subject: ramdisk-cgdroot.fs seems to check for /libdata/firmware/ in the ram disk rather than the mounted cgdroot
X-Send-Pr-Version: www-1.0
X-From4GNATS: "el@horse64.org via gnats" <gnats-admin@NetBSD.org>

>Number:         60956
>Category:       misc
>Synopsis:       ramdisk-cgdroot.fs seems to check for /libdata/firmware/ in the ram disk rather than the mounted cgdroot
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    misc-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Nov 02 18:40:00 +0000 2026
>Last-Modified:  Tue Nov 03 14:45:00 +0000 2026
>Originator:     Ellie
>Release:        11.0
>Organization:
I'm just a chaotic free software tinkerer :)
>Environment:
NetBSD ruby-b-s-delly 11.0_STABLE NetBSD 11.0_STABLE (GENERIC) #0: Fri Oct 30 19:02:53 UTC 2026  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64
>Description:
The ramdisk-cgdroot.fs seems to check for /libdata/firmware/ in the ram disk rather than the mounted cgdroot.

This means if you boot via the normal multi user way, it will start up your system with / being your actual physical unlocked partition inside the cgd container, however, it will apparently retain the RAM fs it used to show the password prompt (from before the unlock being completed) as a hidden / and check there for /libdata/firmware/ when loading firmware.

Since the RAM fs is a minimal rescue shell, it has no firmware. Therefore, e.g. my wifi stick will cause this error to show up:

urtwn0: autoconfiguration error: failed load firmware of file rtl8192eefw.bin (error 2)

And then it will not work.

This can be worked around by:

1. setting rc_configured=NO in /etc/rc.conf to prevent the multi user boot from completing,

2. in the RAM disk, first set up /libdata/firmware as a sym link to the actual physical disk in /altroot and then set up /altroot as a proper chroot https://codeberg.org/ell1e/ellieputercrap/src/commit/1e3b91c3af8691a80e06ef6c2555f16aca9d40ea/NetBSD/ellies_cursed_altroot_chroot.sh (This script does all that),

3. then you have to of course start the "network" and "wpa_supplicant" and all other services yourself since normal multi user boot didn't run. But after that, your wifi will work:
>How-To-Repeat:
1. Set up an encrypted cgd with an FFS 2ea / partition inside. /etc/cgd and /efi should be separate unencrypted partitions each.

2. Install a NetBSD as usual, and make sure that /efi/EFI/BOOT/ contains the necessary boot loader and that /etc/cgd/cgd.cfg contains the cgdroot unlock config. Do a terminal install without any display managers. Make sure to copy the kernel to /efi/netbsd and the ramdisk to /efi/ramdisk-cgdroot.fs.

3. Make sure and double check that your partitions are labelled correctly with LABEL=efi for efi, LABEL=cgdroot for the cgd, and LABEL=cgd.conf for the /etc/cgd partition.

4. Set up your boot.cfg such that you invoke fs /ramdisk-cgdroot.fs and then invoke boot hd0a:/netbsd to get to the password prompt (Don't ask me why exactly the hd0a addition is needed, I got this form chat).

5. You should now be able to boot up and reach multi user stage.

6. After booting into multi user you should be greeted with a TTY. Login and e.g. try to use an rt8192 wifi stick that needs a firmware upload.

(Sorry if I forgot a step, I tried to be as exhaustive as possible. There might be some minor things required in the /efi partition that I forgot about.)
>Fix:
Well in theory the /libdata/firmware inside the ramdisk just needs to point somehow to the real /libdata/firmware on the actual physical disk via a symlink. I'm not sure how one would best do that.

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for
 /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Mon, 2 Nov 2026 20:53:35 +0100

 This is by design and unavoidable as long as a chroot(1) is used
 to set the "real" root.

 You have to populate the original mount.

 You can create your custom boot filesystem for that, see

 	https://wiki.netbsd.org/security/custom_cgdroot/


 Martin

From: Ellie <el@horse64.org>
To: gnats-bugs@netbsd.org, misc-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for
 /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Mon, 2 Nov 2026 20:04:10 +0000

 I think there might be a misunderstanding. On the user side, me, chroot 
 was not used at all:

 In fact, if I use my chroot script then it works (since then I can 
 symlink it as needed) - this is why I mentioned it. Sorry for that being 
 confusing! The default setup is just straight up ramdisk-cgdroot.fs 
 multiuser boot, and there it breaks.

 Regards,

 Ellie

 On 11/2/26 19:55, Martin Husemann via gnats wrote:
 > The following reply was made to PR misc/60956; it has been noted by GNATS.
 > 
 > From: Martin Husemann <martin@duskware.de>
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for
 >   /libdata/firmware/ in the ram disk rather than the mounted cgdroot
 > Date: Mon, 2 Nov 2026 20:53:35 +0100
 > 
 >   This is by design and unavoidable as long as a chroot(1) is used
 >   to set the "real" root.
 >   
 >   You have to populate the original mount.
 >   
 >   You can create your custom boot filesystem for that, see
 >   
 >   	https://wiki.netbsd.org/security/custom_cgdroot/
 >   
 >   
 >   Martin
 >   

From: "David H. Gutteridge" <david@gutteridge.ca>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for
 /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Mon, 02 Nov 2026 17:07:06 -0500

 I haven't tried this myself (I don't use this particular setup), but I
 believe it may be possible instead to add the /altroot path to the
 firmware loading path, e.g.:

 hw.firmware.path=3D/altroot/libdata/firmware:/altroot/usr/libdata/firmware

 ... or such in your sysctl.conf.

 Regards,

 Dave

From: Taylor R Campbell <riastradh@NetBSD.org>
To: "David H. Gutteridge" <david@gutteridge.ca>
Cc: Ellie <el@horse64.org>, Martin Husemann <martin@duskware.de>,
	gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Tue, 3 Nov 2026 14:26:54 +0000

 > Date: Mon, 02 Nov 2026 17:07:06 -0500
 > From: "David H. Gutteridge" <david@gutteridge.ca>
 > 
 > I haven't tried this myself (I don't use this particular setup), but I
 > believe it may be possible instead to add the /altroot path to the
 > firmware loading path, e.g.:
 > 
 > hw.firmware.path=/altroot/libdata/firmware:/altroot/usr/libdata/firmware
 > 
 > ... or such in your sysctl.conf.

 This helps with drivers that load firmware at time of first use by
 userland, but not with drivers that load firmware at config_mountroot
 time.

 For those drivers, we need to either:

 (a) bake the firmware images into cgdroot.fs;

 (b) teach the bootloader to gather firmware images from another source
     than the root file system like it does for the kernel, modules,
     and cgdroot.fs itself; or

 (c) find a way to defer the firmware loading in these drivers until
     the `real' root has been mounted -- which the kernel doesn't have
     a concept of.

 And, if there are any drivers that only load firmware at time of first
 use (not sure offhand which, if any, do this), it would be nice for
 them to not get out of sync between the boot medium, cgdroot.fs, and
 the `real' root.

 It should not be a matter of asking users to manually assemble their
 own custom cgdroot.fs from sets or a custom build -- we should have a
 systematic way to handle this.

From: Martin Husemann <martin@duskware.de>
To: Taylor R Campbell <riastradh@NetBSD.org>
Cc: "David H. Gutteridge" <david@gutteridge.ca>, Ellie <el@horse64.org>,
	gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for
 /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Tue, 3 Nov 2026 15:30:19 +0100

 On Tue, Nov 03, 2026 at 02:26:54PM +0000, Taylor R Campbell wrote:
 > It should not be a matter of asking users to manually assemble their
 > own custom cgdroot.fs from sets or a custom build -- we should have a
 > systematic way to handle this.

 Or do the magic root pivot w/o chrooting ;-)

 Martin

From: Taylor R Campbell <riastradh@NetBSD.org>
To: Martin Husemann <martin@duskware.de>
Cc: "David H. Gutteridge" <david@gutteridge.ca>,
	Ellie <el@horse64.org>, gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Tue, 3 Nov 2026 14:35:52 +0000

 > Date: Tue, 3 Nov 2026 15:30:19 +0100
 > From: Martin Husemann <martin@duskware.de>
 > 
 > On Tue, Nov 03, 2026 at 02:26:54PM +0000, Taylor R Campbell wrote:
 > > It should not be a matter of asking users to manually assemble their
 > > own custom cgdroot.fs from sets or a custom build -- we should have a
 > > systematic way to handle this.
 > 
 > Or do the magic root pivot w/o chrooting ;-)

 This doesn't help.  Whether userland does chroot or mounts a pivot
 root or dances the waltz, the kernel doesn't know (a) that instead of
 running config_mountroot hooks as soon as cgdroot.fs is mounted, it
 has to wait for userland to mount the real root; or (b) when userland
 has mounted the real root and it can finally run the config_mountroot
 hooks.

From: Martin Husemann <martin@duskware.de>
To: Taylor R Campbell <riastradh@NetBSD.org>
Cc: "David H. Gutteridge" <david@gutteridge.ca>, Ellie <el@horse64.org>,
	gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: misc/60956: ramdisk-cgdroot.fs seems to check for
 /libdata/firmware/ in the ram disk rather than the mounted cgdroot
Date: Tue, 3 Nov 2026 15:42:22 +0100

 On Tue, Nov 03, 2026 at 02:35:52PM +0000, Taylor R Campbell wrote:
 > This doesn't help.  Whether userland does chroot or mounts a pivot
 > root or dances the waltz, the kernel doesn't know (a) that instead of
 > running config_mountroot hooks as soon as cgdroot.fs is mounted, it
 > has to wait for userland to mount the real root; or (b) when userland
 > has mounted the real root and it can finally run the config_mountroot
 > hooks.

 Right, it helps only for drivers that do it at first ifconfig up time
 or similar.

 For others we could decouple the config_mountroot() and the "hey this
 is a good time to (re-)try loading your firmware" event, and then
 send that event at config_mountroot() time but also (again) at pivot
 time. For chroots we probably don't want to do that for security reasons.

 Martin

Home	PR Database Search