NetBSD Problem Report #60086

From www@netbsd.org  Mon Mar 16 12:26:28 2026
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.netbsd.org", Issuer "R12" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 1E9421A925F
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 16 Mar 2026 12:26:28 +0000 (UTC)
Message-Id: <20260316122626.C8B1C1A9260@mollari.NetBSD.org>
Date: Mon, 16 Mar 2026 12:26:26 +0000 (UTC)
From: andrew@pmk1.net
Reply-To: andrew@pmk1.net
To: gnats-bugs@NetBSD.org
Subject: cgdconfig may not always find ZFS device labels
X-Send-Pr-Version: www-1.0
X-From4GNATS: "andrew@pmk1.net via gnats" <gnats-admin@NetBSD.org>

>Number:         60086
>Category:       bin
>Synopsis:       cgdconfig may not always find ZFS device labels
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Mar 16 12:30:00 +0000 2026
>Last-Modified:  Tue Mar 17 17:25:01 +0000 2026
>Originator:     Andrew Parker
>Release:        NetBSD 11.0_RC2 (and probably others)
>Organization:
none
>Environment:
Architecture: x86_64       
Machine: amd64
>Description:
When using ZFS on top of cgd devices, the verify_zfs function in cgdconfig may not always find the last two labels.  This appears to depend on the size of the device.  In my case, I'm using named GPT wedges.

The verify_zfs function in cgdconfig calculates the offset of the last two disklabels using the device size.  ZFS zdb function (for example) uses an aligned value for determining the offset.  Because of this, cgdconfig's calculated offset may not be where the actual vdev label exists on the device which results in the verify function not working on boot.
>How-To-Repeat:
This is repeatable using a GPT named disk wedge on a USB device:

# gpt show sd0
     start      size  index  contents
         0         1         PMBR
         1         1         Pri GPT header
         2        32         Pri GPT table
        34  31129533      1  GPT part - NetBSD Cryptographic Disk
  31129567        32         Sec GPT table
  31129599         1         Sec GPT header

Followed the guide to get a working cgd device on top of the named wedge and create a zpool.  Verified the labels using zdb:

# zdb -l /dev/rcgd1d 
--------------------------------------------
LABEL 0
--------------------------------------------
    version: 5000
    name: 'sata'
    state: 0
    txg: 4
    pool_guid: 4175716765529626646
    hostname: 'claire-netbsd.local'
    top_guid: 10667087423283623277
    guid: 10667087423283623277
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 10667087423283623277
        path: '/dev/cgd0d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 32
        ashift: 12
        asize: 512105381888
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
--------------------------------------------
LABEL 1
--------------------------------------------
    version: 5000
    name: 'sata'
    state: 0
    txg: 4
    pool_guid: 4175716765529626646
    hostname: 'claire-netbsd.local'
    top_guid: 10667087423283623277
    guid: 10667087423283623277
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 10667087423283623277
        path: '/dev/cgd0d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 32
        ashift: 12
        asize: 512105381888
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
--------------------------------------------
LABEL 2
--------------------------------------------
    version: 5000
    name: 'usb'
    state: 2
    txg: 36
    pool_guid: 6389698929858570228
    hostname: ''
    top_guid: 4200771912375277257
    guid: 4200771912375277257
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 4200771912375277257
        path: '/dev/cgd1d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 27
        ashift: 9
        asize: 15933374464
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data
--------------------------------------------
LABEL 3
--------------------------------------------
    version: 5000
    name: 'usb'
    state: 2
    txg: 36
    pool_guid: 6389698929858570228
    hostname: ''
    top_guid: 4200771912375277257
    guid: 4200771912375277257
    vdev_children: 1
    vdev_tree:
        type: 'disk'
        id: 0
        guid: 4200771912375277257
        path: '/dev/cgd1d'
        whole_disk: 0
        metaslab_array: 37
        metaslab_shift: 27
        ashift: 9
        asize: 15933374464
        is_log: 0
        create_txg: 4
    features_for_read:
        com.delphix:hole_birth
        com.delphix:embedded_data

# echo "1234" |sudo cgdconfig -p -v -Vzfs cgd1  /dev/dk4 # This is the correct p/w for testing
    with alg aes-cbc keylen 256 blocksize 18446744073709551615 ivmethod encblkno1
cgdconfig: verification failed permanently

After adding some extra logging to both cgdconfig and zdb I could see there were calculating different offsets for the last two labels.

>Fix:
Updating cgdconfig to use the same alignment function as zdb (external/cddl/osnet/dist/cmd/zdb/zdb.c) worked for me:

--- a/sbin/cgdconfig/cgdconfig.c
+++ b/sbin/cgdconfig/cgdconfig.c
@@ -1213,6 +1213,8 @@ verify_zfs(int fd)
                return rv;
        }

+       vdev_size = P2ALIGN(vdev_size, (uint64_t)sizeof (vdev_label_t));
+
        vdev_phys_t *vdev_phys = emalloc(sizeof(*vdev_phys));
        for (size_t i = 0; i < VDEV_LABELS; i++) {
                off_t vdev_phys_off = (i < VDEV_LABELS / 2 ?

>Audit-Trail:
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/60086 CVS commit: src/sbin/cgdconfig
Date: Tue, 17 Mar 2026 13:24:47 -0400

 Module Name:	src
 Committed By:	christos
 Date:		Tue Mar 17 17:24:47 UTC 2026

 Modified Files:
 	src/sbin/cgdconfig: cgdconfig.c

 Log Message:
 PR/60086: Andrew Parker: cgdconfig may not always find ZFS device labels
 Updated cgdconfig to use the same alignment function as zdb
 (external/cddl/osnet/dist/cmd/zdb/zdb.c).


 To generate a diff of this commit:
 cvs rdiff -u -r1.63 -r1.64 src/sbin/cgdconfig/cgdconfig.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Home	PR Database Search