NetBSD Problem Report #56942

From dholland@netbsd.org  Wed Jul 27 01:03:23 2022
Return-Path: <dholland@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id DD3AF1A921F
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 27 Jul 2022 01:03:22 +0000 (UTC)
Message-Id: <20220727010322.7AF4D84E9D@mail.netbsd.org>
Date: Wed, 27 Jul 2022 01:03:22 +0000 (UTC)
From: dholland@NetBSD.org
Reply-To: dholland@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: fsck_ffs can't replay wapbl journals to memory
X-Send-Pr-Version: 3.95

>Number:         56942
>Category:       bin
>Synopsis:       fsck_ffs can't replay wapbl journals to memory
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Jul 27 01:05:00 +0000 2022
>Closed-Date:    
>Last-Modified:  Thu Aug 20 14:05:04 +0000 2026
>Originator:     David A. Holland
>Release:        NetBSD 9.99.99 (20220726)
>Organization:
>Environment:
n/a
>Description:

I discovered recently that although fsck_ffs -n claims it's replaying
the journal, it actually doesn't, because fsck doesn't have the
infrastructure necessary to support replaying the journal without
writing to disk.

I fixed the prints so it no longer claims this; however, since running
fsck without having replayed the journal isn't especially meaningful
or helpful, it would be better to add this functionality.

>How-To-Repeat:

crash, fsck -n, observe inconsistencies, mount and unmount (which
replays the journal), fsck -n again, inconsistencies have disappeared.

>Fix:

SMOP but not entirely trivial :-|

>Release-Note:

>Audit-Trail:

State-Changed-From-To: open->feedback
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Thu, 20 Aug 2026 14:05:04 +0000
State-Changed-Why:
> I discovered recently that although fsck_ffs -n claims it's replaying
> the journal, it actually doesn't, because fsck doesn't have the
> infrastructure necessary to support replaying the journal without
> writing to disk.

How did you conclude this?

When fsck_ffs starts, it checks for a journal and prepares it with
wapbl_replay_start:

    186 	/* ffs_superblock_layout() == 2 */
    187 	if (sblock->fs_magic != FS_UFS1_MAGIC ||
    188 	    (sblock->fs_old_flags & FS_FLAGS_UPDATED) != 0) {
    189 		/* can have WAPBL */
    190 		if (check_wapbl() != 0) {
    191 			doskipclean = 0;
    192 		}

https://nxr.netbsd.org/xref/src/sbin/fsck_ffs/setup.c?r=1.112#186

    162 int
    163 check_wapbl(void)
    164 {
...
    218 			if (sblock->fs_flags & FS_DOWAPBL) {
    219 				error = wapbl_replay_start(
    220 				    &wapbl_replay, 0, addr, count, blksize);

wapbl_replay_start loads the journal into memory and sets up the hash
table of pending block writes:

   2959 int
   2960 wapbl_replay_start(struct wapbl_replay **wrp, struct vnode *vp,
   2961     daddr_t off, size_t count, size_t blksize)
   2962 {
...
   3048 	wapbl_blkhash_init(wr, (used >> wch->wc_fs_dev_bshift));
   3049 
   3050 	error = wapbl_replay_process(wr, wch->wc_head, wch->wc_tail);
...
   3131 static void
   3132 wapbl_replay_process_blocks(struct wapbl_replay *wr, off_t *offp)
   3133 {
   3134 	struct wapbl_wc_blocklist *wc =
   3135 	    (struct wapbl_wc_blocklist *)wr->wr_scratch;
   3136 	int fsblklen = 1 << wr->wr_fs_dev_bshift;
   3137 	int i, j, n;
   3138 
   3139 	for (i = 0; i < wc->wc_blkcount; i++) {
   3140 		/*
   3141 		 * Enter each physical block into the hashtable independently.
   3142 		 */
   3143 		n = wc->wc_blocks[i].wc_dlen >> wr->wr_fs_dev_bshift;
   3144 		for (j = 0; j < n; j++) {
   3145 			wapbl_blkhash_ins(wr,
   3146 			    wapbl_block_daddr(wc, i, j, fsblklen),
   3147 			    *offp);
   3148 			wapbl_circ_advance(wr, fsblklen, offp);
   3149 		}
   3150 	}
   3151 }
...
   3212 static int
   3213 wapbl_replay_process(struct wapbl_replay *wr, off_t head, off_t tail)
   3214 {
...
   3231 		case WAPBL_WC_BLOCKS:
   3232 			wapbl_replay_process_blocks(wr, &off);

https://nxr.netbsd.org/xref/src/sys/kern/vfs_wapbl.c?r=1.117#2959

This is used by wapbl_replay_read:

   3435 
   3436 int
   3437 wapbl_replay_read(struct wapbl_replay *wr, void *data, daddr_t blk, long len)
   3438 {
...
   3446 		struct wapbl_blk *wb = wapbl_blkhash_get(wr, blk);
   3447 		if (wb) {
   3448 			off_t off = wb->wb_off;
   3449 			int error;
   3450 			error = wapbl_circ_read(wr, data, fsblklen, &off);

https://nxr.netbsd.org/xref/src/sys/kern/vfs_wapbl.c?r=1.117#3436

which in turn is used by fsck_ffs's bread when wapbl is enabled:

    347 int
    348 bread(int fd, char *buf, daddr_t blk, long size)
    349 {
    350 	char *cp;
    351 	int i, errs;
    352 	off_t offset;
    353 
    354 	offset = blk;
    355 	offset *= dev_bsize;
    356 	if ((pread(fd, buf, (int)size, offset) == size) &&
    357 	    read_wapbl(buf, size, blk) == 0)
    358 		return (0);

https://nxr.netbsd.org/xref/src/sbin/fsck_ffs/utilities.c?r=1.71#347

    138 int
    139 read_wapbl(char *buf, long size, daddr_t blk)
    140 {
    141 
    142 	if (!wapbl_replay || !wapbl_replay_isopen(wapbl_replay))
    143 		return 0;
    144 	return wapbl_replay_read(wapbl_replay, buf, blk, size);
    145 }

https://nxr.netbsd.org/xref/src/sbin/fsck_ffs/wapbl.c?r=1.6#138

So what's missing?


>Unformatted:

Home	PR Database Search