NetBSD Problem Report #60663

From www@netbsd.org  Fri Aug 28 16:46:10 2026
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits)
	 client-signature RSA-PSS (2048 bits))
	(Client CN "mail.netbsd.org", Issuer "YR2" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 7F46A1A923B
	for <gnats-bugs@gnats.NetBSD.org>; Fri, 28 Aug 2026 16:46:10 +0000 (UTC)
Message-Id: <20260828164609.BCE0C1A923C@mollari.NetBSD.org>
Date: Fri, 28 Aug 2026 16:46:09 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: POSIX.1-2024: dd(1) options
X-Send-Pr-Version: www-1.0
X-From4GNATS: "campbell+netbsd@mumble.net via gnats" <gnats-admin@NetBSD.org>

>Number:         60663
>Category:       standards
>Synopsis:       POSIX.1-2024: dd(1) options
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kre
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Aug 28 16:50:00 +0000 2026
>Closed-Date:    
>Last-Modified:  Sat Aug 29 00:37:15 +0000 2026
>Originator:     Taylor R Campbell
>Release:        current, 11, 10, ...
>Organization:
The FullBlockSD Foundation, Inc.
>Environment:
>Description:

	In POSIX.1-2024, dd(1) sprouted an iflags=fullblock, which --
	if I understood correctly after 6sec of reading and thinking
	-- when combined with bs=N is equivalent to our ibs=N obs=N
	in filling blocks of input before sending them to output,
	rather than sending partial blocks to outpu.

	We should also sync any other options with POSIX.

>How-To-Repeat:

	port openzfs test suite

>Fix:

	Yes, please!

>Release-Note:

>Audit-Trail:
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: standards/60663: POSIX.1-2024: dd(1) options
Date: Sat, 29 Aug 2026 01:08:29 +0700

     Date:        Fri, 28 Aug 2026 16:50:00 +0000 (UTC)
     From:        "campbell+netbsd@mumble.net via gnats" <gnats-admin@NetBSD.org>
     Message-ID:  <20260828165000.8F95A1A923E@mollari.NetBSD.org>


   | 	if I understood correctly after 6sec of reading and thinking
   | 	-- when combined with bs=N is equivalent to our ibs=N obs=N

 No, not that, those always do ibs=N sized reads, then if needed rebuffer
 whatever is obtained into obs=M sized pieces  (ibs and obs are POSIX
 options as well ... given that they have been in dd since forever).

 iflags=fullblock is intended to cause dd to read the complete N bytes
 from ibs=N before doing any other processing (unless EOF occurs, what
 happens on a read error in this case is, very thoughtfully, forgotten)
 That means, first read request is for N bytes, then if n, less than N,
 were received, another read for N-n bytes, and so on, until all N bytes
 are received.

 ie: no longer are all read requests for N bytes; hence this would be useless
 reading from a tape drive, and pointless reading from a file - but might
 make sense reading from a TCP connection, and perhaps from a pipe.

 The fullblock processing can be used with bs= or with ibs=/obs=  -- it really
 just hides data sources that can deliver the data piecemeal, rather than in
 nice pre-prdained blocks all at once ... so much so, that however many
 read()s are needed to fill the input block size (from bs, or ibs) still
 counts as 1 record read for the purposes of count= and the reports.


From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: standards/60663: POSIX.1-2024: dd(1) options
Date: Fri, 28 Aug 2026 20:18:45 +0200

 You mean you could then finally do something like this?

 	zcat disk.img.gz | dd bs=1m iflags=fullblock of=/dev/rdk42

 or

 	progress -zf disk.img.gz dd bs=1m iflags=fullblock of=/dev/rdk42

 Sounds very usefull.

 Martin

From: Taylor R Campbell <tcplquincy20260828@campbell.mumble.net>
To: Martin Husemann <martin@duskware.de>,
	Robert Elz <kre@munnari.OZ.AU>
Cc: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: standards/60663: POSIX.1-2024: dd(1) options
Date: Fri, 28 Aug 2026 19:13:05 +0000

 > Date: Fri, 28 Aug 2026 20:18:45 +0200
 > From: Martin Husemann <martin@duskware.de>
 >
 > You mean you could then finally do something like this?
 > 
 > 	zcat disk.img.gz | dd bs=1m iflags=fullblock of=/dev/rdk42
 > 
 > or
 > 
 > 	progress -zf disk.img.gz dd bs=1m iflags=fullblock of=/dev/rdk42
 > 
 > Sounds very usefull.

 For some reason I had the idea that using ibs and obs together worked
 reliably for this purpose, so I've long been doing stuff like

 progress -zf disk.img.gz dd ibs=64k obs=1m of=dev/rdk42
 nc ... | dd ibs=64k obs=1m of=/dev/rsd2

 instead of

 progress -zf disk.img.gz dd bs=1m of=dev/rdk42
 nc ... | dd bs=1m of=/dev/rsd2

 and while I noticed long ago that the bs=1m version didn't work
 reliably, the ibs=64k obs=1m version has always appeared to work for
 me.  Maybe it's an accident of coinciding with BIG_PIPE_SIZE?  Maybe
 it didn't work reliably and I just wasn't verifying hard enough?  No
 idea!

 Anyway, yes, having this work reliably would be useful -- I dunno why
 it took until 2024 for that to become a standard concept.

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: standards/60663: POSIX.1-2024: dd(1) options
Date: Sat, 29 Aug 2026 02:59:50 +0700

     Date:        Fri, 28 Aug 2026 19:15:01 +0000 (UTC)
     From:        "Taylor R Campbell via gnats" <gnats-admin@NetBSD.org>
     Message-ID:  <20260828191501.896961A923E@mollari.NetBSD.org>

   |  For some reason I had the idea that using ibs and obs together worked
   |  reliably for this purpose, so I've long been doing stuff like
   |  
   |  progress -zf disk.img.gz dd ibs=64k obs=1m of=dev/rdk42
   |  nc ... | dd ibs=64k obs=1m of=/dev/rsd2

 That's fine, and yes, that works, and is reliable.   The only real
 difference between this and the iflags=fullblock together with bs=
 is some less data copying inside dd (at the expense perhaps of a few
 extra read() sys calls, but probably not all that many) in the fullblock
 case.

 Where the difference really appears is when you add count=N or skip=M
 options as well, with the fullblock version, you're getting N*B bytes
 of output with -count=N -bs=B, with the ibs/obs version you're getting
 N*avg-read-size (ie: it does N reads, and however much they produce
 is what is written).



Responsible-Changed-From-To: standards-manager->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Fri, 28 Aug 2026 21:31:19 +0000
Responsible-Changed-Why:
I am looking into this PR


From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: standards/60663: POSIX.1-2024: dd(1) options
Date: Sat, 29 Aug 2026 06:06:25 +0700

 I am waiting for an (absurdly slow) "cvs update" to complete, so
 I can do a build to verify I am not likely to be breaking anything,
 but in the meantime this is what the changes cause:

 jacaranda$ cat /netbsd | ./dd iflags=fullblock bs=128k of=/tmp/TRASH
 207+1 records in
 207+1 records out
 27214200 bytes transferred in 0.040 secs (680355000 bytes/sec)

 jacaranda$ cat /netbsd | ./dd bs=128k of=/tmp/TRASH
 0+1662 records in
 0+1662 records out
 27214200 bytes transferred in 0.048 secs (566962500 bytes/sec)

 I wouldn't place much emphasis on the bytes/sec differences, these were
 singleton runs, the difference is that with "iflags=fullblock" each block
 read is 128k whereas without it, they aren't, which would have affected
 count=10 or something, had I specified it.

 Similarly:

 jacaranda$ cat /netbsd | ./dd iflags=fullblock ibs=128k obs=256k of=/tmp/TRASH
 207+1 records in
 103+1 records out
 27214200 bytes transferred in 0.058 secs (469210344 bytes/sec)

 jacaranda$ cat /netbsd | ./dd ibs=128k obs=256k of=/tmp/TRASH
 0+1662 records in
 103+1 records out
 27214200 bytes transferred in 0.051 secs (533611764 bytes/sec)

 kre

 ps: cvs finished its update while I have been typing, so commit should be soon.


From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: standards/60663: POSIX.1-2024: dd(1) options
Date: Sat, 29 Aug 2026 07:32:02 +0700

 And then I forgot the PR number in the commit log message:

 Here is what should have been sent to gnats:

 Message-id: <20260829000518.A1FCBFA2D@cvs.NetBSD.org>
 Reply-to: source-changes-d@NetBSD.org
 To: source-changes@NetBSD.org
 From: "Robert Elz" <kre@netbsd.org>
 Subject: CVS commit: src/bin/dd
 Date: Sat, 29 Aug 2026 00:05:18 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Sat Aug 29 00:05:18 UTC 2026

 Modified Files:
 	src/bin/dd: args.c dd.1 dd.c dd.h position.c

 Log Message:
 dd(1) - implement POSTX Issue 8 "-iflags=fullblock" option


 To generate a diff of this commit:
 cvs rdiff -u -r1.44 -r1.45 src/bin/dd/args.c
 cvs rdiff -u -r1.41 -r1.42 src/bin/dd/dd.1
 cvs rdiff -u -r1.54 -r1.55 src/bin/dd/dd.c
 cvs rdiff -u -r1.16 -r1.17 src/bin/dd/dd.h
 cvs rdiff -u -r1.18 -r1.19 src/bin/dd/position.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.



State-Changed-From-To: open->feedback
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sat, 29 Aug 2026 00:37:15 +0000
State-Changed-Why:
Are the changes now in HEAD satisfactory?

This is a new feature addition, so I am not planning to request pullups.


>Unformatted:

Home	PR Database Search