NetBSD Problem Report #60369
From www@netbsd.org Thu Jun 25 18:16:56 2026
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits)
client-signature RSA-PSS (2048 bits))
(Client CN "mail.netbsd.org", Issuer "R13" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 975141A923A
for <gnats-bugs@gnats.NetBSD.org>; Thu, 25 Jun 2026 18:16:56 +0000 (UTC)
Message-Id: <20260625181655.194501A923B@mollari.NetBSD.org>
Date: Thu, 25 Jun 2026 18:16:55 +0000 (UTC)
From: bruno@clisp.org
Reply-To: bruno@clisp.org
To: gnats-bugs@NetBSD.org
Subject: mbrtowc, mbrlen have wrong return value for some invalid byte sequences
X-Send-Pr-Version: www-1.0
X-From4GNATS: "bruno@clisp.org via gnats" <gnats-admin@NetBSD.org>
>Number: 60369
>Category: standards
>Synopsis: mbrtowc, mbrlen have wrong return value for some invalid byte sequences
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: standards-manager
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Thu Jun 25 18:20:00 +0000 2026
>Closed-Date:
>Last-Modified: Fri Sep 11 16:15:01 +0000 2026
>Originator: Bruno Haible
>Release: 10.0
>Organization:
GNU
>Environment:
NetBSD ... 10.0 NetBSD 10.0 (GENERIC) ...
>Description:
When encountering an invalid byte sequence, mbrtowc and mbrlen must return (size_t)(-1). The return value (size_t)(-2) is reserved for incomplete byte sequences, that is, for byte sequences to which one or more bytes need to be appended before it can be decided whether the augmented byte sequences is valid or invalid.
In an UTF-8 locale, for some invalid byte sequences, NetBSD's mbrtowc and mbrlen return (size_t)(-2) when in fact they should return (size_t)(-1).
>How-To-Repeat:
Save this program as foo.c.
==============================================================
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <wchar.h>
int
main ()
{
if (setlocale (LC_ALL, "en_US.UTF-8") == NULL)
return 1;
// Expected: -1 -1 -1 -1
// NetBSD: -2 -2 -2 -2
{
mbstate_t state;
memset (&state, 0, sizeof (mbstate_t));
wchar_t wc = 0xDEADBEEF;
size_t ret = mbrtowc (&wc, "\xE0x\xE0", 2, &state);
printf ("ret = %d\n", (int)ret);
}
{
mbstate_t state;
memset (&state, 0, sizeof (mbstate_t));
wchar_t wc = 0xDEADBEEF;
size_t ret = mbrtowc (&wc, "\xF0x\xF0", 3, &state);
printf ("ret = %d\n", (int)ret);
}
{
mbstate_t state;
memset (&state, 0, sizeof (mbstate_t));
size_t ret = mbrlen ("\xE0x\xE0", 2, &state);
printf ("ret = %d\n", (int)ret);
}
{
mbstate_t state;
memset (&state, 0, sizeof (mbstate_t));
size_t ret = mbrlen ("\xF0x\xF0", 3, &state);
printf ("ret = %d\n", (int)ret);
}
return 0;
}
==============================================================
$ cc foo.c
$ ./a.out
Expected output:
-1
-1
-1
-1
Actual output:
-2
-2
-2
-2
>Fix:
>Release-Note:
>Audit-Trail:
From: "Thomas Klausner" <wiz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: src/lib/libc/citrus/modules
Date: Fri, 26 Jun 2026 11:58:57 +0000
Module Name: src
Committed By: wiz
Date: Fri Jun 26 11:58:57 UTC 2026
Modified Files:
src/lib/libc/citrus/modules: citrus_utf8.c
Log Message:
Fix return value for some invalid UTF-8 byte sequences
Update the length definitions to honor RFC 3629 (from 2003):
o The octet values C0, C1, F5 to FF never appear.
Add a check that continuation bytes are valid when adding them to the
internal buffer so that an out-of-bytes error doesn't override the
"invalid bytes" error.
Fixes PR 60369 by Bruno Haible.
To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.19 src/lib/libc/citrus/modules/citrus_utf8.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->needs-pullups
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Fri, 26 Jun 2026 12:03:01 +0000
State-Changed-Why:
[pullup-11 #339]
[pullup-10 #1285]
Fixed in HEAD, pullups requested.
From: Taylor R Campbell <riastradh@NetBSD.org>
To: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Cc: wiz@NetBSD.org, bruno@clisp.org
Subject: Re: standards/60369: mbrtowc, mbrlen have wrong return value for some invalid byte sequences
Date: Fri, 26 Jun 2026 20:32:56 +0000
Need automatic tests for this too!
Should really just exhaustively check that mbrtowc does the right
thing for all possible 4-byte sequences, maybe randomly subsampled for
regular test suite runs.
State-Changed-From-To: needs-pullups->open
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Fri, 26 Jun 2026 21:47:13 +0000
State-Changed-Why:
Looks like this broke some tests we already have -- either need to
fix the tests or fix the fix, not sure which:
https://releng.netbsd.org/b5reports/i386/commits-2026.06.html#build-2026.06.26.11.58.57
Test case: lib/libc/locale/t_mbrtowc/mbrtowc_internal
Duration: 0.003677 seconds
Termination reason
FAILED: Invalid sequence
Standard output stream
Checking string: "ABCD01234_\134"
Using locale: C
Using mbstate: no
First using repeated mbrtowc
Now using mbsrtowcs
Ok.
Checking string: "[\001\177][\200ÿ]["
Using locale: C/en_US.UTF-8/C/C/C/C
Using mbstate: no
First using repeated mbrtowc
Test case: lib/libc/locale/t_mbrtowc/mbrtowc_object
Duration: 0.003976 seconds
Termination reason
FAILED: Invalid sequence
Standard output stream
Checking string: "ABCD01234_\134"
Using locale: C
Using mbstate: yes
First using repeated mbrtowc
Now using mbsrtowcs
Ok.
Checking string: "[\001\177][\200ÿ]["
Using locale: C/en_US.UTF-8/C/C/C/C
Using mbstate: yes
First using repeated mbrtowc
Test case: lib/libc/locale/t_mbstowcs/mbstowcs_basic
Duration: 0.002706 seconds
Termination reason
FAILED: /tmp/build/2026.06.26.11.58.57-i386/src/tests/lib/libc/locale/t_mbstowcs.c:178: (ssize_t)mbstowcs(wbuf, t->data, SIZE-1): Illegal byte sequence
Standard output stream
Checking string: "[\001\177][\200ÿ]["
Using locale: C/en_US.UTF-8/C/C/C/C
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: src/tests/lib/libc/locale
Date: Fri, 26 Jun 2026 22:36:23 +0000
Module Name: src
Committed By: riastradh
Date: Fri Jun 26 22:36:23 UTC 2026
Modified Files:
src/tests/lib/libc/locale: t_mbrtowc.c
Log Message:
t_mbrtowc: Mark UTF-8 test cases xfail.
mbrtowc previously failed to reject invalid (legacy 5/6-byte) UTF-8,
so it accepted this test case. Now it rejects this test case,
because the test case itself is broken.
Need to split this test up into:
1. correctly decoding the valid inputs
2. correctly rejecting the invalid inputs
But for now marking the test case xfail is an adequate approximation
to the more complicated truth.
PR lib/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences: Invalid sequence
To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.3 src/tests/lib/libc/locale/t_mbrtowc.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: [netbsd-11] src/lib/libc/citrus/modules
Date: Sat, 27 Jun 2026 16:46:58 +0000
Module Name: src
Committed By: martin
Date: Sat Jun 27 16:46:58 UTC 2026
Modified Files:
src/lib/libc/citrus/modules [netbsd-11]: citrus_utf8.c
Log Message:
Pull up following revision(s) (requested by wiz in ticket #339):
lib/libc/citrus/modules/citrus_utf8.c: revision 1.19
Fix return value for some invalid UTF-8 byte sequences
Update the length definitions to honor RFC 3629 (from 2003):
o The octet values C0, C1, F5 to FF never appear.
Add a check that continuation bytes are valid when adding them to the
internal buffer so that an out-of-bytes error doesn't override the
"invalid bytes" error.
Fixes PR 60369 by Bruno Haible.
To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.18.42.1 src/lib/libc/citrus/modules/citrus_utf8.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: [netbsd-10] src/lib/libc/citrus/modules
Date: Sat, 27 Jun 2026 16:48:24 +0000
Module Name: src
Committed By: martin
Date: Sat Jun 27 16:48:24 UTC 2026
Modified Files:
src/lib/libc/citrus/modules [netbsd-10]: citrus_utf8.c
Log Message:
Pull up following revision(s) (requested by wiz in ticket #1285):
lib/libc/citrus/modules/citrus_utf8.c: revision 1.19
Fix return value for some invalid UTF-8 byte sequences
Update the length definitions to honor RFC 3629 (from 2003):
o The octet values C0, C1, F5 to FF never appear.
Add a check that continuation bytes are valid when adding them to the
internal buffer so that an out-of-bytes error doesn't override the
"invalid bytes" error.
Fixes PR 60369 by Bruno Haible.
To generate a diff of this commit:
cvs rdiff -u -r1.18 -r1.18.38.1 src/lib/libc/citrus/modules/citrus_utf8.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: src/tests/lib/libc/locale
Date: Sat, 27 Jun 2026 20:43:43 +0000
Module Name: src
Committed By: riastradh
Date: Sat Jun 27 20:43:43 UTC 2026
Modified Files:
src/tests/lib/libc/locale: t_mbrtowc.c t_mbstowcs.c
Log Message:
t_mbstowcs: Mark UTF-8 test cases xfail.
mbrtowc fails to reject invalid (legacy 5/6-byte) UTF-8.
Need to split this test up into:
1. correctly decoding the valid inputs
2. correctly rejecting the invalid inputs
Also don't stop at the first failing test in t_mbrtowc; keep going to
test everything, for better diagnostics at the end in the test
report.
As with t_mbrtowc, this should be split into multiple parts, TBD.
PR lib/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences: Invalid sequence
To generate a diff of this commit:
cvs rdiff -u -r1.3 -r1.4 src/tests/lib/libc/locale/t_mbrtowc.c \
src/tests/lib/libc/locale/t_mbstowcs.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: src/tests/lib/libc/locale
Date: Tue, 30 Jun 2026 05:24:11 +0000
Module Name: src
Committed By: kre
Date: Tue Jun 30 05:24:11 UTC 2026
Modified Files:
src/tests/lib/libc/locale: t_mbrtowc.c t_mbstowcs.c
Log Message:
PR lib/60369 Update tests to match modern UTF-8
This just removes test cases using invalid (by current standards) UTF-8
sequences (in one case the test is modified to switch it from invalid to valid)
The XFAIL that was added is removed. ("removed" in all of this means
hashifd away).
There is, in this change, no attempt to fix either of the other very valid
concerns - actually testing invalid input to ensure it is rejected (would
need to be a whole new test case, the way they are currently structured is
not condusive to that - the input is simply known to be valid), nor having
the test continue to try the remaining cases if an invalid result is obtained
rather than simply abandoning ship at the first opportunity.
Also note that none of this really has anything whatever to do with the
PR, which had nothing at all to do with what is valid UTF-8 and what is
not, but merely when something that is to be treated as invalid is
detetected, that MUST be reported, the (libc, not test) code must not
go on to examine further bytes and end up reporting that more are needed
(even if more would be needed were the input valid). That is, when both
conditions exist, the error is reported, not the insufficient data.
To generate a diff of this commit:
cvs rdiff -u -r1.4 -r1.5 src/tests/lib/libc/locale/t_mbrtowc.c \
src/tests/lib/libc/locale/t_mbstowcs.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: [netbsd-11] src/tests/lib/libc/locale
Date: Fri, 3 Jul 2026 17:49:19 +0000
Module Name: src
Committed By: martin
Date: Fri Jul 3 17:49:19 UTC 2026
Modified Files:
src/tests/lib/libc/locale [netbsd-11]: t_mbrtowc.c t_mbstowcs.c
Log Message:
Additonally pull up following revision(s) (requested by wiz in ticket #339):
tests/lib/libc/locale/t_mbstowcs.c: revision 1.4
tests/lib/libc/locale/t_mbstowcs.c: revision 1.5
tests/lib/libc/locale/t_mbrtowc.c: revision 1.3
tests/lib/libc/locale/t_mbrtowc.c: revision 1.4
tests/lib/libc/locale/t_mbrtowc.c: revision 1.5
t_mbrtowc: Mark UTF-8 test cases xfail.
mbrtowc previously failed to reject invalid (legacy 5/6-byte) UTF-8,
so it accepted this test case. Now it rejects this test case,
because the test case itself is broken.
Need to split this test up into:
1. correctly decoding the valid inputs
2. correctly rejecting the invalid inputs
But for now marking the test case xfail is an adequate approximation
to the more complicated truth.
PR lib/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences: Invalid sequence
t_mbstowcs: Mark UTF-8 test cases xfail.
mbrtowc fails to reject invalid (legacy 5/6-byte) UTF-8.
Need to split this test up into:
1. correctly decoding the valid inputs
2. correctly rejecting the invalid inputs
Also don't stop at the first failing test in t_mbrtowc; keep going to
test everything, for better diagnostics at the end in the test
report.
As with t_mbrtowc, this should be split into multiple parts, TBD.
PR lib/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences: Invalid sequence
PR lib/60369 Update tests to match modern UTF-8
This just removes test cases using invalid (by current standards) UTF-8
sequences (in one case the test is modified to switch it from invalid to valid)
The XFAIL that was added is removed. ("removed" in all of this means
hashifd away).
There is, in this change, no attempt to fix either of the other very valid
concerns - actually testing invalid input to ensure it is rejected (would
need to be a whole new test case, the way they are currently structured is
not condusive to that - the input is simply known to be valid), nor having
the test continue to try the remaining cases if an invalid result is obtained
rather than simply abandoning ship at the first opportunity.
Also note that none of this really has anything whatever to do with the
PR, which had nothing at all to do with what is valid UTF-8 and what is
not, but merely when something that is to be treated as invalid is
detetected, that MUST be reported, the (libc, not test) code must not
go on to examine further bytes and end up reporting that more are needed
(even if more would be needed were the input valid). That is, when both
conditions exist, the error is reported, not the insufficient data.
To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.2.20.1 src/tests/lib/libc/locale/t_mbrtowc.c
cvs rdiff -u -r1.3 -r1.3.4.1 src/tests/lib/libc/locale/t_mbstowcs.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: src
Date: Sat, 4 Jul 2026 13:21:05 +0000
Module Name: src
Committed By: riastradh
Date: Sat Jul 4 13:21:05 UTC 2026
Modified Files:
src/lib/libc/citrus/modules: citrus_utf8.c
src/lib/libc/locale: c8rtomb.c
src/tests/lib/libc/locale: t_c8rtomb.c
Log Message:
libc: Fix two bugs in UTF-8 decoding and add exhaustive tests.
1. Despite the recent slew of changes, mbrtowc(3) in UTF-8 locales
would still fail to return EILSEQ at the first byte where it can,
because it would silently consume the first two bytes encoding a
surrogate code point and only reject it after consuming the third
byte.
2. The byte classification table for the compressed DFA used by
c8rtomb(3) was wrong for the bytes c0-c1 and f5-ff. Somehow the
program I used, over a decade ago, to generate the compressed DFA
and classification table got those, and only those, wrong, and I
can't find that program now, so I'll just have to correct this by
hand. Class 10 was missing from the table (but class 11 was not)
and is obviously the right class for the always-invalid c0-c1 and
f5-ff because all states transition to UTF8_REJECT (=96) on class
10, and the same is not true for any other class.
Now mbrtowc(3) in LC_CTYPE=C.UTF_8 is checked against c8rtomb(3)
systematically to verify they agree on all possible inputs one byte
at a time. There are 4501261 distinct such inputs, including the
valid encodings of all Unicode scalar values, the invalid encodings
of Unicode surrogate code points, and other invalid encodings. We
stop at the first invalid byte, so it is not necessary to examine all
~four billion 4-byte strings. I considered randomly subsampling to
make this test take less time, but decided that would be too
confusing. For debugging purposes, you can run this new test with
`atf-run -v c8rtomb_all_faliures=yes' to show all failures rather
than just the first one; this produces megabytes of output with some
of the bugs we've had, so it's off by default.
PR standards/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences
To generate a diff of this commit:
cvs rdiff -u -r1.21 -r1.22 src/lib/libc/citrus/modules/citrus_utf8.c
cvs rdiff -u -r1.9 -r1.10 src/lib/libc/locale/c8rtomb.c
cvs rdiff -u -r1.7 -r1.8 src/tests/lib/libc/locale/t_c8rtomb.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: [netbsd-11] src
Date: Sat, 4 Jul 2026 15:44:51 +0000
Module Name: src
Committed By: martin
Date: Sat Jul 4 15:44:51 UTC 2026
Modified Files:
src/lib/libc/citrus/modules [netbsd-11]: citrus_utf8.c
src/lib/libc/locale [netbsd-11]: c8rtomb.c
src/tests/lib/libc/locale [netbsd-11]: t_c8rtomb.c
Log Message:
Additionally pull up following revision(s) (requested by riastradh in ticket #339):
tests/lib/libc/locale/t_c8rtomb.c: revision 1.8
lib/libc/locale/c8rtomb.c: revision 1.10
lib/libc/citrus/modules/citrus_utf8.c: revision 1.20
lib/libc/citrus/modules/citrus_utf8.c: revision 1.21
lib/libc/citrus/modules/citrus_utf8.c: revision 1.22
Be truly pedantic about UTF-8 encodings
If we're not going to be accepting "legacy" UTF-8
(5 and 6 byte encodings for code points >= 0x00200000 which the
standards don't allow, as they won't fit in UTF-16) then we
certainly should never be able to generate them, and even more
should certainly be pedantic about not allowing the various
forms of mis-coded strings for which there is no justification
but have been known to be used to attempt to violate security.
This, I believe, now enforces all the current restrictions, eg,
it will no longer be possible to encode ascii in 2 bytes (0xc0 '.')
and similar, the shortest legal encoding is all that will be
accepted (and all that will be generated, but that was always true).
It is quite possible that this will break things, probably many
tests, as now random garbage won't be accepted as valid, things
must be properly encodedd.
mbrtowc() fix a stupid typo in the previous version.
No idea how I managed to miss this previously. This update should
make at least some of the ATF tests (and other stuff) which failed
after the previous change start working again.
libc: Fix two bugs in UTF-8 decoding and add exhaustive tests.
1. Despite the recent slew of changes, mbrtowc(3) in UTF-8 locales
would still fail to return EILSEQ at the first byte where it can,
because it would silently consume the first two bytes encoding a
surrogate code point and only reject it after consuming the third
byte.
2. The byte classification table for the compressed DFA used by
c8rtomb(3) was wrong for the bytes c0-c1 and f5-ff. Somehow the
program I used, over a decade ago, to generate the compressed DFA
and classification table got those, and only those, wrong, and I
can't find that program now, so I'll just have to correct this by
hand. Class 10 was missing from the table (but class 11 was not)
and is obviously the right class for the always-invalid c0-c1 and
f5-ff because all states transition to UTF8_REJECT (=96) on class
10, and the same is not true for any other class.
Now mbrtowc(3) in LC_CTYPE=C.UTF_8 is checked against c8rtomb(3)
systematically to verify they agree on all possible inputs one byte
at a time. There are 4501261 distinct such inputs, including the
valid encodings of all Unicode scalar values, the invalid encodings
of Unicode surrogate code points, and other invalid encodings. We
stop at the first invalid byte, so it is not necessary to examine all
~four billion 4-byte strings. I considered randomly subsampling to
make this test take less time, but decided that would be too
confusing. For debugging purposes, you can run this new test with
`atf-run -v c8rtomb_all_faliures=yes' to show all failures rather
than just the first one; this produces megabytes of output with some
of the bugs we've had, so it's off by default.
PR standards/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences
To generate a diff of this commit:
cvs rdiff -u -r1.18.42.1 -r1.18.42.2 \
src/lib/libc/citrus/modules/citrus_utf8.c
cvs rdiff -u -r1.9 -r1.9.4.1 src/lib/libc/locale/c8rtomb.c
cvs rdiff -u -r1.7 -r1.7.4.1 src/tests/lib/libc/locale/t_c8rtomb.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60369 CVS commit: [netbsd-10] src/tests/lib/libc/locale
Date: Fri, 11 Sep 2026 16:14:24 +0000
Module Name: src
Committed By: martin
Date: Fri Sep 11 16:14:24 UTC 2026
Modified Files:
src/tests/lib/libc/locale [netbsd-10]: t_mbrtowc.c t_mbstowcs.c
Log Message:
Pull up following revision(s) (requested by kre in ticket #1336):
tests/lib/libc/locale/t_mbstowcs.c: revision 1.4
tests/lib/libc/locale/t_mbstowcs.c: revision 1.5
tests/lib/libc/locale/t_mbrtowc.c: revision 1.3
tests/lib/libc/locale/t_mbrtowc.c: revision 1.4
tests/lib/libc/locale/t_mbrtowc.c: revision 1.5
t_mbrtowc: Mark UTF-8 test cases xfail.
mbrtowc previously failed to reject invalid (legacy 5/6-byte) UTF-8,
so it accepted this test case. Now it rejects this test case,
because the test case itself is broken.
Need to split this test up into:
1. correctly decoding the valid inputs
2. correctly rejecting the invalid inputs
But for now marking the test case xfail is an adequate approximation
to the more complicated truth.
PR lib/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences: Invalid sequence
t_mbstowcs: Mark UTF-8 test cases xfail.
mbrtowc fails to reject invalid (legacy 5/6-byte) UTF-8.
Need to split this test up into:
1. correctly decoding the valid inputs
2. correctly rejecting the invalid inputs
Also don't stop at the first failing test in t_mbrtowc; keep going to
test everything, for better diagnostics at the end in the test
report.
As with t_mbrtowc, this should be split into multiple parts, TBD.
PR lib/60369: mbrtowc, mbrlen have wrong return value for some
invalid byte sequences: Invalid sequence
PR lib/60369 Update tests to match modern UTF-8
This just removes test cases using invalid (by current standards) UTF-8
sequences (in one case the test is modified to switch it from invalid to valid)
The XFAIL that was added is removed. ("removed" in all of this means
hashifd away).
There is, in this change, no attempt to fix either of the other very valid
concerns - actually testing invalid input to ensure it is rejected (would
need to be a whole new test case, the way they are currently structured is
not condusive to that - the input is simply known to be valid), nor having
the test continue to try the remaining cases if an invalid result is obtained
rather than simply abandoning ship at the first opportunity.
Also note that none of this really has anything whatever to do with the
PR, which had nothing at all to do with what is valid UTF-8 and what is
not, but merely when something that is to be treated as invalid is
detetected, that MUST be reported, the (libc, not test) code must not
go on to examine further bytes and end up reporting that more are needed
(even if more would be needed were the input valid). That is, when both
conditions exist, the error is reported, not the insufficient data.
To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.2.16.1 src/tests/lib/libc/locale/t_mbrtowc.c
cvs rdiff -u -r1.2.16.1 -r1.2.16.2 src/tests/lib/libc/locale/t_mbstowcs.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home