NetBSD Problem Report #58425
From www@netbsd.org Sat Jul 13 19:53:17 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
client-signature RSA-PSS (2048 bits) client-digest SHA256)
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id CE65C1A923A
for <gnats-bugs@gnats.NetBSD.org>; Sat, 13 Jul 2024 19:53:16 +0000 (UTC)
Message-Id: <20240713195315.A2E021A923B@mollari.NetBSD.org>
Date: Sat, 13 Jul 2024 19:53:15 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: negation of pid or pgid is UB for INT_MIN
X-Send-Pr-Version: www-1.0
>Number: 58425
>Category: kern
>Synopsis: negation of pid or pgid is UB for INT_MIN
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: kre
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Jul 13 19:55:00 +0000 2024
>Closed-Date: Wed Aug 07 11:47:30 +0000 2024
>Last-Modified: Wed Aug 07 11:47:30 +0000 2024
>Originator: Taylor R Campbell
>Release: current, 10, 9, ...
>Organization:
The -PGID Foundation
>Environment:
>Description:
Certain syscalls such as kill(2) and ktrace(2) that take arguments of type pid_t accept a negative value and take its absolute value to be a process group id.
When the negative value is INT_MIN, however, negation is undefined.
We need to systematically avoid these paths, and it looks like we currently don't. This could be a theoretical issue or it could be a practical issue, but the UB is a bug either way.
>How-To-Repeat:
code inspection
>Fix:
Yes, please!
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: kern-bug-people->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Sun, 14 Jul 2024 03:27:30 +0000
Responsible-Changed-Why:
I am looking into this PR
From: Robert Elz <kre@munnari.OZ.AU>
To: netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Sun, 14 Jul 2024 10:52:19 +0700
Date: Sat, 13 Jul 2024 19:55:00 +0000 (UTC)
From: campbell+netbsd@mumble.net
Message-ID: <20240713195500.A81291A923C@mollari.NetBSD.org>
| This could be a theoretical issue or it could be a practical issue,
I believe that unless we have some architecture which traps when
executing -INT_MIN rather than just generating INT_MIN as the result
(which is conceivable, but I doubt anything supported will do that)
then I suspect this is really a non-issue for practical purposes.
Even if the architecture were to generate a random value (which UB
allows) it would not be truly harmful ... an application is allowed
to send any random value to these sys calls, and things must work,
that the random value was generated by this operation, rather than
some other way really doesn't matter.
It is possible for an application to do kill(INT_MIN, sig) and pass
through INT_MIN as the pid, which being negative, is treated as a
pgrp, negated. But assuming -INT_MIN actually becomes -INT_MIN
or something else, not a trap, that would be harmless, it simply results
in a search for something different -- if it remains INT_MIN, the normal
case for most 2's complement architectures, or anything else negative,
it cannot be found, as all pids (and hence all pgrp ids) are positive
(or 0 for the kernel) a negative value (any negative value) can never
match (that's how this hack was invented). So the result would have
just been ESRCH anyway I believe.
I am however in the process of validating the value, and not performing
the undefined operation. Note, that as (ktrace & ptrace excepted, where
the pid arg is actually defined to be an int) generally dealing with pid_t
values, which we cannot assume are int ... it could be short (and once
was, even earlier I think it might have been char, not that the name
"pid_t" existed then, but that was a long long time ago) which would
never give a problem, it gets promoted to int for both the -pid
operation (and so could never be INT_MIN) and to pass as a parameter.
Alternatively, pid_t might be wider than an int (could be long)
in which case the value might be < INT_MIN, so I am making (most of)
the tests check that the value is > INT_MIN not just != INT_MIN.
My test build looks to have succeeded (on amd64) - currently building
the sets - I will do an i386 build, and then commit the changes, and
after a few days, see about pullups to -10 and -9.
The only system place outside the kernel I could see was in the killpg()
function (compat function, for old BSD code, posix defines the kill(-pid)
method) - that one will be fixed as well.
Random applications or other libraries which might accept a pid, and
negate it, will need to wait for some other time (if there are any).
kre
ps: Taylor, if you feel we need ATF test cases for this, by all means,
go ahead.
From: matthew green <mrg@eterna23.net>
To: Robert Elz <kre@munnari.OZ.AU>
Cc: netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Sun, 14 Jul 2024 14:57:10 +1000
> | This could be a theoretical issue or it could be a practical issue,
>
> I believe that unless we have some architecture which traps when
> executing -INT_MIN rather than just generating INT_MIN as the result
this may not be relevant, but it triggered me. :-)
i recently was dealing with a trap related to exactly this.
i was running atf on m68k virt68k qemu and it crashed the emulator
itself. turns out -INT_MAX/-1 traps on x86.
i have this uncommited/unsubmitted patch for qemu:
https://www.netbsd.org/~mrg/patch-target_m68k_op_helper.c
.mrg.
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58425 CVS commit: src
Date: Sun, 14 Jul 2024 05:10:40 +0000
Module Name: src
Committed By: kre
Date: Sun Jul 14 05:10:40 UTC 2024
Modified Files:
src/lib/libc/compat-43: killpg.c
src/sys/kern: kern_descrip.c kern_ktrace.c kern_proc.c sys_sig.c tty.c
Log Message:
PR kern/58425 -- Disallow INT_MIN as a (negative) pid arg.
Since -INT_MIN is undefined, and the point of negative pid args is
to negate them, and use the result as a pgrp id instead, we need
to avoid accidentally negating INT_MIN.
Since pid_t is just an integral type, of unspecified width, when
testing pid_t value test for <= INT_MIN (or > INT_MIN sometimes)
rather than == INT_MIN. When testing int values, just == INT_MIN
is all that is needed, < INT_MIN cannot occur.
XXX pullup -9, -10
To generate a diff of this commit:
cvs rdiff -u -r1.9 -r1.10 src/lib/libc/compat-43/killpg.c
cvs rdiff -u -r1.262 -r1.263 src/sys/kern/kern_descrip.c
cvs rdiff -u -r1.184 -r1.185 src/sys/kern/kern_ktrace.c
cvs rdiff -u -r1.275 -r1.276 src/sys/kern/kern_proc.c
cvs rdiff -u -r1.57 -r1.58 src/sys/kern/sys_sig.c
cvs rdiff -u -r1.312 -r1.313 src/sys/kern/tty.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Robert Elz <kre@munnari.OZ.AU>
To: matthew green <mrg@eterna23.net>
Cc: netbsd-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Sun, 14 Jul 2024 13:26:53 +0700
Date: Sun, 14 Jul 2024 14:57:10 +1000
From: matthew green <mrg@eterna23.net>
Message-ID: <22479.1720933030@splode.eterna23.net>
| i recently was dealing with a trap related to exactly this.
|
| i was running atf on m68k virt68k qemu and it crashed the emulator
| itself. turns out -INT_MAX/-1 traps on x86.
I think you mean INT_MIN / -1 (INT_MIN != -INT_MAX - which is the whole
issue).
That would be a different problem than the pid one, which is just
taking -INT_MIN, no division, by anything, ever performed (I'm not
sure what a half of a pid_t would represent). Further, it should
be fixed in the kernel now.
| i have this uncommited/unsubmitted patch for qemu:
| https://www.netbsd.org/~mrg/patch-target_m68k_op_helper.c
Looks reasonable to me. Though I know nothing about qemu
kre
State-Changed-From-To: open->needs-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sun, 14 Jul 2024 12:05:53 +0000
State-Changed-Why:
Needs pullups to -9 and -10
From: Taylor R Campbell <riastradh@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: kre@NetBSD.org, gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Sun, 14 Jul 2024 13:55:11 +0000
> Date: Sun, 14 Jul 2024 10:52:19 +0700
> From: Robert Elz <kre@munnari.OZ.AU>
>
> Date: Sat, 13 Jul 2024 19:55:00 +0000 (UTC)
> From: campbell+netbsd@mumble.net
> Message-ID: <20240713195500.A81291A923C@mollari.NetBSD.org>
>
> | This could be a theoretical issue or it could be a practical issue,
>
> I believe that unless we have some architecture which traps when
> executing -INT_MIN rather than just generating INT_MIN as the result
> (which is conceivable, but I doubt anything supported will do that)
> then I suspect this is really a non-issue for practical purposes.
> Even if the architecture were to generate a random value (which UB
> allows) it would not be truly harmful ... an application is allowed
> to send any random value to these sys calls, and things must work,
> that the random value was generated by this operation, rather than
> some other way really doesn't matter.
The issue isn't so much that the architecture's SUB or NEG instruction
might trap or produce wild behaviour -- although in principle it
could, and not even in a pathological architecture design, just like
signed division INT_MIN/-1 traps on x86.
The more likely risk is that a _compiler_ may draw an inference from
an unprotected -pgid expression -- it may infer that along any control
flow paths that unconditionally pass through that expression, pgid
must not be INT_MIN, because if it were then the behaviour would be
undefined and we all know C programmers never write programs that
could lead to UB on the possible inputs.
So, for example, a compiler might optimize (a)
*p = -pgid;
return (pgid == INT_MIN);
into
*p = -pgid;
return false;
or it might optimize (b)
return (-pgid == pgid);
into
return (pgid == 0);
where you might expect it (or require it, with -fwrapv) to be
equivalent instead to
return (pgid == 0 || pgid == INT_MIN);
I haven't seen (a) in practice but (b) is easy to exhibit:
https://godbolt.org/z/rzxjf38aY
> I am however in the process of validating the value, and not performing
> the undefined operation. Note, that as (ktrace excepted, where the
> pid arg is actually defined to be an int) generally dealing with pid_t
> values, which we cannot assume are int ... it could be short (and once
> was, even earlier I think it might have been char, not that the name
> "pid_t" existed then, but that was a long long time ago) which would
> never give a problem, it gets promoted to int for both the -pid
> operation (and so could never be INT_MIN) and to pass as a parameter.
> Alternatively, pid_t might be wider than an int (could be long)
> in which case the value might be < INT_MIN, so I am making (most of)
> the tests check that the value is > INT_MIN not just != INT_MIN.
Thanks! Maybe we should use -PID_MAX as the limit instead of INT_MIN,
to make it more resilient to (unlikely but possible) changes of the
type and easier to audit?
...Except PID_MAX apparently isn't actually the max, once we cross
~16k processes. So maybe we should have a different name for the
_real_ maximum pid.
> ps: Taylor, if you feel we need ATF test cases for this, by all means,
> go ahead.
I added one for kill(INT_MIN, 0). We should add more tests for ktrace
and for FIOSETOWN on a tty (uses tty.c path) and a pipe or socket
(uses kern_proc.c fsetown path).
From: Jason Thorpe <thorpej@me.com>
To: gnats-bugs@netbsd.org
Cc: kre@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
campbell+netbsd@mumble.net,
mrg@netbsd.org
Subject: Re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Sun, 14 Jul 2024 07:33:56 -0700
> On Jul 13, 2024, at 10:00=E2=80=AFPM, matthew green <mrg@eterna23.net> =
wrote:
>=20
> i was running atf on m68k virt68k qemu and it crashed the emulator
> itself. turns out -INT_MAX/-1 traps on x86.
>=20
> i have this uncommited/unsubmitted patch for qemu:
>=20
> https://www.netbsd.org/~mrg/patch-target_m68k_op_helper.c
I have an outstanding open issue with qemu about this. I don=E2=80=99t =
know why it=E2=80=99s stalled. I haven=E2=80=99t had much time to spend =
on qemu stuff (long enough for my mailing list sub to time out), but =
I=E2=80=99ll make a point of circling back around to check on it. Mark =
had a patch for it, even.
-- thorpej
From: Jason Thorpe <thorpej@me.com>
To: gnats-bugs@netbsd.org
Cc: kre@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
campbell+netbsd@mumble.net,
mrg@netbsd.org
Subject: Re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Sun, 14 Jul 2024 07:39:00 -0700
> On Jul 14, 2024, at 7:33=E2=80=AFAM, Jason Thorpe <thorpej@me.com> =
wrote:
>=20
> I have an outstanding open issue with qemu about this. I don=E2=80=99t =
know why it=E2=80=99s stalled. I haven=E2=80=99t had much time to spend =
on qemu stuff (long enough for my mailing list sub to time out), but =
I=E2=80=99ll make a point of circling back around to check on it. Mark =
had a patch for it, even.
FWIW: https://gitlab.com/qemu-project/qemu/-/issues/2078
-- thorpej
From: matthew green <mrg@eterna23.net>
To: Jason Thorpe <thorpej@me.com>
Cc: kre@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
campbell+netbsd@mumble.net, gnats-bugs@netbsd.org
Subject: re: kern/58425: negation of pid or pgid is UB for INT_MIN
Date: Mon, 15 Jul 2024 08:19:41 +1000
> FWIW: https://gitlab.com/qemu-project/qemu/-/issues/2078
thanks.
the patch in this issue isn't complete -- there are at least 3
instances of the bug that my patch fixes.
i'll share my version there and see what happens.
.mrg.
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58425 CVS commit: src/tests/lib/libc/sys
Date: Mon, 15 Jul 2024 06:19:07 +0000
Module Name: src
Committed By: riastradh
Date: Mon Jul 15 06:19:07 UTC 2024
Modified Files:
src/tests/lib/libc/sys: t_kill.c
Log Message:
tests/lib/libc/sys/t_kill: Test kill(INT_MIN, ...) fails with ESRCH.
PR kern/58425
To generate a diff of this commit:
cvs rdiff -u -r1.1 -r1.2 src/tests/lib/libc/sys/t_kill.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: needs-pullups->pending-pullups
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sun, 04 Aug 2024 02:00:14 +0000
State-Changed-Why:
[pullup-10 #773]
[pullup-9 #1859]
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58425 CVS commit: [netbsd-10] src
Date: Wed, 7 Aug 2024 10:04:48 +0000
Module Name: src
Committed By: martin
Date: Wed Aug 7 10:04:48 UTC 2024
Modified Files:
src/lib/libc/compat-43 [netbsd-10]: killpg.c
src/sys/kern [netbsd-10]: kern_descrip.c kern_ktrace.c kern_proc.c
sys_sig.c tty.c
src/tests/lib/libc/sys [netbsd-10]: t_kill.c
Log Message:
Pull up following revision(s) (requested by kre in ticket #773):
sys/kern/kern_proc.c: revision 1.276
sys/kern/kern_ktrace.c: revision 1.185
sys/kern/sys_sig.c: revision 1.58
sys/kern/kern_descrip.c: revision 1.263
lib/libc/compat-43/killpg.c: revision 1.10
sys/kern/tty.c: revision 1.313
tests/lib/libc/sys/t_kill.c: revision 1.2
PR kern/58425 -- Disallow INT_MIN as a (negative) pid arg.
Since -INT_MIN is undefined, and to point of negative pid args is
to negate them, and use the result as a pgrp id instead, we need
to avoid accidentally negating INT_MIN.
Since pid_t is just an integral type, of unspecified width, when
testing pid_t value test for <= INT_MIN (or > INT_MIN sometimes)
rather than == INT_MIN. When testing int values, just == INT_MIN
is all that is needed, < INT_MIN cannot occur.
tests/lib/libc/sys/t_kill: Test kill(INT_MIN, ...) fails with ESRCH.
PR kern/58425
To generate a diff of this commit:
cvs rdiff -u -r1.9 -r1.9.14.1 src/lib/libc/compat-43/killpg.c
cvs rdiff -u -r1.251.10.1 -r1.251.10.2 src/sys/kern/kern_descrip.c
cvs rdiff -u -r1.182 -r1.182.4.1 src/sys/kern/kern_ktrace.c
cvs rdiff -u -r1.269 -r1.269.2.1 src/sys/kern/kern_proc.c
cvs rdiff -u -r1.56 -r1.56.4.1 src/sys/kern/sys_sig.c
cvs rdiff -u -r1.307 -r1.307.2.1 src/sys/kern/tty.c
cvs rdiff -u -r1.1 -r1.1.56.1 src/tests/lib/libc/sys/t_kill.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58425 CVS commit: [netbsd-9] src
Date: Wed, 7 Aug 2024 10:11:45 +0000
Module Name: src
Committed By: martin
Date: Wed Aug 7 10:11:45 UTC 2024
Modified Files:
src/lib/libc/compat-43 [netbsd-9]: killpg.c
src/sys/kern [netbsd-9]: kern_descrip.c kern_ktrace.c kern_proc.c
sys_sig.c tty.c
src/tests/lib/libc/sys [netbsd-9]: t_kill.c
Log Message:
Pull up following revision(s) (requested by kre in ticket #1859):
sys/kern/kern_proc.c: revision 1.276 (via patch)
sys/kern/kern_ktrace.c: revision 1.185 (via patch)
sys/kern/sys_sig.c: revision 1.58 (via patch)
sys/kern/kern_descrip.c: revision 1.263 (via patch)
lib/libc/compat-43/killpg.c: revision 1.10
sys/kern/tty.c: revision 1.313 (via patch)
tests/lib/libc/sys/t_kill.c: revision 1.2
PR kern/58425 -- Disallow INT_MIN as a (negative) pid arg.
Since -INT_MIN is undefined, and to point of negative pid args is
to negate them, and use the result as a pgrp id instead, we need
to avoid accidentally negating INT_MIN.
Since pid_t is just an integral type, of unspecified width, when
testing pid_t value test for <= INT_MIN (or > INT_MIN sometimes)
rather than == INT_MIN. When testing int values, just == INT_MIN
is all that is needed, < INT_MIN cannot occur.
tests/lib/libc/sys/t_kill: Test kill(INT_MIN, ...) fails with ESRCH.
PR kern/58425
To generate a diff of this commit:
cvs rdiff -u -r1.9 -r1.9.6.1 src/lib/libc/compat-43/killpg.c
cvs rdiff -u -r1.243 -r1.243.4.1 src/sys/kern/kern_descrip.c
cvs rdiff -u -r1.173 -r1.173.4.1 src/sys/kern/kern_ktrace.c
cvs rdiff -u -r1.233.2.1 -r1.233.2.2 src/sys/kern/kern_proc.c
cvs rdiff -u -r1.47.4.3 -r1.47.4.4 src/sys/kern/sys_sig.c
cvs rdiff -u -r1.281 -r1.281.4.1 src/sys/kern/tty.c
cvs rdiff -u -r1.1 -r1.1.44.1 src/tests/lib/libc/sys/t_kill.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: pending-pullups->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Wed, 07 Aug 2024 11:47:30 +0000
State-Changed-Why:
fixed in HEAD, pulled up to 9 and 10
(although more automatic tests would be nice)
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.