NetBSD Problem Report #60776
From martin@duskware.de Wed Sep 23 15:05:56 2026
Return-Path: <martin@duskware.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits)
client-signature RSA-PSS (2048 bits))
(Client CN "mail.netbsd.org", Issuer "YR1" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 7D56E1A923B
for <gnats-bugs@gnats.NetBSD.org>; Wed, 23 Sep 2026 15:05:56 +0000 (UTC)
Message-Id: <20260923150552.3AF1FE2000C@mail.duskware.de>
Date: Wed, 23 Sep 2026 17:05:47 +0200 (CEST)
From: martin@NetBSD.org
Reply-To: martin@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: reproducable assertion failure with today's current
X-Send-Pr-Version: 3.95
X-From4GNATS: "martin@NetBSD.org via gnats" <gnats-admin@NetBSD.org>
>Number: 60776
>Category: kern
>Synopsis: reproducable assertion failure with today's current
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Sep 23 15:10:01 +0000 2026
>Last-Modified: Thu Sep 24 02:30:02 +0000 2026
>Originator: Martin Husemann
>Release: NetBSD 11.99.8
>Organization:
The NetBSD Foundation, Inc.
>Environment:
System: NetBSD seven-days-to-the-wolves.aprisoft.de 11.99.8 NetBSD 11.99.8 (GENERIC) #892: Wed Sep 23 10:29:43 CEST 2026 martin@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
Today's amd64 kernel reproducably crashes for me when running build.sh.
[ 4043.4787057] panic: kernel diagnostic assertion "lwp_eprio(l) >= ts->ts_eprio" failed: file "../../../../kern/kern_turnstile.c", line 356
[ 4043.6286910] cpu1: Begin traceback...
[ 4043.6686907] vpanic() at netbsd:vpanic+0x189
[ 4043.7186903] kern_assert() at netbsd:kern_assert+0x4b
[ 4043.7786898] turnstile_wakeup() at netbsd:turnstile_wakeup+0x2c5
[ 4043.8586891] ubc_alloc.constprop.0() at netbsd:ubc_alloc.constprop.0+0x680
[ 4043.9386885] ubc_uiomove() at netbsd:ubc_uiomove+0x108
[ 4043.9986879] ffs_write() at netbsd:ffs_write+0x297
[ 4044.0586874] VOP_WRITE() at netbsd:VOP_WRITE+0x78
[ 4044.1086870] vn_write() at netbsd:vn_write+0xce
[ 4044.1686865] dofilewrite() at netbsd:dofilewrite+0x80
[ 4044.2286860] sys_write() at netbsd:sys_write+0x49
[ 4044.2786855] syscall() at netbsd:syscall+0x9d
[ 4044.3286851] --- syscall (number 4) ---
[ 4044.3786847] netbsd:syscall+0x9d:
[ 4044.4186843] cpu1: End traceback...
[ 4044.4586840] fatal breakpoint trap in supervisor mode
[ 4044.5186835] trap type 1 code 0 rip 0xffffffff8023541d cs 0x8 rflags 0x202 cr2 0xffffcc0940a31000 ilevel 0x8 rsp 0xffffcc0954238a90
[ 4044.6586823] curlwp 0xffff8d7fd9359400 pid 1596.1596 lowest kstack 0xffffcc09542342c0
Stopped in pid 1596.1596 (nbmkdep) at netbsd:breakpoint+0x5: leave
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x189
kern_assert() at netbsd:kern_assert+0x4b
turnstile_wakeup() at netbsd:turnstile_wakeup+0x2c5
ubc_alloc.constprop.0() at netbsd:ubc_alloc.constprop.0+0x680
ubc_uiomove() at netbsd:ubc_uiomove+0x108
ffs_write() at netbsd:ffs_write+0x297
VOP_WRITE() at netbsd:VOP_WRITE+0x78
vn_write() at netbsd:vn_write+0xce
dofilewrite() at netbsd:dofilewrite+0x80
sys_write() at netbsd:sys_write+0x49
syscall() at netbsd:syscall+0x9d
--- syscall (number 4) ---
netbsd:syscall+0x9d:
ds 4000
es 1
fs 180
gs 8a40
rdi 0
rsi 3f8
rbp ffffcc0954238a90
rbx ffffffff816368a8 ostype+0x7a008
rdx 1
rcx 0
rax 1
r8 3
r9 0
r10 0
r11 ffffffff81613182 ostype+0x568e2
r12 ffffcc0954238ad8
r13 104
r14 0
r15 ffff8d7fd9359400
rip ffffffff8023541d breakpoint+0x5
cs 8
rflags 202
rsp ffffcc0954238a90
ss 10
netbsd:breakpoint+0x5: leave
db{1}> show all locks
PID LID CPU STRUCT LWP * OWNER BITS LOCK
5705 5705 0 ffff8d803cedd000 0 0 netbsd:kernel_pmap_store+0x180
21887 21887 3 ffff8d801ec67800 ffff8d7fd9359400 7 ffff8d7f904f4400
23930>23930 2 ffff8d8057b39c00 ffff8d7fd9359400 7 ffff8d7f904f4400
0 > 0 3 ffffffff81a8c100 10700 1 netbsd:turnstile_locks+0xd00
db{1}> show all locks/t
PID LID CPU STRUCT LWP * OWNER BITS LOCK
5705 5705 0 ffff8d803cedd000 0 0 netbsd:kernel_pmap_st
ore+0x180
21887 21887 3 ffff8d801ec67800 ffff8d7fd9359400 7 ffff8d7f904f4400
trace: pid 1596 lid 1596 at 0xffffcc0954238a90
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x189
kern_assert() at netbsd:kern_assert+0x4b
turnstile_wakeup() at netbsd:turnstile_wakeup+0x2c5
ubc_alloc.constprop.0() at netbsd:ubc_alloc.constprop.0+0x680
ubc_uiomove() at netbsd:ubc_uiomove+0x108
ffs_write() at netbsd:ffs_write+0x297
VOP_WRITE() at netbsd:VOP_WRITE+0x78
vn_write() at netbsd:vn_write+0xce
dofilewrite() at netbsd:dofilewrite+0x80
sys_write() at netbsd:sys_write+0x49
syscall() at netbsd:syscall+0x9d
--- syscall (number 4) ---
netbsd:syscall+0x9d:
23930>23930 2 ffff8d8057b39c00 ffff8d7fd9359400 7 ffff8d7f904f4400
trace: pid 1596 lid 1596 at 0xffffcc0954238a90
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x189
kern_assert() at netbsd:kern_assert+0x4b
turnstile_wakeup() at netbsd:turnstile_wakeup+0x2c5
ubc_alloc.constprop.0() at netbsd:ubc_alloc.constprop.0+0x680
ubc_uiomove() at netbsd:ubc_uiomove+0x108
ffs_write() at netbsd:ffs_write+0x297
VOP_WRITE() at netbsd:VOP_WRITE+0x78
vn_write() at netbsd:vn_write+0xce
dofilewrite() at netbsd:dofilewrite+0x80
sys_write() at netbsd:sys_write+0x49
syscall() at netbsd:syscall+0x9d
--- syscall (number 4) ---
netbsd:syscall+0x9d:
0 > 0 3 ffffffff81a8c100 10700 1 netbsd:turnstile_locks+0xd00
db{1}> show all tstiles/t
PID LID COMMAND WAITING-FOR TYPE WAIT-CHANNEL
5705 5705 aarch64--netbsd- 0 mutex netbsd:kernel_pmap_store+0x180
7494 7494 nbtsort 0 cv ffff8d8072484108
4482 4482 sh 0 cv ffff8d7fc09ae758
69 69 sh 0 cv ffff8d8033fa58d8
5756 5756 sh 0 cv ffff8d7f911d4a08
2201 2201 sh 0 cv ffff8d8033fa5c58
21887 21887 aarch64--netbsd- ffff8d7fd9359400 rwlock ffff8d7f904f4400
trace: pid 1596 lid 1596 at 0xffffcc0954238a90
breakpoint() at netbsd:breakpoint+0x5
vpanic() at netbsd:vpanic+0x189
kern_assert() at netbsd:kern_assert+0x4b
turnstile_wakeup() at netbsd:turnstile_wakeup+0x2c5
ubc_alloc.constprop.0() at netbsd:ubc_alloc.constprop.0+0x680
ubc_uiomove() at netbsd:ubc_uiomove+0x108
ffs_write() at netbsd:ffs_write+0x297
VOP_WRITE() at netbsd:VOP_WRITE+0x78
vn_write() at netbsd:vn_write+0xce
dofilewrite() at netbsd:dofilewrite+0x80
sys_write() at netbsd:sys_write+0x49
syscall() at netbsd:syscall+0x9d
--- syscall (number 4) ---
netbsd:syscall+0x9d:
4481 4481 aarch64--netbsd- 0 cv ffff8d8125262458
16610 16610 sh 0 cv ffff8d7ff8775bd8
28753 28753 nbmake 0 select ffff8d867d782b00
4247 4247 sh 0 cv ffff8d7fbf7dec58
10038 10038 nbmake 0 select ffff8d867d782f00
4650 4650 sh 0 cv ffff8d7ff9ad7118
9118 9118 nbmake 0 select ffff8d867d782f00
27900 27900 nbmake 0 select ffff8d8682b6f400
19448 19448 nbmake 0 select ffff8d867d782b00
18296 18296 sh 0 cv ffff8d806a077418
15401 15401 sh 0 cv ffff8d7fdc4a9b58
29755 29755 sh 0 cv ffff8d8165250c58
22820 22820 nbmake 0 select ffff8d8682b6f400
13988 13988 sh 0 cv ffff8d8127992498
15682 15682 nbmake 0 select ffff8d867d782700
16365 16365 sh 0 cv ffff8d7f953c5098
15681 15681 nbmake 0 select ffff8d867d782b00
1515 1515 sh 0 cv ffff8d7fa3559818
3807 3807 nbmake 0 select ffff8d867d782700
1374 1374 sh 0 cv ffff8d7f9ae920d8
1504 1504 nbmake 0 select ffff8d867d782700
656 656 sh 0 cv ffff8d7f9ae92b58
653 653 sh 0 cv ffff8d7f98785898
648 648 tcsh 0 cv ffff8d7f91e1d588
644 644 rxvt 0 select ffff8d867d782b00
339 339 ssh-agent 0 select ffff8d867d782700
338 338 tcsh 0 kpause ffff8d7f9af61000
336 336 tcsh 0 kpause ffff8d7f9069f400
335 335 rxvt 0 select ffff8d867d782b00
333 333 tcsh 0 kpause ffff8d7f90f4d800
332 332 sshd-session 0 select ffff8d867d782700
330 330 sshd-session 0 select ffff8d8682b6f400
1850 1850 getty 0 kpause ffff8d7f95ca7c00
1721 1721 getty 0 kpause ffff8d7f99ecdc00
1592 1592 getty 0 kpause ffff8d7f90f4dc00
311 311 getty 0 cv ffff8d7f8f513288
302 302 cron 0 kpause ffff8d7f98632000
2598 2598 inetd 0 cv ffff8d7f950b3b20
401 401 sshd 0 select ffff8d867d782700
394 394 upsmon 0 kpause ffff8d7f9a1c7000
393 393 upsmon 0 cv ffff8d7f911d4808
390 390 upsd 0 select ffff8d867d782700
2485 2485 powerd 0 cv ffff8d7f97d5aca0
2303 2303 ntpd 0 kpause ffff8d7f99ecd000
1917 1917 rpc.lockd 0 select ffff8d867d782700
1586 1722 nfsd 0 cv ffff8d7f93fef720
1586 1568 nfsd 0 cv ffff8d7f93fef3e0
1586 1337 nfsd 0 cv ffff8d7f9a0bdb20
1586 1439 nfsd 0 cv ffff8d7f913dd1e0
1586 1586 nfsd 0 select ffff8d867d782f00
1401 1401 mountd 0 select ffff8d8682b6f400
1087 1087 rpcbind 0 select ffff8d867d782700
1085 1085 syslogd 0 cv ffff8d7f91922fa0
702 702 dhcpcd 0 cv ffff8d7f90d5c520
820 820 dhcpcd 0 cv ffff8d7f9925b3a0
817 817 dhcpcd 0 cv ffff8d7f91496e20
819 819 dhcpcd 0 cv ffff8d7f9a02aca0
1 1 init 0 cv ffff8d7f90775058
0 250 system 0 cv ffff8d7f90d10c48
0 175 system 0 cv ffff8d7f90c9f208
0 174 system 0 cv ffff8d7f909de408
0 173 system 0 cv ffff8d7f909de308
0 172 system 0 cv ffff8d7f907d5d08
0 171 system 0 kpause ffff8d7f90847c00
0 170 system 0 cv netbsd:uvmpd_pool_drain_cv
0 169 system 0 cv ffff8d7f907d5c08
0 168 system 0 sleep netbsd:uvm+0x188
0 165 system 0 cv ffff8d7f90822448
0 163 system 0 cv ffff8d7f8f3b1ac0
0 161 system 0 cv ffffcc0042f5eba0
0 222 system 0 cv ffff8d7f90722b88
0 221 system 0 cv ffff8d7f906c6bc8
0 220 system 0 cv netbsd:unp_thread_cv
0 218 system 0 cv ffff8d7f906f5f08
0 217 system 0 cv ffff8d7f906f5ec8
0 216 system 0 cv ffff8d7f906f5e88
0 215 system 0 cv ffff8d7f906f5e48
0 214 system 0 cv ffff8d7f906c6948
0 213 system 0 cv ffff8d7f906f5b08
0 212 system 0 cv ffff8d7f906f5ac8
0 211 system 0 cv ffff8d7f906f5a88
0 210 system 0 cv ffff8d7f906f5a48
0 209 system 0 cv ffff8d7f906f5708
0 208 system 0 cv ffff8d7f906f56c8
0 207 system 0 cv ffff8d7f906f5688
0 206 system 0 cv ffff8d7f906f5648
0 205 system 0 cv ffff8d7f90623b08
0 204 system 0 cv ffff8d7f90623ac8
0 203 system 0 cv ffff8d7f90623a88
0 202 system 0 cv ffff8d7f90623a48
0 200 system 0 cv ffff8d7f906c6088
0 199 system 0 cv ffff8d7f90651f08
0 198 system 0 cv ffff8d7f90651dc8
0 197 system 0 cv ffff8d7f90651c88
0 196 system 0 cv ffff8d7f90651b48
0 195 system 0 cv ffff8d7f90651a08
0 194 system 0 cv ffff8d7f906513c8
0 193 system 0 cv ffff8d7f8f2b2978
0 192 system 0 cv ffff8d7f8f2b24c0
0 117 system 0 cv netbsd:entropy_global+0xf0
0 116 system 0 cv ffffcc0042f60ba0
0 115 system 0 cv ffffcc0042f603b0
0 114 system 0 cv ffffcc0042f5fbc0
0 113 system 0 cv ffffcc0042f5f3d0
0 112 system 0 cv ffffcc0042f5ebe0
0 111 system 0 cv ffffcc0042f5e3f0
0 110 system 0 cv netbsd:usb_taskq+0x58
0 109 system 0 cv netbsd:usb_taskq+0x18
0 108 system 0 cv ffff8d7f8f8ba108
0 107 system 0 cv ffffcc0937a6d010
0 101 system 0 cv ffffcc0937a32010
0 30 system 0 cv ffffcc0937354010
0 24 system 0 cv ffff8d8682b6fe08
0 23 system 0 cv ffff8d8682b6fd08
0 22 system 0 cv ffff8d8682b6fc08
0 21 system 0 cv ffff8d8682b6fb08
0 20 system 0 cv ffff8d8682b6fa08
0 19 system 0 cv netbsd:gc+0x8
0 18 system 0 cv netbsd:sysmon_task_queue_cv
0 17 system 0 cv ffff8d8681b24948
0 16 system 0 cv ffff8d8681b24808
0 15 system 0 cv netbsd:pendfree_thread_cv
0 14 system 0 cv ffff8d8681b246c8
0 13 system 0 cv ffff8d8681b24588
0 12 system 0 cv ffff8d8681b24448
0 11 system 0 cv ffff8d8681b24308
0 10 system 0 cv ffff8d8681b241c8
0 9 system 0 cv ffff8d8682b6f728
0 8 system 0 cv netbsd:module_thread_cv
0 7 system 0 cv netbsd:cpu_info_primary+0x10
>How-To-Repeat:
Run build.sh with enough concurrency?
>Fix:
n/a
>Audit-Trail:
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/60776 CVS commit: src/sys/kern
Date: Wed, 23 Sep 2026 19:07:40 +0000
Module Name: src
Committed By: riastradh
Date: Wed Sep 23 19:07:40 UTC 2026
Modified Files:
src/sys/kern: kern_turnstile.c
Log Message:
Revert "turnstile(9): Fix priority inheritance if trylock requires restart."
This was originally done in an attempt to resolve the issue described
in:
PR kern/60739: Turnstile priority inheritance short-circuits walking
of waiter chain when trylock fails
But it may have broken other things:
PR kern/60776: reproducable assertion failure with today's current
> Today's amd64 kernel reproducably crashes for me when running build.sh.
>
> [ 4043.4787057] panic: kernel diagnostic assertion "lwp_eprio(l) >= ts->ts_eprio" failed: file "../../../../kern/kern_turnstile.c", line 356
> [ 4043.6286910] cpu1: Begin traceback...
> [ 4043.6686907] vpanic() at netbsd:vpanic+0x189
> [ 4043.7186903] kern_assert() at netbsd:kern_assert+0x4b
> [ 4043.7786898] turnstile_wakeup() at netbsd:turnstile_wakeup+0x2c5
> [ 4043.8586891] ubc_alloc.constprop.0() at netbsd:ubc_alloc.constprop.0+0x680
> [ 4043.9386885] ubc_uiomove() at netbsd:ubc_uiomove+0x108
> [ 4043.9986879] ffs_write() at netbsd:ffs_write+0x297
> [ 4044.0586874] VOP_WRITE() at netbsd:VOP_WRITE+0x78
> [ 4044.1086870] vn_write() at netbsd:vn_write+0xce
> [ 4044.1686865] dofilewrite() at netbsd:dofilewrite+0x80
> [ 4044.2286860] sys_write() at netbsd:sys_write+0x49
> [ 4044.2786855] syscall() at netbsd:syscall+0x9d
So let's back it out for now pending further analysis.
To generate a diff of this commit:
cvs rdiff -u -r1.57 -r1.58 src/sys/kern/kern_turnstile.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: =?UTF-8?Q?Lo=C3=AFc_Gregoire?= <loicgre@gmail.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/60776
Date: Wed, 23 Sep 2026 22:25:58 -0400
Hi,
I submitted the original bug report.
I think I have figured out roughly where the bug is:
/*
* Lend our priority to the 'owner' LWP.
*
* Update lenders info for turnstile_unlendpri.
*/
ts = l->l_ts;
KASSERT(ts->ts_inheritor == owner || ts->ts_inheritor == NULL);
if (ts->ts_inheritor == NULL) {
ts->ts_inheritor = owner;
ts->ts_eprio = prio;
SLIST_INSERT_HEAD(&owner->l_pi_lenders, ts, ts_pichain);
lwp_lendpri(owner, prio);
} else if (prio > ts->ts_eprio) {
ts->ts_eprio = prio;
lwp_lendpri(owner, prio);
}
lwp_lendpri doesn't check whether or not the priority is raised, so
what happens here is that a waiter with a lower priority than the
owner donates its priority to the owner incorrectly.
Intuitively this whole block should be wrapped in
if (prio > lwp_eprio(owner))
As a minimal fix.
However I think this introduces another kind of bug (which was also
present before) where a lower-priority thread is not inserted in
pi_lenders, which means the following situation could occur:
A inherits prio 100 from X
B of priority 50 waits on another lock held by A on turnstile Y
Since A is already of priority > 50, B doesn't get added to lenders.
A releases the lock backed by X, priority is returned
A's priority now could be under 50 even though it still holds Y!
So the real fix would be doing something like this:
ts = l->l_ts;
KASSERT(ts->ts_inheritor == owner || ts->ts_inheritor == NULL);
if (ts->ts_inheritor == NULL) {
ts->ts_inheritor = owner;
ts->ts_eprio = prio;
SLIST_INSERT_HEAD(&owner->l_pi_lenders, ts, ts_pichain);
} else if (prio > ts->ts_eprio) {
ts->ts_eprio = prio;
}
if (prio > owner->l_inheritedprio)
lwp_lendpri(owner, prio);
Instead.
As a bonus I think prio should be set to the eprio of the new owner on each hop.
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home