NetBSD Problem Report #60269

From www@netbsd.org  Sat May 16 04:23:06 2026
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.netbsd.org", Issuer "R13" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id A15411A9239
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 16 May 2026 04:23:06 +0000 (UTC)
Message-Id: <20260516042305.644611A923A@mollari.NetBSD.org>
Date: Sat, 16 May 2026 04:23:05 +0000 (UTC)
From: oshima-ya@yagoto-urayama.jp
Reply-To: oshima-ya@yagoto-urayama.jp
To: gnats-bugs@NetBSD.org
Subject: System hangs NetBSD/evbarm(aarch64) 11.0_RC4
X-Send-Pr-Version: www-1.0
X-From4GNATS: "oshima-ya@yagoto-urayama.jp via gnats" <gnats-admin@NetBSD.org>

>Number:         60269
>Notify-List:    riastradh@NetBSD.org
>Category:       port-evbarm
>Synopsis:       System hangs NetBSD/evbarm(aarch64) 11.0_RC4
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    port-evbarm-maintainer
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat May 16 04:25:00 +0000 2026
>Closed-Date:    
>Last-Modified:  Tue Jun 23 13:50:01 +0000 2026
>Originator:     Yasushi Oshima
>Release:        11.0_RC4
>Organization:
>Environment:
NetBSD nable 11.0_RC4 NetBSD 11.0_RC4 (GENERIC64) #0: Thu May 14 09:46:21 JST 2026  oshima@sudachi:/export/netbsd-11/20260514/obj/evbarm64/sys/arch/evbarm/compile/GENERIC64 evbarm
>Description:
After upgrading NetBSD/evbarm (aarch64)to 11.0_RC4 on my RasPi4, the system frequently hangs. This often occurs while building several applications from pkgsrc. As a result, a response of shell or any commands suddenly stops, and SSH sessions from the network are disconnected.

When the system hangs, it is possible to start ddb from the console using Ctrl-Alt-ESC. I found the process states by ps revealed several processes stuck on rwlock.
 (see https://x.com/oshimyja/status/2054966172375032207/photo/1)

For example, this frequently happens when building www/firefox. It can also occur during daily cron jobs. 
No such issues were observed in 11.0_RC2.
I don't know whether other architectures are affected.
At least, it doesn't happen on my amd64 11.0_RC4 environment.

This system is booted from a USB stick, and no crash dumps have been obtained.
>How-To-Repeat:
On NetBSD/evbarm(GENERIC64) 11.0_RC4, 
Build some pkgsrcs

>Fix:
None, but I check the source change history and found the following changes, then backout these and rebuild libc and related components, no further hangs have occurred so far:

https://mail-index.netbsd.org/source-changes/2026/05/07/msg161941.html

I think some of these changes is causing the issue. 

>Release-Note:

>Audit-Trail:
From: Nick Hudson <nick.hudson@gmx.co.uk>
To: gnats-bugs@netbsd.org,
 port-evbarm-maintainer@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org
Cc: 
Subject: Re: port-evbarm/60269: System hangs NetBSD/evbarm(aarch64) 11.0_RC4
Date: Sat, 16 May 2026 12:08:01 +0100

 On 16/05/2026 05:25, oshima-ya@yagoto-urayama.jp via gnats wrote:
 >> Number:         60269
 >> Category:       port-evbarm
 >> Synopsis:       System hangs NetBSD/evbarm(aarch64) 11.0_RC4
 >> Confidential:   no
 >> Severity:       serious
 >> Priority:       high
 >> Responsible:    port-evbarm-maintainer
 >> State:          open
 >> Class:          sw-bug
 >> Submitter-Id:   net
 >> Arrival-Date:   Sat May 16 04:25:00 +0000 2026
 >> Originator:     Yasushi Oshima
 >> Release:        11.0_RC4
 >> Organization:
 >> Environment:
 > NetBSD nable 11.0_RC4 NetBSD 11.0_RC4 (GENERIC64) #0: Thu May 14 =
 09:46:21 JST 2026  =
 oshima@sudachi:/export/netbsd-11/20260514/obj/evbarm64/sys/arch/evbarm/com=
 pile/GENERIC64 evbarm
 >> Description:
 > After upgrading NetBSD/evbarm (aarch64)to 11.0_RC4 on my RasPi4, the =
 system frequently hangs. This often occurs while building several =
 applications from pkgsrc. As a result, a response of shell or any =
 commands suddenly stops, and SSH sessions from the network are =
 disconnected.
 > When the system hangs, it is possible to start ddb from the console =
 using Ctrl-Alt-ESC. I found the process states by ps revealed several =
 processes stuck on rwlock.
 >  (see https://x.com/oshimyja/status/2054966172375032207/photo/1)
 > For example, this frequently happens when building www/firefox. It can =
 also occur during daily cron jobs.
 > No such issues were observed in 11.0_RC2.
 > I don't know whether other architectures are affected.
 > At least, it doesn't happen on my amd64 11.0_RC4 environment.
 > This system is booted from a USB stick, and no crash dumps have been =
 obtained.
 >> How-To-Repeat:
 > On NetBSD/evbarm(GENERIC64) 11.0_RC4,
 > Build some pkgsrcs
 >> Fix:
 > None, but I check the source change history and found the following =
 changes, then backout these and rebuild libc and related components, no =
 further hangs have occurred so far:
 > https://mail-index.netbsd.org/source-changes/2026/05/07/msg161941.html
 > I think some of these changes is causing the issue.

 Sorry to hear you're having issues with 11.0_RC4.

 I'm not sure this commit is the cause however as the files touch by that =
 commit aren't used on aarch64

 nick@zoom:~/netbsd/netbsd-11/src$ find ../obj.evbarm64-el/ -name =
 "atomic_cas*cas*o"
 nick@zoom:~/netbsd/netbsd-11/src$

 and libc.so doesn't change

 Maybe I'm missing something?

State-Changed-From-To: open->feedback
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Sat, 16 May 2026 19:05:31 +0000
State-Changed-Why:
The transcript of the screenshot you shared shows that:

- ls, cron, and python3.13 are all waiting for some rwlock, probably
  waiting for a vnode, but we're not sure which one.

- estd is busy on the CPU (l_stat = 7 = LSONPROC) and has been noted as
  a CPU hog (l_flag = 0x40000 = LW_BATCH).

Next time this happens, can you share the output of the following
commands in ddb?

1. show all tstiles
2. ps/w
3. bt (if estd is in userland, it might be nonsense, but if it's stuck
   in a loop in the kernel, it's worth a shot)

This might help us to track down who's holding the rwlock that the
waiters are waiting for.


From: Yasushi Oshima <oshima-ya@yagoto-urayama.jp>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: port-evbarm/60269 (System hangs NetBSD/evbarm(aarch64)
 11.0_RC4)
Date: Sun, 17 May 2026 10:48:01 +0900 (JST)

 at Sat, 16 May 2026 19:05:38 +0000 (UTC)
 riastradh@NetBSD.org wrote

 > Next time this happens, can you share the output of the following
 > commands in ddb?
 > 
 > 1. show all tstiles
 > 2. ps/w
 > 3. bt (if estd is in userland, it might be nonsense, but if it's stuck
 >    in a loop in the kernel, it's worth a shot)

 Okay, I try it in the next time.
 (I'm leaving this system-image  USB stick that had the problem without sny overwrite.)

 Thanks,
 --
 Yasushi Oshima

From: Yasushi Oshima <oshima-ya@yagoto-urayama.jp>
To: gnats-bugs@netbsd.org, port-evbarm-maintainer@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: port-evbarm/60269: System hangs NetBSD/evbarm(aarch64) 11.0_RC4
Date: Sun, 17 May 2026 10:23:11 +0900 (JST)

 Thanks for your response!

 at Sat, 16 May 2026 11:10:01 +0000 (UTC) 
 "Nick Hudson via gnats" <gnats-admin@NetBSD.org> :

 >  commit aren't used on aarch64
 >  
 >  nick@zoom:~/netbsd/netbsd-11/src$ find ../obj.evbarm64-el/ -name =
 >  "atomic_cas*cas*o"
 >  nick@zoom:~/netbsd/netbsd-11/src$
 >  
 >  and libc.so doesn't change
 >  
 >  Maybe I'm missing something?
 >  

 Wow!

 Yes, your point is correnct, it was my misunderstanding, sorry.

 I'll check if there are any other differences.

 Thanks,
 --
 Yasushi Oshima

From: Ramiro Aceves <ea1abz@gmail.com>
To: gnats-bugs@netbsd.org, port-evbarm-maintainer@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, oshima-ya@yagoto-urayama.jp
Cc: riastradh@NetBSD.org
Subject: Re: port-evbarm/60269: System hangs NetBSD/evbarm(aarch64) 11.0_RC4
Date: Sun, 17 May 2026 15:11:12 +0200

 El 17/5/26 a las 4:05, Yasushi Oshima via gnats escribió:
 > The following reply was made to PR port-evbarm/60269; it has been noted by GNATS.
 >
 > From: Yasushi Oshima <oshima-ya@yagoto-urayama.jp>
 > To: gnats-bugs@netbsd.org, port-evbarm-maintainer@netbsd.org,
 >   gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
 > Cc:
 > Subject: Re: port-evbarm/60269: System hangs NetBSD/evbarm(aarch64) 11.0_RC4
 > Date: Sun, 17 May 2026 10:23:11 +0900 (JST)
 >
 >   Thanks for your response!
 >   
 >   at Sat, 16 May 2026 11:10:01 +0000 (UTC)
 >   "Nick Hudson via gnats" <gnats-admin@NetBSD.org> :
 >   
 >   >  commit aren't used on aarch64
 >   >
 >   >  nick@zoom:~/netbsd/netbsd-11/src$ find ../obj.evbarm64-el/ -name =
 >   >  "atomic_cas*cas*o"
 >   >  nick@zoom:~/netbsd/netbsd-11/src$
 >   >
 >   >  and libc.so doesn't change
 >   >
 >   >  Maybe I'm missing something?
 >   >
 >   
 >   Wow!
 >   
 >   Yes, your point is correnct, it was my misunderstanding, sorry.
 >   
 >   I'll check if there are any other differences.
 >   
 >   Thanks,
 >   --
 >   Yasushi Oshima
 >   

 Hi, waiting for your conclusions before upgrading from RC3 to RC4 in my RPi4.

 Regards.



From: Yasushi Oshima <oshima-ya@yagoto-urayama.jp>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: port-evbarm/60269 (System hangs NetBSD/evbarm(aarch64)
 11.0_RC4)
Date: Tue, 23 Jun 2026 22:41:09 +0900 (JST)

 >  > Next time this happens, can you share the output of the following
 >  > commands in ddb?
 >  > 
 >  > 1. show all tstiles
 >  > 2. ps/w
 >  > 3. bt (if estd is in userland, it might be nonsense, but if it's stuck
 >  >    in a loop in the kernel, it's worth a shot)
 >  

  Re-produced hangup on NetBSD/evbarm(aarch64) 11.0_RC4.
  This time, it  occurred during pkg_add, not during any build process.
  I ran ddb and enter 'show tstile' and 'ps/w' and get screenshots.
  Unfortunately, the system became unresponsive before I could enter 'bt'.

  Screenshots (by camera) are here:
    https://gist.github.com/oshimaya/ca595df2a9b104e4583304a897588047

 Thanks.

 --
 Yasushi Oshima


>Unformatted:

Home	PR Database Search