NetBSD Problem Report #58730
From martin@duskware.de Tue Oct 8 15:42:28 2024
Return-Path: <martin@duskware.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits)
client-signature RSA-PSS (2048 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 164FD1A923B
for <gnats-bugs@gnats.NetBSD.org>; Tue, 8 Oct 2024 15:42:28 +0000 (UTC)
From: martin@NetBSD.org
Reply-To: martin@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: NFS client locks up waiting on nfscn2
X-Send-Pr-Version: 3.95
>Number: 58730
>Category: kern
>Synopsis: NFS client locks up waiting on nfscn2
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Oct 08 15:45:00 +0000 2024
>Last-Modified: Wed Oct 01 17:25:02 +0000 2025
>Originator: Martin Husemann
>Release: NetBSD 10.99.12
>Organization:
The NetBSD Foundation, Inc.
>Environment:
System: NetBSD unpluged.duskware.de 10.99.12 NetBSD 10.99.12 (UNPLUGED) #588: Tue Oct 8 09:45:16 CEST 2024 martin@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/evbarm/compile/UNPLUGED evbarm
Architecture: earmv5
Machine: evbarm
>Description:
A diskless machine (using mvgbe(4) for NFS / ) started locking up under
(heavy) NFS traffic.
db> ps
PID LID S CPU FLAGS STRUCT LWP * NAME WAIT
29359 29359 3 0 1000000 c50c4480 sed nfscn2
24567 24567 3 0 180 c7f7c240 tcsh ttyraw
189 189 3 0 180 c8093180 tcsh pause
5625 5625 3 0 180 c7f7ca80 rxvt select
3438 3438 3 0 180 c7fa2b80 tcsh pause
2949 2949 3 0 180 c7fbf1c0 sshd-session poll
3922 3922 3 0 180 c8088300 sshd-session poll
1599 1599 3 0 180 c3ecf980 tail kqueue
1022 1022 3 0 180 c2f72500 tcsh pause
710 710 3 0 180 c2fe0300 rxvt select
704 704 3 0 180 c2f72d40 tcsh pause
559 559 3 0 180 c3410380 sshd-session poll
685 685 3 0 180 c2f727c0 sshd-session poll
2397 2397 3 0 180 c3ecf6c0 tee pipe_rd
1047 1047 3 0 180 c3ecf400 atf-report pipe_rd
2076 2076 3 0 180 c3e01680 tee pipe_rd
464 464 3 0 1c0 c2f72a80 atf-run pipe_rd
1880 1880 3 0 180 c3410900 sh wait
466 466 3 0 180 c2f72240 sh wait
252 252 3 0 0 c2f44200 getty biowait
374 374 3 0 40 c3ecf140 cron nfsrcv
558 558 3 0 180 c3e01c00 inetd kqueue
155 155 3 0 180 c3e01940 sshd poll
200 200 3 0 180 c3410bc0 ntpd pause
237 237 3 0 180 c3410640 rpcbind poll
727 639 3 0 180 c2fe0880 named parked
727 729 3 0 10000040 c34100c0 timer biowait
727 615 3 0 180 c3053b80 isc-net-0001 kqueue
727 726 3 0 0 c30538c0 isc-net-0000 nfsrcv
727 727 3 0 180 c3053080 named sigwait
757 757 3 0 0 c3053600 syslogd nfsrcv
494 494 3 0 180 c3053340 dhcpcd poll
497 497 3 0 180 c2fe0b40 dhcpcd poll
486 486 3 0 180 c2fe0040 dhcpcd poll
182 182 3 0 180 c2fe05c0 dhcpcd poll
1 1 3 0 180 c2e99c80 init wait
0 9760 5 0 240 c3e013c0 (zombie)
0 6818 3 0 200 c7e77a40 physiod physiod
0 1395 3 0 200 c2f444c0 swapiod swapiod
0 106 3 0 200 c2f44d00 pooldrain pooldrain
0 105 3 0 240 c2f44a40 ioflush nfsrcv
0 104 3 0 200 c2f44780 pgdaemon pgdaemon
0 101 3 0 200 c2ebdcc0 nfsio nfsiod
0 100 3 0 200 c2ebda00 nfsio nfsiod
0 99 3 0 200 c2ebd740 nfsio nfsiod
0 98 3 0 200 c2ebd480 nfsio nfsiod
0 97 3 0 200 c2e58140 swdmover swdmvr
0 96 3 0 200 c2ebd1c0 usb0 usbevt
0 60 3 0 200 c2e58400 rt_free rt_free
0 59 3 0 200 c2e586c0 unpgc unpgc
0 58 3 0 200 c2e58980 icmp6_wqinput/0 icmp6_wqinput
0 57 3 0 200 c2e58c40 nd6_timer nd6_timer
0 56 3 0 200 c2e99180 icmp_wqinput/0 icmp_wqinput
0 55 3 0 200 c2e99440 rt_timer rt_timer
0 31 3 0 200 c2e99700 vmem_rehash vmem_rehash
0 30 3 0 200 c2e999c0 sdmmc0 mmctaskq
0 21 3 0 200 c2b55c00 entbutler entropy
0 20 3 0 240 c2b55940 atabus1 atath
0 19 3 0 240 c2b55680 atabus0 atath
0 18 3 0 200 c2b553c0 usbtask-dr usbtsk
0 17 3 0 200 c2b55100 usbtask-hc usbtsk
0 16 3 0 200 c2b39bc0 sysmon smtaskq
0 15 3 0 200 c2b39900 pmfsuspend pmfsuspend
0 14 3 0 200 c2b39640 pmfevent pmfevent
0 13 3 0 200 c2b39380 sopendfree sopendfr
0 12 3 0 200 c2b390c0 ifwdog ifwdog
0 11 3 0 200 c2a6db80 iflnkst iflnkst
0 10 3 0 200 c2a6d8c0 nfssilly nfssilly
0 9 3 0 200 c2a6d600 pooldisp pooldisp
0 8 3 0 200 c2a6d340 modunload mod_unld
0 7 3 0 200 c2a6d080 xcall/0 xcall
0 6 1 0 200 c2a6cb40 softser/0
0 5 1 0 40200 c2a6c880 softclk/0
0 4 1 0 200 c2a6c5c0 softbio/0
0 3 1 0 200 c2a6c300 softnet/0
0 > 2 1 0 201 c2a6c040 idle/0
0 0 3 0 200 c051b700 swapper uvm
Note there is an unreaped zombie process (even after hours in this state),
so I guess init did not get a chance to run.
I saw this first with a Oct 1 kernel, the previous Sep 23 kernel was fine.
>How-To-Repeat:
Put some file system load on a diskless machine?
>Fix:
n/a
>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Tue, 8 Oct 2024 18:10:30 +0200
The client seems to loose all networking, on the server I see ARP requests
from the client (with a longish delay between small batches).
I am not sure why getty on the console hangs though (unless it is paged
out, which seems unlikely).
A bit more ddb output from another instance of the hang below.
Martin
db> ps
PID LID S CPU FLAGS STRUCT LWP * NAME WAIT
5593 5593 3 0 1000000 c46d9180 sh nfscn2
22609 22609 3 0 180 c4716480 tail kqueue
506 506 3 0 180 c2f72240 tcsh pause
504 504 3 0 180 c340f900 rxvt select
436 436 3 0 180 c2fdf880 tcsh pause
1077 1077 3 0 180 c2f727c0 sshd-session poll
1970 1970 3 0 180 c2f72d40 sshd-session poll
288 288 3 0 180 c340fbc0 tee pipe_rd
289 289 3 0 180 c36db6c0 atf-report pipe_rd
290 290 3 0 180 c36db400 tee pipe_rd
925 925 3 0 1c0 c2fdf300 atf-run pipe_rd
350 350 3 0 180 c3604680 sh wait
340 340 3 0 180 c2f72500 sh wait
252 252 3 0 0 c2f44200 getty rwlock
1654 1654 3 0 40 c36db140 cron nfssnd
558 558 3 0 180 c3604c00 inetd kqueue
730 730 3 0 180 c3604940 sshd poll
200 200 3 0 180 c3604100 ntpd pause
237 237 3 0 180 c340f640 rpcbind poll
732 538 3 0 180 c340f380 named parked
732 733 3 0 180 c340f0c0 timer parked
732 638 3 0 180 c3055b80 isc-net-0001 kqueue
732 765 3 0 180 c30558c0 isc-net-0000 kqueue
732 732 3 0 180 c3055600 named sigwait
757 757 3 0 0 c3055340 syslogd nfsrcv
494 494 3 0 180 c2fdfb40 dhcpcd poll
497 497 3 0 180 c2fdf5c0 dhcpcd poll
492 492 3 0 180 c3055080 dhcpcd poll
486 486 3 0 180 c2fdf040 dhcpcd poll
91 91 3 0 180 c2f72a80 wdogctl nanoslp
1 1 3 0 180 c2e99c80 init wait
0 21568 5 0 240 c36043c0 (zombie)
0 1650 3 0 200 c2f444c0 swapiod swapiod
0 106 3 0 200 c2f44d00 pooldrain pooldrain
0 105 3 0 240 c2f44a40 ioflush nfssnd
0 104 3 0 200 c2f44780 pgdaemon pgdaemon
0 101 3 0 200 c2ebdcc0 nfsio nfsiod
0 100 3 0 200 c2ebda00 nfsio nfsiod
0 99 3 0 200 c2ebd740 nfsio nfsiod
0 98 3 0 200 c2ebd480 nfsio nfsiod
0 97 3 0 200 c2e58140 swdmover swdmvr
0 96 3 0 200 c2ebd1c0 usb0 usbevt
0 60 3 0 200 c2e58400 rt_free rt_free
0 59 3 0 200 c2e586c0 unpgc unpgc
0 58 3 0 200 c2e58980 icmp6_wqinput/0 icmp6_wqinput
0 57 3 0 200 c2e58c40 nd6_timer nd6_timer
0 56 3 0 200 c2e99180 icmp_wqinput/0 icmp_wqinput
0 55 3 0 200 c2e99440 rt_timer rt_timer
0 31 3 0 200 c2e99700 vmem_rehash vmem_rehash
0 30 3 0 200 c2e999c0 sdmmc0 mmctaskq
0 21 3 0 200 c2b55c00 entbutler entropy
0 20 3 0 240 c2b55940 atabus1 atath
0 19 3 0 240 c2b55680 atabus0 atath
0 18 3 0 200 c2b553c0 usbtask-dr usbtsk
0 17 3 0 200 c2b55100 usbtask-hc usbtsk
0 16 3 0 200 c2b39bc0 sysmon smtaskq
0 15 3 0 200 c2b39900 pmfsuspend pmfsuspend
0 14 3 0 200 c2b39640 pmfevent pmfevent
0 13 3 0 200 c2b39380 sopendfree sopendfr
0 12 3 0 200 c2b390c0 ifwdog ifwdog
0 11 3 0 200 c2a6db80 iflnkst iflnkst
0 10 3 0 200 c2a6d8c0 nfssilly nfssilly
0 9 3 0 200 c2a6d600 pooldisp pooldisp
0 8 3 0 200 c2a6d340 modunload mod_unld
0 7 3 0 200 c2a6d080 xcall/0 xcall
0 6 1 0 200 c2a6cb40 softser/0
0 5 1 0 40200 c2a6c880 softclk/0
0 4 1 0 200 c2a6c5c0 softbio/0
0 3 1 0 200 c2a6c300 softnet/0
0 > 2 1 0 201 c2a6c040 idle/0
0 0 3 0 200 c051b700 swapper uvm
db> show all tstiles
PID LID COMMAND WAITING-FOR TYPE WAIT-CHANNEL
5593 5593 sh 0 kpause c46d9180
22609 22609 tail 0 cv c470e8b8
506 506 tcsh 0 kpause c2f72240
504 504 rxvt 0 select c2a5d9c0
436 436 tcsh 0 kpause c2fdf880
1077 1077 sshd-session 0 select c2a5d9c0
1970 1970 sshd-session 0 select c2a5d9c0
288 288 tee 0 cv c2f6f704
289 289 atf-report 0 cv c2f6f654
290 290 tee 0 cv c2f6f5a4
925 925 atf-run 0 cv c2f6f7b4
350 350 sh 0 cv c35ff2cc
340 340 sh 0 cv c2eb27c4
252 252 getty c36db140 rwlock c2f3e160
1654 1654 cron 0 cv c2b4994c
558 558 inetd 0 cv c36f1af8
730 730 sshd 0 select c2a5d9c0
200 200 ntpd 0 kpause c3604100
237 237 rpcbind 0 select c2a5d9c0
732 538 named 0 lwp_park c340f380
732 733 named 0 lwp_park c340f0c0
732 638 named 0 cv c340b378
732 765 named 0 cv c33fadf8
732 732 named 0 cv c3055768
757 757 syslogd 0 cv c2b49944
494 494 dhcpcd 0 select c2a5d9c0
497 497 dhcpcd 0 select c2a5d9c0
492 492 dhcpcd 0 select c2a5d9c0
486 486 dhcpcd 0 select c2a5d9c0
91 91 wdogctl 0 kpause c2f72a80
1 1 init 0 cv c2eb202c
0 1650 system 0 cv c35f6aec
0 106 system 0 cv netbsd:uvmpd_pool_drain_c
v
0 105 system 0 cv c2b4994c
0 104 system 0 sleep netbsd:uvm+0x68
0 101 system 0 cv c2f38604
0 100 system 0 cv c2f38584
0 99 system 0 cv c2f38504
0 98 system 0 cv c2f38484
0 97 system 0 sleep netbsd:swdmover_cv
0 96 system 0 cv c2b4e264
0 60 system 0 cv c2e9aaec
0 59 system 0 cv netbsd:unp_thread_cv
0 58 system 0 cv c2e9a8ac
0 57 system 0 cv c2e9a7ec
0 56 system 0 cv c2e9a66c
0 55 system 0 cv c2e9a5ac
0 31 system 0 cv c2e9a2ac
0 30 system 0 cv c2e40b8c
0 21 system 0 cv netbsd:entropy_global+0xf
0
0 20 system 0 cv c2e4e10c
0 19 system 0 cv c2b5390c
0 18 system 0 cv netbsd:usb_taskq+0x2c
0 17 system 0 cv netbsd:usb_taskq+0xc
0 16 system 0 cv netbsd:sysmon_task_queue_
cv
0 15 system 0 cv c2a6bbac
0 14 system 0 cv c2a6baec
0 13 system 0 cv netbsd:pendfree_thread_cv
0 12 system 0 cv c2a6b96c
0 11 system 0 cv c2a6b8ac
0 10 system 0 cv c2a6b72c
0 9 system 0 cv c2a6b4d4
0 8 system 0 cv netbsd:module_thread_cv
0 7 system 0 cv netbsd:cpu_info_store+0x8
0 0 system 0 kpause netbsd:lwp0
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, martin@NetBSD.org
Cc:
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Wed, 9 Oct 2024 14:10:08 +0900
Probably same as this?
https://gnats.netbsd.org/56850
From: Martin Husemann <martin@duskware.de>
To: Rin Okuyama <rokuyama.rk@gmail.com>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Wed, 9 Oct 2024 07:57:06 +0200
On Wed, Oct 09, 2024 at 02:10:08PM +0900, Rin Okuyama wrote:
> Probably same as this?
> https://gnats.netbsd.org/56850
Yes, looks like it - strange that it has been nearly easy to reproduce
for me the last two days and ~never hit my setup before.
This is on a GuruPlug (so Kirkwood as well).
Martin
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Fri, 11 Oct 2024 19:42:48 +0200
Interesting - for me this has never happend before the following changes:
cvs rdiff -u -r1.325 -r1.326 src/sys/dev/usb/ehci.c
cvs rdiff -u -r1.40 -r1.41 src/sys/dev/usb/ehcireg.h
cvs rdiff -u -r1.52 -r1.53 src/sys/dev/usb/ehcivar.h
and downgrading those files to the 1.325/1.40/1.52 resp. makes my current
kernel work again (note that I only did test very few sample runs of
the full /usr/tests).
Since this is touching cache alignment - could it be that we get the cache
line size wrong on this machines?
Martin
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Wed, 16 Oct 2024 18:20:03 +0200
Here is the latest dmesg of that machine
(weekly updates at: https://netbsd.org/~martin/evbarm-atf/dmesg.txt,
this is one of my regular test machines)
Martin
Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023,
2024
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
NetBSD 10.99.12 (UNPLUGED) #600: Mon Oct 14 12:03:10 CEST 2024
martin@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/evbarm/compile/UNPLUGED
total memory = 512 MB
avail memory = 495 MB
timecounter: Timecounters tick every 10.000 msec
mainbus0 (root)
cpu0 at mainbus0 core 0: Sheeva 88SV131 rev 1 (ARM9E-S V5TE core)
cpu0: DC enabled IC enabled WB enabled LABT branch prediction enabled
cpu0: L1 16KB/32B 4-way (128 set) VIVT Instruction cache
cpu0: L1 16KB/32B 4-way (128 set) write-back-locking-C VIVT Data cache
mvsoc0 at mainbus0: Marvell 88F6281 Rev. A1 Kirkwood
mvsoc0: CPU Clock 1200.000 MHz SysClock 400.000 MHz TClock 200.000 MHz
mvsoctmr0 at mvsoc0 unit 0 offset 0x20300-0x203ff irq 65: Marvell SoC Timer
mvsocgpp0 at mvsoc0 unit 0 offset 0x10100-0x101ff irq 35: Marvell SoC General Purpose I/O Port Interface
mvsocgpp0: 50 gpio pins
mvsocgpp0: interrupts 96..103, intr 35
mvsocgpp0: interrupts 104..111, intr 36
mvsocgpp0: interrupts 112..119, intr 37
mvsocgpp0: interrupts 120..127, intr 38
mvsocgpp0: interrupts 128..135, intr 39
mvsocgpp0: interrupts 136..143, intr 40
mvsocgpp0: interrupts 144..151, intr 41
mvsocgpp0: Data Out: 0x00000000
mvsocgpp0: Data Out Enable Control: 0xffffffff
mvsocgpp0: Data Blink Enable: 0x00000000
mvsocgpp0: Data In Polarity: 0x00000000
mvsocgpp0: Data In: 0x00000080
mvsocgpp0: Interrupt Cause: 0x00000000
mvsocgpp0: Interrupt Mask: 0x00000000
mvsocgpp0: Interrupt Level Mask: 0x00000000
mvsocgpp0: High Data Out: 0x000f0000
mvsocgpp0: High Data Out Enable Ctrl: 0xffffffff
mvsocgpp0: High Blink Enable: 0x00000000
mvsocgpp0: High Data In Polarity: 0x00000000
mvsocgpp0: High Data In: 0x0003f000
mvsocgpp0: High Interrupt Cause: 0x00000000
mvsocgpp0: High Interrupt Mask: 0x00000000
mvsocgpp0: High Interrupt Level Mask: 0x00000000
gpio0 at mvsocgpp0: 50 pins
mvsocrtc0 at mvsoc0 unit 0 offset 0x10300-0x10317: Marvell SoC Real Time Clock
com0 at mvsoc0 unit 0 offset 0x12000-0x1201f irq 33: ns16550a, 16-byte FIFO
com0: console
com1 at mvsoc0 unit 1 offset 0x12100-0x1211f irq 34: ns16550a, 16-byte FIFO
ehci0 at mvsoc0 unit 0 offset 0x50000-0x50fff irq 19: Marvell USB 2.0 Interface
ehci0: EHCI version 1.0
usb0 at ehci0: USB revision 2.0
gtidmac0 at mvsoc0 unit 0 offset 0x60000-0x60fff: Marvell IDMA Controller/XOR Engine
gtidmac0: XOR Engine 4 channels, intr 5, 6, 7, 8
gttwsi0 at mvsoc0 unit 0 offset 0x11000-0x110ff irq 29: Marvell TWSI controller
iic0 at gttwsi0: I2C bus
mvcesa0 at mvsoc0 unit 0 offset 0x3d000-0x3dfff irq 22: Marvell Cryptographic Engines and Security Accelerator
mvgbec0 at mvsoc0 unit 0 offset 0x70000-0x73fff: Marvell Gigabit Ethernet Controller
mvgbe0 at mvgbec0 port 0 irq 11
mvgbe0: Ethernet address 00:50:43:01:83:1e
makphy0 at mvgbe0 phy 0: Marvell 88E1149 Gigabit PHY, rev. 3
makphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
mvgbec1 at mvsoc0 unit 1 offset 0x74000-0x77fff: Marvell Gigabit Ethernet Controller
mvgbe at mvgbec1 port 0 not configured
mvpex0 at mvsoc0 unit 0 offset 0x40000-0x41fff irq 9: Marvell PCI Express Interface
pci0 at mvpex0
pci0: i/o space, memory space enabled
vendor 11ab product 6281 (miscellaneous memory, revision 0x03) at pci0 dev 0 function 0 not configured
mvsata0 at mvsoc0 unit 0 offset 0x80000-0x87fff irq 21: Marvell Serial-ATA Host Controller (SATAHC)
mvsata0: GenIIe, 1hc, 2port/hc
atabus0 at mvsata0 channel 0
atabus1 at mvsata0 channel 1
mvsdio0 at mvsoc0 unit 0 offset 0x90000-0x9ffff irq 28: Marvell Secure Digital Input/Output Interface
sdmmc0 at mvsdio0
timecounter: Timecounter "mvsoctmr0" frequency 200000000 Hz quality 100
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
WARNING: system needs entropy for security; see entropy(7)
uhub0 at usb0: NetBSD (0x0000) EHCI root hub (0x0000), class 9/0, rev 2.00/1.00, addr 1
uhub0: 1 port with 1 removable, self powered
15 1f 01 00 4d 61 72 76 65 6c 6c 00 38 30 32 2e
31 31 20 53 44 49 4f 20 49 44 3a 20 31 30 00 00
ff
CISTPL_VERS_1
20 04 df 02 03 91
CISTPL_MANFID
21 02 0c 00
CISTPL_FUNCID
22 04 00 00 01 5a
CISTPL_FUNCE: FN0_BLK_SIZE=0x100, MAX_TRAN_SPEED=0x5a(50000kHz)
ff
CISTPL_END
sdmmc0: 4-bit width, 50.000 MHz
sdmmc0: autoconfiguration error: bad CIS ptr 0
sdmmc0: autoconfiguration error: couldn't read CIS
sdmmc0: autoconfiguration error: i/o init failed
sdmmc0: autoconfiguration error: bad CIS ptr 0
sdmmc0: autoconfiguration error: couldn't read CIS
sdmmc0: autoconfiguration error: i/o init failed
"Marvell, 802.11 SDIO ID: 10, " (manufacturer 0x2df, product 0x9103) at sdmmc0 function 1 not configured
"Marvell, 802.11 SDIO ID: 10, " (manufacturer 0x2df, product 0x9103) at sdmmc0 function 2 not configured
uhub1 at uhub0 port 1: Genesys Logic (0x05e3) USB2.0 Hub (0x0608), class 9/0, rev 2.00/77.60, addr 2
uhub1: single transaction translator
uhub1: 4 ports with 4 removable, self powered
umass0 at uhub1 port 1 configuration 1 interface 0
umass0: Genesys Logic (0x05e3) USB Storage (0x0726), rev 2.00/99.09, addr 3
umass0: using SCSI over Bulk-Only
scsibus0 at umass0: 2 targets, 2 luns per target
sd0 at scsibus0 target 0 lun 0: <Generic, STORAGE DEVICE, 9909> disk removable
sd0: drive offline
sd1 at scsibus0 target 0 lun 1: <Generic, STORAGE DEVICE, 9909> disk removable
sd1: drive offline
uhub2 at uhub1 port 3: NEC (0x0409) product 005a (0x005a), class 9/0, rev 2.00/1.00, addr 4
uhub2: single transaction translator
uhub2: 2 ports with 0 removable, self powered
umcs0 at uhub2 port 1: MosChip Semiconductor (0x9710) MCS7840 USB Serial Adapter (0x7840), rev 2.00/0.01, addr 5
umcs0: found 4 active ports
umcs0: On-die configuration: RST: active low, HRD: yes, PLL: avail, POR: avail, Ports: 4, EEPROM write disabled, IrDA is not available
ucom0 at umcs0 portno 0
ucom1 at umcs0 portno 1
ucom2 at umcs0 portno 2
ucom3 at umcs0 portno 3
umcs1 at uhub2 port 2: MosChip Semiconductor (0x9710) MCS7840 USB Serial Adapter (0x7840), rev 2.00/0.01, addr 6
umcs1: found 4 active ports
umcs1: On-die configuration: RST: active low, HRD: yes, PLL: avail, POR: avail, Ports: 4, EEPROM write disabled, IrDA is not available
ucom4 at umcs1 portno 0
ucom5 at umcs1 portno 1
ucom6 at umcs1 portno 2
ucom7 at umcs1 portno 3
WARNING: 6 errors while detecting hardware; check system log.
boot device: <unknown>
root on mvgbe0
nfs_boot: trying DHCP/BOOTP
nfs_boot: DHCP next-server: 192.168.150.188
nfs_boot: my_domain=duskware.de
nfs_boot: my_addr=192.168.150.127
nfs_boot: my_mask=255.255.254.0
nfs_boot: gateway=192.168.151.1
root on 192.168.150.188:/disk/hosts/evbarm
kern.module.path=/stand/evbarm/10.99.12/modules
entropy: best effort
entropy: ready
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Tue, 5 Nov 2024 11:08:18 +0100
I just noticed that on this machine with a -current kernel w/o
backing out the ehci changes probing usb devices takes ages and some
ports are disabled (at boot time).
[ 1.1499648] sdmmc0: couldn't read CIS
[ 1.1499648] sdmmc0: i/o init failed
[ 1.1499648] "Marvell, 802.11 SDIO ID: 10, " (manufacturer 0x2df, product 0x9103) at sdmmc0 function 1 not configured
[ 1.1499648] "Marvell, 802.11 SDIO ID: 10, " (manufacturer 0x2df, product 0x9103) at sdmmc0 function 2 not configured
[ 54.5699816] uhub0: device problem, disabling port 1
[ 54.5699816] WARNING: 6 errors while detecting hardware; check system log.
[ 54.5699816] boot device: <unknown>
[ 54.5699816] root on mvgbe0
Regular dmesg at:
https://netbsd.org/~martin/evbarm-atf/dmesg.txt
Martin
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, martin@NetBSD.org
Cc:
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Tue, 5 Nov 2024 19:34:22 +0900
On 2024/11/05 19:10, Martin Husemann via gnats wrote:
> I just noticed that on this machine with a -current kernel w/o
> backing out the ehci changes probing usb devices takes ages and some
> ports are disabled (at boot time).
Have you found out the blamed commit? I was about opening a PR for
similar problems below, but stalled on narrowing down the commits.
For me, the following problem disappear if the entire dev/usb is
replaced by that of netbsd-10 branch...
I observe a similar problem for ehci(4) on evbarm/NSLU2. Also, for
ehci(4) on landisk/HDL-U, attaching sd(4) takes few minutes:
````
waiting for devices: scsibus0 sd0
waiting for devices: scsibus0 sd0
waiting for devices: scsibus0 sd0
umass0: BBB bulk-in stall clear failed, TIMEOUT
(snip)
waiting for devices: scsibus0 sd0
umass0: BBB reset failed, TIMEOUT
sd0: 232 GB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 488521728 sectors
````
It also stalls during mount (both for RO as root, and RW with fstab):
````
umass0: BBB bulk-in stall clear failed, TIMEOUT
umass0: BBB reset failed, TIMEOUT
````
But once it gets mounted, no additional errors appear anymore.
Thanks,
rin
From: Martin Husemann <martin@duskware.de>
To: Rin Okuyama <rokuyama.rk@gmail.com>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Tue, 5 Nov 2024 12:29:42 +0100
On Tue, Nov 05, 2024 at 07:34:22PM +0900, Rin Okuyama wrote:
> Have you found out the blamed commit? I was about opening a PR for
> similar problems below, but stalled on narrowing down the commits.
Yes:
cvs rdiff -u -r1.325 -r1.326 src/sys/dev/usb/ehci.c
cvs rdiff -u -r1.40 -r1.41 src/sys/dev/usb/ehcireg.h
cvs rdiff -u -r1.52 -r1.53 src/sys/dev/usb/ehcivar.h
broke things for me.
I do
cd sys/dev/usb && cvs up -r1.325 ehci.c && cvs up -r1.40 ehcireg.h && \
cvs up -r1.52 ehcivar.h
before building a kernel for the Kirkwood machine and everything is all fine.
Nick suggested to review mvgbe bus dma sync operations (and I was planning
to do that), but now saw that USB is affected too, so something in this commit
must be wrong (and unrelated to mvgbe bugs).
Martin
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org, netbsd-bugs@NetBSD.org, gnats-admin@netbsd.org
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Wed, 6 Nov 2024 21:42:58 +0900
On 2024/11/05 20:29, Martin Husemann wrote:
> On Tue, Nov 05, 2024 at 07:34:22PM +0900, Rin Okuyama wrote:
>> Have you found out the blamed commit? I was about opening a PR for
>> similar problems below, but stalled on narrowing down the commits.
>
> Yes:
>
> cvs rdiff -u -r1.325 -r1.326 src/sys/dev/usb/ehci.c
> cvs rdiff -u -r1.40 -r1.41 src/sys/dev/usb/ehcireg.h
> cvs rdiff -u -r1.52 -r1.53 src/sys/dev/usb/ehcivar.h
>
> broke things for me.
>
> I do
>
> cd sys/dev/usb && cvs up -r1.325 ehci.c && cvs up -r1.40 ehcireg.h && \
> cvs up -r1.52 ehcivar.h
>
> before building a kernel for the Kirkwood machine and everything is all fine.
>
> Nick suggested to review mvgbe bus dma sync operations (and I was planning
> to do that), but now saw that USB is affected too, so something in this commit
> must be wrong (and unrelated to mvgbe bugs).
Thank you for info! I've confirmed that restoring USBMALLOC_COHERENT for
usb_allocmem() in ehci.c fixes the problems both for landisk/HDL_U and
evbppc/NSLU2.
I've found that behavior for bus_dmamap_sync(9) is different b/w
arm32/bus_dma.c and landisk/bus_dma.c; for BUS_DMASYNC_PREREAD,
whereas the former carries out write-back only for misalined parts,
the latter does for segment if its address or length is misalined.
However, even if the behavior is sync with arm32/bus_dma.c:
https://gist.github.com/rokuyama/6e5ffaf956cd39db4752c69fa792096a
the problem for landisk/HDL_U is not improved for me.
Thanks,
rin
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org, netbsd-bugs@NetBSD.org, gnats-admin@netbsd.org
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Wed, 6 Nov 2024 21:45:49 +0900
On 2024/11/06 21:42, Rin Okuyama wrote:
> However, even if the behavior is sync with arm32/bus_dma.c:
>
> https://gist.github.com/rokuyama/6e5ffaf956cd39db4752c69fa792096a
>
> the problem for landisk/HDL_U is not improved for me.
... and this is an expected result, provided buffers are page-aligned,
and have sizes multiple to 32.
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: Martin Husemann <martin@duskware.de>
Cc: gnats-bugs@netbsd.org, netbsd-bugs@NetBSD.org, gnats-admin@netbsd.org
Subject: Re: kern/58730: NFS client locks up waiting on nfscn2
Date: Thu, 7 Nov 2024 08:10:49 +0900
On 2024/11/06 21:45, Rin Okuyama wrote:
> On 2024/11/06 21:42, Rin Okuyama wrote:
>> However, even if the behavior is sync with arm32/bus_dma.c:
>>
>> https://gist.github.com/rokuyama/6e5ffaf956cd39db4752c69fa792096a
>>
>> the problem for landisk/HDL_U is not improved for me.
>
> ... and this is an expected result, provided buffers are page-aligned,
> and have sizes multiple to 32.
Oops, confusion in my side. While buffer itself is always page-
aligned, a range of sync operation is generally not.
But scenario itself is not modified, if objects are cacheline
(== 32 bytes)-aligned.
Thanks,
rin
From: "Rin Okuyama" <rin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58730 CVS commit: src/sys/dev/usb
Date: Mon, 29 Sep 2025 14:21:46 +0000
Module Name: src
Committed By: rin
Date: Mon Sep 29 14:21:46 UTC 2025
Modified Files:
src/sys/dev/usb: ehci.c
Log Message:
ehci: usb_syncmem against qtd **after** KASSERT for that qtd
Otherwise, we end up with stale data for DIAGNOSTIC kernel.
Fix device-probe failures discussed in PR kern/58730 for me.
(The main topic of the PR, NFS client lock-up, is not addressed.)
To generate a diff of this commit:
cvs rdiff -u -r1.332 -r1.333 src/sys/dev/usb/ehci.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58730 CVS commit: [netbsd-11] src/sys/dev/usb
Date: Wed, 1 Oct 2025 17:21:19 +0000
Module Name: src
Committed By: martin
Date: Wed Oct 1 17:21:19 UTC 2025
Modified Files:
src/sys/dev/usb [netbsd-11]: ehci.c
Log Message:
Pull up following revision(s) (requested by rin in ticket #44):
sys/dev/usb/ehci.c: revision 1.333
ehci: usb_syncmem against qtd **after** KASSERT for that qtd
Otherwise, we end up with stale data for DIAGNOSTIC kernel.
Fix device-probe failures discussed in PR kern/58730 for me.
(The main topic of the PR, NFS client lock-up, is not addressed.)
To generate a diff of this commit:
cvs rdiff -u -r1.332 -r1.332.2.1 src/sys/dev/usb/ehci.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58730 CVS commit: [netbsd-10] src/sys/dev/usb
Date: Wed, 1 Oct 2025 17:23:32 +0000
Module Name: src
Committed By: martin
Date: Wed Oct 1 17:23:32 UTC 2025
Modified Files:
src/sys/dev/usb [netbsd-10]: ehci.c
Log Message:
Pull up following revision(s) (requested by rin in ticket #1169):
sys/dev/usb/ehci.c: revision 1.333 (patch)
ehci: usb_syncmem against qtd **after** KASSERT for that qtd
Otherwise, we end up with stale data for DIAGNOSTIC kernel.
Fix device-probe failures discussed in PR kern/58730 for me.
(The main topic of the PR, NFS client lock-up, is not addressed.)
To generate a diff of this commit:
cvs rdiff -u -r1.315.2.4 -r1.315.2.5 src/sys/dev/usb/ehci.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
(Contact us)
$NetBSD: query-full-pr,v 1.49 2026/05/14 01:52:41 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home