NetBSD Problem Report #49065
From kardel@Kardel.name Sat Aug 2 09:39:34 2014
Return-Path: <kardel@Kardel.name>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id CFC42A8BA9
for <gnats-bugs@gnats.NetBSD.org>; Sat, 2 Aug 2014 09:39:34 +0000 (UTC)
Message-Id: <20140802081949.6FEF444B17@Andromeda.Kardel.name>
Date: Sat, 2 Aug 2014 10:19:49 +0200 (CEST)
From: kardel@netbsd.org
Reply-To: kardel@netbsd.org
To: gnats-bugs@NetBSD.org
Subject: ifconfig tun0 ... sequence locks up system
X-Send-Pr-Version: 3.95
>Number: 49065
>Category: kern
>Synopsis: ifconfig tun0 ... sequence locks up system / lockup: softnet_lock held across usb xfr
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: skrll
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Aug 02 09:40:00 +0000 2014
>Closed-Date: Fri Jan 06 09:21:53 +0000 2017
>Last-Modified: Fri Jan 06 09:25:00 +0000 2017
>Originator: Frank Kardel
>Release: NetBSD 7.99.3
>Organization:
>Environment:
System: NetBSD rpi 6.99.49 NetBSD 6.99.49 (RPISENSOR) #1: Thu Jul 31 16:11:24 CEST 2014 kardel@Andromeda:/usr/srcfresh/src/sys/arch/evbarm/compile/obj.evbarm/RPISENSOR evbarm
Architecture: earmv6hf
Machine: evbarm
>Description:
setting PTP parameters on tun0 twice will lock up the system.
look like an error path does not unlock the socket structure.
ddb state at lockup
db> ps
PID LID S CPU FLAGS STRUCT LWP * NAME WAIT
2991 1 3 0 0 d9c4f120 ifconfig tstile
3803 1 3 0 80 d9c4f960 sh wait
3637 1 3 0 80 dbcf93a0 login wait
3470 1 3 0 80 d9d1abc0 cron nanoslp
3212 1 3 0 80 d9d1b400 inetd kqueue
3093 1 3 0 0 d9c4fc20 mdnsd usbxfer
3192 1 3 0 80 d9c4f6a0 qmgr kqueue
2988 1 3 0 80 d9c4f3e0 pickup kqueue
754 1 3 0 80 d9d1b6c0 master kqueue
2174 1 3 0 80 d9c4e360 sshd select
1718 1 3 0 80 d9c4e8e0 ntpd netio
1679 1 3 0 80 d9c4e0a0 ntpd pause
1317 1 3 0 80 dbcf8320 devpubd devmon
1139 1 3 0 80 db91f3c0 perl select
599 4 3 0 80 db91eb80 named kqueue
599 3 3 0 80 db91ee40 named parked
599 2 3 0 0 db91f100 named tstile
599 1 3 0 10000080 db91f680 named sigwait
570 1 3 0 80 db91f940 syslogd kqueue
1 1 3 0 80 dbda9380 init wait
0 48 3 0 200 db91e600 onewire0 owidle
0 47 3 0 200 dbda9900 physiod physiod
0 46 3 0 280 dbcf9660 VCHIQka-0 lnxcmplt
0 45 3 0 200 dbcf9920 aiodoned aiodoned
0 44 3 0 200 dbcf9be0 ioflush syncer
0 43 3 0 200 dbda9bc0 pgdaemon pgdaemon
0 42 3 0 200 dbda8040 vcaudiowq vcaudiowq
0 41 3 0 280 dbda8300 VCHIQs-0 semacv
0 40 3 0 280 dbda85c0 VCHIQr-0 semacv
0 39 3 0 280 dbe6a020 VCHIQ-0 semacv
0 36 3 0 200 dbe6a2e0 pfpurge pftm
0 35 3 0 200 dbda9640 usb0 usbevt
0 33 3 0 200 dbda90c0 unpgc unpgc
0 32 3 0 200 dbda8e00 vmem_rehash vmem_rehash
0 31 3 0 200 dbda8880 sdmmc0 mmctaskq
0 30 3 0 200 dbda8b40 vcmbox0 vcmbox0
0 21 3 0 200 dbe6a5a0 usbtask-dr usbtsk
0 20 3 0 200 dbe6a860 usbtask-hc usbtsk
0 19 3 0 200 dbe6ab20 dwc2 dwc2
0 18 3 0 200 dbe6ade0 iic1 iicintr
0 17 3 0 200 dbe6b0a0 iic0 iicintr
0 16 3 0 200 dbe6b360 sysmon smtaskq
0 15 3 0 200 dbe6b620 pmfsuspend pmfsuspend
0 14 3 0 200 dbe6b8e0 pmfevent pmfevent
0 13 3 0 200 dbe6bba0 sopendfree sopendfr
0 12 3 0 200 dbf28000 nfssilly nfssilly
0 11 3 0 200 dbf282c0 cachegc cachegc
0 10 3 0 200 dbf28580 vrele vrele
0 9 3 0 200 dbf28840 vdrain vdrain
0 8 3 0 200 dbf28b00 modunload mod_unld
0 7 3 0 200 dbf28dc0 xcall/0 xcall
0 6 1 0 200 dbf29080 softser/0
0 5 3 0 200 dbf29340 softclk/0 tstile
0 4 1 0 200 dbf29600 softbio/0
0 3 3 0 200 dbf298c0 softnet/0 tstile
0 > 2 7 0 201 dbf29b80 idle/0
0 1 3 0 200 c0573360 swapper uvm
db> bt/t BAF
trace: pid 2991 lid 1 at 0xd9ae1cf4
0xd9ae1cf4: netbsd:mi_switch+0xc
0xd9ae1d24: netbsd:sleepq_block+0xa4
0xd9ae1d64: netbsd:turnstile_block+0x3b8
0xd9ae1dac: netbsd:mutex_enter+0x16c
0xd9ae1dd4: netbsd:soclose+0x20
0xd9ae1dec: netbsd:soo_close+0x20
0xd9ae1e2c: netbsd:closef+0x6c
0xd9ae1e74: netbsd:fd_free+0x174
0xd9ae1eec: netbsd:exit1+0x100
0xd9ae1f0c: netbsd:sys_exit+0x3c
0xd9ae1f7c: netbsd:syscall+0x8c
0xd9ae1fac: netbsd:swi_handler+0x98
db>
>How-To-Repeat:
Following sequence will lock up on an rpi.
rpi# ifconfig tun0 create
rpi# ifconfig tun0 10.200.100.1 10.0.0.200 netmask 0xffffffff up
rpi# ifconfig tun0 10.200.100.1 10.0.0.200 netmask 0xffffffff up
this used to work before.
>Fix:
deleting the PTP config first will circumvent the bug
rpi# ifconfig tun0 create
rpi# ifconfig tun0 10.200.100.1 10.0.0.200 netmask 0xffffffff up
rpi# ifconfig tun0 delete 10.200.100.1 10.0.0.200
rpi# ifconfig tun0 10.200.100.1 10.0.0.200 netmask 0xffffffff up
>Release-Note:
>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 3 Aug 2014 13:51:48 +0200
I can not reproduce it - does it need anything special, like DEBUG and/or
LOCKDEBUG?
Martin
From: Frank Kardel <kardel@acrys.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 03 Aug 2014 14:25:43 +0200
This is a cryptographically signed message in MIME format.
--------------ms060104030001040502070803
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: quoted-printable
I have DIAGNOSTIC and DEBUG.
a 99.43 works - I'll upgrade my local RPI (the troubled one is 600km=20
away :-() to the version where I have seen to problem.
Frank
On 08/03/14 14:05, Martin Husemann wrote:
> The following reply was made to PR kern/49065; it has been noted by GNA=
TS.
>
> From: Martin Husemann <martin@duskware.de>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
> Date: Sun, 3 Aug 2014 13:51:48 +0200
>
> I can not reproduce it - does it need anything special, like DEBUG an=
d/or
> LOCKDEBUG?
>
> Martin
>
>
--------------ms060104030001040502070803
Content-Type: application/pkcs7-signature; name="smime.p7s"
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename="smime.p7s"
Content-Description: S/MIME Cryptographic Signature
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIFdDCC
BXAwggRYoAMCAQICAQswDQYJKoZIhvcNAQEFBQAwgcMxCzAJBgNVBAYTAkRFMQ8wDQYDVQQI
EwZIZXNzZW4xMzAxBgNVBAcTKlRoZW9kb3ItSGV1c3MtU3RyLiA1My02MywgNjExMTggQmFk
IFZpbGJlbDEWMBQGA1UEChMNQWNyeXMgQ29uc3VsdDEeMBwGA1UECxMVQ2VydGlmaWNhdGUg
QXV0aG9yaXR5MRkwFwYDVQQDExBBY3J5cyBDb25zdWx0IENBMRswGQYJKoZIhvcNAQkBFgxj
YUBBY3J5cy5DT00wHhcNMTMwNzE3MDAwMDAwWhcNMTgwNzE3MDAwMDAwWjCBhTELMAkGA1UE
BhMCREUxFjAUBgNVBAoTDUFjcnlzIENvbnN1bHQxFzAVBgNVBAsTDkVtYWlsIFNlcnZpY2Vz
MR4wHAYDVQQDExVEci4tSW5nLiBGcmFuayBLYXJkZWwxJTAjBgkqhkiG9w0BCQEWFkZyYW5r
LkthcmRlbEBBY3J5cy5DT00wgd8wDQYJKoZIhvcNAQEBBQADgc0AMIHJAoHBANsLgwFOyXyi
07zzrUFimBvlBSSXq1rdd7R345QPAo80YuR/gI61yRwj0WrEGEbCgJSWyewiIPgxXi1q4yk5
jK0CWKtmXCRdM4tYls58nowoGOkmmnO8LVkzs9qjP6lkD+g4LuL/A0TtXPVuS2dmnobgQfMM
QTspzQggbQGvjE3ePT/QAK6fW3AHX9uWx4Gws+wBujlmalmHf+EN4Mii7/6/VpXOBN3UnN6m
4sreKl9wI93I+ueJ8BQW1NtrSys5uQIDAQABo4IB7TCCAekwCQYDVR0TBAIwADALBgNVHQ8E
BAMCBeAwHQYDVR0lBBYwFAYIKwYBBQUHAwQGCCsGAQUFBwMCMB0GA1UdDgQWBBT5kPNfW8Fy
l+O8f7cf1dBqtPnWzTCB+AYDVR0jBIHwMIHtgBRIYv7NBQzaGqHumifvz1Sib4mZjKGByaSB
xjCBwzELMAkGA1UEBhMCREUxDzANBgNVBAgTBkhlc3NlbjEzMDEGA1UEBxMqVGhlb2Rvci1I
ZXVzcy1TdHIuIDUzLTYzLCA2MTExOCBCYWQgVmlsYmVsMRYwFAYDVQQKEw1BY3J5cyBDb25z
dWx0MR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGTAXBgNVBAMTEEFjcnlzIENv
bnN1bHQgQ0ExGzAZBgkqhkiG9w0BCQEWDGNhQEFjcnlzLkNPTYIJAJv+XpZZVP2cMCEGA1Ud
EQQaMBiBFkZyYW5rLkthcmRlbEBBY3J5cy5DT00wCQYDVR0SBAIwADArBgNVHR8EJDAiMCCg
HqAchhovL3d3dy5hY3J5cy5jb20vY2EvY3JsLmNybDA7BggrBgEFBQcBAQQvMC0wKwYIKwYB
BQUHMAKGH2h0dHA6Ly93d3cuYWNyeXMuY29tL2NhL2NhLmh0bWwwDQYJKoZIhvcNAQEFBQAD
ggEBAF2ilQubzpWev5X+tb3n5e+YqLjCe4oPBEJMDiJ+wLoBejQmzoytxOGhwW77fCYCsn15
RaGpoNNMnJryRsFEokn76zwJaDbILXpA+CbAaZTYdc3EejQEuRjoYZL8iiAkKWZ7juMW/Y+r
UvkCik8XhQrfNmiUFGJbAtrMkopMx9ui/6WcImMhrCECZIFBegB1ubBcw+iSAuQM8CUPkJY0
5UowkMzAbQhPDpR//R5DTfcxcSojsQi3Iao6B6IFUOXA0oKzEsir/C3JQcdUCwnOSFj+U+Kp
T/Yrl7YW9edgbXV7nbpoFvHSyiMmlLfSE8Msb/drZ3qckqA0cowcfMgFrBwxggQ7MIIENwIB
ATCByTCBwzELMAkGA1UEBhMCREUxDzANBgNVBAgTBkhlc3NlbjEzMDEGA1UEBxMqVGhlb2Rv
ci1IZXVzcy1TdHIuIDUzLTYzLCA2MTExOCBCYWQgVmlsYmVsMRYwFAYDVQQKEw1BY3J5cyBD
b25zdWx0MR4wHAYDVQQLExVDZXJ0aWZpY2F0ZSBBdXRob3JpdHkxGTAXBgNVBAMTEEFjcnlz
IENvbnN1bHQgQ0ExGzAZBgkqhkiG9w0BCQEWDGNhQEFjcnlzLkNPTQIBCzAJBgUrDgMCGgUA
oIIChzAYBgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA4MDMx
MjI1NDNaMCMGCSqGSIb3DQEJBDEWBBR3QqgyjA7pNoEqjiR909pdBQsp1DBsBgkqhkiG9w0B
CQ8xXzBdMAsGCWCGSAFlAwQBKjALBglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcN
AwICAgCAMA0GCCqGSIb3DQMCAgFAMAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIHaBgkrBgEE
AYI3EAQxgcwwgckwgcMxCzAJBgNVBAYTAkRFMQ8wDQYDVQQIEwZIZXNzZW4xMzAxBgNVBAcT
KlRoZW9kb3ItSGV1c3MtU3RyLiA1My02MywgNjExMTggQmFkIFZpbGJlbDEWMBQGA1UEChMN
QWNyeXMgQ29uc3VsdDEeMBwGA1UECxMVQ2VydGlmaWNhdGUgQXV0aG9yaXR5MRkwFwYDVQQD
ExBBY3J5cyBDb25zdWx0IENBMRswGQYJKoZIhvcNAQkBFgxjYUBBY3J5cy5DT00CAQswgdwG
CyqGSIb3DQEJEAILMYHMoIHJMIHDMQswCQYDVQQGEwJERTEPMA0GA1UECBMGSGVzc2VuMTMw
MQYDVQQHEypUaGVvZG9yLUhldXNzLVN0ci4gNTMtNjMsIDYxMTE4IEJhZCBWaWxiZWwxFjAU
BgNVBAoTDUFjcnlzIENvbnN1bHQxHjAcBgNVBAsTFUNlcnRpZmljYXRlIEF1dGhvcml0eTEZ
MBcGA1UEAxMQQWNyeXMgQ29uc3VsdCBDQTEbMBkGCSqGSIb3DQEJARYMY2FAQWNyeXMuQ09N
AgELMA0GCSqGSIb3DQEBAQUABIHAlAxnDsqxkSQR+nAILqATxlnUr/TxPH4pd7+MEeJQn572
oVk4W4ofoHcvsGlKih1CW7+UykFLvBwV7ktJYZMq0H606U5lq+R3Nm8avnaNVQkiJE+B5A7k
E0NuMXlYQAXU/tnKocMxizgY5dzhfKH+TvMTmzRQnyyopyGAlrPSnSjaWUp4twGhTo66X5Lk
jjQKgr3hjJA+FCZK5yL0bBuBi2DFe8Ah392q7IKL9td0CWhGgljc6IApFkDCVtEdcXlYAAAA
AAAA
--------------ms060104030001040502070803--
From: Nick Hudson <skrll@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: kardel@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 03 Aug 2014 12:04:13 +0100
What's you full kernel config? Have you tried with LOCKDEBUG?
Thanks,
Nick
From: Frank Kardel <kardel@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 03 Aug 2014 17:14:32 +0200
Hmmm, LOCKDEBUG gives me:
Starting named.
panic: lockdebug_lookup: uninitialized lock (lock=0xc057e204, from=e8c9e670)
cpu0: Begin traceback...
0xdb3e3ba4: netbsd:db_panic+0xc
0xdb3e3bd4: netbsd:vpanic+0x144
0xdb3e3bec: netbsd:snprintf
0xdb3e3c34: netbsd:lockdebug_wantlock+0x1e8
0xdb3e3c7c: netbsd:mutex_enter+0x2b4
0xdb3e3cb4: crypto:cryptoopen+0x84
0xdb3e3ce4: netbsd:cdev_open+0xac
0xdb3e3d3c: netbsd:spec_open+0x1e8
0xdb3e3d5c: netbsd:VOP_OPEN+0x60
0xdb3e3e34: netbsd:vn_open+0x1f8
0xdb3e3eb4: netbsd:do_open+0xb0
0xdb3e3ee4: netbsd:do_sys_openat+0x7c
0xdb3e3f0c: netbsd:sys_open+0x34
0xdb3e3f7c: netbsd:syscall+0x8c
0xdb3e3fac: netbsd:swi_handler+0x98
cpu0: End traceback...
That's strange.
disabling named gets me past that. Strangely on an sysinst upgraded RPI
I cannot reproduce the issue, I thought...
But I bisected through the services running (namely named, mdnsd).
Collected reproduceable facts:
- It turns out that *running mdnsd* will trigger the locking issue.
- named triggers only a LOCKDEBUG (uninitialized lock) panic (see
above).
So it looks like mdnsd's handling of the interfaces will leave a lock on
the socket structure AFAICS.
Best regards
Frank
On 08/03/14 14:55, Nick Hudson wrote:
> The following reply was made to PR kern/49065; it has been noted by GNATS.
>
> From: Nick Hudson <skrll@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc: kardel@netbsd.org, kern-bug-people@netbsd.org,
> gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
> Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
> Date: Sun, 03 Aug 2014 12:04:13 +0100
>
> What's you full kernel config? Have you tried with LOCKDEBUG?
>
> Thanks,
> Nick
>
From: Nick Hudson <skrll@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Frank Kardel <kardel@netbsd.org>, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 03 Aug 2014 18:51:25 +0100
This is a multi-part message in MIME format.
--------------040700090605080508070403
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
On 08/03/14 16:15, Frank Kardel wrote:
> The following reply was made to PR kern/49065; it has been noted by GNATS.
>
> From: Frank Kardel <kardel@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
> Date: Sun, 03 Aug 2014 17:14:32 +0200
>
> Hmmm, LOCKDEBUG gives me:
>
> Starting named.
> panic: lockdebug_lookup: uninitialized lock (lock=0xc057e204, from=e8c9e670)
> cpu0: Begin traceback...
> 0xdb3e3ba4: netbsd:db_panic+0xc
> 0xdb3e3bd4: netbsd:vpanic+0x144
> 0xdb3e3bec: netbsd:snprintf
> 0xdb3e3c34: netbsd:lockdebug_wantlock+0x1e8
> 0xdb3e3c7c: netbsd:mutex_enter+0x2b4
> 0xdb3e3cb4: crypto:cryptoopen+0x84
> 0xdb3e3ce4: netbsd:cdev_open+0xac
> 0xdb3e3d3c: netbsd:spec_open+0x1e8
> 0xdb3e3d5c: netbsd:VOP_OPEN+0x60
> 0xdb3e3e34: netbsd:vn_open+0x1f8
> 0xdb3e3eb4: netbsd:do_open+0xb0
> 0xdb3e3ee4: netbsd:do_sys_openat+0x7c
> 0xdb3e3f0c: netbsd:sys_open+0x34
> 0xdb3e3f7c: netbsd:syscall+0x8c
> 0xdb3e3fac: netbsd:swi_handler+0x98
> cpu0: End traceback...
>
This one I think should be fixed with the attached diff.
> That's strange.
>
> disabling named gets me past that. Strangely on an sysinst upgraded RPI
> I cannot reproduce the issue, I thought...
>
> But I bisected through the services running (namely named, mdnsd).
>
> Collected reproduceable facts:
> - It turns out that *running mdnsd* will trigger the locking issue.
> - named triggers only a LOCKDEBUG (uninitialized lock) panic (see
> above).
>
> So it looks like mdnsd's handling of the interfaces will leave a lock on
> the socket structure AFAICS.
>
>
So, I reproduced locally and can see this...
db> ps/w
PID LID COMMAND EMUL PRI WAIT-MSG WAIT-CHANNEL
861 > 1 sed netbsd 25 0
790 1 ifconfig netbsd 41 tstile cbfc5fc0
948 1 sh netbsd 43 wait cbd8fa58
567 1 sh netbsd 38 wait cbd8f198
954 1 sh netbsd 41 wait cbd8f358
923 1 login netbsd 36 wait cbd8fc18
753 1 cron netbsd 35 nanoslp ca1cb6a0
624 1 inetd netbsd 37 kqueue cbf5085c
956 1 mdnsd netbsd 43 usbxfer cbdddbcc
909 1 qmgr netbsd 38 kqueue cac9ffd4
606 1 pickup netbsd 38 kqueue cbf507bc
831 1 master netbsd 38 kqueue cac9f854
619 1 sshd netbsd 39 select cbf34008
632 1 makemandb netbsd 24 0
437 1 syslogd netbsd 43 kqueue cbf500dc
435 1 dhcpcd netbsd 43 wait cbd8ea98
1 1 init netbsd 36 wait cbd8fdd8
0 48 system netbsd 96 nfsiod cbebceb4
0 47 system netbsd 96 nfsiod cbebc8cc
0 46 system netbsd 96 nfsiod cbebc85c
0 45 system netbsd 96 nfsiod cbebcd2c
0 44 system netbsd 123 physiod cb28d7dc
0 43 system netbsd 96 lnxcmplt cbd75720
0 42 system netbsd 125 aiodoned cbfb6334
0 41 system netbsd 124 syncer cbde85c0
0 40 system netbsd 126 pgdaemon c05b33c8
0 39 system netbsd 96 vcdata cbd7521c
0 38 system netbsd 96 semacv c05b4c68
0 37 system netbsd 96 semacv c05b4c50
0 36 system netbsd 96 semacv c05b4c38
0 32 system netbsd 96 usbevt cbe40570
0 31 system netbsd 96 unpgc c05b2aa8
0 30 system netbsd 125 vmem_rehash cbfb6814
0 29 system netbsd 96 mmctaskq cbef2834
0 20 system netbsd 96 usbtsk c05b2ba0
0 19 system netbsd 96 usbtsk c05b2b80
0 18 system netbsd 123 dwc2 cbfb6d54
0 17 system netbsd 96 iicintr cbfb70f8
0 16 system netbsd 96 iicintr cbfb7278
0 15 system netbsd 43 pmfsuspend cbfb7a74
0 14 system netbsd 43 pmfevent cbfb7ad4
0 13 system netbsd 96 sopendfr c05b2a6c
0 12 system netbsd 43 nfssilly cbfb7d14
0 11 system netbsd 125 cachegc cbf442c0
0 10 system netbsd 125 vrele c0550f80
0 9 system netbsd 125 vdrain c0550f40
0 8 system netbsd 125 mod_unld c0596e20
0 7 system netbsd 127 xcall c0443ac8
0 6 system netbsd 223 0
0 5 system netbsd 220 tstile cbfc5fc0
0 4 system netbsd 221 0
0 3 system netbsd 222 tstile cbfc5fc0
0 2 system netbsd 0 0
0 1 system netbsd 125 uvm c0546020
db> show lock cbfc5fc0
lock address : 0x00000000cbfc5fc0 type : sleep/adaptive
initialized : 0x00000000c014e044
shared holds : 0 exclusive: 1
shares wanted: 0 exclusive: 3
current cpu : 0 last held: 0
current lwp : 0x00000000cb3c93a0 last held: 0x00000000ca1cbc20
last locked* : 0x00000000c02ab654 unlocked : 0x00000000c02ac82c
owner field : 0x00000000ca1cbc20 wait/spin: 1/0
Turnstile chain at 0xc054fbc0.
=> Turnstile at 0xcbf43f48 (wrq=0xcbf43f58, rdq=0xcbf43f60).
=> 0 waiting readers:
=> 3 waiting writers: 0xcbf458c0 0xcbf45340 0xcad333c0
db> ps/l
PID LID S CPU FLAGS STRUCT LWP * NAME WAIT
861 > 1 7 0 0 cb3c93a0 sed
790 1 3 0 0 cad333c0 ifconfig tstile
948 1 3 0 80 cb3c9920 sh wait
567 1 3 0 80 cb3c8b60 sh wait
954 1 3 0 80 cb3c8e20 sh wait
923 1 3 0 80 cb3c9be0 login wait
753 1 3 0 80 ca1cb6a0 cron nanoslp
624 1 3 0 80 cad32600 inetd kqueue
956 1 3 0 0 ca1cbc20 mdnsd usbxfer
909 1 3 0 80 cad32080 qmgr kqueue
606 1 3 0 80 cb3c88a0 pickup kqueue
831 1 3 0 80 ca1cb960 master kqueue
619 1 3 0 80 cad32b80 sshd select
632 1 2 0 0 cb3c8320 makemandb
437 1 3 0 80 cad33940 syslogd kqueue
435 1 3 0 80 cb3c8060 dhcpcd wait
1 1 3 0 80 cbde9640 init wait
0 48 3 0 200 cb3c85e0 nfsio nfsiod
0 47 3 0 200 cad33100 nfsio nfsiod
0 46 3 0 200 cad32e40 nfsio nfsiod
0 45 3 0 200 cad33680 nfsio nfsiod
0 44 3 0 200 cbece5a0 physiod physiod
0 43 3 0 280 cbde8040 VCHIQka-0 lnxcmplt
0 42 3 0 200 cbde8300 aiodoned aiodoned
0 41 3 0 200 cbde85c0 ioflush syncer
0 40 3 0 200 cbde9bc0 pgdaemon pgdaemon
0 39 3 0 280 cbde8880 vcaudio vcdata
0 38 3 0 280 cbde8b40 VCHIQs-0 semacv
0 37 3 0 280 cbece2e0 VCHIQr-0 semacv
0 36 3 0 280 cbece020 VCHIQ-0 semacv
0 32 3 0 200 cbde9900 usb0 usbevt
0 31 3 0 200 cbde9380 unpgc unpgc
0 30 3 0 200 cbde90c0 vmem_rehash vmem_rehash
0 29 3 0 200 cbde8e00 sdmmc0 mmctaskq
0 20 3 0 200 cbece860 usbtask-dr usbtsk
0 19 3 0 200 cbeceb20 usbtask-hc usbtsk
0 18 3 0 200 cbecede0 dwc2 dwc2
0 17 3 0 200 cbecf0a0 iic1 iicintr
0 16 3 0 200 cbecf360 iic0 iicintr
0 15 3 0 200 cbecf620 pmfsuspend pmfsuspend
0 14 3 0 200 cbecf8e0 pmfevent pmfevent
0 13 3 0 200 cbecfba0 sopendfree sopendfr
0 12 3 0 200 cbf44000 nfssilly nfssilly
0 11 3 0 200 cbf442c0 cachegc cachegc
0 10 3 0 200 cbf44580 vrele vrele
0 9 3 0 200 cbf44840 vdrain vdrain
0 8 3 0 200 cbf44b00 modunload mod_unld
0 7 3 0 200 cbf44dc0 xcall/0 xcall
0 6 1 0 200 cbf45080 softser/0
0 5 3 0 200 cbf45340 softclk/0 tstile
0 4 1 0 200 cbf45600 softbio/0
0 3 3 0 200 cbf458c0 softnet/0 tstile
0 2 1 0 201 cbf45b80 idle/0
0 1 3 0 200 c0546020 swapper uvm
db> bt/a cbf458c0
trace: pid 0 lid 3 at 0xcbf3fe7c
0xcbf3fe7c: netbsd:mi_switch+0xc
0xcbf3feac: netbsd:sleepq_block+0xa4
0xcbf3feec: netbsd:turnstile_block+0x2f8
0xcbf3ff34: netbsd:mutex_enter+0x1c4
0xcbf3ff64: netbsd:route_intr+0x30
0xcbf3ffac: netbsd:softint_dispatch+0xec
Bad frame pointer: 0xcac89a74
db> bt/a cbf45340
trace: pid 0 lid 5 at 0xcbf3be4c
0xcbf3be4c: netbsd:mi_switch+0xc
0xcbf3be7c: netbsd:sleepq_block+0xa4
0xcbf3bebc: netbsd:turnstile_block+0x2f8
0xcbf3bf04: netbsd:mutex_enter+0x1c4
0xcbf3bf1c: netbsd:frag6_fasttimo+0x1c
0xcbf3bf34: netbsd:pffasttimo+0x50
0xcbf3bf64: netbsd:callout_softclock+0x18c
0xcbf3bfac: netbsd:softint_dispatch+0xec
Bad frame pointer: 0xcb2abd1c
db> bt/a cad333c0
trace: pid 790 lid 1 at 0xcac89cec
0xcac89cec: netbsd:mi_switch+0xc
0xcac89d1c: netbsd:sleepq_block+0xa4
0xcac89d5c: netbsd:turnstile_block+0x2f8
0xcac89da4: netbsd:mutex_enter+0x1c4
0xcac89dcc: netbsd:soclose+0x20
0xcac89de4: netbsd:soo_close+0x20
0xcac89e24: netbsd:closef+0x6c
0xcac89e6c: netbsd:fd_free+0x174
0xcac89eec: netbsd:exit1+0x100
0xcac89f0c: netbsd:sys_exit+0x3c
0xcac89f7c: netbsd:syscall+0x8c
0xcac89fac: netbsd:swi_handler+0x98
-- tf 0xbfffca44
db>
db> x/x softnet_lock
netbsd:softnet_lock: cbfc5fc0
They're all waiting on softnet_lock
Nick
--------------040700090605080508070403
Content-Type: text/plain; charset=us-ascii;
name="cryptodev.c.diff"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="cryptodev.c.diff"
Index: sys/opencrypto/cryptodev.c
===================================================================
RCS file: /cvsroot/src/sys/opencrypto/cryptodev.c,v
retrieving revision 1.79
diff -u -p -r1.79 cryptodev.c
--- sys/opencrypto/cryptodev.c 25 Jul 2014 08:10:40 -0000 1.79
+++ sys/opencrypto/cryptodev.c 3 Aug 2014 16:14:07 -0000
@@ -2085,6 +2085,8 @@ void cryptoattach(int);
void
cryptoattach(int num)
{
+ crypto_init();
+
pool_init(&fcrpl, sizeof(struct fcrypt), 0, 0, 0, "fcrpl",
NULL, IPL_NET); /* XXX IPL_NET ("splcrypto") */
pool_init(&csepl, sizeof(struct csession), 0, 0, 0, "csepl",
--------------040700090605080508070403--
From: Frank Kardel <kardel@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/49065: ifconfig tun0 ... sequence locks up system
Date: Sun, 03 Aug 2014 22:53:57 +0200
On 08/03/14 21:45, Nick Hudson wrote:
> The following reply was made to PR kern/49065; it has been noted by GNATS.
>
> [...snip..]
> trace: pid 0 lid 3 at 0xcbf3fe7c
> 0xcbf3fe7c: netbsd:mi_switch+0xc
> 0xcbf3feac: netbsd:sleepq_block+0xa4
> 0xcbf3feec: netbsd:turnstile_block+0x2f8
> 0xcbf3ff34: netbsd:mutex_enter+0x1c4
> 0xcbf3ff64: netbsd:route_intr+0x30
> 0xcbf3ffac: netbsd:softint_dispatch+0xec
> Bad frame pointer: 0xcac89a74
> db> bt/a cbf45340
> trace: pid 0 lid 5 at 0xcbf3be4c
> 0xcbf3be4c: netbsd:mi_switch+0xc
> 0xcbf3be7c: netbsd:sleepq_block+0xa4
> 0xcbf3bebc: netbsd:turnstile_block+0x2f8
> 0xcbf3bf04: netbsd:mutex_enter+0x1c4
> 0xcbf3bf1c: netbsd:frag6_fasttimo+0x1c
> 0xcbf3bf34: netbsd:pffasttimo+0x50
> 0xcbf3bf64: netbsd:callout_softclock+0x18c
> 0xcbf3bfac: netbsd:softint_dispatch+0xec
> Bad frame pointer: 0xcb2abd1c
> db> bt/a cad333c0
> trace: pid 790 lid 1 at 0xcac89cec
> 0xcac89cec: netbsd:mi_switch+0xc
> 0xcac89d1c: netbsd:sleepq_block+0xa4
> 0xcac89d5c: netbsd:turnstile_block+0x2f8
> 0xcac89da4: netbsd:mutex_enter+0x1c4
> 0xcac89dcc: netbsd:soclose+0x20
> 0xcac89de4: netbsd:soo_close+0x20
> 0xcac89e24: netbsd:closef+0x6c
> 0xcac89e6c: netbsd:fd_free+0x174
> 0xcac89eec: netbsd:exit1+0x100
> 0xcac89f0c: netbsd:sys_exit+0x3c
> 0xcac89f7c: netbsd:syscall+0x8c
> 0xcac89fac: netbsd:swi_handler+0x98
> -- tf 0xbfffca44
> db>
> db> x/x softnet_lock
> netbsd:softnet_lock: cbfc5fc0
>
>
> They're all waiting on softnet_lock
>
> Nick
>
>
Yes and mdnsd owns the lock. mdnsd (looking at my system) is sleeping here:
db> bt/a dac5fc20
trace: pid 3659 lid 1 at 0xdac61bb4
0xdac61bb4: netbsd:mi_switch+0xc
0xdac61be4: netbsd:sleepq_block+0xa4
0xdac61c1c: netbsd:cv_wait+0x114
0xdac61c44: netbsd:usbd_transfer+0x1e8
0xdac61c7c: netbsd:usbd_do_request_flags_pipe+0xd0
0xdac61ca4: netbsd:usbd_do_request+0x38
0xdac61cd4: netbsd:smsc_write_reg+0x60
0xdac61d04: netbsd:smsc_ioctl+0x110
0xdac61dac: netbsd:if_mcast_op+0x50
0xdac61dd4: netbsd:in_delmulti+0x70
0xdac61df4: netbsd:ip_freemoptions+0x30
0xdac61e14: netbsd:in_pcbdetach+0x58
0xdac61e3c: netbsd:soclose+0xf4
0xdac61e54: netbsd:soo_close+0x20
0xdac61e94: netbsd:closef+0x6c
0xdac61ef4: netbsd:fd_close+0x234
0xdac61f0c: netbsd:sys_close+0x2c
0xdac61f7c: netbsd:syscall+0x8c
0xdac61fac: netbsd:swi_handler+0x98
I cannot see yet why the usb transfer is not terminating, though.
State-Changed-From-To: open->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sun, 03 Aug 2014 22:13:05 +0000
State-Changed-Why:
Can you cvs up and try again?
From: "Mindaugas Rasiukevicius" <rmind@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/49065 CVS commit: src/sys/netinet
Date: Sun, 3 Aug 2014 22:11:50 +0000
Module Name: src
Committed By: rmind
Date: Sun Aug 3 22:11:50 UTC 2014
Modified Files:
src/sys/netinet: in_pcb.c
Log Message:
in_pcbdetach: not that IGMP and multicast groups are MP-safe, we can move
the ip_freemoptions() call outside the softnet_lock. Should fix PR/49065.
To generate a diff of this commit:
cvs rdiff -u -r1.149 -r1.150 src/sys/netinet/in_pcb.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Frank Kardel <kardel@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
Date: Mon, 04 Aug 2014 00:44:43 +0200
On 08/04/14 00:13, rmind@NetBSD.org wrote:
> Synopsis: ifconfig tun0 ... sequence locks up system
>
> State-Changed-From-To: open->feedback
> State-Changed-By: rmind@NetBSD.org
> State-Changed-When: Sun, 03 Aug 2014 22:13:05 +0000
> State-Changed-Why:
> Can you cvs up and try again?
>
>
I tried it, but it only survived to first duplicate tunnel config. The
second one got stuck again like this.
db> bt/a db8dab80
trace: pid 1697 lid 1 at 0xdb429b5c
0xdb429b5c: netbsd:mi_switch+0xc
0xdb429b8c: netbsd:sleepq_block+0xa4
0xdb429bc4: netbsd:cv_wait+0x114
0xdb429bec: netbsd:usbd_transfer+0x1e8
0xdb429c24: netbsd:usbd_do_request_flags_pipe+0xd0
0xdb429c4c: netbsd:usbd_do_request+0x38
0xdb429c7c: netbsd:smsc_write_reg+0x60
0xdb429cac: netbsd:smsc_ioctl+0x110
0xdb429d54: netbsd:if_mcast_op+0x50
0xdb429da4: netbsd:in6_delmulti+0x148
0xdb429dbc: netbsd:in6_leavegroup+0x20
0xdb429dd4: netbsd:ip6_freemoptions+0x3c
0xdb429df4: netbsd:in6_pcbdetach+0x74
0xdb429e14: netbsd:udp6_detach_wrapper+0x3c
0xdb429e3c: netbsd:soclose+0xf4
0xdb429e54: netbsd:soo_close+0x20
0xdb429e94: netbsd:closef+0x6c
0xdb429ef4: netbsd:fd_close+0x234
0xdb429f0c: netbsd:sys_close+0x2c
0xdb429f7c: netbsd:syscall+0x8c
0xdb429fac: netbsd:swi_handler+0x98
BTW: The crypto fix worked.
From: "Mindaugas Rasiukevicius" <rmind@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/49065 CVS commit: src/sys/netinet6
Date: Sun, 3 Aug 2014 22:55:24 +0000
Module Name: src
Committed By: rmind
Date: Sun Aug 3 22:55:24 UTC 2014
Modified Files:
src/sys/netinet6: in6_pcb.c
Log Message:
in6_pcbdetach: now that IGMP and multicast groups are MP-safe, we can move
the ip6_freemoptions() call outside the softnet_lock. Should fix PR/49065.
To generate a diff of this commit:
cvs rdiff -u -r1.126 -r1.127 src/sys/netinet6/in6_pcb.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Mindaugas Rasiukevicius <rmind@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Frank Kardel <kardel@netbsd.org>, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
Date: Sun, 3 Aug 2014 23:56:36 +0100
Frank Kardel <kardel@netbsd.org> wrote:
> The following reply was made to PR kern/49065; it has been noted by GNATS.
>
> From: Frank Kardel <kardel@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
> Date: Mon, 04 Aug 2014 00:44:43 +0200
>
> On 08/04/14 00:13, rmind@NetBSD.org wrote:
> > Synopsis: ifconfig tun0 ... sequence locks up system
> >
> > State-Changed-From-To: open->feedback
> > State-Changed-By: rmind@NetBSD.org
> > State-Changed-When: Sun, 03 Aug 2014 22:13:05 +0000
> > State-Changed-Why:
> > Can you cvs up and try again?
> >
> >
> I tried it, but it only survived to first duplicate tunnel config. The
> second one got stuck again like this.
IPv6 is a sad copy-paste code. Please cvs up and try again.
--
Mindaugas
From: Frank Kardel <kardel@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
Date: Mon, 04 Aug 2014 01:23:23 +0200
On 08/04/14 01:00, Mindaugas Rasiukevicius wrote:
> The following reply was made to PR kern/49065; it has been noted by GNATS.
>
> From: Mindaugas Rasiukevicius <rmind@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc: Frank Kardel <kardel@netbsd.org>, kern-bug-people@netbsd.org,
> gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
> Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
> Date: Sun, 3 Aug 2014 23:56:36 +0100
>
> Frank Kardel <kardel@netbsd.org> wrote:
> > The following reply was made to PR kern/49065; it has been noted by GNATS.
> >
> > From: Frank Kardel <kardel@netbsd.org>
> > To: gnats-bugs@NetBSD.org
> > Cc:
> > Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
> > Date: Mon, 04 Aug 2014 00:44:43 +0200
> >
> > On 08/04/14 00:13, rmind@NetBSD.org wrote:
> > > Synopsis: ifconfig tun0 ... sequence locks up system
> > >
> > > State-Changed-From-To: open->feedback
> > > State-Changed-By: rmind@NetBSD.org
> > > State-Changed-When: Sun, 03 Aug 2014 22:13:05 +0000
> > > State-Changed-Why:
> > > Can you cvs up and try again?
> > >
> > >
> > I tried it, but it only survived to first duplicate tunnel config. The
> > second one got stuck again like this.
>
> IPv6 is a sad copy-paste code. Please cvs up and try again.
>
> --
> Mindaugas
>
Now mdnsd is stuck at:
db> bt/a dac148e0
trace: pid 786 lid 1 at 0xd9f89b54
0xd9f89b54: netbsd:mi_switch+0xc
0xd9f89b84: netbsd:sleepq_block+0xa4
0xd9f89bbc: netbsd:cv_wait+0x114
0xd9f89be4: netbsd:usbd_transfer+0x1e8
0xd9f89c1c: netbsd:usbd_do_request_flags_pipe+0xd0
0xd9f89c44: netbsd:usbd_do_request+0x38
0xd9f89c74: netbsd:smsc_write_reg+0x60
0xd9f89ca4: netbsd:smsc_ioctl+0x110
0xd9f89d4c: netbsd:if_mcast_op+0x50
0xd9f89d9c: netbsd:in6_addmulti+0x1a8
0xd9f89dc4: netbsd:in6_joingroup+0x44
0xd9f89e4c: netbsd:ip6_ctloutput+0x137c
0xd9f89e74: netbsd:udp6_ctloutput+0x9c
0xd9f89ed4: netbsd:sosetopt+0x6c
0xd9f89f0c: netbsd:sys_setsockopt+0x84
0xd9f89f7c: netbsd:syscall+0x8c
0xd9f89fac: netbsd:swi_handler+0x98
db>
I can continue testing next weekend. $DAYJOB takes over.
From: Mindaugas Rasiukevicius <rmind@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Frank Kardel <kardel@netbsd.org>, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
Date: Mon, 4 Aug 2014 00:28:27 +0100
Frank Kardel <kardel@netbsd.org> wrote:
> Now mdnsd is stuck at:
> db> bt/a dac148e0
> trace: pid 786 lid 1 at 0xd9f89b54
> 0xd9f89b54: netbsd:mi_switch+0xc
> 0xd9f89b84: netbsd:sleepq_block+0xa4
> 0xd9f89bbc: netbsd:cv_wait+0x114
> 0xd9f89be4: netbsd:usbd_transfer+0x1e8
> 0xd9f89c1c: netbsd:usbd_do_request_flags_pipe+0xd0
> 0xd9f89c44: netbsd:usbd_do_request+0x38
> 0xd9f89c74: netbsd:smsc_write_reg+0x60
> 0xd9f89ca4: netbsd:smsc_ioctl+0x110
> 0xd9f89d4c: netbsd:if_mcast_op+0x50
> 0xd9f89d9c: netbsd:in6_addmulti+0x1a8
> 0xd9f89dc4: netbsd:in6_joingroup+0x44
> 0xd9f89e4c: netbsd:ip6_ctloutput+0x137c
> 0xd9f89e74: netbsd:udp6_ctloutput+0x9c
> 0xd9f89ed4: netbsd:sosetopt+0x6c
> 0xd9f89f0c: netbsd:sys_setsockopt+0x84
> 0xd9f89f7c: netbsd:syscall+0x8c
> 0xd9f89fac: netbsd:swi_handler+0x98
> db>
Nick is your friend here. The driver should not really block for a
long time with softnet_lock held.
--
Mindaugas
From: "Nick Hudson" <skrll@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/49065 CVS commit: src/sys/opencrypto
Date: Mon, 4 Aug 2014 14:17:19 +0000
Module Name: src
Committed By: skrll
Date: Mon Aug 4 14:17:19 UTC 2014
Modified Files:
src/sys/opencrypto: cryptodev.c
Log Message:
At least crypto_mtx needs initialisation here. Spotted during PR/49065
investigation.
To generate a diff of this commit:
cvs rdiff -u -r1.79 -r1.80 src/sys/opencrypto/cryptodev.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Frank Kardel <kardel@netbsd.org>
To: gnats-bugs@NetBSD.org, kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc:
Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
Date: Sun, 10 Aug 2014 09:55:55 +0200
On 08/04/14 01:30, Mindaugas Rasiukevicius wrote:
> The following reply was made to PR kern/49065; it has been noted by GNATS.
>
> From: Mindaugas Rasiukevicius <rmind@netbsd.org>
> To: gnats-bugs@NetBSD.org
> Cc: Frank Kardel <kardel@netbsd.org>, kern-bug-people@netbsd.org,
> gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
> Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system)
> Date: Mon, 4 Aug 2014 00:28:27 +0100
>
> Frank Kardel <kardel@netbsd.org> wrote:
> > Now mdnsd is stuck at:
> > db> bt/a dac148e0
> > trace: pid 786 lid 1 at 0xd9f89b54
> > 0xd9f89b54: netbsd:mi_switch+0xc
> > 0xd9f89b84: netbsd:sleepq_block+0xa4
> > 0xd9f89bbc: netbsd:cv_wait+0x114
> > 0xd9f89be4: netbsd:usbd_transfer+0x1e8
> > 0xd9f89c1c: netbsd:usbd_do_request_flags_pipe+0xd0
> > 0xd9f89c44: netbsd:usbd_do_request+0x38
> > 0xd9f89c74: netbsd:smsc_write_reg+0x60
> > 0xd9f89ca4: netbsd:smsc_ioctl+0x110
> > 0xd9f89d4c: netbsd:if_mcast_op+0x50
> > 0xd9f89d9c: netbsd:in6_addmulti+0x1a8
> > 0xd9f89dc4: netbsd:in6_joingroup+0x44
> > 0xd9f89e4c: netbsd:ip6_ctloutput+0x137c
> > 0xd9f89e74: netbsd:udp6_ctloutput+0x9c
> > 0xd9f89ed4: netbsd:sosetopt+0x6c
> > 0xd9f89f0c: netbsd:sys_setsockopt+0x84
> > 0xd9f89f7c: netbsd:syscall+0x8c
> > 0xd9f89fac: netbsd:swi_handler+0x98
> > db>
>
> Nick is your friend here. The driver should not really block for a
> long time with softnet_lock held.
>
> --
> Mindaugas
>
With the latest fixes I cannot replicate the issue an more. No idea what
i saw last weekend. So to issue is fixed for me.
State-Changed-From-To: feedback->closed
State-Changed-By: skrll@NetBSD.org
State-Changed-When: Sun, 10 Aug 2014 08:04:27 +0000
State-Changed-Why:
Confirmed fixed.
From: kardel@netbsd.org
To: gnats-bugs@NetBSD.org
Cc:
Subject: lockup: softnet_lock held across usb xfr (usmsc0/PR#49065 again)
Date: Sun, 21 Dec 2014 13:01:18 +0100 (CET)
>Submitter-Id: net
>Originator: Frank Kardel
>Organization:
>Confidential: no
>Synopsis: lockup: softnet_lock held across usb xfr (usmsc0/PR#49065 again)
>Severity: serious
>Priority: high
>Category: kern
>Class: sw-bug
>Release: NetBSD 7.99.3
>Environment:
System: NetBSD rpiahz 7.99.3 NetBSD 7.99.3 (RPISENSOR) #: Wed Dec 17 00:35:23 CET 2014 kardel@Andromeda:/usr/srccur/src/sys/arch/evbarm/compile/obj.evbarm/RPISENSOR evbarm
Architecture: earmv6hf
Machine: evbarm
>Description:
run mdnsd
ifconfig tun1 create
ifconfig tun1 10.201.1.2 10.201.1.1 netmask 0xffffffff
ifconfig tun1 10.201.1.2 10.201.1.1 netmask 0xffffffff
ifconfig tun1 10.201.1.2 10.201.1.1 netmask 0xffffffff
> LOCKUP - see status below
the mcast operations done by mdnsd keep the softnet_lock
while doing an usb xfr to usmsc0. This xfr does not terminate
and all network related code locks up in consequence.
This is the same pattern as in PR#49065.
>How-To-Repeat:
rpiahz# [insure mdnsd is running]
rpiahz# ifconfig tun1 create
rpiahz# ifconfig tun1 10.201.1.2 10.201.1.1 netmask 0xffffffff
rpiahz# ifconfig tun1 10.201.1.2 10.201.1.1 netmask 0xffffffff
rpiahz# ifconfig tun1 10.201.1.2 10.201.1.1 netmask 0xffffffff
~Stopped in pid 0.2 (system) at netbsd:cpu_Debugger+0x4: bx r14
db> ps
PID LID S CPU FLAGS STRUCT LWP * NAME WAIT
680 1 3 0 0 d9c87960 ifconfig tstile
2295 1 3 0 0 db9dc060 mdnsd usbxfer
3242 1 3 0 80 db9dc8a0 sh wait
105 1 3 0 80 db9dd660 login wait
96 1 3 0 80 d6da5400 cron nanoslp
3187 1 3 0 80 d6da56c0 inetd kqueue
2454 1 3 0 80 d9c86360 qmgr kqueue
2382 1 3 0 80 db9dc5e0 pickup kqueue
2973 1 3 0 80 d9c868e0 master kqueue
2199 1 3 0 80 d9c87120 sshd select
2284 1 3 0 80 d9c876a0 ntpd netio
1883 1 3 0 80 d9c873e0 ntpd pause
1525 1 3 0 80 d9c86e60 devpubd devmon
1233 1 3 0 80 db4ce340 perl select
633 4 3 0 0 db4ce8c0 named tstile
633 3 3 0 80 db4ceb80 named parked
633 2 3 0 80 db4cee40 named parked
633 1 3 0 80 db4cf100 named sigwait
606 1 3 0 80 db4cf3c0 syslogd kqueue
506 1 3 0 80 db4cf680 vtund select
388 1 3 0 0 db4cfc00 routed tstile
1 1 3 0 80 dbaf5640 init wait
0 48 3 0 200 db4ce080 onewire0 owidle
0 47 3 0 200 db4ce600 cryptoret crypto_w
0 46 3 0 200 dbe742e0 physiod physiod
0 45 3 0 280 db9dd920 VCHIQka-0 lnxcmplt
0 44 3 0 200 db9ddbe0 aiodoned aiodoned
0 43 3 0 200 dbaf4040 ioflush syncer
0 42 3 0 200 dbaf5bc0 pgdaemon pgdaemon
0 41 3 0 280 dbaf4300 vcaudio data
0 40 3 0 280 dbaf45c0 VCHIQs-0 semacv
0 39 3 0 280 dbaf4880 VCHIQr-0 semacv
0 38 3 0 280 dbe74020 VCHIQ-0 semacv
0 35 3 0 200 dbaf5900 usb0 usbevt
0 33 3 0 200 dbaf5380 unpgc unpgc
0 32 3 0 200 dbaf50c0 vmem_rehash vmem_rehash
0 31 3 0 200 dbaf4e00 vcmbox0 vcmbox0
0 30 3 0 200 dbaf4b40 sdmmc0 mmctaskq
0 21 3 0 200 dbe745a0 usbtask-dr usbtsk
0 20 3 0 200 dbe74860 usbtask-hc usbtsk
0 19 3 0 200 dbe74b20 dwc2 dwc2
0 18 3 0 200 dbe74de0 iic1 iicintr
0 17 3 0 200 dbe750a0 iic0 iicintr
0 16 3 0 200 dbe75360 sysmon smtaskq
0 15 3 0 200 dbe75620 pmfsuspend pmfsuspend
0 14 3 0 200 dbe758e0 pmfevent pmfevent
0 13 3 0 200 dbe75ba0 sopendfree sopendfr
0 12 3 0 200 dbf24000 nfssilly nfssilly
0 11 3 0 200 dbf242c0 cachegc cachegc
0 10 3 0 200 dbf24580 vrele vrele
0 9 3 0 200 dbf24840 vdrain vdrain
0 8 3 0 200 dbf24b00 modunload mod_unld
0 7 3 0 200 dbf24dc0 xcall/0 xcall
0 6 1 0 200 dbf25080 softser/0
0 5 3 0 200 dbf25340 softclk/0 tstile
0 4 1 0 200 dbf25600 softbio/0
0 3 3 0 200 dbf258c0 softnet/0 tstile
0 > 2 7 0 201 dbf25b80 idle/0
0 1 3 0 200 c046abe0 swapper uvm
db> bt/t 0t680
trace: pid 680 lid 1 at 0xd9635ce4
0xd9635ce4: netbsd:mi_switch+0xc
0xd9635d14: netbsd:sleepq_block+0xa4
0xd9635d5c: netbsd:turnstile_block+0x304
0xd9635da4: netbsd:mutex_enter+0x1a4
0xd9635dcc: netbsd:soclose+0x20
0xd9635de4: netbsd:soo_close+0x20
0xd9635e24: netbsd:closef+0x6c
0xd9635e6c: netbsd:fd_free+0x174
0xd9635ee4: netbsd:exit1+0x100
0xd9635f04: netbsd:sys_exit+0x3c
0xd9635f74: netbsd:syscall+0x8c
0xd9635fac: netbsd:swi_handler+0x9c
db> ps/w
PID LID COMMAND EMUL PRI WAIT-MSG WAIT-CHANNEL
680 1 ifconfig netbsd 43 tstile dbfaffc0
2295 1 mdnsd netbsd 43 usbxfer dbae7bcc
3242 1 sh netbsd 43 wait dba9f358
105 1 login netbsd 33 wait dba9fc18
96 1 cron netbsd 43 nanoslp d6da5400
3187 1 inetd netbsd 31 kqueue d9b0f854
2454 1 qmgr netbsd 39 kqueue dbf120dc
2382 1 pickup netbsd 43 kqueue d9b0fa34
2973 1 master netbsd 43 kqueue d9b0f8a4
2199 1 sshd netbsd 32 select dbf14008
2284 1 ntpd netbsd 191 netio db2d8238
1883 1 ntpd netbsd 191 pause d9c873e0
1525 1 devpubd netbsd 37 devmon c0552df8
1233 1 perl netbsd 43 select dbf14008
633 4 named netbsd 43 tstile dbfaffc0
633 3 named netbsd 43 parked 9b1ae560
633 2 named netbsd 43 parked 9b1b1560
633 1 named netbsd 39 sigwait db4cf274
606 1 syslogd netbsd 43 kqueue dbf133ec
506 1 vtund netbsd 44 select dbf14008
388 1 routed netbsd 43 tstile dbfaffc0
1 1 init netbsd 41 wait dba9fdd8
0 48 system netbsd 96 owidle db4ce080
0 47 system netbsd 96 crypto_w e88327c8
0 46 system netbsd 123 physiod db91765c
0 45 system netbsd 96 lnxcmplt dbaa11e0
0 44 system netbsd 125 aiodoned dbfa01b4
0 43 system netbsd 124 syncer dbaf4040
0 42 system netbsd 126 pgdaemon c056f948
0 41 system netbsd 123 data dbaa109c
0 40 system netbsd 96 semacv c0571224
0 39 system netbsd 96 semacv c057120c
0 38 system netbsd 96 semacv c05711f4
0 35 system netbsd 96 usbevt dbdf6570
0 33 system netbsd 96 unpgc c056efe8
0 32 system netbsd 125 vmem_rehash dbfa0754
0 31 system netbsd 43 vcmbox0 dbfa0934
0 30 system netbsd 96 mmctaskq dbe9e834
0 21 system netbsd 96 usbtsk c056f0ec
0 20 system netbsd 96 usbtsk c056f0cc
0 19 system netbsd 123 dwc2 dbfa0d54
0 18 system netbsd 96 iicintr dbfa1158
0 17 system netbsd 96 iicintr dbfa12d8
0 16 system netbsd 96 smtaskq c056d358
0 15 system netbsd 43 pmfsuspend dbfa1a74
0 14 system netbsd 43 pmfevent dbfa1ad4
0 13 system netbsd 96 sopendfr c056efac
0 12 system netbsd 43 nfssilly dbfa1d14
0 11 system netbsd 125 cachegc dbf242c0
0 10 system netbsd 125 vrele c054d300
0 9 system netbsd 125 vdrain c054d2c0
0 8 system netbsd 125 mod_unld c05531c0
0 7 system netbsd 127 xcall c04441b8
0 6 system netbsd 223 0
0 5 system netbsd 220 tstile dbfaffc0
0 4 system netbsd 221 0
0 3 system netbsd 222 tstile dbfaffc0
0 > 2 system netbsd 0 0
0 1 system netbsd 125 uvm c046abe0
db> show lock dbfaffc0
lock address : 0x00000000dbfaffc0 type : sleep/adaptive
initialized : 0x00000000c0147660
shared holds : 0 exclusive: 1
shares wanted: 0 exclusive: 5
current cpu : 0 last held: 0
current lwp : 0x00000000dbf25b80 last held: 0x00000000db9dc060
last locked* : 0x00000000c02ac36c unlocked : 0x00000000c02abdf8
owner field : 0x00000000db9dc060 wait/spin: 1/0
Turnstile chain at 0xc054bf40.
=> Turnstile at 0xdbf23f48 (wrq=0xdbf23f58, rdq=0xdbf23f60).
=> 0 waiting readers:
=> 5 waiting writers: 0xdbf258c0 0xdbf25340 0xdb4ce8c0 0xdb4cfc00 0xd9c87960
db> bt/t 0t2295
trace: pid 2295 lid 1 at 0xdb5e1b04
0xdb5e1b04: netbsd:mi_switch+0xc
0xdb5e1b34: netbsd:sleepq_block+0xa4
0xdb5e1b6c: netbsd:cv_wait+0x114
0xdb5e1ba4: netbsd:usbd_transfer+0x6b0
0xdb5e1bec: netbsd:usbd_do_request_flags_pipe+0x21c
0xdb5e1c14: netbsd:usbd_do_request+0x38
0xdb5e1c44: netbsd:smsc_write_reg+0x60
0xdb5e1c6c: netbsd:smsc_setmulti+0xfc
0xdb5e1c9c: netbsd:smsc_ioctl+0x110
0xdb5e1d44: netbsd:if_mcast_op+0x50
0xdb5e1d9c: netbsd:in6_delmulti+0x144
0xdb5e1db4: netbsd:in6_leavegroup+0x20
0xdb5e1dcc: netbsd:ip6_freemoptions+0x3c
0xdb5e1dec: netbsd:in6_pcbdetach+0xc8
0xdb5e1e0c: netbsd:udp6_detach_wrapper+0x3c
0xdb5e1e34: netbsd:soclose+0xf4
0xdb5e1e4c: netbsd:soo_close+0x20
0xdb5e1e8c: netbsd:closef+0x6c
0xdb5e1eec: netbsd:fd_close+0x234
0xdb5e1f04: netbsd:sys_close+0x2c
0xdb5e1f74: netbsd:syscall+
db> x/x softnet_lock
softnet_lock: dbfaffc0
db> bt/a 0xdbf258c0
trace: pid 0 lid 3 at 0xdbf1fe74
0xdbf1fe74: netbsd:mi_switch+0xc
0xdbf1fea4: netbsd:sleepq_block+0xa4
0xdbf1feec: netbsd:turnstile_block+0x304
0xdbf1ff34: netbsd:mutex_enter+0x1a4
0xdbf1ff64: netbsd:route_intr+0x30
0xdbf1ffac: netbsd:softint_dispatch+0xe4
Bad frame pointer: 0xd9635a6c
db> bt/a 0xdbf25340
trace: pid 0 lid 5 at 0xdbf1be5c
0xdbf1be5c: netbsd:mi_switch+0xc
0xdbf1be8c: netbsd:sleepq_block+0xa4
0xdbf1bed4: netbsd:turnstile_block+0x304
0xdbf1bf1c: netbsd:mutex_enter+0x1a4
0xdbf1bf34: netbsd:tcp_slowtimo+0x18
0xdbf1bf64: netbsd:callout_softclock+0x188
0xdbf1bfac: netbsd:softint_dispatch+0xe4
Bad frame pointer: 0xdb583c64
db> bt/a 0xdb4ce8c0
trace: pid 633 lid 4 at 0xdaa8fc5c
0xdaa8fc5c: netbsd:mi_switch+0xc
0xdaa8fc8c: netbsd:sleepq_block+0xa4
0xdaa8fcd4: netbsd:turnstile_block+0x304
0xdaa8fd1c: netbsd:mutex_enter+0x1a4
0xdaa8fd3c: netbsd:filt_soread+0x30
0xdaa8fed4: netbsd:kevent1+0x534
0xdaa8ff04: netbsd:sys___kevent50+0x48
0xdaa8ff74: netbsd:syscall+0x8c
0xdaa8ffac: netbsd:swi_handler+0x9c
db> bt/a 0xdb4cfc00
trace: pid 388 lid 1 at 0xdb583c34
0xdb583c34: netbsd:mi_switch+0xc
0xdb583c64: netbsd:sleepq_block+0xa4
0xdb583cac: netbsd:turnstile_block+0x304
0xdb583cf4: netbsd:mutex_enter+0x1a4
0xdb583d1c: netbsd:sopoll+0x24
0xdb583da4: netbsd:sel_do_scan+0x2f8
0xdb583eb4: netbsd:selcommon.part.2+0x130
0xdb583f04: netbsd:sys___select50+0x84
0xdb583f74: netbsd:syscall+0x8c
0xdb583fac: netbsd:swi_handler+0x9c
db> bt/a 0xd9c87960
trace: pid 680 lid 1 at 0xd9635ce4
0xd9635ce4: netbsd:mi_switch+0xc
0xd9635d14: netbsd:sleepq_block+0xa4
0xd9635d5c: netbsd:turnstile_block+0x304
0xd9635da4: netbsd:mutex_enter+0x1a4
0xd9635dcc: netbsd:soclose+0x20
0xd9635de4: netbsd:soo_close+0x20
0xd9635e24: netbsd:closef+0x6c
0xd9635e6c: netbsd:fd_free+0x174
0xd9635ee4: netbsd:exit1+0x100
0xd9635f04: netbsd:sys_exit+0x3c
0xd9635f74: netbsd:syscall+0x8c
0xd9635fac: netbsd:swi_handler+0x9c
>Fix:
avoid locking softnet_lock across usb operations
State-Changed-From-To: closed->open
State-Changed-By: kardel@NetBSD.org
State-Changed-When: Sun, 21 Dec 2014 12:09:16 +0000
State-Changed-Why:
issue re-appeared
Responsible-Changed-From-To: kern-bug-people->skrll
Responsible-Changed-By: skrll@NetBSD.org
Responsible-Changed-When: Tue, 10 May 2016 08:32:39 +0000
Responsible-Changed-Why:
Take
State-Changed-From-To: open->analyzed
State-Changed-By: skrll@NetBSD.org
State-Changed-When: Tue, 10 May 2016 08:32:39 +0000
State-Changed-Why:
Problem is understood - see 50491
State-Changed-From-To: analyzed->feedback
State-Changed-By: skrll@NetBSD.org
State-Changed-When: Thu, 05 Jan 2017 11:12:43 +0000
State-Changed-Why:
can this be closed noe?
State-Changed-From-To: feedback->closed
State-Changed-By: skrll@NetBSD.org
State-Changed-When: Fri, 06 Jan 2017 09:21:53 +0000
State-Changed-Why:
Fixed
From: Frank Kardel <kardel@netbsd.org>
To: gnats-bugs@NetBSD.org, skrll@NetBSD.org, netbsd-bugs@netbsd.org,
gnats-admin@netbsd.org
Cc:
Subject: Re: kern/49065 (ifconfig tun0 ... sequence locks up system / lockup:
softnet_lock held across usb xfr)
Date: Fri, 06 Jan 2017 10:20:29 +0100
Checked again with 7.99.52 - can be closed - works fine.
Frank
On 01/05/17 12:12, skrll@NetBSD.org wrote:
> Synopsis: ifconfig tun0 ... sequence locks up system / lockup: softnet_lock held across usb xfr
>
> State-Changed-From-To: analyzed->feedback
> State-Changed-By: skrll@NetBSD.org
> State-Changed-When: Thu, 05 Jan 2017 11:12:43 +0000
> State-Changed-Why:
> can this be closed noe?
>
>
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home