NetBSD Problem Report #54837

From kardel@Kardel.name  Mon Jan  6 08:41:23 2020
Return-Path: <kardel@Kardel.name>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 399567A1A9
	for <gnats-bugs@gnats.NetBSD.org>; Mon,  6 Jan 2020 08:41:23 +0000 (UTC)
Message-Id: <20200106064616.1EDE344B47@Andromeda.Kardel.name>
Date: Mon,  6 Jan 2020 07:46:16 +0100 (CET)
From: kardel@netbsd.org
Reply-To: kardel@netbsd.org
To: gnats-bugs@NetBSD.org
Subject: ixl* + wm* interrupt allocation fails
X-Send-Pr-Version: 3.95

>Number:         54837
>Category:       kern
>Synopsis:       ixl* + wm* interrupt allocation fails
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jan 06 08:45:00 +0000 2020
>Closed-Date:    Mon Aug 03 08:43:39 +0000 2020
>Last-Modified:  Mon Aug 03 08:43:39 +0000 2020
>Originator:     Frank Kardel
>Release:        NetBSD 9.99.33
>Organization:

>Environment:


System: NetBSD abstest2 9.99.33 NetBSD 9.99.33 (ABST2GEN) #0: Fri Jan 3 16:14:02 CET 2020 kardel@:/src/NetBSD/cur/src/obj.amd64/sys/arch/amd64/compile/ABST2GEN amd64
Architecture: x86_64
Machine: amd64
>Description:
	On a system with Intel X722 10GbaseT and Intel I350 Gigabit interfaces the wm* interfaces
	fail allocate interrupts after the X722 interfaces where successfully initialized.

dmesg:
[...]
pci5 at mainbus0 bus 58
pci5: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
ppb3 at pci5 dev 0 function 0: Intel product 2030 (rev. 0x04)
ppb3: PCI Express capability version 2 <Root Port of PCI-E Root Complex> x16 @ 8.0GT/s
pci6 at ppb3 bus 59
pci6: i/o space, memory space enabled
ppb4 at pci6 dev 0 function 0: Intel C620 PCIe x16 Uplink (NPX16) (rev. 0x09)
ppb4: PCI Express capability version 2 <Upstream Port of PCI-E Switch>
pci7 at ppb4 bus 60
pci7: i/o space, memory space enabled
ppb5 at pci7 dev 3 function 0: Intel C620 Virtual Switch Port (for 10GbE LAN) (rev. 0x09)
ppb5: PCI Express capability version 2 <Downstream Port of PCI-E Switch> x1 @ 2.5GT/s
pci8 at ppb5 bus 61
pci8: i/o space, memory space enabled
ixl0 at pci8 dev 0 function 0: port 1, FW 3.1.55727 API 1.5
ixl0: Ethernet address ...
ixl0: interrupt at msix4 vec 0
ixl0: interrupt at msix4 vec 1
ixl0: interrupt at msix4 vec 2
ixl0: interrupt at msix4 vec 3
ixl0: interrupt at msix4 vec 4
allocated pic msix4 type edge pin 5 level 6 to cpu1 slot 0 idt entry 107
ixl0: interrupt at msix4 vec 5
allocated pic msix4 type edge pin 6 level 6 to cpu1 slot 1 idt entry 108
ixl0: interrupt at msix4 vec 6
allocated pic msix4 type edge pin 7 level 6 to cpu1 slot 2 idt entry 109
ixl0: interrupt at msix4 vec 7
allocated pic msix4 type edge pin 8 level 6 to cpu1 slot 3 idt entry 110
ixl0: interrupt at msix4 vec 8
allocated pic msix4 type edge pin 9 level 6 to cpu1 slot 4 idt entry 111
ixl0: interrupt at msix4 vec 9
allocated pic msix4 type edge pin 10 level 6 to cpu1 slot 5 idt entry 112
ixl0: interrupt at msix4 vec 10
allocated pic msix4 type edge pin 11 level 6 to cpu1 slot 6 idt entry 113
ixl0: interrupt at msix4 vec 11
allocated pic msix4 type edge pin 12 level 6 to cpu1 slot 7 idt entry 114
ixl0: interrupt at msix4 vec 12
allocated pic msix4 type edge pin 13 level 6 to cpu1 slot 8 idt entry 115
ixl0: interrupt at msix4 vec 13
allocated pic msix4 type edge pin 14 level 6 to cpu1 slot 9 idt entry 116
ixl0: interrupt at msix4 vec 14
allocated pic msix4 type edge pin 15 level 6 to cpu1 slot 10 idt entry 117
ixl0: interrupt at msix4 vec 15
allocated pic msix4 type edge pin 16 level 6 to cpu1 slot 11 idt entry 118
ixl0: interrupt at msix4 vec 16
allocated pic msix4 type edge pin 17 level 6 to cpu1 slot 12 idt entry 119
ixl0: interrupt at msix4 vec 17
allocated pic msix4 type edge pin 18 level 6 to cpu1 slot 13 idt entry 120
ixl0: interrupt at msix4 vec 18
allocated pic msix4 type edge pin 19 level 6 to cpu1 slot 14 idt entry 121
ixl0: interrupt at msix4 vec 19
allocated pic msix4 type edge pin 20 level 6 to cpu1 slot 15 idt entry 122
ixl0: interrupt at msix4 vec 20
allocated pic msix4 type edge pin 21 level 6 to cpu1 slot 16 idt entry 123
ixl0: interrupt at msix4 vec 21
allocated pic msix4 type edge pin 22 level 6 to cpu1 slot 17 idt entry 124
ixl0: interrupt at msix4 vec 22
allocated pic msix4 type edge pin 23 level 6 to cpu1 slot 18 idt entry 125
ixl0: interrupt at msix4 vec 23
allocated pic msix4 type edge pin 24 level 6 to cpu1 slot 19 idt entry 126
ixl0: interrupt at msix4 vec 24
allocated pic msix4 type edge pin 25 level 6 to cpu1 slot 20 idt entry 127
ixl0: interrupt at msix4 vec 25
allocated pic msix4 type edge pin 26 level 6 to cpu1 slot 21 idt entry 130
ixl0: interrupt at msix4 vec 26
allocated pic msix4 type edge pin 27 level 6 to cpu1 slot 22 idt entry 131
ixl0: interrupt at msix4 vec 27
allocated pic msix4 type edge pin 28 level 6 to cpu1 slot 23 idt entry 132
ixl0: interrupt at msix4 vec 28
allocated pic msix4 type edge pin 29 level 6 to cpu1 slot 29 idt entry 133
ixl0: interrupt at msix4 vec 29
allocated pic msix4 type edge pin 30 level 6 to cpu2 slot 0 idt entry 134
ixl0: interrupt at msix4 vec 30
allocated pic msix4 type edge pin 31 level 6 to cpu2 slot 1 idt entry 135
ixl0: interrupt at msix4 vec 31
allocated pic msix4 type edge pin 32 level 6 to cpu2 slot 2 idt entry 136
ixl0: interrupt at msix4 vec 32
ixl0: for TXRX0 interrupting at msix4 vec 1 affinity to 0
ixl0: for TXRX1 interrupting at msix4 vec 2
ixl0: for TXRX2 interrupting at msix4 vec 3 affinity to 2
ixl0: for TXRX3 interrupting at msix4 vec 4 affinity to 3
ixl0: for TXRX4 interrupting at msix4 vec 5 affinity to 4
ixl0: for TXRX5 interrupting at msix4 vec 6 affinity to 5
ixl0: for TXRX6 interrupting at msix4 vec 7 affinity to 6
ixl0: for TXRX7 interrupting at msix4 vec 8 affinity to 7
ixl0: for TXRX8 interrupting at msix4 vec 9 affinity to 8
ixl0: for TXRX9 interrupting at msix4 vec 10 affinity to 9
ixl0: for TXRX10 interrupting at msix4 vec 11 affinity to 10
ixl0: for TXRX11 interrupting at msix4 vec 12 affinity to 11
ixl0: for TXRX12 interrupting at msix4 vec 13 affinity to 12
ixl0: for TXRX13 interrupting at msix4 vec 14 affinity to 13
ixl0: for TXRX14 interrupting at msix4 vec 15 affinity to 14
ixl0: for TXRX15 interrupting at msix4 vec 16 affinity to 15
ixl0: for TXRX16 interrupting at msix4 vec 17 affinity to 16
ixl0: for TXRX17 interrupting at msix4 vec 18 affinity to 17
ixl0: for TXRX18 interrupting at msix4 vec 19 affinity to 18
ixl0: for TXRX19 interrupting at msix4 vec 20 affinity to 19
ixl0: for TXRX20 interrupting at msix4 vec 21 affinity to 20
ixl0: for TXRX21 interrupting at msix4 vec 22 affinity to 21
ixl0: for TXRX22 interrupting at msix4 vec 23 affinity to 22
ixl0: for TXRX23 interrupting at msix4 vec 24 affinity to 23
ixl0: for TXRX24 interrupting at msix4 vec 25 affinity to 24
ixl0: for TXRX25 interrupting at msix4 vec 26 affinity to 25
ixl0: for TXRX26 interrupting at msix4 vec 27 affinity to 26
ixl0: for TXRX27 interrupting at msix4 vec 28 affinity to 27
ixl0: for TXRX28 interrupting at msix4 vec 29 affinity to 28
ixl0: for TXRX29 interrupting at msix4 vec 30 affinity to 29
ixl0: for TXRX30 interrupting at msix4 vec 31 affinity to 30
ixl0: for TXRX31 interrupting at msix4 vec 32 affinity to 31
ixl0: for other interrupting at msix4 vec 0 affinity to 0
ixl1 at pci8 dev 0 function 1: port 0, FW 3.1.55727 API 1.5
ixl1: Ethernet address ...
ixl1: interrupt at msix5 vec 0
ixl1: interrupt at msix5 vec 1
allocated pic msix5 type edge pin 2 level 6 to cpu1 slot 0 idt entry 139
ixl1: interrupt at msix5 vec 2
allocated pic msix5 type edge pin 3 level 6 to cpu1 slot 1 idt entry 140
ixl1: interrupt at msix5 vec 3
allocated pic msix5 type edge pin 4 level 6 to cpu1 slot 2 idt entry 141
ixl1: interrupt at msix5 vec 4
allocated pic msix5 type edge pin 5 level 6 to cpu1 slot 3 idt entry 142
ixl1: interrupt at msix5 vec 5
allocated pic msix5 type edge pin 6 level 6 to cpu1 slot 4 idt entry 143
ixl1: interrupt at msix5 vec 6
allocated pic msix5 type edge pin 7 level 6 to cpu1 slot 5 idt entry 144
ixl1: interrupt at msix5 vec 7
allocated pic msix5 type edge pin 8 level 6 to cpu1 slot 6 idt entry 145
ixl1: interrupt at msix5 vec 8
allocated pic msix5 type edge pin 9 level 6 to cpu1 slot 7 idt entry 146
ixl1: interrupt at msix5 vec 9
allocated pic msix5 type edge pin 10 level 6 to cpu1 slot 8 idt entry 147
ixl1: interrupt at msix5 vec 10
allocated pic msix5 type edge pin 11 level 6 to cpu1 slot 9 idt entry 148
ixl1: interrupt at msix5 vec 11
allocated pic msix5 type edge pin 12 level 6 to cpu1 slot 10 idt entry 149
ixl1: interrupt at msix5 vec 12
allocated pic msix5 type edge pin 13 level 6 to cpu1 slot 11 idt entry 150
ixl1: interrupt at msix5 vec 13
allocated pic msix5 type edge pin 14 level 6 to cpu1 slot 12 idt entry 151
ixl1: interrupt at msix5 vec 14
allocated pic msix5 type edge pin 15 level 6 to cpu1 slot 13 idt entry 152
ixl1: interrupt at msix5 vec 15
allocated pic msix5 type edge pin 16 level 6 to cpu1 slot 14 idt entry 153
ixl1: interrupt at msix5 vec 16
allocated pic msix5 type edge pin 17 level 6 to cpu1 slot 15 idt entry 154
ixl1: interrupt at msix5 vec 17
allocated pic msix5 type edge pin 18 level 6 to cpu1 slot 16 idt entry 155
ixl1: interrupt at msix5 vec 18
allocated pic msix5 type edge pin 19 level 6 to cpu1 slot 17 idt entry 156
ixl1: interrupt at msix5 vec 19
allocated pic msix5 type edge pin 20 level 6 to cpu1 slot 18 idt entry 157
ixl1: interrupt at msix5 vec 20
allocated pic msix5 type edge pin 21 level 6 to cpu1 slot 19 idt entry 158
ixl1: interrupt at msix5 vec 21
allocated pic msix5 type edge pin 22 level 6 to cpu1 slot 20 idt entry 159
ixl1: interrupt at msix5 vec 22
allocated pic msix5 type edge pin 23 level 6 to cpu1 slot 21 idt entry 160
ixl1: interrupt at msix5 vec 23
allocated pic msix5 type edge pin 24 level 6 to cpu1 slot 22 idt entry 161
ixl1: interrupt at msix5 vec 24
allocated pic msix5 type edge pin 25 level 6 to cpu1 slot 23 idt entry 162
ixl1: interrupt at msix5 vec 25
allocated pic msix5 type edge pin 26 level 6 to cpu1 slot 29 idt entry 163
ixl1: interrupt at msix5 vec 26
allocated pic msix5 type edge pin 27 level 6 to cpu2 slot 0 idt entry 164
ixl1: interrupt at msix5 vec 27
allocated pic msix5 type edge pin 28 level 6 to cpu2 slot 1 idt entry 165
ixl1: interrupt at msix5 vec 28
allocated pic msix5 type edge pin 29 level 6 to cpu2 slot 2 idt entry 166
ixl1: interrupt at msix5 vec 29
allocated pic msix5 type edge pin 30 level 6 to cpu2 slot 4 idt entry 167
ixl1: interrupt at msix5 vec 30
allocated pic msix5 type edge pin 31 level 6 to cpu2 slot 5 idt entry 168
ixl1: interrupt at msix5 vec 31
allocated pic msix5 type edge pin 32 level 6 to cpu2 slot 6 idt entry 169
ixl1: interrupt at msix5 vec 32
ixl1: for TXRX0 interrupting at msix5 vec 1 affinity to 0
ixl1: for TXRX1 interrupting at msix5 vec 2 affinity to 1
ixl1: for TXRX2 interrupting at msix5 vec 3 affinity to 2
ixl1: for TXRX3 interrupting at msix5 vec 4 affinity to 3
ixl1: for TXRX4 interrupting at msix5 vec 5 affinity to 4
ixl1: for TXRX5 interrupting at msix5 vec 6 affinity to 5
ixl1: for TXRX6 interrupting at msix5 vec 7 affinity to 6
ixl1: for TXRX7 interrupting at msix5 vec 8 affinity to 7
ixl1: for TXRX8 interrupting at msix5 vec 9 affinity to 8
ixl1: for TXRX9 interrupting at msix5 vec 10 affinity to 9
ixl1: for TXRX10 interrupting at msix5 vec 11 affinity to 10
ixl1: for TXRX11 interrupting at msix5 vec 12 affinity to 11
ixl1: for TXRX12 interrupting at msix5 vec 13 affinity to 12
ixl1: for TXRX13 interrupting at msix5 vec 14 affinity to 13
ixl1: for TXRX14 interrupting at msix5 vec 15 affinity to 14
ixl1: for TXRX15 interrupting at msix5 vec 16 affinity to 15
ixl1: for TXRX16 interrupting at msix5 vec 17 affinity to 16
ixl1: for TXRX17 interrupting at msix5 vec 18 affinity to 17
ixl1: for TXRX18 interrupting at msix5 vec 19 affinity to 18
ixl1: for TXRX19 interrupting at msix5 vec 20 affinity to 19
ixl1: for TXRX20 interrupting at msix5 vec 21 affinity to 20
ixl1: for TXRX21 interrupting at msix5 vec 22 affinity to 21
ixl1: for TXRX22 interrupting at msix5 vec 23 affinity to 22
ixl1: for TXRX23 interrupting at msix5 vec 24 affinity to 23
ixl1: for TXRX24 interrupting at msix5 vec 25 affinity to 24
ixl1: for TXRX25 interrupting at msix5 vec 26 affinity to 25
ixl1: for TXRX26 interrupting at msix5 vec 27 affinity to 26
ixl1: for TXRX27 interrupting at msix5 vec 28 affinity to 27
ixl1: for TXRX28 interrupting at msix5 vec 29 affinity to 28
ixl1: for TXRX29 interrupting at msix5 vec 30 affinity to 29
ixl1: for TXRX30 interrupting at msix5 vec 31 affinity to 30
ixl1: for TXRX31 interrupting at msix5 vec 32 affinity to 31
ixl1: for other interrupting at msix5 vec 0 affinity to 0
[...]
pci9 at mainbus0 bus 93
pci9: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
ppb6 at pci9 dev 2 function 0: Intel product 2032 (rev. 0x04)
ppb6: PCI Express capability version 2 <Root Port of PCI-E Root Complex> x4 @ 8.0GT/s
pci10 at ppb6 bus 94
pci10: i/o space, memory space enabled
nvme0 at pci10 dev 0 function 0: Intel SSD DC P4500 (rev. 0x00)
nvme0: NVMe 1.2
allocated pic msix6 type edge pin 0 level 6 to cpu1 slot 1 idt entry 170
nvme0: for admin queue interrupting at msix6 vec 0
nvme0: INTEL SSDPE2KX040T8, firmware VDV10131, serial ...
allocated pic msix6 type edge pin 1 level 6 to cpu1 slot 2 idt entry 171
nvme0: for io queue 1 interrupting at msix6 vec 1 affinity to cpu0
allocated pic msix6 type edge pin 2 level 6 to cpu1 slot 2 idt entry 172
nvme0: for io queue 2 interrupting at msix6 vec 2 affinity to cpu1
allocated pic msix6 type edge pin 3 level 6 to cpu1 slot 3 idt entry 173
nvme0: for io queue 3 interrupting at msix6 vec 3 affinity to cpu2
allocated pic msix6 type edge pin 4 level 6 to cpu1 slot 3 idt entry 174
nvme0: for io queue 4 interrupting at msix6 vec 4 affinity to cpu3
allocated pic msix6 type edge pin 5 level 6 to cpu1 slot 3 idt entry 175
nvme0: for io queue 5 interrupting at msix6 vec 5 affinity to cpu4
allocated pic msix6 type edge pin 6 level 6 to cpu1 slot 3 idt entry 176
nvme0: for io queue 6 interrupting at msix6 vec 6 affinity to cpu5
allocated pic msix6 type edge pin 7 level 6 to cpu1 slot 3 idt entry 177
nvme0: for io queue 7 interrupting at msix6 vec 7 affinity to cpu6
allocated pic msix6 type edge pin 8 level 6 to cpu1 slot 3 idt entry 178
nvme0: for io queue 8 interrupting at msix6 vec 8 affinity to cpu7
allocated pic msix6 type edge pin 9 level 6 to cpu1 slot 3 idt entry 179
nvme0: for io queue 9 interrupting at msix6 vec 9 affinity to cpu8
allocated pic msix6 type edge pin 10 level 6 to cpu1 slot 3 idt entry 180
nvme0: for io queue 10 interrupting at msix6 vec 10 affinity to cpu9
allocated pic msix6 type edge pin 11 level 6 to cpu1 slot 3 idt entry 181
nvme0: for io queue 11 interrupting at msix6 vec 11 affinity to cpu10
allocated pic msix6 type edge pin 12 level 6 to cpu1 slot 3 idt entry 182
nvme0: for io queue 12 interrupting at msix6 vec 12 affinity to cpu11
allocated pic msix6 type edge pin 13 level 6 to cpu1 slot 3 idt entry 183
nvme0: for io queue 13 interrupting at msix6 vec 13 affinity to cpu12
allocated pic msix6 type edge pin 14 level 6 to cpu1 slot 3 idt entry 184
nvme0: for io queue 14 interrupting at msix6 vec 14 affinity to cpu13
allocated pic msix6 type edge pin 15 level 6 to cpu1 slot 3 idt entry 185
nvme0: for io queue 15 interrupting at msix6 vec 15 affinity to cpu14
allocated pic msix6 type edge pin 16 level 6 to cpu1 slot 3 idt entry 186
nvme0: for io queue 16 interrupting at msix6 vec 16 affinity to cpu15
allocated pic msix6 type edge pin 17 level 6 to cpu1 slot 3 idt entry 187
nvme0: for io queue 17 interrupting at msix6 vec 17 affinity to cpu16
allocated pic msix6 type edge pin 18 level 6 to cpu1 slot 3 idt entry 188
nvme0: for io queue 18 interrupting at msix6 vec 18 affinity to cpu17
allocated pic msix6 type edge pin 19 level 6 to cpu1 slot 3 idt entry 189
nvme0: for io queue 19 interrupting at msix6 vec 19 affinity to cpu18
allocated pic msix6 type edge pin 20 level 6 to cpu1 slot 3 idt entry 190
nvme0: for io queue 20 interrupting at msix6 vec 20 affinity to cpu19
allocated pic msix6 type edge pin 21 level 6 to cpu1 slot 3 idt entry 191
nvme0: for io queue 21 interrupting at msix6 vec 21 affinity to cpu20
allocated pic msix6 type edge pin 22 level 6 to cpu1 slot 3 idt entry 193
nvme0: for io queue 22 interrupting at msix6 vec 22 affinity to cpu21
allocated pic msix6 type edge pin 23 level 6 to cpu1 slot 3 idt entry 194
nvme0: for io queue 23 interrupting at msix6 vec 23 affinity to cpu22
allocated pic msix6 type edge pin 24 level 6 to cpu1 slot 3 idt entry 195
nvme0: for io queue 24 interrupting at msix6 vec 24 affinity to cpu23
allocated pic msix6 type edge pin 25 level 6 to cpu1 slot 3 idt entry 196
nvme0: for io queue 25 interrupting at msix6 vec 25 affinity to cpu24
allocated pic msix6 type edge pin 26 level 6 to cpu1 slot 3 idt entry 197
nvme0: for io queue 26 interrupting at msix6 vec 26 affinity to cpu25
allocated pic msix6 type edge pin 27 level 6 to cpu1 slot 3 idt entry 198
nvme0: for io queue 27 interrupting at msix6 vec 27 affinity to cpu26
allocated pic msix6 type edge pin 28 level 6 to cpu1 slot 3 idt entry 199
nvme0: for io queue 28 interrupting at msix6 vec 28 affinity to cpu27
allocated pic msix6 type edge pin 29 level 6 to cpu1 slot 3 idt entry 200
nvme0: for io queue 29 interrupting at msix6 vec 29 affinity to cpu28
allocated pic msix6 type edge pin 30 level 6 to cpu1 slot 3 idt entry 201
nvme0: for io queue 30 interrupting at msix6 vec 30 affinity to cpu29
allocated pic msix6 type edge pin 31 level 6 to cpu1 slot 3 idt entry 202
nvme0: for io queue 31 interrupting at msix6 vec 31 affinity to cpu30
allocated pic msix6 type edge pin 32 level 6 to cpu1 slot 3 idt entry 203
nvme0: for io queue 32 interrupting at msix6 vec 32 affinity to cpu31
ld0 at nvme0 nsid 1
ld0: 3726 GB, 486401 cyl, 255 head, 63 sec, 512 bytes/sect x 7814037168 sectors
ppb7 at pci9 dev 3 function 0: Intel product 2033 (rev. 0x04)
ppb7: PCI Express capability version 2 <Root Port of PCI-E Root Complex> x4 @ 8.0GT/s
pci11 at ppb7 bus 95
pci11: i/o space, memory space enabled
nvme1 at pci11 dev 0 function 0: Intel SSD DC P4500 (rev. 0x00)
nvme1: NVMe 1.2
allocated pic msix7 type edge pin 0 level 6 to cpu1 slot 3 idt entry 204
nvme1: for admin queue interrupting at msix7 vec 0
nvme1: INTEL SSDPE2KX040T8, firmware VDV10131, serial ...
allocated pic msix7 type edge pin 1 level 6 to cpu1 slot 4 idt entry 205
nvme1: for io queue 1 interrupting at msix7 vec 1 affinity to cpu0
allocated pic msix7 type edge pin 2 level 6 to cpu1 slot 4 idt entry 206
nvme1: for io queue 2 interrupting at msix7 vec 2 affinity to cpu1
allocated pic msix7 type edge pin 3 level 6 to cpu1 slot 5 idt entry 207
nvme1: for io queue 3 interrupting at msix7 vec 3 affinity to cpu2
allocated pic msix7 type edge pin 4 level 6 to cpu1 slot 5 idt entry 208
nvme1: for io queue 4 interrupting at msix7 vec 4 affinity to cpu3
allocated pic msix7 type edge pin 5 level 6 to cpu1 slot 5 idt entry 209
nvme1: for io queue 5 interrupting at msix7 vec 5 affinity to cpu4
allocated pic msix7 type edge pin 6 level 6 to cpu1 slot 5 idt entry 210
nvme1: for io queue 6 interrupting at msix7 vec 6 affinity to cpu5
allocated pic msix7 type edge pin 7 level 6 to cpu1 slot 5 idt entry 211
nvme1: for io queue 7 interrupting at msix7 vec 7 affinity to cpu6
allocated pic msix7 type edge pin 8 level 6 to cpu1 slot 5 idt entry 212
nvme1: for io queue 8 interrupting at msix7 vec 8 affinity to cpu7
allocated pic msix7 type edge pin 9 level 6 to cpu1 slot 5 idt entry 213
nvme1: for io queue 9 interrupting at msix7 vec 9 affinity to cpu8
allocated pic msix7 type edge pin 10 level 6 to cpu1 slot 5 idt entry 214
nvme1: for io queue 10 interrupting at msix7 vec 10 affinity to cpu9
allocated pic msix7 type edge pin 11 level 6 to cpu1 slot 5 idt entry 215
nvme1: for io queue 11 interrupting at msix7 vec 11 affinity to cpu10
allocated pic msix7 type edge pin 12 level 6 to cpu1 slot 5 idt entry 216
nvme1: for io queue 12 interrupting at msix7 vec 12 affinity to cpu11
allocated pic msix7 type edge pin 13 level 6 to cpu1 slot 5 idt entry 217
nvme1: for io queue 13 interrupting at msix7 vec 13 affinity to cpu12
allocated pic msix7 type edge pin 14 level 6 to cpu1 slot 5 idt entry 218
nvme1: for io queue 14 interrupting at msix7 vec 14 affinity to cpu13
allocated pic msix7 type edge pin 15 level 6 to cpu1 slot 5 idt entry 219
nvme1: for io queue 15 interrupting at msix7 vec 15 affinity to cpu14
allocated pic msix7 type edge pin 16 level 6 to cpu1 slot 5 idt entry 220
nvme1: for io queue 16 interrupting at msix7 vec 16 affinity to cpu15
allocated pic msix7 type edge pin 17 level 6 to cpu1 slot 5 idt entry 221
nvme1: for io queue 17 interrupting at msix7 vec 17 affinity to cpu16
allocated pic msix7 type edge pin 18 level 6 to cpu1 slot 5 idt entry 222
nvme1: for io queue 18 interrupting at msix7 vec 18 affinity to cpu17
allocated pic msix7 type edge pin 19 level 6 to cpu1 slot 5 idt entry 223
nvme1: for io queue 19 interrupting at msix7 vec 19 affinity to cpu18
allocated pic msix7 type edge pin 20 level 6 to cpu1 slot 5 idt entry 226
nvme1: for io queue 20 interrupting at msix7 vec 20 affinity to cpu19
allocated pic msix7 type edge pin 21 level 6 to cpu1 slot 5 idt entry 227
nvme1: for io queue 21 interrupting at msix7 vec 21 affinity to cpu20
allocated pic msix7 type edge pin 22 level 6 to cpu1 slot 5 idt entry 228
nvme1: for io queue 22 interrupting at msix7 vec 22 affinity to cpu21
allocated pic msix7 type edge pin 23 level 6 to cpu1 slot 5 idt entry 229
nvme1: for io queue 23 interrupting at msix7 vec 23 affinity to cpu22
allocated pic msix7 type edge pin 24 level 6 to cpu1 slot 5 idt entry 230
nvme1: for io queue 24 interrupting at msix7 vec 24 affinity to cpu23
allocated pic msix7 type edge pin 25 level 6 to cpu1 slot 5 idt entry 231
nvme1: for io queue 25 interrupting at msix7 vec 25 affinity to cpu24
allocated pic msix7 type edge pin 26 level 6 to cpu1 slot 5 idt entry 232
nvme1: for io queue 26 interrupting at msix7 vec 26 affinity to cpu25
allocated pic msix7 type edge pin 27 level 6 to cpu1 slot 5 idt entry 233
nvme1: for io queue 27 interrupting at msix7 vec 27 affinity to cpu26
allocated pic msix7 type edge pin 28 level 6 to cpu1 slot 5 idt entry 234
nvme1: for io queue 28 interrupting at msix7 vec 28 affinity to cpu27
allocated pic msix7 type edge pin 29 level 6 to cpu1 slot 5 idt entry 235
nvme1: for io queue 29 interrupting at msix7 vec 29 affinity to cpu28
allocated pic msix7 type edge pin 30 level 6 to cpu1 slot 5 idt entry 236
nvme1: for io queue 30 interrupting at msix7 vec 30 affinity to cpu29
allocated pic msix7 type edge pin 31 level 6 to cpu1 slot 5 idt entry 237
nvme1: for io queue 31 interrupting at msix7 vec 31 affinity to cpu30
allocated pic msix7 type edge pin 32 level 6 to cpu1 slot 5 idt entry 238
nvme1: for io queue 32 interrupting at msix7 vec 32 affinity to cpu31
ld1 at nvme1 nsid 1
ld1: 3726 GB, 486401 cyl, 255 head, 63 sec, 512 bytes/sect x 7814037168 sectors
[...]
PCI12 AT mainbus0 bus 128
pci12: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
[...]
pci13 at mainbus0 bus 133
pci13: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
[...]
pci14 at mainbus0 bus 174
pci14: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
ppb8 at pci14 dev 2 function 0: Intel product 2032 (rev. 0x04)
ppb8: PCI Express capability version 2 <Root Port of PCI-E Root Complex> x8 @ 8.0GT/s
ppb8: link is x4 @ 5.0GT/s
pci15 at ppb8 bus 175
pci15: i/o space, memory space enabled
wm0 at pci15 dev 0 function 0: I350 Gigabit Network Connection (rev. 0x01)
failed to allocate interrupt slot for PIC msix8 pin 0
wm0: autoconfiguration error: unable to establish MSI-X(for TX and RX) at msix8 vec 0
failed to allocate interrupt slot for PIC msi8 pin 0
wm0: autoconfiguration error: unable to establish MSI
failed to allocate interrupt slot for PIC ioapic7 pin 2
wm0: autoconfiguration error: unable to establish INTx
wm1 at pci15 dev 0 function 1: I350 Gigabit Network Connection (rev. 0x01)
failed to allocate interrupt slot for PIC msix8 pin 0
wm1: autoconfiguration error: unable to establish MSI-X(for TX and RX) at msix8 vec 0
failed to allocate interrupt slot for PIC msi8 pin 0
wm1: autoconfiguration error: unable to establish MSI
failed to allocate interrupt slot for PIC ioapic7 pin 5
wm1: autoconfiguration error: unable to establish INTx
[...]
	unconfiguring ixl* interface bring the wm* interfaces back to life

	Full dmesg output avaliable on request.
>How-To-Repeat:
	attempt to boot 9.99.33 on a larger system with X722 and I350 ethernet interfaces.

>Fix:
	disable X722 interface, but that defeats the purpose.

>Release-Note:

>Audit-Trail:

From: Masanobu SAITOH <msaitoh@execsw.org>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: msaitoh@execsw.org
Subject: Re: kern/54837: ixl* + wm* interrupt allocation fails
Date: Mon, 6 Jan 2020 18:23:55 +0900

 To mitigate this problem, x86's softint bitmask should be
 increased from 32bit to 64bit. Even if it's done, the problem
 won't be solved on many core systems. To resolve the problem,
 IDTVEC should be per-CPU.

 -- 
 -----------------------------------------------
                 SAITOH Masanobu (msaitoh@execsw.org
                                  msaitoh@netbsd.org)

From: Shoichi Yamaguchi <s.ymgch228@gmail.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, kardel@netbsd.org
Cc: 
Subject: Re: kern/54837: ixl* + wm* interrupt allocation fails
Date: Fri, 10 Jan 2020 14:48:11 +0900

 >>  To mitigate this problem, x86's softint bitmask should be
 > 
 > s/softint bitmask/imask, ipending and some others/
 > 
 >>  increased from 32bit to 64bit. Even if it's done, the problem
 >>  won't be solved on many core systems. To resolve the problem,
 >>  IDTVEC should be per-CPU.
 >>  

 This problem may be avoided by using "IXL_QUEUE_NUM" added at
 http://mail-index.netbsd.org/source-changes/2020/01/09/msg112737.html

 The kernel option can restrict resources related to queue pairs
 including an interrupt when the parameter sets to less than
 the number of cpus.

 This is a workaround, not a solution since the restriction causes the
 performance reduction.

 -- yamaguchi

State-Changed-From-To: open->closed
State-Changed-By: kardel@NetBSD.org
State-Changed-When: Mon, 03 Aug 2020 08:43:39 +0000
State-Changed-Why:
fixed with the per cpu interrupt allocation


>Unformatted:

Home	PR Database Search