NetBSD Problem Report #55534

From clare@csel.org  Mon Aug  3 11:58:59 2020
Return-Path: <clare@csel.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 06FA61A9217
	for <gnats-bugs@gnats.NetBSD.org>; Mon,  3 Aug 2020 11:58:58 +0000 (UTC)
Message-Id: <20200803115852.9353F3874E@mail.csel.org>
Date: Mon,  3 Aug 2020 20:58:52 +0900 (JST)
From: clare@csel.org
Reply-To: clare@csel.org
To: gnats-bugs@NetBSD.org
Subject: ixg(4) does not recognize link-up state at boot time
X-Send-Pr-Version: 3.95

>Number:         55534
>Category:       kern
>Synopsis:       ixg(4) does not recognize link-up state at boot time
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    msaitoh
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Aug 03 12:00:00 +0000 2020
>Closed-Date:    Wed Aug 26 03:14:34 +0000 2020
>Last-Modified:  Wed Aug 26 03:14:34 +0000 2020
>Originator:     Shinichi Doyashiki
>Release:        NetBSD 9.99.69
>Organization:
	at home
>Environment:
System: NetBSD esperia.nas.csel.org 9.99.69 NetBSD 9.99.69 (ESPERIA) #1: Mon Aug  3 19:06:56 JST 2020  clare@esperia.nas.csel.org:/export/tmp/current/src/sys/arch/amd64/compile/ESPERIA amd64
Machine: amd64
>Description:
  the kernel as of version 9.99.69 with ixg(4) driver does not recognize link-up
  state of the SFP+ module attached to the Intel 82599 at boot time, in my hardware
  environment.

[     1.034623] ppb0: PCI Express capability version 2 <Root Port of PCI-E Root Complex> x8 @ 8.0GT/s
[     1.034623] ppb0: link is x8 @ 5.0GT/s
[     1.034623] pci1 at ppb0 bus 1
[     1.034623] pci1: i/o space, memory space enabled, rd/line, wr/inv ok
[     1.034623] ixg0 at pci1 dev 0 function 0: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 4.0.1-k
[     1.034623] ixg0: clearing prefetchable bit
[     1.034623] ixg0: device 82599EB
[     1.034623] ixg0: ETrackID 800003df
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 0, bound queue 0 to cpu 0
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 1, bound queue 1 to cpu 1
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 2, bound queue 2 to cpu 2
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 3, bound queue 3 to cpu 3
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 4, bound queue 4 to cpu 4
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 5, bound queue 5 to cpu 5
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 6, bound queue 6 to cpu 6
[     1.034623] ixg0: for TX/RX, interrupting at msix0 vec 7, bound queue 7 to cpu 7
[     1.034623] ixg0: for link, interrupting at msix0 vec 8, affinity to cpu 0
[     1.034623] ixg0: Using MSI-X interrupts with 9 vectors
[     1.034623] ixg0: Ethernet address 80:61:5f:**:**:**
[     1.034623] ixg0: PCI Express Bus: Speed 5.0GT/s Width x8
[     1.034623] ixg0: feature cap 0x1780<LEGACY_TX,FDIR,MSI,MSIX,LEGACY_IRQ>
[     1.034623] ixg0: feature ena 0x400<MSIX>
[     1.034623] ixg1 at pci1 dev 0 function 1: Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 4.0.1-k
[     1.034623] ixg1: clearing prefetchable bit
[     1.034623] ixg1: device 82599EB
[     1.034623] ixg1: ETrackID 800003df
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 0, bound queue 0 to cpu 0
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 1, bound queue 1 to cpu 1
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 2, bound queue 2 to cpu 2
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 3, bound queue 3 to cpu 3
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 4, bound queue 4 to cpu 4
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 5, bound queue 5 to cpu 5
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 6, bound queue 6 to cpu 6
[     1.034623] ixg1: for TX/RX, interrupting at msix1 vec 7, bound queue 7 to cpu 7
[     1.034623] ixg1: for link, interrupting at msix1 vec 8, affinity to cpu 0
[     1.034623] ixg1: Using MSI-X interrupts with 9 vectors
[     1.034623] ixg1: Ethernet address 80:61:5f:**:**:**
[     1.034623] ixg1: PCI Express Bus: Speed 5.0GT/s Width x8
[     1.034623] ixg1: feature cap 0x1780<LEGACY_TX,FDIR,MSI,MSIX,LEGACY_IRQ>
[     1.034623] ixg1: feature ena 0x400<MSIX>


  hot-plugging of the SFP+ module after boot was correctly recognized by the
  new kernel, and it was okay.

  plugged in the SFP+ module before boot (bad case):

ixg0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9000
	capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
	capabilities=7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
	capabilities=7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
	enabled=0
	ec_capabilities=f<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWFILTER>
	ec_enabled=7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
	address: 80:61:5f:**:**:**
	media: Ethernet autoselect (none)
	status: no carrier
	inet6 fe80::****:**ff:fe**:****%ixg0/64 flags 0x8<DETACHED> scopeid 0x1


  plugged in the SFP+ module after boot (good case):

ixg0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 9000
        capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
        capabilities=7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
        capabilities=7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
        enabled=0
        ec_capabilities=f<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWFILTER>
        ec_enabled=7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
        address: 80:61:5f:**:**:**
        media: Ethernet autoselect (10Gbase-Twinax full-duplex,rxpause,txpause)
        status: active
        inet6 fe80::****:**ff:fe**:****%ixg0/64 flags 0x0 scopeid 0x1


>How-To-Repeat:
1) install kernel with ixg(4) and the Intel 82599 NIC with
   SFP+ module(s).

2) configure /etc/ifconfig.ixg0 as usual (for boot time):
        mtu 9000
	up

3) re-boot the kernel.

4) run ifconfig(8) to review the link status.
	# ifconfig ixg0


>Fix:
  unknown yet.

>Release-Note:

>Audit-Trail:
From: Shinichi Doyashiki <clare@csel.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/55534: ixg(4) does not recognize link-up state at boot time
Date: Tue, 4 Aug 2020 11:14:07 +0900

 The kernel as of 2020-06-25 JST was the last known good kernel for my machine.
 The ixg(4) driver can be missing boot time initialization of SFP+ module related
 status bits.

 The following backouts are workarounds the problem.
 $ cvs update -j 1.7 -j 1.6 if_sriov.c
 $ cvs update -j 1.67 -j 1.66 ixgbe.h
 $ cvs update -j 1.233 -j 1.232 ixgbe.c
 $ cvs update -j 1.27 -j 1.26 ixgbe_osdep.h
 $ cvs update -j 1.15 -j 1.14 ixgbe_netbsd.c
 $ cvs update -j 1.151 -j 1.150 ixv.c


 --
 Shinichi Doyashiki <clare@csel.org>

Responsible-Changed-From-To: kern-bug-people->msaitoh
Responsible-Changed-By: msaitoh@NetBSD.org
Responsible-Changed-When: Wed, 05 Aug 2020 08:15:11 +0000
Responsible-Changed-Why:
mine.


From: "SAITOH Masanobu" <msaitoh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55534 CVS commit: src/sys/dev/pci/ixgbe
Date: Mon, 24 Aug 2020 18:42:17 +0000

 Module Name:	src
 Committed By:	msaitoh
 Date:		Mon Aug 24 18:42:17 UTC 2020

 Modified Files:
 	src/sys/dev/pci/ixgbe: ixgbe.c

 Log Message:
  The admin workqueue can be used even if the interface is not up.
 OK'd by thorpej@. Will fixes PR#55534 reported by Shinichi Doyashiki


 To generate a diff of this commit:
 cvs rdiff -u -r1.242 -r1.243 src/sys/dev/pci/ixgbe/ixgbe.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: msaitoh@NetBSD.org
State-Changed-When: Wed, 26 Aug 2020 03:14:34 +0000
State-Changed-Why:
Fixed in ixgbe.c rev. 1.243.
Thanks.


>Unformatted:

Home	PR Database Search