NetBSD Problem Report #23876

Received: (qmail 4620 invoked by uid 605); 24 Dec 2003 22:13:25 -0000
Message-Id: <200312242210.hBOMAPRr011058@socrates.olib.org>
Date: Wed, 24 Dec 2003 16:10:25 -0600 (CST)
From: rkr@olib.org
Sender: gnats-bugs-owner@NetBSD.org
Reply-To: rkr@olib.org
To: gnats-bugs@gnats.netbsd.org
Subject: ioapic(?) causes problems for my fxp0 NIC on amd64.
X-Send-Pr-Version: 3.95

>Number:         23876
>Category:       kern
>Synopsis:       ioapic(?) causes problems for my fxp0 NIC on amd64.
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Dec 24 22:14:00 +0000 2003
>Closed-Date:    
>Last-Modified:  Thu Mar 31 15:05:01 +0000 2011
>Originator:     Richard Rauch
>Release:        NetBSD 1.6ZG (recent CVS and custom kernel)
>Organization:
  "I probably don't know what I'm talking about."  http://www.olib.org/~rkr/
>Environment:
System: NetBSD socrates 1.6ZG NetBSD 1.6ZG (socrates) #0: Tue Dec 23 06:30:20 CST 2003 root@socrates:/usr/netbsd/current/src/sys/arch/amd64/compile/obj.amd64/socrates amd64
Architecture: x86_64
Machine: amd64
>Description:
	This may be amd64-based, but since the bus handling is supposed
	to be largely platform-independant, and the problems went away
	when I removed the ioapic, I am filing this as a kernel PR.

	I've been unable to use my fxp? Intel NIC.  I wasn't sure of the
	problem and assumed that it was caused by funky motherboard
	chipsets.  I was also having trouble with a ~working rtk? NIC
	(which seems like it may in some way have been clashing with the
	motherboard USB; depending on BIOS options and physical card
	positions (IRQs?), one or the other of USB or rtk0 would apparently
	lose an interrupt and the driver would effectively die).

	Symptoms:
		Several things seemed to get timeouts:

		fxp0 ALWAYS had timeouts on data receive, but could send
		packets.  It was effectively useless as far as I could tell.

		Depending, rtk0 would at RANDOM times get a "watchdog"
		timeout, at which point it would practically cease operation.
		It seemed to process about 1 network packet every 1 (or 10?)
		seconds, so that it was possible to gracefully break network
		connections (such as ssh or NFS) if you are patient.

		Alternately, USB would appear to completely die.  This
		would fail with a "usbd_setup_pipe" TIMEOUT error.  The
		TIMEOUT error would only be displayed after USB froze
		(my pointer device is a USB trackball) and then after
		(re)opening a USB device (e.g., exiting and restarting
		X).

	I never found a state where *both* USB and rtk0 would die.

	I never found a state where GENERIC would let me use fxp0.

	It was suggested to me that I try removing ioapic
	from my kernel (at the time a stock GENERIC amd64).  I also
	removed the MPBIOS-related options.

	Since that time, USB has not died.  rtk0 has not died.  fxp0
	WORKS.

	Here is a dmesg with ioapic enabled; it also includes the USB
	failure message:


NetBSD 1.6ZG (GENERIC) #6: Thu Dec 18 20:28:19 CST 2003
	root@socrates:/usr/netbsd/current/src/sys/arch/amd64/compile/obj.amd64/GENERIC
total memory = 511 MB
avail memory = 457 MB
using 6144 buffers containing 26292 KB of memory
mainbus0 (root)
mainbus0: Intel MP Specification (Version 1.4) (OEM00000 PROD00000000)
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Athlon(tm) 64 Processor 3200+, 1994.98 MHz
cpu0: features: e7dbfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features: e7dbfbff<PGE,MCA,CMOV,PAT,PSE36,MPC,NOX,MMXX,MMX>
cpu0: features: e7dbfbff<FXSR,SSE,SSE2,LONG,3DNOW2,3DNOW>
cpu0: I-cache 64 KB 64b/line 2-way, D-cache 64 KB 64b/line 2-way
cpu0: L2 cache 1 MB 64b/line 16-way
cpu0: ITLB 32 4 KB entries fully associative, 8 4 MB entries fully associative
cpu0: DTLB 32 4 KB entries fully associative, 8 4 MB entries fully associative
cpu0: calibrating local timer
cpu0: apic clock running at 199 MHz
cpu0: 16 page colors
mpbios: bus 0 is type PCI   
mpbios: bus 1 is type PCI   
mpbios: bus 2 is type PCI   
mpbios: bus 3 is type ISA   
ioapic0 at mainbus0 apid 2 (I/O APIC)
ioapic0: pa 0xfec00000, version 11, 24 pins
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: Nvidia Corporation nForce3 Host-PCI bridge (rev. 0xa4)
pcib0 at pci0 dev 1 function 0
pcib0: Nvidia Corporation nForce3 PCI-ISA bridge (rev. 0xa6)
Nvidia Corporation nForce3 SMBus controller (SMBus serial bus, revision 0xa4) at pci0 dev 1 function 1 not configured
ohci0 at pci0 dev 2 function 0: Nvidia Corporation nForce3 USB Host Controller (rev. 0xa5)
pci_intr_map: bus 0 dev 2 func 0 pin 1; line 9
pci_intr_map: no MP mapping found
ohci0: interrupting at irq 9
ohci0: OHCI version 1.0, legacy support
usb0 at ohci0: USB revision 1.0
uhub0 at usb0
uhub0: Nvidia Corporat OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 3 ports with 3 removable, self powered
ohci1 at pci0 dev 2 function 1: Nvidia Corporation nForce3 USB Host Controller (rev. 0xa5)
pci_intr_map: bus 0 dev 2 func 1 pin 2; line 9
pci_intr_map: no MP mapping found
ohci1: interrupting at irq 9
ohci1: OHCI version 1.0, legacy support
usb1 at ohci1: USB revision 1.0
uhub1 at usb1
uhub1: Nvidia Corporat OHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 3 ports with 3 removable, self powered
Nvidia Corporation nForce3 USB2 Host Controller (USB serial bus, interface 0x20, revision 0xa2) at pci0 dev 2 function 2 not configured
auich0 at pci0 dev 6 function 0: nForce3 MCP-T AC-97 Audio
auich0: interrupting at ioapic0 pin 11 (irq 11)
auich0: ac97: Avance Logic unknown (0x414c4780) codec; 20 bit DAC, 18 bit ADC, no 3D stereo
auich0: ac97: ext id 9c7<AC97_23,LDAC,SDAC,CDAC,SPDIF,DRA,VRA>
pciide0 at pci0 dev 8 function 0
pciide0: Nvidia Corporation nForce3 ATA133 IDE (rev. 0xa5)
pciide0: bus-master DMA support present, but unused (no driver support)
pciide0: primary channel configured to compatibility mode
pciide0: primary channel interrupting at ioapic0 pin 14 (irq 14)
atabus0 at pciide0 channel 0
pciide0: secondary channel configured to compatibility mode
pciide0: secondary channel interrupting at ioapic0 pin 15 (irq 15)
atabus1 at pciide0 channel 1
ppb0 at pci0 dev 10 function 0: Nvidia Corporation nforce3 PCI-PCI bridge (rev. 0xa2)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
fxp0 at pci1 dev 7 function 0: i82558 Ethernet, rev 5
fxp0: interrupting at ioapic0 pin 19 (irq 5)
fxp0: Ethernet address 00:03:47:72:7f:87
inphy0 at fxp0 phy 1: i82555 10/100 media interface, rev. 0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
rtk0 at pci1 dev 8 function 0: RealTek 8139 10/100BaseTX
rtk0: interrupting at ioapic0 pin 11 (irq 11)
rtk0: Ethernet address 00:90:47:05:bc:75
ukphy0 at rtk0 phy 7: Generic IEEE 802.3u media interface
ukphy0: OUI 0x000000, model 0x0000, rev. 0
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Realtek Semiconductor 8169 10/100/1000 Ethernet (ethernet network, revision 0x10) at pci1 dev 11 function 0 not configured
Integrated Technology Express, Inc. product 0x8212 (RAID mass storage, revision 0x11) at pci1 dev 12 function 0 not configured
Texas Instruments TSB43AA23 OHCI IEEE 1394 Host Controller (Firewire serial bus, interface 0x10) at pci1 dev 14 function 0 not configured
ppb1 at pci0 dev 11 function 0: Nvidia Corporation product 0x00d2 (rev. 0xa4)
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled
vga0 at pci2 dev 0 function 0: ATI Technologies Radeon 9200SE (rev. 0x01)
wsdisplay0 at vga0 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
pchb1 at pci0 dev 24 function 0
pchb1: Advanced Micro Devices AMD64 HyperTransport configuration (rev. 0x00)
pchb2 at pci0 dev 24 function 1
pchb2: Advanced Micro Devices AMD64 Address Map configuration (rev. 0x00)
pchb3 at pci0 dev 24 function 2
pchb3: Advanced Micro Devices AMD64 DRAM configuration (rev. 0x00)
pchb4 at pci0 dev 24 function 3
pchb4: Advanced Micro Devices AMD64 Miscellaneous configuration (rev. 0x00)
isa0 at pcib0
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
pckbc0 at isa0 port 0x60-0x64
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
lpt0 at isa0 port 0x378-0x37b irq 7
pcppi0 at isa0 port 0x61
midi0 at pcppi0: PC speaker
sysbeep0 at pcppi0
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
ioapic0: enabling
auich0: measured ac97 link rate at 16036 Hz, will use 48000 Hz
audio0 at auich0: full duplex, mmap, independent
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
Kernelized RAIDframe activated
uhidev0 at uhub0 port 3 configuration 1 interface 0
uhidev0: Microsoft Microsoft Trackball OpticalM-., rev 1.10/1.21, addr 2, iclass 3/1
ums0 at uhidev0: 5 buttons and Z dir.
wsmouse0 at ums0 mux 0
wd0 at atabus0 drive 0: <WDC WD400BB-75DKA0>
wd0: drive supports 16-sector PIO transfers, LBA48 addressing
wd0: 38146 MB, 77504 cyl, 16 head, 63 sec, 512 bytes/sect x 78125000 sectors
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
atapibus0 at atabus1: 2 targets
cd0 at atapibus0 drive 1: <LG       DVD-ROM DRD-8160B, , 1.01> cdrom removable
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
boot device: wd0
root on wd0a dumps on wd0b
WARNING: invalid time in clock chip
WARNING: CHECK AND RESET THE DATE!
root file system type: ffs
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x50, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)
fxp0: device timeout
fxp0: device timeout
usbd_setup_pipe: failed to start endpoint, TIMEOUT

>How-To-Repeat:
	On my system, try to use a GENERIC kernel with one of my
	"CINPRO100PCI-A" fxp0 cards, and/or try to use GENERIC with
	X and this particular rtk0 ethernet card for a protracted
	period.

>Fix:
	Dunno about a *fix*.  The workaround is to build a custom
	kernel without ioapic (and/or MPBIOS).
>Release-Note:
>Audit-Trail:

From: Richard Rauch <rkr@olib.org>
To: gnats-bugs@gnats.netbsd.org
Cc:  
Subject: Re: kern/23876: ioapic(?) causes problems for my fxp0 NIC on amd64.
Date: Tue, 30 Dec 2003 18:28:36 -0600

 I don't know if this is definitely related, or if nVidia is well known to
 have problems.  However:

 A GNU/LINUX user recently told me that he had similar problems with ioapic
 (had to turn it off in order to use some PCI hardware).

 He was using GNU/LINUX, *and* an nForce2 motherboard.  (I'm using nForce3.)

 The natural supposition is that nVidia is royally screwing up ioapic, if
 both NetBSD and GNU/LINUX find problems with hardware trying to live on
 ioapic.

 (Of course, I realize that two anecdotes do not make for conclusive evidence.
 (^&)


 -- 
   "I probably don't know what I'm talking about."  http://www.olib.org/~rkr/

From: Richard Rauch <rkr@olib.org>
To: gnats-bugs@netbsd.org
Cc:  
Subject: Re: kern/23876: ioapic(?) causes problems for my fxp0 NIC on amd64.
Date: Thu, 29 Jan 2004 15:41:58 -0600

 As an "interesting" followup to the previous comment from 20031230:

 I had a look at the GNU/LINUX kernel sources.  It seems that they are
 disabling ioapic if they identify an nVidia motherboard.

 While hardly proof that it can't be supported, it does suggest that it
 may be a bit more nasty than just my particular motherboard---and may
 not be easy to fix.


 Is there any evidence that anyone using nVidia-based motherboards
 can use ioapic?  Is it really needed for anything?

 Shaving it off of the INSTALL kernel, completely, might not be a bad
 idea, since (at least on AMD64) the INSTALL kernel is overlarge for
 the 2-floppy limit.  For GENERIC, it might not be a bad idea to do
 what the GNU/LINUX kernel does and disable ioapic on nVidia mother-
 boards unless someone has a workaround or fix.  The size issue prevents
 a full "./build.sh release" from completing without hacking something.
 (One person altered the allowed size of floppy images and added a
 third floppy.  But this won't work if you want to burn a bootable CD.)


 -- 
   "I probably don't know what I'm talking about."  http://www.olib.org/~rkr/
State-Changed-From-To: open->feedback
State-Changed-By: ad@NetBSD.org
State-Changed-When: Wed, 30 Apr 2008 14:34:39 +0000
State-Changed-Why:
Does the problem still occur in -current?


State-Changed-From-To: feedback->open
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 26 Oct 2008 23:50:02 +0000
State-Changed-Why:
regrettably, feedback will not be forthcoming.
Anyone else have this kind of configuration? (amd64, fxp0, nvidia motherboard)


From: "Jonathan A. Kollasch" <jakllsch@kollasch.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/23876 (ioapic(?) causes problems for my fxp0 NIC on
 amd64.)
Date: Thu, 31 Mar 2011 15:00:31 +0000

 I can still reproduce this issue with a netbsd-5 kernel.

>Unformatted:

Home	PR Database Search