NetBSD Problem Report #54984

From gson@gson.org  Wed Feb 19 07:30:56 2020
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 2A8481A9213
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 19 Feb 2020 07:30:56 +0000 (UTC)
Message-Id: <20200219073050.0B698253FA3@guava.gson.org>
Date: Wed, 19 Feb 2020 09:30:50 +0200 (EET)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: USB broken on Dell PowerEdge R630
X-Send-Pr-Version: 3.95

>Number:         54984
>Category:       kern
>Synopsis:       USB and SATA broken on Dell PowerEdge R630 in X2APIC mode
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Feb 19 07:35:00 +0000 2020
>Last-Modified:  Sat Mar 28 11:04:33 +0000 2020
>Originator:     Andreas Gustafsson
>Release:        NetBSD 9.0
>Organization:

>Environment:
System: NetBSD
Architecture: x86_64
Machine: amd64
>Description:

Booting the NetBSD/amd64 9.0 install image on a Dell PowerEdge R630
server from a USB flash drive, the bootloader and kernel load
successfully, but the kernel then fails to detect the flash drive or
any other USB devices.  The console output ends with:

  [   2.4139813] uhub0 at usb0: NetBSD (0000) xHCI root hub (0000), class 9/0, rev 3.00/1.00, addr 0
  [   2.5163526] uhub1 at usb1: NetBSD (0000) xHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 0
  [   2.6256832] uhub2 at usb3: NetBSD (0000) EHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 1
  [   2.7357183] uhub3 at usb2: NetBSD (0000) EHCI root hub (0000), class 9/0, rev 2.00/1.00, addr 1
  [   9.8680077] uhub1: device problem, disabling port 8
  [  15.8099151] uhub1: device problem, disabling port 12
  [  48.6004407] ipmi0: ID 32.1 IPMI 2.0 Available +SDRs
  [  68.6568787] uhub2: device problem, disabling port 1
  [  69.7072159] uhub3: device problem, disabling port 1
  [  69.7672353] WARNING: 27 errors while detecting hardware; check system log.
  [  69.8472608] boot device: <unknown>
  [  69.8872737] root device: 

The full console output is at:

  http://www.gson.org/netbsd/bugs/r630/dmesg-9.0-usb3.txt

This is with the flash drive and a USB keyboard plugged into the rear
USB ports and USB 3 enabled in the BIOS.  Disabling USB 3 does not help,
nor does disabling hyperthreading.

OpenBSD 6.6 and FreeBSD 12.1 install images also fail to boot with USB
related issues.  A Debian 10.3.0 install image boots successfully.

>How-To-Repeat:

>Fix:

>Release-Note:

>Audit-Trail:
From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@netbsd.org
Cc: gson@gson.org (Andreas Gustafsson)
Subject: Re: kern/54984: USB broken on Dell PowerEdge R630
Date: Thu, 20 Feb 2020 17:42:24 +0200

 I tried booting from a SATA hard disk containing an already-installed
 NetBSD-current system instead of the USB flash drive, and it also
 failed, with error messages including:

   wd0: IDENTIFY failed
   wd0: fixing 0 sector size
   wd0: secperunit and ncylinders are zero

 Maybe some kind of interrupt routing issue?
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sat, 7 Mar 2020 16:17:23 +0200

 Any suggestions for how to debug this?  It's a nice 24-core server
 tand it would be a real shame to have to run Linux on it...
 -- 
 Andreas Gustafsson, gson@gson.org

From: Christos Zoulas <christos@zoulas.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org,
 Andreas Gustafsson <gson@gson.org>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sat, 7 Mar 2020 09:28:08 -0500

 --Apple-Mail=_BD584598-A82B-441F-8827-640EFEE2929C
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain;
 	charset=us-ascii

 Did you try booting without ACPI?

 christos

 > On Mar 7, 2020, at 9:20 AM, Andreas Gustafsson <gson@gson.org> wrote:
 > 
 > The following reply was made to PR kern/54984; it has been noted by GNATS.
 > 
 > From: Andreas Gustafsson <gson@gson.org>
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
 > Date: Sat, 7 Mar 2020 16:17:23 +0200
 > 
 > Any suggestions for how to debug this?  It's a nice 24-core server
 > tand it would be a real shame to have to run Linux on it...
 > --
 > Andreas Gustafsson, gson@gson.org
 > 


 --Apple-Mail=_BD584598-A82B-441F-8827-640EFEE2929C
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCXmOveAAKCRBxESqxbLM7
 OrruAJ4u1kLvzTwCUH8n+fcw8ONPvxUwfwCaAxjoXriaZ+65su95jgYmoFe90+M=
 =5UAv
 -----END PGP SIGNATURE-----

 --Apple-Mail=_BD584598-A82B-441F-8827-640EFEE2929C--

From: Andreas Gustafsson <gson@gson.org>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sat, 7 Mar 2020 19:22:48 +0200

 Christos Zoulas wrote:
 > Did you try booting without ACPI?

 I have now.  The console output is at:

   http://www.gson.org/netbsd/bugs/r630/dmesg-9.0-noacpi.txt

 This is booting NetBSD-9.0-amd64-install.img configured for serial
 console from a SATA hard disk.  It does boot, but it's only detecting
 one of the 24 CPUs.

 What next?
 -- 
 Andreas Gustafsson, gson@gson.org

From: Christos Zoulas <christos@zoulas.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org,
 Andreas Gustafsson <gson@gson.org>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sat, 7 Mar 2020 14:29:57 -0500

 --Apple-Mail=_7A4563D0-7EAE-4AFC-810E-59C447FBF54D
 Content-Transfer-Encoding: 7bit
 Content-Type: text/plain;
 	charset=us-ascii

 Can you try -current?

 christos

 --Apple-Mail=_7A4563D0-7EAE-4AFC-810E-59C447FBF54D
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCXmP2NQAKCRBxESqxbLM7
 OnLSAKDnVBRqglqOLR4O49nYdfiythSRKACglW/16eCktPCXTHEHcHHEbaCfEqg=
 =HzPe
 -----END PGP SIGNATURE-----

 --Apple-Mail=_7A4563D0-7EAE-4AFC-810E-59C447FBF54D--

From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org, gson@gson.org (Andreas Gustafsson)
Subject: re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 08 Mar 2020 07:06:17 +1100

 can you run "vmstat -i" and see if the sata or usb interrupts are
 working?  eg, these devices:

 [   1.1108313] ahcisata0: interrupting at msi2 vec 0
 [   1.1108313] xhci0: interrupting at ioapic0 pin 19
 [   1.1108313] ahcisata1: interrupting at msi5 vec 0

 and also, do you know which controller is being used for sata?
 is is possible to switch them to the other controller for testing?


 .mrg.

From: Andreas Gustafsson <gson@gson.org>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 12:02:48 +0200

 Christos Zoulas wrote:
 > Can you try -current?

 Sure.  It fails the same way as 9.0.  A console log is at:

   http://www.gson.org/netbsd/bugs/r630/dmesg-2020.03.07.13.28.45.txt

 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: matthew green <mrg@eterna.com.au>
Cc: gnats-bugs@netbsd.org
Subject: re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 12:28:26 +0200

 matthew green wrote:
 >  can you run "vmstat -i" and see if the sata or usb interrupts are
 >  working?  eg, these devices:
 >  
 >  [   1.1108313] ahcisata0: interrupting at msi2 vec 0
 >  [   1.1108313] xhci0: interrupting at ioapic0 pin 19
 >  [   1.1108313] ahcisata1: interrupting at msi5 vec 0

 Not easily - the machine is unable to mount the root file system from
 either USB or SATA, so I don't get a shell.  A ramdisk kernel might
 work, but that will take some more time to set up.

 >  and also, do you know which controller is being used for sata?

 It's ahcisata0 according to the -current console log at
 http://www.gson.org/netbsd/bugs/r630/dmesg-2020.03.07.13.28.45.txt:

   [   1.1402831] atabus0 at ahcisata0 channel 0
   [...]
   [   4.6751558] wd0 at atabus0 drive 0
   [   9.4271407] uhub1: device problem, disabling port 12
   [  10.5476086] ehci_sync_hc: timed out
   [  11.2979219] wd0: IDENTIFY failed
   [  11.3344302] wd0: fixing 0 sector size
   [  11.3781107] wd0: secperunit and ncylinders are zero

 > is is possible to switch them to the other controller for testing?

 I moved the drive from the first drive bay to the last, and the
 console log then said:

   [   1.1300606] ahcisata1 at pci0 dev 31 function 2: Intel C61x/X99 SATA Controller (AHCI) (rev. 0x05)
   [   1.1300606] ahcisata1: AHCI revision 1.30, 6 ports, 32 slots, CAP 0xcb30ff45<EMS,PSC,SSC,PMD,ISS=0x3=Gen3,SCLO,SAL,SSS,SNCQ,S64A>
   [   1.1300606] ahcisata1: interrupting at msi5 vec 0
   [   1.1300606] atabus4 at ahcisata1 channel 0
   [   1.1300606] atabus5 at ahcisata1 channel 1
   [   1.1300606] atabus6 at ahcisata1 channel 2
   [   1.1300606] atabus7 at ahcisata1 channel 3
   [   1.1300606] atabus8 at ahcisata1 channel 4
   [   1.1300606] atabus9 at ahcisata1 channel 5
   [...]
   [   4.9633932] wd0 at atabus7 drive 0
   [   9.7054639] uhub1: device problem, disabling port 12
   [  10.8659706] ehci_sync_hc: timed out
   [  11.5662764] wd0: IDENTIFY failed
   [  11.6027838] wd0: fixing 0 sector size
   [  11.6464636] wd0: secperunit and ncylinders are zero

 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: matthew green <mrg@eterna.com.au>
Cc: gnats-bugs@netbsd.org
Subject: re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 13:12:43 +0200

 Earlier, I said: 
 > A ramdisk kernel might work, but that will take some more time to set up.

 I tried that by replacing by taking an USB stick containing a
 NetBSD/amd64 install image with the boot block configured for a serial
 console and replacing the kernel on it by an uncompressed -current
 netbsd-INSTALL.gz.  It booted, but at the point where it should have
 printed the first non-kernel message ("Created tmpfs /dev", etc), it
 just printed the letters "Ce" and did not produce any further serial
 console output, nor did it respond to serial console input:

   [  69.7412008] uhub2: device problem, disabling port 1
   [  70.8020093] ehci_sync_hc: timed out
   [  70.8416351] uhub3: device problem, disabling port 1
   [  70.8998752] WARNING: 30 errors while detecting hardware; check system log.
   [  70.9820803] boot device: <unknown>
   [  71.0225966] root on md0a dumps on md0b
   [  71.0641965] root file system type: ffs
   [  71.1120381] kern.module.path=/stand/amd64/9.99.48/modules
   Ce

 -- 
 Andreas Gustafsson, gson@gson.org

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
        Andreas Gustafsson <gson@gson.org>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 15:56:10 +0100

 On Sun, Mar 08, 2020 at 10:30:02AM +0000, Andreas Gustafsson wrote:
 > The following reply was made to PR kern/54984; it has been noted by GNATS.
 > 
 > From: Andreas Gustafsson <gson@gson.org>
 > To: matthew green <mrg@eterna.com.au>
 > Cc: gnats-bugs@netbsd.org
 > Subject: re: kern/54984: USB and SATA broken on Dell PowerEdge R630
 > Date: Sun, 8 Mar 2020 12:28:26 +0200
 > 
 >  matthew green wrote:
 >  >  can you run "vmstat -i" and see if the sata or usb interrupts are
 >  >  working?  eg, these devices:
 >  >  
 >  >  [   1.1108313] ahcisata0: interrupting at msi2 vec 0
 >  >  [   1.1108313] xhci0: interrupting at ioapic0 pin 19
 >  >  [   1.1108313] ahcisata1: interrupting at msi5 vec 0
 >  
 >  Not easily - the machine is unable to mount the root file system from
 >  either USB or SATA, so I don't get a shell.  A ramdisk kernel might
 >  work, but that will take some more time to set up.

 I think you can get them from ddb, with something like
 show event /i

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: Andreas Gustafsson <gson@gson.org>
To: Manuel Bouyer <bouyer@antioche.eu.org>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 19:16:50 +0200

 Manuel Bouyer wrote:
 > I think you can get them from ddb, with something like
 > show event /i

 I had no luck getting the serial console to respond to a break, but
 enabling the front panel NMI button in the BIOS and pressing it got me
 into ddb.  With the -current INSTALL kernel, it says:

   db{12}> show event/i 
   db_read_ptr: cannot find `allevents'

 With the -current GENERIC kernel, it says:

   db{8}> show event/i
   evcnt type 1: TLB shootdown = 96
   evcnt type 1: cpu0 timer = 6868
   db{8}>

   db{8}> show event
   evcnt type 0: bus_dma loads = 189
   evcnt type 0: bus_dma nbouncebufs = 24
   evcnt type 0: pserialize exclusive access = 61
   evcnt type 0: vmem static_bt_inuse = 200
   evcnt type 0: vmem static_bt_count = 200
   evcnt type 1: TLB shootdown = 96
   evcnt type 0: softint clk/0 = 751
   evcnt type 0: softint ser/0 = 83
   evcnt type 0: callout late/0 = 4
   evcnt type 0: crosscall unicast = 168
   evcnt type 0: namecache under scan target = 186
   evcnt type 1: cpu0 timer = 6868
   evcnt type 0: softint clk/1 = 1500
   evcnt type 0: softint ser/1 = 60
   evcnt type 0: cpu1 timer = 18631
   evcnt type 0: cpu1 AST IPI = 1
   evcnt type 0: cpu1 xcall IPI = 60
   evcnt type 0: softint clk/2 = 745
   evcnt type 0: softint ser/2 = 62
   evcnt type 0: cpu2 timer = 18624
   evcnt type 0: cpu2 AST IPI = 1
   evcnt type 0: cpu2 xcall IPI = 60
   evcnt type 0: softint clk/3 = 1860
   evcnt type 0: softint ser/3 = 62
   evcnt type 0: cpu3 timer = 18603
   evcnt type 0: cpu3 AST IPI = 1
   evcnt type 0: cpu3 xcall IPI = 60
   evcnt type 0: softint clk/4 = 1
   evcnt type 0: softint ser/4 = 61
   evcnt type 0: cpu4 timer = 18635
   evcnt type 0: cpu4 AST IPI = 1
   evcnt type 0: cpu4 xcall IPI = 60
   evcnt type 0: softint clk/5 = 401
   evcnt type 0: softint ser/5 = 60
   evcnt type 0: cpu5 timer = 18574
   evcnt type 0: cpu5 AST IPI = 1
   evcnt type 0: cpu5 xcall IPI = 60
   evcnt type 0: softint clk/6 = 3
   evcnt type 0: softint ser/6 = 61
   evcnt type 0: cpu6 timer = 18611
   evcnt type 0: cpu6 AST IPI = 1
   evcnt type 0: cpu6 xcall IPI = 60
   evcnt type 0: softint ser/7 = 60
   evcnt type 0: cpu7 timer = 18582
   evcnt type 0: cpu7 AST IPI = 2
   evcnt type 0: cpu7 xcall IPI = 60
   evcnt type 0: softint clk/8 = 4
   evcnt type 0: softint ser/8 = 60
   evcnt type 0: cpu8 timer = 18613
   evcnt type 0: cpu8 AST IPI = 2
   evcnt type 0: cpu8 xcall IPI = 60
   evcnt type 0: softint clk/9 = 1
   evcnt type 0: softint ser/9 = 61
   evcnt type 0: cpu9 timer = 18585
   evcnt type 0: cpu9 AST IPI = 2
   evcnt type 0: cpu9 xcall IPI = 60
   evcnt type 0: softint clk/10 = 102
   evcnt type 0: softint ser/10 = 60
   evcnt type 0: cpu10 timer = 18621
   evcnt type 0: cpu10 xcall IPI = 60
   evcnt type 0: softint clk/11 = 148
   evcnt type 0: softint ser/11 = 60
   evcnt type 0: cpu11 timer = 18581
   evcnt type 0: cpu11 xcall IPI = 60
   evcnt type 0: softint clk/12 = 102
   evcnt type 0: softint ser/12 = 60
   evcnt type 0: cpu12 timer = 18617
   evcnt type 0: cpu12 xcall IPI = 60
   evcnt type 0: softint clk/13 = 102
   evcnt type 0: softint ser/13 = 60
   evcnt type 0: cpu13 timer = 18584
   evcnt type 0: cpu13 xcall IPI = 60
   evcnt type 0: softint clk/14 = 103
   evcnt type 0: softint ser/14 = 61
   evcnt type 0: cpu14 timer = 18612
   evcnt type 0: cpu14 AST IPI = 1
   evcnt type 0: cpu14 xcall IPI = 60
   evcnt type 0: softint clk/15 = 102
   evcnt type 0: softint ser/15 = 60
   evcnt type 0: cpu15 timer = 18585
   evcnt type 0: cpu15 xcall IPI = 60
   evcnt type 0: softint clk/16 = 102
   evcnt type 0: softint ser/16 = 60
   evcnt type 0: cpu16 timer = 18603
   evcnt type 0: cpu16 AST IPI = 2
   evcnt type 0: cpu16 xcall IPI = 60
   evcnt type 0: softint ser/17 = 60
   evcnt type 0: cpu17 timer = 18587
   evcnt type 0: cpu17 xcall IPI = 60
   evcnt type 0: softint clk/18 = 1
   evcnt type 0: softint ser/18 = 61
   evcnt type 0: cpu18 timer = 18600
   evcnt type 0: cpu18 AST IPI = 2
   evcnt type 0: cpu18 xcall IPI = 60
   evcnt type 0: softint ser/19 = 60
   evcnt type 0: cpu19 timer = 18584
   evcnt type 0: cpu19 xcall IPI = 60
   evcnt type 0: softint clk/20 = 27
   evcnt type 0: softint ser/20 = 60
   evcnt type 0: cpu20 timer = 18580
   evcnt type 0: cpu20 xcall IPI = 60
   evcnt type 0: softint ser/21 = 60
   evcnt type 0: cpu21 timer = 18579
   evcnt type 0: cpu21 xcall IPI = 60
   evcnt type 0: softint ser/22 = 60
   evcnt type 0: cpu22 timer = 18592
   evcnt type 0: cpu22 xcall IPI = 60
   evcnt type 0: softint ser/23 = 60
   evcnt type 0: cpu23 timer = 18580
   evcnt type 0: cpu23 xcall IPI = 60
   evcnt type 0: acpicpu0 C1 (MWAIT) = 6845
   evcnt type 0: acpicpu1 C1 (MWAIT) = 18742
   evcnt type 0: acpicpu2 C1 (MWAIT) = 18754
   evcnt type 0: acpicpu3 C1 (MWAIT) = 18693
   evcnt type 0: acpicpu4 C1 (MWAIT) = 18737
   evcnt type 0: acpicpu5 C1 (MWAIT) = 18735
   evcnt type 0: acpicpu6 C1 (MWAIT) = 18725
   evcnt type 0: acpicpu7 C1 (MWAIT) = 18731
   evcnt type 0: acpicpu8 C1 (MWAIT) = 18708
   evcnt type 0: acpicpu9 C1 (MWAIT) = 18737
   evcnt type 0: acpicpu10 C1 (MWAIT) = 18722
   evcnt type 0: acpicpu11 C1 (MWAIT) = 18704
   evcnt type 0: acpicpu12 C1 (MWAIT) = 18699
   evcnt type 0: acpicpu13 C1 (MWAIT) = 18700
   evcnt type 0: acpicpu14 C1 (MWAIT) = 18726
   evcnt type 0: acpicpu15 C1 (MWAIT) = 18699
   evcnt type 0: acpicpu16 C1 (MWAIT) = 18721
   evcnt type 0: acpicpu17 C1 (MWAIT) = 18712
   evcnt type 0: acpicpu18 C1 (MWAIT) = 18739
   evcnt type 0: acpicpu19 C1 (MWAIT) = 18714
   evcnt type 0: acpicpu20 C1 (MWAIT) = 18683
   evcnt type 0: acpicpu21 C1 (MWAIT) = 18712
   evcnt type 0: acpicpu22 C1 (MWAIT) = 18715
   evcnt type 0: acpicpu23 C1 (MWAIT) = 18712
   evcnt type 0: kpreempt defer: kernel_lock = 28
   evcnt type 0: kpreempt immediate = 3
   db{8}>  

   db{8}> show event /f
   evcnt type 0: bus_dma bounces = 0
   evcnt type 0: bus_dma loads = 189
   evcnt type 0: bus_dma nbouncebufs = 24
   evcnt type 0: pdpolicy reactanon = 0
   evcnt type 0: pdpolicy reactfile = 0
   evcnt type 0: pdpolicy reactexec = 0
   evcnt type 0: vmcmd kills = 0
   evcnt type 0: vmcmd extends = 0
   evcnt type 0: vmcmd calls = 0
   evcnt type 0: pserialize exclusive access = 61
   evcnt type 0: vmem static_bt_inuse = 200
   evcnt type 0: vmem static_bt_count = 200
   evcnt type 0: rndpseudo open hard = 0
   evcnt type 0: rndpseudo open soft = 0
   evcnt type 1: TLB shootdown = 96
   evcnt type 0: x86 io bitmap copy = 0
   evcnt type 0: x86 ldt sync = 0
   evcnt type 0: softint net/0 = 0
   evcnt type 0: softint net block/0 = 0
   evcnt type 0: softint bio/0 = 0
   evcnt type 0: softint bio block/0 = 0
   evcnt type 0: softint clk/0 = 751
   evcnt type 0: softint clk block/0 = 0
   evcnt type 0: softint ser/0 = 83
   evcnt type 0: softint ser block/0 = 0
   evcnt type 0: callout late/0 = 4
   evcnt type 0: callout wait/0 = 0
   evcnt type 0: crosscall unicast = 168
   evcnt type 0: crosscall broadcast = 0
   evcnt type 0: namecache entries scanned = 0
   evcnt type 0: namecache entries collected = 0
   evcnt type 0: namecache over scan target = 0
   evcnt type 0: namecache under scan target = 186
   evcnt type 0: namecache forced reclaims = 0
   evcnt type 0: nfs timer = 0
   evcnt type 0: nfs timer start = 0
   evcnt type 0: nfs timer stop = 0
   evcnt type 1: cpu0 timer = 6868
   evcnt type 0: cpu0 halt IPI = 0
   evcnt type 0: cpu0 AST IPI = 0
   evcnt type 0: cpu0 generic IPI = 0
   evcnt type 0: cpu0 FPU synch IPI = 0
   evcnt type 0: cpu0 MTRR update IPI = 0
   evcnt type 0: cpu0 GDT update IPI = 0
   evcnt type 0: cpu0 xcall IPI = 0
   evcnt type 0: cpu0 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu0 kpreempt IPI = 0
   evcnt type 0: softint net/1 = 0
   evcnt type 0: softint net block/1 = 0
   evcnt type 0: softint bio/1 = 0
   evcnt type 0: softint bio block/1 = 0
   evcnt type 0: softint clk/1 = 1500
   evcnt type 0: softint clk block/1 = 0
   evcnt type 0: softint ser/1 = 60
   evcnt type 0: softint ser block/1 = 0
   evcnt type 0: callout late/1 = 0
   evcnt type 0: callout wait/1 = 0
   evcnt type 0: cpu1 timer = 18631
   evcnt type 0: cpu1 halt IPI = 0
   evcnt type 0: cpu1 AST IPI = 1
   evcnt type 0: cpu1 generic IPI = 0
   evcnt type 0: cpu1 FPU synch IPI = 0
   evcnt type 0: cpu1 MTRR update IPI = 0
   evcnt type 0: cpu1 GDT update IPI = 0
   evcnt type 0: cpu1 xcall IPI = 60
   evcnt type 0: cpu1 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu1 kpreempt IPI = 0
   evcnt type 0: softint net/2 = 0
   evcnt type 0: softint net block/2 = 0
   evcnt type 0: softint bio/2 = 0
   evcnt type 0: softint bio block/2 = 0
   evcnt type 0: softint clk/2 = 745
   evcnt type 0: softint clk block/2 = 0
   evcnt type 0: softint ser/2 = 62
   evcnt type 0: softint ser block/2 = 0
   evcnt type 0: callout late/2 = 0
   evcnt type 0: callout wait/2 = 0
   evcnt type 0: cpu2 timer = 18624
   evcnt type 0: cpu2 halt IPI = 0
   evcnt type 0: cpu2 AST IPI = 1
   evcnt type 0: cpu2 generic IPI = 0
   evcnt type 0: cpu2 FPU synch IPI = 0
   evcnt type 0: cpu2 MTRR update IPI = 0
   evcnt type 0: cpu2 GDT update IPI = 0
   evcnt type 0: cpu2 xcall IPI = 60
   evcnt type 0: cpu2 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu2 kpreempt IPI = 0
   evcnt type 0: softint net/3 = 0
   evcnt type 0: softint net block/3 = 0
   evcnt type 0: softint bio/3 = 0
   evcnt type 0: softint bio block/3 = 0
   evcnt type 0: softint clk/3 = 1860
   evcnt type 0: softint clk block/3 = 0
   evcnt type 0: softint ser/3 = 62
   evcnt type 0: softint ser block/3 = 0
   evcnt type 0: callout late/3 = 0
   evcnt type 0: callout wait/3 = 0
   evcnt type 0: cpu3 timer = 18603
   evcnt type 0: cpu3 halt IPI = 0
   evcnt type 0: cpu3 AST IPI = 1
   evcnt type 0: cpu3 generic IPI = 0
   evcnt type 0: cpu3 FPU synch IPI = 0
   evcnt type 0: cpu3 MTRR update IPI = 0
   evcnt type 0: cpu3 GDT update IPI = 0
   evcnt type 0: cpu3 xcall IPI = 60
   evcnt type 0: cpu3 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu3 kpreempt IPI = 0
   evcnt type 0: softint net/4 = 0
   evcnt type 0: softint net block/4 = 0
   evcnt type 0: softint bio/4 = 0
   evcnt type 0: softint bio block/4 = 0
   evcnt type 0: softint clk/4 = 1
   evcnt type 0: softint clk block/4 = 0
   evcnt type 0: softint ser/4 = 61
   evcnt type 0: softint ser block/4 = 0
   evcnt type 0: callout late/4 = 0
   evcnt type 0: callout wait/4 = 0
   evcnt type 0: cpu4 timer = 18635
   evcnt type 0: cpu4 halt IPI = 0
   evcnt type 0: cpu4 AST IPI = 1
   evcnt type 0: cpu4 generic IPI = 0
   evcnt type 0: cpu4 FPU synch IPI = 0
   evcnt type 0: cpu4 MTRR update IPI = 0
   evcnt type 0: cpu4 GDT update IPI = 0
   evcnt type 0: cpu4 xcall IPI = 60
   evcnt type 0: cpu4 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu4 kpreempt IPI = 0
   evcnt type 0: softint net/5 = 0
   evcnt type 0: softint net block/5 = 0
   evcnt type 0: softint bio/5 = 0
   evcnt type 0: softint bio block/5 = 0
   evcnt type 0: softint clk/5 = 401
   evcnt type 0: softint clk block/5 = 0
   evcnt type 0: softint ser/5 = 60
   evcnt type 0: softint ser block/5 = 0
   evcnt type 0: callout late/5 = 0
   evcnt type 0: callout wait/5 = 0
   evcnt type 0: cpu5 timer = 18574
   evcnt type 0: cpu5 halt IPI = 0
   evcnt type 0: cpu5 AST IPI = 1
   evcnt type 0: cpu5 generic IPI = 0
   evcnt type 0: cpu5 FPU synch IPI = 0
   evcnt type 0: cpu5 MTRR update IPI = 0
   evcnt type 0: cpu5 GDT update IPI = 0
   evcnt type 0: cpu5 xcall IPI = 60
   evcnt type 0: cpu5 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu5 kpreempt IPI = 0
   evcnt type 0: softint net/6 = 0
   evcnt type 0: softint net block/6 = 0
   evcnt type 0: softint bio/6 = 0
   evcnt type 0: softint bio block/6 = 0
   evcnt type 0: softint clk/6 = 3
   evcnt type 0: softint clk block/6 = 0
   evcnt type 0: softint ser/6 = 61
   evcnt type 0: softint ser block/6 = 0
   evcnt type 0: callout late/6 = 0
   evcnt type 0: callout wait/6 = 0
   evcnt type 0: cpu6 timer = 18611
   evcnt type 0: cpu6 halt IPI = 0
   evcnt type 0: cpu6 AST IPI = 1
   evcnt type 0: cpu6 generic IPI = 0
   evcnt type 0: cpu6 FPU synch IPI = 0
   evcnt type 0: cpu6 MTRR update IPI = 0
   evcnt type 0: cpu6 GDT update IPI = 0
   evcnt type 0: cpu6 xcall IPI = 60
   evcnt type 0: cpu6 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu6 kpreempt IPI = 0
   evcnt type 0: softint net/7 = 0
   evcnt type 0: softint net block/7 = 0
   evcnt type 0: softint bio/7 = 0
   evcnt type 0: softint bio block/7 = 0
   evcnt type 0: softint clk/7 = 0
   evcnt type 0: softint clk block/7 = 0
   evcnt type 0: softint ser/7 = 60
   evcnt type 0: softint ser block/7 = 0
   evcnt type 0: callout late/7 = 0
   evcnt type 0: callout wait/7 = 0
   evcnt type 0: cpu7 timer = 18582
   evcnt type 0: cpu7 halt IPI = 0
   evcnt type 0: cpu7 AST IPI = 2
   evcnt type 0: cpu7 generic IPI = 0
   evcnt type 0: cpu7 FPU synch IPI = 0
   evcnt type 0: cpu7 MTRR update IPI = 0
   evcnt type 0: cpu7 GDT update IPI = 0
   evcnt type 0: cpu7 xcall IPI = 60
   evcnt type 0: cpu7 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu7 kpreempt IPI = 0
   evcnt type 0: softint net/8 = 0
   evcnt type 0: softint net block/8 = 0
   evcnt type 0: softint bio/8 = 0
   evcnt type 0: softint bio block/8 = 0
   evcnt type 0: softint clk/8 = 4
   evcnt type 0: softint clk block/8 = 0
   evcnt type 0: softint ser/8 = 60
   evcnt type 0: softint ser block/8 = 0
   evcnt type 0: callout late/8 = 0
   evcnt type 0: callout wait/8 = 0
   evcnt type 0: cpu8 timer = 18613
   evcnt type 0: cpu8 halt IPI = 0
   evcnt type 0: cpu8 AST IPI = 2
   evcnt type 0: cpu8 generic IPI = 0
   evcnt type 0: cpu8 FPU synch IPI = 0
   evcnt type 0: cpu8 MTRR update IPI = 0
   evcnt type 0: cpu8 GDT update IPI = 0
   evcnt type 0: cpu8 xcall IPI = 60
   evcnt type 0: cpu8 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu8 kpreempt IPI = 0
   evcnt type 0: softint net/9 = 0
   evcnt type 0: softint net block/9 = 0
   evcnt type 0: softint bio/9 = 0
   evcnt type 0: softint bio block/9 = 0
   evcnt type 0: softint clk/9 = 1
   evcnt type 0: softint clk block/9 = 0
   evcnt type 0: softint ser/9 = 61
   evcnt type 0: softint ser block/9 = 0
   evcnt type 0: callout late/9 = 0
   evcnt type 0: callout wait/9 = 0
   evcnt type 0: cpu9 timer = 18585
   evcnt type 0: cpu9 halt IPI = 0
   evcnt type 0: cpu9 AST IPI = 2
   evcnt type 0: cpu9 generic IPI = 0
   evcnt type 0: cpu9 FPU synch IPI = 0
   evcnt type 0: cpu9 MTRR update IPI = 0
   evcnt type 0: cpu9 GDT update IPI = 0
   evcnt type 0: cpu9 xcall IPI = 60
   evcnt type 0: cpu9 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu9 kpreempt IPI = 0
   evcnt type 0: softint net/10 = 0
   evcnt type 0: softint net block/10 = 0
   evcnt type 0: softint bio/10 = 0
   evcnt type 0: softint bio block/10 = 0
   evcnt type 0: softint clk/10 = 102
   evcnt type 0: softint clk block/10 = 0
   evcnt type 0: softint ser/10 = 60
   evcnt type 0: softint ser block/10 = 0
   evcnt type 0: callout late/10 = 0
   evcnt type 0: callout wait/10 = 0
   evcnt type 0: cpu10 timer = 18621
   evcnt type 0: cpu10 halt IPI = 0
   evcnt type 0: cpu10 AST IPI = 0
   evcnt type 0: cpu10 generic IPI = 0
   evcnt type 0: cpu10 FPU synch IPI = 0
   evcnt type 0: cpu10 MTRR update IPI = 0
   evcnt type 0: cpu10 GDT update IPI = 0
   evcnt type 0: cpu10 xcall IPI = 60
   evcnt type 0: cpu10 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu10 kpreempt IPI = 0
   evcnt type 0: softint net/11 = 0
   evcnt type 0: softint net block/11 = 0
   evcnt type 0: softint bio/11 = 0
   evcnt type 0: softint bio block/11 = 0
   evcnt type 0: softint clk/11 = 148
   evcnt type 0: softint clk block/11 = 0
   evcnt type 0: softint ser/11 = 60
   evcnt type 0: softint ser block/11 = 0
   evcnt type 0: callout late/11 = 0
   evcnt type 0: callout wait/11 = 0
   evcnt type 0: cpu11 timer = 18581
   evcnt type 0: cpu11 halt IPI = 0
   evcnt type 0: cpu11 AST IPI = 0
   evcnt type 0: cpu11 generic IPI = 0
   evcnt type 0: cpu11 FPU synch IPI = 0
   evcnt type 0: cpu11 MTRR update IPI = 0
   evcnt type 0: cpu11 GDT update IPI = 0
   evcnt type 0: cpu11 xcall IPI = 60
   evcnt type 0: cpu11 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu11 kpreempt IPI = 0
   evcnt type 0: softint net/12 = 0
   evcnt type 0: softint net block/12 = 0
   evcnt type 0: softint bio/12 = 0
   evcnt type 0: softint bio block/12 = 0
   evcnt type 0: softint clk/12 = 102
   evcnt type 0: softint clk block/12 = 0
   evcnt type 0: softint ser/12 = 60
   evcnt type 0: softint ser block/12 = 0
   evcnt type 0: callout late/12 = 0
   evcnt type 0: callout wait/12 = 0
   evcnt type 0: cpu12 timer = 18617
   evcnt type 0: cpu12 halt IPI = 0
   evcnt type 0: cpu12 AST IPI = 0
   evcnt type 0: cpu12 generic IPI = 0
   evcnt type 0: cpu12 FPU synch IPI = 0
   evcnt type 0: cpu12 MTRR update IPI = 0
   evcnt type 0: cpu12 GDT update IPI = 0
   evcnt type 0: cpu12 xcall IPI = 60
   evcnt type 0: cpu12 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu12 kpreempt IPI = 0
   evcnt type 0: softint net/13 = 0
   evcnt type 0: softint net block/13 = 0
   evcnt type 0: softint bio/13 = 0
   evcnt type 0: softint bio block/13 = 0
   evcnt type 0: softint clk/13 = 102
   evcnt type 0: softint clk block/13 = 0
   evcnt type 0: softint ser/13 = 60
   evcnt type 0: softint ser block/13 = 0
   evcnt type 0: callout late/13 = 0
   evcnt type 0: callout wait/13 = 0
   evcnt type 0: cpu13 timer = 18584
   evcnt type 0: cpu13 halt IPI = 0
   evcnt type 0: cpu13 AST IPI = 0
   evcnt type 0: cpu13 generic IPI = 0
   evcnt type 0: cpu13 FPU synch IPI = 0
   evcnt type 0: cpu13 MTRR update IPI = 0
   evcnt type 0: cpu13 GDT update IPI = 0
   evcnt type 0: cpu13 xcall IPI = 60
   evcnt type 0: cpu13 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu13 kpreempt IPI = 0
   evcnt type 0: softint net/14 = 0
   evcnt type 0: softint net block/14 = 0
   evcnt type 0: softint bio/14 = 0
   evcnt type 0: softint bio block/14 = 0
   evcnt type 0: softint clk/14 = 103
   evcnt type 0: softint clk block/14 = 0
   evcnt type 0: softint ser/14 = 61
   evcnt type 0: softint ser block/14 = 0
   evcnt type 0: callout late/14 = 0
   evcnt type 0: callout wait/14 = 0
   evcnt type 0: cpu14 timer = 18612
   evcnt type 0: cpu14 halt IPI = 0
   evcnt type 0: cpu14 AST IPI = 1
   evcnt type 0: cpu14 generic IPI = 0
   evcnt type 0: cpu14 FPU synch IPI = 0
   evcnt type 0: cpu14 MTRR update IPI = 0
   evcnt type 0: cpu14 GDT update IPI = 0
   evcnt type 0: cpu14 xcall IPI = 60
   evcnt type 0: cpu14 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu14 kpreempt IPI = 0
   evcnt type 0: softint net/15 = 0
   evcnt type 0: softint net block/15 = 0
   evcnt type 0: softint bio/15 = 0
   evcnt type 0: softint bio block/15 = 0
   evcnt type 0: softint clk/15 = 102
   evcnt type 0: softint clk block/15 = 0
   evcnt type 0: softint ser/15 = 60
   evcnt type 0: softint ser block/15 = 0
   evcnt type 0: callout late/15 = 0
   evcnt type 0: callout wait/15 = 0
   evcnt type 0: cpu15 timer = 18585
   evcnt type 0: cpu15 halt IPI = 0
   evcnt type 0: cpu15 AST IPI = 0
   evcnt type 0: cpu15 generic IPI = 0
   evcnt type 0: cpu15 FPU synch IPI = 0
   evcnt type 0: cpu15 MTRR update IPI = 0
   evcnt type 0: cpu15 GDT update IPI = 0
   evcnt type 0: cpu15 xcall IPI = 60
   evcnt type 0: cpu15 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu15 kpreempt IPI = 0
   evcnt type 0: softint net/16 = 0
   evcnt type 0: softint net block/16 = 0
   evcnt type 0: softint bio/16 = 0
   evcnt type 0: softint bio block/16 = 0
   evcnt type 0: softint clk/16 = 102
   evcnt type 0: softint clk block/16 = 0
   evcnt type 0: softint ser/16 = 60
   evcnt type 0: softint ser block/16 = 0
   evcnt type 0: callout late/16 = 0
   evcnt type 0: callout wait/16 = 0
   evcnt type 0: cpu16 timer = 18603
   evcnt type 0: cpu16 halt IPI = 0
   evcnt type 0: cpu16 AST IPI = 2
   evcnt type 0: cpu16 generic IPI = 0
   evcnt type 0: cpu16 FPU synch IPI = 0
   evcnt type 0: cpu16 MTRR update IPI = 0
   evcnt type 0: cpu16 GDT update IPI = 0
   evcnt type 0: cpu16 xcall IPI = 60
   evcnt type 0: cpu16 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu16 kpreempt IPI = 0
   evcnt type 0: softint net/17 = 0
   evcnt type 0: softint net block/17 = 0
   evcnt type 0: softint bio/17 = 0
   evcnt type 0: softint bio block/17 = 0
   evcnt type 0: softint clk/17 = 0
   evcnt type 0: softint clk block/17 = 0
   evcnt type 0: softint ser/17 = 60
   evcnt type 0: softint ser block/17 = 0
   evcnt type 0: callout late/17 = 0
   evcnt type 0: callout wait/17 = 0
   evcnt type 0: cpu17 timer = 18587
   evcnt type 0: cpu17 halt IPI = 0
   evcnt type 0: cpu17 AST IPI = 0
   evcnt type 0: cpu17 generic IPI = 0
   evcnt type 0: cpu17 FPU synch IPI = 0
   evcnt type 0: cpu17 MTRR update IPI = 0
   evcnt type 0: cpu17 GDT update IPI = 0
   evcnt type 0: cpu17 xcall IPI = 60
   evcnt type 0: cpu17 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu17 kpreempt IPI = 0
   evcnt type 0: softint net/18 = 0
   evcnt type 0: softint net block/18 = 0
   evcnt type 0: softint bio/18 = 0
   evcnt type 0: softint bio block/18 = 0
   evcnt type 0: softint clk/18 = 1
   evcnt type 0: softint clk block/18 = 0
   evcnt type 0: softint ser/18 = 61
   evcnt type 0: softint ser block/18 = 0
   evcnt type 0: callout late/18 = 0
   evcnt type 0: callout wait/18 = 0
   evcnt type 0: cpu18 timer = 18600
   evcnt type 0: cpu18 halt IPI = 0
   evcnt type 0: cpu18 AST IPI = 2
   evcnt type 0: cpu18 generic IPI = 0
   evcnt type 0: cpu18 FPU synch IPI = 0
   evcnt type 0: cpu18 MTRR update IPI = 0
   evcnt type 0: cpu18 GDT update IPI = 0
   evcnt type 0: cpu18 xcall IPI = 60
   evcnt type 0: cpu18 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu18 kpreempt IPI = 0
   evcnt type 0: softint net/19 = 0
   evcnt type 0: softint net block/19 = 0
   evcnt type 0: softint bio/19 = 0
   evcnt type 0: softint bio block/19 = 0
   evcnt type 0: softint clk/19 = 0
   evcnt type 0: softint clk block/19 = 0
   evcnt type 0: softint ser/19 = 60
   evcnt type 0: softint ser block/19 = 0
   evcnt type 0: callout late/19 = 0
   evcnt type 0: callout wait/19 = 0
   evcnt type 0: cpu19 timer = 18584
   evcnt type 0: cpu19 halt IPI = 0
   evcnt type 0: cpu19 AST IPI = 0
   evcnt type 0: cpu19 generic IPI = 0
   evcnt type 0: cpu19 FPU synch IPI = 0
   evcnt type 0: cpu19 MTRR update IPI = 0
   evcnt type 0: cpu19 GDT update IPI = 0
   evcnt type 0: cpu19 xcall IPI = 60
   evcnt type 0: cpu19 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu19 kpreempt IPI = 0
   evcnt type 0: softint net/20 = 0
   evcnt type 0: softint net block/20 = 0
   evcnt type 0: softint bio/20 = 0
   evcnt type 0: softint bio block/20 = 0
   evcnt type 0: softint clk/20 = 27
   evcnt type 0: softint clk block/20 = 0
   evcnt type 0: softint ser/20 = 60
   evcnt type 0: softint ser block/20 = 0
   evcnt type 0: callout late/20 = 0
   evcnt type 0: callout wait/20 = 0
   evcnt type 0: cpu20 timer = 18580
   evcnt type 0: cpu20 halt IPI = 0
   evcnt type 0: cpu20 AST IPI = 0
   evcnt type 0: cpu20 generic IPI = 0
   evcnt type 0: cpu20 FPU synch IPI = 0
   evcnt type 0: cpu20 MTRR update IPI = 0
   evcnt type 0: cpu20 GDT update IPI = 0
   evcnt type 0: cpu20 xcall IPI = 60
   evcnt type 0: cpu20 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu20 kpreempt IPI = 0
   evcnt type 0: softint net/21 = 0
   evcnt type 0: softint net block/21 = 0
   evcnt type 0: softint bio/21 = 0
   evcnt type 0: softint bio block/21 = 0
   evcnt type 0: softint clk/21 = 0
   evcnt type 0: softint clk block/21 = 0
   evcnt type 0: softint ser/21 = 60
   evcnt type 0: softint ser block/21 = 0
   evcnt type 0: callout late/21 = 0
   evcnt type 0: callout wait/21 = 0
   evcnt type 0: cpu21 timer = 18579
   evcnt type 0: cpu21 halt IPI = 0
   evcnt type 0: cpu21 AST IPI = 0
   evcnt type 0: cpu21 generic IPI = 0
   evcnt type 0: cpu21 FPU synch IPI = 0
   evcnt type 0: cpu21 MTRR update IPI = 0
   evcnt type 0: cpu21 GDT update IPI = 0
   evcnt type 0: cpu21 xcall IPI = 60
   evcnt type 0: cpu21 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu21 kpreempt IPI = 0
   evcnt type 0: softint net/22 = 0
   evcnt type 0: softint net block/22 = 0
   evcnt type 0: softint bio/22 = 0
   evcnt type 0: softint bio block/22 = 0
   evcnt type 0: softint clk/22 = 0
   evcnt type 0: softint clk block/22 = 0
   evcnt type 0: softint ser/22 = 60
   evcnt type 0: softint ser block/22 = 0
   evcnt type 0: callout late/22 = 0
   evcnt type 0: callout wait/22 = 0
   evcnt type 0: cpu22 timer = 18592
   evcnt type 0: cpu22 halt IPI = 0
   evcnt type 0: cpu22 AST IPI = 0
   evcnt type 0: cpu22 generic IPI = 0
   evcnt type 0: cpu22 FPU synch IPI = 0
   evcnt type 0: cpu22 MTRR update IPI = 0
   evcnt type 0: cpu22 GDT update IPI = 0
   evcnt type 0: cpu22 xcall IPI = 60
   evcnt type 0: cpu22 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu22 kpreempt IPI = 0
   evcnt type 0: softint net/23 = 0
   evcnt type 0: softint net block/23 = 0
   evcnt type 0: softint bio/23 = 0
   evcnt type 0: softint bio block/23 = 0
   evcnt type 0: softint clk/23 = 0
   evcnt type 0: softint clk block/23 = 0
   evcnt type 0: softint ser/23 = 60
   evcnt type 0: softint ser block/23 = 0
   evcnt type 0: callout late/23 = 0
   evcnt type 0: callout wait/23 = 0
   evcnt type 0: cpu23 timer = 18580
   evcnt type 0: cpu23 halt IPI = 0
   evcnt type 0: cpu23 AST IPI = 0
   evcnt type 0: cpu23 generic IPI = 0
   evcnt type 0: cpu23 FPU synch IPI = 0
   evcnt type 0: cpu23 MTRR update IPI = 0
   evcnt type 0: cpu23 GDT update IPI = 0
   evcnt type 0: cpu23 xcall IPI = 60
   evcnt type 0: cpu23 ACPI CPU sleep IPI = 0
   evcnt type 0: cpu23 kpreempt IPI = 0
   evcnt type 1: ioapic0 pin 9 = 0
   evcnt type 0: midi0 xmt bytes discarded = 0
   evcnt type 0: midi0 xmt incomplete msgs = 0
   evcnt type 1: msix0 vec 0 = 0
   evcnt type 1: msix0 vec 1 = 0
   evcnt type 1: msix0 vec 2 = 0
   evcnt type 1: msix0 vec 3 = 0
   evcnt type 1: msix0 vec 4 = 0
   evcnt type 1: msix0 vec 5 = 0
   evcnt type 1: msix0 vec 6 = 0
   evcnt type 1: msix0 vec 7 = 0
   evcnt type 1: msix0 vec 8 = 0
   evcnt type 1: msix0 vec 9 = 0
   evcnt type 1: msix0 vec 10 = 0
   evcnt type 1: msix0 vec 11 = 0
   evcnt type 1: msix0 vec 12 = 0
   evcnt type 1: msix0 vec 13 = 0
   evcnt type 1: msix0 vec 14 = 0
   evcnt type 1: msix0 vec 15 = 0
   evcnt type 1: msix0 vec 16 = 0
   evcnt type 1: msix0 vec 17 = 0
   evcnt type 1: msix0 vec 18 = 0
   evcnt type 1: msix0 vec 19 = 0
   evcnt type 1: msix0 vec 20 = 0
   evcnt type 1: msix0 vec 21 = 0
   evcnt type 1: msix0 vec 22 = 0
   evcnt type 1: msix0 vec 23 = 0
   evcnt type 1: msix0 vec 24 = 0
   evcnt type 0: ixg0 Driver tx dma soft fail EFBIG = 0
   evcnt type 0: ixg0 m_defrag() failed = 0
   evcnt type 0: ixg0 Driver tx dma hard fail EFBIG = 0
   evcnt type 0: ixg0 Driver tx dma hard fail EINVAL = 0
   evcnt type 0: ixg0 Driver tx dma hard fail other = 0
   evcnt type 0: ixg0 Driver tx dma soft fail EAGAIN = 0
   evcnt type 0: ixg0 Driver tx dma soft fail ENOMEM = 0
   evcnt type 0: ixg0 Watchdog timeouts = 0
   evcnt type 0: ixg0 TSO errors = 0
   evcnt type 1: ixg0 Link MSI-X IRQ Handled = 0
   evcnt type 1: ixg0 Link softint = 0
   evcnt type 1: ixg0 module softint = 0
   evcnt type 1: ixg0 multimode softint = 0
   evcnt type 1: ixg0 external PHY softint = 0
   evcnt type 0: ixg0 tc0 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc0 pxontxc = 0
   evcnt type 0: ixg0 tc0 pxonrxc = 0
   evcnt type 0: ixg0 tc0 pxofftxc = 0
   evcnt type 0: ixg0 tc0 pxoffrxc = 0
   evcnt type 0: ixg0 tc0 pxon2offc = 0
   evcnt type 0: ixg0 tc1 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc1 pxontxc = 0
   evcnt type 0: ixg0 tc1 pxonrxc = 0
   evcnt type 0: ixg0 tc1 pxofftxc = 0
   evcnt type 0: ixg0 tc1 pxoffrxc = 0
   evcnt type 0: ixg0 tc1 pxon2offc = 0
   evcnt type 0: ixg0 tc2 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc2 pxontxc = 0
   evcnt type 0: ixg0 tc2 pxonrxc = 0
   evcnt type 0: ixg0 tc2 pxofftxc = 0
   evcnt type 0: ixg0 tc2 pxoffrxc = 0
   evcnt type 0: ixg0 tc2 pxon2offc = 0
   evcnt type 0: ixg0 tc3 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc3 pxontxc = 0
   evcnt type 0: ixg0 tc3 pxonrxc = 0
   evcnt type 0: ixg0 tc3 pxofftxc = 0
   evcnt type 0: ixg0 tc3 pxoffrxc = 0
   evcnt type 0: ixg0 tc3 pxon2offc = 0
   evcnt type 0: ixg0 tc4 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc4 pxontxc = 0
   evcnt type 0: ixg0 tc4 pxonrxc = 0
   evcnt type 0: ixg0 tc4 pxofftxc = 0
   evcnt type 0: ixg0 tc4 pxoffrxc = 0
   evcnt type 0: ixg0 tc4 pxon2offc = 0
   evcnt type 0: ixg0 tc5 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc5 pxontxc = 0
   evcnt type 0: ixg0 tc5 pxonrxc = 0
   evcnt type 0: ixg0 tc5 pxofftxc = 0
   evcnt type 0: ixg0 tc5 pxoffrxc = 0
   evcnt type 0: ixg0 tc5 pxon2offc = 0
   evcnt type 0: ixg0 tc6 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc6 pxontxc = 0
   evcnt type 0: ixg0 tc6 pxonrxc = 0
   evcnt type 0: ixg0 tc6 pxofftxc = 0
   evcnt type 0: ixg0 tc6 pxoffrxc = 0
   evcnt type 0: ixg0 tc6 pxon2offc = 0
   evcnt type 0: ixg0 tc7 RX Missed Packet Count = 0
   evcnt type 0: ixg0 tc7 pxontxc = 0
   evcnt type 0: ixg0 tc7 pxonrxc = 0
   evcnt type 0: ixg0 tc7 pxofftxc = 0
   evcnt type 0: ixg0 tc7 pxoffrxc = 0
   evcnt type 0: ixg0 tc7 pxon2offc = 0
   evcnt type 1: ixg0 q0 IRQs on queue = 0
   evcnt type 0: ixg0 q0 Handled queue in softint = 0
   evcnt type 0: ixg0 q0 Requeued in softint = 0
   evcnt type 0: ixg0 q0 TSO = 0
   evcnt type 0: ixg0 q0 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q0 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q0 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q0 qprc = 0
   evcnt type 0: ixg0 q0 qptc = 0
   evcnt type 0: ixg0 q0 qbrc = 0
   evcnt type 0: ixg0 q0 qbtc = 0
   evcnt type 0: ixg0 q0 qprdc = 0
   evcnt type 0: ixg0 q0 Queue Packets Received = 0
   evcnt type 0: ixg0 q0 Queue Bytes Received = 0
   evcnt type 0: ixg0 q0 Copied RX Frames = 0
   evcnt type 0: ixg0 q0 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q0 Rx discarded = 0
   evcnt type 1: ixg0 q1 IRQs on queue = 0
   evcnt type 0: ixg0 q1 Handled queue in softint = 0
   evcnt type 0: ixg0 q1 Requeued in softint = 0
   evcnt type 0: ixg0 q1 TSO = 0
   evcnt type 0: ixg0 q1 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q1 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q1 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q1 qprc = 0
   evcnt type 0: ixg0 q1 qptc = 0
   evcnt type 0: ixg0 q1 qbrc = 0
   evcnt type 0: ixg0 q1 qbtc = 0
   evcnt type 0: ixg0 q1 qprdc = 0
   evcnt type 0: ixg0 q1 Queue Packets Received = 0
   evcnt type 0: ixg0 q1 Queue Bytes Received = 0
   evcnt type 0: ixg0 q1 Copied RX Frames = 0
   evcnt type 0: ixg0 q1 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q1 Rx discarded = 0
   evcnt type 1: ixg0 q2 IRQs on queue = 0
   evcnt type 0: ixg0 q2 Handled queue in softint = 0
   evcnt type 0: ixg0 q2 Requeued in softint = 0
   evcnt type 0: ixg0 q2 TSO = 0
   evcnt type 0: ixg0 q2 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q2 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q2 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q2 qprc = 0
   evcnt type 0: ixg0 q2 qptc = 0
   evcnt type 0: ixg0 q2 qbrc = 0
   evcnt type 0: ixg0 q2 qbtc = 0
   evcnt type 0: ixg0 q2 qprdc = 0
   evcnt type 0: ixg0 q2 Queue Packets Received = 0
   evcnt type 0: ixg0 q2 Queue Bytes Received = 0
   evcnt type 0: ixg0 q2 Copied RX Frames = 0
   evcnt type 0: ixg0 q2 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q2 Rx discarded = 0
   evcnt type 1: ixg0 q3 IRQs on queue = 0
   evcnt type 0: ixg0 q3 Handled queue in softint = 0
   evcnt type 0: ixg0 q3 Requeued in softint = 0
   evcnt type 0: ixg0 q3 TSO = 0
   evcnt type 0: ixg0 q3 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q3 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q3 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q3 qprc = 0
   evcnt type 0: ixg0 q3 qptc = 0
   evcnt type 0: ixg0 q3 qbrc = 0
   evcnt type 0: ixg0 q3 qbtc = 0
   evcnt type 0: ixg0 q3 qprdc = 0
   evcnt type 0: ixg0 q3 Queue Packets Received = 0
   evcnt type 0: ixg0 q3 Queue Bytes Received = 0
   evcnt type 0: ixg0 q3 Copied RX Frames = 0
   evcnt type 0: ixg0 q3 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q3 Rx discarded = 0
   evcnt type 1: ixg0 q4 IRQs on queue = 0
   evcnt type 0: ixg0 q4 Handled queue in softint = 0
   evcnt type 0: ixg0 q4 Requeued in softint = 0
   evcnt type 0: ixg0 q4 TSO = 0
   evcnt type 0: ixg0 q4 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q4 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q4 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q4 qprc = 0
   evcnt type 0: ixg0 q4 qptc = 0
   evcnt type 0: ixg0 q4 qbrc = 0
   evcnt type 0: ixg0 q4 qbtc = 0
   evcnt type 0: ixg0 q4 qprdc = 0
   evcnt type 0: ixg0 q4 Queue Packets Received = 0
   evcnt type 0: ixg0 q4 Queue Bytes Received = 0
   evcnt type 0: ixg0 q4 Copied RX Frames = 0
   evcnt type 0: ixg0 q4 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q4 Rx discarded = 0
   evcnt type 1: ixg0 q5 IRQs on queue = 0
   evcnt type 0: ixg0 q5 Handled queue in softint = 0
   evcnt type 0: ixg0 q5 Requeued in softint = 0
   evcnt type 0: ixg0 q5 TSO = 0
   evcnt type 0: ixg0 q5 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q5 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q5 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q5 qprc = 0
   evcnt type 0: ixg0 q5 qptc = 0
   evcnt type 0: ixg0 q5 qbrc = 0
   evcnt type 0: ixg0 q5 qbtc = 0
   evcnt type 0: ixg0 q5 qprdc = 0
   evcnt type 0: ixg0 q5 Queue Packets Received = 0
   evcnt type 0: ixg0 q5 Queue Bytes Received = 0
   evcnt type 0: ixg0 q5 Copied RX Frames = 0
   evcnt type 0: ixg0 q5 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q5 Rx discarded = 0
   evcnt type 1: ixg0 q6 IRQs on queue = 0
   evcnt type 0: ixg0 q6 Handled queue in softint = 0
   evcnt type 0: ixg0 q6 Requeued in softint = 0
   evcnt type 0: ixg0 q6 TSO = 0
   evcnt type 0: ixg0 q6 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q6 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q6 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q6 qprc = 0
   evcnt type 0: ixg0 q6 qptc = 0
   evcnt type 0: ixg0 q6 qbrc = 0
   evcnt type 0: ixg0 q6 qbtc = 0
   evcnt type 0: ixg0 q6 qprdc = 0
   evcnt type 0: ixg0 q6 Queue Packets Received = 0
   evcnt type 0: ixg0 q6 Queue Bytes Received = 0
   evcnt type 0: ixg0 q6 Copied RX Frames = 0
   evcnt type 0: ixg0 q6 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q6 Rx discarded = 0
   evcnt type 1: ixg0 q7 IRQs on queue = 0
   evcnt type 0: ixg0 q7 Handled queue in softint = 0
   evcnt type 0: ixg0 q7 Requeued in softint = 0
   evcnt type 0: ixg0 q7 TSO = 0
   evcnt type 0: ixg0 q7 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q7 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q7 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q7 qprc = 0
   evcnt type 0: ixg0 q7 qptc = 0
   evcnt type 0: ixg0 q7 qbrc = 0
   evcnt type 0: ixg0 q7 qbtc = 0
   evcnt type 0: ixg0 q7 qprdc = 0
   evcnt type 0: ixg0 q7 Queue Packets Received = 0
   evcnt type 0: ixg0 q7 Queue Bytes Received = 0
   evcnt type 0: ixg0 q7 Copied RX Frames = 0
   evcnt type 0: ixg0 q7 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q7 Rx discarded = 0
   evcnt type 1: ixg0 q8 IRQs on queue = 0
   evcnt type 0: ixg0 q8 Handled queue in softint = 0
   evcnt type 0: ixg0 q8 Requeued in softint = 0
   evcnt type 0: ixg0 q8 TSO = 0
   evcnt type 0: ixg0 q8 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q8 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q8 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q8 qprc = 0
   evcnt type 0: ixg0 q8 qptc = 0
   evcnt type 0: ixg0 q8 qbrc = 0
   evcnt type 0: ixg0 q8 qbtc = 0
   evcnt type 0: ixg0 q8 qprdc = 0
   evcnt type 0: ixg0 q8 Queue Packets Received = 0
   evcnt type 0: ixg0 q8 Queue Bytes Received = 0
   evcnt type 0: ixg0 q8 Copied RX Frames = 0
   evcnt type 0: ixg0 q8 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q8 Rx discarded = 0
   evcnt type 1: ixg0 q9 IRQs on queue = 0
   evcnt type 0: ixg0 q9 Handled queue in softint = 0
   evcnt type 0: ixg0 q9 Requeued in softint = 0
   evcnt type 0: ixg0 q9 TSO = 0
   evcnt type 0: ixg0 q9 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q9 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q9 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q9 qprc = 0
   evcnt type 0: ixg0 q9 qptc = 0
   evcnt type 0: ixg0 q9 qbrc = 0
   evcnt type 0: ixg0 q9 qbtc = 0
   evcnt type 0: ixg0 q9 qprdc = 0
   evcnt type 0: ixg0 q9 Queue Packets Received = 0
   evcnt type 0: ixg0 q9 Queue Bytes Received = 0
   evcnt type 0: ixg0 q9 Copied RX Frames = 0
   evcnt type 0: ixg0 q9 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q9 Rx discarded = 0
   evcnt type 1: ixg0 q10 IRQs on queue = 0
   evcnt type 0: ixg0 q10 Handled queue in softint = 0
   evcnt type 0: ixg0 q10 Requeued in softint = 0
   evcnt type 0: ixg0 q10 TSO = 0
   evcnt type 0: ixg0 q10 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q10 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q10 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q10 qprc = 0
   evcnt type 0: ixg0 q10 qptc = 0
   evcnt type 0: ixg0 q10 qbrc = 0
   evcnt type 0: ixg0 q10 qbtc = 0
   evcnt type 0: ixg0 q10 qprdc = 0
   evcnt type 0: ixg0 q10 Queue Packets Received = 0
   evcnt type 0: ixg0 q10 Queue Bytes Received = 0
   evcnt type 0: ixg0 q10 Copied RX Frames = 0
   evcnt type 0: ixg0 q10 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q10 Rx discarded = 0
   evcnt type 1: ixg0 q11 IRQs on queue = 0
   evcnt type 0: ixg0 q11 Handled queue in softint = 0
   evcnt type 0: ixg0 q11 Requeued in softint = 0
   evcnt type 0: ixg0 q11 TSO = 0
   evcnt type 0: ixg0 q11 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q11 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q11 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q11 qprc = 0
   evcnt type 0: ixg0 q11 qptc = 0
   evcnt type 0: ixg0 q11 qbrc = 0
   evcnt type 0: ixg0 q11 qbtc = 0
   evcnt type 0: ixg0 q11 qprdc = 0
   evcnt type 0: ixg0 q11 Queue Packets Received = 0
   evcnt type 0: ixg0 q11 Queue Bytes Received = 0
   evcnt type 0: ixg0 q11 Copied RX Frames = 0
   evcnt type 0: ixg0 q11 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q11 Rx discarded = 0
   evcnt type 1: ixg0 q12 IRQs on queue = 0
   evcnt type 0: ixg0 q12 Handled queue in softint = 0
   evcnt type 0: ixg0 q12 Requeued in softint = 0
   evcnt type 0: ixg0 q12 TSO = 0
   evcnt type 0: ixg0 q12 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q12 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q12 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q12 qprc = 0
   evcnt type 0: ixg0 q12 qptc = 0
   evcnt type 0: ixg0 q12 qbrc = 0
   evcnt type 0: ixg0 q12 qbtc = 0
   evcnt type 0: ixg0 q12 qprdc = 0
   evcnt type 0: ixg0 q12 Queue Packets Received = 0
   evcnt type 0: ixg0 q12 Queue Bytes Received = 0
   evcnt type 0: ixg0 q12 Copied RX Frames = 0
   evcnt type 0: ixg0 q12 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q12 Rx discarded = 0
   evcnt type 1: ixg0 q13 IRQs on queue = 0
   evcnt type 0: ixg0 q13 Handled queue in softint = 0
   evcnt type 0: ixg0 q13 Requeued in softint = 0
   evcnt type 0: ixg0 q13 TSO = 0
   evcnt type 0: ixg0 q13 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q13 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q13 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q13 qprc = 0
   evcnt type 0: ixg0 q13 qptc = 0
   evcnt type 0: ixg0 q13 qbrc = 0
   evcnt type 0: ixg0 q13 qbtc = 0
   evcnt type 0: ixg0 q13 qprdc = 0
   evcnt type 0: ixg0 q13 Queue Packets Received = 0
   evcnt type 0: ixg0 q13 Queue Bytes Received = 0
   evcnt type 0: ixg0 q13 Copied RX Frames = 0
   evcnt type 0: ixg0 q13 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q13 Rx discarded = 0
   evcnt type 1: ixg0 q14 IRQs on queue = 0
   evcnt type 0: ixg0 q14 Handled queue in softint = 0
   evcnt type 0: ixg0 q14 Requeued in softint = 0
   evcnt type 0: ixg0 q14 TSO = 0
   evcnt type 0: ixg0 q14 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q14 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q14 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q14 qprc = 0
   evcnt type 0: ixg0 q14 qptc = 0
   evcnt type 0: ixg0 q14 qbrc = 0
   evcnt type 0: ixg0 q14 qbtc = 0
   evcnt type 0: ixg0 q14 qprdc = 0
   evcnt type 0: ixg0 q14 Queue Packets Received = 0
   evcnt type 0: ixg0 q14 Queue Bytes Received = 0
   evcnt type 0: ixg0 q14 Copied RX Frames = 0
   evcnt type 0: ixg0 q14 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q14 Rx discarded = 0
   evcnt type 1: ixg0 q15 IRQs on queue = 0
   evcnt type 0: ixg0 q15 Handled queue in softint = 0
   evcnt type 0: ixg0 q15 Requeued in softint = 0
   evcnt type 0: ixg0 q15 TSO = 0
   evcnt type 0: ixg0 q15 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q15 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q15 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q15 qprc = 0
   evcnt type 0: ixg0 q15 qptc = 0
   evcnt type 0: ixg0 q15 qbrc = 0
   evcnt type 0: ixg0 q15 qbtc = 0
   evcnt type 0: ixg0 q15 qprdc = 0
   evcnt type 0: ixg0 q15 Queue Packets Received = 0
   evcnt type 0: ixg0 q15 Queue Bytes Received = 0
   evcnt type 0: ixg0 q15 Copied RX Frames = 0
   evcnt type 0: ixg0 q15 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q15 Rx discarded = 0
   evcnt type 1: ixg0 q16 IRQs on queue = 0
   evcnt type 0: ixg0 q16 Handled queue in softint = 0
   evcnt type 0: ixg0 q16 Requeued in softint = 0
   evcnt type 0: ixg0 q16 TSO = 0
   evcnt type 0: ixg0 q16 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q16 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q16 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q16 Queue Packets Received = 0
   evcnt type 0: ixg0 q16 Queue Bytes Received = 0
   evcnt type 0: ixg0 q16 Copied RX Frames = 0
   evcnt type 0: ixg0 q16 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q16 Rx discarded = 0
   evcnt type 1: ixg0 q17 IRQs on queue = 0
   evcnt type 0: ixg0 q17 Handled queue in softint = 0
   evcnt type 0: ixg0 q17 Requeued in softint = 0
   evcnt type 0: ixg0 q17 TSO = 0
   evcnt type 0: ixg0 q17 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q17 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q17 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q17 Queue Packets Received = 0
   evcnt type 0: ixg0 q17 Queue Bytes Received = 0
   evcnt type 0: ixg0 q17 Copied RX Frames = 0
   evcnt type 0: ixg0 q17 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q17 Rx discarded = 0
   evcnt type 1: ixg0 q18 IRQs on queue = 0
   evcnt type 0: ixg0 q18 Handled queue in softint = 0
   evcnt type 0: ixg0 q18 Requeued in softint = 0
   evcnt type 0: ixg0 q18 TSO = 0
   evcnt type 0: ixg0 q18 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q18 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q18 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q18 Queue Packets Received = 0
   evcnt type 0: ixg0 q18 Queue Bytes Received = 0
   evcnt type 0: ixg0 q18 Copied RX Frames = 0
   evcnt type 0: ixg0 q18 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q18 Rx discarded = 0
   evcnt type 1: ixg0 q19 IRQs on queue = 0
   evcnt type 0: ixg0 q19 Handled queue in softint = 0
   evcnt type 0: ixg0 q19 Requeued in softint = 0
   evcnt type 0: ixg0 q19 TSO = 0
   evcnt type 0: ixg0 q19 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q19 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q19 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q19 Queue Packets Received = 0
   evcnt type 0: ixg0 q19 Queue Bytes Received = 0
   evcnt type 0: ixg0 q19 Copied RX Frames = 0
   evcnt type 0: ixg0 q19 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q19 Rx discarded = 0
   evcnt type 1: ixg0 q20 IRQs on queue = 0
   evcnt type 0: ixg0 q20 Handled queue in softint = 0
   evcnt type 0: ixg0 q20 Requeued in softint = 0
   evcnt type 0: ixg0 q20 TSO = 0
   evcnt type 0: ixg0 q20 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q20 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q20 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q20 Queue Packets Received = 0
   evcnt type 0: ixg0 q20 Queue Bytes Received = 0
   evcnt type 0: ixg0 q20 Copied RX Frames = 0
   evcnt type 0: ixg0 q20 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q20 Rx discarded = 0
   evcnt type 1: ixg0 q21 IRQs on queue = 0
   evcnt type 0: ixg0 q21 Handled queue in softint = 0
   evcnt type 0: ixg0 q21 Requeued in softint = 0
   evcnt type 0: ixg0 q21 TSO = 0
   evcnt type 0: ixg0 q21 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q21 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q21 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q21 Queue Packets Received = 0
   evcnt type 0: ixg0 q21 Queue Bytes Received = 0
   evcnt type 0: ixg0 q21 Copied RX Frames = 0
   evcnt type 0: ixg0 q21 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q21 Rx discarded = 0
   evcnt type 1: ixg0 q22 IRQs on queue = 0
   evcnt type 0: ixg0 q22 Handled queue in softint = 0
   evcnt type 0: ixg0 q22 Requeued in softint = 0
   evcnt type 0: ixg0 q22 TSO = 0
   evcnt type 0: ixg0 q22 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q22 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q22 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q22 Queue Packets Received = 0
   evcnt type 0: ixg0 q22 Queue Bytes Received = 0
   evcnt type 0: ixg0 q22 Copied RX Frames = 0
   evcnt type 0: ixg0 q22 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q22 Rx discarded = 0
   evcnt type 1: ixg0 q23 IRQs on queue = 0
   evcnt type 0: ixg0 q23 Handled queue in softint = 0
   evcnt type 0: ixg0 q23 Requeued in softint = 0
   evcnt type 0: ixg0 q23 TSO = 0
   evcnt type 0: ixg0 q23 Queue No Descriptor Available = 0
   evcnt type 0: ixg0 q23 Queue Packets Transmitted = 0
   evcnt type 0: ixg0 q23 Packets dropped in pcq = 0
   evcnt type 0: ixg0 q23 Queue Packets Received = 0
   evcnt type 0: ixg0 q23 Queue Bytes Received = 0
   evcnt type 0: ixg0 q23 Copied RX Frames = 0
   evcnt type 0: ixg0 q23 Rx no jumbo mbuf = 0
   evcnt type 0: ixg0 q23 Rx discarded = 0
   evcnt type 0: ixg0 MAC Statistics rx csum offload - IP = 0
   evcnt type 0: ixg0 MAC Statistics rx csum offload - L4 = 0
   evcnt type 0: ixg0 MAC Statistics rx csum offload - IP bad = 0
   evcnt type 0: ixg0 MAC Statistics rx csum offload - L4 bad = 0
   evcnt type 0: ixg0 MAC Statistics Interrupt conditions zero = 0
   evcnt type 0: ixg0 MAC Statistics Legacy interrupts = 0
   evcnt type 0: ixg0 MAC Statistics CRC Errors = 0
   evcnt type 0: ixg0 MAC Statistics Illegal Byte Errors = 0
   evcnt type 0: ixg0 MAC Statistics Byte Errors = 0
   evcnt type 0: ixg0 MAC Statistics MAC Short Packets Discarded = 0
   evcnt type 0: ixg0 MAC Statistics Total Packets Missed = 0
   evcnt type 0: ixg0 MAC Statistics MAC Local Faults = 0
   evcnt type 0: ixg0 MAC Statistics MAC Remote Faults = 0
   evcnt type 0: ixg0 MAC Statistics Receive Length Errors = 0
   evcnt type 0: ixg0 MAC Statistics Link XON Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Link XON Received = 0
   evcnt type 0: ixg0 MAC Statistics Link XOFF Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Link XOFF Received = 0
   evcnt type 0: ixg0 MAC Statistics Total Octets Received = 0
   evcnt type 0: ixg0 MAC Statistics Good Octets Received = 0
   evcnt type 0: ixg0 MAC Statistics Total Packets Received = 0
   evcnt type 0: ixg0 MAC Statistics Good Packets Received = 0
   evcnt type 0: ixg0 MAC Statistics Multicast Packets Received = 0
   evcnt type 0: ixg0 MAC Statistics Broadcast Packets Received = 0
   evcnt type 0: ixg0 MAC Statistics 64 byte frames received  = 0
   evcnt type 0: ixg0 MAC Statistics 65-127 byte frames received = 0
   evcnt type 0: ixg0 MAC Statistics 128-255 byte frames received = 0
   evcnt type 0: ixg0 MAC Statistics 256-511 byte frames received = 0
   evcnt type 0: ixg0 MAC Statistics 512-1023 byte frames received = 0
   evcnt type 0: ixg0 MAC Statistics 1023-1522 byte frames received = 0
   evcnt type 0: ixg0 MAC Statistics Receive Undersized = 0
   evcnt type 0: ixg0 MAC Statistics Fragmented Packets Received  = 0
   evcnt type 0: ixg0 MAC Statistics Oversized Packets Received = 0
   evcnt type 0: ixg0 MAC Statistics Received Jabber = 0
   evcnt type 0: ixg0 MAC Statistics Management Packets Received = 0
   evcnt type 0: ixg0 MAC Statistics Management Packets Dropped = 0
   evcnt type 0: ixg0 MAC Statistics Checksum Errors = 0
   evcnt type 0: ixg0 MAC Statistics Good Octets Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Total Packets Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Good Packets Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Broadcast Packets Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Multicast Packets Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics Management Packets Transmitted = 0
   evcnt type 0: ixg0 MAC Statistics 64 byte frames transmitted  = 0
   evcnt type 0: ixg0 MAC Statistics 65-127 byte frames transmitted = 0
   evcnt type 0: ixg0 MAC Statistics 128-255 byte frames transmitted = 0
   evcnt type 0: ixg0 MAC Statistics 256-511 byte frames transmitted = 0
   evcnt type 0: ixg0 MAC Statistics 512-1023 byte frames transmitted = 0
   evcnt type 0: ixg0 MAC Statistics 1024-1522 byte frames transmitted = 0
   evcnt type 1: msix1 vec 0 = 0
   evcnt type 1: msix1 vec 1 = 0
   evcnt type 1: msix1 vec 2 = 0
   evcnt type 1: msix1 vec 3 = 0
   evcnt type 1: msix1 vec 4 = 0
   evcnt type 1: msix1 vec 5 = 0
   evcnt type 1: msix1 vec 6 = 0
   evcnt type 1: msix1 vec 7 = 0
   evcnt type 1: msix1 vec 8 = 0
   evcnt type 1: msix1 vec 9 = 0
   evcnt type 1: msix1 vec 10 = 0
   evcnt type 1: msix1 vec 11 = 0
   evcnt type 1: msix1 vec 12 = 0
   evcnt type 1: msix1 vec 13 = 0
   evcnt type 1: msix1 vec 14 = 0
   evcnt type 1: msix1 vec 15 = 0
   evcnt type 1: msix1 vec 16 = 0
   evcnt type 1: msix1 vec 17 = 0
   evcnt type 1: msix1 vec 18 = 0
   evcnt type 1: msix1 vec 19 = 0
   evcnt type 1: msix1 vec 20 = 0
   evcnt type 1: msix1 vec 21 = 0
   evcnt type 1: msix1 vec 22 = 0
   evcnt type 1: msix1 vec 23 = 0
   evcnt type 1: msix1 vec 24 = 0
   evcnt type 0: ixg1 Driver tx dma soft fail EFBIG = 0
   evcnt type 0: ixg1 m_defrag() failed = 0
   evcnt type 0: ixg1 Driver tx dma hard fail EFBIG = 0
   evcnt type 0: ixg1 Driver tx dma hard fail EINVAL = 0
   evcnt type 0: ixg1 Driver tx dma hard fail other = 0
   evcnt type 0: ixg1 Driver tx dma soft fail EAGAIN = 0
   evcnt type 0: ixg1 Driver tx dma soft fail ENOMEM = 0
   evcnt type 0: ixg1 Watchdog timeouts = 0
   evcnt type 0: ixg1 TSO errors = 0
   evcnt type 1: ixg1 Link MSI-X IRQ Handled = 0
   evcnt type 1: ixg1 Link softint = 0
   evcnt type 1: ixg1 module softint = 0
   evcnt type 1: ixg1 multimode softint = 0
   evcnt type 1: ixg1 external PHY softint = 0
   evcnt type 0: ixg1 tc0 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc0 pxontxc = 0
   evcnt type 0: ixg1 tc0 pxonrxc = 0
   evcnt type 0: ixg1 tc0 pxofftxc = 0
   evcnt type 0: ixg1 tc0 pxoffrxc = 0
   evcnt type 0: ixg1 tc0 pxon2offc = 0
   evcnt type 0: ixg1 tc1 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc1 pxontxc = 0
   evcnt type 0: ixg1 tc1 pxonrxc = 0
   evcnt type 0: ixg1 tc1 pxofftxc = 0
   evcnt type 0: ixg1 tc1 pxoffrxc = 0
   evcnt type 0: ixg1 tc1 pxon2offc = 0
   evcnt type 0: ixg1 tc2 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc2 pxontxc = 0
   evcnt type 0: ixg1 tc2 pxonrxc = 0
   evcnt type 0: ixg1 tc2 pxofftxc = 0
   evcnt type 0: ixg1 tc2 pxoffrxc = 0
   evcnt type 0: ixg1 tc2 pxon2offc = 0
   evcnt type 0: ixg1 tc3 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc3 pxontxc = 0
   evcnt type 0: ixg1 tc3 pxonrxc = 0
   evcnt type 0: ixg1 tc3 pxofftxc = 0
   evcnt type 0: ixg1 tc3 pxoffrxc = 0
   evcnt type 0: ixg1 tc3 pxon2offc = 0
   evcnt type 0: ixg1 tc4 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc4 pxontxc = 0
   evcnt type 0: ixg1 tc4 pxonrxc = 0
   evcnt type 0: ixg1 tc4 pxofftxc = 0
   evcnt type 0: ixg1 tc4 pxoffrxc = 0
   evcnt type 0: ixg1 tc4 pxon2offc = 0
   evcnt type 0: ixg1 tc5 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc5 pxontxc = 0
   evcnt type 0: ixg1 tc5 pxonrxc = 0
   evcnt type 0: ixg1 tc5 pxofftxc = 0
   evcnt type 0: ixg1 tc5 pxoffrxc = 0
   evcnt type 0: ixg1 tc5 pxon2offc = 0
   evcnt type 0: ixg1 tc6 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc6 pxontxc = 0
   evcnt type 0: ixg1 tc6 pxonrxc = 0
   evcnt type 0: ixg1 tc6 pxofftxc = 0
   evcnt type 0: ixg1 tc6 pxoffrxc = 0
   evcnt type 0: ixg1 tc6 pxon2offc = 0
   evcnt type 0: ixg1 tc7 RX Missed Packet Count = 0
   evcnt type 0: ixg1 tc7 pxontxc = 0
   evcnt type 0: ixg1 tc7 pxonrxc = 0
   evcnt type 0: ixg1 tc7 pxofftxc = 0
   evcnt type 0: ixg1 tc7 pxoffrxc = 0
   evcnt type 0: ixg1 tc7 pxon2offc = 0
   evcnt type 1: ixg1 q0 IRQs on queue = 0
   evcnt type 0: ixg1 q0 Handled queue in softint = 0
   evcnt type 0: ixg1 q0 Requeued in softint = 0
   evcnt type 0: ixg1 q0 TSO = 0
   evcnt type 0: ixg1 q0 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q0 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q0 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q0 qprc = 0
   evcnt type 0: ixg1 q0 qptc = 0
   evcnt type 0: ixg1 q0 qbrc = 0
   evcnt type 0: ixg1 q0 qbtc = 0
   evcnt type 0: ixg1 q0 qprdc = 0
   evcnt type 0: ixg1 q0 Queue Packets Received = 0
   evcnt type 0: ixg1 q0 Queue Bytes Received = 0
   evcnt type 0: ixg1 q0 Copied RX Frames = 0
   evcnt type 0: ixg1 q0 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q0 Rx discarded = 0
   evcnt type 1: ixg1 q1 IRQs on queue = 0
   evcnt type 0: ixg1 q1 Handled queue in softint = 0
   evcnt type 0: ixg1 q1 Requeued in softint = 0
   evcnt type 0: ixg1 q1 TSO = 0
   evcnt type 0: ixg1 q1 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q1 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q1 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q1 qprc = 0
   evcnt type 0: ixg1 q1 qptc = 0
   evcnt type 0: ixg1 q1 qbrc = 0
   evcnt type 0: ixg1 q1 qbtc = 0
   evcnt type 0: ixg1 q1 qprdc = 0
   evcnt type 0: ixg1 q1 Queue Packets Received = 0
   evcnt type 0: ixg1 q1 Queue Bytes Received = 0
   evcnt type 0: ixg1 q1 Copied RX Frames = 0
   evcnt type 0: ixg1 q1 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q1 Rx discarded = 0
   evcnt type 1: ixg1 q2 IRQs on queue = 0
   evcnt type 0: ixg1 q2 Handled queue in softint = 0
   evcnt type 0: ixg1 q2 Requeued in softint = 0
   evcnt type 0: ixg1 q2 TSO = 0
   evcnt type 0: ixg1 q2 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q2 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q2 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q2 qprc = 0
   evcnt type 0: ixg1 q2 qptc = 0
   evcnt type 0: ixg1 q2 qbrc = 0
   evcnt type 0: ixg1 q2 qbtc = 0
   evcnt type 0: ixg1 q2 qprdc = 0
   evcnt type 0: ixg1 q2 Queue Packets Received = 0
   evcnt type 0: ixg1 q2 Queue Bytes Received = 0
   evcnt type 0: ixg1 q2 Copied RX Frames = 0
   evcnt type 0: ixg1 q2 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q2 Rx discarded = 0
   evcnt type 1: ixg1 q3 IRQs on queue = 0
   evcnt type 0: ixg1 q3 Handled queue in softint = 0
   evcnt type 0: ixg1 q3 Requeued in softint = 0
   evcnt type 0: ixg1 q3 TSO = 0
   evcnt type 0: ixg1 q3 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q3 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q3 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q3 qprc = 0
   evcnt type 0: ixg1 q3 qptc = 0
   evcnt type 0: ixg1 q3 qbrc = 0
   evcnt type 0: ixg1 q3 qbtc = 0
   evcnt type 0: ixg1 q3 qprdc = 0
   evcnt type 0: ixg1 q3 Queue Packets Received = 0
   evcnt type 0: ixg1 q3 Queue Bytes Received = 0
   evcnt type 0: ixg1 q3 Copied RX Frames = 0
   evcnt type 0: ixg1 q3 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q3 Rx discarded = 0
   evcnt type 1: ixg1 q4 IRQs on queue = 0
   evcnt type 0: ixg1 q4 Handled queue in softint = 0
   evcnt type 0: ixg1 q4 Requeued in softint = 0
   evcnt type 0: ixg1 q4 TSO = 0
   evcnt type 0: ixg1 q4 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q4 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q4 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q4 qprc = 0
   evcnt type 0: ixg1 q4 qptc = 0
   evcnt type 0: ixg1 q4 qbrc = 0
   evcnt type 0: ixg1 q4 qbtc = 0
   evcnt type 0: ixg1 q4 qprdc = 0
   evcnt type 0: ixg1 q4 Queue Packets Received = 0
   evcnt type 0: ixg1 q4 Queue Bytes Received = 0
   evcnt type 0: ixg1 q4 Copied RX Frames = 0
   evcnt type 0: ixg1 q4 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q4 Rx discarded = 0
   evcnt type 1: ixg1 q5 IRQs on queue = 0
   evcnt type 0: ixg1 q5 Handled queue in softint = 0
   evcnt type 0: ixg1 q5 Requeued in softint = 0
   evcnt type 0: ixg1 q5 TSO = 0
   evcnt type 0: ixg1 q5 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q5 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q5 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q5 qprc = 0
   evcnt type 0: ixg1 q5 qptc = 0
   evcnt type 0: ixg1 q5 qbrc = 0
   evcnt type 0: ixg1 q5 qbtc = 0
   evcnt type 0: ixg1 q5 qprdc = 0
   evcnt type 0: ixg1 q5 Queue Packets Received = 0
   evcnt type 0: ixg1 q5 Queue Bytes Received = 0
   evcnt type 0: ixg1 q5 Copied RX Frames = 0
   evcnt type 0: ixg1 q5 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q5 Rx discarded = 0
   evcnt type 1: ixg1 q6 IRQs on queue = 0
   evcnt type 0: ixg1 q6 Handled queue in softint = 0
   evcnt type 0: ixg1 q6 Requeued in softint = 0
   evcnt type 0: ixg1 q6 TSO = 0
   evcnt type 0: ixg1 q6 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q6 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q6 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q6 qprc = 0
   evcnt type 0: ixg1 q6 qptc = 0
   evcnt type 0: ixg1 q6 qbrc = 0
   evcnt type 0: ixg1 q6 qbtc = 0
   evcnt type 0: ixg1 q6 qprdc = 0
   evcnt type 0: ixg1 q6 Queue Packets Received = 0
   evcnt type 0: ixg1 q6 Queue Bytes Received = 0
   evcnt type 0: ixg1 q6 Copied RX Frames = 0
   evcnt type 0: ixg1 q6 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q6 Rx discarded = 0
   evcnt type 1: ixg1 q7 IRQs on queue = 0
   evcnt type 0: ixg1 q7 Handled queue in softint = 0
   evcnt type 0: ixg1 q7 Requeued in softint = 0
   evcnt type 0: ixg1 q7 TSO = 0
   evcnt type 0: ixg1 q7 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q7 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q7 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q7 qprc = 0
   evcnt type 0: ixg1 q7 qptc = 0
   evcnt type 0: ixg1 q7 qbrc = 0
   evcnt type 0: ixg1 q7 qbtc = 0
   evcnt type 0: ixg1 q7 qprdc = 0
   evcnt type 0: ixg1 q7 Queue Packets Received = 0
   evcnt type 0: ixg1 q7 Queue Bytes Received = 0
   evcnt type 0: ixg1 q7 Copied RX Frames = 0
   evcnt type 0: ixg1 q7 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q7 Rx discarded = 0
   evcnt type 1: ixg1 q8 IRQs on queue = 0
   evcnt type 0: ixg1 q8 Handled queue in softint = 0
   evcnt type 0: ixg1 q8 Requeued in softint = 0
   evcnt type 0: ixg1 q8 TSO = 0
   evcnt type 0: ixg1 q8 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q8 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q8 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q8 qprc = 0
   evcnt type 0: ixg1 q8 qptc = 0
   evcnt type 0: ixg1 q8 qbrc = 0
   evcnt type 0: ixg1 q8 qbtc = 0
   evcnt type 0: ixg1 q8 qprdc = 0
   evcnt type 0: ixg1 q8 Queue Packets Received = 0
   evcnt type 0: ixg1 q8 Queue Bytes Received = 0
   evcnt type 0: ixg1 q8 Copied RX Frames = 0
   evcnt type 0: ixg1 q8 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q8 Rx discarded = 0
   evcnt type 1: ixg1 q9 IRQs on queue = 0
   evcnt type 0: ixg1 q9 Handled queue in softint = 0
   evcnt type 0: ixg1 q9 Requeued in softint = 0
   evcnt type 0: ixg1 q9 TSO = 0
   evcnt type 0: ixg1 q9 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q9 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q9 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q9 qprc = 0
   evcnt type 0: ixg1 q9 qptc = 0
   evcnt type 0: ixg1 q9 qbrc = 0
   evcnt type 0: ixg1 q9 qbtc = 0
   evcnt type 0: ixg1 q9 qprdc = 0
   evcnt type 0: ixg1 q9 Queue Packets Received = 0
   evcnt type 0: ixg1 q9 Queue Bytes Received = 0
   evcnt type 0: ixg1 q9 Copied RX Frames = 0
   evcnt type 0: ixg1 q9 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q9 Rx discarded = 0
   evcnt type 1: ixg1 q10 IRQs on queue = 0
   evcnt type 0: ixg1 q10 Handled queue in softint = 0
   evcnt type 0: ixg1 q10 Requeued in softint = 0
   evcnt type 0: ixg1 q10 TSO = 0
   evcnt type 0: ixg1 q10 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q10 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q10 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q10 qprc = 0
   evcnt type 0: ixg1 q10 qptc = 0
   evcnt type 0: ixg1 q10 qbrc = 0
   evcnt type 0: ixg1 q10 qbtc = 0
   evcnt type 0: ixg1 q10 qprdc = 0
   evcnt type 0: ixg1 q10 Queue Packets Received = 0
   evcnt type 0: ixg1 q10 Queue Bytes Received = 0
   evcnt type 0: ixg1 q10 Copied RX Frames = 0
   evcnt type 0: ixg1 q10 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q10 Rx discarded = 0
   evcnt type 1: ixg1 q11 IRQs on queue = 0
   evcnt type 0: ixg1 q11 Handled queue in softint = 0
   evcnt type 0: ixg1 q11 Requeued in softint = 0
   evcnt type 0: ixg1 q11 TSO = 0
   evcnt type 0: ixg1 q11 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q11 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q11 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q11 qprc = 0
   evcnt type 0: ixg1 q11 qptc = 0
   evcnt type 0: ixg1 q11 qbrc = 0
   evcnt type 0: ixg1 q11 qbtc = 0
   evcnt type 0: ixg1 q11 qprdc = 0
   evcnt type 0: ixg1 q11 Queue Packets Received = 0
   evcnt type 0: ixg1 q11 Queue Bytes Received = 0
   evcnt type 0: ixg1 q11 Copied RX Frames = 0
   evcnt type 0: ixg1 q11 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q11 Rx discarded = 0
   evcnt type 1: ixg1 q12 IRQs on queue = 0
   evcnt type 0: ixg1 q12 Handled queue in softint = 0
   evcnt type 0: ixg1 q12 Requeued in softint = 0
   evcnt type 0: ixg1 q12 TSO = 0
   evcnt type 0: ixg1 q12 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q12 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q12 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q12 qprc = 0
   evcnt type 0: ixg1 q12 qptc = 0
   evcnt type 0: ixg1 q12 qbrc = 0
   evcnt type 0: ixg1 q12 qbtc = 0
   evcnt type 0: ixg1 q12 qprdc = 0
   evcnt type 0: ixg1 q12 Queue Packets Received = 0
   evcnt type 0: ixg1 q12 Queue Bytes Received = 0
   evcnt type 0: ixg1 q12 Copied RX Frames = 0
   evcnt type 0: ixg1 q12 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q12 Rx discarded = 0
   evcnt type 1: ixg1 q13 IRQs on queue = 0
   evcnt type 0: ixg1 q13 Handled queue in softint = 0
   evcnt type 0: ixg1 q13 Requeued in softint = 0
   evcnt type 0: ixg1 q13 TSO = 0
   evcnt type 0: ixg1 q13 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q13 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q13 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q13 qprc = 0
   evcnt type 0: ixg1 q13 qptc = 0
   evcnt type 0: ixg1 q13 qbrc = 0
   evcnt type 0: ixg1 q13 qbtc = 0
   evcnt type 0: ixg1 q13 qprdc = 0
   evcnt type 0: ixg1 q13 Queue Packets Received = 0
   evcnt type 0: ixg1 q13 Queue Bytes Received = 0
   evcnt type 0: ixg1 q13 Copied RX Frames = 0
   evcnt type 0: ixg1 q13 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q13 Rx discarded = 0
   evcnt type 1: ixg1 q14 IRQs on queue = 0
   evcnt type 0: ixg1 q14 Handled queue in softint = 0
   evcnt type 0: ixg1 q14 Requeued in softint = 0
   evcnt type 0: ixg1 q14 TSO = 0
   evcnt type 0: ixg1 q14 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q14 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q14 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q14 qprc = 0
   evcnt type 0: ixg1 q14 qptc = 0
   evcnt type 0: ixg1 q14 qbrc = 0
   evcnt type 0: ixg1 q14 qbtc = 0
   evcnt type 0: ixg1 q14 qprdc = 0
   evcnt type 0: ixg1 q14 Queue Packets Received = 0
   evcnt type 0: ixg1 q14 Queue Bytes Received = 0
   evcnt type 0: ixg1 q14 Copied RX Frames = 0
   evcnt type 0: ixg1 q14 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q14 Rx discarded = 0
   evcnt type 1: ixg1 q15 IRQs on queue = 0
   evcnt type 0: ixg1 q15 Handled queue in softint = 0
   evcnt type 0: ixg1 q15 Requeued in softint = 0
   evcnt type 0: ixg1 q15 TSO = 0
   evcnt type 0: ixg1 q15 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q15 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q15 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q15 qprc = 0
   evcnt type 0: ixg1 q15 qptc = 0
   evcnt type 0: ixg1 q15 qbrc = 0
   evcnt type 0: ixg1 q15 qbtc = 0
   evcnt type 0: ixg1 q15 qprdc = 0
   evcnt type 0: ixg1 q15 Queue Packets Received = 0
   evcnt type 0: ixg1 q15 Queue Bytes Received = 0
   evcnt type 0: ixg1 q15 Copied RX Frames = 0
   evcnt type 0: ixg1 q15 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q15 Rx discarded = 0
   evcnt type 1: ixg1 q16 IRQs on queue = 0
   evcnt type 0: ixg1 q16 Handled queue in softint = 0
   evcnt type 0: ixg1 q16 Requeued in softint = 0
   evcnt type 0: ixg1 q16 TSO = 0
   evcnt type 0: ixg1 q16 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q16 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q16 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q16 Queue Packets Received = 0
   evcnt type 0: ixg1 q16 Queue Bytes Received = 0
   evcnt type 0: ixg1 q16 Copied RX Frames = 0
   evcnt type 0: ixg1 q16 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q16 Rx discarded = 0
   evcnt type 1: ixg1 q17 IRQs on queue = 0
   evcnt type 0: ixg1 q17 Handled queue in softint = 0
   evcnt type 0: ixg1 q17 Requeued in softint = 0
   evcnt type 0: ixg1 q17 TSO = 0
   evcnt type 0: ixg1 q17 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q17 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q17 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q17 Queue Packets Received = 0
   evcnt type 0: ixg1 q17 Queue Bytes Received = 0
   evcnt type 0: ixg1 q17 Copied RX Frames = 0
   evcnt type 0: ixg1 q17 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q17 Rx discarded = 0
   evcnt type 1: ixg1 q18 IRQs on queue = 0
   evcnt type 0: ixg1 q18 Handled queue in softint = 0
   evcnt type 0: ixg1 q18 Requeued in softint = 0
   evcnt type 0: ixg1 q18 TSO = 0
   evcnt type 0: ixg1 q18 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q18 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q18 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q18 Queue Packets Received = 0
   evcnt type 0: ixg1 q18 Queue Bytes Received = 0
   evcnt type 0: ixg1 q18 Copied RX Frames = 0
   evcnt type 0: ixg1 q18 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q18 Rx discarded = 0
   evcnt type 1: ixg1 q19 IRQs on queue = 0
   evcnt type 0: ixg1 q19 Handled queue in softint = 0
   evcnt type 0: ixg1 q19 Requeued in softint = 0
   evcnt type 0: ixg1 q19 TSO = 0
   evcnt type 0: ixg1 q19 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q19 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q19 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q19 Queue Packets Received = 0
   evcnt type 0: ixg1 q19 Queue Bytes Received = 0
   evcnt type 0: ixg1 q19 Copied RX Frames = 0
   evcnt type 0: ixg1 q19 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q19 Rx discarded = 0
   evcnt type 1: ixg1 q20 IRQs on queue = 0
   evcnt type 0: ixg1 q20 Handled queue in softint = 0
   evcnt type 0: ixg1 q20 Requeued in softint = 0
   evcnt type 0: ixg1 q20 TSO = 0
   evcnt type 0: ixg1 q20 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q20 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q20 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q20 Queue Packets Received = 0
   evcnt type 0: ixg1 q20 Queue Bytes Received = 0
   evcnt type 0: ixg1 q20 Copied RX Frames = 0
   evcnt type 0: ixg1 q20 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q20 Rx discarded = 0
   evcnt type 1: ixg1 q21 IRQs on queue = 0
   evcnt type 0: ixg1 q21 Handled queue in softint = 0
   evcnt type 0: ixg1 q21 Requeued in softint = 0
   evcnt type 0: ixg1 q21 TSO = 0
   evcnt type 0: ixg1 q21 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q21 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q21 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q21 Queue Packets Received = 0
   evcnt type 0: ixg1 q21 Queue Bytes Received = 0
   evcnt type 0: ixg1 q21 Copied RX Frames = 0
   evcnt type 0: ixg1 q21 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q21 Rx discarded = 0
   evcnt type 1: ixg1 q22 IRQs on queue = 0
   evcnt type 0: ixg1 q22 Handled queue in softint = 0
   evcnt type 0: ixg1 q22 Requeued in softint = 0
   evcnt type 0: ixg1 q22 TSO = 0
   evcnt type 0: ixg1 q22 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q22 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q22 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q22 Queue Packets Received = 0
   evcnt type 0: ixg1 q22 Queue Bytes Received = 0
   evcnt type 0: ixg1 q22 Copied RX Frames = 0
   evcnt type 0: ixg1 q22 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q22 Rx discarded = 0
   evcnt type 1: ixg1 q23 IRQs on queue = 0
   evcnt type 0: ixg1 q23 Handled queue in softint = 0
   evcnt type 0: ixg1 q23 Requeued in softint = 0
   evcnt type 0: ixg1 q23 TSO = 0
   evcnt type 0: ixg1 q23 Queue No Descriptor Available = 0
   evcnt type 0: ixg1 q23 Queue Packets Transmitted = 0
   evcnt type 0: ixg1 q23 Packets dropped in pcq = 0
   evcnt type 0: ixg1 q23 Queue Packets Received = 0
   evcnt type 0: ixg1 q23 Queue Bytes Received = 0
   evcnt type 0: ixg1 q23 Copied RX Frames = 0
   evcnt type 0: ixg1 q23 Rx no jumbo mbuf = 0
   evcnt type 0: ixg1 q23 Rx discarded = 0
   evcnt type 0: ixg1 MAC Statistics rx csum offload - IP = 0
   evcnt type 0: ixg1 MAC Statistics rx csum offload - L4 = 0
   evcnt type 0: ixg1 MAC Statistics rx csum offload - IP bad = 0
   evcnt type 0: ixg1 MAC Statistics rx csum offload - L4 bad = 0
   evcnt type 0: ixg1 MAC Statistics Interrupt conditions zero = 0
   evcnt type 0: ixg1 MAC Statistics Legacy interrupts = 0
   evcnt type 0: ixg1 MAC Statistics CRC Errors = 0
   evcnt type 0: ixg1 MAC Statistics Illegal Byte Errors = 0
   evcnt type 0: ixg1 MAC Statistics Byte Errors = 0
   evcnt type 0: ixg1 MAC Statistics MAC Short Packets Discarded = 0
   evcnt type 0: ixg1 MAC Statistics Total Packets Missed = 0
   evcnt type 0: ixg1 MAC Statistics MAC Local Faults = 0
   evcnt type 0: ixg1 MAC Statistics MAC Remote Faults = 0
   evcnt type 0: ixg1 MAC Statistics Receive Length Errors = 0
   evcnt type 0: ixg1 MAC Statistics Link XON Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Link XON Received = 0
   evcnt type 0: ixg1 MAC Statistics Link XOFF Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Link XOFF Received = 0
   evcnt type 0: ixg1 MAC Statistics Total Octets Received = 0
   evcnt type 0: ixg1 MAC Statistics Good Octets Received = 0
   evcnt type 0: ixg1 MAC Statistics Total Packets Received = 0
   evcnt type 0: ixg1 MAC Statistics Good Packets Received = 0
   evcnt type 0: ixg1 MAC Statistics Multicast Packets Received = 0
   evcnt type 0: ixg1 MAC Statistics Broadcast Packets Received = 0
   evcnt type 0: ixg1 MAC Statistics 64 byte frames received  = 0
   evcnt type 0: ixg1 MAC Statistics 65-127 byte frames received = 0
   evcnt type 0: ixg1 MAC Statistics 128-255 byte frames received = 0
   evcnt type 0: ixg1 MAC Statistics 256-511 byte frames received = 0
   evcnt type 0: ixg1 MAC Statistics 512-1023 byte frames received = 0
   evcnt type 0: ixg1 MAC Statistics 1023-1522 byte frames received = 0
   evcnt type 0: ixg1 MAC Statistics Receive Undersized = 0
   evcnt type 0: ixg1 MAC Statistics Fragmented Packets Received  = 0
   evcnt type 0: ixg1 MAC Statistics Oversized Packets Received = 0
   evcnt type 0: ixg1 MAC Statistics Received Jabber = 0
   evcnt type 0: ixg1 MAC Statistics Management Packets Received = 0
   evcnt type 0: ixg1 MAC Statistics Management Packets Dropped = 0
   evcnt type 0: ixg1 MAC Statistics Checksum Errors = 0
   evcnt type 0: ixg1 MAC Statistics Good Octets Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Total Packets Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Good Packets Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Broadcast Packets Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Multicast Packets Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics Management Packets Transmitted = 0
   evcnt type 0: ixg1 MAC Statistics 64 byte frames transmitted  = 0
   evcnt type 0: ixg1 MAC Statistics 65-127 byte frames transmitted = 0
   evcnt type 0: ixg1 MAC Statistics 128-255 byte frames transmitted = 0
   evcnt type 0: ixg1 MAC Statistics 256-511 byte frames transmitted = 0
   evcnt type 0: ixg1 MAC Statistics 512-1023 byte frames transmitted = 0
   evcnt type 0: ixg1 MAC Statistics 1024-1522 byte frames transmitted = 0
   evcnt type 1: msi2 vec 0 = 0
   evcnt type 1: ioapic0 pin 19 = 0
   evcnt type 1: ioapic0 pin 18 = 0
   evcnt type 1: msix3 vec 0 = 0
   evcnt type 1: msix3 vec 1 = 0
   evcnt type 1: msix3 vec 2 = 0
   evcnt type 1: msix3 vec 3 = 0
   evcnt type 1: msix3 vec 4 = 0
   evcnt type 1: msix3 vec 5 = 0
   evcnt type 1: msix3 vec 6 = 0
   evcnt type 1: msix3 vec 7 = 0
   evcnt type 1: msix3 vec 8 = 0
   evcnt type 1: msix4 vec 0 = 0
   evcnt type 1: msix4 vec 1 = 0
   evcnt type 1: msix4 vec 2 = 0
   evcnt type 1: msix4 vec 3 = 0
   evcnt type 1: msix4 vec 4 = 0
   evcnt type 1: msix4 vec 5 = 0
   evcnt type 1: msix4 vec 6 = 0
   evcnt type 1: msix4 vec 7 = 0
   evcnt type 1: msix4 vec 8 = 0
   evcnt type 1: msi5 vec 0 = 0
   evcnt type 1: ioapic0 pin 4 = 0
   evcnt type 1: ioapic0 pin 3 = 0
   evcnt type 0: acpicpu0 C1 (MWAIT) = 6845
   evcnt type 0: acpicpu1 C1 (MWAIT) = 18742
   evcnt type 0: acpicpu2 C1 (MWAIT) = 18754
   evcnt type 0: acpicpu3 C1 (MWAIT) = 18693
   evcnt type 0: acpicpu4 C1 (MWAIT) = 18737
   evcnt type 0: acpicpu5 C1 (MWAIT) = 18735
   evcnt type 0: acpicpu6 C1 (MWAIT) = 18725
   evcnt type 0: acpicpu7 C1 (MWAIT) = 18731
   evcnt type 0: acpicpu8 C1 (MWAIT) = 18708
   evcnt type 0: acpicpu9 C1 (MWAIT) = 18737
   evcnt type 0: acpicpu10 C1 (MWAIT) = 18722
   evcnt type 0: acpicpu11 C1 (MWAIT) = 18704
   evcnt type 0: acpicpu12 C1 (MWAIT) = 18699
   evcnt type 0: acpicpu13 C1 (MWAIT) = 18700
   evcnt type 0: acpicpu14 C1 (MWAIT) = 18726
   evcnt type 0: acpicpu15 C1 (MWAIT) = 18699
   evcnt type 0: acpicpu16 C1 (MWAIT) = 18721
   evcnt type 0: acpicpu17 C1 (MWAIT) = 18712
   evcnt type 0: acpicpu18 C1 (MWAIT) = 18739
   evcnt type 0: acpicpu19 C1 (MWAIT) = 18714
   evcnt type 0: acpicpu20 C1 (MWAIT) = 18683
   evcnt type 0: acpicpu21 C1 (MWAIT) = 18712
   evcnt type 0: acpicpu22 C1 (MWAIT) = 18715
   evcnt type 0: acpicpu23 C1 (MWAIT) = 18712
   evcnt type 0: kpreempt defer: critical section = 0
   evcnt type 0: kpreempt defer: kernel_lock = 28
   evcnt type 0: kpreempt immediate = 3
   evcnt type 0: ipi full = 0
   db{8}> 

 -- 
 Andreas Gustafsson, gson@gson.org

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 18:52:22 +0100

 On Sun, Mar 08, 2020 at 07:16:50PM +0200, Andreas Gustafsson wrote:
 > Manuel Bouyer wrote:
 > > I think you can get them from ddb, with something like
 > > show event /i
 > 
 > I had no luck getting the serial console to respond to a break, but
 > enabling the front panel NMI button in the BIOS and pressing it got me
 > into ddb.  With the -current INSTALL kernel, it says:
 > 
 >   db{12}> show event/i 
 >   db_read_ptr: cannot find `allevents'
 > 
 > With the -current GENERIC kernel, it says:
 > 
 >   db{8}> show event/i
 >   evcnt type 1: TLB shootdown = 96
 >   evcnt type 1: cpu0 timer = 6868
 >   db{8}>
 > [...]
 >   db{8}> show event /f
 >   [...]
 >   evcnt type 1: msi2 vec 0 = 0
 >   evcnt type 1: ioapic0 pin 19 = 0
 >   evcnt type 1: ioapic0 pin 18 = 0
 >   evcnt type 1: msi5 vec 0 = 0

 these are the interrupts used by ahcisata and usb controllers. They should
 not be 0 at this point.

 So we have interrupts routing issue. Someone with more clues about ACPI
 and x86 interrupt routing should look at this.

 Any option about this in the BIOS ?

 Would NetBSD 8.1 boot on this hardware ?
 I have 8.1 running on poweredge R540 and R440 without problems.

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: Andreas Gustafsson <gson@gson.org>
To: Manuel Bouyer <bouyer@antioche.eu.org>
Cc: gnats-bugs@netbsd.org,
    matthew green <mrg@eterna.com.au>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 22:10:25 +0200

 Manuel Bouyer wrote:
 > So we have interrupts routing issue. Someone with more clues about ACPI
 > and x86 interrupt routing should look at this.
 > 
 > Any option about this in the BIOS ?

 Not that I have noticed.

 > Would NetBSD 8.1 boot on this hardware ?

 No, it behaves the same as 9.0 and -current.  See the console log at

   http://www.gson.org/netbsd/bugs/r630/dmesg-8.1.txt

 -- 
 Andreas Gustafsson, gson@gson.org

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org, matthew green <mrg@eterna.com.au>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 21:28:08 +0100

 On Sun, Mar 08, 2020 at 10:10:25PM +0200, Andreas Gustafsson wrote:
 > Manuel Bouyer wrote:
 > > So we have interrupts routing issue. Someone with more clues about ACPI
 > > and x86 interrupt routing should look at this.
 > > 
 > > Any option about this in the BIOS ?
 > 
 > Not that I have noticed.
 > 
 > > Would NetBSD 8.1 boot on this hardware ?
 > 
 > No, it behaves the same as 9.0 and -current.  See the console log at
 > 
 >   http://www.gson.org/netbsd/bugs/r630/dmesg-8.1.txt

 So it's really a problem with the way ACPI provides interrupt
 routing (or the way we use it).

 Silly question: is the BIOS up to date ?

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: Andreas Gustafsson <gson@gson.org>
To: Manuel Bouyer <bouyer@antioche.eu.org>
Cc: gnats-bugs@netbsd.org,
    matthew green <mrg@eterna.com.au>
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Sun, 8 Mar 2020 22:41:31 +0200

 Manuel Bouyer wrote:
 > Silly question: is the BIOS up to date ?

 Yes.  The machine came with BIOS version 2.3.4, and when NetBSD failed
 to boot with that, one of the first things I did was to update it to
 the current version, 2.11.0.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Mon, 16 Mar 2020 15:06:20 +0200

 I tried switching the BIOS to boot in UEFI mode and booting
 NetBSD-9.0-amd64-uefi-install.img from a USB flash drive.
 It fails the same way as NetBSD-9.0-amd64-install.img.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: Manuel Bouyer <bouyer@antioche.eu.org>, gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/54984: USB and SATA broken on Dell PowerEdge R630
Date: Wed, 18 Mar 2020 12:36:41 +0200

 On March 8, Manuel Bouyer wrote:
 >  So we have interrupts routing issue. Someone with more clues about ACPI
 >  and x86 interrupt routing should look at this.
 >  
 >  Any option about this in the BIOS ?

 Turns out the BIOS has more Processor Settings than fit on the
 screen, and scrolling down past the end reveals some additional
 settings including one called "X2Apic Mode".  After changing that
 from "Enabled" to "Disabled", NetBSD-current boots with working
 USB and SATA!  Thanks to Manuel for the clue.

 So there is now a work-around, but I'm leaving the PR open as the
 X2Apic mode is still broken.
 -- 
 Andreas Gustafsson, gson@gson.org

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.