NetBSD Problem Report #57596

From www@netbsd.org  Wed Aug 30 17:36:35 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id B31811A9238
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 30 Aug 2023 17:36:35 +0000 (UTC)
Message-Id: <20230830173633.9F8C01A9239@mollari.NetBSD.org>
Date: Wed, 30 Aug 2023 17:36:33 +0000 (UTC)
From: jaimef@linbsd.org
Reply-To: jaimef@linbsd.org
To: gnats-bugs@NetBSD.org
Subject: OCTEON kernel panic on MP enablement.
X-Send-Pr-Version: www-1.0

>Number:         57596
>Category:       port-evbmips
>Synopsis:       OCTEON kernel panic on MP enablement.
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-evbmips-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Aug 30 17:40:00 +0000 2023
>Originator:     Jaime Fournier
>Release:        
>Organization:
Me
>Environment:
NetBSD octeon 10.0_BETA NetBSD 10.0_BETA (OCTEON) #0: Tue Aug 29 19:58:59 UTC 2023  root@octeon:/usr/src/sys/arch/evbmips/compile/OCTEON evbmips
Although this crash is booting the latest trunk with options MULTIPROCESSOR enabled.
>Description:

USB1:   Starting the controller
USB XHCI 1.00
scanning bus 1 for devices... 1 USB Device(s) found
       scanning usb for storage devices... 1 Storage Device(s) found
reading netbsd
7643512 bytes read in 188 ms (38.8 MiB/s)
Allocating memory for ELF segment: addr: 0xffffffff80200000 (adjusted to: 0x200000), size 0x698ac0
## Loading big-endian Linux kernel with entry point: 0xffffffff80200000 ...
Bootloader: Done loading app on coremask:
 0xf
Starting cores:
 0xf
[   1.0000000] MIPS32/64 params: cpu arch: 256
[   1.0000000] MIPS32/64 params: TLB entries: 256
[   1.0000000] MIPS32/64 params: Icache: line=128, total=79872, ways=39, sets=16, colors=0
[   1.0000000] MIPS32/64 params: Dcache: line=128, total=32768, ways=32, sets=8, colors=0
[   1.0000000] MIPS32/64 params: SDcache: line=128, total=1048576, ways=8, sets=1024, colors=16
[   1.0000000]   Dcache is coherent
[   1.0000000] u-boot bootmem desc @ 0x6c108 version 3.0
[   1.0000000] phys segment: 0xf8000 @ 0x108000
[   1.0000000] adding 0xf8000 @ 0x108000 to freelist 0
[   1.0000000] phys segment: 0xf516000 @ 0x89a000
[   1.0000000] adding 0xf516000 @ 0x89a000 to freelist 0
[   1.0000000] phys segment: 0xc000 @ 0xfdb4000
[   1.0000000] adding 0xc000 @ 0xfdb4000 to freelist 0
[   1.0000000] phys segment: 0x800 @ 0xfdc2000 (short)
[   1.0000000] phys segment: 0x2f0 @ 0xffb6000 (short)
[   1.0000000] phys segment: 0x80 @ 0xffd6000 (short)
[   1.0000000] phys segment: 0x40 @ 0xfff6000 (short)
[   1.0000000] phys segment: 0x60 @ 0xfff6000 (short)
[   1.0000000] phys segment: 0x2effe000 @ 0x20000000
[   1.0000000] adding 0x2effe000 @ 0x20000000 to freelist 0
[   1.0000000] phys segment: 0x1080 @ 0x4f000000 (short)
[   1.0000000] Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003,
[   1.0000000]     2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013,
[   1.0000000]     2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021, 2022, 2023
[   1.0000000]     The NetBSD Foundation, Inc.  All rights reserved.
[   1.0000000] Copyright (c) 1982, 1986, 1989, 1991, 1993
[   1.0000000]     The Regents of the University of California.  All rights reserved.

[   1.0000000] NetBSD 10.99.7 (OCTEON) #0: Wed Aug 30 17:12:37 UTC 2023
[   1.0000000]  root@:/usr/obj/sys/arch/evbmips/compile/OCTEON
[   1.0000000] Cavium Octeon CN7130-1000
[   1.0000000] total memory = 1024 MB
[   1.0000000] avail memory = 980 MB
[   1.0000000] mainbus0 (root)
[   1.0000000] cpunode0 at mainbus0: 4 cores, crypto+kasumi, 64bit-mul, unaligned-access ok, booted 0xf
[   1.0000000] cpu0 at cpunode0 core 0: 1000.00MHz
[   1.0000000] cpu0: Cavium CN7130-1000 (0xd9602) Rev. 2 with built-in FPU
[   1.0000000] cpu0: 256 TLB entries, 512TB (49-bit) VAs, 512TB (49-bit) PAs, 256MB max page size
[   1.0000000] cpu0: 78KB/128B 39-way set-associative L1 instruction cache
[   1.0000000] cpu0: 32KB/128B 32-way set-associative write-through coherent L1 data cache
[   1.0000000] cpu0: 1024KB/128B 8-way set-associative write-back L2 unified cache
[   1.0000000] cpu1 at cpunode0 core 1: 1000.00MHz
[   1.0000000] cpu2 at cpunode0 core 2: 1000.00MHz
[   1.0000000] cpu3 at cpunode0 core 3: 1000.00MHz
[   1.0000000] wdog0 at cpunode0: default period is 4 seconds
[   1.0000000] iobus0 at mainbus0
[   1.0000000] iobus0: initializing POW
[   1.0000000] iobus0: initializing FPA
[   1.0000000] octrnm0 at iobus0 address 0x0001180040000000
[   1.0000000] entropy: ready
[   1.0000000] simplebus0 at mainbus0: cavium,ubnt_e300
[   1.0000000] simplebus1 at simplebus0
[   1.0000000] octintc0 at simplebus1: CIU
[   1.0000000] octcib0 at simplebus1: CIB
[   1.0000000] octintc0: CIU irq 144 (2, 16) out of range
[   1.0000000] octcib0: couldn't establish interrupt on CIU irq 144
[   1.0000000] octcib1 at simplebus1: CIB
[   1.0000000] octcib1: interrupting on CIU irq 97
[   1.0000000] octcib2 at simplebus1: CIB
[   1.0000000] octcib2: interrupting on CIU irq 81
[   1.0000000] octcib3 at simplebus1: CIB
[   1.0000000] octintc0: CIU irq 145 (2, 17) out of range
[   1.0000000] octcib3: couldn't establish interrupt on CIU irq 145
[   1.0000000] octcib4 at simplebus1: CIB
[   1.0000000] octcib4: interrupting on CIU irq 88
[   1.0000000] octcib5 at simplebus1: CIB
[   1.0000000] octcib5: interrupting on CIU irq 116
[   1.0000000] octcib6 at simplebus1: CIB
[   1.0000000] octcib6: interrupting on CIU irq 127
[   1.0000000] octsmi0 at simplebus1
[   1.0000000] octsmi1 at simplebus1
[   1.0000000] octpip0 at simplebus1
[   1.0000000] octgmx0 at octpip0
[   1.0000000] cnmac0 at octgmx0: address=0x1180008000000: SGMII
[   1.0000000] cnmac0: Ethernet address 18:e8:29:45:c5:6f
[   1.0000000] ukphy0 at cnmac0 phy 4: Vitesse Semiconductor Vitesse VSC8504 quad 10/100/1000TX PHY (OUI 0x008083, model 0x000c), rev. 2
[   1.0000000] ukphy0: 1000baseT, 1000baseT-FDX, auto
[   1.0000000] cnmac1 at octgmx0: address=0x1180008000000: SGMII
[   1.0000000] cnmac1: Ethernet address 18:e8:29:45:c5:70
[   1.0000000] ukphy1 at cnmac1 phy 5: Vitesse Semiconductor Vitesse VSC8504 quad 10/100/1000TX PHY (OUI 0x008083, model 0x000c), rev. 2
[   1.0000000] ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
[   1.0000000] cnmac2 at octgmx0: address=0x1180008000000: SGMII
[   1.0000000] cnmac2: Ethernet address 18:e8:29:45:c5:71
[   1.0000000] ukphy2 at cnmac2 phy 6: Vitesse Semiconductor Vitesse VSC8504 quad 10/100/1000TX PHY (OUI 0x008083, model 0x000c), rev. 2
[   1.0000000] ukphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
[   1.0000000] cnmac3 at octgmx0: address=0x1180008000000: SGMII
[   1.0000000] cnmac3: Ethernet address 18:e8:29:45:c5:72
[   1.0000000] ukphy3 at cnmac3 phy 7: Vitesse Semiconductor Vitesse VSC8504 quad 10/100/1000TX PHY (OUI 0x008083, model 0x000c), rev. 2
[   1.0000000] ukphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
[   1.0000000] /soc@0/gpio-controller@1070000000800 at simplebus1 not configured
[   1.0000000] /gpio-leds at simplebus0 not configured
[   1.0000000] /soc@0/bootbus@1180000000000 at simplebus1 not configured
[   1.0000000] /soc@0/uctl@118006c000000 at simplebus1 not configured
[   1.0000000] xhci0 at simplebus1: DWC3 rev 0x250a
[   1.0000000] xhci0: interrupting on octcib1 intr 9
[   1.0000000] usb0 at xhci0: USB revision 3.0
[   1.0000000] usb1 at xhci0: USB revision 2.0
[   1.0000000] xhci1 at simplebus1: DWC3 rev 0x250a
[   1.0000000] xhci1: interrupting on octcib2 intr 9
[   1.0000000] usb2 at xhci1: USB revision 3.0
[   1.0000000] usb3 at xhci1: USB revision 2.0
[   1.0000000] /soc@0/i2c@1180000001000 at simplebus1 not configured
[   1.0000000] /soc@0/i2c@1180000001200 at simplebus1 not configured
[   1.0000000] com0 at simplebus1: ns16650, no ERS, 16-byte FIFO
[   1.0000000] com0: console
[   1.0000000] com0: interrupting on CIU irq 34
[   1.0000000] com1 at simplebus1: ns16650, no ERS, 16-byte FIFO
[   1.0000000] com1: interrupting on CIU irq 35
[   1.0000000] /soc@0/mmc@1180000002000 at simplebus1 not configured
[   1.0000000] /soc@0/spi@1070000001000 at simplebus1 not configured
[   1.0000000] /soc@0/ocla0@11800A8000000 at simplebus1 not configured
[   1.0000000] /soc@0/dma-engine@1180000000100 at simplebus1 not configured
[   1.0000000] /soc@0/dma-engine@1180000000108 at simplebus1 not configured
[   1.0000030] cpu1: Cavium CN7130-1000 (0xd9602) Rev. 2 with built-in FPU
[   1.0000030] cpu1: 256 TLB entries, 512TB (49-bit) VAsReader / writer lock error: rw_vector_enter,291: assertion failed: !cpu_intr_p()

[   1.0100030] lock address : netbsd:kernel_map_store+0x8
[   1.0200030] current cpu  :                  1
[   1.0200030] current lwp  : 0x980000004eda0080
[   1.0300030] owner/count  : 000000000000000000 flags    : 000000000000000000

[   1.0400030] panic: lock error: Reader / writer lock: rw_vector_enter,291: assertion failed: !cpu_intr_p(): lock 0xffffffff80884170 cpu 1 lwp 0x980000004eda0080
[   1.0500030] cpu1: Begin traceback...
[   1.0500030] pid 2028464 not found
[   1.0600030] cpu1: End traceback...
[   1.0600030] kernel: breakpoint trap
Stopped in pid 0.16 (system) at netbsd:cpu_Debugger+0x4:        jr      ra
                bdslot: nop
db{1}> ,bt
db{1}> ,bed
db{1}> bt
0x98000000001ef3d0: cpu_Debugger+0x4 (0x4,0x7000,0x10,0xffffffff80001580) ra 0xf
fffffff804ed6f0 sz 0
0x98000000001ef3d0: vpanic+0x1a0 (0x4,0x7000,0x10,0xffffffff80001580) ra 0xfffff
fff804ed7c0 sz 64
0x98000000001ef410: panic+0x30 (0x4,0x98000000001ef438,0xffffffff80655430,0x123)
 ra 0xffffffff804e0460 sz 96
0x98000000001ef470: lockdebug_abort+0x118 (0x4,0x98000000001ef438,0xffffffff8065
5430,0x123) ra 0xffffffff804a3ba8 sz 208
0x98000000001ef540: rw_enter+0x3d0 (0x4,0,0xffffffff80655430,0x123) ra 0xfffffff
f8042e558 sz 144
0x98000000001ef5d0: uvm_fault_internal+0x130 (0,0,0x1,0x123) ra 0xffffffff802536
98 sz 560
0x98000000001ef800: trap+0x768 (0,0,0x1,0xffffffff804c124c) ra 0xffffffff80242f5
4 sz 560
0x98000000001efa30: mips64r2_kern_gen_exception+0xd8 (0x70a3,0x8,0x6,0xffffffff8
04c124c) ra 0xffffffff802012e0 sz 352
0x98000000001efb90: cpu_intr+0xf8 (0x70a3,0x8,0x6,0xffffffff804c124c) ra 0xfffff
fff806701a8 sz 144
0x98000000001efc20: ostype+0xd2a8 (0x70a3,0x8,0x6,0xffffffff804c124c) ra 0 sz 0
User-level: pid 0.16

>How-To-Repeat:
Add `options MULTIPROCESSOR` to sys/arch/evbmips/conf/OCTEON.
Boot latest HEAD.
>Fix:

Home	PR Database Search