NetBSD Problem Report #26383

Received: (qmail 21789 invoked by uid 605); 20 Jul 2004 22:13:27 -0000
Message-Id: <m1Bn2rU-00000AC@building.weird.com>
Date: Tue, 20 Jul 2004 18:13:24 -0400 (EDT)
From: "Greg A. Woods" <woods@weird.com>
Sender: gnats-bugs-owner@NetBSD.org
Reply-To: "Greg A. Woods" <woods@planix.com>
To: gnats-bugs@gnats.netbsd.org
Subject: "panic: fpsave ipi didn't" on alpha SMP
X-Send-Pr-Version: 3.95

>Number:         26383
>Category:       port-alpha
>Synopsis:       "panic: fpsave ipi didn't" on alpha SMP
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    thorpej
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jul 20 22:14:00 +0000 2004
>Closed-Date:    Thu Jul 28 14:04:22 +0000 2005
>Last-Modified:  Mon Aug 29 00:58:01 +0000 2005
>Originator:     Greg A. Woods
>Release:        NetBSD 1.6.2_STABLE
>Organization:
Planix, Inc.; Toronto, Ontario; Canada
>Environment:
System: NetBSD 1.6.2_STABLE 
Architecture: alpha
Machine: alpha
>Description:

	NetBSD-1.6.2_STABLE running on a dual processor AS4000

	note: as you can see below, the reboot failed and a manual reset
	was necessary

	This all appears to be the same as the problem reported in the
	"Random panics with 2-0 on CS20" thread starting at:

		http://mail-index.netbsd.org/port-alpha/2004/05/28/0000.html

	other related threads:

		http://mail-index.netbsd.org/port-alpha/2002/11/10/0000.html
		http://mail-index.netbsd.org/port-alpha/2002/10/22/0001.html

	I don't see a related PR anywhere though, thus this one.

panic: fpsave ipi didn't
Stopped in pid 230 (ntpd) at    cpu_Debugger+0x4:       ret     zero,(ra)
db{1}> where
cpu_Debugger() at cpu_Debugger+0x4
panic() at panic+0x160
fpusave_proc() at fpusave_proc+0x2fc
sendsig() at sendsig+0x150
postsig() at postsig+0x1c8
syscall_plain() at syscall_plain+0x238
XentSys() at XentSys+0x5c
--- syscall (4) ---
--- user mode ---
db{1}> show reg
v0                 0x6
t0          0xfffffc00006c2290  db_onpanic
t1                 0x1
t2          0xfffffc005fff0000
t3                   0
t4          0xfffffc0000608c6a  fmt.162+0x1068
t5               0x3e9  rn+0x3c9
t6                 0x1
t7          0xfffffc00007c89a0  cycles_per_usec
s0          0xfffffe003292fac8
s1          0xfffffc0000000008
s2          0xfffffc00007b4d20  msgbufp
s3          0xfffffc00006c30e4  msgbufenabled
s4               0x100  rn+0xe0
s5                0x20  rn
s6          0xfffffc0000695efd  msgperiod.210+0xac5
a0                   0
a1                 0x1
a2                 0x5
a3                 0x8  rettmp
a4                 0x3
a5                 0x8  rettmp
t8                   0
t9          0xfffffc00005ba864  microtime+0xc4
t10         0x134f14b7ede64
t11         0x17d174f6
ra          0xfffffc000045ba20  panic+0x160
t12         0xfffffc00005cc3e0  cpu_Debugger
at          0xfffffc00006c30cc  sched_whichqs
gp          0xfffffc00006b2dd8  special_symbols+0x8080
sp          0xfffffe003292fa48
pc          0xfffffc00005cc3e4  cpu_Debugger+0x4
ps                   0
ai          0x17d174f6
pv          0xfffffc00005cc3e0  cpu_Debugger
cpu_Debugger+0x4:       ret     zero,(ra)
db{1}> reboot
syncing disks... ^]^]rcm

RCM>status

Firmware Rev: V1.1
Escape Sequence: ^]^]RCM
Remote Access: DISABLE
Alerts: DISABLE
Alert Pending: NO
Temp (C): 42.0
RCM Power Control: ON
External Power: OFF
Server Power: ON

RCM>reset

Focus returned to COM port



>How-To-Repeat:

	unknown (it's only happened once so far)

	(two mozilla-gtk2 processes were running at the time, plus
	named, ntpd, and various xterms, ssh2d, ssh2 clients, cron,
	inetd, nfsd, mountd, rpcbind, rpc.lockd, rpc.statd, syslogd,
	smail, shells, emacs, gettys, etc.)

>Fix:

	unknown

>Release-Note:
>Audit-Trail:

From: "Michael L. Hitch" <mhitch@lightning.msu.montana.edu>
To: gnats-bugs@gnats.netbsd.org
Cc:  
Subject: Re: port-alpha/26383: "panic: fpsave ipi didn't" on alpha SMP
Date: Sun, 8 Aug 2004 22:50:02 -0600 (MDT)

 > 	I don't see a related PR anywhere though, thus this one.

   You have beaten me to the PR.

 > panic: fpsave ipi didn't
 > Stopped in pid 230 (ntpd) at    cpu_Debugger+0x4:       ret     zero,(ra)
 > db{1}> where

   The panic occurs because CPU1 needs to synchronize the FPU state for the
 current process, but the FPU state is on a different CPU (CPU0 in this
 case, as there are only 2).  The CPU holding the current FPU state fails
 to respond to the IPI request, and CPU1 times out and panics.

   The other CPU is probably spinning waiting for a lock, and when CPU1 is
 running DDB, the other CPUs are paused.  If they detect the pause IPI
 while spinning for a lock, the CPU does not have any register state and
 DDB is unable to get any information about that CPU.  In addition, a bug
 in handling the paused CPUs will leave the other CPUs spinning waiting to
 be resumed if you attempt to reboot or halt via DDB.  [This bug has been
 fixed in -current and the 2.0 branch now].  I found this bug while trying
 to debug the fpsave panic, and one thing I did was to tell the other CPU
 to resume by clearing cpus_paused manually before continuing DDB to let it
 finish the panic.  The other CPU was still spinning on the lock and
 eventually paniced.  At this point, I had the stack trace for CPU0
 available, which provides some clues as the the reason for this panic.

   What appears to be happening is that CPU0 gets an FPU fault because the
 FPU has not been enabled.  The FPU state for CPU0 is for a different
 process, so it tries to save the FPU state.  In the middle of the save, it
 gets an I/O interrupt and eventually attempts to get the kernel_lock.
 However, CPU1 has that kernel lock and attempts to get CPU0 to save the
 FPU state, but because CPU0 is in the middle of saveing the FPU state, the
 IPI request is ignored.  Then CPU1 panics.

   I have been testing the following patch, which disables the interrupts
 during the FPU state save operation.  The CPUF_FPUSAVE flag is what causes
 the IPI request to save FPU state to be ignored.

 Index: sys/arch/alpha/alpha/machdep.c
 ===================================================================
 RCS file: /cvsroot/src/sys/arch/alpha/alpha/machdep.c,v
 retrieving revision 1.283
 diff -u -r1.283 machdep.c
 --- sys/arch/alpha/alpha/machdep.c      7 Jul 2004 02:46:14 -0000       1.283
 +++ sys/arch/alpha/alpha/machdep.c      9 Aug 2004 04:27:01 -0000
 @@ -1700,12 +1701,13 @@
  {
         struct lwp *l;
  #if defined(MULTIPROCESSOR)
 -       int s;
 +       int s, s2;
  #endif

         KDASSERT(ci == curcpu());

  #if defined(MULTIPROCESSOR)
 +       s2 = splhigh();
         atomic_setbits_ulong(&ci->ci_flags, CPUF_FPUSAVE);
  #endif

 @@ -1730,6 +1732,7 @@
   out:
  #if defined(MULTIPROCESSOR)
         atomic_clearbits_ulong(&ci->ci_flags, CPUF_FPUSAVE);
 +       splx(s2);
  #endif
         return;
  }
 ~

 --
 Michael L. Hitch			mhitch@montana.edu
 Computer Consultant
 Information Technology Center
 Montana State University	Bozeman, MT	USA

State-Changed-From-To: open->feedback
State-Changed-By: thorpej@netbsd.org
State-Changed-When: Thu, 28 Jul 2005 13:57:55 +0000
State-Changed-Why:
Fixed in -current, requesting pullup to netbsd-3.


From: Jason R Thorpe <thorpej@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/26383 CVS commit: src/sys/arch/alpha
Date: Thu, 28 Jul 2005 13:57:06 +0000 (UTC)

 Module Name:	src
 Committed By:	thorpej
 Date:		Thu Jul 28 13:57:06 UTC 2005

 Modified Files:
 	src/sys/arch/alpha/alpha: machdep.c trap.c
 	src/sys/arch/alpha/include: pcb.h

 Log Message:
 - Change FPCPU_LOCK() such that the caller is responsible for blocking
   IPIs in the MULTIPROCESSOR case.  Adjust all callers.
 - In fpusave_cpu(), block IPIs for the entire duration (while we have
   CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
   "panic: fpsave ipi didn't", as described in PR port-alpha/26383.
   Many thanks to Michael Hitch for the analysis and initial patch which
   this one is derived from.


 To generate a diff of this commit:
 cvs rdiff -r1.286 -r1.287 src/sys/arch/alpha/alpha/machdep.c
 cvs rdiff -r1.97 -r1.98 src/sys/arch/alpha/alpha/trap.c
 cvs rdiff -r1.12 -r1.13 src/sys/arch/alpha/include/pcb.h

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: port-alpha-maintainer->thorpej
Responsible-Changed-By: thorpej@netbsd.org
Responsible-Changed-When: Thu, 28 Jul 2005 14:02:14 +0000
Responsible-Changed-Why:
I checked in the patch.


State-Changed-From-To: feedback->closed
State-Changed-By: thorpej@netbsd.org
State-Changed-When: Thu, 28 Jul 2005 14:04:22 +0000
State-Changed-Why:
Received feedback from Michael already.


From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/26383 CVS commit: [netbsd-3] src/sys/arch/alpha/include
Date: Sat, 30 Jul 2005 17:55:47 +0000 (UTC)

 Module Name:	src
 Committed By:	tron
 Date:		Sat Jul 30 17:55:47 UTC 2005

 Modified Files:
 	src/sys/arch/alpha/include [netbsd-3]: pcb.h

 Log Message:
 Pull up revision 1.13 (requested by thorpej in ticket #629):
 - Change FPCPU_LOCK() such that the caller is responsible for blocking
   IPIs in the MULTIPROCESSOR case.  Adjust all callers.
 - In fpusave_cpu(), block IPIs for the entire duration (while we have
   CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
   "panic: fpsave ipi didn't", as described in PR port-alpha/26383.
   Many thanks to Michael Hitch for the analysis and initial patch which
   this one is derived from.


 To generate a diff of this commit:
 cvs rdiff -r1.12 -r1.12.16.1 src/sys/arch/alpha/include/pcb.h

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/26383 CVS commit: [netbsd-3] src/sys/arch/alpha/alpha
Date: Sat, 30 Jul 2005 17:55:54 +0000 (UTC)

 Module Name:	src
 Committed By:	tron
 Date:		Sat Jul 30 17:55:54 UTC 2005

 Modified Files:
 	src/sys/arch/alpha/alpha [netbsd-3]: trap.c

 Log Message:
 Pull up revision 1.98 (requested by thorpej in ticket #629):
 - Change FPCPU_LOCK() such that the caller is responsible for blocking
   IPIs in the MULTIPROCESSOR case.  Adjust all callers.
 - In fpusave_cpu(), block IPIs for the entire duration (while we have
   CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
   "panic: fpsave ipi didn't", as described in PR port-alpha/26383.
   Many thanks to Michael Hitch for the analysis and initial patch which
   this one is derived from.


 To generate a diff of this commit:
 cvs rdiff -r1.96 -r1.96.10.1 src/sys/arch/alpha/alpha/trap.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/26383 CVS commit: [netbsd-3] src/sys/arch/alpha/alpha
Date: Sat, 30 Jul 2005 17:56:02 +0000 (UTC)

 Module Name:	src
 Committed By:	tron
 Date:		Sat Jul 30 17:56:02 UTC 2005

 Modified Files:
 	src/sys/arch/alpha/alpha [netbsd-3]: machdep.c

 Log Message:
 Pull up revision 1.287 (requested by thorpej in ticket #629):
 - Change FPCPU_LOCK() such that the caller is responsible for blocking
   IPIs in the MULTIPROCESSOR case.  Adjust all callers.
 - In fpusave_cpu(), block IPIs for the entire duration (while we have
   CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
   "panic: fpsave ipi didn't", as described in PR port-alpha/26383.
   Many thanks to Michael Hitch for the analysis and initial patch which
   this one is derived from.


 To generate a diff of this commit:
 cvs rdiff -r1.283 -r1.283.10.1 src/sys/arch/alpha/alpha/machdep.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/26383 CVS commit: [netbsd-2] src/sys/arch/alpha
Date: Mon, 29 Aug 2005 00:57:12 +0000 (UTC)

 Module Name:	src
 Committed By:	tron
 Date:		Mon Aug 29 00:57:12 UTC 2005

 Modified Files:
 	src/sys/arch/alpha/alpha [netbsd-2]: machdep.c trap.c
 	src/sys/arch/alpha/include [netbsd-2]: pcb.h

 Log Message:
 Pull up following revision(s) (requested by riz in ticket #5623):
 	sys/arch/alpha/alpha/machdep.c: revision 1.287
 	sys/arch/alpha/alpha/trap.c: revision 1.98
 	sys/arch/alpha/include/pcb.h: revision 1.13
 - Change FPCPU_LOCK() such that the caller is responsible for blocking
   IPIs in the MULTIPROCESSOR case.  Adjust all callers.
 - In fpusave_cpu(), block IPIs for the entire duration (while we have
   CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
   "panic: fpsave ipi didn't", as described in PR port-alpha/26383.
   Many thanks to Michael Hitch for the analysis and initial patch which
   this one is derived from.


 To generate a diff of this commit:
 cvs rdiff -r1.282.2.1 -r1.282.2.1.2.1 src/sys/arch/alpha/alpha/machdep.c
 cvs rdiff -r1.92.2.2 -r1.92.2.2.2.1 src/sys/arch/alpha/alpha/trap.c
 cvs rdiff -r1.12 -r1.12.8.1 src/sys/arch/alpha/include/pcb.h

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.