NetBSD Problem Report #26383
Received: (qmail 21789 invoked by uid 605); 20 Jul 2004 22:13:27 -0000
Message-Id: <m1Bn2rU-00000AC@building.weird.com>
Date: Tue, 20 Jul 2004 18:13:24 -0400 (EDT)
From: "Greg A. Woods" <woods@weird.com>
Sender: gnats-bugs-owner@NetBSD.org
Reply-To: "Greg A. Woods" <woods@planix.com>
To: gnats-bugs@gnats.netbsd.org
Subject: "panic: fpsave ipi didn't" on alpha SMP
X-Send-Pr-Version: 3.95
>Number: 26383
>Category: port-alpha
>Synopsis: "panic: fpsave ipi didn't" on alpha SMP
>Confidential: no
>Severity: critical
>Priority: medium
>Responsible: thorpej
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Jul 20 22:14:00 +0000 2004
>Closed-Date: Thu Jul 28 14:04:22 +0000 2005
>Last-Modified: Mon Aug 29 00:58:01 +0000 2005
>Originator: Greg A. Woods
>Release: NetBSD 1.6.2_STABLE
>Organization:
Planix, Inc.; Toronto, Ontario; Canada
>Environment:
System: NetBSD 1.6.2_STABLE
Architecture: alpha
Machine: alpha
>Description:
NetBSD-1.6.2_STABLE running on a dual processor AS4000
note: as you can see below, the reboot failed and a manual reset
was necessary
This all appears to be the same as the problem reported in the
"Random panics with 2-0 on CS20" thread starting at:
http://mail-index.netbsd.org/port-alpha/2004/05/28/0000.html
other related threads:
http://mail-index.netbsd.org/port-alpha/2002/11/10/0000.html
http://mail-index.netbsd.org/port-alpha/2002/10/22/0001.html
I don't see a related PR anywhere though, thus this one.
panic: fpsave ipi didn't
Stopped in pid 230 (ntpd) at cpu_Debugger+0x4: ret zero,(ra)
db{1}> where
cpu_Debugger() at cpu_Debugger+0x4
panic() at panic+0x160
fpusave_proc() at fpusave_proc+0x2fc
sendsig() at sendsig+0x150
postsig() at postsig+0x1c8
syscall_plain() at syscall_plain+0x238
XentSys() at XentSys+0x5c
--- syscall (4) ---
--- user mode ---
db{1}> show reg
v0 0x6
t0 0xfffffc00006c2290 db_onpanic
t1 0x1
t2 0xfffffc005fff0000
t3 0
t4 0xfffffc0000608c6a fmt.162+0x1068
t5 0x3e9 rn+0x3c9
t6 0x1
t7 0xfffffc00007c89a0 cycles_per_usec
s0 0xfffffe003292fac8
s1 0xfffffc0000000008
s2 0xfffffc00007b4d20 msgbufp
s3 0xfffffc00006c30e4 msgbufenabled
s4 0x100 rn+0xe0
s5 0x20 rn
s6 0xfffffc0000695efd msgperiod.210+0xac5
a0 0
a1 0x1
a2 0x5
a3 0x8 rettmp
a4 0x3
a5 0x8 rettmp
t8 0
t9 0xfffffc00005ba864 microtime+0xc4
t10 0x134f14b7ede64
t11 0x17d174f6
ra 0xfffffc000045ba20 panic+0x160
t12 0xfffffc00005cc3e0 cpu_Debugger
at 0xfffffc00006c30cc sched_whichqs
gp 0xfffffc00006b2dd8 special_symbols+0x8080
sp 0xfffffe003292fa48
pc 0xfffffc00005cc3e4 cpu_Debugger+0x4
ps 0
ai 0x17d174f6
pv 0xfffffc00005cc3e0 cpu_Debugger
cpu_Debugger+0x4: ret zero,(ra)
db{1}> reboot
syncing disks... ^]^]rcm
RCM>status
Firmware Rev: V1.1
Escape Sequence: ^]^]RCM
Remote Access: DISABLE
Alerts: DISABLE
Alert Pending: NO
Temp (C): 42.0
RCM Power Control: ON
External Power: OFF
Server Power: ON
RCM>reset
Focus returned to COM port
>How-To-Repeat:
unknown (it's only happened once so far)
(two mozilla-gtk2 processes were running at the time, plus
named, ntpd, and various xterms, ssh2d, ssh2 clients, cron,
inetd, nfsd, mountd, rpcbind, rpc.lockd, rpc.statd, syslogd,
smail, shells, emacs, gettys, etc.)
>Fix:
unknown
>Release-Note:
>Audit-Trail:
From: "Michael L. Hitch" <mhitch@lightning.msu.montana.edu>
To: gnats-bugs@gnats.netbsd.org
Cc:
Subject: Re: port-alpha/26383: "panic: fpsave ipi didn't" on alpha SMP
Date: Sun, 8 Aug 2004 22:50:02 -0600 (MDT)
> I don't see a related PR anywhere though, thus this one.
You have beaten me to the PR.
> panic: fpsave ipi didn't
> Stopped in pid 230 (ntpd) at cpu_Debugger+0x4: ret zero,(ra)
> db{1}> where
The panic occurs because CPU1 needs to synchronize the FPU state for the
current process, but the FPU state is on a different CPU (CPU0 in this
case, as there are only 2). The CPU holding the current FPU state fails
to respond to the IPI request, and CPU1 times out and panics.
The other CPU is probably spinning waiting for a lock, and when CPU1 is
running DDB, the other CPUs are paused. If they detect the pause IPI
while spinning for a lock, the CPU does not have any register state and
DDB is unable to get any information about that CPU. In addition, a bug
in handling the paused CPUs will leave the other CPUs spinning waiting to
be resumed if you attempt to reboot or halt via DDB. [This bug has been
fixed in -current and the 2.0 branch now]. I found this bug while trying
to debug the fpsave panic, and one thing I did was to tell the other CPU
to resume by clearing cpus_paused manually before continuing DDB to let it
finish the panic. The other CPU was still spinning on the lock and
eventually paniced. At this point, I had the stack trace for CPU0
available, which provides some clues as the the reason for this panic.
What appears to be happening is that CPU0 gets an FPU fault because the
FPU has not been enabled. The FPU state for CPU0 is for a different
process, so it tries to save the FPU state. In the middle of the save, it
gets an I/O interrupt and eventually attempts to get the kernel_lock.
However, CPU1 has that kernel lock and attempts to get CPU0 to save the
FPU state, but because CPU0 is in the middle of saveing the FPU state, the
IPI request is ignored. Then CPU1 panics.
I have been testing the following patch, which disables the interrupts
during the FPU state save operation. The CPUF_FPUSAVE flag is what causes
the IPI request to save FPU state to be ignored.
Index: sys/arch/alpha/alpha/machdep.c
===================================================================
RCS file: /cvsroot/src/sys/arch/alpha/alpha/machdep.c,v
retrieving revision 1.283
diff -u -r1.283 machdep.c
--- sys/arch/alpha/alpha/machdep.c 7 Jul 2004 02:46:14 -0000 1.283
+++ sys/arch/alpha/alpha/machdep.c 9 Aug 2004 04:27:01 -0000
@@ -1700,12 +1701,13 @@
{
struct lwp *l;
#if defined(MULTIPROCESSOR)
- int s;
+ int s, s2;
#endif
KDASSERT(ci == curcpu());
#if defined(MULTIPROCESSOR)
+ s2 = splhigh();
atomic_setbits_ulong(&ci->ci_flags, CPUF_FPUSAVE);
#endif
@@ -1730,6 +1732,7 @@
out:
#if defined(MULTIPROCESSOR)
atomic_clearbits_ulong(&ci->ci_flags, CPUF_FPUSAVE);
+ splx(s2);
#endif
return;
}
~
--
Michael L. Hitch mhitch@montana.edu
Computer Consultant
Information Technology Center
Montana State University Bozeman, MT USA
State-Changed-From-To: open->feedback
State-Changed-By: thorpej@netbsd.org
State-Changed-When: Thu, 28 Jul 2005 13:57:55 +0000
State-Changed-Why:
Fixed in -current, requesting pullup to netbsd-3.
From: Jason R Thorpe <thorpej@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/26383 CVS commit: src/sys/arch/alpha
Date: Thu, 28 Jul 2005 13:57:06 +0000 (UTC)
Module Name: src
Committed By: thorpej
Date: Thu Jul 28 13:57:06 UTC 2005
Modified Files:
src/sys/arch/alpha/alpha: machdep.c trap.c
src/sys/arch/alpha/include: pcb.h
Log Message:
- Change FPCPU_LOCK() such that the caller is responsible for blocking
IPIs in the MULTIPROCESSOR case. Adjust all callers.
- In fpusave_cpu(), block IPIs for the entire duration (while we have
CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
"panic: fpsave ipi didn't", as described in PR port-alpha/26383.
Many thanks to Michael Hitch for the analysis and initial patch which
this one is derived from.
To generate a diff of this commit:
cvs rdiff -r1.286 -r1.287 src/sys/arch/alpha/alpha/machdep.c
cvs rdiff -r1.97 -r1.98 src/sys/arch/alpha/alpha/trap.c
cvs rdiff -r1.12 -r1.13 src/sys/arch/alpha/include/pcb.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Responsible-Changed-From-To: port-alpha-maintainer->thorpej
Responsible-Changed-By: thorpej@netbsd.org
Responsible-Changed-When: Thu, 28 Jul 2005 14:02:14 +0000
Responsible-Changed-Why:
I checked in the patch.
State-Changed-From-To: feedback->closed
State-Changed-By: thorpej@netbsd.org
State-Changed-When: Thu, 28 Jul 2005 14:04:22 +0000
State-Changed-Why:
Received feedback from Michael already.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/26383 CVS commit: [netbsd-3] src/sys/arch/alpha/include
Date: Sat, 30 Jul 2005 17:55:47 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sat Jul 30 17:55:47 UTC 2005
Modified Files:
src/sys/arch/alpha/include [netbsd-3]: pcb.h
Log Message:
Pull up revision 1.13 (requested by thorpej in ticket #629):
- Change FPCPU_LOCK() such that the caller is responsible for blocking
IPIs in the MULTIPROCESSOR case. Adjust all callers.
- In fpusave_cpu(), block IPIs for the entire duration (while we have
CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
"panic: fpsave ipi didn't", as described in PR port-alpha/26383.
Many thanks to Michael Hitch for the analysis and initial patch which
this one is derived from.
To generate a diff of this commit:
cvs rdiff -r1.12 -r1.12.16.1 src/sys/arch/alpha/include/pcb.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/26383 CVS commit: [netbsd-3] src/sys/arch/alpha/alpha
Date: Sat, 30 Jul 2005 17:55:54 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sat Jul 30 17:55:54 UTC 2005
Modified Files:
src/sys/arch/alpha/alpha [netbsd-3]: trap.c
Log Message:
Pull up revision 1.98 (requested by thorpej in ticket #629):
- Change FPCPU_LOCK() such that the caller is responsible for blocking
IPIs in the MULTIPROCESSOR case. Adjust all callers.
- In fpusave_cpu(), block IPIs for the entire duration (while we have
CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
"panic: fpsave ipi didn't", as described in PR port-alpha/26383.
Many thanks to Michael Hitch for the analysis and initial patch which
this one is derived from.
To generate a diff of this commit:
cvs rdiff -r1.96 -r1.96.10.1 src/sys/arch/alpha/alpha/trap.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/26383 CVS commit: [netbsd-3] src/sys/arch/alpha/alpha
Date: Sat, 30 Jul 2005 17:56:02 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Sat Jul 30 17:56:02 UTC 2005
Modified Files:
src/sys/arch/alpha/alpha [netbsd-3]: machdep.c
Log Message:
Pull up revision 1.287 (requested by thorpej in ticket #629):
- Change FPCPU_LOCK() such that the caller is responsible for blocking
IPIs in the MULTIPROCESSOR case. Adjust all callers.
- In fpusave_cpu(), block IPIs for the entire duration (while we have
CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
"panic: fpsave ipi didn't", as described in PR port-alpha/26383.
Many thanks to Michael Hitch for the analysis and initial patch which
this one is derived from.
To generate a diff of this commit:
cvs rdiff -r1.283 -r1.283.10.1 src/sys/arch/alpha/alpha/machdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/26383 CVS commit: [netbsd-2] src/sys/arch/alpha
Date: Mon, 29 Aug 2005 00:57:12 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Mon Aug 29 00:57:12 UTC 2005
Modified Files:
src/sys/arch/alpha/alpha [netbsd-2]: machdep.c trap.c
src/sys/arch/alpha/include [netbsd-2]: pcb.h
Log Message:
Pull up following revision(s) (requested by riz in ticket #5623):
sys/arch/alpha/alpha/machdep.c: revision 1.287
sys/arch/alpha/alpha/trap.c: revision 1.98
sys/arch/alpha/include/pcb.h: revision 1.13
- Change FPCPU_LOCK() such that the caller is responsible for blocking
IPIs in the MULTIPROCESSOR case. Adjust all callers.
- In fpusave_cpu(), block IPIs for the entire duration (while we have
CPUF_FPUSAVE set in ci_flags) to fix the deadlock that leads to
"panic: fpsave ipi didn't", as described in PR port-alpha/26383.
Many thanks to Michael Hitch for the analysis and initial patch which
this one is derived from.
To generate a diff of this commit:
cvs rdiff -r1.282.2.1 -r1.282.2.1.2.1 src/sys/arch/alpha/alpha/machdep.c
cvs rdiff -r1.92.2.2 -r1.92.2.2.2.1 src/sys/arch/alpha/alpha/trap.c
cvs rdiff -r1.12 -r1.12.8.1 src/sys/arch/alpha/include/pcb.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.