NetBSD Problem Report #49691
From kamada@nanohz.org Tue Feb 24 13:10:41 2015
Return-Path: <kamada@nanohz.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 90C30A57FE
for <gnats-bugs@gnats.NetBSD.org>; Tue, 24 Feb 2015 13:10:41 +0000 (UTC)
Message-Id: <20150224221031NN%kamada@nanohz.org>
Date: Tue, 24 Feb 2015 22:10:31 +0900
From: KAMADA Ken'ichi <kamada@nanohz.org>
To: gnats-bugs@NetBSD.org
Cc: kamada@nanohz.org
Subject: KASSERT(l->l_md.md_gc_ptp == NULL) failed on NetBSD/amd64 7.0_BETA
>Number: 49691
>Category: kern
>Synopsis: KASSERT(l->l_md.md_gc_ptp == NULL) failed on NetBSD/amd64 7.0_BETA
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Feb 24 13:15:00 +0000 2015
>Closed-Date: Sun Dec 18 21:14:23 +0000 2016
>Last-Modified: Sun Dec 18 21:14:23 +0000 2016
>Originator: KAMADA Ken'ichi
>Release: NetBSD 7.0_BETA
>Organization:
>Environment:
NetBSD example.org 7.0_BETA NetBSD 7.0_BETA (DIAG) #0: Mon Feb 23 23:16:18 JST 2015 ken@example.org:/usr/src/sys/arch/amd64/compile/DIAG amd64
Architecture: x86_64
Machine: amd64
kernel: GENERIC + options DIAGNOSTIC
>Description:
I see occasional panics on NetBSD/amd64 7.0_BETA boxes (checked out
on Feb. 23). Seen on both a real hardware (custom kernconf) and
a VirtualBox guest (GENERIC kernel).
The kernels have the DIAGNOSTIC option enabled.
panic: kernel diagnostic assertion "l->l_md.md_gc_ptp == NULL" failed: file "../../../../arch/x86/x86/vm_machdep.c", line 262
cpu0: Begin traceback...
vpanic() at netbsd:vpanic+0x13c
kern_assert() at netbsd:kern_assert+0x4f
cpu_lwp_free2() at netbsd:cpu_lwp_free2+0x42
lwp_free() at netbsd:lwp_free+0x22f
lwp_wait() at netbsd:lwp_wait+0x7c
exit_lwps() at netbsd:exit_lwps+0xca
exit1() at netbsd:exit1+0x65
sys_exit() at netbsd:sys_exit+0x3a
syscall() at netbsd:syscall+0x9a
--- syscall (number 1) ---
4a0a64:
cpu0: End traceback...
dumping to dev 0,1 (offset=1050072, size=130959):
dump
(gdb) bt
#0 0xffffffff805b2885 in cpu_reboot (howto=howto@entry=260,
bootstr=bootstr@entry=0x0) at ../../../../arch/amd64/amd64/machdep.c:671
#1 0xffffffff807ae1b4 in vpanic (
fmt=0xffffffff80bab850 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=ap@entry=0xfffffe80027f4ca0) at ../../../../kern/subr_prf.c:340
#2 0xffffffff8093f003 in kern_assert (
fmt=fmt@entry=0xffffffff80bab850 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ") at ../../../../../../lib/libkern/kern_assert.c:51
#3 0xffffffff808dbb34 in cpu_lwp_free2 (l=l@entry=0xfffffe800fc6baa0)
at ../../../../arch/x86/x86/vm_machdep.c:262
#4 0xffffffff8052d2f8 in lwp_free (l=0xfffffe800fc6baa0,
recycle=recycle@entry=false, last=last@entry=false)
at ../../../../kern/kern_lwp.c:1268
#5 0xffffffff8052e423 in lwp_wait (l=l@entry=0xfffffe8010a13640,
lid=lid@entry=0, departed=departed@entry=0x0, exiting=exiting@entry=true)
at ../../../../kern/kern_lwp.c:569
#6 0xffffffff8052453e in exit_lwps (l=l@entry=0xfffffe8010a13640)
at ../../../../kern/kern_exit.c:632
#7 0xffffffff805247a0 in exit1 (l=l@entry=0xfffffe8010a13640, rv=768)
at ../../../../kern/kern_exit.c:219
#8 0xffffffff80525056 in sys_exit (l=0xfffffe8010a13640,
uap=0xfffffe80027f4f00, retval=<optimized out>)
at ../../../../kern/kern_exit.c:181
#9 0xffffffff807c810a in sy_call (rval=0xfffffe80027f4eb8,
uap=0xfffffe80027f4f00, l=0xfffffe8010a13640,
sy=0xffffffff80f1a5b0 <sysent+16>) at ../../../../sys/syscallvar.h:61
#10 sy_invoke (code=1, rval=0xfffffe80027f4eb8, uap=0xfffffe80027f4f00,
l=0xfffffe8010a13640, sy=0xffffffff80f1a5b0 <sysent+16>)
at ../../../../sys/syscallvar.h:85
#11 syscall (frame=0xfffffe80027f4f00)
at ../../../../arch/x86/x86/syscall.c:156
#12 0xffffffff80100691 in Xsyscall ()
>How-To-Repeat:
Not known.
>Fix:
>Release-Note:
>Audit-Trail:
From: KAMADA Ken'ichi <kamada@nanohz.org>
To: gnats-bugs@NetBSD.org
Cc: kamada@nanohz.org
Subject: Re: kern/49691: KASSERT(l->l_md.md_gc_ptp == NULL) failed on NetBSD/amd64 7.0_BETA
Date: Sun, 08 Mar 2015 14:49:17 +0900
>How-To-Repeat:
The following program triggers this panic.
% cat > crash.go
package main
import "time"
func main() {
var s []int
s = make([]int, 1024 * 1024)
s = make([]int, 1024 * 1024)
s[0] = 1
time.Sleep(550 * time.Second)
}
^D
% go version
go version go1.4.2 netbsd/amd64
% go build crash.go
% ./crash
(Wait until the program exit.)
From: Mihai Chelaru <chelaru@gmail.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/49691: KASSERT(l->l_md.md_gc_ptp == NULL) failed on
NetBSD/amd64 7.0_BETA
Date: Sun, 7 Aug 2016 20:50:03 +0100
The below patch fixes this issue, but I'd like someone with more
knowledge than me in this area to review it before anything else.
Index: sys/arch/x86/x86/vm_machdep.c
===================================================================
RCS file: /cvsroot/src/sys/arch/x86/x86/vm_machdep.c,v
retrieving revision 1.25
diff -d -u -p -r1.25 vm_machdep.c
--- sys/arch/x86/x86/vm_machdep.c 11 Mar 2014 20:55:19 -0000 1.25
+++ sys/arch/x86/x86/vm_machdep.c 7 Aug 2016 19:04:49 -0000
@@ -249,6 +249,20 @@ cpu_lwp_free(struct lwp *l, int proc)
if (proc && l->l_proc->p_md.md_flags & MDP_USEDMTRR)
mtrr_clean(l->l_proc);
#endif
+ /*
+ * Free the deferred mappings
+ */
+ if (l->l_md.md_gc_ptp != NULL) {
+ struct vm_page *ptp;
+ struct pmap_page *pp;
+
+ while ((ptp = l->l_md.md_gc_ptp) != NULL) {
+ pp = &ptp->mdpage.mp_pp;
+ l->l_md.md_gc_ptp = pp->pp_link;
+ LIST_INIT(&pp->pp_head.pvh_list);
+ uvm_pagefree(ptp);
+ }
+ }
}
/*
From: KAMADA Ken'ichi <kamada@nanohz.org>
To: gnats-bugs@NetBSD.org
Cc: chelaru@gmail.com
Subject: Re: kern/49691: KASSERT(l->l_md.md_gc_ptp == NULL) failed on NetBSD/amd64 7.0_BETA
Date: Mon, 08 Aug 2016 14:40:36 +0900
My previous patch
<https://mail-index.netbsd.org/port-amd64/2015/06/23/msg002276.html>
does the same thing as Mihai's one. My concern was that the patch may
not fix the root cause but just hide the symptom.
From: coypu@SDF.ORG
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/49691
Date: Mon, 8 Aug 2016 10:47:00 +0000
I thought it might have to do with PR/42663.
From: Mihai Chelaru <chelaru@gmail.com>
To: "KAMADA Ken'ichi" <kamada@nanohz.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/49691: KASSERT(l->l_md.md_gc_ptp == NULL) failed on
NetBSD/amd64 7.0_BETA
Date: Tue, 9 Aug 2016 09:19:36 +0100
Oh, I wasn't aware there is a patch already. Indeed, I don't know if
we should have deferred mappings by the time cpu_lwp_free() is called.
On Mon, Aug 8, 2016 at 6:40 AM, KAMADA Ken'ichi <kamada@nanohz.org> wrote:
> My previous patch
> <https://mail-index.netbsd.org/port-amd64/2015/06/23/msg002276.html>
> does the same thing as Mihai's one. My concern was that the patch may
> not fix the root cause but just hide the symptom.
From: Thomas Klausner <tk@giga.or.at>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc:
Subject: Re: kern/49691
Date: Mon, 7 Nov 2016 23:19:25 +0100
I just saw this on -current (7.99.42 from Nov 2), also with a go
program (syncthing).
Below my mail to current-users:
----- Forwarded message from Thomas Klausner <tk@giga.or.at> -----
Hi!
Today I was running a bulk build and an idle syncthing
(net/syncthing). When I tried to quit syncthing using CTRL-C (which
worked once before on the same day), I got a kernel panic:
panic: kernel diagnostic assertion "l->l_md.md_gc_ptp == NULL" failed: file "/disk/6/archive/foreign/src/sys/arch/x86/x86/vm_machdep.c", line 262 WARNING: SPL NOT LOWERED ON SYSCALL 146 4 EXIT 81c27c08 6
dmesg on boot contained lots of spam at the start:
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 233 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 233 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 233 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
lwp_userret() at WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 250 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
netbsd:lwp_userret+0xf9
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 257 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWEREDW OANR NSIYNSGC:A LSLP L1 N6O TE XLIOTW E0R E7D
ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 233 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 233 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 77 13 EXIT 60139800 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL W1A R6N IENXGI:T S0P L7
NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 33s3y s4c aElXlI(T) W 8Aa1RtcN 2I7NcG0:8 S6P
L NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 288 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
netbsd:syscall+0x208
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED O- -S- YsSysCcAaLlLl 1( n6u mbEerX 4I)T -0- -
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 279 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
4930ba:
WARNING: SPL NOT LOWERED ON SYSCALL 283 4 EXIT 81c27c08 6
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
cpu6: End traceback...
WARNING: SPL NOT LOWERED ON TRAP EXIT 6 0
WARNING: SPL NOT LOWERED ON SYSCALL 1 6 EXIT 0 7
dumping to dev 168,3 (offset=8, size=8373576):
dump Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016
The NetBSD Foundation, Inc. All rights reserved.
Copyright (c) 1982, 1986, 1989, 1991, 1993
The Regents of the University of California. All rights reserved.
NetBSD 7.99.42 (KVOTHE) #23: Wed Nov 2 09:38:07 CET 2016
wiz@yt.nih.at:/disk/6/archive/foreign/src/sys/arch/amd64/compile/obj/KVOTHE
...
netbsd.core.101.gz exists, so I can provide more detail:
(gdb) bt
#0 0xffffffff80119a85 in cpu_reboot (howto=howto@entry=260, bootstr=bootstr@entry=0x0) at /disk/6/archive/foreign/src/sys/arch/amd64/amd64/machdep.c:676
#1 0xffffffff8089072c in vpanic (fmt=0xffffffff80ed31f8 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ", ap=ap@entry=0xfffffe8720904d78)
at /disk/6/archive/foreign/src/sys/kern/subr_prf.c:342
#2 0xffffffff80bd0b65 in kern_assert (fmt=fmt@entry=0xffffffff80ed31f8 "kernel %sassertion \"%s\" failed: file \"%s\", line %d ")
at /disk/6/archive/foreign/src/sys/lib/libkern/kern_assert.c:51
#3 0xffffffff8013efa7 in cpu_lwp_free2 (l=l@entry=0xfffffe82c7577360) at /disk/6/archive/foreign/src/sys/arch/x86/x86/vm_machdep.c:262
#4 0xffffffff8084be91 in lwp_free (l=0xfffffe82c7577360, recycle=recycle@entry=false, last=last@entry=false) at /disk/6/archive/foreign/src/sys/kern/kern_lwp.c:1263
#5 0xffffffff8084cc66 in lwp_exit (l=l@entry=0xfffffe824b4a1700) at /disk/6/archive/foreign/src/sys/kern/kern_lwp.c:1093
#6 0xffffffff8084cf39 in lwp_userret (l=l@entry=0xfffffe824b4a1700) at /disk/6/archive/foreign/src/sys/kern/kern_lwp.c:1555
#7 0xffffffff8013e5c9 in mi_userret (l=0xfffffe824b4a1700) at /disk/6/archive/foreign/src/sys/sys/userret.h:94
#8 userret (l=0xfffffe824b4a1700) at ./machine/userret.h:82
#9 syscall (frame=0xfffffe8720904f00) at /disk/6/archive/foreign/src/sys/arch/x86/x86/syscall.c:184
#10 0xffffffff80100761 in Xsyscall ()
Let me know if I should file a PR and what more information you'd like to see.
Thanks,
Thomas
----- End forwarded message -----
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/49691 CVS commit: src/sys/arch/x86
Date: Mon, 7 Nov 2016 22:05:36 -0500
Module Name: src
Committed By: christos
Date: Tue Nov 8 03:05:36 UTC 2016
Modified Files:
src/sys/arch/x86/include: pmap.h
src/sys/arch/x86/x86: pmap.c vm_machdep.c
Log Message:
PR/49691: KAMADA Ken'ichi: free deferred ptp mappings if present.
XXX: pullup-7
To generate a diff of this commit:
cvs rdiff -u -r1.60 -r1.61 src/sys/arch/x86/include/pmap.h
cvs rdiff -u -r1.222 -r1.223 src/sys/arch/x86/x86/pmap.c
cvs rdiff -u -r1.25 -r1.26 src/sys/arch/x86/x86/vm_machdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 21 Nov 2016 06:55:39 +0000
State-Changed-Why:
is this fixed?
From: KAMADA Ken'ichi <kamada@nanohz.org>
To: gnats-bugs@NetBSD.org
Cc: dholland@NetBSD.org, kamada@nanohz.org
Subject: Re: kern/49691 (KASSERT(l->l_md.md_gc_ptp == NULL) failed on NetBSD/amd64 7.0_BETA)
Date: Tue, 22 Nov 2016 22:45:09 +0900
Fixed in -current, but not yet in netbsd-7.
From: "Soren Jacobsen" <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/49691 CVS commit: [netbsd-7] src/sys/arch/x86
Date: Sun, 18 Dec 2016 07:01:57 +0000
Module Name: src
Committed By: snj
Date: Sun Dec 18 07:01:57 UTC 2016
Modified Files:
src/sys/arch/x86/include [netbsd-7]: pmap.h
src/sys/arch/x86/x86 [netbsd-7]: pmap.c vm_machdep.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1316):
sys/arch/x86/x86/pmap.c: revision 1.223
sys/arch/x86/x86/vm_machdep.c: revision 1.26
sys/arch/x86/include/pmap.h: revision 1.61
PR/49691: KAMADA Ken'ichi: free deferred ptp mappings if present.
XXX: pullup-7
To generate a diff of this commit:
cvs rdiff -u -r1.55.4.1 -r1.55.4.2 src/sys/arch/x86/include/pmap.h
cvs rdiff -u -r1.183.2.4 -r1.183.2.5 src/sys/arch/x86/x86/pmap.c
cvs rdiff -u -r1.25 -r1.25.4.1 src/sys/arch/x86/x86/vm_machdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Soren Jacobsen" <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/49691 CVS commit: [netbsd-7-0] src/sys/arch/x86
Date: Sun, 18 Dec 2016 07:02:59 +0000
Module Name: src
Committed By: snj
Date: Sun Dec 18 07:02:59 UTC 2016
Modified Files:
src/sys/arch/x86/include [netbsd-7-0]: pmap.h
src/sys/arch/x86/x86 [netbsd-7-0]: pmap.c vm_machdep.c
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #1316):
sys/arch/x86/x86/pmap.c: revision 1.223
sys/arch/x86/x86/vm_machdep.c: revision 1.26
sys/arch/x86/include/pmap.h: revision 1.61
PR/49691: KAMADA Ken'ichi: free deferred ptp mappings if present.
XXX: pullup-7
To generate a diff of this commit:
cvs rdiff -u -r1.55.4.1 -r1.55.4.1.2.1 src/sys/arch/x86/include/pmap.h
cvs rdiff -u -r1.183.2.2.2.1 -r1.183.2.2.2.2 src/sys/arch/x86/x86/pmap.c
cvs rdiff -u -r1.25 -r1.25.8.1 src/sys/arch/x86/x86/vm_machdep.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 18 Dec 2016 21:14:23 +0000
State-Changed-Why:
pullups completed, and I guess this is fixed.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home