NetBSD Problem Report #53863

From www@NetBSD.org  Sun Jan 13 03:14:25 2019
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 10AB07A181
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 13 Jan 2019 03:14:25 +0000 (UTC)
Message-Id: <20190113031423.910B67A222@mollari.NetBSD.org>
Date: Sun, 13 Jan 2019 03:14:23 +0000 (UTC)
From: marcotte@panix.com
Reply-To: marcotte@panix.com
To: gnats-bugs@NetBSD.org
Subject: panic: xen_failsafe_handler called! while running 32-bit binary on 64-bit.
X-Send-Pr-Version: www-1.0

>Number:         53863
>Category:       port-xen
>Synopsis:       panic: xen_failsafe_handler called! while running 32-bit binary on 64-bit.
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-xen-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Jan 13 03:15:00 +0000 2019
>Last-Modified:  Mon Jan 14 04:20:00 +0000 2019
>Originator:     Brian Marcotte
>Release:        8.0
>Organization:
Public Access Networks, Corp.
>Environment:
NetBSD dw10.panix.com 8.0 NetBSD 8.0 (PANIX-XEN-WEB) #5: Sat Jan 12 20:28:06 EST 2019  root@juggler.panix.com:/misc/obj/misc/devel/netbsd/8.0/src/sys/arch/amd64/compile/PANIX-XEN-WEB amd64

>Description:
NetBSD/xen 64-bit domU.

When running an old 32-bit perl binary which loads modules, I sometimes get this:

  panic: xen_failsafe_handler called!

  cpu0: Begin traceback...
  vpanic() at netbsd:vpanic+0x140
  snprintf() at netbsd:snprintf
  xpq_flush_queue() at netbsd:xpq_flush_queue
  failsafe_callback() at netbsd:failsafe_callback+0xa1
  x86_64_tls_switch() at netbsd:x86_64_tls_switch+0x8a
  cpu_switchto() at netbsd:cpu_switchto+0x88
  preempt() at netbsd:preempt+0x4d
  trap() at netbsd:trap+0x747
  --- trap (number 3) ---
  7f7fd860c36a:
  cpu0: End traceback...

The binary is an old 32-bit perl which when loading certain modules, changes the text section to read/write. By default this should fail in NetBSD-8 because of mprotect, but it shoudn't panic the kernel. Turning off mprotect allows the program to run, but it sometimes still panics the kernel on startup.


>How-To-Repeat:
I wasn't able to make a simple program to cause the panic, but I can reproduce it pretty reliably here.

>Fix:
Don't know what the fix is, but I tracked it down to this change:

  src/sys/arch/amd64/amd64/machdep.c
  1.255.6.1 -> 1.255.6.2

That's a pullup of version 1.267.

When I revert that, I no longer get the panic.

>Audit-Trail:
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: port-xen-maintainer@netbsd.org, gnats-admin@netbsd.org,
        netbsd-bugs@netbsd.org
Subject: Re: port-xen/53863: panic: xen_failsafe_handler called! while
 running 32-bit binary on 64-bit.
Date: Sun, 13 Jan 2019 11:01:52 +0100

 On Sun, Jan 13, 2019 at 03:15:00AM +0000, marcotte@panix.com wrote:
 > NetBSD/xen 64-bit domU.
 > 
 > When running an old 32-bit perl binary which loads modules, I sometimes get this:
 > 
 >   panic: xen_failsafe_handler called!
 > 
 >   cpu0: Begin traceback...
 >   vpanic() at netbsd:vpanic+0x140
 >   snprintf() at netbsd:snprintf
 >   xpq_flush_queue() at netbsd:xpq_flush_queue
 >   failsafe_callback() at netbsd:failsafe_callback+0xa1
 >   x86_64_tls_switch() at netbsd:x86_64_tls_switch+0x8a
 >   cpu_switchto() at netbsd:cpu_switchto+0x88
 >   preempt() at netbsd:preempt+0x4d
 >   trap() at netbsd:trap+0x747
 >   --- trap (number 3) ---
 >   7f7fd860c36a:
 >   cpu0: End traceback...
 > 
 > The binary is an old 32-bit perl which when loading certain modules, changes the text section to read/write. By default this should fail in NetBSD-8 because of mprotect, but it shoudn't panic the kernel. Turning off mprotect allows the program to run, but it sometimes still panics the kernel on startup.
 > 
 > 
 > >How-To-Repeat:
 > I wasn't able to make a simple program to cause the panic, but I can reproduce it pretty reliably here.
 > 
 > >Fix:
 > Don't know what the fix is, but I tracked it down to this change:
 > 
 >   src/sys/arch/amd64/amd64/machdep.c
 >   1.255.6.1 -> 1.255.6.2
 > 
 > That's a pullup of version 1.267.
 > 
 > When I revert that, I no longer get the panic.

 Do you have anything relevant in xl dmesg on the dom0 ? booting with
 a xen-debug.gz kernel may be needed to get usefull information

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: Brian Marcotte <marcotte@panix.com>
To: Manuel Bouyer <bouyer@antioche.eu.org>
Cc: port-xen-maintainer@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org, gnats-bugs@NetBSD.org,
	Brian Marcotte <marcotte@panix.com>
Subject: Re: port-xen/53863: panic: xen_failsafe_handler called! while
 running 32-bit binary on 64-bit.
Date: Sun, 13 Jan 2019 23:15:23 -0500

 >  Do you have anything relevant in xl dmesg on the dom0 ? booting with
 >  a xen-debug.gz kernel may be needed to get usefull information

 Is xen-debug.gz still a thing? I don't see any reference to it in modern
 documentation. I've compiled with DEBUG and turned the log options to
 their max.

 This is Xen 4.10.3-pre (git around November 8) and dom0 is Linux 4.16.16.
 I had also seen the problem with slightly earlier versions of each.

 At domain start I get these:

 (XEN) grant_table.c:1770:d0v14 Expanding d1 grant table from 0 to 1 frames
 (XEN) d1 attempted to change d1v1's CR4 flags 00002660 -> 00000620
 (XEN) d1 L1TF-vulnerable L4e 0000000623ff8000 - Shadowing

 I now get this at the time of the NetBSD panic:

 (XEN) grant_table.c:1770:d1v0 Expanding d1 grant table from 1 to 2 frames
 (XEN) traps.c:1543: GPF (0088): ffff82d0802784ce [context_switch+0x91b/0xf89] -> ffff82d08037435b

 --
 - Brian

Home	PR Database Search