NetBSD Problem Report #51251
From hannken@eis.cs.tu-bs.de Thu Jun 16 16:18:07 2016
Return-Path: <hannken@eis.cs.tu-bs.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id B25817A110
for <gnats-bugs@gnats.NetBSD.org>; Thu, 16 Jun 2016 16:18:07 +0000 (UTC)
Message-Id: <20160616161343.D230174F64@builder.isf.cs.tu-bs.de>
Date: Thu, 16 Jun 2016 18:13:43 +0200 (MEST)
From: hannken@eis.cs.tu-bs.de
Reply-To: hannken@eis.cs.tu-bs.de
To: gnats-bugs@NetBSD.org
Subject: ACPI device probe raises NMI
X-Send-Pr-Version: 3.95
>Number: 51251
>Category: kern
>Synopsis: ACPI device probe raises NMI
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: hannken
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Thu Jun 16 16:20:00 +0000 2016
>Closed-Date: Tue Jul 12 09:47:39 +0000 2016
>Last-Modified: Tue Jul 12 09:47:39 +0000 2016
>Originator: Juergen Hannken-Illjes
>Release: NetBSD 7.99.30
>Organization:
>Environment:
System: NetBSD toaster.dd 7.99.30 NetBSD 7.99.30 (work.amd64) #2: Thu Jun 16 17:38:28 MEST 2016 hannken@builder.isf.cs.tu-bs.de:/work/build/obj/obj.amd64/sys/arch/amd64/compile/work.amd64 amd64
Architecture: x86_64
Machine: amd64
>Description:
Booting an IBM eserver xSeries 336 -[883721Y]- I get a NMI during PCI bus probe:
total memory = 16383 MB
avail memory = 15871 MB
SMBIOS rev. 2.3 @ 0xf5f9e (52 entries)
mainbus0 (root)
efi: missing or invalid systbl
ACPI: RSDP 0x00000000000FDFB0 000024 (v02 IBM )
ACPI: XSDT 0x00000000D7FCFF00 00003C (v01 IBM SERONYXP 00001001 IBM 45444F43)
ACPI: FACP 0x00000000D7FCFE40 000084 (v02 IBM SERONYXP 00001001 IBM 45444F43)
ACPI: DSDT 0x00000000D7FCCF00 002346 (v02 IBM SERTURQU 00001000 INTL 20041203)
ACPI: FACS 0x00000000D7FCFD00 000040
ACPI: APIC 0x00000000D7FCFD80 00009C (v01 IBM SERONYXP 00001001 IBM 45444F43)
ACPI: MCFG 0x00000000D7FCFD40 00003C (v01 IBM SERONYXP 00001001 IBM 45444F43)
ACPI: 1 ACPI AML tables successfully acquired and loaded
efi: missing or invalid systbl
ioapic0 at mainbus0 apid 14
ioapic1 at mainbus0 apid 13
ioapic2 at mainbus0 apid 12
cpu0 at mainbus0 apid 0
cpu0: 32 page colors
cpu0: calibrating local timer
cpu0: apic clock running at 200 MHz
cpu0: Intel(R) Xeon(TM) CPU 3.20GHz, id 0xf41
cpu0: PAT enabled
cpu1 at mainbus0 apid 6
cpu1: 2 page colors
x86_ipi_init: ESR 00000040
cpu1: Intel(R) Xeon(TM) CPU 3.20GHz, id 0xf41
cpu1: PAT enabled
cpu2 at mainbus0 apid 1
cpu2: 2 page colors
cpu2: Intel(R) Xeon(TM) CPU 3.20GHz, id 0xf41
cpu2: PAT enabled
cpu3 at mainbus0 apid 7
cpu3: 2 page colors
cpu3: Intel(R) Xeon(TM) CPU 3.20GHz, id 0xf41
cpu3: PAT enabled
acpi0 at mainbus0: Intel ACPICA 20160422
efi: missing or invalid systbl
acpi0: MCFG: segment 0, bus 0-7, address 0x00000000e0000000
MCFG: MEMMAP: 0x0000000000000000-0x000000000009d3ff, size=0x000000000009d400, type=1(Memory)
MCFG: MEMMAP: 0x000000000009d400-0x000000000009ffff, size=0x0000000000002c00, type=2(Reserved)
MCFG: MEMMAP: 0x00000000000e0000-0x00000000000fffff, size=0x0000000000020000, type=2(Reserved)
MCFG: MEMMAP: 0x0000000000100000-0x00000000d7fcceff, size=0x00000000d7eccf00, type=1(Memory)
MCFG: MEMMAP: 0x00000000d7fd0000-0x00000000d7ffffff, size=0x0000000000030000, type=2(Reserved)
MCFG: MEMMAP: 0x00000000d7fccf00-0x00000000d7fcffff, size=0x0000000000003100, type=3(ACPI)
MCFG: MEMMAP: 0x0000000100000000-0x0000000427ffffff, size=0x0000000328000000, type=1(Memory)
MCFG: MEMMAP: 0x00000000e0000000-0x00000000efffffff, size=0x0000000010000000, type=2(Reserved)
acpi0: fixed power button present
acpi0: ACPI-Fast 24-bit timer
SIOM (PNP0C02) at acpi0 not configured
pckbc1 at acpi0 (PS2K, PNP0303) (kbd port): io 0x64,0x60 irq 1
pckbc2 at acpi0 (PS2M, PNP0F13) (aux port): irq 12
COM2 (PNP0501) at acpi0 not configured
PIC2 (PNP0003) at acpi0 not configured
PIC3 (PNP0003) at acpi0 not configured
PIC4 (PNP0003) at acpi0 not configured
attimer1 at acpi0 (TMR, PNP0100): io 0x40-0x43 irq 0
pcppi1 at acpi0 (SPKR, PNP0800): io 0x61
midi0 at pcppi1: PC speaker
sysbeep0 at pcppi1
SBD1 (PNP0C02) at acpi0 not configured
pckbdprobe: reset error 5
pmsprobe: reset error 5
attimer1: attached to pcppi1
ipmi0 at mainbus0
ipmi0: WARNING: power management not supported
pci0 at mainbus0 bus 0: configuration mode 1
[ probe 000:00:0 ]
[ ffff80011e331000 + 100 ]
[ ffff80011e331000 + 100 ]
acpi0: MCFG: 000:00:0: Ok (cfg[0x100]=0x00000000)
[ probe 000:00:1 ]
[ ffff80011e332000 + 100 ]
[ ffff80011e332000 + 100 ]
acpi0: MCFG: 000:00:1: Ok (cfg[0x100]=0x00000000)
[ probe 000:02:0 ]
[ ffff80011e341000 + 100 ]
[ ffff80011e341000 + 100 ]
acpi0: MCFG: 000:02:0: Ok (cfg[0x100]=0x00010001)
[ probe 000:04:0 ]
[ ffff80011e351000 + 100 ]
[ ffff80011e351000 + 100 ]
acpi0: MCFG: 000:04:0: Ok (cfg[0x100]=0x00010001)
[ probe 000:06:0 ]
[ ffff80011e361000 + 100 ]
[ ffff80011e361000 + 100 ]
acpi0: MCFG: 000:06:0: Ok (cfg[0x100]=0x00010001)
[ probe 000:07:0 ]
[ ffff80011e369000 + 100 ]
[ ffff80011e369000 + 100 ]
acpi0: MCFG: 000:07:0: Ok (cfg[0x100]=0x00010001)
[ probe 000:08:0 ]
[ ffff80011e371000 + 100 ]
[ ffff80011e371000 + 100 ]
acpi0: MCFG: 000:08:0: Ok (cfg[0x100]=0x00000000)
[ probe 000:29:0 ]
[ ffff80011e419000 + 100 ]
NMI ... going to debugger
The lines reading "[ probe ..." come from a printf in acpimcfg_device_probe()
before it probes extended configuration space.
The lines reading "[ fff..." come from a printf in acpi_md_mcfg_read()
before it does "__asm(...", they print "bsh" and "addr".
With "pcictl pci0 list" I get:
000:00:0: Intel E7525 Memory Controller Hub (host bridge, revision 0x0c)
000:00:1: Intel E7525 Error Reporting Device (undefined, subclass 0x00, revision 0x0c)
000:02:0: Intel E7525 PCI Express Port A (PCI bridge, revision 0x0c)
000:04:0: Intel E7525 PCI Express Port B (PCI bridge, revision 0x0c)
000:06:0: Intel E7520 PCI Express Port C (PCI bridge, revision 0x0c)
000:07:0: Intel E7520 PCI Express Port C1 (PCI bridge, revision 0x0c)
000:08:0: Intel E7520 Extended Configuration (miscellaneous system, revision 0x0c)
000:29:0: Intel 82801EB/ER USB UHCI Controller (USB serial bus, UHCI, revision 0x02)
...
>How-To-Repeat:
Boot a -current kernel on an IBM x336.
>Fix:
>Release-Note:
>Audit-Trail:
From: "J. Hannken-Illjes" <hannken@eis.cs.tu-bs.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Sat, 2 Jul 2016 17:46:23 +0200
Some further info:
The machine has a TCO device "tco0 at ichlpcib0" and it boots with
LPCIB_SMI_EN enabled. There is no BIOS option to disable this bit.
tco0 will unset this bit but it is much too late. With the attached
diff (an ugly proof-of-concept) the machine boots multiuser.
--
J. Hannken-Illjes - hannken@eis.cs.tu-bs.de - TU Braunschweig (Germany)
Index: acpi_mcfg.c
===================================================================
RCS file: /cvsroot/src/sys/dev/acpi/acpi_mcfg.c,v
retrieving revision 1.2
diff -p -u -2 -r1.2 acpi_mcfg.c
--- acpi_mcfg.c 11 Oct 2015 21:49:22 -0000 1.2
+++ acpi_mcfg.c 22 Jun 2016 10:31:33 -0000
@@ -42,4 +42,6 @@ __KERNEL_RCSID(0, "$NetBSD: acpi_mcfg.c,
#include <dev/acpi/acpi_mcfg.h>
+#include <dev/ic/i82801lpcreg.h>
+
#include "locators.h"
@@ -565,4 +567,26 @@ acpimcfg_map_bus(device_t self, pci_chip
aprint_debug("\n");
+ for (i = 0; i < 32; i++) {
+ pcitag_t tag;
+ pcireg_t reg, ioreg;
+ bus_space_handle_t ioh;
+
+ tag = pci_make_tag(pc, bus, i, 0);
+ reg = pci_conf_read(pc, tag, 0);
+ if (reg != PCI_ID_CODE(PCI_VENDOR_INTEL,
+ PCI_PRODUCT_INTEL_82801EB_LPC))
+ continue;
+
+ reg = pci_conf_read(pc, tag, LPCIB_PCI_PMBASE);
+ if (bus_space_map(x86_bus_space_io, PCI_MAPREG_IO_ADDR(reg),
+ LPCIB_PCI_PM_SIZE, 0, &ioh) != 0)
+ continue;
+
+ ioreg = bus_space_read_4(x86_bus_space_io, ioh, LPCIB_SMI_EN);
+ ioreg &= ~LPCIB_SMI_EN_TCO_EN;
+ bus_space_write_4(x86_bus_space_io, ioh, LPCIB_SMI_EN, ioreg);
+ bus_space_unmap(x86_bus_space_io, ioh, LPCIB_PCI_PM_SIZE);
+ }
+
/* Probe extended configuration space of all devices. */
memset(mb->valid_devs, 0xff, sizeof(mb->valid_devs));
From: David Young <dyoung@pobox.com>
To: gnats-bugs@NetBSD.org, hannken@eis.cs.tu-bs.de
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Sat, 2 Jul 2016 19:05:21 -0500
On Thu, Jun 16, 2016 at 04:20:00PM +0000, hannken@eis.cs.tu-bs.de wrote:
> pci0 at mainbus0 bus 0: configuration mode 1
> [ probe 000:00:0 ]
... snip snip ...
ISTR there is a bit you can clear in the bus bridge to stop SERR from
being forwarded upstream? PCI_COMMAND_SERR_ENABLE, is it?
It would be interesting to see the PCI Status Register on pci0 before
and after the probe of 000:29:0. Likewise, I'm curious what the PCI
Command & Status Registers on 000:29:0 contain.
Dave
--
David Young //\ Trestle Technology Consulting
(217) 721-9981 Urbana, IL http://trestle.tech/
From: "J. Hannken-Illjes" <hannken@eis.cs.tu-bs.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Sun, 3 Jul 2016 15:03:25 +0200
On Sun, Jul 03, 2016 at 12:10:01AM +0000, David Young wrote:
[snip]
> ISTR there is a bit you can clear in the bus bridge to stop SERR from
> being forwarded upstream? PCI_COMMAND_SERR_ENABLE, is it?
>
> It would be interesting to see the PCI Status Register on pci0 before
> and after the probe of 000:29:0. Likewise, I'm curious what the PCI
> Command & Status Registers on 000:29:0 contain.
With my ugly patch and this
if (bus == 0 && dev == 29 && func == 0) {
pcitag_t rtag;
pcireg_t rval;
rtag = pci_make_tag(pc, bus, 0, 0);
rval = pci_conf_read(pc, rtag, PCI_COMMAND_STATUS_REG);
printf("[ - 0:0:0 %#8x", rval);
rval = pci_conf_read(pc, tag, PCI_COMMAND_STATUS_REG);
printf(" 0:29:0 %#8x ]\n", rval);
}
before and after "Probe extended configuration space." I get:
[ - 0:0:0 0x00900146 0:29:0 0x02800005 ]
acpi0: MCFG: 000:29:0: invalid config space (cfg[0x100]=0x24d28086, alias=true)
[ + 0:0:0 0x00900146 0:29:0 0x02800005 ]
--
J. Hannken-Illjes - hannken@eis.cs.tu-bs.de - TU Braunschweig (Germany)
From: David Young <dyoung@pobox.com>
To: gnats-bugs@NetBSD.org, hannken@eis.cs.tu-bs.de
Cc:
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Sun, 3 Jul 2016 13:45:01 -0500
On Sun, Jul 03, 2016 at 01:05:01PM +0000, J. Hannken-Illjes wrote:
> The following reply was made to PR kern/51251; it has been noted by GNATS.
>
> From: "J. Hannken-Illjes" <hannken@eis.cs.tu-bs.de>
> To: gnats-bugs@NetBSD.org
> Cc:
> Subject: Re: kern/51251: ACPI device probe raises NMI
> Date: Sun, 3 Jul 2016 15:03:25 +0200
>
> On Sun, Jul 03, 2016 at 12:10:01AM +0000, David Young wrote:
> [snip]
> > ISTR there is a bit you can clear in the bus bridge to stop SERR from
> > being forwarded upstream? PCI_COMMAND_SERR_ENABLE, is it?
> >
> > It would be interesting to see the PCI Status Register on pci0 before
> > and after the probe of 000:29:0. Likewise, I'm curious what the PCI
> > Command & Status Registers on 000:29:0 contain.
>
> With my ugly patch and this
>
> if (bus == 0 && dev == 29 && func == 0) {
> pcitag_t rtag;
> pcireg_t rval;
>
> rtag = pci_make_tag(pc, bus, 0, 0);
> rval = pci_conf_read(pc, rtag, PCI_COMMAND_STATUS_REG);
> printf("[ - 0:0:0 %#8x", rval);
> rval = pci_conf_read(pc, tag, PCI_COMMAND_STATUS_REG);
> printf(" 0:29:0 %#8x ]\n", rval);
> }
>
> before and after "Probe extended configuration space." I get:
>
> [ - 0:0:0 0x00900146 0:29:0 0x02800005 ]
> acpi0: MCFG: 000:29:0: invalid config space (cfg[0x100]=0x24d28086, alias=true)
> [ + 0:0:0 0x00900146 0:29:0 0x02800005 ]
IIUC, extended configuration space is memory-mapped? Looks like
memory-mapped I/O is not enabled on 0:29:0. Perhaps that explains
the NMI. Does ACPI expect for NetBSD to set the enables? If you set
PCI_COMMAND_MEM_ENABLE on the CSR for 0:29:0 before ACPI probes it,
maybe that will also prevent the NMI?
Dave
--
David Young //\ Trestle Technology Consulting
(217) 721-9981 Urbana, IL http://trestle.tech/
From: "J. Hannken-Illjes" <hannken@eis.cs.tu-bs.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Mon, 4 Jul 2016 10:17:11 +0200
On Sun, Jul 03, 2016 at 06:50:01PM +0000, David Young wrote:
[snip]
> IIUC, extended configuration space is memory-mapped? Looks like
> memory-mapped I/O is not enabled on 0:29:0. Perhaps that explains
> the NMI. Does ACPI expect for NetBSD to set the enables? If you set
> PCI_COMMAND_MEM_ENABLE on the CSR for 0:29:0 before ACPI probes it,
> maybe that will also prevent the NMI?
With this change
--- acpi_mcfg.c 11 Oct 2015 21:49:22 -0000 1.2
+++ acpi_mcfg.c 4 Jul 2016 08:14:03 -0000
@@ -489,4 +489,12 @@ acpimcfg_device_probe(const struct pci_a
mb->last_probed = tag;
+reg = pci_conf_read(pc, tag, PCI_COMMAND_STATUS_REG);
+printf("[ %03d:%02d:%d: %08x", bus, dev, func, reg);
+if ((reg & PCI_COMMAND_MEM_ENABLE) == 0) {
+ reg |= PCI_COMMAND_MEM_ENABLE;
+ printf(" -> %08x", reg);
+ pci_conf_write(pc, tag, PCI_COMMAND_STATUS_REG, reg);
+}
+printf(" ]\n");
/* Probe extended configuration space. */
if (((reg = pci_conf_read(pc, tag, PCI_CONF_SIZE)) == (pcireg_t)-1) ||
I get
pci0 at mainbus0 bus 0: configuration mode 1
[ 000:00:0: 00900146 ]
acpi0: MCFG: 000:00:0: Ok (cfg[0x100]=0x00000000)
[ 000:00:1: 00000100 -> 00000102 ]
acpi0: MCFG: 000:00:1: Ok (cfg[0x100]=0x00000000)
[ 000:02:0: 00100147 ]
acpi0: MCFG: 000:02:0: Ok (cfg[0x100]=0x00010001)
[ 000:04:0: 00100147 ]
acpi0: MCFG: 000:04:0: Ok (cfg[0x100]=0x00010001)
[ 000:06:0: 00100147 ]
acpi0: MCFG: 000:06:0: Ok (cfg[0x100]=0x00010001)
[ 000:07:0: 00100147 ]
acpi0: MCFG: 000:07:0: Ok (cfg[0x100]=0x00010001)
[ 000:08:0: 00800000 -> 00800002 ]
acpi0: MCFG: 000:08:0: Ok (cfg[0x100]=0x00000000)
[ 000:29:0: 02800005 -> 02800007 ]
NMI ... going to debugger
--
J. Hannken-Illjes - hannken@eis.cs.tu-bs.de - TU Braunschweig (Germany)
From: "J. Hannken-Illjes" <hannken@eis.cs.tu-bs.de>
To: gnats-bugs@NetBSD.org
Cc: David Young <dyoung@pobox.com>
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Sun, 10 Jul 2016 16:30:22 +0200
The following diff fixes the problem for me:
Index: acpi_mcfg.c
===================================================================
RCS file: /cvsroot/src/sys/dev/acpi/acpi_mcfg.c,v
retrieving revision 1.2
diff -p -u -2 -r1.2 acpi_mcfg.c
--- acpi_mcfg.c 11 Oct 2015 21:49:22 -0000 1.2
+++ acpi_mcfg.c 10 Jul 2016 14:26:33 -0000
@@ -491,5 +491,5 @@ acpimcfg_device_probe(const struct pci_a
/* Probe extended configuration space. */
if (((reg = pci_conf_read(pc, tag, PCI_CONF_SIZE)) == (pcireg_t)-1) ||
- (alias = acpimcfg_ext_conf_is_aliased(pc, tag))) {
+ (reg == 0) || (alias = acpimcfg_ext_conf_is_aliased(pc, tag))) {
aprint_debug_dev(acpi_sc->sc_dev,
"MCFG: %03d:%02d:%d: invalid config space "
@@ -527,4 +527,7 @@ acpimcfg_map_bus(device_t self, pci_chip
bus_space_handle_t bsh;
bus_addr_t baddr;
+ pcitag_t tag;
+ pcireg_t reg;
+ bool is_e7520_mch;
int boff;
int last_dev, last_func;
@@ -570,6 +573,27 @@ acpimcfg_map_bus(device_t self, pci_chip
mb->last_probed = pci_make_tag(pc, bus, 0, 0);
+ /*
+ * On an Intel E7520 we have to temporarily disable
+ * Enhanced Config Access error detection and reporting
+ * by setting the appropriate error mask in HI_ERRMASK register.
+ *
+ * See "Intel E7520 Memory Controller Hub (MCH) Datasheet",
+ * Document 303006-002, pg. 82
+ */
+ tag = pci_make_tag(pc, 0, 0, 1);
+ reg = pci_conf_read(pc, tag, PCI_ID_REG);
+ is_e7520_mch = (reg ==
+ PCI_ID_CODE(PCI_VENDOR_INTEL, PCI_PRODUCT_INTEL_E7525_MCHER));
+ if (is_e7520_mch) {
+ reg = pci_conf_read(pc, tag, 0x54);
+ pci_conf_write(pc, tag, 0x54, reg | 0x20);
+ }
+
acpimcfg_scan_bus(sc, pc, bus);
+ if (is_e7520_mch) {
+ pci_conf_write(pc, tag, 0x54, reg);
+ }
+
/* Unmap extended configration space of all dev/func. */
bus_space_unmap(seg->ms_bst, bsh, ACPIMCFG_SIZE_PER_BUS);
Does it qualify for commit?
--
J. Hannken-Illjes - hannken@eis.cs.tu-bs.de - TU Braunschweig (Germany)
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, hannken@eis.cs.tu-bs.de
Cc:
Subject: Re: kern/51251: ACPI device probe raises NMI
Date: Sun, 10 Jul 2016 10:37:15 -0400
On Jul 10, 2:35pm, hannken@eis.cs.tu-bs.de ("J. Hannken-Illjes") wrote:
-- Subject: Re: kern/51251: ACPI device probe raises NMI
| Does it qualify for commit?
I think it does.
christos
From: "Juergen Hannken-Illjes" <hannken@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/51251 CVS commit: src/sys/dev/acpi
Date: Tue, 12 Jul 2016 09:29:32 +0000
Module Name: src
Committed By: hannken
Date: Tue Jul 12 09:29:32 UTC 2016
Modified Files:
src/sys/dev/acpi: acpi_mcfg.c
Log Message:
On an Intel E7520 temporarily disable Enhanced Config Access error
detection and reporting during bus scan.
Fixes PR kern/51251: ACPI device probe raises NMI
To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.3 src/sys/dev/acpi/acpi_mcfg.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Responsible-Changed-From-To: kern-bug-people->hannken
Responsible-Changed-By: hannken@NetBSD.org
Responsible-Changed-When: Tue, 12 Jul 2016 09:47:39 +0000
Responsible-Changed-Why:
Take.
State-Changed-From-To: open->closed
State-Changed-By: hannken@NetBSD.org
State-Changed-When: Tue, 12 Jul 2016 09:47:39 +0000
State-Changed-Why:
Committed a fix.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home