NetBSD Problem Report #46763

From mw@fenrir.wzff.de  Wed Aug  1 22:44:07 2012
Return-Path: <mw@fenrir.wzff.de>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 1F97F63B85F
	for <gnats-bugs@gnats.NetBSD.org>; Wed,  1 Aug 2012 22:44:07 +0000 (UTC)
Message-Id: <20120801224400.9FA5BEEA54B@fenrir.wzff.de>
Date: Wed,  1 Aug 2012 22:44:00 +0000 (UTC)
From: Moritz Wilhelmy <mw@wzff.de>
Reply-To: Moritz Wilhelmy <mw@wzff.de>
To: gnats-bugs@gnats.NetBSD.org
Subject: Standby on IBM Thinkpad X41 does not work anymore
X-Send-Pr-Version: 3.95

>Number:         46763
>Category:       port-i386
>Synopsis:       Standby on IBM Thinkpad X41 does not work anymore
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-i386-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Aug 01 22:45:00 +0000 2012
>Last-Modified:  Fri Aug 03 02:50:02 +0000 2012
>Originator:     Moritz Wilhelmy
>Release:        NetBSD 6.0_BETA2
>Organization:
Fictional Enterprises Ltd
>Environment:
System: NetBSD fenrir.wzff.de 6.0_BETA2 NetBSD 6.0_BETA2 (GENERIC) i386
Architecture: i386
Machine: i386
>Description:
	Running zzz(8) on my Thinkpad X41 results in a 80x25 console being
	displayed with a white cursor at the top left position of the screen.
	Suspend (and even resume) used to work as intended via zzz in an
	earlier release of the 5 series, 5.1 if I remember correctly, but it
	might also have been 5.0.2 and broke after an upgrade to -current
	(which was then about 5.99.54). After reinstallation of 6.0_BETA2, the
	problem persists.

	I decided that I lack the skills to debug this myself within a
	reasonable amount of time and hereby file this PR.
>How-To-Repeat:
	Type zzz as root on my machine. Can anyone reproduce this?
>Fix:
	None known.

>Audit-Trail:
From: Jukka Ruohonen <jruohonen@iki.fi>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work anymore
Date: Thu, 2 Aug 2012 10:03:44 +0300

 On Wed, Aug 01, 2012 at 10:45:00PM +0000, Moritz Wilhelmy wrote:
 > 	Running zzz(8) on my Thinkpad X41 results in a 80x25 console being
 > 	displayed with a white cursor at the top left position of the screen.

 The dmesg(8) is a good start.

 - Jukka.

From: Moritz Wilhelmy <moritz@wzff.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work
 anymore
Date: Thu, 2 Aug 2012 11:39:13 +0200

 Hi Jukka,

 On Thu, Aug 02, 2012 at 07:05:04 +0000, Jukka Ruohonen wrote:
 > The dmesg(8) is a good start.

 Ah, right, I knew I forgot something.

 Best regards,

 Moritz

 -snip-
 Copyright (c) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005,
     2006, 2007, 2008, 2009, 2010, 2011, 2012
     The NetBSD Foundation, Inc.  All rights reserved.
 Copyright (c) 1982, 1986, 1989, 1991, 1993
     The Regents of the University of California.  All rights reserved.

 NetBSD 6.0_BETA2 (GENERIC)
 total memory = 2038 MB
 avail memory = 1991 MB
 timecounter: Timecounters tick every 10.000 msec
 timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
 IBM 2525C86 (ThinkPad X41)
 mainbus0 (root)
 ACPI Warning: 32/64X length mismatch in Gpe1Block: 0/32 (20110623/tbfadt-556)
 ACPI Warning: Optional field Gpe1Block has zero address or length: 0x000000000000102C/0x0 (20110623/tbfadt-586)
 cpu0 at mainbus0 apid 0: Intel(R) Pentium(R) M processor 1.60GHz, id 0x6d8
 ioapic0 at mainbus0 apid 1: pa 0xfec00000, version 20, 24 pins
 acpi0 at mainbus0: Intel ACPICA 20110623
 acpi0: X/RSDT: OemId <IBM   ,TP-74   ,00002020>, AslId < LTP,00000000>
 acpiecdt0 at acpi0: ACPI Embedded Controller via ECDT
 acpi0: SCI interrupting at int 9
 timecounter: Timecounter "ACPI-Safe" frequency 3579545 Hz quality 900
 acpiec0 at acpi0 (EC, PNP0C09-0): using acpiecdt0
 MEM (PNP0C01) at acpi0 not configured
 acpilid0 at acpi0 (LID, PNP0C0D): ACPI Lid Switch
 acpibut0 at acpi0 (SLPB, PNP0C0E): ACPI Sleep Button
 SIO (PNP0C02) at acpi0 not configured
 attimer1 at acpi0 (TIMR, PNP0100): io 0x40-0x43 irq 0
 pcppi1 at acpi0 (SPKR, PNP0800): io 0x61
 midi0 at pcppi1: PC speaker
 sysbeep0 at pcppi1
 npx1 at acpi0 (FPU, PNP0C04): io 0xf0 irq 13
 npx1: reported by CPUID; using exception 16
 pckbc1 at acpi0 (KBD, PNP0303) (kbd port): io 0x60,0x64 irq 1
 pckbc2 at acpi0 (MOU, IBM3780) (aux port): irq 12
 TPM (ATM1100) at acpi0 not configured
 acpibat0 at acpi0 (BAT0, PNP0C0A-0): ACPI Battery
 acpibat0: SANYO LION rechargeable battery
 acpibat0: granularity: low->warn 0.001 Wh, warn->full 0.001 Wh
 acpiacad0 at acpi0 (AC, ACPI0003-0): ACPI AC Adapter
 thinkpad0 at acpi0 (HKEY, IBM0068)
 acpivga0 at acpi0 (VID): ACPI Display Adapter
 acpiout0 at acpivga0 (LCD0, 0x0400): ACPI Display Output Device
 acpiout1 at acpivga0 (CRT0, 0x0100): ACPI Display Output Device
 acpivga0: connected output devices:
 acpivga0:   0x0100 (acpiout1): Ext. Monitor, head 0
 acpivga0:   0x0400 (acpiout0): Unknown Output Device, head 0
 acpitz0 at acpi0 (THM0): cpu0
 acpitz0: levels: critical 97.0 C, passive 92.5 C, passive cooling
 apm0 at acpi0: Power Management spec V1.2
 attimer1: attached to pcppi1
 pckbd0 at pckbc1 (kbd slot)
 pckbc1: using irq 1 for kbd slot
 wskbd0 at pckbd0: console keyboard
 pms0 at pckbc1 (aux slot)
 pckbc1: using irq 12 for aux slot
 wsmouse0 at pms0 mux 0
 pci0 at mainbus0 bus 0: configuration mode 1
 pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
 pchb0 at pci0 dev 0 function 0: vendor 0x8086 product 0x2590 (rev. 0x03)
 agp0 at pchb0: detected 7932k stolen memory
 agp0: aperture at 0xc0000000, size 0x10000000
 vga1 at pci0 dev 2 function 0: vendor 0x8086 product 0x2592 (rev. 0x03)
 wsdisplay0 at vga1 kbdmux 1: console (80x25, vt100 emulation), using wskbd0
 wsmux1: connecting to wsdisplay0
 i915drm0 at vga1: Intel i915GM
 i915drm0: AGP at 0xc0000000 256MB
 i915drm0: Initialized i915 1.6.0 20080730
 vendor 0x8086 product 0x2792 (miscellaneous display, revision 0x03) at pci0 dev 2 function 1 not configured
 ppb0 at pci0 dev 28 function 0: vendor 0x8086 product 0x2660 (rev. 0x03)
 ppb0: PCI Express 1.0 <Root Port of PCI-E Root Complex>
 pci1 at ppb0 bus 2
 pci1: i/o space, memory space enabled, rd/line, wr/inv ok
 bge0 at pci1 dev 0 function 0: Broadcom BCM5751M Gigabit Ethernet
 bge0: interrupting at ioapic0 pin 16
 bge0: adjust device control 0x102000 -> 0x105000
 bge0: ASIC BCM5750 B1 (0x4101), Ethernet address 00:0a:e4:3f:4d:ef
 bge0: setting short Tx thresholds
 brgphy0 at bge0 phy 1: BCM5750 1000BASE-T media interface, rev. 0
 brgphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseT, 1000baseT-FDX, auto
 uhci0 at pci0 dev 29 function 0: vendor 0x8086 product 0x2658 (rev. 0x03)
 uhci0: interrupting at ioapic0 pin 16
 usb0 at uhci0: USB revision 1.0
 uhci1 at pci0 dev 29 function 1: vendor 0x8086 product 0x2659 (rev. 0x03)
 uhci1: interrupting at ioapic0 pin 17
 usb1 at uhci1: USB revision 1.0
 uhci2 at pci0 dev 29 function 2: vendor 0x8086 product 0x265a (rev. 0x03)
 uhci2: interrupting at ioapic0 pin 18
 usb2 at uhci2: USB revision 1.0
 uhci3 at pci0 dev 29 function 3: vendor 0x8086 product 0x265b (rev. 0x03)
 uhci3: interrupting at ioapic0 pin 19
 usb3 at uhci3: USB revision 1.0
 ehci0 at pci0 dev 29 function 7: vendor 0x8086 product 0x265c (rev. 0x03)
 ehci0: interrupting at ioapic0 pin 19
 ehci0: EHCI version 1.0
 ehci0: companion controllers, 2 ports each: uhci0 uhci1 uhci2 uhci3
 usb4 at ehci0: USB revision 2.0
 ppb1 at pci0 dev 30 function 0: vendor 0x8086 product 0x2448 (rev. 0xd3)
 pci2 at ppb1 bus 4
 pci2: i/o space, memory space enabled
 cbb0 at pci2 dev 0 function 0: vendor 0x1180 product 0x0476 (rev. 0x8d)
 sdhc0 at pci2 dev 0 function 1: vendor 0x1180 product 0x0822 (rev. 0x13)
 sdhc0: interrupting at ioapic0 pin 17
 sdmmc0 at sdhc0
 ath0 at pci2 dev 2 function 0: Atheros 5212
 ath0: interrupting at ioapic0 pin 21
 ath0: 11a rates: 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
 ath0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
 ath0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
 ath0: mac 5.9 phy 4.3 radio 3.6
 cbb0: cacheline 0x0 lattimer 0x40
 cbb0: bhlc 0x824000
 cbb0: interrupting at ioapic0 pin 16
 cardslot0 at cbb0
 cardbus0 at cardslot0: bus 5
 pcmcia0 at cardslot0
 auich0 at pci0 dev 30 function 2: i82801FB (ICH6) AC-97 Audio
 auich0: interrupting at ioapic0 pin 22
 auich0: ac97: Analog Devices AD1981B codec; headphone, 20 bit DAC, no 3D stereo
 auich0: ac97: ext id 0x601<AC97_22,AMAP,VRA>
 vendor 0x8086 product 0x266d (modem communications, revision 0x03) at pci0 dev 30 function 3 not configured
 ichlpcib0 at pci0 dev 31 function 0: vendor 0x8086 product 0x2641 (rev. 0x03)
 timecounter: Timecounter "ichlpcib0" frequency 3579545 Hz quality 1000
 ichlpcib0: 24-bit timer
 ichlpcib0: TCO (watchdog) timer configured.
 gpio0 at ichlpcib0: 64 pins
 piixide0 at pci0 dev 31 function 2: Intel 82801FBM Serial ATA Controller (ICH6) (rev. 0x03)
 piixide0: bus-master DMA support present
 piixide0: primary channel wired to compatibility mode
 piixide0: primary channel interrupting at ioapic0 pin 14
 atabus0 at piixide0 channel 0
 piixide0: secondary channel wired to compatibility mode
 piixide0: secondary channel interrupting at ioapic0 pin 15
 atabus1 at piixide0 channel 1
 ichsmb0 at pci0 dev 31 function 3: vendor 0x8086 product 0x266a (rev. 0x03)
 ichsmb0: interrupting at ioapic0 pin 23
 iic0 at ichsmb0: I2C bus
 isa0 at ichlpcib0
 acpicpu0 at cpu0: ACPI CPU
 acpicpu0: C1: HLT, lat   1 us, pow  1000 mW
 acpicpu0: C2: I/O, lat   1 us, pow   500 mW
 acpicpu0: C3: I/O, lat  85 us, pow   250 mW
 acpicpu0: P0: FFH, lat 500 us, pow 10000 mW, 1600 MHz
 acpicpu0: P1: FFH, lat 500 us, pow  9700 mW, 1500 MHz
 acpicpu0: P2: FFH, lat 500 us, pow  9500 mW, 1400 MHz
 acpicpu0: P3: FFH, lat 500 us, pow  9200 mW, 1300 MHz
 acpicpu0: P4: FFH, lat 500 us, pow  9000 mW, 1200 MHz
 acpicpu0: P5: FFH, lat 500 us, pow  8700 mW, 1100 MHz
 acpicpu0: P6: FFH, lat 500 us, pow  8500 mW, 1000 MHz
 acpicpu0: P7: FFH, lat 500 us, pow  8200 mW,  900 MHz
 acpicpu0: P8: FFH, lat 500 us, pow  8000 mW,  800 MHz
 acpicpu0: P9: FFH, lat 500 us, pow  7500 mW,  600 MHz
 acpicpu0: T0: I/O, lat   1 us, pow     0 mW, 100 %
 acpicpu0: T1: I/O, lat   1 us, pow     0 mW,  88 %
 acpicpu0: T2: I/O, lat   1 us, pow     0 mW,  76 %
 acpicpu0: T3: I/O, lat   1 us, pow     0 mW,  64 %
 acpicpu0: T4: I/O, lat   1 us, pow     0 mW,  52 %
 acpicpu0: T5: I/O, lat   1 us, pow     0 mW,  40 %
 acpicpu0: T6: I/O, lat   1 us, pow     0 mW,  28 %
 acpicpu0: T7: I/O, lat   1 us, pow     0 mW,  16 %
 timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
 auich0: measured ac97 link rate at 48001 Hz, will use 48000 Hz
 audio0 at auich0: full duplex, playback, capture, mmap, independent
 acpiacad0: AC adapter online.
 uhub0 at usb0: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub0: 2 ports with 2 removable, self powered
 uhub1 at usb1: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub1: 2 ports with 2 removable, self powered
 uhub2 at usb2: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub2: 2 ports with 2 removable, self powered
 uhub3 at usb3: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
 uhub3: 2 ports with 2 removable, self powered
 uhub4 at usb4: vendor 0x8086 EHCI root hub, class 9/0, rev 2.00/1.00, addr 1
 uhub4: 8 ports with 8 removable, self powered
 ehci0: handing over low speed device on port 3 to uhci1
 ehci0: handing over full speed device on port 5 to uhci2
 wd0 at atabus0 drive 0
 wd0: <HTC426060G9AT00>
 wd0: drive supports 16-sector PIO transfers, LBA addressing
 wd0: 57231 MB, 116280 cyl, 16 head, 63 sec, 512 bytes/sect x 117210240 sectors
 uhidev0 at uhub1 port 1 configuration 1 interface 0
 uhidev0: Logitech USB-PS/2 Optical Mouse, rev 2.00/27.20, addr 2, iclass 3/1
 ums0 at uhidev0: 8 buttons, W and Z dirs
 wsmouse1 at ums0 mux 0
 wd0: 32-bit data port
 wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
 wd0(piixide0:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA)
 Kernelized RAIDframe activated
 boot device: wd0
 root on wd0a dumps on wd0b
 root file system type: ffs
 ubt0 at uhub2 port 1
 ubt0: Broadcom IBM Bluetooth Module, rev 1.10/0.09, addr 2
 wsdisplay0: screen 1 added (80x25, vt100 emulation)
 wsdisplay0: screen 2 added (80x25, vt100 emulation)
 wsdisplay0: screen 3 added (80x25, vt100 emulation)
 wsdisplay0: screen 4 added (80x25, vt100 emulation)
 ath0: device timeout (txq 1, txintrperiod 5)
 ath0: device timeout (txq 1, txintrperiod 4)
 i915drm0: interrupting at ioapic0 pin 16
 wsmouse1: detached
 ums0: detached
 uhidev0: detached
 uhidev0: at uhub1 port 1 (addr 2) disconnected
 ath0: device timeout (txq 1, txintrperiod 3)
 ath0: device timeout (txq 1, txintrperiod 2)

From: Jukka Ruohonen <jruohonen@iki.fi>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work anymore
Date: Thu, 2 Aug 2012 13:21:57 +0300

 On Wed, Aug 01, 2012 at 10:45:00PM +0000, Moritz Wilhelmy wrote:
 > 	Running zzz(8) on my Thinkpad X41 results in a 80x25 console being
 > 	displayed with a white cursor at the top left position of the screen.
 > 	Suspend (and even resume) used to work as intended via zzz in an
 > 	earlier release of the 5 series, 5.1 if I remember correctly

 Don't use zzz(8) but hw.acpi.sleep.state=3. Can you try suspend/resume from
 within X? Try different values of hw.acpi.sleep.vbios, as described in the
 acpi(4) manual page. Report back with/without X.org.

 - Jukka.

From: Moritz Wilhelmy <moritz@wzff.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work
 anymore
Date: Thu, 2 Aug 2012 13:00:59 +0200

 On Thu, Aug 02, 2012 at 10:25:02 +0000, Jukka Ruohonen wrote:
 > Don't use zzz(8) but hw.acpi.sleep.state=3.

 Same thing, I continued my experiments without starting X after that.

 >  Can you try suspend/resume from within X?

 It used to work (using zzz(8)), both from within X and without.

 > Try different values of hw.acpi.sleep.vbios, as described in the
 > acpi(4) manual page. Report back with/without X.org.

 Doesn't make a difference. Always the screen shows the 80x25 console.
 Without X, I get a few informational messages from ACPI, but nothing
 unusual. However, the Suspend-LED on the LED panel does glow up.

 Mind you, suspension doesn't really work. I'm not trying to get it to
 wake up again, it doesn't even complete the suspension. The screen
 doesn't blank and it doesn't react to keyboard or pressing the power
 button for <5s.

 Thanks for helping me out here. If you need additional information,
 please speak up.

 Best regards,

 Moritz

From: Jukka Ruohonen <jruohonen@iki.fi>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work anymore
Date: Thu, 2 Aug 2012 14:23:41 +0300

 On Thu, Aug 02, 2012 at 11:05:03AM +0000, Moritz Wilhelmy wrote:
 >  It used to work (using zzz(8)), both from within X and without.

 Yes but the zzz(8) is kind of legacy command (involves also more untested
 code).

 >  Mind you, suspension doesn't really work. I'm not trying to get it to
 >  wake up again, it doesn't even complete the suspension. The screen
 >  doesn't blank and it doesn't react to keyboard or pressing the power
 >  button for <5s.

 This sounds like a panic somewhere during the suspend sequence. If you don't
 have remote debugging capabilities, it could be hard to catch that though.
 I'd personally try to disable as many devices from userconf(4) and try
 again. If it works, it is a driver-problem.

 - Jukka.

From: Moritz Wilhelmy <moritz@wzff.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work
 anymore
Date: Fri, 3 Aug 2012 03:41:46 +0200

 On Thu, Aug 02, 2012 at 11:25:02 +0000, Jukka Ruohonen wrote:
 >  On Thu, Aug 02, 2012 at 11:05:03AM +0000, Moritz Wilhelmy wrote:
 >  >  It used to work (using zzz(8)), both from within X and without.
 >  
 >  Yes but the zzz(8) is kind of legacy command (involves also more untested
 >  code).

 I like zzz(8) because the name is so short and convenient. Given that
 apm is mostly deprecated, wouldn't it be possible to switch zzz over to
 write that sysctl value instead?

 >  >  Mind you, suspension doesn't really work. I'm not trying to get it to
 >  >  wake up again, it doesn't even complete the suspension. The screen
 >  >  doesn't blank and it doesn't react to keyboard or pressing the power
 >  >  button for <5s.
 >  
 >  This sounds like a panic somewhere during the suspend sequence. If you don't
 >  have remote debugging capabilities, it could be hard to catch that though.
 >  I'd personally try to disable as many devices from userconf(4) and try
 >  again. If it works, it is a driver-problem.

 Alright, I tried disabling whatever I could think of, in various
 combinations. So far, it didn't help. I see a message about 
 "Syncing disks" and a number.
 I tried disabling audio, video, bluetooth, sdhc/sdmmc, usb etc., but
 nothing so far helped. Why do I need remote debugging capabilities and
 how do I do that? Via a serial line? Maybe the machine has a serial
 connector on the inside...

 Any other ideas? If you were me, and looking at this dmesg, what would
 you disable first? Are any of the drivers in the dmesg known to be
 substandard quality or known to cause breakage, or have they recently
 (i.e. between 5.1 and 5.99.5x) undergone changes?

 I might be overlooking things.


 Best regards,

 Moritz

From: Jukka Ruohonen <jruohonen@iki.fi>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/46763: Standby on IBM Thinkpad X41 does not work anymore
Date: Fri, 3 Aug 2012 05:46:34 +0300

 On Fri, Aug 03, 2012 at 01:45:02AM +0000, Moritz Wilhelmy wrote:
 >  Alright, I tried disabling whatever I could think of, in various
 >  combinations. So far, it didn't help. I see a message about 
 >  "Syncing disks" and a number.
 >  I tried disabling audio, video, bluetooth, sdhc/sdmmc, usb etc., but
 >  nothing so far helped. Why do I need remote debugging capabilities and
 >  how do I do that? Via a serial line? Maybe the machine has a serial
 >  connector on the inside...

 A serial line would be perfect for this.

 >  Any other ideas? If you were me, and looking at this dmesg, what would
 >  you disable first? Are any of the drivers in the dmesg known to be
 >  substandard quality or known to cause breakage, or have they recently
 >  (i.e. between 5.1 and 5.99.5x) undergone changes?

 Not much (I asked about devices because stuff like "ubt0: Broadcom IBM
 Bluetooth Module, rev 1.10/0.09, addr"; otherwise seems to be rather normal,
 old Intel configuration).

 >  I might be overlooking things.

 Not really. Blunt as it may seem, these are bugs that usually can not be
 solved without machine/system/laptop/etc.

 Another regression, nevertheless.

 - Jukka.

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.