NetBSD Problem Report #9856

Received: (qmail 16647 invoked from network); 10 Apr 2000 21:31:44 -0000
Message-Id: <200004102131.RAA01006@snorklewacker.mit.edu>
Date: Mon, 10 Apr 2000 17:31:41 -0400 (EDT)
From: jhawk@MIT.EDU
Reply-To: jhawk@MIT.EDU
To: gnats-bugs@gnats.netbsd.org
Subject: wd driver loses seriouslly in face of bad blocks
X-Send-Pr-Version: 3.95

>Number:         9856
>Category:       kern
>Synopsis:       wd driver loses seriouslly in face of bad blocks
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    mycroft
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Apr 10 21:32:01 +0000 2000
>Closed-Date:    Wed Jul 07 00:37:37 +0000 2004
>Last-Modified:  Wed Jul 07 00:37:37 +0000 2004
>Originator:     John Hawkinson
>Release:        1.4.2
>Organization:

>Environment:

System: NetBSD 1.4.2 (ZORKMID) #63: Sat Apr  8 12:19:54 EDT 2000
    jhawk@zorkmid.mit.edu:/usr/src/sys/arch/i386/compile/ZORKMID

>Description:
	The wd driver misbehaves rather spectacular in the face of
bad blocks. There are a number of problems. 

1)	The drive returns atapi error number 1 on some reads.
According to the ATAv4 draft I was able to find, this indicates
"obsolete". sys/dev/ata.c's atapi_errno() pritns a nulls string
for this error, rather than anything useful. This results in:

   wd0e:   reading fsbn ...

which is confusing.  I've patched my kernel to report it as "(obsolete)"
so when you see that below, that is what is meant by it. I don't really know
what to conclude by this error. Perhaps my drive is really ATAv3
and happens to support some ATAv4 features like Ultra DMA? I don't know how
to debug this. Perhaps CFA REQUEST EXTENDED ERROR CODE could/should be used?

2)	The wd driver can take *forever* to timeout. Where forever == minutes.
Please see below (How-to-Repeat) for an example of a 1-block read 
that took 11 minutes to fail. Note that the kernel printfs regarding
disk errors did not take place until 9 minutes into it. Specfically,
9m6s and 9m14s for the (obsolete) errors, 9m23s for the
uncorrectable+downgrade, and 9m35s for the next (obsolete).

This is pretty hokey.

3)	The driver seems to spend a lot of time in each read, and then
downgrading the transfer mode, and then rereading the same blocks.
I don't have precise timings on this one, but I earlier had a case
where it went through ~30seconds each trying to read a single
block and getting 3 (obsolete) errors, then downgraded to Ultra-DMA1,
tried twice with (obsolete) errors, then saw:

pciide0:0:0: lost interrupt
        type: ata
        c_bcount: 37888
        c_skip: 19456
wd0e: device timeout reading fsbn 7455094 of 7455056-7455167 (wd0 bn 12899239; c
n 13649 tn 14 sn 52)
wd0e:  uncorrectable data error reading fsbn 7455056 of 7455056-7455167 (wd0 bn 
12899201; cn 13649 tn 14 sn 14), retrying
wd0e:  uncorrectable data error reading fsbn 7455056 of 7455056-7455167 (wd0 bn 
12899201; cn 13649 tn 14 sn 14), retrying

then downgraded to PIO mode 4.

It doens't seem like the downgrades were ever necessary or appropriate
(this is not a DMA problem, it is a physical problem, presumably), yet
they happened regardless and took a long time to effect themselves.


>How-To-Repeat:

Kernel probe:

NetBSD 1.4.2 (ZORKMID) #63: Sat Apr  8 12:19:54 EDT 2000
    jhawk@zorkmid.mit.edu:/usr/src/sys/arch/i386/compile/ZORKMID
cpu0: family 6 model 8 step 1
cpu0: Intel Pentium Pro, II or III (686-class)
real mem  = 66650112
avail mem = 57622528
using 839 buffers containing 3436544 bytes of memory
mainbus0 (root)
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o enabled, memory enabled
pchb0 at pci0 dev 0 function 0
pchb0: Intel 82443BX Host Bridge/Controller (rev. 0x03)
ppb0 at pci0 dev 1 function 0: Intel 82443BX AGP Interface (rev. 0x03)
pci1 at ppb0 bus 1
pci1: i/o enabled, memory enabled
vga1 at pci1 dev 0 function 0: Neomagic product 0x0005 (rev. 0x20)
wsdisplay0 at vga1: console (80x25, vt100 emulation)
pcib0 at pci0 dev 7 function 0
pcib0: Intel 82371AB PCI-to-ISA Bridge (PIIX4) (rev. 0x02)
pciide0 at pci0 dev 7 function 1: Intel 82371AB IDE controller (PIIX4)
pciide0: bus-master DMA support present
pciide0: primary channel wired to compatibility mode
wd0 at pciide0 channel 0 drive 0: <TOSHIBA MK8113MAT>
wd0: drive supports 16-sector pio transfers, lba addressing
wd0: 7815MB, 16938 cyl, 15 head, 63 sec, 512 bytes/sect x 16006410 sectors
wd0: 32-bits data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2
pciide0: primary channel interrupting at irq 14
pciide0: secondary channel wired to compatibility mode
pciide0: disabling secondary channel (no drives)
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 (using DMA data transfers)

Try reading from a previously-logged bad sector:

# date; time dd skip=7455056 count=1 if=/dev/rwd0e of=/dev/null; date
Mon Apr 10 16:48:33 EDT 2000
wd0e:  (obsolete) reading fsbn 7251341 (wd0 bn 12695486; cn 13434 tn 5 sn 41), r
etrying
wd0e:  (obsolete) reading fsbn 7251341 (wd0 bn 12695486; cn 13434 tn 5 sn 41), r
etrying
wd0e:  uncorrectable data error reading fsbn 7251341 (wd0 bn 12695486; cn 13434 
tn 5 sn 41), retrying
wd0: transfer error, downgrading to Ultra-DMA mode 1
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 1 (using DMA data transfers)
wd0e:  (obsolete) reading fsbn 7251341 (wd0 bn 12695486; cn 13434 tn 5 sn 41), r
etrying
wd0e:  (obsolete) reading fsbn 7251341 (wd0 bn 12695486; cn 13434 tn 5 sn 41), r
etrying
wd0e:  (obsolete) reading fsbn 7251341 (wd0 bn 12695486; cn 13434 tn 5 sn 41)
dd: /dev/rwd0e: Input/output error
      630.95 real         4.11 user            322.28 sys
Mon Apr 10 16:59:05 EDT 2000
>Fix:
	I have no idea, and it's very frustrating. There doesn't seem to be
any bad-block remapping or marking mechanism available.
>Release-Note:
>Audit-Trail:

From: Manuel Bouyer <bouyer@antioche.lip6.fr>
To: jhawk@MIT.EDU
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/9856: wd driver loses seriouslly in face of bad blocks
Date: Fri, 14 Apr 2000 01:31:14 +0200

 On Mon, Apr 10, 2000 at 05:31:41PM -0400, jhawk@MIT.EDU wrote:
 > 
 > 	The wd driver misbehaves rather spectacular in the face of
 > bad blocks. There are a number of problems. 
 > 
 > 1)	The drive returns atapi error number 1 on some reads.
 > According to the ATAv4 draft I was able to find, this indicates
 > "obsolete". sys/dev/ata.c's atapi_errno() pritns a nulls string
 > for this error, rather than anything useful. This results in:
 > 
 >    wd0e:   reading fsbn ...
 > 
 > which is confusing.  I've patched my kernel to report it as "(obsolete)"

 Ok, I changed it to : "obsolete (was address mark not found)", to make
 it clear.

 > so when you see that below, that is what is meant by it. I don't really know
 > what to conclude by this error. Perhaps my drive is really ATAv3
 > and happens to support some ATAv4 features like Ultra DMA? I don't know how

 This is the case of most drives; manufacturers usually adds new features and
 then try to include them in a standart. Ultra-DMA hardware was available
 before ATA-4 was out (and now the same is true for Ultra/66 :)
 The code guess that if a drive supports Ultra-DMA it supports ATA-4 specs.
 Maybe I should switch back to trust what the disks says, it won't be rigth
 in all cases anyway, as some drives supports only parts of ATA-4 :(

 > to debug this. Perhaps CFA REQUEST EXTENDED ERROR CODE could/should be used?

 I'll look at this; but later (when I'll add disonnect/reselect and tagged
 command queuing). The problem is that this command is optionnal, and
 I don't have a good way to deal with this yet.

 > 
 > 2)	The wd driver can take *forever* to timeout. Where forever == minutes.
 > Please see below (How-to-Repeat) for an example of a 1-block read 
 > that took 11 minutes to fail. Note that the kernel printfs regarding
 > disk errors did not take place until 9 minutes into it. Specfically,
 > 9m6s and 9m14s for the (obsolete) errors, 9m23s for the
 > uncorrectable+downgrade, and 9m35s for the next (obsolete).
 > 
 > This is pretty hokey.

 It should have timed out after 10s. Can you reproduce this reliably ?
 In the trace below there doens't appear to be a timeout.
 Actually I think I've found a bug in kern_clock.c that could explain
 this kind of behavior ...

 > 
 > 3)	The driver seems to spend a lot of time in each read, and then
 > downgrading the transfer mode, and then rereading the same blocks.
 > I don't have precise timings on this one, but I earlier had a case
 > where it went through ~30seconds each trying to read a single
 > block and getting 3 (obsolete) errors, then downgraded to Ultra-DMA1,
 > tried twice with (obsolete) errors, then saw:
 > 
 > pciide0:0:0: lost interrupt
 >         type: ata
 >         c_bcount: 37888
 >         c_skip: 19456
 > wd0e: device timeout reading fsbn 7455094 of 7455056-7455167 (wd0 bn 12899239; c
 > n 13649 tn 14 sn 52)
 > wd0e:  uncorrectable data error reading fsbn 7455056 of 7455056-7455167 (wd0 bn 
 > 12899201; cn 13649 tn 14 sn 14), retrying
 > wd0e:  uncorrectable data error reading fsbn 7455056 of 7455056-7455167 (wd0 bn 
 > 12899201; cn 13649 tn 14 sn 14), retrying
 > 
 > then downgraded to PIO mode 4.
 > 
 > It doens't seem like the downgrades were ever necessary or appropriate
 > (this is not a DMA problem, it is a physical problem, presumably), yet
 > they happened regardless and took a long time to effect themselves.

 A "lost interrupt" is always handled as a DMA error. I think the drive should
 have issued an IRQ anyway here. I'm not sure this is the rigth behavior but
 it's pretty easy to fix: please try this :

 --- ata_wdc.c.old	Fri Apr 14 01:27:50 2000
 +++ ata_wdc.c	Fri Apr 14 01:27:53 2000
 @@ -498,7 +498,9 @@
  		}
  		if (drv_err != WDC_ATA_ERR)
  			goto end;
 -		ata_dmaerr(drvp);
 +		if (drvp->drive_flags & DRIVE_UDMA &&
 +		    (ata_bio->r_error & WDCE_CRC))
 +			ata_dmaerr(drvp);
  	}

  	/* if we had an error, end */

 > >Fix:
 > 	I have no idea, and it's very frustrating. There doesn't seem to be
 > any bad-block remapping or marking mechanism available.

 No. Modern IDE disks should auto-remap bad blocks, just like SCSI.
 I guess the bad block table of your disk is full (this is where supporting
 SMART would be usefull :)

 --
 Manuel Bouyer <bouyer@antioche.eu.org>
 --

From: John Hawkinson <jhawk@MIT.EDU>
To: Manuel Bouyer <bouyer@antioche.lip6.fr>
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/9856: wd driver loses seriouslly in face of bad blocks 
Date: Thu, 13 Apr 2000 21:23:53 -0400

 In message <20000414013114.A398@antioche.eu.org>, Manuel Bouyer writes:
 >Ok, I changed it to : "obsolete (was address mark not found)", to make
 >it clear.

 Sounds good.

 >> so when you see that below, that is what is meant by it. I don't really know
 >> what to conclude by this error. Perhaps my drive is really ATAv3
 >> and happens to support some ATAv4 features like Ultra DMA? I don't know how
 >
 >This is the case of most drives; manufacturers usually adds new features and
 >then try to include them in a standart. Ultra-DMA hardware was available
 >before ATA-4 was out (and now the same is true for Ultra/66 :)
 >The code guess that if a drive supports Ultra-DMA it supports ATA-4 specs.
 >Maybe I should switch back to trust what the disks says, it won't be rigth
 >in all cases anyway, as some drives supports only parts of ATA-4 :(

 Well, perhaps it deserves a kernel printf() to observe when
 they disagree? I guess I can change the #if 0 code in
 wdc.c...

 >> to debug this. Perhaps CFA REQUEST EXTENDED ERROR CODE could/should be used?
 >
 >I'll look at this; but later (when I'll add disonnect/reselect and tagged
 >command queuing). The problem is that this command is optionnal, and
 >I don't have a good way to deal with this yet.

 Ah, OK. 

 >It should have timed out after 10s. Can you reproduce this reliably ?

 Yes, if I try to dd a single block from the drive in 
 a quiescenet state (e.g. boot -s; dd if=/dev/rwd0e skip=NNNNNN count=1 of=/dev/null)

 >In the trace below there doens't appear to be a timeout.
 >Actually I think I've found a bug in kern_clock.c that could explain
 >this kind of behavior ...

 Depends on which trace you look at ;-) See the one immediately
 following "Try reading from a previously-loged bad sector" under
 How-To-Repeat. No timeout is logged.


 >A "lost interrupt" is always handled as a DMA error. I think the drive should
 >have issued an IRQ anyway here. I'm not sure this is the rigth behavior but
 >it's pretty easy to fix: please try this :
 >
 >--- ata_wdc.c.old	Fri Apr 14 01:27:50 2000
 >+++ ata_wdc.c	Fri Apr 14 01:27:53 2000

 OK, I'll give it a shot.

 >> 	I have no idea, and it's very frustrating. There doesn't seem to be
 >> any bad-block remapping or marking mechanism available.
 >
 >No. Modern IDE disks should auto-remap bad blocks, just like SCSI.
 >I guess the bad block table of your disk is full (this is where supporting
 >SMART would be usefull :)

 I looked at the SMART frobs in the ATA4 spec and it wasn't clear to me
 that it gave you any useful data at all. Perhaps I was misunderstanding
 it, but there were lots of commands to enable/disable SMART but
 not anything that I saw that gave useful data on "how many error events"
 or any such.

 --jhawk


From: John Hawkinson <jhawk@MIT.EDU>
To: Manuel Bouyer <bouyer@antioche.lip6.fr>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/9856: timeout problem. 
Date: Thu, 27 Apr 2000 22:46:40 -0400

 Wow, sorry for the delay.

 I was out of town last weekend and had a nasty cold spending
 most of the week in bed and not being productive.

 >>Hum, wait a minute ... Ok, I got it ! In 1.4.x dd doesn't use lseek on raw
 >>disk devices, so it'll do 745508 reads here.
 >
 >*ARGH*
 >
 >That's terribly frustrating, but it explains a lot, and it also
 >explains a bizarre phenonmenon where I saw errors on the raw device
 >but not on the block device.
 >
 >Argh.
 >
 >I'll bitch and probably do a doc fix and pullup request for 1.4.

 Done (last week).

 I think this basically addresses PR 9856. Point #2 is certainly covered.

 Point #3 you've suggested is fixed by your patch to check WDCE_CRC
 and DRIVE_UDMA before ata_dmaerr()-ing. I haven't had a great
 opportunity to test that. I'm running with it and haven't seen
 a problem, though. Let me know if you'd like something more rigourous
 done.

 With respect to point #1, in <20000414013114.A398@antioche.eu.org>,
 you said:

 | > which is confusing.  I've patched my kernel to report it as "(obsolete)"
 | 
 | Ok, I changed it to : "obsolete (was address mark not found)", to make
 | it clear.
 | 

 But this doesn't seem to have been committed.

 In <20000414013114.A398@antioche.eu.org> et seq we hypothesized that
 the drive was not really ATA4 compliant. Removing the #if 0 code in
 wdc.c, it reports itself as a v4 drive. For what it's worth.


 I'll change the PR to 'analyzed' with you as responsible.
 Feel free to close it when the errno==1 stuff is fixed.

 Thanks for your help.

 --jhawk
State-Changed-From-To: open->analyzed 
State-Changed-By: jhawk 
State-Changed-When: Thu Apr 27 19:47:13 PDT 2000 
State-Changed-Why:  
Principal problem addressed, solutions drafted for anciallary issues. 


Responsible-Changed-From-To: kern-bug-people->Manuel Bouyer <bouyer@antioche.lip6.fr> 
Responsible-Changed-By: jhawk 
Responsible-Changed-When: Thu Apr 27 19:47:13 PDT 2000 
Responsible-Changed-Why:  
Manuel is dealing. 
Responsible-Changed-From-To: Manuel->bouyer 
Responsible-Changed-By: jhawk 
Responsible-Changed-When: Thu Apr 27 19:54:42 PDT 2000 
Responsible-Changed-Why:  
typo fixup 

From: John Hawkinson <jhawk@MIT.EDU>
To: Manuel Bouyer <bouyer@antioche.lip6.fr>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/9856: timeout problem. 
Date: Sat, 29 Apr 2000 20:06:32 -0400

 In message <200004280246.WAA304314@steve-dallas.mit.edu>, John Hawkinson writes
 :
 >Point #3 you've suggested is fixed by your patch to check WDCE_CRC
 >and DRIVE_UDMA before ata_dmaerr()-ing. I haven't had a great
 >opportunity to test that. I'm running with it and haven't seen
 >a problem, though. Let me know if you'd like something more rigourous
 >done.

 Tested. The downgrades no longer happen.

 --jhawk
State-Changed-From-To: analyzed->closed 
State-Changed-By: mycroft 
State-Changed-When: Wed Jul 7 00:35:52 UTC 2004 
State-Changed-Why:  
The real problems here have been addressed. 


Responsible-Changed-From-To: bouyer->mycroft 
Responsible-Changed-By: mycroft 
Responsible-Changed-When: Wed Jul 7 00:35:52 UTC 2004 
Responsible-Changed-Why:  
. 
>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.