NetBSD Problem Report #30531
From prlw1@newn.cam.ac.uk Wed Jun 15 16:38:40 2005
Return-Path: <prlw1@newn.cam.ac.uk>
Received: from henry.newn.cam.ac.uk (henry.newn.cam.ac.uk [131.111.204.130])
by narn.netbsd.org (Postfix) with ESMTP id CAAC663B116
for <gnats-bugs@gnats.NetBSD.org>; Wed, 15 Jun 2005 16:38:39 +0000 (UTC)
Message-Id: <E1DiauP-0007cq-Gh@quartz.newn.cam.ac.uk>
Date: Wed, 15 Jun 2005 17:38:33 +0100
From: prlw1@cam.ac.uk
Sender: "Patrick Welche,SCC,ext.35710," <prlw1@newn.cam.ac.uk>
Reply-To: prlw1@cam.ac.uk
To: gnats-bugs@netbsd.org
Subject: lsilogic raid array crawls
X-Send-Pr-Version: 3.95
>Number: 30531
>Category: kern
>Synopsis: 4MB/s out of mpt(4) 53c1030 striped drives
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Jun 15 16:39:00 +0000 2005
>Closed-Date:
>Last-Modified: Sun Jan 20 10:47:32 +0000 2008
>Originator: Patrick Welche
>Release: NetBSD 3.99.6
>Organization:
>Environment:
source of 15 June 2005, i386
>Description:
The problematic box is a Dell Precision 360, which comes with an
LSI21320-IS card, with matching MPTBIOS-IS-5.02.33 firmware,
(IS = integrated striping)
mpt1 at pci2 dev 3 function 1: LSI Logic 53c1030 Ultra320 SCSI
and 2 73GB drives which are striped,
sd0 at scsibus1 target 0 lun 0: <LSILOGIC, RAID ARRAY IS, 1000> disk fixed
sd0: fabricating a geometry
sd0: 136 GB, 139712 cyl, 64 head, 32 sec, 512 bytes/sect x 286130176 sectors
sd0: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
The problem is that the data transfer speed is miserable e.g.
% dd if=/dev/rsd0d of=/dev/null bs=64k count=1000
1000+0 records in
1000+0 records out
65536000 bytes transferred in 15.151 secs (4325523 bytes/sec)
By comparison, a Nov 2004 Fedora Core rescue CD
Linux localhost.localdomain 2.6.9-1.667 #1 Tue Nov 2 14:41:31 EST 2004 i686 unknown
gives:
% time dd if=/dev/sda of=/dev/null bs=64k count=1000
1000+0 records in
1000+0 records out
real 0m0.727s
user 0m0.006s
sys 0m0.060s
which is useable.
>How-To-Repeat:
>Fix:
>Release-Note:
>Audit-Trail:
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: kern-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Wed, 15 Jun 2005 20:18:16 +0200
On Wed, Jun 15, 2005 at 04:39:00PM +0000, prlw1@cam.ac.uk wrote:
> [...]
> By comparison, a Nov 2004 Fedora Core rescue CD
> Linux localhost.localdomain 2.6.9-1.667 #1 Tue Nov 2 14:41:31 EST 2004 i686 unknown
> gives:
>
> % time dd if=/dev/sda of=/dev/null bs=64k count=1000
> 1000+0 records in
> 1000+0 records out
>
> real 0m0.727s
> user 0m0.006s
> sys 0m0.060s
Note that on linux /dev/sda goes though the buffer cache. Your dd may not
have touched the disk at all. To be sure, can you try with a transfer at last
2x larger than your RAM (twice, to be sure to flush the entire cache).
--
Manuel Bouyer <bouyer@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
From: Patrick Welche <prlw1@newn.cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc: bouyer@antioche.eu.org, netbsd-bugs@netbsd.org,
gnats-admin@netbsd.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Thu, 16 Jun 2005 16:46:11 +0100
The computer has 2Gb of memory, so I just repeated the dd with the
aforementioned Fedora and got:
% time dd if=/dev/sda of=/dev/null bs=1024k count=4096
4096+0 records in
4096+0 records out
real 0m49.366s
user 0m0.015s
sys 0m6.148s
So a rough and ready estimate is 4096/49.366 = 83 MB/s >> 4 MB/s
(so I don't think I will repeat with 4GB under NetBSD..)
Are any of you using mpt(4) successfully? If so with what hardware?
(It would make playing "spot the difference" easier..)
Cheers,
Patrick
From: Patrick Welche <prlw1@newn.cam.ac.uk>
To: gnats-bugs@netbsd.org
Cc: bouyer@antioche.eu.org, netbsd-bugs@netbsd.org,
gnats-admin@netbsd.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Tue, 21 Jun 2005 16:35:40 +0100
Looking at mptlinux-3.03.02-src.tar.gz, mptbase.c, mpt_attach() I see:
/* 1030 Chip Fix. Disable Split transactions
* for PCIX. Set MOST bits to zero if Rev < C0( = 8).
*/
pci_read_config_byte(pdev, PCI_CLASS_REVISION, &revision);
if (revision < C0_1030) {
pci_read_config_byte(pdev, 0x6a, &pcixcmd);
pcixcmd &= 0x8F;
pci_write_config_byte(pdev, 0x6a, pcixcmd);
}
and naturally this card is a revision 7. I haven't seen a similar work
around in the NetBSD source (I've been looking at /sys/dev/{pci,ic}/mpt*).
BTW there are quite a few like that for the fibre channel cards.
Cheers,
Patrick
From: Jason Thorpe <thorpej@shagadelic.org>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Tue, 21 Jun 2005 09:30:12 -0700
On Jun 21, 2005, at 8:36 AM, Patrick Welche wrote:
> and naturally this card is a revision 7. I haven't seen a similar
> work
> around in the NetBSD source (I've been looking at /sys/dev/
> {pci,ic}/mpt*).
> BTW there are quite a few like that for the fibre channel cards.
Are you plugged into a PCIX slot? If not, then this work-around
won't make any difference.
-- thorpej
From: Patrick Welche <prlw1@newn.cam.ac.uk>
To: thorpej@shagadelic.org
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Thu, 30 Jun 2005 10:27:24 +0100
I am not plugged into a PCIX slot.
[re the bug system: I only just checked the bug database because I was about
to report that this patch:
Index: mpt_pci.c
===================================================================
RCS file: /cvsroot/src/sys/dev/pci/mpt_pci.c,v
retrieving revision 1.5
diff -u -r1.5 mpt_pci.c
--- mpt_pci.c 27 Feb 2005 00:27:33 -0000 1.5
+++ mpt_pci.c 30 Jun 2005 09:17:01 -0000
@@ -210,7 +210,20 @@
pci_conf_write(pa->pa_pc, pa->pa_tag, PCI_COMMAND_STATUS_REG, reg);
/*
- * Ensure that the ROM is diabled.
+ * quirk from mptbase.c,mptlinux-3.03.02-src.tar.gz
+ * Disable split transactions for rev < 8.
+ * XXX here be magic constants...
+ */
+ if ((PCI_PRODUCT(pa->pa_id) == PCI_PRODUCT_SYMBIOS_1030)
+ && (PCI_REVISION(pa->pa_class) < 0x08)) {
+ aprint_normal("%s: applying 1030 quirk\n",mpt->sc_dev.dv_xname);
+ reg = pci_conf_read(pa->pa_pc, pa->pa_tag, 0x6a);
+ reg &= 0x8f;
+ pci_conf_write(pa->pa_pc, pa->pa_tag, 0x6a, reg);
+ }
+
+ /*
+ * Ensure that the ROM is disabled.
*/
reg = pci_conf_read(pa->pa_pc, pa->pa_tag, PCI_MAPREG_ROM);
reg &= ~1;
@@ -253,7 +266,7 @@
* Hard resets are known to screw up the BAR for diagnostic
* memory accesses (Mem1).
*
- * Using Mem1 is know to make the chip stop responding to
+ * Using Mem1 is known to make the chip stop responding to
* configuration cycles, so we need to save it now.
*/
mpt_pci_read_config_regs(mpt);
achieves nothing. Had I actually received Jason's mail that would have
saved much head scratching!]
From: Patrick Welche <prlw1@newn.cam.ac.uk>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
netbsd-bugs@netbsd.org
Cc: bouyer@antioche.eu.org, thorpej@shagadelic.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Wed, 13 Jul 2005 20:04:10 +0100
The LSI21320-IS card has "IS" = "integrated striping", in other words
mirroring. If I use its firmware utility to break the mirror back into
two disks:
sd0 at scsibus1 target 0 lun 0: <FUJITSU, MAS3735NP, 5B08> disk fixed
sd0: 70007 MB, 27094 cyl, 8 head, 661 sec, 512 bytes/sect x 143374650 sectors
sd0: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
sd1 at scsibus1 target 1 lun 0: <FUJITSU, MAS3735NP, 5B08> disk fixed
sd1: 70007 MB, 27094 cyl, 8 head, 661 sec, 512 bytes/sect x 143374650 sectors
sd1: sync (6.25ns offset 127), 16-bit (320.000MB/s) transfers, tagged queueing
then the simple test of
dd if=/dev/rsd0d of=/dev/null bs=64k count=1000
gives a rather more respectable speed of 78392344 bytes/sec (about 75MB/s)
Any thoughts on how to spot what linux is doing that we aren't?
(have kgdb set up + chip manual)
Cheers,
Patrick
From: rivo nurges <rix@estpak.ee>
To: thorpej@shagadelic.org, Patrick Welche <prlw1@newn.cam.ac.uk>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/30531: lsilogic raid array crawls
Date: Mon, 13 Feb 2006 11:23:17 +0000
Hi!
> I am not plugged into a PCIX slot.
>
>
> [re the bug system: I only just checked the bug database because I was about
> to report that this patch:
>
>
> Index: mpt_pci.c
> ===================================================================
> RCS file: /cvsroot/src/sys/dev/pci/mpt_pci.c,v
> retrieving revision 1.5
> diff -u -r1.5 mpt_pci.c
> --- mpt_pci.c 27 Feb 2005 00:27:33 -0000 1.5
> +++ mpt_pci.c 30 Jun 2005 09:17:01 -0000
> @@ -210,7 +210,20 @@
> pci_conf_write(pa->pa_pc, pa->pa_tag, PCI_COMMAND_STATUS_REG, reg);
>
>
> /*
> - * Ensure that the ROM is diabled.
> + * quirk from mptbase.c,mptlinux-3.03.02-src.tar.gz
> + * Disable split transactions for rev < 8.
> + * XXX here be magic constants...
> + */
> + if ((PCI_PRODUCT(pa->pa_id) == PCI_PRODUCT_SYMBIOS_1030)
> + && (PCI_REVISION(pa->pa_class) < 0x08)) {
> + aprint_normal("%s: applying 1030
> quirk\n",mpt->sc_dev.dv_xname);
> + reg = pci_conf_read(pa->pa_pc, pa->pa_tag, 0x6a);
> + reg &= 0x8f;
> + pci_conf_write(pa->pa_pc, pa->pa_tag, 0x6a, reg);
> + }
> +
> + /*
> + * Ensure that the ROM is disabled.
> */
> reg = pci_conf_read(pa->pa_pc, pa->pa_tag, PCI_MAPREG_ROM);
> reg &= ~1;
> @@ -253,7 +266,7 @@
> * Hard resets are known to screw up the BAR for diagnostic
> * memory accesses (Mem1).
> *
> - * Using Mem1 is know to make the chip stop responding to
> + * Using Mem1 is known to make the chip stop responding to
> * configuration cycles, so we need to save it now.
> */
> mpt_pci_read_config_regs(mpt);
This patch doesn't help on kern/30531 however it will fix kern/26825 like
symptoms on 1030 plugged into PCIX slot.
$ dmesg | grep mpt
mpt0 at pci6 dev 3 function 0: LSI Logic 53c1030 Ultra320 SCSI
mpt0: applying 1030 quirk
mpt0: interrupting at irq 5
--
rix
http://www.ripe.net/perl/whois?rix@estpak.ee
Responsible-Changed-From-To: kern-bug-people->tron
Responsible-Changed-By: tron@netbsd.org
Responsible-Changed-When: Thu, 03 Jan 2008 18:14:46 +0000
Responsible-Changed-Why:
I'll handle this PR.
State-Changed-From-To: open->analyzed
State-Changed-By: tron@netbsd.org
State-Changed-When: Thu, 03 Jan 2008 18:16:54 +0000
State-Changed-Why:
I've committed your patch after a bit of clean-up. I'll ask for testers
and request a pullup to "netbsd-4" later.
From: Matthias Scheler <tron@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: PR/30531 CVS commit: src/sys/dev/pci
Date: Thu, 3 Jan 2008 18:16:20 +0000 (UTC)
Module Name: src
Committed By: tron
Date: Thu Jan 3 18:16:20 UTC 2008
Modified Files:
src/sys/dev/pci: mpt_pci.c
Log Message:
Fix perfomance problems for old revisions of the Symbios 53c1030.
Patch supplied by Patrick Welche in PR kern/30531.
To generate a diff of this commit:
cvs rdiff -r1.12 -r1.13 src/sys/dev/pci/mpt_pci.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Patrick Welche <prlw1@newn.cam.ac.uk>
To: gnats-bugs@NetBSD.org
Cc: tron@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
prlw1@cam.ac.uk
Subject: Re: PR/30531 CVS commit: src/sys/dev/pci
Date: Thu, 3 Jan 2008 18:31:30 +0000
On Thu, Jan 03, 2008 at 06:20:02PM +0000, Matthias Scheler wrote:
> Fix perfomance problems for old revisions of the Symbios 53c1030.
Just to avoid confusion: sadly it doesn't fix the performance problem,
as noted by thorpej, but will (probably) fix kern/26825 as noted by
rivo nurges.
Responsible-Changed-From-To: tron->kern-bug-people
Responsible-Changed-By: tron@narn.netbsd.org
Responsible-Changed-When: Sun, 20 Jan 2008 10:47:32 +0000
Responsible-Changed-Why:
I don't have any mpt(4) hardware.
State-Changed-From-To: analyzed->open
State-Changed-By: tron@narn.netbsd.org
State-Changed-When: Sun, 20 Jan 2008 10:47:32 +0000
State-Changed-Why:
I've committed Patrick Welche's patch. But it apparently doesn't fix
the performance problem.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home