NetBSD Problem Report #58684

From he@smistad.uninett.no  Fri Sep 20 21:09:44 2024
Return-Path: <he@smistad.uninett.no>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id E0E771A923D
	for <gnats-bugs@gnats.NetBSD.org>; Fri, 20 Sep 2024 21:09:44 +0000 (UTC)
Message-Id: <20240920210938.911FF43F257@smistad.uninett.no>
Date: Fri, 20 Sep 2024 23:09:38 +0200 (CEST)
From: he@NetBSD.org
Reply-To: he@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: mpii driver wedges during install of 10.0
X-Send-Pr-Version: 3.95

>Number:         58684
>Category:       kern
>Synopsis:       mpii driver wedges during install of 10.0
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Sep 20 21:10:00 +0000 2024
>Last-Modified:  Mon Aug 17 15:35:01 +0000 2026
>Originator:     Havard Eidnes
>Release:        NetBSD 10.0
>Organization:
I Try...
>Environment:
System: NetBSD 10.0 USB UEFI amd64 install image
Architecture: x86_64
Machine: amd64
>Description:
	I'm trying to install NetBSD/amd64 10.0 on a Fujitsu RX2540 M4
	equipped with a 600GB SAS disk, connected via a controller
	identifying as

mpii0 at pci9 dev 0 function 0: Symbios Logic SAS3008 (rev. 0x02)
mpii0: interrupting at msix7 vec 0
mpii0: PSAS CP400i, firmware 16.0.0.0IR, MPI 2.5
mpii0: phsyical device inserted in slot 4

	Before I started the installation, I exited to the shell
	and successfully did

# dd if=/dev/urandom bs=32k of=/dev/rsd0d

	and I let it run to completion (took a few hours).

	However, for the extraction of the install sets from within
	sysinst, from the booted-from USB drive, the target file
	system is mounted async, so a large amount of data is being
	buffered, and when the buffering "runs out" there is going
	to be a thundering herd of I/O requests.  It appears that
	the driver is unable to cope with this situation, because
	the drive light extinguishes, and the extraction stalls
	from within sysinst.  After more than a minute I get a long
	slew of kernel messages saying

mpii0: mpii_scsi_cmd_tmo

	I think in the order of 50 such messages in quick succession.
	After this happens, any attempt at doing I/O to sd0 will
	stall.

	Sadly, the host does not have a serial port, and I've yet
	to figure out how to configure and use the basic system
	management on the host (they are "all different", grr!),
	so the above is transcribed from pictures from my phone.

	At one attempt, I suspended sysinst quickly after starting
	the set extraction, and did

# mount -u -o noasync /targetroot

	in an attempt to even out the I/O requests, but apparently
	it was at that point already too late, the extraction had
	already come too far at that point.

	Any information I should collect at that point?

	I've broken into DDB and did "show uvmexp", but there's
	more than 7M pages free (the host has 32GB memory).

	I also did "vmstat -m", but didn't see anything untoward,
	but then again I don't really know what would be abnormal
	in that output.  Sadly, I didn't take a picture of that one...

>How-To-Repeat:
	Try to install with something similar to the above hardware.
>Fix:
	That would be nice.

>Release-Note:

>Audit-Trail:

From: "Havard Eidnes" <he@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58684 CVS commit: src/sys/dev/pci
Date: Mon, 3 Aug 2026 14:43:23 +0000

 Module Name:	src
 Committed By:	he
 Date:		Mon Aug  3 14:43:23 UTC 2026

 Modified Files:
 	src/sys/dev/pci: mpii.c

 Log Message:
 sys/dev/pci/mpii.c: Reduce MPII_REQUEST_CREDIT to improve stability.

 This should fix PR#58684, at least it does in my case on a
 host with a "Symbios Logic SAS3008 (rev. 0x02)" controller.

 Admittedly, this is a band-aid fix, but it's the best we could come
 up with to solve the PR, and it does not appear to have significant
 performance implications, at least not in a single-drive configuration.
 So, better to be stable / functional than not.


 To generate a diff of this commit:
 cvs rdiff -u -r1.31 -r1.32 src/sys/dev/pci/mpii.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58684 CVS commit: [netbsd-11] src/sys/dev/pci
Date: Mon, 17 Aug 2026 15:27:56 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Mon Aug 17 15:27:56 UTC 2026

 Modified Files:
 	src/sys/dev/pci [netbsd-11]: mpii.c

 Log Message:
 Pull up following revision(s) (requested by he in ticket #412):

 	sys/dev/pci/mpii.c: revision 1.32

 sys/dev/pci/mpii.c: Reduce MPII_REQUEST_CREDIT to improve stability.

 This should fix PR#58684, at least it does in my case on a
 host with a "Symbios Logic SAS3008 (rev. 0x02)" controller.

 Admittedly, this is a band-aid fix, but it's the best we could come
 up with to solve the PR, and it does not appear to have significant
 performance implications, at least not in a single-drive configuration.

 So, better to be stable / functional than not.


 To generate a diff of this commit:
 cvs rdiff -u -r1.31 -r1.31.4.1 src/sys/dev/pci/mpii.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58684 CVS commit: [netbsd-10] src/sys/dev/pci
Date: Mon, 17 Aug 2026 15:30:06 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Mon Aug 17 15:30:06 UTC 2026

 Modified Files:
 	src/sys/dev/pci [netbsd-10]: mpii.c

 Log Message:
 Pull up following revision(s) (requested by he in ticket #1323):

 	sys/dev/pci/mpii.c: revision 1.32

 sys/dev/pci/mpii.c: Reduce MPII_REQUEST_CREDIT to improve stability.

 This should fix PR#58684, at least it does in my case on a
 host with a "Symbios Logic SAS3008 (rev. 0x02)" controller.

 Admittedly, this is a band-aid fix, but it's the best we could come
 up with to solve the PR, and it does not appear to have significant
 performance implications, at least not in a single-drive configuration.

 So, better to be stable / functional than not.


 To generate a diff of this commit:
 cvs rdiff -u -r1.29.6.1 -r1.29.6.2 src/sys/dev/pci/mpii.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

 From: Havard Eidnes <he@NetBSD.org>
 To: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
  netbsd-bugs@netbsd.org
 Cc: 
 Subject: Re: kern/58684: mpii driver wedges during install of 10.0
 Date: Sat, 28 Sep 2024 15:53:20 +0200 (CEST)

  Hm,

  I see I didn't post an update to this, so here is one such.

  "Behind the scenes" I got some hints which led me to being able
  to resolve this problem.  At least in this particular case,
  adjusting the number of outstanding requests to the "scsi"
  controller (the drives are SAS, the controller also has RAID
  functionality, but I use it in JBOD mode).

  The dmesg (mis-labeled as RX2540, it's an RX2530) can be seen at

     https://dmesgd.nycbug.org/index.cgi?do=view&id=7934

  and the SAS controller is identified as

  [     1.062153] mpii0 at pci9 dev 0 function 0: Symbios Logic SAS3008 (rev. 0x02)
  [     1.062153] mpii0: interrupting at msix7 vec 0
  [     1.062153] mpii0: PSAS CP400i, firmware 16.0.0.0 IR, MPI 2.5

  The change which made this system work was:

  Index: sys/dev/pci/mpii.c
  ===================================================================
  RCS file: /cvsroot/src/sys/dev/pci/mpii.c,v
  retrieving revision 1.29.6.1
  diff -u -r1.29.6.1 mpii.c
  --- sys/dev/pci/mpii.c  26 Oct 2023 15:11:02 -0000      1.29.6.1
  +++ sys/dev/pci/mpii.c  28 Sep 2024 13:43:31 -0000
  @@ -91,7 +91,7 @@
   #endif

   #define MPII_REQUEST_SIZE              (512)
  -#define MPII_REQUEST_CREDIT            (128)
  +#define MPII_REQUEST_CREDIT            (64)    /* Was: 128 */

   struct mpii_dmamem {
          bus_dmamap_t            mdm_map;

  An earlier attempt at going just down to 100 did not resolve the
  problem, it merely postponed the wedge of the IO to a bit later
  stage in the set extraction process during the installation.

  The obvious question is whether we should apply this as a
  workaround or fix or whether more analysis is required before
  doing so.

  It would be nice being able to install plain 10.1 once that's
  released on this kind of systems...

  Best regards,

  - Havard

Home	PR Database Search