NetBSD Problem Report #57669

From www@netbsd.org  Thu Oct 19 11:58:43 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 565991A9238
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 19 Oct 2023 11:58:43 +0000 (UTC)
Message-Id: <20231019115841.CA8311A9239@mollari.NetBSD.org>
Date: Thu, 19 Oct 2023 11:58:41 +0000 (UTC)
From: joel.bertrand@systella.fr
Reply-To: joel.bertrand@systella.fr
To: gnats-bugs@NetBSD.org
Subject: [iscsi] Panic in ccb_timeout
X-Send-Pr-Version: www-1.0

>Number:         57669
>Category:       kern
>Synopsis:       [iscsi] Panic in ccb_timeout
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    mlelstv
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 19 12:00:01 +0000 2023
>Last-Modified:  Sun Dec 17 10:55:01 +0000 2023
>Originator:     BERTRAND Joël
>Release:        -10 beta
>Organization:
>Environment:
NetBSD legendre.systella.fr 10.0_BETA NetBSD 10.0_BETA (CUSTOM) #4: Sun Sep 24 21:59:27 CEST 2023  root@legendre.systella.fr:/usr/src/netbsd-10/obj/sys/arch/amd64/compile/CUSTOM amd64
CUSTOM=GENERIC+ALTQ support

>Description:
Hello,

I was pretty sure I have sent a PR with this issue but I unable to find it in database.

For a long time (since -9.1), iscsi initiator crashes and kernel panics with :

[ 3604460.041870] S-1C-1: ccb_timeout: num=1 total=1 disp=0
[ 3604460.041870] uvm_fault(0xffffffff819755e0, 0x0, 2) -> e
[ 3604460.041870] fatal page fault in supervisor mode
[ 3604460.041870] trap type 6 code 0x2 rip 0xffffffff8023fa4c cs 0x8 rflags 0x10246 cr2 0x10 ilevel 0 rsp 0xffffc70448066f68
[ 3604460.041870] curlwp 0xfffffe51a2c8a940 pid 0.325 lowest kstack 0xffffc704480622c0
[ 3604460.041870] panic: trap
[ 3604460.041870] cpu2: Begin traceback...
[ 3604460.041870] vpanic() at netbsd:vpanic+0x183
[ 3604460.041870] panic() at netbsd:panic+0x3c
[ 3604460.051870] trap() at netbsd:trap+0xbaf
[ 3604460.051870] --- trap (number 6) ---
[ 3604460.051870] mutex_enter() at netbsd:mutex_enter+0xc
[ 3604460.051870] ccb_timeout() at netbsd:ccb_timeout+0xa6
[ 3604460.061869] iscsi_cleanup_thread() at netbsd:iscsi_cleanup_thread+0x333
[ 3604460.061869] cpu2: End traceback...
[ 3604460.061869] dumping to dev 18,1 (offset=251919, size=4162814):

Unfortunately, I have upgraded my tree juste before panic and I don't have netbsd.gdb. Kernel notices that it dumps memory but after reboot /var/crash is empty.

>How-To-Repeat:
Two NAS (qNAP) using iSCSI. Each NAS is connected to an ethernet controler (only one IP addresse as wm0 and wm1 as grouped in bridge0).

bridge0: flags=0x41<UP,RUNNING> mtu 9000
   capabilities=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
   capabilities=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
   capabilities=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
   enabled=0x3ff00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx>
   enabled=0x3ff00<UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx,TCP6CSUM_Tx>
   enabled=0x3ff00<UDP6CSUM_Rx,UDP6CSUM_Tx>
   status: active

iSCSI is configured without special feature:
name="iscsictl"
rcvar=$name
command="/sbin/${name}"
pidfile="/var/run/${name}.pid"

case $1 in
start)
    sleep 5
    ${command} add_send_target -a 192.168.12.2
    ${command} add_send_target -a 192.168.12.3
    ${command} refresh_targets
    ${command} list_targets
    ${command} login -P `iscsictl list_targets | \
        awk '/192.168.12.2/ { print $1; }' | tr -d ":"`
    ${command} login -P `iscsictl list_targets | \
        awk '/192.168.12.3/ { print $1; }' | tr -d ":"`
    sleep 1
    mount -o rw,log NAME=bacula /opt/bacula
    mount -o rw,log NAME=video /opt/video
    /etc/rc.d/mountd reload
    ;;

Wait now for panic (last panic occurs after 40 days).

>Fix:

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: kern-bug-people->mlelstv
Responsible-Changed-By: mlelstv@NetBSD.org
Responsible-Changed-When: Tue, 28 Nov 2023 05:58:50 +0000
Responsible-Changed-Why:
my code


From: Michael van Elst <mlelstv@serpens.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/57669 [iscsi] Panic in ccb_timeout
Date: Tue, 28 Nov 2023 07:00:46 +0100

 I have committed a few changes to -current that may fix this panic.

 Can you apply the changes from -current to netbsd-10, build new modules
 and see if these help ?

 This patch should apply to netbsd-10 (or just copy over the .c and .h
 files in sys/dev/iscsi from -current):

 http://cdn.netbsd.org/pub/NetBSD/misc/mlelstv/10-iscsi.diff


 Greetings,
 -- 
                                 Michael van Elst
 Internet: mlelstv@serpens.de
                                 "A potential Snark may lurk in every tree."

From: =?UTF-8?Q?BERTRAND_Jo=c3=abl?= <joel.bertrand@systella.fr>
To: gnats-bugs@netbsd.org, mlelstv@netbsd.org, gnats-admin@netbsd.org,
        netbsd-bugs@netbsd.org
Cc: 
Subject: Re: kern/57669 [iscsi] Panic in ccb_timeout
Date: Tue, 28 Nov 2023 10:47:41 +0100

 This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
 --dyscXsfZ0rFKoBY5BNlrwPtpUH7Iq8Oiy
 Content-Type: multipart/mixed; boundary="3sMFLoyHTYPY3ru9AsHNFQBXQjdWr0xwW"

 --3sMFLoyHTYPY3ru9AsHNFQBXQjdWr0xwW
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable

 	Hello,

 	Patch applied and in test.

 	Regards,

 	JB


 --3sMFLoyHTYPY3ru9AsHNFQBXQjdWr0xwW--

 --dyscXsfZ0rFKoBY5BNlrwPtpUH7Iq8Oiy
 Content-Type: application/pgp-signature; name="signature.asc"
 Content-Description: OpenPGP digital signature
 Content-Disposition: attachment; filename="signature.asc"

 -----BEGIN PGP SIGNATURE-----

 iHUEABYIAB0WIQQj8MW8iOsC2RXEznnFW/s/mMLXCAUCZWW3bQAKCRDFW/s/mMLX
 CHevAP93Czepr010ucYTCpcskVz/oxflsTiKIqh1fSCzI4gyfAD9EV4mhsSmbjyC
 CSpOfKYYL2t1Acji2Zf4K0dGM0l0CQQ=
 =U60Y
 -----END PGP SIGNATURE-----

 --dyscXsfZ0rFKoBY5BNlrwPtpUH7Iq8Oiy--

From: =?UTF-8?Q?BERTRAND_Jo=c3=abl?= <joel.bertrand@systella.fr>
To: gnats-bugs@netbsd.org, mlelstv@netbsd.org, gnats-admin@netbsd.org,
        netbsd-bugs@netbsd.org
Cc: 
Subject: Re: kern/57669 [iscsi] Panic in ccb_timeout
Date: Sun, 17 Dec 2023 11:53:17 +0100

 This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
 --RiBLxTcakrjCcczGV62D1J8ZPsWnbv5Jo
 Content-Type: multipart/mixed; boundary="UNRIIlfR1O7NmzlRPGaqiUXwS3Ji72FMH"

 --UNRIIlfR1O7NmzlRPGaqiUXwS3Ji72FMH
 Content-Type: text/plain; charset=UTF-8
 Content-Transfer-Encoding: quoted-printable

 Michael van Elst a =C3=A9crit=C2=A0:
 > The following reply was made to PR kern/57669; it has been noted by GNA=
 TS.
 >=20
 > From: Michael van Elst <mlelstv@serpens.de>
 > To: gnats-bugs@netbsd.org
 > Cc:=20
 > Subject: Re: kern/57669 [iscsi] Panic in ccb_timeout
 > Date: Tue, 28 Nov 2023 07:00:46 +0100
 >=20
 >  I have committed a few changes to -current that may fix this panic.
 > =20
 >  Can you apply the changes from -current to netbsd-10, build new module=
 s
 >  and see if these help ?
 > =20
 >  This patch should apply to netbsd-10 (or just copy over the .c and .h
 >  files in sys/dev/iscsi from -current):
 > =20
 >  http://cdn.netbsd.org/pub/NetBSD/misc/mlelstv/10-iscsi.diff

 	Hello,

 	I think you can apply this patch to -10. Since I have applied patch, I
 haven't seen any new panic:

 legendre:[~] > uptime
 11:51AM  up 19 days,  2:51, 4 users, load averages: 1.63, 1.27, 1.29

 	Best regards,

 	JB


 --UNRIIlfR1O7NmzlRPGaqiUXwS3Ji72FMH--

 --RiBLxTcakrjCcczGV62D1J8ZPsWnbv5Jo
 Content-Type: application/pgp-signature; name="signature.asc"
 Content-Description: OpenPGP digital signature
 Content-Disposition: attachment; filename="signature.asc"

 -----BEGIN PGP SIGNATURE-----

 iHUEABYIAB0WIQQj8MW8iOsC2RXEznnFW/s/mMLXCAUCZX7THgAKCRDFW/s/mMLX
 CARUAP9WqNvTirYKH1ShAz6jW6ZSt2ZEFyJ+macxB68xOWHDOwD+I9AqlCfbauGk
 7M+B4yzIPFjE2L0CXaxj32CXWJS7xQM=
 =TTqr
 -----END PGP SIGNATURE-----

 --RiBLxTcakrjCcczGV62D1J8ZPsWnbv5Jo--

>Unformatted:

Home	PR Database Search