NetBSD Problem Report #49072

From www@NetBSD.org  Mon Aug  4 23:28:42 2014
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id CA682A9327
	for <gnats-bugs@gnats.NetBSD.org>; Mon,  4 Aug 2014 23:28:41 +0000 (UTC)
Message-Id: <20140804232840.9B5D4A932F@mollari.NetBSD.org>
Date: Mon,  4 Aug 2014 23:28:40 +0000 (UTC)
From: zafer@aydogan.de
Reply-To: zafer@aydogan.de
To: gnats-bugs@NetBSD.org
Subject: in-kernel iscsi initiator not working
X-Send-Pr-Version: www-1.0

>Number:         49072
>Category:       kern
>Synopsis:       in-kernel iscsi initiator not working
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Aug 04 23:30:00 +0000 2014
>Last-Modified:  Wed Aug 06 13:00:00 +0000 2014
>Originator:     Zafer Aydogan
>Release:        6.99.16 onward
>Organization:
TNF
>Environment:
6.99.49 amd64
>Description:
The in-kernel iscsi initiator looks like to be broken since 6.99.16.
Symptoms are scsi SENSE KEY errors with "Overlapped Commands Attempted"
when you write data to disk.
If you continue to write data to the iscsi disk, the write process will wedge in tstile.
In this state a reboot will not succeed. You need to power-cycle.
Sometimes writing to the disk will panic the system, especially overwriting the same file frequently.

I could trace the error back to two commits from mlelstv@ on 2012-12-29:
http://freshbsd.org/commit/netbsd/0407d4dcf2917f3861618fc1cbc54e6bd5b583b3
and
http://freshbsd.org/commit/netbsd/77a0b440f42ba059ed279f043194a4b589203054
If you revert both or use a version prior to that date, iscsi works as expected without any issues.

Screenshots:
http://www.aydogan.de/wtfs.png
http://www.aydogan.de/iscsi_panic.png

>How-To-Repeat:
mount an iscsi share and
newfs -O1 on a partition

newfs will exit with a wtfs write error

Attention: newfs -O2 will show no errors.


>Fix:
A workaround is to revert to the version before 2012-12-29 08:00:00

>Audit-Trail:
From: Greg Oster <oster@cs.usask.ca>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/49072: in-kernel iscsi initiator not working
Date: Tue, 5 Aug 2014 08:57:39 -0600

 On Mon,  4 Aug 2014 23:30:00 +0000 (UTC)
 zafer@aydogan.de wrote:

 > >Number:         49072
 > >Category:       kern
 > >Synopsis:       in-kernel iscsi initiator not working
 > >Confidential:   no
 > >Severity:       serious
 > >Priority:       high
 > >Responsible:    kern-bug-people
 > >State:          open
 > >Class:          sw-bug
 > >Submitter-Id:   net
 > >Arrival-Date:   Mon Aug 04 23:30:00 +0000 2014
 > >Originator:     Zafer Aydogan
 > >Release:        6.99.16 onward
 > >Organization:
 > TNF
 > >Environment:
 > 6.99.49 amd64
 > >Description:
 > The in-kernel iscsi initiator looks like to be broken since 6.99.16.
 > Symptoms are scsi SENSE KEY errors with "Overlapped Commands
 > Attempted" when you write data to disk.
 > If you continue to write data to the iscsi disk, the write process
 > will wedge in tstile. In this state a reboot will not succeed. You
 > need to power-cycle. Sometimes writing to the disk will panic the
 > system, especially overwriting the same file frequently.
 > 
 > I could trace the error back to two commits from mlelstv@ on
 > 2012-12-29:
 > http://freshbsd.org/commit/netbsd/0407d4dcf2917f3861618fc1cbc54e6bd5b583b3
 > and
 > http://freshbsd.org/commit/netbsd/77a0b440f42ba059ed279f043194a4b589203054
 > If you revert both or use a version prior to that date, iscsi works
 > as expected without any issues.
 > 
 > Screenshots:
 > http://www.aydogan.de/wtfs.png
 > http://www.aydogan.de/iscsi_panic.png
 > 
 > >How-To-Repeat:
 > mount an iscsi share and
 > newfs -O1 on a partition
 > 
 > newfs will exit with a wtfs write error
 > 
 > Attention: newfs -O2 will show no errors.
 > 
 > 
 > >Fix:
 > A workaround is to revert to the version before 2012-12-29 08:00:00

 If you can, two things to check:

 1) In your 'working' version, if you fix the ISCSI_THROTTLING_ENABLED
 #define in src/sys/dev/iscsi/iscsi_globals.h to make it look like this:

  #define ISCSI_THROTTLING_ENABLED	1

 do things continue to work?  (i.e. turn on the throttling code in the
 'working' code..)

 2) If you try -current using the following:

  #define ISCSI_THROTTLING_ENABLED	0

 in src/sys/dev/iscsi/iscsi_globals.h do things work? (i.e. disable the
 throttling code)

 It looks like the throttling bits weren't enabled (because of the typo
 in the #define ) originally.  They are now enabled, but with slightly
 different code.  The above checks might help narrow things down...

 Later...

 Greg Oster

From: =?UTF-8?Q?Zafer_Aydo=C4=9Fan?= <zafer@aydogan.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/49072: in-kernel iscsi initiator not working
Date: Wed, 6 Aug 2014 14:58:21 +0200

 Hi Greg,

 I can confirm that it does continue to work with the typo fixed
 ISCSI_THROTTLING_ENABLED 1

 Zafer

 2014-08-05 17:00 GMT+02:00 Greg Oster <oster@cs.usask.ca>:
 > The following reply was made to PR kern/49072; it has been noted by GNATS.
 >
 > From: Greg Oster <oster@cs.usask.ca>
 > To: gnats-bugs@NetBSD.org
 > Cc:
 > Subject: Re: kern/49072: in-kernel iscsi initiator not working
 > Date: Tue, 5 Aug 2014 08:57:39 -0600
 >
 >  On Mon,  4 Aug 2014 23:30:00 +0000 (UTC)
 >  zafer@aydogan.de wrote:
 >
 >  > >Number:         49072
 >  > >Category:       kern
 >  > >Synopsis:       in-kernel iscsi initiator not working
 >  > >Confidential:   no
 >  > >Severity:       serious
 >  > >Priority:       high
 >  > >Responsible:    kern-bug-people
 >  > >State:          open
 >  > >Class:          sw-bug
 >  > >Submitter-Id:   net
 >  > >Arrival-Date:   Mon Aug 04 23:30:00 +0000 2014
 >  > >Originator:     Zafer Aydogan
 >  > >Release:        6.99.16 onward
 >  > >Organization:
 >  > TNF
 >  > >Environment:
 >  > 6.99.49 amd64
 >  > >Description:
 >  > The in-kernel iscsi initiator looks like to be broken since 6.99.16.
 >  > Symptoms are scsi SENSE KEY errors with "Overlapped Commands
 >  > Attempted" when you write data to disk.
 >  > If you continue to write data to the iscsi disk, the write process
 >  > will wedge in tstile. In this state a reboot will not succeed. You
 >  > need to power-cycle. Sometimes writing to the disk will panic the
 >  > system, especially overwriting the same file frequently.
 >  >
 >  > I could trace the error back to two commits from mlelstv@ on
 >  > 2012-12-29:
 >  > http://freshbsd.org/commit/netbsd/0407d4dcf2917f3861618fc1cbc54e6bd5b583b3
 >  > and
 >  > http://freshbsd.org/commit/netbsd/77a0b440f42ba059ed279f043194a4b589203054
 >  > If you revert both or use a version prior to that date, iscsi works
 >  > as expected without any issues.
 >  >
 >  > Screenshots:
 >  > http://www.aydogan.de/wtfs.png
 >  > http://www.aydogan.de/iscsi_panic.png
 >  >
 >  > >How-To-Repeat:
 >  > mount an iscsi share and
 >  > newfs -O1 on a partition
 >  >
 >  > newfs will exit with a wtfs write error
 >  >
 >  > Attention: newfs -O2 will show no errors.
 >  >
 >  >
 >  > >Fix:
 >  > A workaround is to revert to the version before 2012-12-29 08:00:00
 >
 >  If you can, two things to check:
 >
 >  1) In your 'working' version, if you fix the ISCSI_THROTTLING_ENABLED
 >  #define in src/sys/dev/iscsi/iscsi_globals.h to make it look like this:
 >
 >   #define ISCSI_THROTTLING_ENABLED      1
 >
 >  do things continue to work?  (i.e. turn on the throttling code in the
 >  'working' code..)
 >
 >  2) If you try -current using the following:
 >
 >   #define ISCSI_THROTTLING_ENABLED      0
 >
 >  in src/sys/dev/iscsi/iscsi_globals.h do things work? (i.e. disable the
 >  throttling code)
 >
 >  It looks like the throttling bits weren't enabled (because of the typo
 >  in the #define ) originally.  They are now enabled, but with slightly
 >  different code.  The above checks might help narrow things down...
 >
 >  Later...
 >
 >  Greg Oster
 >

Home	PR Database Search