NetBSD Problem Report #6652

Received: (qmail 6330 invoked from network); 26 Dec 1998 05:24:57 -0000
Message-Id: <199812260523.AAA00733@horowitz.ne.mediaone.net>
Date: Sat, 26 Dec 1998 00:23:33 -0500 (EST)
From: marc@netbsd.org
Reply-To: marc@netbsd.org
To: gnats-bugs@gnats.netbsd.org
Subject: swapping to a sparse file fails unpredictably.
X-Send-Pr-Version: 3.95

>Number:         6652
>Category:       kern
>Synopsis:       swapping to a sparse file fails unpredictably.
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Dec 25 21:35:01 +0000 1998
>Closed-Date:    
>Last-Modified:  
>Originator:     Marc Horowitz
>Release:        24 Dec 1998
>Organization:

>Environment:

System: NetBSD horowitz 1.3I NetBSD 1.3I (MARC) #14: Wed Dec 2 02:35:57 EST 1998 marc@horowitz:/u1/netbsd/src/sys/arch/i386/compile/MARC i386


>Description:

If the system tries to swap to a sparse file, it becomes highly
unstable with no visible log or warning.  In my situation, I saw three
failure modes:

1) The system would spontaneously reset.

2) The system would wedge.

3) init's memory would get corrupted.  This would have weird secondary
effects, such as tty's getting reset, init crashing with SIGILL or
SIGSEGV, or init spinning.  The one time I got a good stack trace on
an init spin, it was in
disaster->strsignal->__strsignal->_catopen->getenv->findenv.  (As a
second possible bug, it seems poor for init do be doing this much
extra work when something catastrophic enough happens to disaster to
get called.)

>How-To-Repeat:

	create a sparse file
	swapctl -a /sparsefile
	do something which tries to page into the sparse file

>Fix:

I'm going to commit a change to uvm_swap.c to panic if a write to a
sparse file occurrs.  It would be better to handle this case more
elegantly, but I'm not familiar enough with the code to do that.  A
useful panic() message is better than arbitrary system instability,
IMHO.
>Release-Note:
>Audit-Trail:

From: Jason Thorpe <thorpej@nas.nasa.gov>
To: marc@netbsd.org
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/6652: swapping to a sparse file fails unpredictably. 
Date: Mon, 04 Jan 1999 17:11:51 -0800

 On Sat, 26 Dec 1998 00:23:33 -0500 (EST) 
  marc@netbsd.org wrote:

  > >How-To-Repeat:
  > 
  > 	create a sparse file
  > 	swapctl -a /sparsefile
  > 	do something which tries to page into the sparse file

 I would say that any attempt to swapon to a sparse file should fail
 with EINVAL.  Same goes for the vnd code.

 Jason R. Thorpe                                       thorpej@nas.nasa.gov
 NASA Ames Research Center                            Home: +1 408 866 1912
 NAS: M/S 258-5                                       Work: +1 650 604 0935
 Moffett Field, CA 94035                             Pager: +1 650 940 5942

From: Marc Horowitz <marc@mit.edu>
To: Jason Thorpe <thorpej@nas.nasa.gov>
Cc: gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/6652: swapping to a sparse file fails unpredictably.
Date: 05 Jan 1999 16:53:13 -0500

 Jason Thorpe <thorpej@nas.nasa.gov> writes:

 >> I would say that any attempt to swapon to a sparse file should fail
 >> with EINVAL.  Same goes for the vnd code.

 Well, eventually, growing the file would be nice.  But this is a
 reasonable start.

 We discussed this on ICB a bit.  The only issue here is that it would
 be nice if there was a non-O(n) way to determine if a file is sparse.
 I'm not sure how to do this (or if it even makes sense) in a
 filesystem generic way.  I think this might be possible using the
 indirect block macros, but I'm not ffs-clueful enough to know the
 details.  If you could give me some clue, or a pointer to it, I'll try
 to fix it.

 		Marc

From: Daniel Carosone <dan@geek.com.au>
To: Marc Horowitz <marc@mit.edu>
Cc: Jason Thorpe <thorpej@nas.nasa.gov>, gnats-bugs@gnats.netbsd.org,
        netbsd-bugs@netbsd.org, dan@geek.com.au
Subject: Re: kern/6652: swapping to a sparse file fails unpredictably. 
Date: Wed, 06 Jan 1999 10:50:22 +1100

 > We discussed this on ICB a bit.  The only issue here is that it would
 > be nice if there was a non-O(n) way to determine if a file is sparse.
 > I'm not sure how to do this (or if it even makes sense) in a
 > filesystem generic way.  I think this might be possible using the
 > indirect block macros, but I'm not ffs-clueful enough to know the

 I'm not sure there is a problem here, though the "fs generic" issue
 might be a problem for some weird case i can't think of right now
 (in any case i can think of, the underlying filesystem doesn't
 support sparse files and the point is moot).

 "ls -ls" will let me determine if a file is sparse, by comparing
 the "length" and "number of blocks" columns.  This info comes from
 the stat structure and is O(1)..

 --
 Dan.

From: Marc Horowitz <marc@mit.edu>
To: Daniel Carosone <dan@geek.com.au>
Cc: Jason Thorpe <thorpej@nas.nasa.gov>, gnats-bugs@gnats.netbsd.org,
        netbsd-bugs@netbsd.org
Subject: Re: kern/6652: swapping to a sparse file fails unpredictably.
Date: 05 Jan 1999 19:06:24 -0500

 Daniel Carosone <dan@geek.com.au> writes:

 >> "ls -ls" will let me determine if a file is sparse, by comparing
 >> the "length" and "number of blocks" columns.  This info comes from
 >> the stat structure and is O(1)..

 And fails if the file uses indirect blocks.  Like I said, I'm pretty
 sure there's a way to do this, but I don't know what it is.

 		Marc

From: Robert Elz <kre@munnari.OZ.AU>
To: Marc Horowitz <marc@mit.edu>
Cc: Daniel Carosone <dan@geek.com.au>, Jason Thorpe <thorpej@nas.nasa.gov>,
        gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/6652: swapping to a sparse file fails unpredictably. 
Date: Wed, 06 Jan 1999 19:57:11 +1100

     Date:        05 Jan 1999 19:06:24 -0500
     From:        Marc Horowitz <marc@mit.edu>
     Message-ID:  <t53lnjh1czj.fsf@rover.cygnus.com>

   | Daniel Carosone <dan@geek.com.au> writes:
   | >> "ls -ls" will let me determine if a file is sparse

   | And fails if the file uses indirect blocks.

 Only if the algorithm used is too simplistic like
 	 (size + blocksize - 1) / blocksize

 You can always determine exactly how many blocks it takes to cover an entire
 file of any given size (on ffs) - if the number of blocks the inode claims is
 equal to that, then the file is not sparse, if the number of blocks is less
 then the file is sparse, if it is greater, the filesystem is broken...

 kre


From: Marc Horowitz <marc@mit.edu>
To: Robert Elz <kre@munnari.OZ.AU>
Cc: Daniel Carosone <dan@geek.com.au>, Jason Thorpe <thorpej@nas.nasa.gov>,
        gnats-bugs@gnats.netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/6652: swapping to a sparse file fails unpredictably.
Date: 06 Jan 1999 15:06:45 -0500

 Robert Elz <kre@munnari.OZ.AU> writes:

 >> You can always determine exactly how many blocks it takes to cover
 >> an entire file of any given size (on ffs) - if the number of blocks
 >> the inode claims is equal to that, then the file is not sparse, if
 >> the number of blocks is less then the file is sparse, if it is
 >> greater, the filesystem is broken...

 You're about the fifth person to tell me that it's possible.  Can
 someone please tell me *how* so I can fix the code?

 		Marc
>Unformatted:

Home	PR Database Search