NetBSD Problem Report #1718

From gnats  Thu Nov  2 21:13:02 1995
Received: from sun-lamp.pc.cs.cmu.edu by pain.lcs.mit.edu (8.6.9/8.6.9) with ESMTP id UAA14425 for <gnats-bugs@pain.lcs.mit.edu>; Thu, 2 Nov 1995 20:55:44 -0500
Message-Id: <199511030155.UAA15980@sun-lamp.pc.cs.cmu.edu>
Date: Thu, 2 Nov 1995 20:55:42 -0500
From: "Chris G. Demetriou" <cgd@netbsd.org>
Reply-To: cgd@netbsd.org
To: gnats-bugs@netbsd.org
Subject: MAXBSIZE too low.
X-Send-Pr-Version: 3.2

>Number:         1718
>Category:       kern
>Synopsis:       MAXBSIZE changed from the Lite value, & performance suffered
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    kern-bug-people
>State:          closed
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Thu Nov 02 21:20:01 +0000 1995
>Closed-Date:    Mon Dec 02 07:30:52 +0000 1996
>Last-Modified:  Mon Dec 02 07:31:04 +0000 1996
>Originator:     Chris G. Demetriou
>Release:        NetBSD-current 951102
>Organization:
Kernel Hackers 'r' Us
>Environment:
Measured on NetBSD/Alpha, but similar problem on all ports.

>Description:
	MAXBSIZE was #defined to be MAXPHYS in 4.4BSD-Lite (and Lite2).
	It was changed to be 16k in NetBSD.  (If my memory serves me,
	the reason given at the time was to save kernel page table space
	on the i386.

	This drastically reduces the effectiveness of clustering (esp.
	write clustering), because the number of I/Os which can be combined
	is reduced.  (Coalsced I/Os must still fit into a buffer with
	maximum size of MAXBSIZE.)

	For instance, on a NetBSD/Alpha system with a 53c810 SCSI
	controller, doing I/O to an rz25 (not too fast, but not horrible)
	disk, with a standard file system (i.e. no special options to newfs,
	8k block size, maxcontig of 8, etc.), i see the following:

	MAXBSIZE=16k:

	# time dd if=/dev/zero of=/mnt/bar bs=1024k count=86
	86+0 records in
	86+0 records out
	90177536 bytes transferred in 105 secs (858833 bytes/sec)
	0.0u 9.1s 1:44.95 8.6% 0+0k 3+5548io 0pf+0w

	MAXBSIZE=MAXPHYS=64k:

	# time dd if=/dev/zero of=/mnt/bar bs=1024k count=86
	86+0 records in
	86+0 records out
	90177536 bytes transferred in 52 secs (1734183 bytes/sec)
	0.0u 7.5s 0:52.70 14.4% 0+0k 11+9684io 0pf+0w

	that's a 8.2% reduction in CPU time, and 50% reduction
	in elapsed times.

	For simple large-read tests (e.g. dd with 1MB block size),
	CPU time was reduced by between 20% and 40%, but elapsed
	time remained the same.  (CPU time reduction is because
	number of I/Os processed by the SCSI code would have been
	reduced.  Elapsed time probably stayed the same because of
	effectiveness of block read-ahead and on-disk buffering.)

>How-To-Repeat:
	Time file system operations with a kernel with MAXBSIZE set to 16k
	(the default).

	The only thing i changed in my tests was the kernel that i booted,
	and the only difference there was that MAXBSIZE had changed.

	Obviously, the exact results you get will depend on your system
	configuration, and will be sensitive to driver quality and disk speed.
	In particular, systems which have a large per-I/O cost should see
	the most benefit from increasing MAXBSIZE.

>Fix:
	(1) Change the #define of MAXBSIZE in <sys/param.h> to be
		MAXPHYS, as it was in releases from Berkeley.

	(2) Don't be so quick to hack system constants which
		have wide-ranging performance implications.
>Release-Note:
>Audit-Trail:

From: John Dyson <dyson@freefall.freebsd.org>
To: cgd@NetBSD.ORG
Cc: gnats-bugs@NetBSD.ORG
Subject: Re: kern/1718: MAXBSIZE too low.
Date: Thu, 2 Nov 1995 21:54:42 -0800 (PST)

 > 	MAXBSIZE was #defined to be MAXPHYS in 4.4BSD-Lite (and Lite2).
 > 	It was changed to be 16k in NetBSD.  (If my memory serves me,
 > 	the reason given at the time was to save kernel page table space
 > 	on the i386.
 > 
 The problem is that filesystem buffers are also used for I/O clustering.  The
 original 4.4Lite clustering scheme depends on this.  That is the major reason
 that the scheme was changed on FreeBSD.  (Also the scheme that moved the pages
 from buffers in the cluster to the 'cluster buffer' (the first one in the
 cluster) was 'unpleasant' to look at: 'pagemove'.)  Note also that the problem
 with kva space on FreeBSD is a bit less critical because it is built with
 256MB of allocatable space to begin with, and kernel pts are created on the
 fly.  If the kernel pts were not, then the maximum that would be used by the
 system would have had to have been allocated at startup like on the original
 code.
 >
 > 	This drastically reduces the effectiveness of clustering (esp.
 > 	write clustering), because the number of I/Os which can be combined
 > 	is reduced.  (Coalsced I/Os must still fit into a buffer with
 > 	maximum size of MAXBSIZE.)
 > 
 We have a seperate pool of buffers (much fewer) that contain only 64K kva space,
 each and the buffer cache buffers are 16K.  Since the clustering code remaps the
 buffers anyway -- segregating the buffers into cache/(cluster-IO) makes
 sense.  Note that if an I/O isn't clustered, the I/O will be done directly
 to/from the buffer cache buffers.  On FreeBSD any I/O for a VREG file is
 done directly to the VM object for the file.  All I/O is done through those
 I/O buffers including both physical and leaf filesystem I/O.

 John
 dyson@freebsd.org
State-Changed-From-To: open->closed 
State-Changed-By: thorpej 
State-Changed-When: Sun Dec 1 23:30:52 PST 1996 
State-Changed-Why:  
Fixed. 
>Unformatted:

Home	PR Database Search