NetBSD Problem Report #47899

From www@NetBSD.org  Wed Jun  5 20:28:37 2013
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 1599F71B3C
	for <gnats-bugs@gnats.NetBSD.org>; Wed,  5 Jun 2013 20:28:37 +0000 (UTC)
Message-Id: <20130605202835.C81B371B42@mollari.NetBSD.org>
Date: Wed,  5 Jun 2013 20:28:35 +0000 (UTC)
From: 7heo@7heo.tk
Reply-To: 7heo@7heo.tk
To: gnats-bugs@NetBSD.org
Subject: xen block script does not create vnd devices when needed.
X-Send-Pr-Version: www-1.0

>Number:         47899
>Category:       port-xen
>Synopsis:       xen block script does not create vnd devices when needed.
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-xen-maintainer
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Jun 05 20:30:00 +0000 2013
>Closed-Date:    Wed Jan 09 19:08:18 +0000 2019
>Last-Modified:  Wed Jan 09 19:08:18 +0000 2019
>Originator:     7heo
>Release:        6.0.1
>Organization:
>Environment:
NetBSD host.domain.com 6.0.1 NetBSD 6.0.1 (XEN3PAE_DOM0) i386
>Description:
By default, NetBSD has 4 (four) vnd devices, /dev/vnd0, /dev/vnd1, /dev/vnd2 and /dev/vnd3.

When one creates a virtual machine with a disk from a file, the block script uses vnconfig to "mount" the image on a vnd loopback device.

When the Xen toolstack (xm) needs to mount more than 4 images at the same time, it fails, instead of increasing the number of vnd devices to a suited quantity.

The error message looks like:

"Error: Device <nr> (vbd) could not be connected. Hotplug scripts not working."

and does not give any valuable input on the real reason behind the problem.
>How-To-Repeat:
Install NetBSD, install Xen, create a virtual machine with 5 drives.
>Fix:
$ cd /dev
$ ./MAKEDEV vnd4
$ ./MAKEDEV vnd...
$ xm start /path/to/domain.cfg

>Release-Note:

>Audit-Trail:
From: Pierre Pronchery <khorben@defora.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-xen/47899: xen block script does not create vnd devices
 when needed.
Date: Thu, 06 Jun 2013 04:00:34 +0200

 This is a multi-part message in MIME format.
 --------------040700010609060508010203
 Content-Type: text/plain; charset=ISO-8859-1
 Content-Transfer-Encoding: 7bit

 			Hi 7heo,

 On 05/06/2013 22:30, 7heo@7heo.tk wrote:
 > [...]
 > By default, NetBSD has 4 (four) vnd devices, /dev/vnd0, /dev/vnd1,
 > /dev/vnd2 and /dev/vnd3.
 > 
 > When one creates a virtual machine with a disk from a file, the
 > block script uses vnconfig to "mount" the image on a vnd loopback device.
 > 
 > When the Xen toolstack (xm) needs to mount more than 4 images at the
 > same time, it fails, instead of increasing the number of vnd devices to
 > a suited quantity.

 Good catch.

 Can you try the patch attached? It should attempt to create the missing
 nodes automatically.

 For some reason "vnconfig -l" seems to only ever lists the four first
 vnd devices, regardless of the existence of the corresponding device
 nodes, but that's a separate issue.

 HTH,
 -- 
 khorben

 --------------040700010609060508010203
 Content-Type: text/plain; charset=ISO-8859-15;
  name="patch-xentools_block_create_vnd.diff"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
  filename="patch-xentools_block_create_vnd.diff"

 --- /usr/pkg/etc/xen/scripts/block	2013-04-10 01:51:03.000000000 +0200
 +++ /usr/pkg/etc/xen/scripts/block.new	2013-06-06 03:53:49.000000000 +0200
 @@ -46,14 +46,15 @@
  	file)
  		# Store the list of available vnd(4) devices in
  		#``available_disks'', and mark them as ``free''.
 -		list=`ls -1 /dev/vnd[0-9]*d | sed "s,/dev/vnd,,;s,d,," | sort -n`
 -		for i in $list; do
 -			disk="vnd$i"
 +		i=0
 +		while [ -b /dev/vnd${i}d ]; do
 +			disk=vnd$i
  			available_disks="$available_disks $disk"
  			eval $disk=free
 +			i=$((i + 1))
  		done
  		# Mark the used vnd(4) devices as ``used''.
 -		for disk in `sysctl hw.disknames`; do
 +		for disk in $(sysctl hw.disknames); do
  			case $disk in
  			vnd[0-9]*) eval $disk=used ;;
  			esac
 @@ -68,6 +69,13 @@
  				break	
  			fi
  		done
 +		# Attempt to create a new device node
 +		if [ x$device = x ] && \
 +				(cd /dev && ./MAKEDEV vnd$i) && \
 +				vnconfig /dev/vnd${i}d $xparams >/dev/null; then
 +			device=/dev/vnd${i}d
 +			echo vnconfig /dev/vnd${i}d $xparams
 +		fi
  		if [ x$device = x ] ; then
  			error "no available vnd device"
  		fi

 --------------040700010609060508010203--

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: port-xen-maintainer@NetBSD.org, gnats-admin@NetBSD.org,
        netbsd-bugs@NetBSD.org, 7heo@7heo.tk
Subject: Re: port-xen/47899: xen block script does not create vnd devices
 when needed.
Date: Thu, 6 Jun 2013 09:44:28 +0200

 On Thu, Jun 06, 2013 at 02:05:00AM +0000, Pierre Pronchery wrote:
 > The following reply was made to PR port-xen/47899; it has been noted by GNATS.
 > 
 > From: Pierre Pronchery <khorben@defora.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: port-xen/47899: xen block script does not create vnd devices
 >  when needed.
 > Date: Thu, 06 Jun 2013 04:00:34 +0200
 > 
 >  This is a multi-part message in MIME format.
 >  --------------040700010609060508010203
 >  Content-Type: text/plain; charset=ISO-8859-1
 >  Content-Transfer-Encoding: 7bit
 >  
 >  			Hi 7heo,
 >  
 >  On 05/06/2013 22:30, 7heo@7heo.tk wrote:
 >  > [...]
 >  > By default, NetBSD has 4 (four) vnd devices, /dev/vnd0, /dev/vnd1,
 >  > /dev/vnd2 and /dev/vnd3.
 >  > 
 >  > When one creates a virtual machine with a disk from a file, the
 >  > block script uses vnconfig to "mount" the image on a vnd loopback device.
 >  > 
 >  > When the Xen toolstack (xm) needs to mount more than 4 images at the
 >  > same time, it fails, instead of increasing the number of vnd devices to
 >  > a suited quantity.
 >  
 >  Good catch.
 >  
 >  Can you try the patch attached? It should attempt to create the missing
 >  nodes automatically.

 I'm not sure it's the right thing to do. If things go wrong (e.g.
 a domU being repeteadly restarted without the previous one being
 properly destroyed, as with on_crash = rename-restart)
 you can fill up /dev, and probably more bad things will happen.

 I think it's up to the admin to make sure he has enough vnd devices
 created.

 The real problem may be that 'xm create' doens't report the real
 reason for the failure; the real reason is in xenbackendd.log.
 I'm not sure if this can easily be fixed.

 >  
 >  For some reason "vnconfig -l" seems to only ever lists the four first
 >  vnd devices, regardless of the existence of the corresponding device
 >  nodes, but that's a separate issue.

 vnconfig -l works fine for me:
 ragga:/domains#vnconfig -l
 vnd0: not in use
 vnd1: /domains (/dev/raid1f) inode 4
 vnd2: /domains (/dev/raid1f) inode 11
 vnd3: /domains (/dev/raid1f) inode 12
 vnd4: not in use
 vnd5: not in use
 vnd6: not in use
 [...]
 vnd29: not in use
 vnd30: not in use
 vnd31: not in use

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: port-xen/47899: xen block script does not create vnd devices when needed.
Date: Thu, 6 Jun 2013 12:02:35 +0000 (UTC)

 bouyer@antioche.eu.org (Manuel Bouyer) writes:

 >vnconfig -l works fine for me:
 >ragga:/domains#vnconfig -l
 >vnd0: not in use
 >vnd1: /domains (/dev/raid1f) inode 4
 >vnd2: /domains (/dev/raid1f) inode 11
 >vnd3: /domains (/dev/raid1f) inode 12
 >vnd4: not in use
 >vnd5: not in use
 >vnd6: not in use
 >[...]
 >vnd29: not in use
 >vnd30: not in use
 >vnd31: not in use

 vnconfig will clone the driver instance when necessary, but
 never remove it. The first 4 instances are precreated by
 the driver to be compatible with earlier kernels where
 the driver wasn't cloning.

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: port-xen-maintainer@NetBSD.org, gnats-admin@NetBSD.org,
        netbsd-bugs@NetBSD.org, 7heo@7heo.tk
Subject: Re: port-xen/47899: xen block script does not create vnd devices
 when needed.
Date: Thu, 6 Jun 2013 15:26:16 +0200

 On Thu, Jun 06, 2013 at 12:05:01PM +0000, Michael van Elst wrote:
 >  >vnconfig -l works fine for me:
 >  >ragga:/domains#vnconfig -l
 >  >vnd0: not in use
 >  >vnd1: /domains (/dev/raid1f) inode 4
 >  >vnd2: /domains (/dev/raid1f) inode 11
 >  >vnd3: /domains (/dev/raid1f) inode 12
 >  >vnd4: not in use
 >  >vnd5: not in use
 >  >vnd6: not in use
 >  >[...]
 >  >vnd29: not in use
 >  >vnd30: not in use
 >  >vnd31: not in use
 >  
 >  vnconfig will clone the driver instance when necessary, but
 >  never remove it. The first 4 instances are precreated by
 >  the driver to be compatible with earlier kernels where
 >  the driver wasn't cloning.

 but AFAIK vnconfig doesn't know how much instances are available in
 the kernel. It only checks if existing /dev/vnd* nodes are busy or
 not: it does a VNDIOCGET for each vnu_unit against /dev/vnd0d
 (up do cd_ndevs if I read the code properly).
 I guess cd_ndevs is increased dynamically when devices are used ...

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, port-xen-maintainer@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, 7heo@7heo.tk
Cc: 
Subject: Re: port-xen/47899: xen block script does not create vnd devices when needed.
Date: Thu, 6 Jun 2013 11:22:41 -0400

 On Jun 6,  1:30pm, bouyer@antioche.eu.org (Manuel Bouyer) wrote:
 -- Subject: Re: port-xen/47899: xen block script does not create vnd devices 

 | The following reply was made to PR port-xen/47899; it has been noted by GNATS.
 | 
 | From: Manuel Bouyer <bouyer@antioche.eu.org>
 | To: gnats-bugs@NetBSD.org
 | Cc: port-xen-maintainer@NetBSD.org, gnats-admin@NetBSD.org,
 |         netbsd-bugs@NetBSD.org, 7heo@7heo.tk
 | Subject: Re: port-xen/47899: xen block script does not create vnd devices
 |  when needed.
 | Date: Thu, 6 Jun 2013 15:26:16 +0200
 | 
 |  On Thu, Jun 06, 2013 at 12:05:01PM +0000, Michael van Elst wrote:
 |  >  >vnconfig -l works fine for me:
 |  >  >ragga:/domains#vnconfig -l
 |  >  >vnd0: not in use
 |  >  >vnd1: /domains (/dev/raid1f) inode 4
 |  >  >vnd2: /domains (/dev/raid1f) inode 11
 |  >  >vnd3: /domains (/dev/raid1f) inode 12
 |  >  >vnd4: not in use
 |  >  >vnd5: not in use
 |  >  >vnd6: not in use
 |  >  >[...]
 |  >  >vnd29: not in use
 |  >  >vnd30: not in use
 |  >  >vnd31: not in use
 |  >  
 |  >  vnconfig will clone the driver instance when necessary, but
 |  >  never remove it. The first 4 instances are precreated by
 |  >  the driver to be compatible with earlier kernels where
 |  >  the driver wasn't cloning.
 |  
 |  but AFAIK vnconfig doesn't know how much instances are available in
 |  the kernel. It only checks if existing /dev/vnd* nodes are busy or
 |  not: it does a VNDIOCGET for each vnu_unit against /dev/vnd0d
 |  (up do cd_ndevs if I read the code properly).
 |  I guess cd_ndevs is increased dynamically when devices are used ...

 No, this is a bug. Let me fix it. It should not be using cd_ndevs.

 christos

State-Changed-From-To: open->feedback
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Sun, 24 Jun 2018 20:48:51 +0000
State-Changed-Why:
The vnd(4) problem looks like something fixed by Christos on 2013-06-09,
in rev. 1.225 of sys/dev/vnd.c. Can you confirm if the problem still
happens with 8.0_RC1?


State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Wed, 09 Jan 2019 19:08:18 +0000
State-Changed-Why:
Feedback mail has been bouncing.


>Unformatted:

Home	PR Database Search