NetBSD Problem Report #48220

From www@NetBSD.org  Thu Sep 19 18:05:49 2013
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 47F2571415
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 19 Sep 2013 18:05:49 +0000 (UTC)
Message-Id: <20130919180547.F416B72533@mollari.NetBSD.org>
Date: Thu, 19 Sep 2013 18:05:47 +0000 (UTC)
From: toby.karyadi@gmail.com
Reply-To: toby.karyadi@gmail.com
To: gnats-bugs@NetBSD.org
Subject: xentools-4.1: xl has problems with file/vnd based disks
X-Send-Pr-Version: www-1.0

>Number:         48220
>Category:       port-xen
>Synopsis:       xentools-4.1: xl has problems with file/vnd based disks
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    port-xen-maintainer
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Sep 19 18:10:00 +0000 2013
>Closed-Date:    Sun Jun 24 20:37:25 +0000 2018
>Last-Modified:  Sun Jun 24 20:37:25 +0000 2018
>Originator:     Toby Karyadi
>Release:        NetBSD 6.1
>Organization:
>Environment:
NetBSD non.ofyour.biz 6.1 NetBSD 6.1 (XEN3_DOM0) amd64
>Description:
xl in pkgsrc/sysutils/xentools41, the xen domain control utility to replace xm, has the following problems:

1) Does not properly support file based (vnd based) disk configuration, e.g.:
disk = [ 'file:/vol/1/vhosts/host01/host01.disk0.img,hda,w', 
    'file:/vol/1/vhosts/host01/host01.disk1.img,hdb,w' ]
The domain would just hang during startup.

2) Once problem 1) is fixed, xl also has another bug that caused xenbackendd to not call the 'block' script to properly undo the vnd device. 

This issue was described originally here:
http://mail-index.netbsd.org/port-xen/2012/04/11/msg007216.html
http://mail-index.netbsd.org/port-xen/2012/05/29/msg007252.html

>How-To-Repeat:
Use xl that comes with xentools41 with a domain that uses one or more file-based disk.
>Fix:
I'll attach the patches for xentools41 in a following update to the bug report (it's just easier to do so through email). For those who are not too familiar with pkgsrc, put these files in the patches directory of pkgsrc/sysutils/xentools41 and do 'make makepatchsum' and rebuild. These patches apply cleanly to xen 4.1.5 and xen 4.1.6.1 sources.

>Release-Note:

>Audit-Trail:
From: Toby Karyadi <toby.karyadi@gmail.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based
 disks
Date: Thu, 19 Sep 2013 14:46:18 -0400

 This is a multi-part message in MIME format.
 --------------090406040207000003080209
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed
 Content-Transfer-Encoding: 7bit

 Here are the patches.



 --------------090406040207000003080209
 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0";
  name="patch-libxl_z0-vbd_dtor_fix"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
  filename="patch-libxl_z0-vbd_dtor_fix"

 This patch corrects libxl__device_destroy() to also handle the case when the
 device is already in the XenbusStateClosing state, by adding watch on this
 device and letting the caller, libxl__devices_destroy() wait until the device 
 closes. This would allow xenbackendd to properly call the 'block' script to 
 undo the vnd device.
 --- libxl/libxl_device.c.orig	2013-04-23 12:44:20.000000000 -0400
 +++ libxl/libxl_device.c	2013-08-07 23:02:09.000000000 -0400
 @@ -249,7 +249,18 @@ int libxl__device_destroy(libxl_ctx *ctx

      if (!state)
          goto out;
 -    if (atoi(state) != 4) {
 +    switch (atoi(state)) {
 +    case 4: /* XenbusStateConnected */
 +        break;
 +    case 5: /* XenbusStateClosing */
 +        /* When the device is already 'closing', skip the code below that 
 +         * begins at the retry_transaction label that changes the state to
 +         * the 'closing' state. If it is not a forced destroy, a watch needs
 +         * to be added so that the current device will be waited upon by 
 +         * libxl__devices_destroy() so that proper cleanup, e.g. by 
 +         * xenbackendd, can occur. */
 +        goto wait;
 +    default: /* Presumed 6, XenbusStateClosed */
          libxl__device_destroy_tapdisk(&gc, be_path);
          xs_rm(ctx->xsh, XBT_NULL, be_path);
          goto out;
 @@ -267,12 +278,14 @@ retry_transaction:
              goto out;
          }
      }
 +
 +wait:
      if (!force) {
          xs_watch(ctx->xsh, state_path, be_path);
          rc = 1;
      } else {
 -		xs_rm(ctx->xsh, XBT_NULL, be_path);
 -	}
 +        xs_rm(ctx->xsh, XBT_NULL, be_path);
 +    }
  out:
      libxl__free_all(&gc);
      return rc;

 --------------090406040207000003080209
 Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0";
  name="patch-libxl_z1-vnd_support"
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
  filename="patch-libxl_z1-vnd_support"

 This patch adds support to 'xl' so that it will properly setup a vnd device
 for each file based disk defined in the domU configuration file. This would 
 provide the same behavior in dealing with file based disk as in 'xm' for 
 NetBSD. This patch won't be needed for xentools >= 4.2, but file based disk 
 needs to be described as a 'phy' device with a 'script' argument.

 Without this patch, xl delegates dealing with file based disk to blktap, and 
 since it does not exist on NetBSD, it then tries to use qemu-dm, which also 
 does not work for file based disk on NetBSD, and the domain would just hang. 
 --- libxl/libxl.h.orig	2011-10-20 13:05:42.000000000 -0400
 +++ libxl/libxl.h	2012-05-08 02:17:57.000000000 -0400
 @@ -185,6 +185,9 @@
      DISK_BACKEND_PHY,
      DISK_BACKEND_TAP,
      DISK_BACKEND_QDISK,
 +#if defined(__NetBSD__)
 +    DISK_BACKEND_VND,
 +#endif
  } libxl_disk_backend;

  typedef enum {
 --- libxl/libxl.c.orig	2013-04-23 12:44:20.000000000 -0400
 +++ libxl/libxl.c	2013-08-08 14:05:30.000000000 -0400
 @@ -955,7 +955,21 @@ int libxl_device_disk_add(libxl_ctx *ctx
      device.kind = DEVICE_VBD;

      switch (disk->backend) {
 -        case DISK_BACKEND_PHY: 
 +#if defined(__NetBSD__)
 +        case DISK_BACKEND_VND:
 +            /* Use bogus device; block script must redefine */
 +            major = 2; minor = 12; /* /dev/zero char device */
 +
 +            flexarray_append(back, "physical-device");
 +            flexarray_append(back, libxl__sprintf(&gc, "%x:%x", major, minor));
 +
 +            flexarray_append(back, "params");
 +            flexarray_append(back, disk->pdev_path);
 +
 +            device.backend_kind = DEVICE_VBD;
 +            break;
 +#endif
 +        case DISK_BACKEND_PHY:
              libxl__device_physdisk_major_minor(disk->pdev_path, &major, &minor);
              flexarray_append(back, "physical-device");
              flexarray_append(back, libxl__sprintf(&gc, "%x:%x", major, minor));
 @@ -1059,8 +1073,19 @@ int libxl_device_disk_del(libxl_ctx *ctx
      devid = libxl__device_disk_dev_number(disk->vdev);
      device.backend_domid    = disk->backend_domid;
      device.backend_devid    = devid;
 +#if defined(__NetBSD__)
 +    switch (disk->backend) {
 +    case DISK_BACKEND_PHY:
 +    case DISK_BACKEND_VND:
 +        device.backend_kind = DEVICE_VBD;
 +        break;
 +    default:
 +        device.backend_kind = DEVICE_TAP;
 +    }
 +#else
      device.backend_kind     = 
          (disk->backend == DISK_BACKEND_PHY) ? DEVICE_VBD : DEVICE_TAP;
 +#endif
      device.domid            = disk->domid;
      device.devid            = devid;
      device.kind             = DEVICE_VBD;
 @@ -1078,6 +1103,9 @@ char * libxl_device_disk_local_attach(li
      char *ret = NULL;

      switch (disk->backend) {
 +#if defined(__NetBSD__)
 +	case DISK_BACKEND_VND:
 +#endif
          case DISK_BACKEND_PHY: 
              if (disk->format != DISK_FORMAT_RAW) {
                  LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "physical block device must"
 --- libxl/libxl_device.c.orig	2012-05-28 00:55:38.000000000 -0400
 +++ libxl/libxl_device.c	2012-05-27 23:55:47.000000000 -0400
 @@ -139,6 +139,9 @@
          case DISK_BACKEND_QDISK: return "qdisk";
          case DISK_BACKEND_TAP: return "tap";
          case DISK_BACKEND_PHY: return "phy";
 +#if defined(__NetBSD__)
 +        case DISK_BACKEND_VND: return "file";
 +#endif
          default: return NULL;
      }
  }
 --- libxl/libxl_dm.c.orig	2011-10-20 13:05:42.000000000 -0400
 +++ libxl/libxl_dm.c	2012-05-08 02:17:01.000000000 -0400
 @@ -845,6 +845,9 @@
                  ret = 1;
                  goto out;

 +#if defined(__NetBSD__)
 +            case DISK_BACKEND_VND:
 +#endif
              case DISK_BACKEND_PHY:
              case DISK_BACKEND_UNKNOWN:
                  break;
 --- libxl/xl_cmdimpl.c.orig	2011-10-20 13:05:43.000000000 -0400
 +++ libxl/xl_cmdimpl.c	2012-05-27 14:39:08.000000000 -0400
 @@ -484,7 +484,11 @@
                  }else if ( !strcmp(tok, "file") ) {
                      state = DSTATE_PHYSPATH;
                      disk->format = DISK_FORMAT_RAW;
 +#if defined(__NetBSD__)
 +                    disk->backend = DISK_BACKEND_VND;
 +#else
                      disk->backend = DISK_BACKEND_TAP;
 +#endif
                  }else if ((!strcmp(tok, "tap")) ||
                            (!strcmp(tok, "tap2"))) {
                      state = DSTATE_TAP;
 @@ -4432,7 +4436,11 @@
      if (!strcmp(tok, "phy")) {
          disk.backend = DISK_BACKEND_PHY;
      } else if (!strcmp(tok, "file")) {
 +#if defined(__NetBSD__)
 +        disk.backend = DISK_BACKEND_VND;
 +#else
          disk.backend = DISK_BACKEND_TAP;
 +#endif
      } else if (!strcmp(tok, "tap")) {
          tok = strtok(NULL, ":");
          if (!strcmp(tok, "aio")) {

 --------------090406040207000003080209--

From: John Nemeth <jnemeth@cue.bc.ca>
To: gnats-bugs@netbsd.org, port-xen-maintainer@netbsd.org,
        gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based disks
Date: Thu, 19 Sep 2013 12:28:25 -0700

 On Sep 19,  6:10pm, toby.karyadi@gmail.com wrote:
 }
 } >Number:         48220
 } >Category:       port-xen
 } >Synopsis:       xentools-4.1: xl has problems with file/vnd based disks
 } >Arrival-Date:   Thu Sep 19 18:10:00 +0000 2013
 } >Originator:     Toby Karyadi
 } >Environment:
 } NetBSD non.ofyour.biz 6.1 NetBSD 6.1 (XEN3_DOM0) amd64
 } >Description:
 } xl in pkgsrc/sysutils/xentools41, the xen domain control utility
 } to replace xm, has the following problems:

      xl in xentools41 is known not to work properly.  If you want to
 use xl, then you should be using xentools42 (and xenkernel42).

 } 2) Once problem 1) is fixed, xl also has another bug that caused
 } xenbackendd to not call the 'block' script to properly undo the
 } vnd device.

      If you are using xl then you should not be running xenbackendd.
 That is only for use with xm.  Using the combination of xl and
 xenbackendd can mess things up.  This would be another reason to
 use xentools42 since the startup scripts have been redone to get
 this correct.

 }-- End of excerpt from toby.karyadi@gmail.com

From: Toby Karyadi <toby.karyadi@gmail.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based
 disks
Date: Thu, 19 Sep 2013 16:37:46 -0400

 On 9/19/13 3:55 PM, John Nemeth wrote:
 > The following reply was made to PR port-xen/48220; it has been noted by GNATS.
 >
 > From: John Nemeth <jnemeth@cue.bc.ca>
 > To: gnats-bugs@netbsd.org, port-xen-maintainer@netbsd.org,
 >          gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
 > Cc:
 > Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based disks
 > Date: Thu, 19 Sep 2013 12:28:25 -0700
 >
 >   On Sep 19,  6:10pm, toby.karyadi@gmail.com wrote:
 >   }
 >   } >Number:         48220
 >   } >Category:       port-xen
 >   } >Synopsis:       xentools-4.1: xl has problems with file/vnd based disks
 >   } >Arrival-Date:   Thu Sep 19 18:10:00 +0000 2013
 >   } >Originator:     Toby Karyadi
 >   } >Environment:
 >   } NetBSD non.ofyour.biz 6.1 NetBSD 6.1 (XEN3_DOM0) amd64
 >   } >Description:
 >   } xl in pkgsrc/sysutils/xentools41, the xen domain control utility
 >   } to replace xm, has the following problems:
 >   
 >        xl in xentools41 is known not to work properly.  If you want to
 >   use xl, then you should be using xentools42 (and xenkernel42).
 Actually I'm trying to submit patches for xentools41 to fix the problems 
 with xl that I mentioned. I'm hoping that it would provide a smoother 
 migration from 3.3 to 4.*. I thought xentools42 doesn't support HVM domU 
 just yet, is that no longer true? Trying xentools42 is my next step.
 >   
 >   } 2) Once problem 1) is fixed, xl also has another bug that caused
 >   } xenbackendd to not call the 'block' script to properly undo the
 >   } vnd device.
 >   
 >        If you are using xl then you should not be running xenbackendd.
 >   That is only for use with xm.  Using the combination of xl and
 >   xenbackendd can mess things up.  This would be another reason to
 >   use xentools42 since the startup scripts have been redone to get
 >   this correct.
 >   
 If you look at rc.d/xencommons for 4.1, it actually does start 
 xenbackendd. On the other hand, xend should not be started. 
 Unfortunately xencommons and xend rc scripts uses the same rcvar 'xend', 
 so if you specify xend=yes in /etc/rc.conf you'll start both xencommons 
 and xend. The work around is to have xencommons and xend files under 
 /etc/rc.conf.d where the xencommons file should contain xend=yes where 
 the xend file should contain xend=no.

 In netsbd and xen 4.1, AFAIK, there is no way to invoke the block script 
 without the help of the xenbackendd. In 4.2, each disk spec can have a 
 block script which should be invoked by xl itself, and hence negates the 
 need for xenbackendd. For linux, there is no need of xenbackendd even in 
 4.1 because xl uses blktap, which netbsd doesn't have.

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: port-xen-maintainer@NetBSD.org, gnats-admin@NetBSD.org,
        netbsd-bugs@NetBSD.org, toby.karyadi@gmail.com
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd
 based disks
Date: Thu, 19 Sep 2013 23:27:46 +0200

 On Thu, Sep 19, 2013 at 08:40:01PM +0000, Toby Karyadi wrote:
 >  >   use xl, then you should be using xentools42 (and xenkernel42).
 >  Actually I'm trying to submit patches for xentools41 to fix the problems 
 >  with xl that I mentioned. I'm hoping that it would provide a smoother 
 >  migration from 3.3 to 4.*. I thought xentools42 doesn't support HVM domU 
 >  just yet, is that no longer true? Trying xentools42 is my next step.

 HVM domU works with xen 4.2

 >  >   
 >  >   } 2) Once problem 1) is fixed, xl also has another bug that caused
 >  >   } xenbackendd to not call the 'block' script to properly undo the
 >  >   } vnd device.
 >  >   
 >  >        If you are using xl then you should not be running xenbackendd.
 >  >   That is only for use with xm.  Using the combination of xl and
 >  >   xenbackendd can mess things up.  This would be another reason to
 >  >   use xentools42 since the startup scripts have been redone to get
 >  >   this correct.
 >  >   
 >  If you look at rc.d/xencommons for 4.1, it actually does start 
 >  xenbackendd. On the other hand, xend should not be started. 
 >  Unfortunately xencommons and xend rc scripts uses the same rcvar 'xend', 
 >  so if you specify xend=yes in /etc/rc.conf you'll start both xencommons 
 >  and xend. The work around is to have xencommons and xend files under 
 >  /etc/rc.conf.d where the xencommons file should contain xend=yes where 
 >  the xend file should contain xend=no.

 At this time, xl is not recommended on xen 4.1, that's why xend is still
 started. xl is known to have various problems and limitations
 (i.e. features present in xm which have not been intgrated in xl,
 like shutdown -a)

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

State-Changed-From-To: open->closed
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Sun, 24 Jun 2018 20:37:25 +0000
State-Changed-Why:
Xen 4.1 was EOLed by upstream.


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.