NetBSD Problem Report #48220
From www@NetBSD.org Thu Sep 19 18:05:49 2013
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 47F2571415
for <gnats-bugs@gnats.NetBSD.org>; Thu, 19 Sep 2013 18:05:49 +0000 (UTC)
Message-Id: <20130919180547.F416B72533@mollari.NetBSD.org>
Date: Thu, 19 Sep 2013 18:05:47 +0000 (UTC)
From: toby.karyadi@gmail.com
Reply-To: toby.karyadi@gmail.com
To: gnats-bugs@NetBSD.org
Subject: xentools-4.1: xl has problems with file/vnd based disks
X-Send-Pr-Version: www-1.0
>Number: 48220
>Category: port-xen
>Synopsis: xentools-4.1: xl has problems with file/vnd based disks
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: port-xen-maintainer
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Thu Sep 19 18:10:00 +0000 2013
>Closed-Date: Sun Jun 24 20:37:25 +0000 2018
>Last-Modified: Sun Jun 24 20:37:25 +0000 2018
>Originator: Toby Karyadi
>Release: NetBSD 6.1
>Organization:
>Environment:
NetBSD non.ofyour.biz 6.1 NetBSD 6.1 (XEN3_DOM0) amd64
>Description:
xl in pkgsrc/sysutils/xentools41, the xen domain control utility to replace xm, has the following problems:
1) Does not properly support file based (vnd based) disk configuration, e.g.:
disk = [ 'file:/vol/1/vhosts/host01/host01.disk0.img,hda,w',
'file:/vol/1/vhosts/host01/host01.disk1.img,hdb,w' ]
The domain would just hang during startup.
2) Once problem 1) is fixed, xl also has another bug that caused xenbackendd to not call the 'block' script to properly undo the vnd device.
This issue was described originally here:
http://mail-index.netbsd.org/port-xen/2012/04/11/msg007216.html
http://mail-index.netbsd.org/port-xen/2012/05/29/msg007252.html
>How-To-Repeat:
Use xl that comes with xentools41 with a domain that uses one or more file-based disk.
>Fix:
I'll attach the patches for xentools41 in a following update to the bug report (it's just easier to do so through email). For those who are not too familiar with pkgsrc, put these files in the patches directory of pkgsrc/sysutils/xentools41 and do 'make makepatchsum' and rebuild. These patches apply cleanly to xen 4.1.5 and xen 4.1.6.1 sources.
>Release-Note:
>Audit-Trail:
From: Toby Karyadi <toby.karyadi@gmail.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based
disks
Date: Thu, 19 Sep 2013 14:46:18 -0400
This is a multi-part message in MIME format.
--------------090406040207000003080209
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Here are the patches.
--------------090406040207000003080209
Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0";
name="patch-libxl_z0-vbd_dtor_fix"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="patch-libxl_z0-vbd_dtor_fix"
This patch corrects libxl__device_destroy() to also handle the case when the
device is already in the XenbusStateClosing state, by adding watch on this
device and letting the caller, libxl__devices_destroy() wait until the device
closes. This would allow xenbackendd to properly call the 'block' script to
undo the vnd device.
--- libxl/libxl_device.c.orig 2013-04-23 12:44:20.000000000 -0400
+++ libxl/libxl_device.c 2013-08-07 23:02:09.000000000 -0400
@@ -249,7 +249,18 @@ int libxl__device_destroy(libxl_ctx *ctx
if (!state)
goto out;
- if (atoi(state) != 4) {
+ switch (atoi(state)) {
+ case 4: /* XenbusStateConnected */
+ break;
+ case 5: /* XenbusStateClosing */
+ /* When the device is already 'closing', skip the code below that
+ * begins at the retry_transaction label that changes the state to
+ * the 'closing' state. If it is not a forced destroy, a watch needs
+ * to be added so that the current device will be waited upon by
+ * libxl__devices_destroy() so that proper cleanup, e.g. by
+ * xenbackendd, can occur. */
+ goto wait;
+ default: /* Presumed 6, XenbusStateClosed */
libxl__device_destroy_tapdisk(&gc, be_path);
xs_rm(ctx->xsh, XBT_NULL, be_path);
goto out;
@@ -267,12 +278,14 @@ retry_transaction:
goto out;
}
}
+
+wait:
if (!force) {
xs_watch(ctx->xsh, state_path, be_path);
rc = 1;
} else {
- xs_rm(ctx->xsh, XBT_NULL, be_path);
- }
+ xs_rm(ctx->xsh, XBT_NULL, be_path);
+ }
out:
libxl__free_all(&gc);
return rc;
--------------090406040207000003080209
Content-Type: text/plain; charset=UTF-8; x-mac-type="0"; x-mac-creator="0";
name="patch-libxl_z1-vnd_support"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="patch-libxl_z1-vnd_support"
This patch adds support to 'xl' so that it will properly setup a vnd device
for each file based disk defined in the domU configuration file. This would
provide the same behavior in dealing with file based disk as in 'xm' for
NetBSD. This patch won't be needed for xentools >= 4.2, but file based disk
needs to be described as a 'phy' device with a 'script' argument.
Without this patch, xl delegates dealing with file based disk to blktap, and
since it does not exist on NetBSD, it then tries to use qemu-dm, which also
does not work for file based disk on NetBSD, and the domain would just hang.
--- libxl/libxl.h.orig 2011-10-20 13:05:42.000000000 -0400
+++ libxl/libxl.h 2012-05-08 02:17:57.000000000 -0400
@@ -185,6 +185,9 @@
DISK_BACKEND_PHY,
DISK_BACKEND_TAP,
DISK_BACKEND_QDISK,
+#if defined(__NetBSD__)
+ DISK_BACKEND_VND,
+#endif
} libxl_disk_backend;
typedef enum {
--- libxl/libxl.c.orig 2013-04-23 12:44:20.000000000 -0400
+++ libxl/libxl.c 2013-08-08 14:05:30.000000000 -0400
@@ -955,7 +955,21 @@ int libxl_device_disk_add(libxl_ctx *ctx
device.kind = DEVICE_VBD;
switch (disk->backend) {
- case DISK_BACKEND_PHY:
+#if defined(__NetBSD__)
+ case DISK_BACKEND_VND:
+ /* Use bogus device; block script must redefine */
+ major = 2; minor = 12; /* /dev/zero char device */
+
+ flexarray_append(back, "physical-device");
+ flexarray_append(back, libxl__sprintf(&gc, "%x:%x", major, minor));
+
+ flexarray_append(back, "params");
+ flexarray_append(back, disk->pdev_path);
+
+ device.backend_kind = DEVICE_VBD;
+ break;
+#endif
+ case DISK_BACKEND_PHY:
libxl__device_physdisk_major_minor(disk->pdev_path, &major, &minor);
flexarray_append(back, "physical-device");
flexarray_append(back, libxl__sprintf(&gc, "%x:%x", major, minor));
@@ -1059,8 +1073,19 @@ int libxl_device_disk_del(libxl_ctx *ctx
devid = libxl__device_disk_dev_number(disk->vdev);
device.backend_domid = disk->backend_domid;
device.backend_devid = devid;
+#if defined(__NetBSD__)
+ switch (disk->backend) {
+ case DISK_BACKEND_PHY:
+ case DISK_BACKEND_VND:
+ device.backend_kind = DEVICE_VBD;
+ break;
+ default:
+ device.backend_kind = DEVICE_TAP;
+ }
+#else
device.backend_kind =
(disk->backend == DISK_BACKEND_PHY) ? DEVICE_VBD : DEVICE_TAP;
+#endif
device.domid = disk->domid;
device.devid = devid;
device.kind = DEVICE_VBD;
@@ -1078,6 +1103,9 @@ char * libxl_device_disk_local_attach(li
char *ret = NULL;
switch (disk->backend) {
+#if defined(__NetBSD__)
+ case DISK_BACKEND_VND:
+#endif
case DISK_BACKEND_PHY:
if (disk->format != DISK_FORMAT_RAW) {
LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "physical block device must"
--- libxl/libxl_device.c.orig 2012-05-28 00:55:38.000000000 -0400
+++ libxl/libxl_device.c 2012-05-27 23:55:47.000000000 -0400
@@ -139,6 +139,9 @@
case DISK_BACKEND_QDISK: return "qdisk";
case DISK_BACKEND_TAP: return "tap";
case DISK_BACKEND_PHY: return "phy";
+#if defined(__NetBSD__)
+ case DISK_BACKEND_VND: return "file";
+#endif
default: return NULL;
}
}
--- libxl/libxl_dm.c.orig 2011-10-20 13:05:42.000000000 -0400
+++ libxl/libxl_dm.c 2012-05-08 02:17:01.000000000 -0400
@@ -845,6 +845,9 @@
ret = 1;
goto out;
+#if defined(__NetBSD__)
+ case DISK_BACKEND_VND:
+#endif
case DISK_BACKEND_PHY:
case DISK_BACKEND_UNKNOWN:
break;
--- libxl/xl_cmdimpl.c.orig 2011-10-20 13:05:43.000000000 -0400
+++ libxl/xl_cmdimpl.c 2012-05-27 14:39:08.000000000 -0400
@@ -484,7 +484,11 @@
}else if ( !strcmp(tok, "file") ) {
state = DSTATE_PHYSPATH;
disk->format = DISK_FORMAT_RAW;
+#if defined(__NetBSD__)
+ disk->backend = DISK_BACKEND_VND;
+#else
disk->backend = DISK_BACKEND_TAP;
+#endif
}else if ((!strcmp(tok, "tap")) ||
(!strcmp(tok, "tap2"))) {
state = DSTATE_TAP;
@@ -4432,7 +4436,11 @@
if (!strcmp(tok, "phy")) {
disk.backend = DISK_BACKEND_PHY;
} else if (!strcmp(tok, "file")) {
+#if defined(__NetBSD__)
+ disk.backend = DISK_BACKEND_VND;
+#else
disk.backend = DISK_BACKEND_TAP;
+#endif
} else if (!strcmp(tok, "tap")) {
tok = strtok(NULL, ":");
if (!strcmp(tok, "aio")) {
--------------090406040207000003080209--
From: John Nemeth <jnemeth@cue.bc.ca>
To: gnats-bugs@netbsd.org, port-xen-maintainer@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based disks
Date: Thu, 19 Sep 2013 12:28:25 -0700
On Sep 19, 6:10pm, toby.karyadi@gmail.com wrote:
}
} >Number: 48220
} >Category: port-xen
} >Synopsis: xentools-4.1: xl has problems with file/vnd based disks
} >Arrival-Date: Thu Sep 19 18:10:00 +0000 2013
} >Originator: Toby Karyadi
} >Environment:
} NetBSD non.ofyour.biz 6.1 NetBSD 6.1 (XEN3_DOM0) amd64
} >Description:
} xl in pkgsrc/sysutils/xentools41, the xen domain control utility
} to replace xm, has the following problems:
xl in xentools41 is known not to work properly. If you want to
use xl, then you should be using xentools42 (and xenkernel42).
} 2) Once problem 1) is fixed, xl also has another bug that caused
} xenbackendd to not call the 'block' script to properly undo the
} vnd device.
If you are using xl then you should not be running xenbackendd.
That is only for use with xm. Using the combination of xl and
xenbackendd can mess things up. This would be another reason to
use xentools42 since the startup scripts have been redone to get
this correct.
}-- End of excerpt from toby.karyadi@gmail.com
From: Toby Karyadi <toby.karyadi@gmail.com>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based
disks
Date: Thu, 19 Sep 2013 16:37:46 -0400
On 9/19/13 3:55 PM, John Nemeth wrote:
> The following reply was made to PR port-xen/48220; it has been noted by GNATS.
>
> From: John Nemeth <jnemeth@cue.bc.ca>
> To: gnats-bugs@netbsd.org, port-xen-maintainer@netbsd.org,
> gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
> Cc:
> Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd based disks
> Date: Thu, 19 Sep 2013 12:28:25 -0700
>
> On Sep 19, 6:10pm, toby.karyadi@gmail.com wrote:
> }
> } >Number: 48220
> } >Category: port-xen
> } >Synopsis: xentools-4.1: xl has problems with file/vnd based disks
> } >Arrival-Date: Thu Sep 19 18:10:00 +0000 2013
> } >Originator: Toby Karyadi
> } >Environment:
> } NetBSD non.ofyour.biz 6.1 NetBSD 6.1 (XEN3_DOM0) amd64
> } >Description:
> } xl in pkgsrc/sysutils/xentools41, the xen domain control utility
> } to replace xm, has the following problems:
>
> xl in xentools41 is known not to work properly. If you want to
> use xl, then you should be using xentools42 (and xenkernel42).
Actually I'm trying to submit patches for xentools41 to fix the problems
with xl that I mentioned. I'm hoping that it would provide a smoother
migration from 3.3 to 4.*. I thought xentools42 doesn't support HVM domU
just yet, is that no longer true? Trying xentools42 is my next step.
>
> } 2) Once problem 1) is fixed, xl also has another bug that caused
> } xenbackendd to not call the 'block' script to properly undo the
> } vnd device.
>
> If you are using xl then you should not be running xenbackendd.
> That is only for use with xm. Using the combination of xl and
> xenbackendd can mess things up. This would be another reason to
> use xentools42 since the startup scripts have been redone to get
> this correct.
>
If you look at rc.d/xencommons for 4.1, it actually does start
xenbackendd. On the other hand, xend should not be started.
Unfortunately xencommons and xend rc scripts uses the same rcvar 'xend',
so if you specify xend=yes in /etc/rc.conf you'll start both xencommons
and xend. The work around is to have xencommons and xend files under
/etc/rc.conf.d where the xencommons file should contain xend=yes where
the xend file should contain xend=no.
In netsbd and xen 4.1, AFAIK, there is no way to invoke the block script
without the help of the xenbackendd. In 4.2, each disk spec can have a
block script which should be invoked by xl itself, and hence negates the
need for xenbackendd. For linux, there is no need of xenbackendd even in
4.1 because xl uses blktap, which netbsd doesn't have.
From: Manuel Bouyer <bouyer@antioche.eu.org>
To: gnats-bugs@NetBSD.org
Cc: port-xen-maintainer@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org, toby.karyadi@gmail.com
Subject: Re: port-xen/48220: xentools-4.1: xl has problems with file/vnd
based disks
Date: Thu, 19 Sep 2013 23:27:46 +0200
On Thu, Sep 19, 2013 at 08:40:01PM +0000, Toby Karyadi wrote:
> > use xl, then you should be using xentools42 (and xenkernel42).
> Actually I'm trying to submit patches for xentools41 to fix the problems
> with xl that I mentioned. I'm hoping that it would provide a smoother
> migration from 3.3 to 4.*. I thought xentools42 doesn't support HVM domU
> just yet, is that no longer true? Trying xentools42 is my next step.
HVM domU works with xen 4.2
> >
> > } 2) Once problem 1) is fixed, xl also has another bug that caused
> > } xenbackendd to not call the 'block' script to properly undo the
> > } vnd device.
> >
> > If you are using xl then you should not be running xenbackendd.
> > That is only for use with xm. Using the combination of xl and
> > xenbackendd can mess things up. This would be another reason to
> > use xentools42 since the startup scripts have been redone to get
> > this correct.
> >
> If you look at rc.d/xencommons for 4.1, it actually does start
> xenbackendd. On the other hand, xend should not be started.
> Unfortunately xencommons and xend rc scripts uses the same rcvar 'xend',
> so if you specify xend=yes in /etc/rc.conf you'll start both xencommons
> and xend. The work around is to have xencommons and xend files under
> /etc/rc.conf.d where the xencommons file should contain xend=yes where
> the xend file should contain xend=no.
At this time, xl is not recommended on xen 4.1, that's why xend is still
started. xl is known to have various problems and limitations
(i.e. features present in xm which have not been intgrated in xl,
like shutdown -a)
--
Manuel Bouyer <bouyer@antioche.eu.org>
NetBSD: 26 ans d'experience feront toujours la difference
--
State-Changed-From-To: open->closed
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Sun, 24 Jun 2018 20:37:25 +0000
State-Changed-Why:
Xen 4.1 was EOLed by upstream.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.