NetBSD Problem Report #54611
From www@netbsd.org Wed Oct 9 17:17:33 2019
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id BFC9B7A1DC
for <gnats-bugs@gnats.NetBSD.org>; Wed, 9 Oct 2019 17:17:32 +0000 (UTC)
Message-Id: <20191009171731.E23457A245@mollari.NetBSD.org>
Date: Wed, 9 Oct 2019 17:17:31 +0000 (UTC)
From: rnestor@mac.com
Reply-To: rnestor@mac.com
To: gnats-bugs@NetBSD.org
Subject: NVMM faults on a Windows 95 install
X-Send-Pr-Version: www-1.0
>Number: 54611
>Category: kern
>Synopsis: NVMM faults on a Windows 95 install
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: kern-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Oct 09 17:20:00 +0000 2019
>Closed-Date: Sat Apr 11 15:37:26 +0000 2020
>Last-Modified: Sat Apr 11 15:37:26 +0000 2020
>Originator: Bob Nestor
>Release: NetBSD-9.0 BETA Oct-8, qem-nvmm 3.1
>Organization:
>Environment:
NetBSD bandit.home.net 9.0_BETA NetBSD 9.0_BETA (GENERIC) #0: Sun Oct 6 11:20:50 UTC 2019 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64
QEMU emulator version 3.1.0
Copyright (c) 2003-2018 Fabrice Bellard and the QEMU Project developers
>Description:
Windows-95C installation from an official CD fails with the following:
NetBSD Virtual Machine Monitor accelerator is operational
qemu-system-x86_64: NVMM: Mem Assist Failed [gpa=0xb8040]
qemu-system-x86_64: NVMM: Failed to execute a VCPU.
The same install works under XEN using the same QEMU parameters as follows:
qemu-img create -f raw /home/Win95C.dsk 2G
qemu-system-x86_64 -accel nvmm -cpu pentium -smp 1 \
-cdrom /home/Win95C.iso -name "Windows-95 Install" \
-netdev tap,id=nd0,ifname=tap0,script=no,downscript=no \
-device rtl8139,netdev=nd0 \
-drive file=/home/Win95C.dsk,media=disk,format=raw
Attempts to find a magic set of parameters to QEMU have so far all produced the same results.
>How-To-Repeat:
As described above.
>Fix:
Unknown.
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->feedback
State-Changed-By: maxv@NetBSD.org
State-Changed-When: Sat, 12 Oct 2019 12:33:35 +0000
State-Changed-Why:
Is it possible for you to put this ISO online? You can answer to me offlist
with the link.
I think I see the problem, just a missing instruction in the disassembler,
but I need to test on my side.
From: "Maxime Villard" <maxv@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54611 CVS commit: src
Date: Sun, 13 Oct 2019 17:32:15 +0000
Module Name: src
Committed By: maxv
Date: Sun Oct 13 17:32:15 UTC 2019
Modified Files:
src/lib/libnvmm: libnvmm_x86.c
src/tests/lib/libnvmm: h_mem_assist.c h_mem_assist_asm.S
Log Message:
Fix incorrect parsing: the R/M field uses a special GPR map when the
address size is 16 bits, regardless of the actual operating mode. With
this special map there can be two registers referenced at once, and
also disp16-only.
Implement this special behavior, and add associated tests. While here
simplify a few things.
With this in place, the Windows 95 installer initializes correctly.
Part of PR/54611.
To generate a diff of this commit:
cvs rdiff -u -r1.31 -r1.32 src/lib/libnvmm/libnvmm_x86.c
cvs rdiff -u -r1.11 -r1.12 src/tests/lib/libnvmm/h_mem_assist.c
cvs rdiff -u -r1.6 -r1.7 src/tests/lib/libnvmm/h_mem_assist_asm.S
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Robert Nestor <rnestor@mac.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: PR/54611 CVS commit: src
Date: Sun, 13 Oct 2019 12:45:47 -0500
Fantastic! Will this be pulled up to 9.0 as well?
Thanks,
-bob
On Oct 13, 2019, at 12:35 PM, Maxime Villard <maxv@netbsd.org> wrote:
> The following reply was made to PR kern/54611; it has been noted by GNATS.
>
> From: "Maxime Villard" <maxv@netbsd.org>
> To: gnats-bugs@gnats.NetBSD.org
> Cc:
> Subject: PR/54611 CVS commit: src
> Date: Sun, 13 Oct 2019 17:32:15 +0000
>
> Module Name: src
> Committed By: maxv
> Date: Sun Oct 13 17:32:15 UTC 2019
>
> Modified Files:
> src/lib/libnvmm: libnvmm_x86.c
> src/tests/lib/libnvmm: h_mem_assist.c h_mem_assist_asm.S
>
> Log Message:
> Fix incorrect parsing: the R/M field uses a special GPR map when the
> address size is 16 bits, regardless of the actual operating mode. With
> this special map there can be two registers referenced at once, and
> also disp16-only.
>
> Implement this special behavior, and add associated tests. While here
> simplify a few things.
>
> With this in place, the Windows 95 installer initializes correctly.
>
> Part of PR/54611.
>
>
> To generate a diff of this commit:
> cvs rdiff -u -r1.31 -r1.32 src/lib/libnvmm/libnvmm_x86.c
> cvs rdiff -u -r1.11 -r1.12 src/tests/lib/libnvmm/h_mem_assist.c
> cvs rdiff -u -r1.6 -r1.7 src/tests/lib/libnvmm/h_mem_assist_asm.S
>
> Please note that diffs are not public domain; they are subject to the
> copyright notices on the relevant files.
>
From: "Maxime Villard" <maxv@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54611 CVS commit: src
Date: Mon, 14 Oct 2019 10:39:24 +0000
Module Name: src
Committed By: maxv
Date: Mon Oct 14 10:39:24 UTC 2019
Modified Files:
src/lib/libnvmm: libnvmm_x86.c
src/tests/lib/libnvmm: h_mem_assist.c h_mem_assist_asm.S
Log Message:
Implement XCHG, add associated tests, and add comments to explain. With
this in place the Windows 95 installer completes successfuly.
Part of PR/54611.
To generate a diff of this commit:
cvs rdiff -u -r1.32 -r1.33 src/lib/libnvmm/libnvmm_x86.c
cvs rdiff -u -r1.13 -r1.14 src/tests/lib/libnvmm/h_mem_assist.c
cvs rdiff -u -r1.7 -r1.8 src/tests/lib/libnvmm/h_mem_assist_asm.S
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Robert Nestor <rnestor@mac.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/54611
Date: Tue, 5 Nov 2019 08:17:21 -0600
Not sure this particular problem is completely resolved yet. Tried =
installing Windows-95 on a -current system from 3-Nov with qemu-nvmm =
rebuilt from the same date. When the system completes the first stage =
of the Windows installation and goes to reboot Windows from the new HD =
for the first time it errors out with:
NetBSD Virtual Machine Monitor accelerator is operational
qemu-system-x86_64: NVMM: Mem Assist Failed [gpa=3D0xa3a81]
qemu-system-x86_64: NVMM: Failed to execute a VCPU.=
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54611 CVS commit: [netbsd-9] src
Date: Sun, 10 Nov 2019 12:58:31 +0000
Module Name: src
Committed By: martin
Date: Sun Nov 10 12:58:31 UTC 2019
Modified Files:
src/distrib/sets/lists/base [netbsd-9]: md.amd64
src/distrib/sets/lists/debug [netbsd-9]: md.amd64
src/distrib/sets/lists/man [netbsd-9]: mi
src/etc [netbsd-9]: MAKEDEV.tmpl group
src/lib/libnvmm [netbsd-9]: libnvmm.3 libnvmm.c libnvmm_x86.c nvmm.h
src/share/mk [netbsd-9]: bsd.hostprog.mk
src/sys/dev/nvmm [netbsd-9]: nvmm.c nvmm.h nvmm_internal.h nvmm_ioctl.h
src/sys/dev/nvmm/x86 [netbsd-9]: nvmm_x86.h nvmm_x86_svm.c
nvmm_x86_vmx.c
src/tests/lib/libnvmm [netbsd-9]: h_io_assist.c h_mem_assist.c
h_mem_assist_asm.S
src/usr.sbin [netbsd-9]: Makefile
src/usr.sbin/postinstall [netbsd-9]: postinstall.in
Added Files:
src/usr.sbin/nvmmctl [netbsd-9]: Makefile nvmmctl.8 nvmmctl.c
Log Message:
Pull up following revision(s) (requested by maxv in ticket #405):
usr.sbin/nvmmctl/nvmmctl.8: revision 1.2
lib/libnvmm/libnvmm.3: revision 1.24
sys/dev/nvmm/nvmm.h: revision 1.11
lib/libnvmm/libnvmm.3: revision 1.25
sys/dev/nvmm/x86/nvmm_x86.h: revision 1.16
sys/dev/nvmm/nvmm.h: revision 1.12
sys/dev/nvmm/x86/nvmm_x86.h: revision 1.17
tests/lib/libnvmm/h_mem_assist.c: revision 1.12
sys/dev/nvmm/x86/nvmm_x86.h: revision 1.18
share/mk/bsd.hostprog.mk: revision 1.82
lib/libnvmm/libnvmm.c: revision 1.15
distrib/sets/lists/base/md.amd64: revision 1.281
tests/lib/libnvmm/h_mem_assist.c: revision 1.13
lib/libnvmm/libnvmm.c: revision 1.16
tests/lib/libnvmm/h_mem_assist.c: revision 1.14
lib/libnvmm/libnvmm_x86.c: revision 1.32
lib/libnvmm/libnvmm.c: revision 1.17
tests/lib/libnvmm/h_mem_assist.c: revision 1.15
lib/libnvmm/libnvmm_x86.c: revision 1.33
lib/libnvmm/libnvmm.c: revision 1.18
usr.sbin/nvmmctl/Makefile: revision 1.1
tests/lib/libnvmm/h_mem_assist_asm.S: revision 1.7
tests/lib/libnvmm/h_mem_assist.c: revision 1.16
lib/libnvmm/libnvmm_x86.c: revision 1.34
usr.sbin/nvmmctl/Makefile: revision 1.2
tests/lib/libnvmm/h_mem_assist_asm.S: revision 1.8
tests/lib/libnvmm/h_mem_assist.c: revision 1.17
sys/dev/nvmm/nvmm_internal.h: revision 1.13
lib/libnvmm/libnvmm_x86.c: revision 1.35
lib/libnvmm/libnvmm_x86.c: revision 1.36
usr.sbin/postinstall/postinstall.in: revision 1.8
lib/libnvmm/libnvmm_x86.c: revision 1.37
lib/libnvmm/libnvmm_x86.c: revision 1.38
lib/libnvmm/libnvmm_x86.c: revision 1.39
usr.sbin/Makefile: revision 1.282
lib/libnvmm/nvmm.h: revision 1.13
lib/libnvmm/nvmm.h: revision 1.14
lib/libnvmm/nvmm.h: revision 1.15
sys/dev/nvmm/nvmm.c: revision 1.23
lib/libnvmm/nvmm.h: revision 1.16
sys/dev/nvmm/nvmm.c: revision 1.24
lib/libnvmm/nvmm.h: revision 1.17
sys/dev/nvmm/nvmm.c: revision 1.25
tests/lib/libnvmm/h_io_assist.c: revision 1.9
etc/MAKEDEV.tmpl: revision 1.209
tests/lib/libnvmm/h_io_assist.c: revision 1.10
tests/lib/libnvmm/h_io_assist.c: revision 1.11
etc/group: revision 1.35
distrib/sets/lists/man/mi: revision 1.1660
sys/dev/nvmm/x86/nvmm_x86_vmx.c: revision 1.40
sys/dev/nvmm/x86/nvmm_x86_vmx.c: revision 1.41
sys/dev/nvmm/x86/nvmm_x86_vmx.c: revision 1.42
sys/dev/nvmm/x86/nvmm_x86_vmx.c: revision 1.43
sys/dev/nvmm/x86/nvmm_x86_vmx.c: revision 1.44
sys/dev/nvmm/x86/nvmm_x86_svm.c: revision 1.51
sys/dev/nvmm/nvmm_ioctl.h: revision 1.8
sys/dev/nvmm/x86/nvmm_x86_svm.c: revision 1.52
sys/dev/nvmm/nvmm_ioctl.h: revision 1.9
sys/dev/nvmm/x86/nvmm_x86_svm.c: revision 1.53
usr.sbin/nvmmctl/nvmmctl.c: revision 1.1
lib/libnvmm/libnvmm.3: revision 1.20
distrib/sets/lists/debug/md.amd64: revision 1.106
lib/libnvmm/libnvmm.3: revision 1.21
lib/libnvmm/libnvmm.3: revision 1.22
usr.sbin/nvmmctl/nvmmctl.8: revision 1.1
lib/libnvmm/libnvmm.3: revision 1.23
Fix incorrect parsing: the R/M field uses a special GPR map when the
address size is 16 bits, regardless of the actual operating mode. With
this special map there can be two registers referenced at once, and
also disp16-only.
Implement this special behavior, and add associated tests. While here
simplify a few things.
With this in place, the Windows 95 installer initializes correctly.
Part of PR/54611.
add missing initializer
Implement XCHG, add associated tests, and add comments to explain. With
this in place the Windows 95 installer completes successfuly.
Part of PR/54611.
Improve nvmm_vcpu_dump().
Put back 'default', because llvm apparently doesn't realize that all cases
are covered in the switch.
Miscellaneous changes in NVMM, to address several inconsistencies and
issues in the libnvmm API.
- Rename NVMM_CAPABILITY_VERSION to NVMM_KERN_VERSION, and check it in
libnvmm. Introduce NVMM_USER_VERSION, for future use.
- In libnvmm, open "/dev/nvmm" as read-only and with O_CLOEXEC. This is to
avoid sharing the VMs with the children if the process forks. In the
NVMM driver, force O_CLOEXEC on open().
- Rename the following things for consistency:
nvmm_exit* -> nvmm_vcpu_exit*
nvmm_event* -> nvmm_vcpu_event*
NVMM_EXIT_* -> NVMM_VCPU_EXIT_*
NVMM_EVENT_INTERRUPT_HW -> NVMM_VCPU_EVENT_INTR
NVMM_EVENT_EXCEPTION -> NVMM_VCPU_EVENT_EXCP
Delete NVMM_EVENT_INTERRUPT_SW, unused already.
- Slightly reorganize the MI/MD definitions, for internal clarity.
- Split NVMM_VCPU_EXIT_MSR in two: NVMM_VCPU_EXIT_{RD,WR}MSR. Also provide
separate u.rdmsr and u.wrmsr fields. This is more consistent with the
other exit reasons.
- Change the types of several variables:
event.type enum -> u_int
event.vector uint64_t -> uint8_t
exit.u.*msr.msr: uint64_t -> uint32_t
exit.u.io.type: enum -> bool
exit.u.io.seg: int -> int8_t
cap.arch.mxcsr_mask: uint64_t -> uint32_t
cap.arch.conf_cpuid_maxops: uint64_t -> uint32_t
- Delete NVMM_VCPU_EXIT_MWAIT_COND, it is AMD-only and confusing, and we
already intercept 'monitor' so it is never armed.
- Introduce vmx_exit_insn() for NVMM-Intel, similar to svm_exit_insn().
The 'npc' field wasn't getting filled properly during certain VMEXITs.
- Introduce nvmm_vcpu_configure(). Similar to nvmm_machine_configure(),
but as its name indicates, the configuration is per-VCPU and not per-VM.
Migrate and rename NVMM_MACH_CONF_X86_CPUID to NVMM_VCPU_CONF_CPUID.
This becomes per-VCPU, which makes more sense than per-VM.
- Extend the NVMM_VCPU_CONF_CPUID conf to allow triggering VMEXITs on
specific leaves. Until now we could only mask the leaves. An uint32_t
is added in the structure:
uint32_t mask:1;
uint32_t exit:1;
uint32_t rsvd:30;
The two first bits select the desired behavior on the leaf. Specifying
zero on both resets the leaf to the default behavior. The new
NVMM_VCPU_EXIT_CPUID exit reason is added.
Three changes in libnvmm:
- Add 'mach' and 'vcpu' backpointers in the nvmm_io and nvmm_mem
structures.
- Rename 'nvmm_callbacks' to 'nvmm_assist_callbacks'.
- Rename and migrate NVMM_MACH_CONF_CALLBACKS to NVMM_VCPU_CONF_CALLBACKS,
it now becomes per-VCPU.
Update the libnvmm man page:
- Sync the naming with reality.
- Replace "relevant" by "desired" and "virtualizer" by "emulator", closer
to what I meant.
- Add a "VCPU Configuration" section.
- Add a "Machine Ownership" section.
Add the "nvmm" group, and make nvmm_init() public. Sent to tech-kern@ a few
days ago.
Use the new PTE naming, and define CR3_FRAME_* separately. No functional
change.
Add a new VCPU conf option, that allows userland to request VMEXITs after a
TPR change. This is supported on all Intel CPUs, and not-too-old AMD CPUs.
The reason for wanting this option is that certain OSes (like Win10 64bit)
manage interrupt priority in hardware via CR8 directly, and for these OSes,
the emulator may want to sync its internal TPR state on each change.
Add two new fields in cap.arch, to report the conf capabilities. Report TPR
only on Intel for now, not AMD, because I don't have a recent AMD CPU on
which to test.
Mask CPUID leaf 0x0A on Intel, because we don't want the guest to try (and
fail) to probe the PMC MSRs. This avoids "Unexpected WRMSR" warnings in
qemu-nvmm.
Add PCID support in the guests. This speeds up most 64bit guests, because
since Meltdown, everybody uses PCID (including NetBSD).
Change the way root_owner works: consider the calling process as root_owner
not if it has root privileges, but if the /dev/nvmm device was opened with
write permissions. Introduce the undocumented nvmm_root_init() function to
achieve that.
The goal is to simplify the logic and have more granularity, eg if we want
a monitoring agent to access VMs but don't want to give this agent real
root access on the system.
A few changes:
- Use smaller types in struct nvmm_capability.
- Use smaller type for nvmm_io.port.
- Switch exitstate to a compacted structure.
Add nram in struct nvmm_ctl_mach_info.
Add nvmmctl, with two commands for now.
Macro tidyness.
Sort SEE ALSO.
should be fork(2), noticed by wiz
Add debug entry for newly introduced nvmmctl utility.
Annotate a covering switch as such to avoid warnings about missing
returns.
Forgot to put nvmmctl in the "nvmm" group.
Add nvmm group.
To generate a diff of this commit:
cvs rdiff -u -r1.277.2.2 -r1.277.2.3 src/distrib/sets/lists/base/md.amd64
cvs rdiff -u -r1.105 -r1.105.2.1 src/distrib/sets/lists/debug/md.amd64
cvs rdiff -u -r1.1649.2.2 -r1.1649.2.3 src/distrib/sets/lists/man/mi
cvs rdiff -u -r1.205.2.1 -r1.205.2.2 src/etc/MAKEDEV.tmpl
cvs rdiff -u -r1.34 -r1.34.16.1 src/etc/group
cvs rdiff -u -r1.19 -r1.19.4.1 src/lib/libnvmm/libnvmm.3
cvs rdiff -u -r1.14 -r1.14.4.1 src/lib/libnvmm/libnvmm.c
cvs rdiff -u -r1.31 -r1.31.4.1 src/lib/libnvmm/libnvmm_x86.c
cvs rdiff -u -r1.12 -r1.12.4.1 src/lib/libnvmm/nvmm.h
cvs rdiff -u -r1.81 -r1.81.2.1 src/share/mk/bsd.hostprog.mk
cvs rdiff -u -r1.22 -r1.22.2.1 src/sys/dev/nvmm/nvmm.c
cvs rdiff -u -r1.10 -r1.10.4.1 src/sys/dev/nvmm/nvmm.h
cvs rdiff -u -r1.12 -r1.12.2.1 src/sys/dev/nvmm/nvmm_internal.h
cvs rdiff -u -r1.7 -r1.7.4.1 src/sys/dev/nvmm/nvmm_ioctl.h
cvs rdiff -u -r1.15 -r1.15.4.1 src/sys/dev/nvmm/x86/nvmm_x86.h
cvs rdiff -u -r1.46.4.1 -r1.46.4.2 src/sys/dev/nvmm/x86/nvmm_x86_svm.c
cvs rdiff -u -r1.36.2.2 -r1.36.2.3 src/sys/dev/nvmm/x86/nvmm_x86_vmx.c
cvs rdiff -u -r1.8 -r1.8.4.1 src/tests/lib/libnvmm/h_io_assist.c
cvs rdiff -u -r1.11 -r1.11.4.1 src/tests/lib/libnvmm/h_mem_assist.c
cvs rdiff -u -r1.6 -r1.6.4.1 src/tests/lib/libnvmm/h_mem_assist_asm.S
cvs rdiff -u -r1.280 -r1.280.2.1 src/usr.sbin/Makefile
cvs rdiff -u -r0 -r1.2.2.2 src/usr.sbin/nvmmctl/Makefile \
src/usr.sbin/nvmmctl/nvmmctl.8
cvs rdiff -u -r0 -r1.1.2.2 src/usr.sbin/nvmmctl/nvmmctl.c
cvs rdiff -u -r1.5.2.1 -r1.5.2.2 src/usr.sbin/postinstall/postinstall.in
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: feedback->open
State-Changed-By: maya@NetBSD.org
State-Changed-When: Sat, 11 Apr 2020 13:16:11 +0000
State-Changed-Why:
Author replies there are still issues with Windows 95 install.
From: Robert Nestor <rnestor@mac.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org,
netbsd-bugs@netbsd.org,
gnats-admin@netbsd.org,
maya@NetBSD.org
Subject: Re: kern/54611 (NVMM faults on a Windows 95 install)
Date: Sat, 11 Apr 2020 08:44:47 -0500
I think this PR should be closed. There don=92t appear to be =
significant issues with the install of Windows-95, but there may be =
issues with being able to run it under NVMM on some machines. There are =
a lot of people who have reported that Windows-95 is very sensitive to =
machine speed and does not run reliably in many virtual environments =
where the host machine is just too fast. So I suspect this isn=92t =
really an issue for NVMM.
On Apr 11, 2020, at 8:16 AM, maya@NetBSD.org wrote:
> Synopsis: NVMM faults on a Windows 95 install
>=20
> State-Changed-From-To: feedback->open
> State-Changed-By: maya@NetBSD.org
> State-Changed-When: Sat, 11 Apr 2020 13:16:11 +0000
> State-Changed-Why:
> Author replies there are still issues with Windows 95 install.
>=20
>=20
>=20
State-Changed-From-To: open->closed
State-Changed-By: maya@NetBSD.org
State-Changed-When: Sat, 11 Apr 2020 15:37:26 +0000
State-Changed-Why:
Author believes this PR should be closed, and is more of a Windows 95 issue than NVMM.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home