NetBSD Problem Report #48071

From gson@gson.org  Sun Jul 21 15:01:46 2013
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 7D0B9705C7
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 21 Jul 2013 15:01:46 +0000 (UTC)
Message-Id: <20130721145843.3BA8375E14@guava.gson.org>
Date: Sun, 21 Jul 2013 17:58:43 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@gnats.NetBSD.org
Subject: qemu 1.5.1 serial console output lossage
X-Send-Pr-Version: 3.95

>Number:         48071
>Category:       pkg
>Synopsis:       qemu 1.5.1 serial console output lossage
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    pkg-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Jul 21 15:05:00 +0000 2013
>Closed-Date:    Wed Jan 04 10:22:00 +0000 2017
>Last-Modified:  Wed Jan 04 10:22:00 +0000 2017
>Originator:     Andreas Gustafsson
>Release:        NetBSD 6.0.1
>Organization:
>Environment:
System: NetBSD gusto.gson.org 6.0.1
Architecture: amd64
Machine: amd64
>Description:

When attempting to install a NetBSD/i386 or NetBSD/amd64 guest in a
qemu virtual machine on a NetBSD 6 host using qemu 1.5.1 from pkgsrc,
parts of the serial console output are missing.  In practice, this
makes it impossible to complete an installation, as much of the
information needed to correctly respond to sysinst's prompts is
never displayed.

A typical behavior is that you get as far as the following screen in
sysinst:

   You have chosen to install NetBSD on your hard disk.  This will change         
   information on your hard disk.  You should have made a full backup before      
   this procedure!  This procedure will do the following things:                  
	   a) Partition your disk                                                 
	   b) Create new BSD file systems                                         
	   c) Load and install distribution sets                                  
	   d) Some initial system configuration                                   

   (After you enter the partition information but before your disk is changed,    
   you will have the opportunity to quit this procedure.)                         

   Shall we continue?                                                             
                                 ---------------
                                |  yes or no?   |

but the screen fails to display in its entirety.  Sometimes the box
containing the "yes or no?" prompt is missing entirely, and sometimes
it is displayed only partially as shown above.

When this happens, sysinst still responds to a control-L by redrawing
the screen as expected, but the redrawn screen is again incomplete
(though it usually ends in a different place than before).  Typing "b"
and ENTER (if you happen to known that "b" is the right answer even
though the part of the screen telling you that is missing) has the
expected effect of continuing to the next screen, but that screen is
also displayed only in part.

Typing control-C drops to a shell prompt as expected, but typing
"ls -l /bin" shows only the beginning of the expected output.

This behavior all seems consistent with serial console output somehow
getting discarded whenever more than a certain amount is output at once.

Doing a similar installation using qemu-0.15.1nb5 works with no problems.

>How-To-Repeat:

On a NetBSD 6.x system, install qemu 1.5.1 from pkgsrc.
Then run:

  ftp http://ftp.netbsd.org/pub/NetBSD/NetBSD-6.1/amd64/installation/cdrom/boot-com.iso
  dd if=/dev/zero of=wd0.img bs=1m count=1024
  qemu-system-x86_64 -cdrom boot-com.iso -hda wd0.img -nographic

Follow the prompts and attempt to install NetBSD in the virtual
machine.  At some point during the installation (not necessarily the
exact screen shown above), notice how a sysinst dialog fails to
display fully.

>Fix:

>Release-Note:

>Audit-Trail:
From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Tue, 20 Aug 2013 21:46:04 +0300

 Since qemu version 1.6.0 is now in pkgsrc, I tested it, and the
 problem is still there.  I have reproduced it both running qemu
 manually as described in the original PR (though it took a few tries),
 and by running

   anita install http://ftp.netbsd.org/pub/NetBSD/NetBSD-6.1/amd64/

 and having it fail with the last output showing an incomplete sysinst
 screen.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Thu, 20 Feb 2014 22:31:12 +0200

 I just tried the current pkgsrc version, qemu-1.7.0, on a NetBSD/amd64
 host, and it's still broken.

 Qemu versions built on Linux from the qemu git master branch have been
 working this whole time (defining "working" as "anita can successfully
 install NetBSD in the qemu VM"), so the problem seems to be specific
 to NetBSD hosts, or possibly pkgsrc, or conceivably the qemu release
 process.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Greg Troxel <gdt@ir.bbn.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Thu, 08 May 2014 12:56:01 -0400

 I see the same partial-output problem with NetBSD-6 i386 host,
 qemu-2.0.0nb1, and anita 1.32, installing -current i386 or amd64.

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@netbsd.org,
    gnats-admin@netbsd.org,
    pkgsrc-bugs@netbsd.org
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Thu, 8 May 2014 20:09:58 +0300

 As another data point, I didn't see this problem when doing an anita
 install of NetBSD using the qemu 1.7.1 release on a Linux host (having
 built this Linux qemu from source by manually running configure and
 make, not using pkgsrc).

 Therefore, I think we can eliminate the possibility I raised earlier
 in this PR that problem might be triggered by the use of a released
 version of qemu as opposed to the git mainline.  That leaves two
 candidates: the use of NetBSD as the host (as opposed to Linux), or
 the use of pkgsrc as the build environment (as opposed to configure +
 make).
 -- 
 Andreas Gustafsson, gson@gson.org

From: Thomas Klausner <wiz@NetBSD.org>
To: Andreas Gustafsson <gson@gson.org>
Cc: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Thu, 8 May 2014 19:13:31 +0200

 On Thu, May 08, 2014 at 08:09:58PM +0300, Andreas Gustafsson wrote:
 > Therefore, I think we can eliminate the possibility I raised earlier
 > in this PR that problem might be triggered by the use of a released
 > version of qemu as opposed to the git mainline.  That leaves two
 > candidates: the use of NetBSD as the host (as opposed to Linux), or
 > the use of pkgsrc as the build environment (as opposed to configure +
 > make).

 Can you please test configure+make on NetBSD first, to find out if it's
 pkgsrc's fault?

 Thanks,
  Thomas

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@netbsd.org,
    gnats-admin@netbsd.org,
    pkgsrc-bugs@netbsd.org
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Thu, 8 May 2014 21:12:39 +0300

 Thomas Klausner wrote:
 >  Can you please test configure+make on NetBSD first, to find out if it's
 >  pkgsrc's fault?

 Done (not sure what you meant by "first", though - is there a "second"?).
 I built qemu 2.0.0 on NetBSD/amd64 6.1.3 as follows:

  sh configure --python=/usr/pkg/bin/python2.7 --prefix=/usr/pkg --disable-sdl
  # edit util/hbitmap.c replacing all occurrences of popcountl by qemu_popcountl
  gmake

 The resulting qemu-system-i386 binary does suffer from the problem, so it
 looks like the triggering factor is not pkgsrc, but the NetBSD host.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@netbsd.org,
    gnats-admin@netbsd.org,
    pkgsrc-bugs@netbsd.org
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Sat, 10 May 2014 19:54:35 +0300

 Earlier, I said:
 > it looks like the triggering factor is not pkgsrc, but the NetBSD host

 I have now also confirmed this by building qemu 2.0.0 using pkgsrc on
 Linux, and successfully doing an "anita install" using that qemu on
 the Linux host.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@netbsd.org,
    gnats-admin@netbsd.org,
    paul@whooppee.com,
    Greg Troxel <gdt@ir.bbn.com>
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Sun, 22 Jun 2014 20:55:26 +0300

 I have now analyzed this problem further by adding some debug logging
 to the character device handling code in qemu, and found that it is a
 bug in qemu, not NetBSD.

 What happens is that qemu is setting its standard output to
 non-blocking mode, causing writes to return EAGAIN when the tty output
 buffer fills up.  When that happens, qemu is supposed to retry the
 write once the file descriptor becomes ready for output, but it fails
 to retry and ends up discarding the data.

 This problem is easily triggered on NetBSD hosts where the default tty
 buffer size is only 1 kB, which means that even a relatively small
 write() to a tty or pty can cause the buffer to fill up and EAGAIN to
 be returned.  That it is not happening on Linux hosts may be due to
 the larger size of the Linux tty output buffer, or there may be other
 differences that make Linux less likely than NetBSD to return EAGAIN
 from a write to a tty.

 Increasing the tty output buffer size on the NetBSD host using

   sysctl -w kern.tty.qsize=65536

 alleviates the problem significantly, but does not fix it completely:
 with this setting, "anita install" completes successfully, but
 "anita test" sometimes fails because the final summary of the ATF test
 results still fails to appear in its entirety.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: Greg Troxel <gdt@ir.bbn.com>, Paul Goyette <paul@whooppee.com>
Subject: Re: pkg/48071: qemu 1.5.1 serial console output lossage
Date: Sat, 28 Jun 2014 17:58:41 +0300

 I have now devised a couple of ways of reproducing this bug using a
 Linux guest and Linux host, and reported it upstream as

   https://bugs.launchpad.net/qemu/+bug/1335444

 -- 
 Andreas Gustafsson, gson@gson.org

From: "Andreas Gustafsson" <gson@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/48071 CVS commit: pkgsrc/emulators/qemu
Date: Fri, 11 Jul 2014 19:15:08 +0000

 Module Name:	pkgsrc
 Committed By:	gson
 Date:		Fri Jul 11 19:15:08 UTC 2014

 Modified Files:
 	pkgsrc/emulators/qemu: Makefile distinfo
 Added Files:
 	pkgsrc/emulators/qemu/patches: patch-hw_char_serial.c patch-qemu-char.c

 Log Message:
 Fix qemu bug 1335444 aka PR pkg/48071.  Patches from Kirill Batuzov,
 with minor changes to make them apply to 2.0.0.  Bump PKGREVISION.


 To generate a diff of this commit:
 cvs rdiff -u -r1.129 -r1.130 pkgsrc/emulators/qemu/Makefile
 cvs rdiff -u -r1.95 -r1.96 pkgsrc/emulators/qemu/distinfo
 cvs rdiff -u -r0 -r1.1 pkgsrc/emulators/qemu/patches/patch-hw_char_serial.c \
     pkgsrc/emulators/qemu/patches/patch-qemu-char.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: gson@NetBSD.org
State-Changed-When: Fri, 11 Jul 2014 19:35:15 +0000
State-Changed-Why:
Should be fixed by patches in 2.0.0nb4.


State-Changed-From-To: closed->open
State-Changed-By: gson@NetBSD.org
State-Changed-When: Sat, 16 Aug 2014 15:34:04 +0000
State-Changed-Why:
qemu-system-sparc is still broken.


From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/48071 (qemu 1.5.1 serial console output lossage)
Date: Sat, 16 Aug 2014 18:33:12 +0300

 Although this bug is now fixed in qemu-system-i386 and
 qemu-system-x86_64, it is still present in qemu-system-sparc.  See the
 upstream ticket https://bugs.launchpad.net/qemu/+bug/1335444 for
 details.
 -- 
 Andreas Gustafsson, gson@gson.org

State-Changed-From-To: open->closed
State-Changed-By: gson@NetBSD.org
State-Changed-When: Wed, 04 Jan 2017 10:22:00 +0000
State-Changed-Why:
Fixed for sparc in qemu 2.8.  For details, see upstream bug
https://bugs.launchpad.net/qemu/+bug/1399943


>Unformatted:

Home	PR Database Search