NetBSD Problem Report #56582
From www@netbsd.org Sat Dec 25 23:42:21 2021
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 62C541A9239
for <gnats-bugs@gnats.NetBSD.org>; Sat, 25 Dec 2021 23:42:21 +0000 (UTC)
Message-Id: <20211225234219.D1D5C1A923C@mollari.NetBSD.org>
Date: Sat, 25 Dec 2021 23:42:19 +0000 (UTC)
From: dross@pobox.com
Reply-To: dross@pobox.com
To: gnats-bugs@NetBSD.org
Subject: port-atari freeze on installation when extracting base.tgz
X-Send-Pr-Version: www-1.0
>Number: 56582
>Category: port-atari
>Synopsis: port-atari freeze on installation when extracting base.tgz
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: port-atari-maintainer
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Dec 25 23:45:00 +0000 2021
>Last-Modified: Sun Jul 10 01:49:01 +0000 2022
>Originator: David Ross
>Release: 9.2
>Organization:
>Environment:
>Description:
port-atari 9.2 installation on an atari TT030 freezes during extraction of base.tgz. The extraction proceeds for a few minutes, but then appears to stop (no HD activity). Ctrl-C does not work to exit the extraction, the system just appears frozen. This issue reproduces reliably.
The issue does not happen on 8.2, which I am able to install and run reliably on the same system.
On 9.2, there seems to be an issue with how text is rendered that may (or may not) be related to the install failure. For example, the 'progress' command doesn't render progress at all. I would normally be able to see how far set extraction has completed, but it's not possible to do so because 'progress' is broken.
>How-To-Repeat:
Install NetBSD port-atari onto an Atari TT030, mostly as described here:
https://netbsd-ataritt.fandom.com/wiki/NetBSD_on_the_Atari_TT030
Choose to download sets over HTTP. When the base.tgz set is extracted, observe the system stops responding after a few minutes.
This is a standard install, nothing special. I can help test, etc. as necessary, please feel free to reach out.
>Fix:
>Release-Note:
>Audit-Trail:
From: David Ross <randomdross00@gmail.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sat, 25 Dec 2021 15:57:10 -0800
(I'm not sure if my reply to this mail will land in the bug text, but
hopefully that's how it works...)
I mean to file this under port-atari instead of install. Let me know
if it makes sense and I can re-file it there, or feel free to move it
over if you can.
On Sat, Dec 25, 2021 at 3:45 PM <gnats-admin@netbsd.org> wrote:
>
> Thank you very much for your problem report.
> It has the internal identification `install/56582'.
> The individual assigned to look at your
> report is: install-manager.
>
> >Category: install
> >Responsible: install-manager
> >Synopsis: port-atari freeze on installation when extracting base.tgz
> >Arrival-Date: Sat Dec 25 23:45:00 +0000 2021
>
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 07:10:13 -0000 (UTC)
dross@pobox.com writes:
>>Description:
>port-atari 9.2 installation on an atari TT030 freezes during extraction of base.tgz. The extraction proceeds for a few minutes, but then appears to stop (no HD activity). Ctrl-C does not work to exit the extraction, the system just appears frozen. This issue reproduces reliably.
>The issue does not happen on 8.2, which I am able to install and run reliably on the same system.
Can you give details about the hardware, in particular how much RAM exists ?
When the system hangs, can you use ALT+LSHIFT+F9 to enter the kernel
debugger ?
Responsible-Changed-From-To: install-manager->port-atari-maintainer
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Sun, 26 Dec 2021 12:06:56 +0000
Responsible-Changed-Why:
atari (or m68k) kernel bug
From: David Ross <randomdross00@gmail.com>
To: gnats-bugs@netbsd.org
Cc: install-manager@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
David Ross <dross@pobox.com>
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 10:20:57 -0800
> Can you give details about the hardware, in particular how much RAM exist=
s ?
The system has 10MB of ST RAM and 16MB of TT RAM. I think NetBSD only
uses the ST RAM for video and floppy, and the TT RAM is what's used
normally for everything else.
> When the system hangs, can you use ALT+LSHIFT+F9 to enter the kernel debu=
gger ?
I will try this and report back soon. Thanks!
On Sat, Dec 25, 2021 at 11:15 PM Michael van Elst <mlelstv@serpens.de> wrot=
e:
>
> The following reply was made to PR install/56582; it has been noted by GN=
ATS.
>
> From: mlelstv@serpens.de (Michael van Elst)
> To: gnats-bugs@netbsd.org
> Cc:
> Subject: Re: install/56582: port-atari freeze on installation when extrac=
ting base.tgz
> Date: Sun, 26 Dec 2021 07:10:13 -0000 (UTC)
>
> dross@pobox.com writes:
>
> >>Description:
> >port-atari 9.2 installation on an atari TT030 freezes during extraction=
of base.tgz. The extraction proceeds for a few minutes, but then appears =
to stop (no HD activity). Ctrl-C does not work to exit the extraction, the=
system just appears frozen. This issue reproduces reliably.
>
> >The issue does not happen on 8.2, which I am able to install and run re=
liably on the same system.
>
> Can you give details about the hardware, in particular how much RAM exis=
ts ?
>
> When the system hangs, can you use ALT+LSHIFT+F9 to enter the kernel
> debugger ?
>
From: David Holland <dholland-gnats@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: install/56582: port-atari freeze on installation when extracting
base.tgz
Date: Sun, 10 Jul 2022 01:42:09 +0000
Also not sent to gnats.
------
From: David Ross <randomdross00@gmail.com>
To: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, David Ross <dross@pobox.com>
Subject: Re: install/56582: port-atari freeze on installation when extracting
base.tgz
Date: Sun, 26 Dec 2021 10:44:53 -0800
I'm unable to break into the kernel debugger using ALT+LSHIFT+F9 after
booting the 9.2 kernel / sysinst. This is while sysinst is operating
normally, before the hang. So there is probably a different key
sequence I need to use to get into the kernel debugger on the Atari.
I'll do a bit of research to see if I can figure that out.
It's also worth pointing out Izumi Tsutsui's tweet here, describing
what was observed during a 9.0 RC1 install:
https://twitter.com/tsutsuii/status/1475065059777064960
---
there were two problems. - "qqq.." is only visible issue. NetBSD/atari
ite console doesn't support ruled lines but sysinst assumes full
terminfo (smacs,rmacs,acsc). - sysinst on atari omits "progress bar"
support due to size restriction. The installation goes background.
---
Izumi's install was apparently successful, though it was an upgrade
and not a clean install.
So it sounds like the hang that I'm seeing on my 9.2 install may be
entirely unrelated to the text display issues / missing progress bar.
Dave
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 18:57:58 -0000 (UTC)
randomdross00@gmail.com (David Ross) writes:
>Izumi's install was apparently successful, though it was an upgrade
>and not a clean install.
>So it sounds like the hang that I'm seeing on my 9.2 install may be
>entirely unrelated to the text display issues / missing progress bar.
My prime suspect would be memory. 16MB usuable memory could be too tight,
and NetBSD doesn't handle memory shortage well.
N.B. if the installation system uses a SMALL kernel, then DDB doesn't
exist.
From: David Ross <randomdross00@gmail.com>
To: gnats-bugs@netbsd.org
Cc: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, David Ross <dross@pobox.com>
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 15:55:37 -0800
Thanks! That's certainly possible. I've never been able to get my
TT030 stable with >16MB, however I know Izumi has. I'll be trying a
new memory card soon, and hopefully this will help. I also have some
experience cross-building netbsd, so I may do that soon and compile
everything with -Os instead of -O2 in case that gives me just enough
memory to get past the base.tgz extraction.
For right now, I'm going to try installing the 9.0 release and see how
that goes. I know 8.2 installs and works great, but from there I had
jumped right to 9.2 without trying the earlier 9.x releases.
Dave
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: install/56582: port-atari freeze on installation when extracting
base.tgz
Date: Sun, 10 Jul 2022 01:22:48 +0000
Not sent to gnats. (Will shift to the correct position in the
discussion in the database.)
------
From: David Ross <randomdross00@gmail.com>
To: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: install/56582: port-atari freeze on installation when extracting
base.tgz
Date: Wed, 29 Dec 2021 17:20:25 -0800
I was able to complete a NetBSD 9.2 install by booting with an 8.2
kernel. However, on first boot I get:
Dec 29 10:56:44 tt030 ntpd[373]: mlockall(): Cannot allocate memory
...and a few postfix related errors. You can see it all in this
Twitter thread: https://twitter.com/nbtt030/status/1476355335493324802
The system does ultimately come up, though I'm not sure yet if it
runs stable.
So this tells me the install problem is not simply a matter of running
out of memory during the initial boot / sysinst process. The ntpd
issue happens on normal boot with the 9.2 kernel, so the system will
have access to swap. So it seems there may be a memory related
problem that crept into the kernel between 8.2 and 9.0.
This is not something I've observed previously -- NetBSD 8.2 runs fine
on this machine.
Dave
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Thu, 30 Dec 2021 07:08:15 -0000 (UTC)
randomdross00@gmail.com (David Ross) writes:
>I was able to complete a NetBSD 9.2 install by booting with an 8.2
>kernel. However, on first boot I get:
>Dec 29 10:56:44 tt030 ntpd[373]: mlockall(): Cannot allocate memory
mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
you can ignore it. Worst thing is that ntpd gets swapped out and cannot
sync time anymore.
ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
to have this locked in memory on a small machine. Adding
rlimit memlock -1
to ntp.conf should prevent it from trying.
From: Jason Thorpe <thorpej@me.com>
To: gnats-bugs@netbsd.org
Cc: port-atari-maintainer@netbsd.org,
gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org,
dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting
base.tgz
Date: Thu, 30 Dec 2021 07:48:57 -0700
> On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> =
wrote:
> mlockall() returns ENOMEM when the system- or per-process-limit is =
exceeded,
> you can ignore it. Worst thing is that ntpd gets swapped out and =
cannot
> sync time anymore.
>=20
> ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
> to have this locked in memory on a small machine. Adding
>=20
> rlimit memlock -1
>=20
> to ntp.conf should prevent it from trying.
I wonder if we should add a heuristic in ntpd to auto-tune for this.
-- thorpej
From: David Ross <randomdross00@gmail.com>
To: Jason Thorpe <thorpej@me.com>
Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Thu, 30 Dec 2021 10:44:22 -0800
Thanks! I thought ntpd might be useful for keeping the time sync'd,
but I didn't realize how much memory it's eating. And probably
ntpdate is sufficient if I just want to keep the system time in sync
with Internet time (?). I'll stop running ntpd now.
It appears that right now at least the 9.2 kernel may in fact be
stable on this machine. That makes me have faith in Michael van
Elst's suggestion that the freeze during install could be simply due
to a lack of memory.
I'll be experimenting with a few solutions going forward:
- Getting a memory card for my TT030 that is >16MB and runs stable.
- Cross-building netbsd for atari with -Os and using the sysinst.fs
and ataritt kernel that it generates for the install. I can also
remove some things from my build of the ataritt kernel, which could
help.
It might make sense to use this PR to track any potential changes that
could help get 9.2 to install successfully on a 16MB TT030. The most
common RAM board for the TT (the Atari branded board) will do 16MB
max, and in my experience many >16MB RAM boards don't run stable with
netbsd.
Dave
On Thu, Dec 30, 2021 at 6:49 AM Jason Thorpe <thorpej@me.com> wrote:
>
>
> > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
>
> > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
> > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
> > sync time anymore.
> >
> > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
> > to have this locked in memory on a small machine. Adding
> >
> > rlimit memlock -1
> >
> > to ntp.conf should prevent it from trying.
>
> I wonder if we should add a heuristic in ntpd to auto-tune for this.
>
> -- thorpej
>
From: David Brownlee <abs@absd.org>
To: Jason Thorpe <thorpej@me.com>
Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Thu, 30 Dec 2021 23:05:31 +0000
On Thu, 30 Dec 2021 at 14:49, Jason Thorpe <thorpej@me.com> wrote:
>
> > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
>
> > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
> > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
> > sync time anymore.
> >
> > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
> > to have this locked in memory on a small machine. Adding
> >
> > rlimit memlock -1
> >
> > to ntp.conf should prevent it from trying.
>
> I wonder if we should add a heuristic in ntpd to auto-tune for this.
I'd be loudly in favour of this - people should be able to set things
explicitly, but we really should be defaulting more things based on
current memory and or relative compute level.
David
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting
base.tgz
Date: Fri, 31 Dec 2021 00:35:43 +0100
On Thu, Dec 30, 2021 at 11:10:02PM +0000, David Brownlee wrote:
> The following reply was made to PR port-atari/56582; it has been noted by GNATS.
>
> From: David Brownlee <abs@absd.org>
> To: Jason Thorpe <thorpej@me.com>
> Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org,
> gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
> Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
> Date: Thu, 30 Dec 2021 23:05:31 +0000
>
> On Thu, 30 Dec 2021 at 14:49, Jason Thorpe <thorpej@me.com> wrote:
> >
> > > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
> >
> > > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
> > > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
> > > sync time anymore.
> > >
> > > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
> > > to have this locked in memory on a small machine. Adding
> > >
> > > rlimit memlock -1
> > >
> > > to ntp.conf should prevent it from trying.
> >
> > I wonder if we should add a heuristic in ntpd to auto-tune for this.
>
> I'd be loudly in favour of this - people should be able to set things
> explicitly, but we really should be defaulting more things based on
> current memory and or relative compute level.
Speaking as the devil's adocate: why should I care about machines with
16MB as ntp author in 2021? Getting swapped out is a very real risk even
on larger machines and ntpd by nature is timing sensitive. So what
justifies the added complexity of such an option or tuning heuristic?
Joerg
From: David Brownlee <abs@absd.org>
To: Joerg Sonnenberger <joerg@bec.de>
Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Fri, 7 Jan 2022 15:42:58 +0000
On Thu, 30 Dec 2021 at 23:35, Joerg Sonnenberger <joerg@bec.de> wrote:
>
> On Thu, Dec 30, 2021 at 11:10:02PM +0000, David Brownlee wrote:
> > The following reply was made to PR port-atari/56582; it has been noted by GNATS.
> >
> > From: David Brownlee <abs@absd.org>
> > To: Jason Thorpe <thorpej@me.com>
> > Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org,
> > gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
> > Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
> > Date: Thu, 30 Dec 2021 23:05:31 +0000
> >
> > On Thu, 30 Dec 2021 at 14:49, Jason Thorpe <thorpej@me.com> wrote:
> > >
> > > > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
> > >
> > > > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
> > > > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
> > > > sync time anymore.
> > > >
> > > > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
> > > > to have this locked in memory on a small machine. Adding
> > > >
> > > > rlimit memlock -1
> > > >
> > > > to ntp.conf should prevent it from trying.
> > >
> > > I wonder if we should add a heuristic in ntpd to auto-tune for this.
> >
> > I'd be loudly in favour of this - people should be able to set things
> > explicitly, but we really should be defaulting more things based on
> > current memory and or relative compute level.
>
> Speaking as the devil's adocate: why should I care about machines with
> 16MB as ntp author in 2021? Getting swapped out is a very real risk even
> on larger machines and ntpd by nature is timing sensitive. So what
> justifies the added complexity of such an option or tuning heuristic?
... and that would be a perfectly legitimate position to take. If the
heuristic is easy and accepted o[stream then I think it's worthwhile,
if it's too complicated, or just rejected upstream then probably not.
I'd be inclined to have something along the lines of a sysctl value
which would indicate whether "low memory behaviour" should be
triggered, which could be picked up by a default ntp conf, or if rc.d
should run makemandb.
Interestingly nfp.conf indicates that form rlimit memlock "The default
is 32 megabytes on non-Linux machines, and -1 under Linux", so I
wonder if we should just look to default to -1 on shipping and allow
people to tweak away from that
David
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.