NetBSD Problem Report #56582

From www@netbsd.org  Sat Dec 25 23:42:21 2021
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 62C541A9239
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 25 Dec 2021 23:42:21 +0000 (UTC)
Message-Id: <20211225234219.D1D5C1A923C@mollari.NetBSD.org>
Date: Sat, 25 Dec 2021 23:42:19 +0000 (UTC)
From: dross@pobox.com
Reply-To: dross@pobox.com
To: gnats-bugs@NetBSD.org
Subject: port-atari freeze on installation when extracting base.tgz
X-Send-Pr-Version: www-1.0

>Number:         56582
>Category:       port-atari
>Synopsis:       port-atari freeze on installation when extracting base.tgz
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    port-atari-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Dec 25 23:45:00 +0000 2021
>Last-Modified:  Sun Jul 10 01:49:01 +0000 2022
>Originator:     David Ross
>Release:        9.2
>Organization:
>Environment:
>Description:
port-atari 9.2 installation on an atari TT030 freezes during extraction of base.tgz.  The extraction proceeds for a few minutes, but then appears to stop (no HD activity).  Ctrl-C does not work to exit the extraction, the system just appears frozen.  This issue reproduces reliably.

The issue does not happen on 8.2, which I am able to install and run reliably on the same system.

On 9.2, there seems to be an issue with how text is rendered that may (or may not) be related to the install failure.  For example, the 'progress' command doesn't render progress at all.  I would normally be able to see how far set extraction has completed, but it's not possible to do so because 'progress' is broken.
>How-To-Repeat:
Install NetBSD port-atari onto an Atari TT030, mostly as described here:
https://netbsd-ataritt.fandom.com/wiki/NetBSD_on_the_Atari_TT030

Choose to download sets over HTTP.  When the base.tgz set is extracted, observe the system stops responding after a few minutes.

This is a standard install, nothing special.  I can help test, etc. as necessary, please feel free to reach out.
>Fix:

>Release-Note:

>Audit-Trail:
From: David Ross <randomdross00@gmail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sat, 25 Dec 2021 15:57:10 -0800

 (I'm not sure if my reply to this mail will land in the bug text, but
 hopefully that's how it works...)
 I mean to file this under port-atari instead of install.  Let me know
 if it makes sense and I can re-file it there, or feel free to move it
 over if you can.

 On Sat, Dec 25, 2021 at 3:45 PM <gnats-admin@netbsd.org> wrote:
 >
 > Thank you very much for your problem report.
 > It has the internal identification `install/56582'.
 > The individual assigned to look at your
 > report is: install-manager.
 >
 > >Category:       install
 > >Responsible:    install-manager
 > >Synopsis:       port-atari freeze on installation when extracting base.tgz
 > >Arrival-Date:   Sat Dec 25 23:45:00 +0000 2021
 >

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 07:10:13 -0000 (UTC)

 dross@pobox.com writes:

 >>Description:
 >port-atari 9.2 installation on an atari TT030 freezes during extraction of base.tgz.  The extraction proceeds for a few minutes, but then appears to stop (no HD activity).  Ctrl-C does not work to exit the extraction, the system just appears frozen.  This issue reproduces reliably.

 >The issue does not happen on 8.2, which I am able to install and run reliably on the same system.

 Can you give details about the hardware, in particular how much RAM exists ?

 When the system hangs, can you use ALT+LSHIFT+F9 to enter the kernel
 debugger ?

Responsible-Changed-From-To: install-manager->port-atari-maintainer
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Sun, 26 Dec 2021 12:06:56 +0000
Responsible-Changed-Why:
atari (or m68k) kernel bug


From: David Ross <randomdross00@gmail.com>
To: gnats-bugs@netbsd.org
Cc: install-manager@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, 
	David Ross <dross@pobox.com>
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 10:20:57 -0800

 > Can you give details about the hardware, in particular how much RAM exist=
 s ?
 The system has 10MB of ST RAM and 16MB of TT RAM.  I think NetBSD only
 uses the ST RAM for video and floppy, and the TT RAM is what's used
 normally for everything else.

 > When the system hangs, can you use ALT+LSHIFT+F9 to enter the kernel debu=
 gger ?
 I will try this and report back soon.  Thanks!

 On Sat, Dec 25, 2021 at 11:15 PM Michael van Elst <mlelstv@serpens.de> wrot=
 e:
 >
 > The following reply was made to PR install/56582; it has been noted by GN=
 ATS.
 >
 > From: mlelstv@serpens.de (Michael van Elst)
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: install/56582: port-atari freeze on installation when extrac=
 ting base.tgz
 > Date: Sun, 26 Dec 2021 07:10:13 -0000 (UTC)
 >
 >  dross@pobox.com writes:
 >
 >  >>Description:
 >  >port-atari 9.2 installation on an atari TT030 freezes during extraction=
  of base.tgz.  The extraction proceeds for a few minutes, but then appears =
 to stop (no HD activity).  Ctrl-C does not work to exit the extraction, the=
  system just appears frozen.  This issue reproduces reliably.
 >
 >  >The issue does not happen on 8.2, which I am able to install and run re=
 liably on the same system.
 >
 >  Can you give details about the hardware, in particular how much RAM exis=
 ts ?
 >
 >  When the system hangs, can you use ALT+LSHIFT+F9 to enter the kernel
 >  debugger ?
 >

From: David Holland <dholland-gnats@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/56582: port-atari freeze on installation when extracting
 base.tgz
Date: Sun, 10 Jul 2022 01:42:09 +0000

 Also not sent to gnats.

    ------

 From: David Ross <randomdross00@gmail.com>
 To: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
 	netbsd-bugs@netbsd.org, David Ross <dross@pobox.com>
 Subject: Re: install/56582: port-atari freeze on installation when extracting
 	base.tgz
 Date: Sun, 26 Dec 2021 10:44:53 -0800

 I'm unable to break into the kernel debugger using ALT+LSHIFT+F9 after
 booting the 9.2 kernel / sysinst.  This is while sysinst is operating
 normally, before the hang.  So there is probably a different key
 sequence I need to use to get into the kernel debugger on the Atari.
 I'll do a bit of research to see if I can figure that out.

 It's also worth pointing out Izumi Tsutsui's tweet here, describing
 what was observed during a 9.0 RC1 install:
 https://twitter.com/tsutsuii/status/1475065059777064960
 ---
 there were two problems. - "qqq.." is only visible issue. NetBSD/atari
 ite console doesn't support ruled lines but sysinst assumes full
 terminfo (smacs,rmacs,acsc). - sysinst on atari omits "progress bar"
 support due to size restriction. The installation goes background.
 ---

 Izumi's install was apparently successful, though it was an upgrade
 and not a clean install.
 So it sounds like the hang that I'm seeing on my 9.2 install may be
 entirely unrelated to the text display issues / missing progress bar.

 Dave

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 18:57:58 -0000 (UTC)

 randomdross00@gmail.com (David Ross) writes:

 >Izumi's install was apparently successful, though it was an upgrade
 >and not a clean install.
 >So it sounds like the hang that I'm seeing on my 9.2 install may be
 >entirely unrelated to the text display issues / missing progress bar.

 My prime suspect would be memory. 16MB usuable memory could be too tight,
 and NetBSD doesn't handle memory shortage well.

 N.B. if the installation system uses a SMALL kernel, then DDB doesn't
 exist.

From: David Ross <randomdross00@gmail.com>
To: gnats-bugs@netbsd.org
Cc: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org, 
	netbsd-bugs@netbsd.org, David Ross <dross@pobox.com>
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Sun, 26 Dec 2021 15:55:37 -0800

 Thanks!  That's certainly possible.  I've never been able to get my
 TT030 stable with >16MB, however I know Izumi has.  I'll be trying a
 new memory card soon, and hopefully this will help.  I also have some
 experience cross-building netbsd, so I may do that soon and compile
 everything with -Os instead of -O2 in case that gives me just enough
 memory to get past the base.tgz extraction.

 For right now, I'm going to try installing the 9.0 release and see how
 that goes.  I know 8.2 installs and works great, but from there I had
 jumped right to 9.2 without trying the earlier 9.x releases.

 Dave

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/56582: port-atari freeze on installation when extracting
 base.tgz
Date: Sun, 10 Jul 2022 01:22:48 +0000

 Not sent to gnats. (Will shift to the correct position in the
 discussion in the database.)

    ------

 From: David Ross <randomdross00@gmail.com>
 To: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
 	netbsd-bugs@netbsd.org
 Subject: Re: install/56582: port-atari freeze on installation when extracting
 	base.tgz
 Date: Wed, 29 Dec 2021 17:20:25 -0800

 I was able to complete a NetBSD 9.2 install by booting with an 8.2
 kernel.  However, on first boot I get:
 Dec 29 10:56:44 tt030 ntpd[373]: mlockall(): Cannot allocate memory
 ...and a few postfix related errors.  You can see it all in this
 Twitter thread: https://twitter.com/nbtt030/status/1476355335493324802
  The system does ultimately come up, though I'm not sure yet if it
 runs stable.

 So this tells me the install problem is not simply a matter of running
 out of memory during the initial boot / sysinst process.  The ntpd
 issue happens on normal boot with the 9.2 kernel, so the system will
 have access to swap.  So it seems there may be a memory related
 problem that crept into the kernel between 8.2 and 9.0.

 This is not something I've observed previously -- NetBSD 8.2 runs fine
 on this machine.

 Dave

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Thu, 30 Dec 2021 07:08:15 -0000 (UTC)

 randomdross00@gmail.com (David Ross) writes:

 >I was able to complete a NetBSD 9.2 install by booting with an 8.2
 >kernel.  However, on first boot I get:
 >Dec 29 10:56:44 tt030 ntpd[373]: mlockall(): Cannot allocate memory

 mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
 you can ignore it. Worst thing is that ntpd gets swapped out and cannot
 sync time anymore.

 ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
 to have this locked in memory on a small machine. Adding

 rlimit memlock -1

 to ntp.conf should prevent it from trying.

From: Jason Thorpe <thorpej@me.com>
To: gnats-bugs@netbsd.org
Cc: port-atari-maintainer@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org,
 dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting
 base.tgz
Date: Thu, 30 Dec 2021 07:48:57 -0700

 > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> =
 wrote:

 > mlockall() returns ENOMEM when the system- or per-process-limit is =
 exceeded,
 > you can ignore it. Worst thing is that ntpd gets swapped out and =
 cannot
 > sync time anymore.
 >=20
 > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
 > to have this locked in memory on a small machine. Adding
 >=20
 > rlimit memlock -1
 >=20
 > to ntp.conf should prevent it from trying.

 I wonder if we should add a heuristic in ntpd to auto-tune for this.

 -- thorpej

From: David Ross <randomdross00@gmail.com>
To: Jason Thorpe <thorpej@me.com>
Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Thu, 30 Dec 2021 10:44:22 -0800

 Thanks!  I thought ntpd might be useful for keeping the time sync'd,
 but I didn't realize how much memory it's eating.  And probably
 ntpdate is sufficient if I just want to keep the system time in sync
 with Internet time (?).  I'll stop running ntpd now.

 It appears that right now at least the 9.2 kernel may in fact be
 stable on this machine.  That makes me have faith in Michael van
 Elst's suggestion that the freeze during install could be simply due
 to a lack of memory.

 I'll be experimenting with a few solutions going forward:
   - Getting a memory card for my TT030 that is >16MB and runs stable.
   - Cross-building netbsd for atari with -Os and using the sysinst.fs
 and ataritt kernel that it generates for the install.  I can also
 remove some things from my build of the ataritt kernel, which could
 help.

 It might make sense to use this PR to track any potential changes that
 could help get 9.2 to install successfully on a 16MB TT030.  The most
 common RAM board for the TT (the Atari branded board) will do 16MB
 max, and in my experience many >16MB RAM boards don't run stable with
 netbsd.

 Dave

 On Thu, Dec 30, 2021 at 6:49 AM Jason Thorpe <thorpej@me.com> wrote:
 >
 >
 > > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
 >
 > > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
 > > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
 > > sync time anymore.
 > >
 > > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
 > > to have this locked in memory on a small machine. Adding
 > >
 > > rlimit memlock -1
 > >
 > > to ntp.conf should prevent it from trying.
 >
 > I wonder if we should add a heuristic in ntpd to auto-tune for this.
 >
 > -- thorpej
 >

From: David Brownlee <abs@absd.org>
To: Jason Thorpe <thorpej@me.com>
Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Thu, 30 Dec 2021 23:05:31 +0000

 On Thu, 30 Dec 2021 at 14:49, Jason Thorpe <thorpej@me.com> wrote:
 >
 > > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
 >
 > > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
 > > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
 > > sync time anymore.
 > >
 > > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
 > > to have this locked in memory on a small machine. Adding
 > >
 > > rlimit memlock -1
 > >
 > > to ntp.conf should prevent it from trying.
 >
 > I wonder if we should add a heuristic in ntpd to auto-tune for this.

 I'd be loudly in favour of this - people should be able to set things
 explicitly, but we really should be defaulting more things based on
 current memory and or relative compute level.

 David

From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc: port-atari-maintainer@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org, dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting
 base.tgz
Date: Fri, 31 Dec 2021 00:35:43 +0100

 On Thu, Dec 30, 2021 at 11:10:02PM +0000, David Brownlee wrote:
 > The following reply was made to PR port-atari/56582; it has been noted by GNATS.
 > 
 > From: David Brownlee <abs@absd.org>
 > To: Jason Thorpe <thorpej@me.com>
 > Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org, 
 > 	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
 > Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
 > Date: Thu, 30 Dec 2021 23:05:31 +0000
 > 
 >  On Thu, 30 Dec 2021 at 14:49, Jason Thorpe <thorpej@me.com> wrote:
 >  >
 >  > > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
 >  >
 >  > > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
 >  > > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
 >  > > sync time anymore.
 >  > >
 >  > > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
 >  > > to have this locked in memory on a small machine. Adding
 >  > >
 >  > > rlimit memlock -1
 >  > >
 >  > > to ntp.conf should prevent it from trying.
 >  >
 >  > I wonder if we should add a heuristic in ntpd to auto-tune for this.
 >  
 >  I'd be loudly in favour of this - people should be able to set things
 >  explicitly, but we really should be defaulting more things based on
 >  current memory and or relative compute level.

 Speaking as the devil's adocate: why should I care about machines with
 16MB as ntp author in 2021? Getting swapped out is a very real risk even
 on larger machines and ntpd by nature is timing sensitive. So what
 justifies the added complexity of such an option or tuning heuristic?

 Joerg

From: David Brownlee <abs@absd.org>
To: Joerg Sonnenberger <joerg@bec.de>
Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
Date: Fri, 7 Jan 2022 15:42:58 +0000

 On Thu, 30 Dec 2021 at 23:35, Joerg Sonnenberger <joerg@bec.de> wrote:
 >
 > On Thu, Dec 30, 2021 at 11:10:02PM +0000, David Brownlee wrote:
 > > The following reply was made to PR port-atari/56582; it has been noted by GNATS.
 > >
 > > From: David Brownlee <abs@absd.org>
 > > To: Jason Thorpe <thorpej@me.com>
 > > Cc: gnats-bugs@netbsd.org, port-atari-maintainer@netbsd.org,
 > >       gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, dross@pobox.com
 > > Subject: Re: install/56582: port-atari freeze on installation when extracting base.tgz
 > > Date: Thu, 30 Dec 2021 23:05:31 +0000
 > >
 > >  On Thu, 30 Dec 2021 at 14:49, Jason Thorpe <thorpej@me.com> wrote:
 > >  >
 > >  > > On Dec 30, 2021, at 12:10 AM, Michael van Elst <mlelstv@serpens.de> wrote:
 > >  >
 > >  > > mlockall() returns ENOMEM when the system- or per-process-limit is exceeded,
 > >  > > you can ignore it. Worst thing is that ntpd gets swapped out and cannot
 > >  > > sync time anymore.
 > >  > >
 > >  > > ntpd uses about 7MB RAM (about 3MB shared). You probably don't want
 > >  > > to have this locked in memory on a small machine. Adding
 > >  > >
 > >  > > rlimit memlock -1
 > >  > >
 > >  > > to ntp.conf should prevent it from trying.
 > >  >
 > >  > I wonder if we should add a heuristic in ntpd to auto-tune for this.
 > >
 > >  I'd be loudly in favour of this - people should be able to set things
 > >  explicitly, but we really should be defaulting more things based on
 > >  current memory and or relative compute level.
 >
 > Speaking as the devil's adocate: why should I care about machines with
 > 16MB as ntp author in 2021? Getting swapped out is a very real risk even
 > on larger machines and ntpd by nature is timing sensitive. So what
 > justifies the added complexity of such an option or tuning heuristic?

 ... and that would be a perfectly legitimate position to take. If the
 heuristic is easy and accepted o[stream then I think it's worthwhile,
 if it's too complicated, or just rejected upstream then probably not.

 I'd be inclined to have something along the lines of a sysctl value
 which would indicate whether "low memory behaviour" should be
 triggered, which could be picked up by a default ntp conf, or if rc.d
 should run makemandb.

 Interestingly nfp.conf indicates that form rlimit memlock "The default
 is 32 megabytes on non-Linux machines, and -1 under Linux", so I
 wonder if we should just look to default to -1 on shipping and allow
 people to tweak away from that

 David

>Unformatted:

Home	PR Database Search