NetBSD Problem Report #57185

From www@netbsd.org  Mon Jan 16 20:12:54 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id AD3811A9239
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 16 Jan 2023 20:12:54 +0000 (UTC)
Message-Id: <20230116201253.04F651A923A@mollari.NetBSD.org>
Date: Mon, 16 Jan 2023 20:12:53 +0000 (UTC)
From: jspath55@gmail.com
Reply-To: jspath55@gmail.com
To: gnats-bugs@NetBSD.org
Subject: Python build fails on 10_BETA due to no entropy on Atom CPU system
X-Send-Pr-Version: www-1.0

>Number:         57185
>Category:       kern
>Synopsis:       Python build fails on 10_BETA due to no entropy on Atom CPU system
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          doc-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jan 16 20:15:00 +0000 2023
>Closed-Date:    Mon Aug 21 03:07:46 +0000 2023
>Last-Modified:  Mon Aug 21 03:07:46 +0000 2023
>Originator:     Jim Spath
>Release:        10_BETA
>Organization:
>Environment:
NetBSD nnnn 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Sat Dec 31 04:55:53 UTC 2022  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC i386 i386 Intel 686-class NetBSD
>Description:
After an upgrade from NetBSD 9 to 10_BETA, a build of python3.10 stalled with no obvious feedback.
On boot, the system showed:

Jan  3 21:07:29 nnnn /netbsd: [   1.0065252] WARNING: system needs entropy for security; see entropy(7)

There were console screen messages about the block which I did not see right away.

One message about the python hang appeared in /var/log/messages during a shutdown:

Jan 11 18:03:44 nnnn /netbsd: [ 176217.1118082] entropy: pid 19289 (python) blocking due to lack of entropy
Jan 11 18:03:44 nnnn /netbsd: [ 339845.8159510] syncing disks... done
Jan 11 18:03:44 nnnn /netbsd: [ 339845.8259565] unmounted procfs on /proc type procfs

After researching, I found suggested workarounds here:

http://mail-index.netbsd.org/current-users/2021/06/14/msg041118.html
http://mail-index.netbsd.org/current-users/2021/06/15/msg041128.html 

After creating a valid seed, the python build completed.
Boot messages show:

Jan 11 18:03:44 nnnn /netbsd: [  15.5957645] entropy: ready

>How-To-Repeat:
Install or upgrade to NetBSD 10 beta with a CPU similar to:

Intel(R) Atom(TM) CPU D425   @ 1.80GHz

Build Python 3.10 from source (i.e. pkgsrc)
>Fix:
Several workarounds exist, such as overriding the system entropy status, or creating a seed file on a compliant system and importing.
I opened this PR as the build stalling reason was unclear at first (no immediate user feedback).

>Release-Note:

>Audit-Trail:
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/57185 CVS commit: src
Date: Fri, 30 Jun 2023 21:42:06 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Fri Jun 30 21:42:06 UTC 2023

 Modified Files:
 	src/share/man/man7: entropy.7
 	src/sys/kern: kern_clock.c kern_entropy.c

 Log Message:
 entropy(9): Reintroduce netbsd<=9 time-delta estimator for unblocking.

 The system will (in a subsequent change) by default block for this
 condition before almost all of userland is running (including
 /etc/rc.d/sshd key generation).  That way, a never-blocking
 getentropy(3) API will never return any data without at least
 best-effort entropy like netbsd<=9 did to applications except in
 single-user mode (where you have to be careful about everything
 anyway) or in the few processes that run before a seed can even be
 loaded (where blocking indefinitely, e.g. when generating a stack
 protector cookie in libc, could pose a severe availability problem
 that can't be configured away, but where the security impact is low).

 However, (in another subsequent change) we will continue to use
 _only_ HWRNG driver estimates and seed estimates, and _not_
 time-delta estimator, for _warning_ about security in motd, daily
 security report, etc.  And if HWRNG/seed provides enough entropy
 before time-delta estimator does, that will unblock /dev/random too.

 The result is:

 - Machines with HWRNG or seed won't warn about entropy and will
   essentially never block -- even on first boot without a seed, it
   will take only as long as the fastest HWRNG to unblock.

 - Machines with neither HWRNG nor seed:
   . will warn about entropy, giving feedback about security;
     and
   . will avoid returning anything more predictable than netbsd<=9;
     but
   . won't block (much) longer than netbsd<=9 would (and won't block
     again after blocking once, except with kern.entropy.depletion=1 for
     testing).

   (The threshold for unblocking is now somewhat higher than before:
   512 samples that pass the time-delta estimator, rather than 80 as
   it used to be.)

   And, of course, adding a seed (or HWRNG) will prevent both warnings
   and blocking.

 The mechanism is:

 1. /dev/random will block until _either_

    (a) enough bits of entropy (256) from reliable sources have been
        added to the pool, _or_

    (b) enough samples have been added from any sources (512), passing
        the old time-delta entropy estimator, that the possible
        security benefit doesn't justify holding up availability any
        longer (`best effort'), except on systems with higher security
        requirements like securelevel=2 which can disable non-HWRNG,
        non-seed sources with rndctl_flags in rc.conf(5).

 2. dmesg will report `entropy: ready' when 1(a) is satisfied, but if
    1(b) is satisfied first, it will report `entropy: best effort', so
    the concise log messages will reflect the timing and whether in
    any period of time any of the system might be relying on best
    effort entropy.

 3. The sysctl knob kern.entropy.needed (and the ioctl RNDGETPOOLSTAT
    variable rndpoolstat_t::added) still reflects the number of bits
    of entropy from reliable sources, so we can still use this to
    suggest regenerating ssh keys.

    This matters on platforms that can only be reached, after flashing
    an installation image, by sshing in over a (private) network, like
    small network appliances or remote virtual machines without
    (interactive) serial consoles.  If we blocked indefinitely at boot
    when generating ssh keys, such platforms would be unusable.  This
    way, platforms are usable, but operators can still be advised at
    login time to regenerate keys as soon as they can actually load
    entropy onto the system, e.g. with rndctl(8) on a seed file copied
    from a local machine over the (private) network.

 4. On machines without HWRNG, using a seed file still suppresses
    warnings for users who need more confident security.  But it is no
    longer necessary for availability.

 This is a compromise between availability and security:

 - The security mechanism of blocking indefinitely on machines without
   HWRNG hurts availability too much, as painful experience over the
   multiple years since I made the mistake of introducing it have
   shown.  (Sorry!)

 - The other main alternative, not having a blocking path at all (as I
   pushed for, and as OpenBSD has done for a long time) could
   potentially reduce security vs netbsd<=9, and would run against the
   expectations set by many popular operating systems to the severe
   detriment of public perception of NetBSD security.

 Even though we can't _confidently_ assess enough entropy from, e.g.,
 sampling interrupt timings, this is the traditional behaviour that
 most operating systems provide -- and the result here is a net
 nondecrease in security over netbsd<=9, because all paths from the
 entropy pool to userland now have at least as high a standard before
 returning data as they did in netbsd<=9.

 PR kern/55641
 PR pkg/55847
 PR kern/57185
 https://mail-index.netbsd.org/current-users/2020/09/02/msg039470.html
 https://mail-index.netbsd.org/current-users/2020/11/21/msg039931.html
 https://mail-index.netbsd.org/current-users/2020/12/05/msg040019.html

 XXX pullup-10


 To generate a diff of this commit:
 cvs rdiff -u -r1.8 -r1.9 src/share/man/man7/entropy.7
 cvs rdiff -u -r1.148 -r1.149 src/sys/kern/kern_clock.c
 cvs rdiff -u -r1.61 -r1.62 src/sys/kern/kern_entropy.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: gutteridge@NetBSD.org
State-Changed-When: Sun, 20 Aug 2023 19:41:20 +0000
State-Changed-Why:
Is this now addressed, from your perspective?

From: Jim Spath <jspath55@gmail.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/57185 (Python build fails on 10_BETA due to no entropy on
 Atom CPU system)
Date: Sun, 20 Aug 2023 19:26:48 -0400

  > Is this now addressed, from your perspective?

 Quite so. Coincidentally, I just upgraded to an August beta build and
 compiled Python 3.11 with no issues. The entropy paths are documented
 well enough.
 Thanks.

 On Sun, Aug 20, 2023 at 3:41=E2=80=AFPM <gutteridge@netbsd.org> wrote:
 >
 > Synopsis: Python build fails on 10_BETA due to no entropy on Atom CPU sys=
 tem
 >
 > State-Changed-From-To: open->feedback
 > State-Changed-By: gutteridge@NetBSD.org
 > State-Changed-When: Sun, 20 Aug 2023 19:41:20 +0000
 > State-Changed-Why:
 > Is this now addressed, from your perspective?
 >
 >

State-Changed-From-To: feedback->closed
State-Changed-By: gutteridge@NetBSD.org
State-Changed-When: Mon, 21 Aug 2023 03:07:46 +0000
State-Changed-Why:
Submitter fine to close. Thanks for the PR!

>Unformatted:

Home	PR Database Search