NetBSD Problem Report #55659

From gson@gson.org  Mon Sep 14 14:52:40 2020
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 5B7471A9239
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 14 Sep 2020 14:52:40 +0000 (UTC)
Message-Id: <20200914145235.0C8F4253EDA@guava.gson.org>
Date: Mon, 14 Sep 2020 17:52:35 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: ssh-keygen may generate predictable keys
X-Send-Pr-Version: 3.95

>Number:         55659
>Category:       security
>Synopsis:       ssh-keygen may generate predictable keys
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    security-officer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Sep 14 14:55:00 +0000 2020
>Last-Modified:  Thu Sep 24 16:40:01 +0000 2020
>Originator:     Andreas Gustafsson
>Release:        NetBSD 9.0, -current, and others
>Organization:

>Environment:
System: NetBSD
Architecture: x86_64
Machine: amd64
>Description:

In NetBSD 9.0, ssh-keygen(1) uses /dev/urandom as its randomness
source, and will therefore silently generate potentially guessable
keys when run on a system that lacks entropy.  In -current, it uses
the kern.arandom sysctl, with the same result.

It makes no sense that NetBSD has strict criteria for when it
considers the system to have sufficient entropy, but then completely
ignores those criteria when performing the very operations where
having entropy is most important.

Other cryptographic functions such as the generation of SSH and TLS
session keys have similar issues, but I'm focusing on ssh-keygen
because I think it is the clearest example of a case where using
/dev/urandom is not acceptable.

I have been told NetBSD has behaved this way for decades, but that
does not make it any less of a bug.  In theory, fixing it is easy:
make ssh-keygen use /dev/random instead of /dev/urandom (or their
respective equivalents).  In practice, it's harder, because unless
other issues are fixed first, making this change can have a severe
usability impact, such as causing newly installed systems to
mysteriously hang on first boot as a result of ssh-keygen silently
blocking on /dev/random.

>How-To-Repeat:

Run ssh-keygen under ktrace and inspect the system calls issued.

>Fix:

I'm proposing the following rough plan:

First, make the randomness source of ssh-keygen configurable, so that
those who prefer it to block rather than generate predictable keys can
choose this behavior even if it is not yet the default.  This will
also aid testing of the following steps.

Second, make sure that most systems have sufficient entropy, for
example by creating a random seed file at installation or upgrade
time.  Also, make sure that if ssh-keygen nonetheless ends up unable
to generate keys due to a lack of entropy, a clear error message is
issued and points the user to a reasonable way of supplying the
entropy manually.

Finally, enable the configuration option of the first step by default.

>Audit-Trail:
From: Taylor R Campbell <campbell@mumble.net>
To: gnats-bugs@netbsd.org
Cc: security-officer@netbsd.org, gnats-admin@netbsd.org, security-alert@netbsd.org
Subject: Re: security/55659: ssh-keygen may generate predictable keys
Date: Tue, 22 Sep 2020 22:37:16 +0000

 > Date: Mon, 14 Sep 2020 14:55:00 +0000 (UTC)
 > From: gson@gson.org (Andreas Gustafsson)
 > 
 > First, make the randomness source of ssh-keygen configurable, so that
 > those who prefer it to block rather than generate predictable keys can
 > choose this behavior even if it is not yet the default.  This will
 > also aid testing of the following steps.

 We already have a mechanism for achieving the same effect:

 head -c 1 < /dev/random > /dev/null
 ssh-keygen ...

 I don't see much value in adding nonstandard options to ssh-keygen
 just for this.

 > Second, make sure that most systems have sufficient entropy, for
 > example by creating a random seed file at installation or upgrade
 > time.  

 We already do this example.

 > Also, make sure that if ssh-keygen nonetheless ends up unable
 > to generate keys due to a lack of entropy, a clear error message is
 > issued and points the user to a reasonable way of supplying the
 > entropy manually.

 I agree that we should make this information clear to the operator.

 However, it's not obvious to me that ssh-keygen is the right place to
 do this -- and if we extended it to every program that might need to
 generate keys, I expect we would overwhelm the user with warning
 fatigue.

 There are already too many warnings in kern_entropy.c (and users
 fatigued by them), despite my efforts to be judicious about adding
 them and to rate-limit them, and I'd like to cut down on them.

 I think a better way forward would be:

 1. prominent but concise text in afterboot(8) on the subject
 2. an optional addendum to the motd if there is not enough entropy
 3. a note in the /etc/security report if there is not enough entropy

From: Andreas Gustafsson <gson@gson.org>
To: Taylor R Campbell <campbell@mumble.net>, gnats-bugs@netbsd.org
Cc: 
Subject: Re: security/55659: ssh-keygen may generate predictable keys
Date: Wed, 23 Sep 2020 11:21:58 +0300

 Taylor R Campbell wrote:
 >  > First, make the randomness source of ssh-keygen configurable, so that
 >  > those who prefer it to block rather than generate predictable keys can
 >  > choose this behavior even if it is not yet the default.  This will
 >  > also aid testing of the following steps.
 >  
 >  We already have a mechanism for achieving the same effect:
 >  
 >  head -c 1 < /dev/random > /dev/null
 >  ssh-keygen ...
 >  
 >  I don't see much value in adding nonstandard options to ssh-keygen
 >  just for this.

 I don't mean a command line option to ssh-keygen to select blocking
 behavior, I mean a way to make it happen by default.  For example, a
 build-time option in the form of a make variable that you can set in
 /etc/mk.conf, or perhaps an installation-time choice.

 >  > Second, make sure that most systems have sufficient entropy, for
 >  > example by creating a random seed file at installation or upgrade
 >  > time.  
 >  
 >  We already do this example.

 We already create a seed file at installation time, but in many cases
 it has an entropy estimate of zero, so it doesn't actually actually help.
 I believe Martin is working on fixing this.

 >  > Also, make sure that if ssh-keygen nonetheless ends up unable
 >  > to generate keys due to a lack of entropy, a clear error message is
 >  > issued and points the user to a reasonable way of supplying the
 >  > entropy manually.
 >  
 >  I agree that we should make this information clear to the operator.
 >  
 >  However, it's not obvious to me that ssh-keygen is the right place to
 >  do this

 I didn't say the message would necessarily be issued by ssh-keygen
 itself.

 > -- and if we extended it to every program that might need to
 >  generate keys, I expect we would overwhelm the user with warning
 >  fatigue.

 I'm not proposing adding warnings, I am proposing making the lack of
 entropy an error.  This can mean either making ssh-keygen exit or
 making it block, but in either case, since it has not (yet) performed
 its intended function, I would consider any messages issued in this
 situation to be error messages, not warning messages.

 >  There are already too many warnings in kern_entropy.c (and users
 >  fatigued by them), despite my efforts to be judicious about adding
 >  them and to rate-limit them, and I'd like to cut down on them.

 This should be addressed by providing entropy, not by hiding the lack
 of it.

 >  I think a better way forward would be:
 >  
 >  1. prominent but concise text in afterboot(8) on the subject
 >  2. an optional addendum to the motd if there is not enough entropy
 >  3. a note in the /etc/security report if there is not enough entropy

 And continue to generate predictable keys?
 -- 
 Andreas Gustafsson, gson@gson.org

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: security/55659: ssh-keygen may generate predictable keys
Date: Wed, 23 Sep 2020 10:28:13 +0200

 On Wed, Sep 23, 2020 at 08:25:01AM +0000, Andreas Gustafsson wrote:
 >  We already create a seed file at installation time, but in many cases
 >  it has an entropy estimate of zero, so it doesn't actually actually help.
 >  I believe Martin is working on fixing this.

 Yes, but want to get all sysinst changes that need pullup to 9.1 out of the
 way first.

 Martin

From: Taylor R Campbell <campbell@mumble.net>
To: Andreas Gustafsson <gson@gson.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: security/55659: ssh-keygen may generate predictable keys
Date: Wed, 23 Sep 2020 17:53:58 +0000

 > Date: Wed, 23 Sep 2020 11:21:58 +0300
 > From: Andreas Gustafsson <gson@gson.org>
 >=20
 > >  There are already too many warnings in kern_entropy.c (and users
 > >  fatigued by them), despite my efforts to be judicious about adding
 > >  them and to rate-limit them, and I'd like to cut down on them.
 >=20
 > This should be addressed by providing entropy, not by hiding the lack
 > of it.

 I'm not saying we should hide the lack of entropy; I'm saying we
 should report it judiciously so that users are not inundated with
 warnings about essentially the same thing over and over again -- that
 leads users to ignore warnings.

 > >  I think a better way forward would be:
 > > =20
 > >  1. prominent but concise text in afterboot(8) on the subject
 > >  2. an optional addendum to the motd if there is not enough entropy
 > >  3. a note in the /etc/security report if there is not enough entropy
 >=20
 > And continue to generate predictable keys?

 Correct.  The point is that I don't think ssh-keygen is a useful place
 at which to make this decision.  As I wrote in rnd(4):

      Applications should read from /dev/urandom when they need randomly
      generated data, e.g. key material for cryptography or seeds for
      simulations.

      Systems should be engineered to judiciously read at least once from
      /dev/random at boot before running any services that talk to the inter=
 net
      or otherwise require cryptography, in order to avoid generating keys
      predictably.

 What I'm trying to do is not just answer the nearsighted question
 `what remedial action should be taken at the point in every program
 when it tries to generate keys and there turns out to be no entropy?'.

 Rather, I'm trying to broaden the scope of the question to a matter of
 system integration so that we can usefully provoke operator
 intervention before a security problem happens -- whether the security
 problem is predictable keys or denial of service by blocking forever.

 Two other suggestions that came up privately are:

 4. add an rc.conf variable to make /etc/rc.d/random_seed halt boot and
    return to single-user mode if there is no entropy

 5. add an rc.conf variable to make /etc/rc.d/sshd refuse to generate
    keys if there's no entropy

 Neither of these requires any changes to ssh-keygen, and they are both
 focused on points of system integration that an operator is likely to
 be paying attention to.

From: Andreas Gustafsson <gson@gson.org>
To: Taylor R Campbell <campbell@mumble.net>, gnats-bugs@netbsd.org
Cc: netbsd-bugs@NetBSD.org
Subject: Re: security/55659: ssh-keygen may generate predictable keys
Date: Thu, 24 Sep 2020 19:39:28 +0300

 Taylor R Campbell wrote:
 >  > >  I think a better way forward would be:
 >  > > 
 >  > >  1. prominent but concise text in afterboot(8) on the subject
 >  > >  2. an optional addendum to the motd if there is not enough entropy
 >  > >  3. a note in the /etc/security report if there is not enough entropy
 >  >
 >  > And continue to generate predictable keys?
 >  
 >  Correct.  The point is that I don't think ssh-keygen is a useful place
 >  at which to make this decision.

 I didn't say that the decision necessarily has to be made in ssh-keygen
 itself.  For example, I'm currently testing patches to make openssl
 use getrandom(..., 0), which would address both ssh-keygen and many
 other programs that generate keys.

 > As I wrote in rnd(4):
 >  
 >       Applications should read from /dev/urandom when they need randomly
 >       generated data, e.g. key material for cryptography or seeds for
 >       simulations.

 I strongly disagree with the "key material for cryptography" part of
 this advice.

 >       Systems should be engineered to judiciously read at least once from
 >       /dev/random at boot before running any services that talk to the internet
 >       or otherwise require cryptography, in order to avoid generating keys
 >       predictably.

 Yes, they should, but there are two problems.  First, the system
 called NetBSD is not currently engineered this way, which makes the
 use of /dev/urandom insecure in NetBSD, even though it may be secure
 in a system built on top of NetBSD and engineered according to the
 above.  This is not acceptable - NetBSD is supposed to be a secure
 operating system, not just a base on which secure systems can be
 built.

 Second, even if NetBSD is changed to conform to the above, if that
 mechanism fails (for example, if the system boots to single user mode
 bypassing the read from /dev/random, and the administrator then runs
 ssh-keygen manually), we are back to silently generating predictable
 keys.  In effect, the system "fails open" rather than "failing
 closed".

 >  What I'm trying to do is not just answer the nearsighted question
 >  `what remedial action should be taken at the point in every program
 >  when it tries to generate keys and there turns out to be no entropy?'.
 >
 >  Rather, I'm trying to broaden the scope of the question to a matter of
 >  system integration so that we can usefully provoke operator
 >  intervention before a security problem happens -- whether the security
 >  problem is predictable keys or denial of service by blocking forever.

 I agree that operator intervention should be provoked before the
 problem happens.  And again, this should be a feature of NetBSD, not
 something that has to be added by a third party integrator.

 Regardless of whether one thinks predictable keys or denial of service
 by blocking is the bigger problem, we all want a system where neither
 problem occurs, so we should be able to work together towards that
 goal, for example by improving seeding by the installer, adding more
 entropy sources, provoking operator intervention at boot time, and any
 other means available.

 But there will always remain a possibility that despite all our
 efforts, a system nonetheless ends up without sufficient entropy.
 And I still say that if you run ssh-keygen on such a system, it must
 not generate a predictable key.  I don't care if the decision is
 implemented in ssh-keygen itself, in OpenSSL, or the kernel.  And if
 it also applies to other programs, so much the better.  But I do care
 about the outcome, and while I'm happy to discuss the broader system
 integration issues in the context of this PR, the PR remains
 specifically about what will happen in that error case, and I won't
 consider the PR resolved until that question has been resolved one way
 or the other.

 >  Two other suggestions that came up privately are:
 >  
 >  4. add an rc.conf variable to make /etc/rc.d/random_seed halt boot and
 >     return to single-user mode if there is no entropy
 >  
 >  5. add an rc.conf variable to make /etc/rc.d/sshd refuse to generate
 >     keys if there's no entropy
 >  
 >  Neither of these requires any changes to ssh-keygen, and they are both
 >  focused on points of system integration that an operator is likely to
 >  be paying attention to.

 That would count as "provoking operator intervention before a security
 problem happens".  What do you think the default settings of those
 variables should be?

 [I'm manually CCing this to netbsd-bugs since it won't be posted there
 automatically.]
 -- 
 Andreas Gustafsson, gson@gson.org

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.