NetBSD Problem Report #57456

From www@netbsd.org  Thu Jun  8 09:01:13 2023
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 5A9F21A923D
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  8 Jun 2023 09:01:13 +0000 (UTC)
Message-Id: <20230608090112.A4EB61A9241@mollari.NetBSD.org>
Date: Thu,  8 Jun 2023 09:01:12 +0000 (UTC)
From: abs@absd.org
Reply-To: abs@absd.org
To: gnats-bugs@NetBSD.org
Subject: ftp fails for https in netbsd-10 due to missing certificates
X-Send-Pr-Version: www-1.0

>Number:         57456
>Category:       bin
>Synopsis:       ftp fails for https in netbsd-10 due to missing certificates
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Jun 08 09:05:00 +0000 2023
>Closed-Date:    Tue Nov 28 01:48:12 +0000 2023
>Last-Modified:  Tue Nov 28 01:48:12 +0000 2023
>Originator:     David Brownlee
>Release:        NetBSD-10
>Organization:
-
>Environment:
NetBSD iris.absd.org 10.0_BETA NetBSD 10.0_BETA (GENERIC) #0: Mon May 15 10:40:13 UTC 2023  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64

>Description:
ftp on netbsd-10 has started to fail against https URLs with the following error

4294967295:error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:/usr/src/crypto/external/bsd/openssl/dist/ssl/statem/statem_clnt.c:1919:

Running "env FTPSSLNOVERIFY=1 ftp ..." works around the issue
>How-To-Repeat:
Try to use ftp on the latest netbsd-10 to download an https URL
>Fix:
Add basic root ssl certificates to netbsd-10?

>Release-Note:

>Audit-Trail:
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing certificates
Date: Thu, 8 Jun 2023 11:44:13 -0000 (UTC)

 abs@absd.org writes:

 >Add basic root ssl certificates to netbsd-10?

 There are no "basic" root ssl certificates.

 You may install the cert package of your choice and update
 as necessary to track changes. I also suggest to make this
 part of the sysinst process (after seeding the random generator
 where necessary).


From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Thu, 8 Jun 2023 13:56:20 +0200

 On Thu, Jun 08, 2023 at 11:45:02AM +0000, Michael van Elst wrote:
 >  You may install the cert package of your choice and update
 >  as necessary to track changes. I also suggest to make this
 >  part of the sysinst process (after seeding the random generator
 >  where necessary).

 That is too late.

 One of the options to install the base sets (and actually one of my
 personal favorites for many install media, especially as it gets me
 "latest" binaries instead of the ones that came with the [maybe older]
 installer) is to download them via https - which relies on ftp(1) on
 the install medium being able to download them.

 So some sets of certificates *must* be bundled with the installers
 or that feature in ftp(1) needs to be turned off by default again.

 Martin

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing certificates
Date: Thu, 8 Jun 2023 17:22:16 -0000 (UTC)

 martin@duskware.de (Martin Husemann) writes:

 >The following reply was made to PR bin/57456; it has been noted by GNATS.

 >From: Martin Husemann <martin@duskware.de>
 >To: gnats-bugs@netbsd.org
 >Cc: 
 >Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 > certificates
 >Date: Thu, 8 Jun 2023 13:56:20 +0200

 > On Thu, Jun 08, 2023 at 11:45:02AM +0000, Michael van Elst wrote:
 > >  You may install the cert package of your choice and update
 > >  as necessary to track changes. I also suggest to make this
 > >  part of the sysinst process (after seeding the random generator
 > >  where necessary).
 > 
 > That is too late.

 > So some sets of certificates *must* be bundled with the installers
 > or that feature in ftp(1) needs to be turned off by default again.

 Wouldn't that be even "later" ?

 If you want to download from an unproven source, the installer can
 tell ftp to do that without changing defaults.

 If you want more trust, you could sign the sets (and deliver a cert
 with the installer for validation). This also works for other kinds
 of downloads.

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Thu, 8 Jun 2023 19:37:26 +0200

 On Thu, Jun 08, 2023 at 05:25:01PM +0000, Michael van Elst wrote:
 >  If you want to download from an unproven source, the installer can
 >  tell ftp to do that without changing defaults.

 I would like to have a workable plan before we make changes like this
 to very basic utilities. And ideally do all steps required at the
 same time. If I now add an override to sysinst we certainly will forget
 about it should we ever add trust anchors to the installer media.

 Martin

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing certificates
Date: Thu, 8 Jun 2023 18:35:08 -0000 (UTC)

 martin@duskware.de (Martin Husemann) writes:

 > On Thu, Jun 08, 2023 at 05:25:01PM +0000, Michael van Elst wrote:
 > >  If you want to download from an unproven source, the installer can
 > >  tell ftp to do that without changing defaults.
 > 
 > I would like to have a workable plan before we make changes like this
 > to very basic utilities.

 Perfect, we can easily revert ftp to its unconditionally insecure behaviour,
 so nobody forgets it the next 10 years. :)




From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Thu, 8 Jun 2023 20:53:01 +0200

 On Thu, Jun 08, 2023 at 06:40:01PM +0000, Michael van Elst wrote:
 >  Perfect, we can easily revert ftp to its unconditionally insecure behaviour,
 >  so nobody forgets it the next 10 years. :)

 I'm not arguing secure vs. insecure - but plain broken (both in the default
 install and in the installers) as it is now is not a good step in between.
 With a bit better planning it could have been avoided, but on the other
 hand the planning should not prevent the security fix for ever. So I do
 understand both sides, and we should quickly find a good plan to move
 forward.

 I am not sure your initial suggestion (let the end user pick any trust
 anchor set and leave the updating problem to them too) is the best, but
 it may be the only one workable now w/o getting deeply into net
 politicis or having to make promises from TNF side that we would better
 stay away from.

 For sysinst I don't want to show confusing warnings about untrusted
 downloads or missing verification - even if true.
 But I also don't like to return to the old state (by setting
 sslnoverify). Open to any suggestions (and probably this should not be
 in this particular PR, but better be discussed on some mailing list).

 Martin

From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Thu, 8 Jun 2023 23:02:02 +0200

 Can we add the concrete certificate(s) needed for nbftp or cdn or
 whatever's used in the installer right now, or will those expire too
 quickly (compared to the lifetime of a NetBSD 10 install image)?
  Thomas

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Thu, 8 Jun 2023 23:07:54 +0200

 Actually the installer issue is quite simple: I'll make it check
 for any root certs in it's /etc/openssl/certs and if none is found
 set FTPSSLNOVERIFY.

 Martin

From: David Brownlee <abs@absd.org>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing certificates
Date: Wed, 21 Jun 2023 20:50:01 +0100

 On Thu, 8 Jun 2023 at 22:10, Martin Husemann <martin@duskware.de> wrote:
 >
 >  Actually the installer issue is quite simple: I'll make it check
 >  for any root certs in it's /etc/openssl/certs and if none is found
 >  set FTPSSLNOVERIFY.

 That looks like a good change for the installer (and worth making
 independent of any other fix), but this issue is still outstanding for
 netbsd-10

 Without installing mozilla-rootcerts-openssl or similar from pkgsrc,
 ftp will fail on any http URL with:

 18446744073709551615:error:1416F086:SSL
 routines:tls_process_server_certificate:certificate verify
 failed:/usr/src/crypto/external/bsd/openssl/dist/ssl/statem/statem_clnt.c:1919:

 including downloading https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc.tar.xz

 I'm inclined to suggest this should be a blocker for netbsd-10

 David

State-Changed-From-To: open->pending-pullups
State-Changed-By: martin@NetBSD.org
State-Changed-When: Wed, 21 Jun 2023 20:01:16 +0000
State-Changed-Why:
[pullup-10 #212]


From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing certificates
Date: Wed, 21 Jun 2023 20:00:04 -0000 (UTC)

 abs@absd.org (David Brownlee) writes:

 > Without installing mozilla-rootcerts-openssl or similar from pkgsrc,
 > ftp will fail on any http URL with:
 > 
 > 18446744073709551615:error:1416F086:SSL
 > routines:tls_process_server_certificate:certificate verify
 > failed:/usr/src/crypto/external/bsd/openssl/dist/ssl/statem/statem_clnt.c:1919:

 Of course only for https URLs.

 That's why the installer should offer to install such a package,
 maybe similar to chose whether to install the pkgin package.

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Wed, 21 Jun 2023 22:11:59 +0200

 On Wed, Jun 21, 2023 at 08:05:02PM +0000, Michael van Elst wrote:
 >  That's why the installer should offer to install such a package,
 >  maybe similar to chose whether to install the pkgin package.

 And download it via https :-)

 From where? (We should create a well know URL for it and play redirect
 tricks on the server or something...)
 Does pkgsrc support arch independent packages?

 Or it should be bundled with the installer, but that is likely a political
 battle.

 Martin

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing certificates
Date: Wed, 21 Jun 2023 20:26:00 -0000 (UTC)

 martin@duskware.de (Martin Husemann) writes:

 > And download it via https :-)

 Sure.


 > From where? (We should create a well know URL for it and play redirect
 > tricks on the server or something...)
 > Does pkgsrc support arch independent packages?

 pkg_install.conf has a well known URL, it's not arch independent but this
 isn't necessary. When pkgsrc has deprecated the release, the user should
 be able to supply a different URL.

 This is more an issue for archs that we fail to provide packages for.

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/57456: ftp fails for https in netbsd-10 due to missing
 certificates
Date: Thu, 22 Jun 2023 08:16:14 +0200

 On Wed, Jun 21, 2023 at 08:30:02PM +0000, Michael van Elst wrote:
 >  This is more an issue for archs that we fail to provide packages for.

 Indeed. But combined with (a) pkgsrc apparently being unable to provide
 arch-neutral pkgs and (b) the net content of this pkg being totaly arch
 neutral makes me think that pkgsrc is not the best answer for this.

 Or that we should add some special treatement, like: when we swich the
 quaterly symlink for amd64 (for example) to run a special script that
 creates a simple "mozilla-certs.tgz" pseudo-set (from the amd64 pkg)
 and stores it at some fixed place for download. Maybe plus a
 "mozilla-certs-hash.txt" that could even be properly signed (by whom?)

 But I guess this discussion better should happen elsewhere.

 Martin

State-Changed-From-To: pending-pullups->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Tue, 28 Nov 2023 01:48:12 +0000
State-Changed-Why:
addressed by shipping certs in base


>Unformatted:

Home	PR Database Search