NetBSD Problem Report #58273

From www@netbsd.org  Wed May 22 17:12:50 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 198481A926A
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 22 May 2024 17:12:50 +0000 (UTC)
Message-Id: <20240522171248.7F83E1A926B@mollari.NetBSD.org>
Date: Wed, 22 May 2024 17:12:48 +0000 (UTC)
From: 2857@gmx.de
Reply-To: 2857@gmx.de
To: gnats-bugs@NetBSD.org
Subject: pkgin cannot download repo index over SSL in default install
X-Send-Pr-Version: www-1.0

>Number:         58273
>Category:       pkg
>Synopsis:       pkgin cannot download repo index over SSL in default install
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed May 22 17:15:00 +0000 2024
>Closed-Date:    
>Last-Modified:  Sat May 25 02:48:05 +0000 2024
>Originator:     zip100
>Release:        10
>Organization:
>Environment:
NetBSD netbsd 10.0 NetBSD 10.0 (GENERIC) #0: Thu Mar 28 08:33:33 UTC 2024  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/amd64/compile/GENERIC amd64
>Description:
All defaults install (Xorgless) on VirtualBox. I've picked an option to add pkgin during the install, but that didn't work, so I decided to sort it out after booting into fresh install. However, pkgin doesn't seem to like "https" in repo address unless I install mozilla-certs manually.

=================================
trying with HTTPS from the start:


netbsd# export PKG_PATH="https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/"
netbsd# pkg_add pkgin
pkgin-23.8.1nb3: copying /usr/pkg/share/examples/pkgin/repositories.conf.example to /usr/pkg/etc/pkgin/repositories.conf
netbsd# pkgin install nano
processing remote summary (https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All)...
pkg_summary.gz                                                                                       50% 3136KB 308.0KB/s   00:09 ETApkgin: failure during fetch of file: Input/output error

===================================================
removing HTTPS and trying it out, installing certs:


netbsd# vi /usr/pkg/etc/pkgin/repositories.conf
netbsd# pkgin install nano
cleaning database from https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All entries...
reading local summary...
processing local summary...
processing remote summary (http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All)...
pkg_summary.gz                                                                                      100% 6210KB 207.0KB/s   00:30    
calculating dependencies...done.

1 package to install:
  nano-7.2nb2

0 to remove, 0 to refresh, 0 to upgrade, 1 to install
539K to download, 2646K of additional disk space will be used

proceed ? [Y/n] y
[1/1] nano-7.2nb2.tgz                                                                               100%  539KB 539.5KB/s   00:00    
[1/1] installing nano-7.2nb2...
nano-7.2nb2: copying /usr/pkg/share/examples/nano/nanorc to /usr/pkg/etc/nanorc
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
netbsd# pkgin install mozilla-rootcerts-openssl
calculating dependencies...done.

1 package to install:
  mozilla-rootcerts-openssl-2.14

0 to remove, 0 to refresh, 0 to upgrade, 1 to install
325K to download, 1722K of additional disk space will be used

proceed ? [Y/n] y
[1/1] mozilla-rootcerts-openssl-2.14.tgz                                                            100%  325KB 324.7KB/s   00:00    
[1/1] installing mozilla-rootcerts-openssl-2.14...
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...
netbsd# pkgin install mozilla-rootcerts
calculating dependencies...done.

1 package to install:
  mozilla-rootcerts-1.0.20240214

0 to remove, 0 to refresh, 0 to upgrade, 1 to install
532K to download, 2196K of additional disk space will be used

proceed ? [Y/n] y
[1/1] mozilla-rootcerts-1.0.20240214.tgz                                                            100%  532KB 532.4KB/s   00:00    
[1/1] installing mozilla-rootcerts-1.0.20240214...
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...

=======================
changing back to HTTPS:

netbsd# vi /usr/pkg/etc/pkgin/repositories.conf
netbsd# pkgin install htop
cleaning database from http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All entries...
reading local summary...
processing local summary...
processing remote summary (https://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All)...
pkg_summary.gz                                                                                      100% 6210KB 207.0KB/s   00:30    
calculating dependencies...done.

2 packages to install:
  hicolor-icon-theme-0.17nb1 htop-3.3.0

0 to remove, 0 to refresh, 0 to upgrade, 2 to install
108K to download, 320K of additional disk space will be used

proceed ? [Y/n] y
[1/2] hicolor-icon-theme-0.17nb1.tgz                                                                100%   11KB  11.0KB/s   00:00    
[2/2] htop-3.3.0.tgz                                                                                100%   97KB  96.5KB/s   00:00    
[1/2] installing hicolor-icon-theme-0.17nb1...
[2/2] installing htop-3.3.0...
pkg_install warnings: 0, errors: 0
reading local summary...
processing local summary...

>How-To-Repeat:
Install 10 iso in VirtualBox, all defaults but without X. Try to use pkgsrc with HTTPS after.
>Fix:
I remember there was some discussion over IRC that VirtualBox *may* corrupt some network packets ocassionally, but it seems to me that root certs are missing in default install and thus even NetBSD mirrors are affected. Maybe they're installed as some dependencies of X set, that's why it's not always triggered in a console install.

>Release-Note:

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/58273: pkgin cannot download repo index over SSL in default
 install
Date: Wed, 22 May 2024 19:39:10 +0200

 The default installation of 10.0 has certs installed (of course totally
 independent of X being installed or not).

 But you said the pkgin installation did not work, so maybe something went
 wrong in the last part of the install process that broke certs for you
 too.

 Can you check if

 	certctl list

 shows anything? It should show a bit over 140 trust anchors in the default
 installation.

 Also: which version of pkgin are you using and where did you get it?

 Martin

From: Taylor R Campbell <riastradh@NetBSD.org>
To: zip100 <2857@gmx.de>
Cc: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: pkg/58273: pkgin cannot download repo index over SSL in default install
Date: Wed, 22 May 2024 18:08:53 +0000

 > I've picked an option to add pkgin during the install, but that
 > didn't work, [...]

 What does `that didn't work' mean?  What was the symptom?

 > it seems to me that root certs are missing in default install and
 > thus even NetBSD mirrors are affected.

 This is probably what happened, but it's not clear why it happened.

 The mozilla-rootcerts-openssl package should no longer be necessary as
 of 10.  If you delete it and run `certctl list', that will tell you
 what root certs NetBSD thinks should be configured in
 /etc/openssl/certs, and `certctl rehash' will clear out
 /etc/openssl/certs and repopulate it to make it so.

 If you have more time, can you:

 1. boot the installer in a fresh VM,
 2. enter the utility menu and enable logging and scripting,
 3. otherwise run through the same installation procedure again, and
 4. reproduce the pkgin failure?

 If so, can you break into a shell (hit ^Z or go into the utility menu
 and start a shell) and share /tmp/sysinst.log and /tmp/sysinst.sh?
 (E.g., transmit them with nc(1) to another host.)

 Once you've done all that, can you:

 5. reboot into the fresh installation,
 6. check whether pkg_add and pkgin work with https,
 7. check whether `ls /etc/openssl/certs' is empty,
 8. run `certctl rehash', and
 9. check again whether whether pkg_add and pkgin work with https, and
 10. check again wehther `ls /etc/openssl/certs' is empty?

 > Maybe they're installed as some dependencies of X set, that's why
 > it's not always triggered in a console install.

 The certificates are in the base set, and they are always configured
 in /etc/openssl/certs when extracting sets during installation.

State-Changed-From-To: open->feedback
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Wed, 22 May 2024 18:14:27 +0000
State-Changed-Why:
feedback requested


From: 2857@gmx.de
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/58273: pkgin cannot download repo index over SSL in default
 install
Date: Thu, 23 May 2024 23:56:28 +0200

 Hi,

 Thank you for the detailed debug steps. Sadly I couldn't reproduce this
 exact issue, but after pkgin was installed, I chose to get pkgsrc
 archive from the installer menu, and that failed. I have extracted the
 logs, you can get them via

 wget -qO- https://bpa.st/download/KPHA | base64 -d | tar -xvz

 You can see it tries to unpack tar archive which's corrupted (failed
 download?). I then hit ^C and installer has marked that menu item as
 "Abandoned". After rebooting the machine, I've went to check the certs,
 and they were installed:

 # cd /etc/openssl/certs && wc -l
 294

 I've installed wget over https and it worked, so certs are good. I've
 then downloaded pkgsrc tarball over https and it also worked.

 I also noticed that `certctl rehash' happened right before entering
 "final" installer menu, before prompting to install pkgin and friends.

 I will try to reproduce it again, but my idea is that pkgin failed to
 install the same way pkgsrc archive did. No idea about what went wrong
 with certs.

 Thanks!



 On 22.05.24 20:10, Taylor R Campbell wrote:
 > The following reply was made to PR pkg/58273; it has been noted by GNATS=
 .
 >
 > From: Taylor R Campbell <riastradh@NetBSD.org>
 > To: zip100 <2857@gmx.de>
 > Cc: gnats-bugs@NetBSD.org, netbsd-bugs@NetBSD.org
 > Subject: Re: pkg/58273: pkgin cannot download repo index over SSL in def=
 ault install
 > Date: Wed, 22 May 2024 18:08:53 +0000
 >
 >   > I've picked an option to add pkgin during the install, but that
 >   > didn't work, [...]
 >
 >   What does `that didn't work' mean?  What was the symptom?
 >
 >   > it seems to me that root certs are missing in default install and
 >   > thus even NetBSD mirrors are affected.
 >
 >   This is probably what happened, but it's not clear why it happened.
 >
 >   The mozilla-rootcerts-openssl package should no longer be necessary as
 >   of 10.  If you delete it and run `certctl list', that will tell you
 >   what root certs NetBSD thinks should be configured in
 >   /etc/openssl/certs, and `certctl rehash' will clear out
 >   /etc/openssl/certs and repopulate it to make it so.
 >
 >   If you have more time, can you:
 >
 >   1. boot the installer in a fresh VM,
 >   2. enter the utility menu and enable logging and scripting,
 >   3. otherwise run through the same installation procedure again, and
 >   4. reproduce the pkgin failure?
 >
 >   If so, can you break into a shell (hit ^Z or go into the utility menu
 >   and start a shell) and share /tmp/sysinst.log and /tmp/sysinst.sh?
 >   (E.g., transmit them with nc(1) to another host.)
 >
 >   Once you've done all that, can you:
 >
 >   5. reboot into the fresh installation,
 >   6. check whether pkg_add and pkgin work with https,
 >   7. check whether `ls /etc/openssl/certs' is empty,
 >   8. run `certctl rehash', and
 >   9. check again whether whether pkg_add and pkgin work with https, and
 >   10. check again wehther `ls /etc/openssl/certs' is empty?
 >
 >   > Maybe they're installed as some dependencies of X set, that's why
 >   > it's not always triggered in a console install.
 >
 >   The certificates are in the base set, and they are always configured
 >   in /etc/openssl/certs when extracting sets during installation.
 >

State-Changed-From-To: feedback->open
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sat, 25 May 2024 02:48:05 +0000
State-Changed-Why:
feedback received


>Unformatted:

Home	PR Database Search