NetBSD Problem Report #58506
From www@netbsd.org Sun Jul 28 22:15:34 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits)
client-signature RSA-PSS (2048 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 4D7C81A923E
for <gnats-bugs@gnats.NetBSD.org>; Sun, 28 Jul 2024 22:15:34 +0000 (UTC)
Message-Id: <20240728221533.49BB21A9241@mollari.NetBSD.org>
Date: Sun, 28 Jul 2024 22:15:33 +0000 (UTC)
From: phil+netbsd@krylov.eu
Reply-To: phil+netbsd@krylov.eu
To: gnats-bugs@NetBSD.org
Subject: lang/rust-1.79 build fails due to APIs removed in curl-8.8
X-Send-Pr-Version: www-1.0
>Number: 58506
>Category: pkg
>Synopsis: lang/rust-1.79 build fails due to APIs removed in curl-8.8
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: pkg-manager
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sun Jul 28 22:20:00 +0000 2024
>Closed-Date: Fri Aug 02 14:18:42 +0000 2024
>Last-Modified: Fri Aug 02 14:18:42 +0000 2024
>Originator: Phil Krylov
>Release: pkgsrc-current
>Organization:
krylov.eu
>Environment:
Linux localhost.localdomain 3.10.0-1160.119.1.el7.x86_64 #1 SMP Tue Jun 4 14:43:51 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
>Description:
lang/rust build fails on CentOS 7, Rocky Linux 8, Rocky Linux 9 because newer curl-8.8 headers are found instead of the bundled 8.6.
>How-To-Repeat:
bmake -C lang/rust bin-install
>Fix:
Index: lang/rust/Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/lang/rust/Makefile,v
retrieving revision 1.312
diff -p -u -r1.312 Makefile
--- lang/rust/Makefile 15 Jul 2024 03:32:23 -0000 1.312
+++ lang/rust/Makefile 28 Jul 2024 22:09:09 -0000
@@ -67,6 +67,7 @@ MAKE_ENV+= CARGO_BUILD_JOBS=${_MAKE_JOB
# Rust builds some bundled components with strict version requirements, ensure
# that any conflicting packages pulled in via dependencies are not buildlinked.
+BUILDLINK_FILES_CMD.curl= ${TRUE}
BUILDLINK_FILES_CMD.libssh2= ${TRUE}
BUILDLINK_FILES_CMD.xz= ${TRUE}
MAKE_ENV+= LZMA_API_STATIC=1
>Release-Note:
>Audit-Trail:
From: Thomas Klausner <wiz@gatalith.at>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 08:23:01 +0200
On Sun, Jul 28, 2024 at 10:20:01PM +0000, phil+netbsd@krylov.eu wrote:
> lang/rust build fails on CentOS 7, Rocky Linux 8, Rocky Linux 9 because newer curl-8.8 headers are found instead of the bundled 8.6.
I don't understand this bug report.
The pkgsrc package by default pulls in pkgsrc's curl package, which
right now is at 8.9.0.
rust builds for me on NetBSD.
Why doesn't this work on your platform?
Is curl pulled in or not?
If yes, the pkgsrc curl headers (for 8.9) should be used, not the
system's 8.8 ones.
Thomas
From: Tobias Nygren <tnn@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 08:43:01 +0200
It could be that base curl headers are installed and picked up, or
that it for some reason lacks the pkgsrc run path to the library
when linking, leading to a mismatch. Are base curl headers installed?
It builds for me on Fedora so at least it's not completely borked on
Linux.
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: Thomas Klausner <wiz@NetBSD.org>
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 11:08:17 +0200
On 2024-07-29 08:25, Thomas Klausner wrote:
> I don't understand this bug report.
>
> The pkgsrc package by default pulls in pkgsrc's curl package, which
> right now is at 8.9.0.
>
> rust builds for me on NetBSD.
>
> Why doesn't this work on your platform?
>
> Is curl pulled in or not?
>
> If yes, the pkgsrc curl headers (for 8.9) should be used, not the
> system's 8.8 ones.
Rust 1.79 sources contain a bundled vendor crate (curl-sys
v0.4.72+curl-8.6.0). When bundled curl C sources are being compiled
during the building of that crate, GCC finds the headers of pkgsrc's
curl-8.8.0 instead, which do not contain the needed definitions any
more. E.g. the latest Rocky Linux 9 report:
https://us-central.manta.mnx.io/pkgsrc/public/reports/Linux/el9/trunk/x86_64/20240728.1102/rust-1.79.0/build.log
-- Ph.
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: pkgsrc-bugs@netbsd.org
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 11:10:36 +0200
On 2024-07-29 08:45, Tobias Nygren wrote:
> It could be that base curl headers are installed and picked up, or
> that it for some reason lacks the pkgsrc run path to the library
> when linking, leading to a mismatch. Are base curl headers installed?
> It builds for me on Fedora so at least it's not completely borked on
> Linux.
You can see the failure in any of the recent Linux pbulk logs here:
https://us-central.manta.mnx.io/pkgsrc/public/reports/Linux/el9/trunk/x86_64/20240728.1102/rust-1.79.0/build.log
-- Ph.
From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 11:12:57 +0200
Before changing this I'd still like to understand why this works on
e.g. NetBSD where all the same arguments apply but fails on CentOS.
What's the difference?
Thomas
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: Thomas Klausner <wiz@NetBSD.org>
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 11:29:38 +0200
On 2024-07-29 11:15, Thomas Klausner wrote:
> Before changing this I'd still like to understand why this works on
> e.g. NetBSD where all the same arguments apply but fails on CentOS.
Well, I am not sure if it really works on NetBSD, I do not have a clean
NetBSD installation to test, and just look at the pbulk logs in
pkgsrc-bulk, and they have not yet been built with both rust-1.79 and
curl>=8.8 at the same time.
-- Ph.
From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 11:38:09 +0200
On Mon, Jul 29, 2024 at 11:29:38AM +0200, Phil Krylov wrote:
> Well, I am not sure if it really works on NetBSD, I do not have a clean
> NetBSD installation to test, and just look at the pbulk logs in pkgsrc-bulk,
> and they have not yet been built with both rust-1.79 and curl>=8.8 at the
> same time.
I've successfully built it in a pbulk environment on NetBSD with curl 8.9.0.
Thomas
From: Tobias Nygren <tnn@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 12:37:35 +0200
On Mon, 29 Jul 2024 09:15:01 +0000 (UTC)
Phil Krylov <phil+netbsd@krylov.eu> wrote:
> You can see the failure in any of the recent Linux pbulk logs here:
>
> https://us-central.manta.mnx.io/pkgsrc/public/reports/Linux/el9/trunk/x86_64/20240728.1102/rust-1.79.0/build.log
Ok, so my hypothesis then is that it normally builds the bundled curl
but does not end up using it. But this build is now failing on Linux
because the pkgsrc libunwind patch injects -I/usr/pkg/include too early
in the command line.
In the bulk build we can see that it has
"-I/usr/pkg/include" before "-I" "curl/include" which is why it fails.
On NetBSD, rust (or rather the internal llvm) does not need/use
libunwind so we don't have to add the stuff to have it available.
I will think about how to do the patch differently of we can
fix this in the curl crate.
Meanwhile you can try to revert this commit and see if anything changes:
http://mail-index.netbsd.org/pkgsrc-changes/2024/07/27/msg303751.html
For reference, prior thread on this topic and why we have the
libunwind patch:
http://mail-index.netbsd.org/pkgsrc-users/2024/04/03/msg039271.html
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: tnn@NetBSD.org
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 13:51:36 +0200
On 2024-07-29 12:40, Tobias Nygren wrote:
> I will think about how to do the patch differently of we can
> fix this in the curl crate.
> Meanwhile you can try to revert this commit and see if anything
> changes:
>
> http://mail-index.netbsd.org/pkgsrc-changes/2024/07/27/msg303751.html
>
> For reference, prior thread on this topic and why we have the
> libunwind patch:
>
> http://mail-index.netbsd.org/pkgsrc-users/2024/04/03/msg039271.html
Thanks for the context! I believe, trying to revert the libunwind patch
will revert to the situation from
https://us-central.manta.mnx.io/pkgsrc/public/reports/Linux/el9/trunk/x86_64/20240725.2158/rust-1.79.0/build.log
(a Linux build just before the libunwind patch): it fails with `/bin/ld:
cannot find -lunwind`.
However, my `BUILDLINK_FILES_CMD.curl= ${TRUE}` patch seems to fix
at least CentOS 7 build. Is anything wrong with it?
Patching the crate version seems hard (I have tried it and AFAICR cargo
complains about checksums etc.), probably another option is upgrading to
rust-1.80 where the crate is already updated. It needs to be done
anyway, but it can take longer, and then Linux is left with broken Rust
and 5000 of its dependents for the time being.
-- Ph.
From: Tobias Nygren <tnn@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 14:32:24 +0200
On Mon, 29 Jul 2024 13:51:36 +0200
Phil Krylov <phil+netbsd@krylov.eu> wrote:
> However, my `BUILDLINK_FILES_CMD.curl= ${TRUE}` patch seems to fix
> at least CentOS 7 build. Is anything wrong with it?
It is wrong in principle because cargo should still link against the
library in /usr/pkg and will need the lib buildlinked for that.
But all this points to the curl crate using it's bundled headers
regardless which doesn't sound safe on any pkgsrc platform so we'll
need to have a closer look at what it does.
A less wrong but still wrong workaround would be to only omit
buildlinking the headers. You can do this with
pre-configure:
${RM} -rf ${BUILDLINK_DIR}/include/curl
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 20:38:47 +0200
On 2024-07-29 14:35, Tobias Nygren wrote:
> It is wrong in principle because cargo should still link against the
> library in /usr/pkg and will need the lib buildlinked for that.
> But all this points to the curl crate using it's bundled headers
> regardless which doesn't sound safe on any pkgsrc platform so we'll
> need to have a closer look at what it does.
Thank you! In Linux pbulk logs, there is the following fragment:
> ===============================<
Couldn't find libcurl from pkgconfig (
pkg-config exited with status code 1
> PKG_CONFIG_PATH=
PKG_CONFIG_LIBDIR=/home/pbulk/build/lang/rust/work/.buildlink/lib64/pkgconfig:/home/pbulk/build/lang/rust/work/.buildlink/lib/pkgconfig:/home/pbulk/build/lang/rust/work/.buildlink/share/pkgconfig
PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1
/home/pbulk/build/lang/rust/work/.tools/bin/pkg-config --libs --cflags
libcurl
The system library `libcurl` required by crate `curl-sys` was not
found.
The file `libcurl.pc` needs to be installed and the PKG_CONFIG_PATH
environment variable must contain its parent directory.
PKG_CONFIG_PATH contains the following:
-
HINT: you may need to install a package such as libcurl, libcurl-dev
or libcurl-devel.
), compiling it from source...
> ===============================<
Ok, I have removed my patch, started the build and stopped it with ^Z,
and tried this command (which is approximately how cargo finds libcurl):
$ PKG_CONFIG_PATH=
PKG_CONFIG_LIBDIR=work/.buildlink/lib64/pkgconfig:work/.buildlink/lib/pkgconfig:work/.buildlink/share/pkgconfig
PKG_CONFIG_ALLOW_SYSTEM_LIBS=1 PKG_CONFIG_ALLOW_SYSTEM_CFLAGS=1
work/.tools/bin/pkg-config --libs --cflags libcurl
Package libssh2 was not found in the pkg-config search path.
Perhaps you should add the directory containing `libssh2.pc'
to the PKG_CONFIG_PATH environment variable
Package 'libssh2', required by 'libcurl', not found
So, `BUILDLINK_FILES_CMD.libssh2= ${TRUE}` is guilty then? The
question is how to make it build with bundled libssh2 and pkgsrc's curl.
And if it is needed/possible at all.
Now I am trying to build without `BUILDLINK_FILES_CMD.libssh2=
${TRUE}` .
-- Ph.
From: Tobias Nygren <tnn@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 22:55:51 +0200
On Mon, 29 Jul 2024 18:40:01 +0000 (UTC)
Phil Krylov <phil+netbsd@krylov.eu> wrote:
> Package 'libssh2', required by 'libcurl', not found
>
> So, `BUILDLINK_FILES_CMD.libssh2= ${TRUE}` is guilty then? The
> question is how to make it build with bundled libssh2 and pkgsrc's curl.
> And if it is needed/possible at all.
Is your curl package built with curl's libssh2 option enabled?
It's a non-default option. If so, that would explain the problem.
rust will need to do the pre-configure trick to unbuildlink only the
libssh2 headers and possibly .pc file.
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: Tobias Nygren <tnn@NetBSD.org>
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 23:29:42 +0200
On 2024-07-29 23:00, Tobias Nygren wrote:
> Is your curl package built with curl's libssh2 option enabled?
> It's a non-default option. If so, that would explain the problem.
> rust will need to do the pre-configure trick to unbuildlink only the
> libssh2 headers and possibly .pc file.
Ouch! I have actually installed curl implicitly using `bmake -C
lang/rust bin-install`, so it's the binary package from
https://pkgsrc.smartos.org/packages/Linux/el7/trunk/x86_64/All/curl-8.9.0.tgz
which is configured --with-libssh2=/usr/pkg . This explains why I have
the same lang/rust build failures that
https://mail-index.netbsd.org/pkgsrc-bulk/ . Probably we could ping the
person doing these MNX builds?
-- Ph.
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: Tobias Nygren <tnn@netbsd.org>
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Mon, 29 Jul 2024 23:47:20 +0200
On 2024-07-29 23:35, Phil Krylov wrote:
> Ouch! I have actually installed curl implicitly using `bmake -C
> lang/rust bin-install`, so it's the binary package from
>
> https://pkgsrc.smartos.org/packages/Linux/el7/trunk/x86_64/All/curl-8.9.0.tgz
> which is configured --with-libssh2=/usr/pkg . This explains why I have
> the same lang/rust build failures that
> https://mail-index.netbsd.org/pkgsrc-bulk/ . Probably we could ping
> the
> person doing these MNX builds?
And probably it's intentional:
https://github.com/TritonDataCenter/pkgbuild/blob/master/include/pkgoptions/linux.mk
So it would be better to find a way to still build rust in such an
environment.
-- Ph
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: Tobias Nygren <tnn@NetBSD.org>
Subject: Re: pkg/58506: lang/rust-1.79 build fails due to APIs removed in
curl-8.8
Date: Tue, 30 Jul 2024 19:36:28 +0200
On 2024-07-29 23:00, Tobias Nygren wrote:
> Is your curl package built with curl's libssh2 option enabled?
> It's a non-default option. If so, that would explain the problem.
> rust will need to do the pre-configure trick to unbuildlink only the
> libssh2 headers and possibly .pc file.
So, I have first tried this:
diff -p -u -r1.312 Makefile
--- lang/rust/Makefile 15 Jul 2024 03:32:23 -0000 1.312
+++ lang/rust/Makefile 30 Jul 2024 14:01:52 -0000
@@ -67,7 +67,7 @@ MAKE_ENV+= CARGO_BUILD_JOBS=${_MAKE_JOB
# Rust builds some bundled components with strict version requirements,
ensure
# that any conflicting packages pulled in via dependencies are not
buildlinked.
-BUILDLINK_FILES_CMD.libssh2= ${TRUE}
+#BUILDLINK_FILES_CMD.libssh2= ${TRUE}
BUILDLINK_FILES_CMD.xz= ${TRUE}
MAKE_ENV+= LZMA_API_STATIC=1
@@ -82,6 +82,9 @@ MAKE_ENV.SunOS+= AR=gar
# Annoyingly, this makes the powerpc build succeed...
#MAKE_ENV+= RUSTC_LOG=rustc_metadata
+pre-configure:
+ ${RM} -rf ${BUILDLINK_DIR}/include/libssh2*
${BUILDLINK_DIR}/lib/libssh2*
+
UNLIMIT_RESOURCES+= cputime stacksize datasize virtualsize
TEST_TARGET= check
It seemingly did the trick, but kept -lssh2 in libcurl.pc. It is not
immediately evident if it works properly. So I have made another
version, which I consider much more robust although a bit wordy:
Index: lang/rust/Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/lang/rust/Makefile,v
retrieving revision 1.312
diff -p -u -r1.312 Makefile
--- lang/rust/Makefile 15 Jul 2024 03:32:23 -0000 1.312
+++ lang/rust/Makefile 30 Jul 2024 14:54:51 -0000
@@ -67,6 +67,7 @@ MAKE_ENV+= CARGO_BUILD_JOBS=${_MAKE_JOB
# Rust builds some bundled components with strict version requirements,
ensure
# that any conflicting packages pulled in via dependencies are not
buildlinked.
+# See also pre-configure below.
BUILDLINK_FILES_CMD.libssh2= ${TRUE}
BUILDLINK_FILES_CMD.xz= ${TRUE}
MAKE_ENV+= LZMA_API_STATIC=1
@@ -476,6 +477,16 @@ post-extract:
${LN} -s gcc-wrap ar-wrap
.endif
+# Don't contaminate the build with curl's optional libssh2 dependency
+pre-configure:
+ cd ${BUILDLINK_DIR}/lib/pkgconfig && \
+ if grep -q libssh2 libcurl.pc; then \
+ ${SED} -e '/^Requires.private:/s/,libssh2\>//' \
+ -e '/^Libs.private:/ -lssh2\>//g' \
+ < libcurl.pc > libcurl.pc.new && \
+ ${MV} -f libcurl.pc.new libcurl.pc; \
+ fi
+
.if ${OPSYS} == "FreeBSD"
MAKE_ENV+= OPENSSL_DIR=${SSLBASE}
.endif
-- Ph.
From: "Tobias Nygren" <tnn@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58506 CVS commit: pkgsrc/lang
Date: Tue, 30 Jul 2024 18:01:16 +0000
Module Name: pkgsrc
Committed By: tnn
Date: Tue Jul 30 18:01:16 UTC 2024
Modified Files:
pkgsrc/lang/rust: Makefile
pkgsrc/lang/rust176: Makefile
Log Message:
rust: fix build when curl has libssh2 option enabled
PR pkg/58506 from Phil Krylov
To generate a diff of this commit:
cvs rdiff -u -r1.312 -r1.313 pkgsrc/lang/rust/Makefile
cvs rdiff -u -r1.2 -r1.3 pkgsrc/lang/rust176/Makefile
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: tnn@NetBSD.org
State-Changed-When: Tue, 30 Jul 2024 18:15:51 +0000
State-Changed-Why:
I committed the first version because I think some versions
of ld(1) and/or pkg-config might complain if the private libs
are absent. Maybe ld on SunOS?
Please test and let me know if we can close the PR. Thanks.
From: Phil Krylov <phil+netbsd@krylov.eu>
To: gnats-bugs@netbsd.org
Cc: tnn@netbsd.org
Subject: Re: pkg/58506 (lang/rust-1.79 build fails due to APIs removed in
curl-8.8)
Date: Thu, 01 Aug 2024 02:23:49 +0200
On 2024-07-30 20:15, tnn@NetBSD.org wrote:
> I committed the first version because I think some versions
> of ld(1) and/or pkg-config might complain if the private libs
> are absent. Maybe ld on SunOS?
> Please test and let me know if we can close the PR. Thanks.
Thank you! It's working well, please close the PR.
-- Ph.
State-Changed-From-To: feedback->closed
State-Changed-By: bsiegert@NetBSD.org
State-Changed-When: Fri, 02 Aug 2024 14:18:42 +0000
State-Changed-Why:
Confirmed fixed, thanks!
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.