NetBSD Problem Report #55997
From martin@aprisoft.de Wed Feb 17 14:17:27 2021
Return-Path: <martin@aprisoft.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 962201A9217
for <gnats-bugs@gnats.NetBSD.org>; Wed, 17 Feb 2021 14:17:27 +0000 (UTC)
Message-Id: <20210217141718.76FE25CC8CF@emmas.aprisoft.de>
Date: Wed, 17 Feb 2021 15:17:18 +0100 (CET)
From: martin@NetBSD.org
Reply-To: martin@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: lang/perl5 fails to build on NetBSD-current
X-Send-Pr-Version: 3.95
>Number: 55997
>Category: pkg
>Synopsis: lang/perl5 fails to build on NetBSD-current under COMPAT_NETBSD32
>Confidential: no
>Severity: critical
>Priority: high
>Responsible: pkg-manager
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Feb 17 14:20:00 +0000 2021
>Closed-Date: Tue Jun 01 15:07:22 +0000 2021
>Last-Modified: Tue Jun 01 15:07:22 +0000 2021
>Originator: Martin Husemann
>Release: NetBSD 9.99.80
>Organization:
The NetBSD Foundation, Inc.
>Environment:
System: NetBSD seven-days-to-the-wolves.aprisoft.de 9.99.80 NetBSD 9.99.80 (GENERIC) #437: Wed Feb 17 11:59:57 CET 2021 martin@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:
I am not sure this is a pkgsrc problem - I can reproduce the issue in various
setups using COMPAT_NETBSD32 (in this case: a i386 chroot on an amd64 system).
I am pretty sure the same perl pkg did build a few days ago, and the
build failure looks like a bogus @INC environment - so could be caused
by a broken /bin/sh or broken /usr/bin/make as well.
The perl build fails for me with:
--- lib/auto/Encode/Encode.so ---
LD_LIBRARY_PATH=/tmp/pkgobj/lang/perl5/work/perl-5.32.1 ./miniperl -Ilib make_ext.pl lib/auto/Encode/Encode.so MAKE="/usr/bin/make" LIBPERL_A=libperl.so LINKTYPE=dynamic
ERROR from evaluation of /tmp/pkgobj/lang/perl5/work/perl-5.32.1/cpan/Encode/Byte/Makefile.PL: Can't locate File/Spec/Functions.pm in @INC (you may need to install the File::Spec::Functions module) (@INC contains: / /.work.log/cv.h/Carp/lib /.work.log/cv.h/PathTools /.work.log/cv.h/PathTools/Cwd.pm / / / / /.work.log/embedvar.h/re / / / / / / / / /.work.log/intrpvar.h /.work.log/charclass_invlists.h/Encode .) at ./Makefile.PL line 4.
BEGIN failed--compilation aborted at ./Makefile.PL line 4.
Unsuccessful Makefile.PL(cpan/Encode): code=512 at make_ext.pl line 536.
*** [lib/auto/Encode/Encode.so] Error code 2
and the file exists:
# ll ./work/perl-5.32.1/lib/File/Spec/Functions.pm
-r--r--r-- 1 root 513 2348 Dec 18 11:04 ./work/perl-5.32.1/lib/File/Spec/Functions.pm
but notice the apparently totaly broken @INC listed.
>How-To-Repeat:
build pkgsrc/lang/perl5 on latest -current? or maybe only COMPAT_NETBSD32
latest -current?
>Fix:
n/a
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: pkg-manager->kern-bug-people
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Wed, 17 Feb 2021 14:34:04 +0000
Responsible-Changed-Why:
Seems to be a COMPAT_NETBSD32 issue
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/55997: lang/perl5 fails to build on NetBSD-current
Date: Wed, 17 Feb 2021 15:34:57 +0100
I tried with an older /bin/sh which did not change anything, but the
pkg builds fine natively on amd64 - so I guess it is some strange
COMPAT_NETBSD32 fallout.
Martin
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/55997: lang/perl5 fails to build in compat_netbsd32 chroot
Date: Sun, 14 Mar 2021 12:56:27 +0100
This seems to not always be reproducable, though on the machines/setups
where I hit it it reproduces 100% every time I tried.
The perl build worked for me on sparc64/sparc and mips64 with N32 userland.
It still fails for me on aarch64/evbearmv7hf-el and amd64/i386.
Changing a minor detail on the i386 setup makes it work:
I have the following mounts for the chroot:
tmpfs on /test32/tmp type tmpfs (local)
kernfs on /test32/kern type kernfs (local)
ptyfs on /test32/dev/pts type ptyfs (local)
procfs on /test32/proc type procfs (local)
emmas:/var/nbsd/pkgsrc on /test32/usr/pkgsrc type nfs
and do the build chroot'ed to /test32. Inside the chroot, /etc/mk.conf
has: WRKOBJDIR=/tmp/pkgobj
so perl is extracted and build on a tmpfs. When I change that line to point
WRKOBJDIR to a ffs on an ssd instead, ther build works.
I tried to reproduce it on native amd64 (outside a chroot) with a
similar setup, but that build also works. So I created a amd64 chroot
with identical setup to the i386 one and build it there - this also
works.
I have no idea if this is timing related or really file system induced (and
maybe broken by chroot handling). The @INC paths for the perl build obviously
are wrong, as if the real path of some file/dir got expanded bogusly and
became / or empty.
So to reproduce it seems you need:
- be running COMPAT_NETBSD32 binaries
- be inside a chroot
- have WRKOBJDIR on a tmpfs
Martin
From: Rin Okuyama <rokuyama.rk@gmail.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, martin@NetBSD.org
Cc:
Subject: Re: kern/55997: lang/perl5 fails to build in compat_netbsd32 chroot
Date: Sun, 14 Mar 2021 21:28:22 +0900
Ah, I just remembered perl5 vs tmpfs on COMPAT_NETBSD32 problem; I believe
this is a bug of perl5 bootstrap rather than ours.
Support for 64-bit integer is not built in miniperl for ILP32 environments,
unless it is explicitly enabled by 64bitint option (and similar ones?).
This is problematic since ino_t is uint64_t for NetBSD.
For most cases, inode number is within 32-bit width. But tmpfs on LP64
machines uses inode number larger than UINT32_MAX frequently. In this case,
miniperl cannot handle that file/directory at all, which results in build
failure.
This problem can be worked around by something like
----
.if ${MACHINE_ARCH} != alpha && \
!(!empty(MACHINE_ARCH:M*64*) && empty(MACHINE_ARCH:Mmips*))
PKG_OPTIONS.perl= -64bitauto 64bitint
.endif
----
in /etc/mk.conf. Probably we can add a similar hack to pkgsrc/lang/perl5.
Again, this is a bug in perl5. Since ino_t is uint64_t, we can freely use
inode number larger than UINT32_MAX. miniperl should handle such an inode
regardless of options (except for 64bitnone?).
Thanks,
rin
Responsible-Changed-From-To: kern-bug-people->pkg-manager
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Sun, 14 Mar 2021 12:36:22 +0000
Responsible-Changed-Why:
duh - it is a plain perl (miniperl) issue after all
From: matthew green <mrg@eterna.com.au>
To: Rin Okuyama <rokuyama.rk@gmail.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, martin@NetBSD.org
Subject: re: kern/55997: lang/perl5 fails to build in compat_netbsd32 chroot
Date: Mon, 15 Mar 2021 04:03:06 +1100
> in /etc/mk.conf. Probably we can add a similar hack to pkgsrc/lang/perl5.
we should patch perl's "hints/netbsd.sh" to enable this on all
of our platforms by default.
.mrg.
From: "Rin Okuyama" <rin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/55997 CVS commit: pkgsrc/lang/perl5
Date: Sat, 29 May 2021 13:04:22 +0000
Module Name: pkgsrc
Committed By: rin
Date: Sat May 29 13:04:21 UTC 2021
Modified Files:
pkgsrc/lang/perl5: Makefile distinfo
pkgsrc/lang/perl5/patches:
patch-cpan_ExtUtils-MakeMaker_lib_ExtUtils_MM__Unix.pm
Added Files:
pkgsrc/lang/perl5/patches: patch-dist_PathTools_Cwd.pm
patch-lib_File_Copy.pm
Log Message:
perl5: Compare inode numbers as string. Bump revision.
PR pkg/55997
Internal stat() function for perl stores inode number as string, if it
cannot be represented by host's integer. However, unfortunately, some
components compare them as integer.
Therefore, if 64-bit integers are not supported, files cannot be handled,
whose inode number is larger than UINT32_MAX.
Usually, inode numbers on real filesystems are well below UINT32_MAX. But,
inode numbers larger than UINT32_MAX are assigned for tmpfs on LP64 kernels.
This results in build failures for perl on COMPAT_NETBSD32 if working
directory is tmpfs, and perl-64bitint and friends are not specified.
Now, inode numbers are compared as string, which works just fine even if
64-bit integers are not supported.
Cherry-picked from upstream. See https://github.com/Perl/perl5/pull/18788
and related pull-requests for more details.
To generate a diff of this commit:
cvs rdiff -u -r1.270 -r1.271 pkgsrc/lang/perl5/Makefile
cvs rdiff -u -r1.170 -r1.171 pkgsrc/lang/perl5/distinfo
cvs rdiff -u -r1.1 -r1.2 \
pkgsrc/lang/perl5/patches/patch-cpan_ExtUtils-MakeMaker_lib_ExtUtils_MM__Unix.pm
cvs rdiff -u -r0 -r1.1 pkgsrc/lang/perl5/patches/patch-dist_PathTools_Cwd.pm \
pkgsrc/lang/perl5/patches/patch-lib_File_Copy.pm
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: rin@NetBSD.org
State-Changed-When: Sat, 29 May 2021 14:27:25 +0000
State-Changed-Why:
Works for me. Can you please confirm?
State-Changed-From-To: feedback->closed
State-Changed-By: martin@NetBSD.org
State-Changed-When: Tue, 01 Jun 2021 15:07:22 +0000
State-Changed-Why:
Works fine for me - thanks!
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.