NetBSD Problem Report #55997

From martin@aprisoft.de  Wed Feb 17 14:17:27 2021
Return-Path: <martin@aprisoft.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 962201A9217
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 17 Feb 2021 14:17:27 +0000 (UTC)
Message-Id: <20210217141718.76FE25CC8CF@emmas.aprisoft.de>
Date: Wed, 17 Feb 2021 15:17:18 +0100 (CET)
From: martin@NetBSD.org
Reply-To: martin@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: lang/perl5 fails to build on NetBSD-current
X-Send-Pr-Version: 3.95

>Number:         55997
>Category:       pkg
>Synopsis:       lang/perl5 fails to build on NetBSD-current under COMPAT_NETBSD32
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    pkg-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Feb 17 14:20:00 +0000 2021
>Closed-Date:    Tue Jun 01 15:07:22 +0000 2021
>Last-Modified:  Tue Jun 01 15:07:22 +0000 2021
>Originator:     Martin Husemann
>Release:        NetBSD 9.99.80
>Organization:
The NetBSD Foundation, Inc.
>Environment:
System: NetBSD seven-days-to-the-wolves.aprisoft.de 9.99.80 NetBSD 9.99.80 (GENERIC) #437: Wed Feb 17 11:59:57 CET 2021 martin@seven-days-to-the-wolves.aprisoft.de:/work/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:

I am not sure this is a pkgsrc problem - I can reproduce the issue in various
setups using COMPAT_NETBSD32 (in this case: a i386 chroot on an amd64 system).
I am pretty sure the same perl pkg did build a few days ago, and the
build failure looks like a bogus @INC environment - so could be caused
by a broken /bin/sh or broken /usr/bin/make as well.

The perl build fails for me with:

--- lib/auto/Encode/Encode.so ---
LD_LIBRARY_PATH=/tmp/pkgobj/lang/perl5/work/perl-5.32.1 ./miniperl -Ilib make_ext.pl lib/auto/Encode/Encode.so  MAKE="/usr/bin/make" LIBPERL_A=libperl.so LINKTYPE=dynamic
ERROR from evaluation of /tmp/pkgobj/lang/perl5/work/perl-5.32.1/cpan/Encode/Byte/Makefile.PL: Can't locate File/Spec/Functions.pm in @INC (you may need to install the File::Spec::Functions module) (@INC contains: / /.work.log/cv.h/Carp/lib /.work.log/cv.h/PathTools /.work.log/cv.h/PathTools/Cwd.pm / / / / /.work.log/embedvar.h/re / / / / / / / / /.work.log/intrpvar.h /.work.log/charclass_invlists.h/Encode .) at ./Makefile.PL line 4.
BEGIN failed--compilation aborted at ./Makefile.PL line 4.
Unsuccessful Makefile.PL(cpan/Encode): code=512 at make_ext.pl line 536.
*** [lib/auto/Encode/Encode.so] Error code 2


and the file exists:

# ll ./work/perl-5.32.1/lib/File/Spec/Functions.pm 
-r--r--r--  1 root  513  2348 Dec 18 11:04 ./work/perl-5.32.1/lib/File/Spec/Functions.pm

but notice the apparently totaly broken @INC listed.

>How-To-Repeat:

build pkgsrc/lang/perl5 on latest -current? or maybe only COMPAT_NETBSD32
latest -current?

>Fix:
n/a

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->kern-bug-people
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Wed, 17 Feb 2021 14:34:04 +0000
Responsible-Changed-Why:
Seems to be a COMPAT_NETBSD32 issue


From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/55997: lang/perl5 fails to build on NetBSD-current
Date: Wed, 17 Feb 2021 15:34:57 +0100

 I tried with an older /bin/sh which did not change anything, but the
 pkg builds fine natively on amd64 - so I guess it is some strange
 COMPAT_NETBSD32 fallout.

 Martin

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/55997: lang/perl5 fails to build in compat_netbsd32 chroot
Date: Sun, 14 Mar 2021 12:56:27 +0100

 This seems to not always be reproducable, though on the machines/setups
 where I hit it it reproduces 100% every time I tried.

 The perl build worked for me on sparc64/sparc and mips64 with N32 userland.

 It still fails for me on aarch64/evbearmv7hf-el and amd64/i386.

 Changing a minor detail on the i386 setup makes it work:

 I have the following mounts for the chroot:

 tmpfs on /test32/tmp type tmpfs (local)
 kernfs on /test32/kern type kernfs (local)
 ptyfs on /test32/dev/pts type ptyfs (local)
 procfs on /test32/proc type procfs (local)
 emmas:/var/nbsd/pkgsrc on /test32/usr/pkgsrc type nfs

 and do the build chroot'ed to /test32. Inside the chroot, /etc/mk.conf
 has: WRKOBJDIR=/tmp/pkgobj

 so perl is extracted and build on a tmpfs. When I change that line to point
 WRKOBJDIR to a ffs on an ssd instead, ther build works.

 I tried to reproduce it on native amd64 (outside a chroot) with a
 similar setup, but that build also works. So I created a amd64 chroot
 with identical setup to the i386 one and build it there - this also
 works.

 I have no idea if this is timing related or really file system induced (and
 maybe broken by chroot handling). The @INC paths for the perl build obviously
 are wrong, as if the real path of some file/dir got expanded bogusly and
 became / or empty.

 So to reproduce it seems you need:

  - be running COMPAT_NETBSD32 binaries
  - be inside a chroot
  - have WRKOBJDIR on a tmpfs

 Martin

From: Rin Okuyama <rokuyama.rk@gmail.com>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, martin@NetBSD.org
Cc: 
Subject: Re: kern/55997: lang/perl5 fails to build in compat_netbsd32 chroot
Date: Sun, 14 Mar 2021 21:28:22 +0900

 Ah, I just remembered perl5 vs tmpfs on COMPAT_NETBSD32 problem; I believe
 this is a bug of perl5 bootstrap rather than ours.

 Support for 64-bit integer is not built in miniperl for ILP32 environments,
 unless it is explicitly enabled by 64bitint option (and similar ones?).
 This is problematic since ino_t is uint64_t for NetBSD.

 For most cases, inode number is within 32-bit width. But tmpfs on LP64
 machines uses inode number larger than UINT32_MAX frequently. In this case,
 miniperl cannot handle that file/directory at all, which results in build
 failure.

 This problem can be worked around by something like

 ----
 .if ${MACHINE_ARCH} != alpha && \
      !(!empty(MACHINE_ARCH:M*64*) && empty(MACHINE_ARCH:Mmips*))
 PKG_OPTIONS.perl=       -64bitauto 64bitint
 .endif
 ----

 in /etc/mk.conf. Probably we can add a similar hack to pkgsrc/lang/perl5.

 Again, this is a bug in perl5. Since ino_t is uint64_t, we can freely use
 inode number larger than UINT32_MAX. miniperl should handle such an inode
 regardless of options (except for 64bitnone?).

 Thanks,
 rin

Responsible-Changed-From-To: kern-bug-people->pkg-manager
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Sun, 14 Mar 2021 12:36:22 +0000
Responsible-Changed-Why:
duh - it is a plain perl (miniperl) issue after all


From: matthew green <mrg@eterna.com.au>
To: Rin Okuyama <rokuyama.rk@gmail.com>
Cc: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
    gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, martin@NetBSD.org
Subject: re: kern/55997: lang/perl5 fails to build in compat_netbsd32 chroot
Date: Mon, 15 Mar 2021 04:03:06 +1100

 > in /etc/mk.conf. Probably we can add a similar hack to pkgsrc/lang/perl5.

 we should patch perl's "hints/netbsd.sh" to enable this on all
 of our platforms by default.


 .mrg.

From: "Rin Okuyama" <rin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55997 CVS commit: pkgsrc/lang/perl5
Date: Sat, 29 May 2021 13:04:22 +0000

 Module Name:	pkgsrc
 Committed By:	rin
 Date:		Sat May 29 13:04:21 UTC 2021

 Modified Files:
 	pkgsrc/lang/perl5: Makefile distinfo
 	pkgsrc/lang/perl5/patches:
 	    patch-cpan_ExtUtils-MakeMaker_lib_ExtUtils_MM__Unix.pm
 Added Files:
 	pkgsrc/lang/perl5/patches: patch-dist_PathTools_Cwd.pm
 	    patch-lib_File_Copy.pm

 Log Message:
 perl5: Compare inode numbers as string. Bump revision.

 PR pkg/55997

 Internal stat() function for perl stores inode number as string, if it
 cannot be represented by host's integer. However, unfortunately, some
 components compare them as integer.

 Therefore, if 64-bit integers are not supported, files cannot be handled,
 whose inode number is larger than UINT32_MAX.

 Usually, inode numbers on real filesystems are well below UINT32_MAX. But,
 inode numbers larger than UINT32_MAX are assigned for tmpfs on LP64 kernels.
 This results in build failures for perl on COMPAT_NETBSD32 if working
 directory is tmpfs, and perl-64bitint and friends are not specified.

 Now, inode numbers are compared as string, which works just fine even if
 64-bit integers are not supported.

 Cherry-picked from upstream. See https://github.com/Perl/perl5/pull/18788
 and related pull-requests for more details.


 To generate a diff of this commit:
 cvs rdiff -u -r1.270 -r1.271 pkgsrc/lang/perl5/Makefile
 cvs rdiff -u -r1.170 -r1.171 pkgsrc/lang/perl5/distinfo
 cvs rdiff -u -r1.1 -r1.2 \
     pkgsrc/lang/perl5/patches/patch-cpan_ExtUtils-MakeMaker_lib_ExtUtils_MM__Unix.pm
 cvs rdiff -u -r0 -r1.1 pkgsrc/lang/perl5/patches/patch-dist_PathTools_Cwd.pm \
     pkgsrc/lang/perl5/patches/patch-lib_File_Copy.pm

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: rin@NetBSD.org
State-Changed-When: Sat, 29 May 2021 14:27:25 +0000
State-Changed-Why:
Works for me. Can you please confirm?


State-Changed-From-To: feedback->closed
State-Changed-By: martin@NetBSD.org
State-Changed-When: Tue, 01 Jun 2021 15:07:22 +0000
State-Changed-Why:
Works fine for me - thanks!


>Unformatted:

Home	PR Database Search