NetBSD Problem Report #52766

From hf@spg.tu-darmstadt.de  Mon Nov 27 11:20:40 2017
Return-Path: <hf@spg.tu-darmstadt.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 4F0F07A0F8
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 27 Nov 2017 11:20:40 +0000 (UTC)
Message-Id: <201711271117.vARBHWMM028868@Gstoder.nt.e-technik.tu-darmstadt.de>
Date: Mon, 27 Nov 2017 12:17:32 +0100 (CET)
From: Hauke Fath <hf@spg.tu-darmstadt.de>
Reply-To: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@NetBSD.org
Cc: Hauke Fath <hf@spg.tu-darmstadt.de>
Subject: lang/python27 fails to build on arch linux
X-Send-Pr-Version: 3.95

>Number:         52766
>Category:       pkg
>Synopsis:       lang/python27 fails to build on arch linux>
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Nov 27 11:25:00 +0000 2017
>Closed-Date:    Tue May 05 13:21:40 +0000 2020
>Last-Modified:  Tue May 05 13:21:40 +0000 2020
>Originator:     Hauke Fath
>Release:        Arch Linux x86_64
>Organization:
Technische Universitaet Darmstadt
>Environment:


System: Linux 4.13.12-1-ARCH #1 SMP PREEMPT Wed Nov 8 11:54:06 CET 2017 x86_64 GNU/Linux
Architecture: x86_64
Machine: x86_64
>Description:

	On Arch Linux, building lang/python27 fails in the packaging
	phase with

[...]
=> Generating post-install file lists
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/Python-2.7.14-py2.7.egg-info'
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/_ctypes.so'
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/_multiprocessing.so'
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/array.so'
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/mmap.so'
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib2to3/Grammar2.7.14.final.0.pickle'
pkg_create: can't stat `/var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib2to3/PatternGrammar2.7.14.final.0.pickle'
=> Checking file-check results for python27-2.7.14
ERROR: ************************************************************
ERROR: The following files are in the PLIST but not in /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local:
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/Python-2.7.14-py2.7.egg-info
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/_ctypes.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/_multiprocessing.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/array.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/mmap.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib2to3/Grammar2.7.14.final.0.pickle
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib2to3/PatternGrammar2.7.14.final.0.pickle
ERROR: ************************************************************
ERROR: The following files are in /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local but not in the PLIST:
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/Python-2.7.13-py2.7.egg-info
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/_ctypes_failed.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/_multiprocessing_failed.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/array_failed.so
ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/mmap_failed.so
*** Error code 1

Stop.
nbmake[1]: stopped in /misc/pkgsrc/lang/python27
*** Error code 1
[...]

	Looking closer, it turns out that parts of the build fail, but
	the errors are not propagated and do not cause the build to
	fail:

[...]
creating build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -IModules/_multiprocessing -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/multiprocessing.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/multiprocessing.o
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -IModules/_multiprocessing -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/socket_connection.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/socket_connection.o
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -IModules/_multiprocessing -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/semaphore.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/semaphore.o
cc -shared -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -pthread -L/usr/lib64 -Wl,-R/usr/lib64 -L/opt/local/lib -Wl,-R/opt/local/lib -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -pthread -L/usr/lib64 -Wl,-R/usr/lib64 -L/opt/local/lib -Wl,-R/opt/local/lib -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/usr/include -I/opt/local/include -I/opt/local/include/db4 build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/multiprocessing.o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/socket_connection.o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_multiprocessing/semaphore.o -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -L/usr/lib64 -L/opt/local/lib -L. -lpython2.7 -o build/lib.linux-x86_64-2.7/_multiprocessing.so -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -pthread -L/usr/!
 lib64 -Wl,-R/usr/lib64 -L/opt/local/lib -Wl,-R/opt/local/lib
*** WARNING: renaming "_multiprocessing" since importing it failed: cannot import name _remove_dead_weakref
building '_ctypes' extension
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/opt/local/include -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/_ctypes.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/_ctypes.o
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/opt/local/include -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/callbacks.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/callbacks.o
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/opt/local/include -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/callproc.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/callproc.o
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/opt/local/include -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/stgdict.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/stgdict.o
cc -fPIC -fno-strict-aliasing -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/opt/local/include -I. -IInclude -I./Include -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Include -I/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -c /var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/cfield.c -o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/cfield.o
cc -shared -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -pthread -L/usr/lib64 -Wl,-R/usr/lib64 -L/opt/local/lib -Wl,-R/opt/local/lib -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -pthread -L/usr/lib64 -Wl,-R/usr/lib64 -L/opt/local/lib -Wl,-R/opt/local/lib -O2 -D_FORTIFY_SOURCE=2 -pthread -I/usr/include -I/opt/local/include -I/opt/local/include/db4 -I/usr/include -I/opt/local/include -I/opt/local/include/db4 build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/_ctypes.o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/callbacks.o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/callproc.o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/stgdict.o build/temp.linux-x86_64-2.7/var/obj/pkgsrc/lang/python27/work/Python-2.7.14/Modules/_ctypes/cfield.o -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -L/u!
 sr/lib64 -L/opt/local/lib -L. -Wl,-R/opt/local/lib -lffi -lpython2.7 -o build/lib.linux-x86_64-2.7/_ctypes.so -L/var/obj/pkgsrc/lang/python27/work/Python-2.7.14 -pthread -L/usr/lib64 -Wl,-R/usr/lib64 -L/opt/local/lib -Wl,-R/opt/local/lib
*** WARNING: renaming "_ctypes" since importing it failed: build/lib.linux-x86_64-2.7/_ctypes.so: undefined symbol: _PySlice_Unpack

Python build finished, but the necessary bits to build these modules were not found:
dl                 gdbm               imageop         
sunaudiodev                                           
To find the necessary bits, look in setup.py in detect_modules() for the module's name.


Failed to build these modules:
_ctypes            _multiprocessing   array           
mmap                                                  

running build_scripts
creating build/scripts-2.7
[...]


	Full build logs are at
	<https://www2.nt.tu-darmstadt.de/~hf/pkgsrc/pkgsrc-archlinux-python27.tgz>



>How-To-Repeat:

	Build and package python27 on Arch.


>Fix:
	No Idea.



>Release-Note:

>Audit-Trail:
From: Benny Siegert <bsiegert@gmail.com>
To: gnats-bugs@netbsd.org
Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/52766: lang/python27 fails to build on arch linux
Date: Mon, 27 Nov 2017 14:32:57 +0100

 > ERROR: The following files are in the PLIST but not in /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local:
 > ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/Python-2.7.14-py2.7.egg-info

 > ERROR: The following files are in /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local but not in the PLIST:
 > ERROR:         /var/obj/pkgsrc/lang/python27/work/.destdir/opt/local/lib/python2.7/lib-dynload/Python-2.7.13-py2.7.egg-info

 This sounds like you have an incompletely updated checkout, and/or a
 stale work directory.

 -- 
 Benny

From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@NetBSD.org, pkg-manager@NetBSD.org, gnats-admin@NetBSD.org
Cc: 
Subject: Re: pkg/52766: lang/python27 fails to build on arch linux
Date: Mon, 27 Nov 2017 16:58:16 +0100

 On 11/27/17 14:35, Benny Siegert wrote:
 >   This sounds like you have an incompletely updated checkout, and/or a
 >   stale work directory.

 Re-updated cvs, made clean, successfully built python27 on netbsd-8 from 
 the same sources -- not so easy. Then, there's this:

 % fgrep WARNING /tmp/pkgsrc-archlinux-python27/*
 /tmp/pkgsrc-archlinux-python27/python27.buildlog:*** WARNING: renaming 
 "array" since importing it failed: build/lib.linux-x86_64-2.7/array.so: 
 undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.buildlog:*** WARNING: renaming 
 "mmap" since importing it failed: build/lib.linux-x86_64-2.7/mmap.so: 
 undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.buildlog:*** WARNING: renaming 
 "_multiprocessing" since importing it failed: cannot import name 
 _remove_dead_weakref
 /tmp/pkgsrc-archlinux-python27/python27.buildlog:*** WARNING: renaming 
 "_ctypes" since importing it failed: 
 build/lib.linux-x86_64-2.7/_ctypes.so: undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "array" since importing it failed: build/lib.linux-x86_64-2.7/array.so: 
 undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "mmap" since importing it failed: build/lib.linux-x86_64-2.7/mmap.so: 
 undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "_multiprocessing" since importing it failed: cannot import name 
 _remove_dead_weakref
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "_ctypes" since importing it failed: 
 build/lib.linux-x86_64-2.7/_ctypes.so: undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "array" since importing it failed: build/lib.linux-x86_64-2.7/array.so: 
 undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "mmap" since importing it failed: build/lib.linux-x86_64-2.7/mmap.so: 
 undefined symbol: _PySlice_Unpack
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "_multiprocessing" since importing it failed: cannot import name 
 _remove_dead_weakref
 /tmp/pkgsrc-archlinux-python27/python27.packagelog:*** WARNING: renaming 
 "_ctypes" since importing it failed: 
 build/lib.linux-x86_64-2.7/_ctypes.so: undefined symbol: _PySlice_Unpack
 %

 and

 % gcc --version | head -1
 gcc (GCC) 7.2.0
 %

From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@NetBSD.org
Cc: pkg-manager@NetBSD.org, gnats-admin@NetBSD.org
Subject: Re: pkg/52766: lang/python27 fails to build on arch linux
Date: Wed, 17 Jan 2018 13:15:14 +0100

 On Mon, 27 Nov 2017 16:05:01 +0000 (UTC) Hauke Fath wrote:

 >  % fgrep WARNING /tmp/pkgsrc-archlinux-python27/*
 >  /tmp/pkgsrc-archlinux-python27/python27.buildlog:*** WARNING: renaming 
 >  "array" since importing it failed: build/lib.linux-x86_64-2.7/array.so: 
 >  undefined symbol: _PySlice_Unpack

 See also <https://bugs.python.org/issue30074>.

From: Leonardo Taccari <leot@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/52766: lang/python27 fails to build on arch linux
Date: Thu, 04 Apr 2019 11:59:57 +0200

 JFTR this is very similar to PR pkg/53312
 (except for the _cytpes.so, _multiprocessing.so, array.so and mmap.so
 missing files.)

 (At least the mismatching version is probably still a problem on some
 systems and probably extended to other versions of lang/python*.)

State-Changed-From-To: open->feedback
State-Changed-By: leot@NetBSD.org
State-Changed-When: Thu, 04 Apr 2019 10:06:24 +0000
State-Changed-Why:
Are the _cytpes.so, _multiprocessing.so, array.so and mmap.so missing
files still problematic with current lang/python27?


State-Changed-From-To: feedback->closed
State-Changed-By: hauke@NetBSD.org
State-Changed-When: Tue, 05 May 2020 13:21:40 +0000
State-Changed-Why:
Confirmed fixed.


>Unformatted:

Home	PR Database Search