NetBSD Problem Report #56774

From www@netbsd.org  Thu Mar 31 16:47:06 2022
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id B493A1A921F
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 31 Mar 2022 16:47:06 +0000 (UTC)
Message-Id: <20220331164705.9DF1F1A9239@mollari.NetBSD.org>
Date: Thu, 31 Mar 2022 16:47:05 +0000 (UTC)
From: mdehling@gmail.com
Reply-To: mdehling@gmail.com
To: gnats-bugs@NetBSD.org
Subject: Python 3.9 & 3.10 build fails with pkgsrc-2022Q1 on NetBSD-current on RPi4
X-Send-Pr-Version: www-1.0

>Number:         56774
>Category:       pkg
>Synopsis:       Python 3.9 & 3.10 build fails with pkgsrc-2022Q1 on NetBSD-current on RPi4
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    khorben
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Mar 31 16:50:00 +0000 2022
>Closed-Date:    
>Last-Modified:  Wed Jul 13 21:49:59 +0000 2022
>Originator:     Malte Dehling
>Release:        NetBSD 9.99.96, pkgsrc 2022Q1
>Organization:
>Environment:
NetBSD retropi.local 9.99.96 NetBSD 9.99.96 (GENERIC64) #0: Wed Mar 30 00:06:50 UTC 2022  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/evbarm/compile/GENERIC64 evbarm
>Description:
On a clean NetBSD current install on my Raspberry Pi 4, trying to compile lang/python39 or lang/python310 from pkgsrc-2022Q1 leads to the following error:

--- sharedmods ---
LD_LIBRARY_PATH=/usr/pkg/obj/lang/python39/work/Python-3.9.12 CC='gcc -pthread' LDSHARED='gcc -pthread -shared -L. -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib -Wl,-zrelro -L/usr/lib -Wl,-R/usr/lib -L/usr/pkg/lib -Wl,-R/usr/pkg/lib  ' OPT='-DNDEBUG -O2 -fPIC -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include'  _TCLTK_INCLUDES='' _TCLTK_LIBS=''  ./python -E ./setup.py  build
Traceback (most recent call last):
  File "/usr/pkg/obj/lang/python39/work/Python-3.9.12/./setup.py", line 17, in <module>
    import subprocess
ModuleNotFoundError: No module named 'subprocess'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/pkg/obj/lang/python39/work/Python-3.9.12/./setup.py", line 28, in <module>
    import _bootsubprocess
ModuleNotFoundError: No module named '_bootsubprocess'
*** [sharedmods] Error code 1

make: stopped in /usr/pkg/obj/lang/python39/work/Python-3.9.12
1 error
>How-To-Repeat:
see description
>Fix:
I don’t have a fix, but installing an older version of lang/python39 or lang/python310 from pkgsrc-2021Q4 works fine.

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->tnn
Responsible-Changed-By: khorben@NetBSD.org
Responsible-Changed-When: Wed, 06 Jul 2022 00:38:53 +0000
Responsible-Changed-Why:
The underlying issue was found.


State-Changed-From-To: open->analyzed
State-Changed-By: khorben@NetBSD.org
State-Changed-When: Wed, 06 Jul 2022 00:38:53 +0000
State-Changed-Why:


From: Pierre Pronchery <khorben@defora.org>
To: gnats-bugs@netbsd.org
Cc: Malte Dehling <mdehling@gmail.com>,
 Tobias Nygren <tnn@netbsd.org>
Subject: Re: pkg/56774: Python 3.9 & 3.10 build fails with pkgsrc-2022Q1 on
 NetBSD-current on RPi4
Date: Wed, 6 Jul 2022 02:33:09 +0200

 --Apple-Mail=_3DBB6D82-D39B-4C5F-A601-28EC55487025
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/plain;
 	charset=us-ascii

 			Hi Malte, (Tobias)

 Thank you for reporting this issue. I have ran into the same problem on =
 macOS/amd64, with an unprivileged build, and managed to find the issue.

 It is due to a patch specific to pkgsrc, which modifies setup.py by =
 adding the following line:

 > sys.path =3D [p for p in sys.path if not re.compile('^' + =
 sys.base_prefix + '/').match(p)]


 Unfortunately this effectively empties sys.path on setups where the work =
 directory is a subfolder of the destination prefix (as is the case in =
 your setup and in mine).
 In practice this prevents Python from finding its own modules during the =
 build, triggering the error you reported.

 I am not aware of a better patch than simply removing this line from =
 setup.py at the moment.

 @Tobias: any ideas?

 HTH,
 --=20
 khorben=

 --Apple-Mail=_3DBB6D82-D39B-4C5F-A601-28EC55487025
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/html;
 	charset=us-ascii

 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; =
 charset=3Dus-ascii"></head><body style=3D"word-wrap: break-word; =
 -webkit-nbsp-mode: space; line-break: after-white-space;" class=3D""><span=
  class=3D"Apple-tab-span" style=3D"white-space:pre">			=
 </span>Hi Malte, (Tobias)<div class=3D""><br class=3D""></div><div =
 class=3D"">Thank you for reporting this issue. I have ran into the same =
 problem on macOS/amd64, with an unprivileged build, and managed to find =
 the issue.</div><div class=3D""><br class=3D""></div><div class=3D"">It =
 is due to a patch specific to pkgsrc, which modifies setup.py by adding =
 the following line:</div><div class=3D""><br class=3D""></div><div =
 class=3D""><blockquote type=3D"cite" class=3D""><span class=3D"" =
 style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">sys.path =3D =
 [p for p in sys.path if not re.compile('^' + sys.base_prefix + =
 '/').match(p)]</span><br class=3D"" style=3D"caret-color: rgb(0, 0, 0); =
 color: rgb(0, 0, 0);"></blockquote></div><div class=3D""><span class=3D"" =
 style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br =
 class=3D""></span></div><div class=3D""><span class=3D"" =
 style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">Unfortunately =
 this effectively empties sys.path on setups where the work directory is =
 a subfolder of the destination prefix (as is the case in your setup and =
 in mine).</span></div><div class=3D""><span class=3D"" =
 style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">In practice =
 this prevents Python from finding its own modules during the build, =
 triggering the error you reported.</span></div><div class=3D""><span =
 class=3D"" style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br =
 class=3D""></span></div><div class=3D""><span class=3D"" =
 style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">I am not aware =
 of a better patch than simply removing this line from setup.py at the =
 moment.</span></div><div class=3D""><br class=3D""></div><div =
 class=3D"">@Tobias: any ideas?</div><div class=3D""><br =
 class=3D""></div><div class=3D"">HTH,<br class=3D""><div class=3D"">
 <meta charset=3D"UTF-8" class=3D""><div dir=3D"auto" style=3D"caret-color:=
  rgb(0, 0, 0); color: rgb(0, 0, 0); letter-spacing: normal; text-align: =
 start; text-indent: 0px; text-transform: none; white-space: normal; =
 word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: =
 none; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: =
 after-white-space;" =
 class=3D""><div>--&nbsp;</div><div>khorben</div></div></div></div></body><=
 /html>=

 --Apple-Mail=_3DBB6D82-D39B-4C5F-A601-28EC55487025--

From: Tobias Nygren <tnn@NetBSD.org>
To: Pierre Pronchery <khorben@defora.org>
Cc: gnats-bugs@netbsd.org, Malte Dehling <mdehling@gmail.com>
Subject: Re: pkg/56774: Python 3.9 & 3.10 build fails with pkgsrc-2022Q1 on
 NetBSD-current on RPi4
Date: Thu, 7 Jul 2022 12:23:32 +0200

 On Wed, 6 Jul 2022 02:33:09 +0200
 Pierre Pronchery <khorben@defora.org> wrote:

 > I am not aware of a better patch than simply removing this line from setup.py at the moment.
 > 
 > @Tobias: any ideas?

 The purpose of the line is to prevent upgrade problems when setuptools
 comes with backwards-incompatible changes as happened earlier this year.

 Putting the work directory inside prefix sounds somewhat unorthodox to
 me but if you'd like you can change the regex to
 '^' + sys.base_prefix + '/lib'
 in order to avoid this specific problem [unless you name your work
 directory libwork at which point I'll wave my hands].
 I'm fine with that change but am not in a position to commit it myself
 at the moment.

From: Pierre Pronchery <khorben@defora.org>
To: Tobias Nygren <tnn@NetBSD.org>
Cc: gnats-bugs@netbsd.org,
 Malte Dehling <mdehling@gmail.com>
Subject: Re: pkg/56774: Python 3.9 & 3.10 build fails with pkgsrc-2022Q1 on
 NetBSD-current on RPi4
Date: Thu, 7 Jul 2022 17:20:22 +0200

 		Hey Tobias, (Malte)

 > On 7. Jul 2022, at 12:23, Tobias Nygren <tnn@NetBSD.org> wrote:
 >=20
 > On Wed, 6 Jul 2022 02:33:09 +0200
 > Pierre Pronchery <khorben@defora.org> wrote:
 >=20
 >> I am not aware of a better patch than simply removing this line from =
 setup.py at the moment.
 >>=20
 >> @Tobias: any ideas?
 >=20
 > The purpose of the line is to prevent upgrade problems when setuptools
 > comes with backwards-incompatible changes as happened earlier this =
 year.

 Understood.

 > Putting the work directory inside prefix sounds somewhat unorthodox to

 I believe I have a very good reason to do it.
 I regularly switch between pkgsrc branches:

 - -current for regular work on pkgsrc,
 - stable for pull-ups, and
 - frozen-or-soon-to-be-stable for tests (and building binaries ahead
   of upgrades)

 In practice this means one CVS tree with -current (everything goes in
 ~/opt/pkgsrc-current including packages and work directories, easy to
 reset and start from scratch if necessary or for tests) and one tree
 with Git (EdgeBSD) where I can switch between branches at will (to
 match /usr/pkg or work on eg ~/opt/pkgsrc-2022Q2 before the local
 system is up to date).

 > me but if you'd like you can change the regex to
 > '^' + sys.base_prefix + '/lib'
 > in order to avoid this specific problem [unless you name your work
 > directory libwork at which point I'll wave my hands].

 :)

 > I'm fine with that change but am not in a position to commit it myself
 > at the moment.

 I just tested your suggestion successfully on -current with
 lang/python{39,310}; I am going to commit it now.

 Thank you!
 --=20
 khorben=

Responsible-Changed-From-To: tnn->khorben
Responsible-Changed-By: khorben@NetBSD.org
Responsible-Changed-When: Thu, 07 Jul 2022 15:55:25 +0000
Responsible-Changed-Why:
I committed the fix.


State-Changed-From-To: analyzed->pending-pullups
State-Changed-By: khorben@NetBSD.org
State-Changed-When: Thu, 07 Jul 2022 15:55:25 +0000
State-Changed-Why:
pullup-pkgsrc #6647


Responsible-Changed-From-To: khorben->mdehling@gmail.com
Responsible-Changed-By: khorben@NetBSD.org
Responsible-Changed-When: Tue, 12 Jul 2022 16:02:48 +0000
Responsible-Changed-Why:
Bug analyzed and (hopefully) fixed, time for feedback :)


State-Changed-From-To: pending-pullups->feedback
State-Changed-By: khorben@NetBSD.org
State-Changed-When: Tue, 12 Jul 2022 16:02:48 +0000
State-Changed-Why:
Pull-up to 2022Q2 has been performed.


Responsible-Changed-From-To: mdehling@gmail.com->khorben
Responsible-Changed-By: dholland@NetBSD.org
Responsible-Changed-When: Wed, 13 Jul 2022 21:49:59 +0000
Responsible-Changed-Why:
Only developers are supposed to be Responsible:
(it won't nag you while it's in feedback)


>Unformatted:

Home	PR Database Search