NetBSD Problem Report #53673

From www@NetBSD.org  Wed Oct 17 01:49:12 2018
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 03DAB7A1CC
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 17 Oct 2018 01:49:12 +0000 (UTC)
Message-Id: <20181017014909.88BF47A271@mollari.NetBSD.org>
Date: Wed, 17 Oct 2018 01:49:09 +0000 (UTC)
From: aikchar@codeghar.com
Reply-To: aikchar@codeghar.com
To: gnats-bugs@NetBSD.org
Subject: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
X-Send-Pr-Version: www-1.0

>Number:         53673
>Notify-List:    dave@weller-fahy.com
>Category:       pkg
>Synopsis:       Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    linux-pkg-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 17 01:50:00 +0000 2018
>Last-Modified:  Thu Apr 25 16:00:02 +0000 2019
>Originator:     Hamza Sheikh
>Release:        trunk - git commit c18957a255f58b90000cad752b5c9a2764faf62d
>Organization:
>Environment:
Linux ec41ea7b3621 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 Linux
>Description:
I am building Python 3.7 on Alpine Linux 3.8 within ``docker build``. The build fails with the following messages. The full build log is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/build.log.


        running install_scripts
        copying build/scripts-3.7/pydoc3.7 -> /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin
        copying build/scripts-3.7/2to3-3.7 -> /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin
        copying build/scripts-3.7/pyvenv-3.7 -> /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin
        changing mode of /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin/pydoc3.7 to 755
        changing mode of /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin/2to3-3.7 to 755
        changing mode of /root/pkg/work/lang/python37/work/.destdir/usr/pkg/bin/pyvenv-3.7 to 755
        rm /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/_sysconfigdata_linux.py
        rm -r /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/__pycache__
        Creating directory /usr/pkg/man
        Creating directory /usr/pkg/man/man1
        /usr/bin/install -c -o root -g root -m 644 ./Misc/python.man  /root/pkg/work/lang/python37/work/.destdir/usr/pkg/man/man1/python3.7.1
        if test "xno" != "xno"  ; then  case no in  upgrade) ensurepip="--altinstall --upgrade" ;;  install|*) ensurepip="--altinstall" ;;  esac;  LD_LIBRARY_PATH=/root/pkg/work/lang/python37/work/Python-3.7.0 ./python -E -m ensurepip  $ensurepip --root=/root/pkg/work/lang/python37/work/.destdir/ ;  fi
        => Automatic manual page handling
        => Generating post-install file lists
        pkg_create: can't stat `/root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so'
        => Checking file-check results for python37-3.7.0
        ERROR: ************************************************************
        ERROR: The following files are in the PLIST but not in /root/pkg/work/lang/python37/work/.destdir/usr/pkg:
        ERROR:         /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so
        *** Error code 1

        Stop.
        bmake[1]: stopped in /root/pkgsrc/lang/python37
        *** Error code 1

        Stop.
        bmake: stopped in /root/pkgsrc/lang/python37
>How-To-Repeat:
I was working with pkgsrc from https://github.com/NetBSD/pkgsrc and commit c18957a255f58b90000cad752b5c9a2764faf62d made on 2018-10-15.

The mk.conf file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/mk.conf. It's also pasted below.


        # Example /usr/pkg/etc/mk.conf file produced by bootstrap-pkgsrc
        # Tue Jun 26 23:08:17 UTC 2018

        .ifdef BSD_PKG_MK	# begin pkgsrc settings

        ABI=			64

        PKG_DBDIR=		/usr/pkg/pkgdb
        LOCALBASE=		/usr/pkg
        VARBASE=		/var
        PKG_TOOLS_BIN=		/usr/pkg/sbin
        PKGINFODIR=		info
        PKGMANDIR=		man

        TOOLS_PLATFORM.awk?=		/usr/bin/awk
        TOOLS_PLATFORM.sh?=		/bin/bash

        WRKOBJDIR=/root/pkg/work
        DISTDIR=/root/pkg/distfiles
        PYTHON_VERSION_DEFAULT=37

        .endif			# end pkgsrc settings


The pkg_install.conf file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/pkg_install.conf. It's also pasted below.


        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-10754
        IGNORE_URL=https://www.openssl.org/news/secadv/20180612.txt
        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-1000035
        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-1060
        IGNORE_URL=https://nvd.nist.gov/vuln/detail/CVE-2018-1061


The profile file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/profile. It's also pasted below.


        export PATH
        PATH="/usr/pkg/sbin:/usr/pkg/bin:${PATH}"

        export MANPATH
        MANPATH="/usr/pkg/man:${MANPATH}"

        export LC_CTYPE
        LC_CTYPE="en_US.UTF-8"
        export LC_COLLATE
        LC_COLLATE="C"
        export LC_TIME
        LC_TIME="C"
        export LC_NUMERIC
        LC_NUMERIC="C"
        export LC_MONETARY
        LC_MONETARY="C"
        export LC_MESSAGES
        LC_MESSAGES="en_US.UTF-8"
        export LC_ALL
        LC_ALL=""


The Dockerfile I'm using is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/Dockerfile. It's also pasted below.

With this Dockerfile, run ``docker build -t localhost/tox:latest .``. It should fail in the RUN stanza on line 37 of the Dockerfile.


        FROM alpine as builder

        # https://github.com/justincormack/alpine-pkgsrc/blob/master/Dockerfile
        ENV \
        PATH=/usr/pkg/bin:$PATH \
        LIBABISUFFIX=""

        RUN apk --update add automake autoconf bash bzip2 curl file gcc g++ gawk git gmp-dev grep gzip libffi-dev libtool linux-headers m4 make ncurses-dev openssl-dev rsync sed tar unzip wget zlib-dev

        RUN    mkdir -p /root/pkgsrc \
            && curl -L -o /root/pkgsrc.tar.gz https://github.com/NetBSD/pkgsrc/archive/c18957a255f58b90000cad752b5c9a2764faf62d.tar.gz \
            && tar -C /root/pkgsrc -xzf /root/pkgsrc.tar.gz --strip=1

        RUN    export SH=/bin/bash \
            && cd ~/pkgsrc/bootstrap \
            && ./bootstrap

        RUN    mkdir -p /root/pkg/work \
            && mkdir -p /root/pkg/distfiles

        COPY profile /etc/profile.d/pkgsrc
        COPY mk.conf /usr/pkg/etc/mk.conf
        COPY pkg_install.conf /usr/pkg/etc/pkg_install.conf

        RUN    . /etc/profile.d/pkgsrc \
            && pkg_admin -K /usr/pkg/pkgdb fetch-pkg-vulnerabilities

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/nawk \
            && bmake install clean clean-depends \
            && sed -i.bak 's!TOOLS_PLATFORM\.awk.*!TOOLS_PLATFORM.awk?=		/usr/pkg/bin/nawk!' /usr/pkg/etc/mk.conf

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/pkgtools/pkg_alternatives \
            && bmake install clean clean-depends

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python37 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python36 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=36 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=36

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python27 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=27 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=27

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python35 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=35 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=35

        RUN    . /etc/profile.d/pkgsrc \
            && cd /root/pkgsrc/lang/python34 \
            && bmake install clean clean-depends \
            && cd /root/pkgsrc/devel/py-pip \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=34 \
            && cd /root/pkgsrc/devel/py-virtualenv \
            && bmake install clean clean-depends PYTHON_VERSION_DEFAULT=34

        RUN tar czf /root/pkg.tgz /usr/pkg


        FROM alpine as distro
        RUN    apk update \
            && apk upgrade \
            && apk add bash
        COPY --from=builder /root/pkg.tgz /root/pkg.tgz
        RUN tar xzf /root/pkg.tgz -C /usr --strip=1
        COPY profile /etc/profile.d/pkgsrc
        COPY bashrc_pkgsrc_edits.sh /root
        RUN    chmod +x /root/bashrc_pkgsrc_edits.sh \
            && /root/bashrc_pkgsrc_edits.sh
        RUN rm -rf /root/bashrc_pkgsrc_edits.sh /root/pkg.tgz /etc/profile.d/pkgsrc
        RUN /usr/pkg/bin/python3.7 -m pip install tox


The bashrc_pkgsrc_edits.sh file is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/bashrc_pkgsrc_edits.sh. It's also pasted below.


        #!/bin/sh
        cat /etc/profile.d/pkgsrc >> /etc/bash.bashrc



>Fix:

>Release-Note:

>Audit-Trail:
From: Leonardo Taccari <leot@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Wed, 17 Oct 2018 10:01:23 +0200

 Hello Hamza,

 aikchar@codeghar.com writes:
 > >Number:         53673
 > >Category:       pkg
 > >Synopsis:       Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
 > >Confidential:   no
 > >Severity:       serious
 > >Priority:       medium
 > >Responsible:    pkg-manager
 > >State:          open
 > >Class:          sw-bug
 > >Submitter-Id:   net
 > >Arrival-Date:   Wed Oct 17 01:50:00 +0000 2018
 > >Originator:     Hamza Sheikh
 > >Release:        trunk - git commit c18957a255f58b90000cad752b5c9a2764faf62d
 > >Organization:
 > >Environment:
 > Linux ec41ea7b3621 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 Linux
 > >Description:
 > I am building Python 3.7 on Alpine Linux 3.8 within ``docker build``. The build fails with the following messages. The full build log is available from https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/build.log.
 >
 >
 > [...]
 >         => Generating post-install file lists
 >         pkg_create: can't stat `/root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so'
 >         => Checking file-check results for python37-3.7.0
 >         ERROR: ************************************************************
 >         ERROR: The following files are in the PLIST but not in /root/pkg/work/lang/python37/work/.destdir/usr/pkg:
 >         ERROR:         /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so
 >         *** Error code 1
 > [...]

 Any idea regarding which Alpine Linux package provides the
 `<rpc/rpc.h>', `<rpcsvc/yp_prot.h>', `<rpcsvc/ypclnt.h>' header
 files and possible related libraries?

 In pkgsrc/lang/python37 the relevant part that handles that are:

  | .  if ${OPSYS} != "NetBSD" || exists(/usr/bin/ypcat)
  | .    if "${OS_VARIANT}" != "chromeos"
  | PLIST.nis=      yes
  | .    endif
  | .  else
  | PLIST.no-nis=   yes
  | .  endif
  | .endif

Responsible-Changed-From-To: pkg-manager->linux-pkg-people
Responsible-Changed-By: leot@NetBSD.org
Responsible-Changed-When: Wed, 17 Oct 2018 08:20:59 +0000
Responsible-Changed-Why:
Linux problem (sort of, maybe it is more a "libc used problem")


From: coypu@sdf.org
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Wed, 17 Oct 2018 10:14:10 +0000

 I'm hinted at this being a bigger problem we should probably resolve
 cleanly. glibc has apparently deprecated nis as well, so newer builds
 might not have it too (this is the default).

 This is a skeleton idea of what we could do, I don't know where to put
 HAVE_NIS in pkgsrc/mk (completely untested, just writing down my
 thoughts as code)
 I can make everyone do the nis test and it would probably be cleaner,
 but that is a small risk of breaking platforms.

 http://coypu.sdf.org/bootstrap.diff

From: Hamza Sheikh <hamza@codeghar.com>
To: leot@netbsd.org
Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org, 
	gnats-bugs@netbsd.org
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Wed, 17 Oct 2018 13:33:29 -0700

 On Wed, Oct 17, 2018 at 1:05 AM Leonardo Taccari <leot@netbsd.org> wrote:
 >
 >  Any idea regarding which Alpine Linux package provides the
 >  `<rpc/rpc.h>', `<rpcsvc/yp_prot.h>', `<rpcsvc/ypclnt.h>' header
 >  files and possible related libraries?
 >

 I found the following in Alpine Linux 3.8*:

 - /usr/include/rpc/rpc.h is provided by package portablexdr-dev
 - /usr/include/rpcsvc/yp_prot.h is provided by package libnsl-dev
 - /usr/include/rpcsvc/ypclnt.h is provided by package libnsl-dev
 - /usr/share/bash-completion/completions/ypcat is provided by package
 bash-completion

 I installed packages bash-completion, libnsl-dev, and portablexdr-dev
 and tried again but the build failed with the same error message.

 The new build2.log** and the updated Dockerfile (Dockerfile2)*** are
 available from their respective locations.

 Thanks for your help.

 * https://pkgs.alpinelinux.org/contents
 ** https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/build2.log
 *** https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/Dockerfile2

From: Leonardo Taccari <leot@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Wed, 17 Oct 2018 22:52:04 +0200

 Hello Hamza,

 Hamza Sheikh writes:
 > On Wed, Oct 17, 2018 at 1:05 AM Leonardo Taccari <leot@netbsd.org> wrote=
 :
 > >
 > >  Any idea regarding which Alpine Linux package provides the
 > >  `<rpc/rpc.h>', `<rpcsvc/yp_prot.h>', `<rpcsvc/ypclnt.h>' header
 > >  files and possible related libraries?
 > >
 >
 > I found the following in Alpine Linux 3.8*:
 >
 > - /usr/include/rpc/rpc.h is provided by package portablexdr-dev
 > - /usr/include/rpcsvc/yp_prot.h is provided by package libnsl-dev
 > - /usr/include/rpcsvc/ypclnt.h is provided by package libnsl-dev
 > - /usr/share/bash-completion/completions/ypcat is provided by package
 > bash-completion
 >
 > I installed packages bash-completion, libnsl-dev, and portablexdr-dev
 > and tried again but the build failed with the same error message.
 >
 > The new build2.log** and the updated Dockerfile (Dockerfile2)*** are
 > available from their respective locations.
 >
 > Thanks for your help.
 >
 > * https://pkgs.alpinelinux.org/contents
 > ** https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/build2.log
 > *** https://mushtarka.codeghar.com/netbsd/2018/gnats/python37/Dockerfile=
 2

 Thank you for the quick feedback and for investigating it!

 I think these are just the important lines of build2.log:

  | building 'nis' extension
  | cc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -O2 -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -O2 -D_FORTIFY_SOURCE=2 -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -std=gnu99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include -I. -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -I/root/pkg/work/lang/python37/work/Python-3.7.0/Include -I/root/pkg/work/lang/python37/work/Python-3.7.0 -c /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c -o build/temp.linux-x86_64-3.7/root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.o
  | In file included from /usr/include/rpcsvc/yp_prot.h:11:0,
  |                  from /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:18:
  | /usr/include/rpcsvc/ypclnt.h:23:23: fatal error: sys/cdefs.h: No such file or directory
  |  #include <sys/cdefs.h>
  |                        ^
  | compilation terminated.

 If I understand the error correctly I think that now the problem
 is in libnsl-dev that could tries to include <sys/cdefs.h>.

 Can you please try to share that with the Alpine community and keep us
 updated?


 Thank you again!

From: Hamza Sheikh <hamza@codeghar.com>
To: leot@netbsd.org
Cc: linux-pkg-people@netbsd.org, gnats-admin@netbsd.org, 
	pkgsrc-bugs@netbsd.org, gnats-bugs@netbsd.org
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Wed, 24 Oct 2018 15:43:56 -0700

 Thanks for your help.

 On Wed, Oct 17, 2018 at 1:55 PM Leonardo Taccari <leot@netbsd.org> wrote:
 >  Thank you for the quick feedback and for investigating it!
 >
 >  I think these are just the important lines of build2.log:
 >
 >   | building 'nis' extension
 >   | cc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -D_FORTIFY_SO=
 URCE=3D2 -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -O2 -D_FO=
 RTIFY_SOURCE=3D2 -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -=
 O2 -D_FORTIFY_SOURCE=3D2 -I/usr/include -I/usr/pkg/include -I/usr/pkg/inclu=
 de/db4 -std=3Dgnu99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-m=
 issing-field-initializers -Werror=3Dimplicit-function-declaration -I./Inclu=
 de -I. -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -I/root/pkg=
 /work/lang/python37/work/Python-3.7.0/Include -I/root/pkg/work/lang/python3=
 7/work/Python-3.7.0 -c /root/pkg/work/lang/python37/work/Python-3.7.0/Modul=
 es/nismodule.c -o build/temp.linux-x86_64-3.7/root/pkg/work/lang/python37/w=
 ork/Python-3.7.0/Modules/nismodule.o
 >   | In file included from /usr/include/rpcsvc/yp_prot.h:11:0,
 >   |                  from /root/pkg/work/lang/python37/work/Python-3.7.0/=
 Modules/nismodule.c:18:
 >   | /usr/include/rpcsvc/ypclnt.h:23:23: fatal error: sys/cdefs.h: No such=
  file or directory
 >   |  #include <sys/cdefs.h>
 >   |                        ^
 >   | compilation terminated.
 >
 >  If I understand the error correctly I think that now the problem
 >  is in libnsl-dev that could tries to include <sys/cdefs.h>.
 >

 I installed bsd-compat-headers package, which installs file
 /usr/include/sys/cdefs.h, and tried again. Build failed, and following
 is a chunk of the output.

 I also tried with the libbsd-dev package, which installs
 /usr/include/bsd/sys/cdefs.h, but the result was the same. There's
 another package, dev86, which installs
 /usr/lib/bcc/include/sys/cdefs.h , but I didn't try with that.

 | building 'nis' extension
 | cc -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -O2
 -D_FORTIFY_SOURCE=3D2 -I/usr/include -I/usr/pkg/include
 -I/usr/pkg/include/db4 -O2 -D_FORTIFY_SOURCE=3D2 -I/usr/include
 -I/usr/pkg/include -I/usr/pkg/include/db4 -O2 -D_FORTIFY_SOURCE=3D2
 -I/usr/include -I/usr/pkg/include -I/usr/pkg/include/db4 -std=3Dgnu99
 -Wextra -Wno-unused-result -Wno-unused-parameter
 -Wno-missing-field-initializers -Werror=3Dimplicit-function-declaration
 -I./Include -I. -I/usr/include -I/usr/pkg/include
 -I/usr/pkg/include/db4
 -I/root/pkg/work/lang/python37/work/Python-3.7.0/Include
 -I/root/pkg/work/lang/python37/work/Python-3.7.0 -c
 /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c -o
 build/temp.linux-x86_64-3.7/root/pkg/work/lang/python37/work/Python-3.7.0/M=
 odules/nismodule.o
 | In file included from /usr/include/rpcsvc/ypclnt.h:23:0,
 |                  from /usr/include/rpcsvc/yp_prot.h:11,
 |                  from
 /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:18:
 | /usr/include/sys/cdefs.h:1:2: warning: #warning usage of
 non-standard #include <sys/cdefs.h> is deprecated [-Wcpp]
 |  #warning usage of non-standard #include <sys/cdefs.h> is deprecated
 |   ^~~~~~~
 | In file included from
 /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:18:0:
 | /usr/include/rpcsvc/yp_prot.h:291:18: error: field
 'ypbind_binding_addr' has incomplete type
 |    struct in_addr ypbind_binding_addr;         /* In network order */
 |                   ^~~~~~~~~~~~~~~~~~~
 | /usr/include/rpcsvc/yp_prot.h:332:3: error: unknown type name 'rpcvers_t'
 |    rpcvers_t ypbind_hi_vers;
 |    ^~~~~~~~~
 | /usr/include/rpcsvc/yp_prot.h:335:3: error: unknown type name 'rpcvers_t'
 |    rpcvers_t ypbind_lo_vers;
 |    ^~~~~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:338:37=
 :
 error: unknown type name 'CLIENT'
 |  nisproc_maplist_2(domainname *argp, CLIENT *clnt)
 |                                      ^~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:
 In function 'nis_maplist':
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:358:5:
 error: unknown type name 'CLIENT'
 |      CLIENT *cl;
 |      ^~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:370:10=
 :
 error: implicit declaration of function 'clnt_create'
 [-Werror=3Dimplicit-function-declaration]
 |      cl =3D clnt_create(server, YPPROG, YPVERS, "tcp");
 |           ^~~~~~~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:370:8:
 warning: assignment makes pointer from integer without a cast
 [-Wint-conversion]
 |      cl =3D clnt_create(server, YPPROG, YPVERS, "tcp");
 |         ^
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:372:35=
 :
 error: implicit declaration of function 'clnt_spcreateerror'
 [-Werror=3Dimplicit-function-declaration]
 |          PyErr_SetString(NisError, clnt_spcreateerror(server));
 |                                    ^~~~~~~~~~~~~~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:372:35=
 :
 warning: passing argument 2 of 'PyErr_SetString' makes pointer from
 integer without a cast [-Wint-conversion]
 | In file included from ./Include/Python.h:112:0,
 |                  from
 /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:13:
 | ./Include/pyerrors.h:83:18: note: expected 'const char *' but
 argument is of type 'int'
 |  PyAPI_FUNC(void) PyErr_SetString(
 |                   ^~~~~~~~~~~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:375:12=
 :
 error: implicit declaration of function 'nisproc_maplist_2'
 [-Werror=3Dimplicit-function-declaration]
 |      list =3D nisproc_maplist_2 (&dom, cl);
 |             ^~~~~~~~~~~~~~~~~
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:375:10=
 :
 warning: assignment makes pointer from integer without a cast
 [-Wint-conversion]
 |      list =3D nisproc_maplist_2 (&dom, cl);
 |           ^
 | /root/pkg/work/lang/python37/work/Python-3.7.0/Modules/nismodule.c:376:5:
 error: implicit declaration of function 'clnt_destroy'
 [-Werror=3Dimplicit-function-declaration]
 |      clnt_destroy(cl);
 |      ^~~~~~~~~~~~
 | cc1: some warnings being treated as errors
 |
 | Python build finished successfully!
 | The necessary bits to build these optional modules were not found:
 | _gdbm                 _sqlite3              _tkinter
 | To find the necessary bits, look in setup.py in detect_modules() for
 the module's name.
 |
 |
 | The following modules found by detect_modules() in setup.py, have been
 | built by the Makefile instead, as configured by the Setup files:
 | _abc                  atexit                pwd
 | time
 |
 |
 | Failed to build these modules:
 | nis

 >  Can you please try to share that with the Alpine community and keep us
 >  updated?
 >

 Should I share with the Alpine community that it's a dependency issue?
 That libnsl-dev expects <sys/cdefs.h> but it's not installed and that
 either libbsd-dev or bsd-compat-headers or something else should be a
 dependency?

 Thanks.

From: Leonardo Taccari <leot@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Thu, 25 Oct 2018 00:56:35 +0200

 Hello Hamza,

 Hamza Sheikh writes:
 > [...]
 > Should I share with the Alpine community that it's a dependency issue?
 > That libnsl-dev expects <sys/cdefs.h> but it's not installed and that
 > either libbsd-dev or bsd-compat-headers or something else should be a
 > dependency?
 >

 Yes, please share that with the Alpine community, libnsl-dev expects
 <sys/cdefs.h> so it should probably depends on bsd-compat-headers
 package.

 Unfortunately also with that it seems that libnsl-dev and
 bsd-compat-headers are not enough to build Python `nis'.  I think
 that's also worth discussing with the Alpine community.


 Thank you!

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Thu, 14 Mar 2019 15:56:59 +0000

 On Wed, Oct 17, 2018 at 01:50:00AM +0000, aikchar@codeghar.com wrote:
  > ERROR: The following files are in the PLIST but not in /root/pkg/work/lang/python37/work/.destdir/usr/pkg:
  > ERROR:         /root/pkg/work/lang/python37/work/.destdir/usr/pkg/lib/python3.7/lib-dynload/nis.so

 This also affects python27 (see PR 54056) and almost certainly all
 python versions as well.

 -- 
 David A. Holland
 dholland@netbsd.org

From: "David H. Gutteridge" <david@gutteridge.ca>
To: gnats-bugs@netbsd.org
Cc: thomas.orgis@uni-hamburg.de
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Mon, 25 Mar 2019 19:16:09 -0400

 On Mon, 25 Mar 2019 at 13:20:35 +0100, Dr. Thomas Orgis wrote:
 > I just hit this on my Linux box. There is no NIS support and python27
 > and python36 fail. Regarding this logic:
 > 
 > .  elif ${OPSYS} != "NetBSD" || exists(/usr/bin/ypcat)
 > 
 > Is this really what is intended? It seems more sensible to me when
 > changed to
 > 
 > .  elif ${OPSYS} != "NetBSD" && exists(/usr/bin/ypcat)
 > 
 > I actually have a Linux system where NIS is employed, and there
 > is /usr/bin/ypcat. Shouldn't the logic use that to decide if NIS is
 > built or not, instead of forking out at != NetBSD and happily assuming
 > that there is NIS?

 This has been the subject of more than one bug report, and a recent
 discussion on pkgsrc-users began here:
 http://mail-index.netbsd.org/pkgsrc-users/2019/02/22/msg028082.html

 A proposed alternate means of fixing the issue was logged in another
 PR:
 https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=54056

 The absence of /usr/bin/ypcat isn't necessarily a reliable indicator
 that the nis module can't be built, e.g. on recent versions of Fedora.
 By the way, which Linux distribution(s) are you using?

 Dave


From: "Dr. Thomas Orgis" <thomas.orgis@uni-hamburg.de>
To: <gnats-bugs@netbsd.org>
Cc: "David H. Gutteridge" <david@gutteridge.ca>
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Tue, 26 Mar 2019 12:34:30 +0100

 Am Mon, 25 Mar 2019 19:16:09 -0400
 schrieb "David H. Gutteridge" <david@gutteridge.ca>:

 > This has been the subject of more than one bug report, and a recent
 > discussion on pkgsrc-users began here:
 > http://mail-index.netbsd.org/pkgsrc-users/2019/02/22/msg028082.html
 > 
 > A proposed alternate means of fixing the issue was logged in another
 > PR:
 > https://gnats.netbsd.org/cgi-bin/query-pr-single.pl?number=54056

 I'm all for applying _any_ fix;-)

 > The absence of /usr/bin/ypcat isn't necessarily a reliable indicator
 > that the nis module can't be built, e.g. on recent versions of Fedora.
 > By the way, which Linux distribution(s) are you using?

 Right. I am testing on Source Mage right now, with is a rather custom
 distro much like doing pkgsrc for the whole system. Incidentally, I
 _do_ have libnsl installed and the header

 	/usr/include/rpcsvc/yp_prot.h

 exists. Apparenlty it is not picked up by the pythons. I do not have
 ypcat. I can try to look again where the python configuration goes
 wrong and then post to pkg/54056.


 -- 
 Dr. Thomas Orgis
 HPC @ Universität Hamburg

From: "Dr. Thomas Orgis" <thomas.orgis@uni-hamburg.de>
To: <gnats-bugs@netbsd.org>
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Mon, 1 Apr 2019 22:14:41 +0200

 For my system, the issue seems to be that python does not link with -lnsl:

 building 'nis' extension
 gcc -pthread -fPIC -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -I/usr/local/include -I/dev/shm/Python-2.7.16/Include -I/dev/shm/Python-2.7.16 -c /dev/shm/Python-2.7.16/Modules/nismodule.c -o build/temp.linux-x86_64-2.7/dev/shm/Python-2.7.16/Modules/nismodule.o
 gcc -pthread -shared build/temp.linux-x86_64-2.7/dev/shm/Python-2.7.16/Modules/nismodule.o -L/usr/local/lib -ltirpc -o build/lib.linux-x86_64-2.7/nis.so
 *** WARNING: renaming "nis" since importing it failed: build/lib.linux-x86_64-2.7/nis.so: undefined symbol: yp_master

 $ nm -g /usr/lib/libnsl.so.2 | grep yp_master
 00000000000075c0 T yp_master

 It needs a check if glibc provides the symbols, and else check for
 libnsl. It should be able to build and use the module just fine on my
 box with glibc 2.29, libnsl and libtirpc.


 -- 
 Dr. Thomas Orgis
 HPC @ Universität Hamburg

From: "Dr. Thomas Orgis" <thomas.orgis@uni-hamburg.de>
To: <gnats-bugs@netbsd.org>
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
 "pkg_create: can't stat" at stage "Generating post-install file lists"
Date: Tue, 2 Apr 2019 17:12:57 +0200

 I just noticed that my libnsl installation was broken. There was no
 libnsl.so, only libnsl.so.2. After fixing that, I was able to build
 python27 and python37. Still, the test in the Makefile needs to look
 for the correct headers to fix the decision for installations without
 libnsl or libtirpc. What python27 looks for (if I read it correctly):

 $PREFIX/include/rpcsvc/yp_prot.h
 $PREFIX/include/nsl/rpcsvc/yp_prot.h
 $PREFIX/include/rpc/rpc.h
 $PREFIX/include/tirpc/rpc/rpc.h

 I assume $PREFIX is /usr for our case. So … if we check for presence of

 	/usr/include/rpcsvc/yp_prot.h || /usr/include/nsl/rpcsvc/yp_prot.h

 AND

 	/usr/include/rpc/rpc.h || /usr/include/tirpc/rpc/rpc.h

 and set nis and no-nis accordingly, we should be safe, right?


 -- 
 Dr. Thomas Orgis
 HPC @ Universität Hamburg

From: "Maya Rashish" <maya@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53673 CVS commit: pkgsrc/lang
Date: Wed, 3 Apr 2019 13:17:28 +0000

 Module Name:	pkgsrc
 Committed By:	maya
 Date:		Wed Apr  3 13:17:28 UTC 2019

 Modified Files:
 	pkgsrc/lang/python27: Makefile
 	pkgsrc/lang/python34: Makefile
 	pkgsrc/lang/python35: Makefile
 	pkgsrc/lang/python36: Makefile
 	pkgsrc/lang/python37: Makefile

 Log Message:
 python*: rework logic used to determine if PLIST.nis is used.

 Match the logic used by setup.py: it looks for two headers in the default
 include path. This helps newer glibc linux.

 Omit PLIST.dll on python3* because it doesn't appear in the PLIST.
 Make PLIST.dll true on all non-IRIX.

 tested: NetBSD-current, FreeBSD 11.2, Ubuntu 18.10, CentOS 6.9, Source Mage

 From Dr. Thomas Orgis, myself, and with pointers to a change from leot.

 PR pkg/53673


 To generate a diff of this commit:
 cvs rdiff -u -r1.78 -r1.79 pkgsrc/lang/python27/Makefile
 cvs rdiff -u -r1.30 -r1.31 pkgsrc/lang/python34/Makefile
 cvs rdiff -u -r1.17 -r1.18 pkgsrc/lang/python35/Makefile
 cvs rdiff -u -r1.16 -r1.17 pkgsrc/lang/python36/Makefile
 cvs rdiff -u -r1.5 -r1.6 pkgsrc/lang/python37/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Roy Marples <roy@marples.name>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/53673: Building Python 3.7 on Linux fails with error
Date: Tue, 23 Apr 2019 02:13:12 +0100

 This change breaks NetBSD systems built with MKYP=no because the headers 
 are installed but yp_master is no longer defined in libc.
 As such PLIST expects nis to build, but the reality is that it fails.

 I see two solutions
 1) Don't install the yp headers when MKYP=no
 2) See if yp_master is defined in libc
 3) Add the exclusion back for NetBSD and /usr/bin/ypcat doesn't exist
 4) Consider a dynamic PLIST?

 1 won't work if a generic NetBSD is installed and later customised to 
 MKYP=no (which mine is in this case).
 2 is more invasive - and does pkgsrc even know which libc will be used 
 and where it is?
 3 probably path of least resistance
 4 probably the most robust solution, but maybe one which will meet 
 resistance to add as it shows we're not in control of what we install.


 Roy

From: "Roy Marples" <roy@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53673 CVS commit: pkgsrc/lang
Date: Thu, 25 Apr 2019 15:59:55 +0000

 Module Name:	pkgsrc
 Committed By:	roy
 Date:		Thu Apr 25 15:59:54 UTC 2019

 Modified Files:
 	pkgsrc/lang/python27: Makefile PLIST
 	pkgsrc/lang/python34: Makefile PLIST
 	pkgsrc/lang/python35: Makefile PLIST
 	pkgsrc/lang/python36: Makefile PLIST
 	pkgsrc/lang/python37: Makefile PLIST

 Log Message:
 python: Don't install nis_failed.so if no yp headers

 Because python won't even try to build it.
 You only see nis_failed.so if there is an error building it, when
 it wanted to build it, and that should be fixed accordingly.

 In a nutshell, if the yp headers are installed, python will build
 the nis module, otherwise it won't.

 On netbsd systems at least, if you have the yp headers installed and
 subsequently re-install over the top with MKYP=no you get into the state
 where the headers are installed, but the functions are no longer in libc.
 This is an error with *your* system - either rebuild with MKYP=yes OR
 remove the yp headers from include/rpc and include/rpcsvc.

 Follow on fix for PR pkg/53673.


 To generate a diff of this commit:
 cvs rdiff -u -r1.79 -r1.80 pkgsrc/lang/python27/Makefile
 cvs rdiff -u -r1.2 -r1.3 pkgsrc/lang/python27/PLIST
 cvs rdiff -u -r1.31 -r1.32 pkgsrc/lang/python34/Makefile
 cvs rdiff -u -r1.11 -r1.12 pkgsrc/lang/python34/PLIST
 cvs rdiff -u -r1.18 -r1.19 pkgsrc/lang/python35/Makefile
 cvs rdiff -u -r1.9 -r1.10 pkgsrc/lang/python35/PLIST
 cvs rdiff -u -r1.17 -r1.18 pkgsrc/lang/python36/Makefile
 cvs rdiff -u -r1.13 -r1.14 pkgsrc/lang/python36/PLIST
 cvs rdiff -u -r1.6 -r1.7 pkgsrc/lang/python37/Makefile
 cvs rdiff -u -r1.5 -r1.6 pkgsrc/lang/python37/PLIST

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.