NetBSD Problem Report #53686

From www@NetBSD.org  Sat Oct 27 05:22:57 2018
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 8A3677A154
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 27 Oct 2018 05:22:57 +0000 (UTC)
Message-Id: <20181027052256.4AAC37A218@mollari.NetBSD.org>
Date: Sat, 27 Oct 2018 05:22:56 +0000 (UTC)
From: triaxx@NetBSD.org
Reply-To: triaxx@NetBSD.org
To: gnats-bugs@NetBSD.org
Subject: shells/pdksh building fails due to wrong preprocessor checking
X-Send-Pr-Version: www-1.0

>Number:         53686
>Category:       pkg
>Synopsis:       shells/pdksh building fails due to wrong preprocessor checking
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    bsiegert
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Oct 27 05:25:00 +0000 2018
>Closed-Date:    Sat Nov 24 10:53:12 +0000 2018
>Last-Modified:  Sat Nov 24 10:53:12 +0000 2018
>Originator:     Frédéric Fauberteau
>Release:        pkgsrc-current
>Organization:
>Environment:
Linux guardian 4.18.16-arch1-1-ARCH #1 SMP PREEMPT Sat Oct 20 22:06:45 UTC 2018 x86_64 GNU/Linux
>Description:
The preprocessor checking fails with the following errors from config.log:
==========
configure:867: checking how to run the C preprocessor
configure:888: echo cc -E  1>&5; cc -E  conftest.c >/dev/null 2>conftest.out
cc -E
In file included from /usr/include/assert.h:35,
                 from configure:883:
/usr/include/features.h:381:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
 #  warning _FORTIFY_SOURCE requires compiling with optimization (-O) 
    ^~~~~~~
configure: failed program was: 
#line 882 "configure"
#include "confdefs.h"
#include <assert.h>
Syntax Error
configure:905: echo cc -E -traditional-cpp  1>&5; cc -E -traditional-cpp  conftest.c >/dev/null 2>conftest.out
cc -E -traditional-cpp
In file included from /usr/include/assert.h:36,
                 from configure:901:
/usr/include/features.h:381:4: warning: #warning _FORTIFY_SOURCE requires compiling with optimization (-O) [-Wcpp]
 #  warning _FORTIFY_SOURCE requires compiling with optimization (-O) 
    ^~~~~~~
In file included from /usr/include/features.h:429,
                 from /usr/include/assert.h:36,
                 from configure:901:
/usr/include/sys/cdefs.h:30:3: error: #error "You need a ISO C conforming compiler to use the glibc headers"
 # error "You need a ISO C conforming compiler to use the glibc headers"
   ^~~~~
configure: failed program was: 
#line 899 "configure"
#include "confdefs.h"
#include <assert.h>
Syntax Error
configure:929: checking whether cc needs -traditional
/lib/cpp
./configure: line 940: /lib/cpp: No such file or directory
==========

Then, a lot of type checking fails that implies compilation errors
>How-To-Repeat:
On my Linux box:
bmake build
>Fix:
I don't really understand why #warning _FORTIFY_SOURCE

The following fix works for me, but it seems not portable and not really elegant:
==========
$NetBSD$

--- configure.orig  2018-10-27 05:08:25.609807539 +0000
+++ configure
@@ -570,7 +570,7 @@ fi
 ac_ext=c
 # CFLAGS is not in ac_cpp because -g, -O, etc. are not valid cpp options.
 ac_cpp='echo $CPP $CPPFLAGS 1>&5;
-$CPP $CPPFLAGS'
+$CPP $CPPFLAGS $CFLAGS'
 ac_compile='echo ${CC-cc} -c $CFLAGS $CPPFLAGS conftest.$ac_ext 1>&5;
 ${CC-cc} -c $CFLAGS $CPPFLAGS conftest.$ac_ext 1>&5 2>&5'
 ac_link='echo ${CC-cc} -o conftest$ac_exe_suffix $CFLAGS $CPPFLAGS $LDFLAGS conftest.$ac_ext $LIBS 1>&5;
==========

>Release-Note:

>Audit-Trail:
From: coypu@sdf.org
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/53686: shells/pdksh building fails due to wrong preprocessor
 checking
Date: Sat, 27 Oct 2018 05:55:45 +0000

 A bunch of things:
 - You are getting this because your linux distribution defaults to
   enabling FORTIFY in the compiler configuration, most likely.
   This is fine but very problematic when combined with choosing to keep
   warnings regarding FORTIFY CFLAGS configuration. It's weird that they
   haven't noticed this because it would have more effects, but maybe
   they don't want to edit their glibc headers.

 - It's probably fine to patch this. It would be better to do the
   non-generated file but I can't generate it easily so I can see why you
   did it that way.

From: =?UTF-8?Q?Fr=C3=A9d=C3=A9ric_Fauberteau?= <triaxx@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/53686: shells/pdksh building fails due to wrong preprocessor  checking
Date: Sat, 27 Oct 2018 16:26:25 +0200

 Le 2018-10-27 08:00, coypu@sdf.org a écrit :
 > The following reply was made to PR pkg/53686; it has been noted by 
 > GNATS.
 > 
 > From: coypu@sdf.org
 > To: gnats-bugs@netbsd.org
 > Cc:
 > Subject: Re: pkg/53686: shells/pdksh building fails due to wrong 
 > preprocessor
 >  checking
 > Date: Sat, 27 Oct 2018 05:55:45 +0000
 > 
 >  A bunch of things:
 >  - You are getting this because your linux distribution defaults to
 >    enabling FORTIFY in the compiler configuration, most likely.
 >    This is fine but very problematic when combined with choosing to 
 > keep
 >    warnings regarding FORTIFY CFLAGS configuration. It's weird that 
 > they
 >    haven't noticed this because it would have more effects, but maybe
 >    they don't want to edit their glibc headers.

 I am not sure that Arch guys enable FORTIFY by default.

 configure:888: echo cc -E  1>&5; cc -E  conftest.c >/dev/null 
 2>conftest.out
 cc -E
 In file included from /usr/include/assert.h:35,
                   from configure:883:
 /usr/include/features.h:381:4: warning: #warning _FORTIFY_SOURCE 
 requires compiling with optimization (-O) [-Wcpp]
   #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
      ^~~~~~~

 I have nor error neither warning if I type the following line in my 
 term:
 $ echo '#include<assert.h>' | cc -E -

 But I get the #warnging by running:
 $ echo '#include<assert.h>' | CWRAPPERS_CONFIG_DIR=<long_path> 
 <another_long_path>/cc-wrapper -E -

 >  - It's probably fine to patch this. It would be better to do the
 >    non-generated file but I can't generate it easily so I can see why 
 > you
 >    did it that way.

 Instead of checking cc -E, configure could check cpp. The cpp-wrapper 
 works fine without enabling FORTIFY. But now, I don't know how to do 
 that.

From: "David H. Gutteridge" <david@gutteridge.ca>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/53686: shells/pdksh building fails due to wrong
 preprocessor checking
Date: Sat, 27 Oct 2018 16:37:26 -0400

 Questions and comments (and apologies if this is out of sync with
 what's running through Gnats, and someone else has already replied
 with these):

 Are you building this as a result of bootstrapping pkgsrc (i.e.,
 using a "full" bootstrap, since the default bootstrap for Linux
 doesn't pull in pdksh, from what I can see)? Or are you building this
 for your own use? If the latter, have you considered mksh instead? I
 would strongly recommend using it over pdksh, if you simply want a ksh
 implementation. mksh is more actively maintained, and is my go-to on
 Linux.

 My understanding is that Arch Linux defines _FORTIFY_SOURCE in
 makepkg.conf[1]. Is this file being pulled in somehow (i.e., you're
 doing a "full" bootstrap, and so compiling using the native Arch
 environment)?

 Note that the configure script for the pdksh package was generated
 using autoconf 2.12, so you're relying upon output from a tool from
 1996 that may not interact well with more current environments.

 The Internet is scattered with references to this very issue on Arch.
 A useful report is one for GRASS[2], which summarizes this problem
 well. Comment #7 explains the pattern of errors you're seeing: "This
 all arises because autoconf tends to treat ANYTHING written to stderr
 as indicating a failure." (Comments #7 and #12 are detailed about
 this.)

 As a local workaround, you could also append your own redefinition of
 _FORTIFY_SOURCE to CPPFLAGS, either with -U or with -D setting the
 value to zero.

 [1] 
 https://lists.archlinux.org/pipermail/arch-dev-public/2013-April/024776.html
 [2] https://trac.osgeo.org/grass/ticket/2916

 Dave


From: =?UTF-8?Q?Fr=C3=A9d=C3=A9ric_Fauberteau?= <triaxx@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: pkg-manager@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org
Subject: Re: pkg/53686: shells/pdksh building fails due to wrong preprocessor  checking
Date: Tue, 30 Oct 2018 05:50:49 +0100

 Le 2018-10-27 22:40, David H. Gutteridge a écrit :
 >  Are you building this as a result of bootstrapping pkgsrc (i.e.,
 >  using a "full" bootstrap, since the default bootstrap for Linux
 >  doesn't pull in pdksh, from what I can see)? Or are you building this
 >  for your own use? If the latter, have you considered mksh instead? I
 >  would strongly recommend using it over pdksh, if you simply want a ksh
 >  implementation. mksh is more actively maintained, and is my go-to on
 >  Linux.

 It is a dependency for libuuid

 >  My understanding is that Arch Linux defines _FORTIFY_SOURCE in
 >  makepkg.conf[1]. Is this file being pulled in somehow (i.e., you're
 >  doing a "full" bootstrap, and so compiling using the native Arch
 >  environment)?

 The error occurs in both context: full and not full bootstrap

 >  As a local workaround, you could also append your own redefinition of
 >  _FORTIFY_SOURCE to CPPFLAGS, either with -U or with -D setting the
 >  value to zero.

 Could we imagine a supplementary test to use cc -E with -Wno-cpp? FMHO, 
 it does
 not break anything. The following patch works but if somebody does not 
 agree, I
 just modify my /etc/makepkg.conf (even if I prefer when it works 
 out-of-the-box).

 ----------8<--------------------------------------------------------------------
 $NetBSD$

 * On Arch Linux, cc-wrapper is built such as _FORTIFY_SOURCE is defined. 
 This
    fix avoids #warning to make the preprocessor checking failed.

 --- configure.orig	2018-10-30 04:18:53.913128552 +0000
 +++ configure
 @@ -910,12 +910,34 @@ else
     echo "configure: failed program was:" >&5
     cat conftest.$ac_ext >&5
     rm -rf conftest*
 +  # Arch Linux defines _FORTIFY_SOURCE in CPPFLAGS but the warnings 
 should not
 +  # failed the checking process
 +  CPP="${CC-cc} -E -Wno-cpp"
 +  cat > conftest.$ac_ext <<EOF
 +#line 899 "configure"
 +#include "confdefs.h"
 +#include <assert.h>
 +Syntax Error
 +EOF
 +ac_try="$ac_cpp conftest.$ac_ext >/dev/null 2>conftest.out"
 +{ (eval echo configure:905: \"$ac_try\") 1>&5; (eval $ac_try) 2>&5; }
 +ac_err=`grep -v '^ *+' conftest.out`
 +if test -z "$ac_err"; then
 +  :
 +else
 +  echo "$ac_err" >&5
 +  echo "configure: failed program was:" >&5
 +  cat conftest.$ac_ext >&5
 +  rm -rf conftest*
     CPP=/lib/cpp
   fi
   rm -f conftest*
   fi
   rm -f conftest*
     ac_cv_prog_CPP="$CPP"
 +fi
 +rm -f conftest*
 +  ac_cv_prog_CPP="$CPP"
   fi
     CPP="$ac_cv_prog_CPP"
   else

From: "David H. Gutteridge" <david@gutteridge.ca>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/53686: shells/pdksh building fails due to wrong
 preprocessor checking
Date: Wed, 31 Oct 2018 20:54:09 -0400

 On Tue, 30 Oct 2018, at 05:50:49 +0100, Frédéric Fauberteau wrote:
 >It is a dependency for libuuid

 Well, my guesses were completely wrong, then. I've seen bash as a
 build requirement before, but never ksh. I looked at revision 1.3 of
 Makefile.common, and this dependency was introduced as a workaround
 for NetBSD 6. I reverted that change and tested on NetBSD 8, and the
 package compiled.

 Have you considered something more like this?

 --- Makefile.common.old 2018-09-29 20:21:11.781672810 -0400
 +++ Makefile.common     2018-10-31 20:40:52.949178690 -0400
 @@ -16,9 +16,7 @@

  USE_GCC_RUNTIME=       yes
  USE_LIBTOOL=           yes
 -USE_TOOLS+=            pkg-config ksh
 -# XXX PR pkg/52877 configure is broken on netbsd6
 -CONFIG_SHELL=          ksh
 +USE_TOOLS+=            pkg-config
  GNU_CONFIGURE=         yes
  CONFIGURE_ARGS+=       --disable-all-programs
  CONFIGURE_ARGS+=       --disable-bash-completion
 @@ -30,3 +28,11 @@
  CONFIGURE_ARGS+=       --disable-nls

  TEST_TARGET=           check
 +
 +.include "../../mk/bsd.fast.prefs.mk"
 +
 +# XXX PR pkg/52877: configure is broken on netbsd6
 +.if (${OPSYS} == "NetBSD" && !empty(OS_VERSION:M[0123456].*))
 +USE_TOOLS+=            ksh
 +CONFIG_SHELL=          ksh
 +.endif

 Dave


From: "Benny Siegert" <bsiegert@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/53686 CVS commit: pkgsrc/devel/libuuid
Date: Sun, 4 Nov 2018 16:51:31 +0000

 Module Name:	pkgsrc
 Committed By:	bsiegert
 Date:		Sun Nov  4 16:51:31 UTC 2018

 Modified Files:
 	pkgsrc/devel/libuuid: Makefile.common

 Log Message:
 Only configure libuuid using ksh on NetBSD 6.

 The build of ksh is all but broken on modern systems. PR pkg/52877
 introduced ksh as CONFIGURE_SHELL to fix NetBSD 6, but this causes issues
 on NetBSD-8 and later (PR pkg/53686). Arguably, the workaround could be
 dropped, since we stopped supporting NetBSD 6.

 Fix proposed by David H. Gutteridge, thanks!


 To generate a diff of this commit:
 cvs rdiff -u -r1.4 -r1.5 pkgsrc/devel/libuuid/Makefile.common

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: pkg-manager->bsiegert
Responsible-Changed-By: bsiegert@NetBSD.org
Responsible-Changed-When: Sun, 04 Nov 2018 17:39:20 +0000
Responsible-Changed-Why:


State-Changed-From-To: open->feedback
State-Changed-By: bsiegert@NetBSD.org
State-Changed-When: Sun, 04 Nov 2018 17:39:20 +0000
State-Changed-Why:
Does the workaround fix the issue for you?


From: =?UTF-8?Q?Fr=C3=A9d=C3=A9ric_Fauberteau?= <triaxx@NetBSD.org>
To: gnats-bugs@netbsd.org
Cc: bsiegert@netbsd.org, pkg-manager@netbsd.org, pkgsrc-bugs@netbsd.org,
 gnats-admin@netbsd.org
Subject: Re: pkg/53686 (shells/pdksh building fails due to wrong preprocessor  checking)
Date: Fri, 23 Nov 2018 16:54:45 +0100

 Le 2018-11-04 18:39, bsiegert@NetBSD.org a écrit :
 > Synopsis: shells/pdksh building fails due to wrong preprocessor 
 > checking
 > 
 > Responsible-Changed-From-To: pkg-manager->bsiegert
 > Responsible-Changed-By: bsiegert@NetBSD.org
 > Responsible-Changed-When: Sun, 04 Nov 2018 17:39:20 +0000
 > Responsible-Changed-Why:
 > 
 > 
 > State-Changed-From-To: open->feedback
 > State-Changed-By: bsiegert@NetBSD.org
 > State-Changed-When: Sun, 04 Nov 2018 17:39:20 +0000
 > State-Changed-Why:
 > Does the workaround fix the issue for you?

 Yes, thanks!

State-Changed-From-To: feedback->closed
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Sat, 24 Nov 2018 10:53:12 +0000
State-Changed-Why:
Confirmed fixed, thanks!


>Unformatted:

Home	PR Database Search