NetBSD Problem Report #45374

From mm_lists@pulsar-zone.net  Sun Sep 18 07:49:51 2011
Return-Path: <mm_lists@pulsar-zone.net>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id F2ABA63B86B
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 18 Sep 2011 07:49:50 +0000 (UTC)
Message-Id: <201109180749.p8I7nj1N004718@ginseng.pulsar-zone.net>
Date: Sun, 18 Sep 2011 03:49:45 -0400
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@gnats.NetBSD.org
Subject: -current tools using machine-local CPU CFLAGS overriding mk.conf

>Number:         45374
>Category:       port-i386
>Synopsis:       -current tools using machine-local CPU CFLAGS overriding mk.conf
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    port-i386-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Sep 18 07:50:00 +0000 2011
>Last-Modified:  Sun Sep 18 23:05:01 +0000 2011
>Originator:     Matthew Mondor
>Release:        -current
>Organization:
>Environment:
Architecture: i386
Machine: i386
>Description:

I'm not really sure why GMP is now needed as part of /usr/tools, but I
noticed that while other tools will by default ignore mk.conf
CFLAGS/CCOPTS and ensure to use x86 agnostic CFLAGS (so that they're
portable among x86 systems), GMP will use its own heuristics and build
using CFLAGS optimized for the current machine.

While this could potentially increase the speed of system builds, it is
new behaviour, and /usr/tools can no longer be shared among x86
variants of a network.  I tried setting conservative CFLAGS
via /etc/mk.conf but those also are ignored by GMP.

>How-To-Repeat:

Build /usr/tools using build.sh on a build host; attempt to use them
from other x86 variants across a network.  I.e. a mix of 32-bit-mode
core2, p4 and other i686.

>Fix:

Currently unknown; tell GMP at configuration to not do heuristics,
or give it conservative x86 CFLAGS to use?  Or observe mk.conf provided
CFLAGS instead of ignoring them when building tools?  So that
CFLAGS+='-march=i686 -O2' would work for /usr/tools/ ...

Thanks,

>Audit-Trail:
From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@NetBSD.org
Cc: port-i386-maintainer@netbsd.org, gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org
Subject: re: port-i386/45374: -current tools using machine-local CPU CFLAGS overriding mk.conf
Date: Sun, 18 Sep 2011 18:17:07 +1000

 > I'm not really sure why GMP is now needed as part of /usr/tools, but I

 GCC >= 4.2 require GMP.

 > noticed that while other tools will by default ignore mk.conf
 > CFLAGS/CCOPTS and ensure to use x86 agnostic CFLAGS (so that they're
 > portable among x86 systems), GMP will use its own heuristics and build
 > using CFLAGS optimized for the current machine.
 > 
 > While this could potentially increase the speed of system builds, it is
 > new behaviour, and /usr/tools can no longer be shared among x86
 > variants of a network.  I tried setting conservative CFLAGS
 > via /etc/mk.conf but those also are ignored by GMP.
 > 
 > >How-To-Repeat:
 > 
 > Build /usr/tools using build.sh on a build host; attempt to use them
 > from other x86 variants across a network.  I.e. a mix of 32-bit-mode
 > core2, p4 and other i686.

 while i'd like to fix this problem, i don't think that we ever have
 tried to provide this level of compatibility for the tooldir.  our
 src/tools are designed to run on the machine you build netbsd, not on
 some other system.

 > >Fix:
 > 
 > Currently unknown; tell GMP at configuration to not do heuristics,
 > or give it conservative x86 CFLAGS to use?  Or observe mk.conf provided
 > CFLAGS instead of ignoring them when building tools?  So that
 > CFLAGS+='-march=i686 -O2' would work for /usr/tools/ ...

 can someone have a look at what happens here?  this is also the same
 basic issue that people had trying to build on some OSX versions,
 where GMP and MPFR would choose different defaults.  clearly there's
 some extra we need to do for GMP -- but i don't know it well enough
 to really fix it, and won't have a lot of time in the near future.

 FWIW, setting "ABI" when configuring GMP can alter the settings it
 chosses, so maybe picking a valid LCD here is what we need.


 .mrg.

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: port-i386/45374: -current tools using machine-local CPU CFLAGS
 overriding mk.conf
Date: Sun, 18 Sep 2011 12:23:39 -0400

 On Sun, 18 Sep 2011 08:20:04 +0000 (UTC)
 matthew green <mrg@eterna.com.au> wrote:

 >  while i'd like to fix this problem, i don't think that we ever have
 >  tried to provide this level of compatibility for the tooldir.  our
 >  src/tools are designed to run on the machine you build netbsd, not on
 >  some other system.
 >  
 >  > >Fix:
 >  > 
 >  > Currently unknown; tell GMP at configuration to not do heuristics,
 >  > or give it conservative x86 CFLAGS to use?  Or observe mk.conf provided
 >  > CFLAGS instead of ignoring them when building tools?  So that
 >  > CFLAGS+='-march=i686 -O2' would work for /usr/tools/ ...
 >  
 >  can someone have a look at what happens here?  this is also the same
 >  basic issue that people had trying to build on some OSX versions,
 >  where GMP and MPFR would choose different defaults.  clearly there's
 >  some extra we need to do for GMP -- but i don't know it well enough
 >  to really fix it, and won't have a lot of time in the near future.
 >  
 >  FWIW, setting "ABI" when configuring GMP can alter the settings it
 >  chosses, so maybe picking a valid LCD here is what we need.

 I think that indeed, traditionally the reason to avoid optimizations
 (and even ignore mk.conf provided CFLAGS, if any) was to avoid
 potential toolchain bugs, more than inter-machine compatibility.

 As long as it'd be possible to enforce mk.conf CFLAGS (at one's own
 risk, of course), in theory my problem would be fixed :)  Perhaps that
 it'd also be useful for benchmarks as custom optimizations could be
 used for the toolchain, potentially affecting the distribution building
 time...

 I also am not very familiar with GMP, however the package I built from
 pkgsrc observed the CFLAGS provided via /etc/mk.conf without issues.  I
 didn't previously have MPFR installed, but I just did a test and it
 also observes mk.conf CFLAGS.  Of course, that's a pkgsrc-conditional
 CFLAGS (.ifdef BSD_PKG_MK) in this case, but I also have a global one.

 It could perhaps simply be that GMP/MPFR take CFLAGS in consideration
 if in the environment, with the current tool building system unsetting
 it if present to avoid potential toolchain bugs?  If so, is it a
 problem in itself to just let CFLAGS survive, but set a default
 conservative one (i.e. "-O2" or even "") if absent in the parent
 environment?  It wouldn't matter to me if it also required setting
 another option to tell it to observe CFLAGS to build tools (like
 ALLOWTOOLSCFLAGS=yes or similar)...

 Thanks,
 -- 
 Matt

From: Manuel Bouyer <bouyer@antioche.eu.org>
To: matthew green <mrg@eterna.com.au>
Cc: gnats-bugs@NetBSD.org, port-i386-maintainer@NetBSD.org,
        gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org
Subject: Re: port-i386/45374: -current tools using machine-local CPU CFLAGS
 overriding mk.conf
Date: Sun, 18 Sep 2011 21:46:16 +0200

 On Sun, Sep 18, 2011 at 06:17:07PM +1000, matthew green wrote:
 > [...]
 > 
 > can someone have a look at what happens here?  this is also the same
 > basic issue that people had trying to build on some OSX versions,
 > where GMP and MPFR would choose different defaults.  clearly there's

 I guess it's the same issue when I tried build.sh tools on mips64.

 -- 
 Manuel Bouyer <bouyer@antioche.eu.org>
      NetBSD: 26 ans d'experience feront toujours la difference
 --

From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@NetBSD.org
Cc: port-i386-maintainer@netbsd.org, gnats-admin@netbsd.org,
    netbsd-bugs@netbsd.org, Matthew Mondor <mm_lists@pulsar-zone.net>
Subject: re: port-i386/45374: -current tools using machine-local CPU CFLAGS overriding mk.conf
Date: Mon, 19 Sep 2011 09:02:42 +1000

 i'm not sure what the diff for native vs. cross target GMP or MPFR
 is, really.  it could be that that is part of the issue here...


 .mrg.

Home	PR Database Search