NetBSD Problem Report #50830

From www@NetBSD.org  Thu Feb 18 19:08:36 2016
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.NetBSD.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 91AC67ACAB
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 18 Feb 2016 19:08:36 +0000 (UTC)
Message-Id: <20160218190835.79D5D7ACC2@mollari.NetBSD.org>
Date: Thu, 18 Feb 2016 19:08:35 +0000 (UTC)
From: code@mmayer.net
Reply-To: code@mmayer.net
To: gnats-bugs@NetBSD.org
Subject: Building pkgsrc packages broken on Darwin/OS X
X-Send-Pr-Version: www-1.0

>Number:         50830
>Category:       pkg
>Synopsis:       Building pkgsrc packages broken on Darwin/OS X
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    jperkin
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Feb 18 19:10:00 +0000 2016
>Closed-Date:    Wed Feb 21 08:59:16 +0000 2018
>Last-Modified:  Wed Feb 21 08:59:16 +0000 2018
>Originator:     Markus Mayer
>Release:        pkgsrc HEAD
>Organization:
>Environment:
Darwin triton.home.lan 15.3.0 Darwin Kernel Version 15.3.0: Thu Dec 10 18:40:58 PST 2015; root:xnu-3248.30.4~1/RELEASE_X86_64 x86_64

>Description:
Changes made during the commit below break pkgsrc on Darwin in 64 bit mode. Packages will build properly (as 64 bit binaries), but they won't install, because they have been mis-labeled as being for "i386" (i.e. 32 bit platforms) while the host machine will be seen as "x86_64" (i.e. 64 bit platform). pkgsrc reports an architecture mismatch and refuses to install the package.

Author: jperkin <jperkin>
Date:   Sun Jan 24 16:14:44 2016 +0000

    Attempt to bring sanity to how ABI and MACHINE_ARCH are set.

    Previously there were at least 5 different ways MACHINE_ARCH could be set,
    some statically and some at run time, and in many cases these settings
    differed, leading to issues at pkg_add time where there was conflict
    between the setting encoded into the package and that used by pkg_install.

    Instead, move to a single source of truth where the correct value based on
    the host and the chosen (or default) ABI is determined in the bootstrap
    script.  The value can still be overridden in mk.conf if necessary, e.g.
    for cross-compiling.

    ABI is now set by default and if unset a default is calculated based on
    MACHINE_ARCH.  This fixes some OS, e.g. Linux, where the wrong default was
    previously chosen.

    As a result of the refactoring there is no need for LOWER_ARCH, with
    references to it replaced by MACHINE_ARCH.  SPARC_TARGET_ARCH is also
    removed.

The problem is this change in mk/bsd.prefs.mk:

 .elif ${OPSYS} == "Darwin"
 LOWER_OPSYS?=          darwin
-.if empty(OS_VERSION:M[1-9].*.*)
-# Automatically select the ABI under Mac OS X Snow Leopard. We don't
-# use this at the moment because too many third party programs don't
-# work with it.
-#
-# _SYSCTL_HW_OPTIONAL_X86_64!= /usr/sbin/sysctl -n hw.optional.x86_64
-# .  if ${_SYSCTL_HW_OPTIONAL_X86_64} == "1"
-# ABI=                 64
-# .else
-# ABI=                 32
-#.  endif
-ABI=                   32
-LOWER_ARCH.32=         i386
-LOWER_ARCH.64=         x86_64
-LOWER_ARCH=            ${LOWER_ARCH.${ABI}}
-.else
-LOWER_ARCH!=           ${UNAME} -p
-.endif
-MACHINE_ARCH=          ${LOWER_ARCH}
-MAKEFLAGS+=            LOWER_ARCH=${LOWER_ARCH:Q}
 LOWER_OPSYS_VERSUFFIX= ${LOWER_OS_VERSION:C/([0-9]*).*/\1/}
 LOWER_VENDOR?=         apple

Specifically, it is the removal of "MACHINE_ARCH=${LOWER_ARCH}" that is causing the problems. I don't know pkgsrc internals well enough, but I am assuming that MACHINE_ARCH gets set to the output of "uname -p" by default, which is "i386" on Darwin, even on 64 bit machines. So, MACHINE_ARCH needs to be overridden for Darwin, otherwise it won't work.

>How-To-Repeat:
Attempt to build and install any package on OS X using a recent pkgsrc checkout and observe the (wrongfully reported) i386 vs. x86_64 architecture mismatch.
>Fix:
Probably not the best fix, but this worked for me in a pinch.

diff --git a/mk/bsd.prefs.mk b/mk/bsd.prefs.mk
index 9f7ab87..b2966d7 100644
--- a/mk/bsd.prefs.mk
+++ b/mk/bsd.prefs.mk
@@ -153,6 +153,16 @@ OS_VERSION=                ${_OS_VERSION:C/\(.*\)//}
 OS_VARIANT!=           ${UNAME} -s

 .elif ${OPSYS} == "Darwin"
+.if empty(OS_VERSION:M[1-9].*.*)
+ABI=                   32
+LOWER_ARCH.32=         i386
+LOWER_ARCH.64=         x86_64
+LOWER_ARCH=            ${LOWER_ARCH.${ABI}}
+.else
+LOWER_ARCH!=           ${UNAME} -p
+.endif
+MACHINE_ARCH=          ${LOWER_ARCH}
+MAKEFLAGS+=            LOWER_ARCH=${LOWER_ARCH:Q}
 LOWER_OPSYS?=          darwin
 LOWER_OPSYS_VERSUFFIX= ${LOWER_OS_VERSION:C/([0-9]*).*/\1/}
 LOWER_VENDOR?=         apple

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->jperkin
Responsible-Changed-By: bsiegert@NetBSD.org
Responsible-Changed-When: Tue, 08 Mar 2016 19:06:20 +0000
Responsible-Changed-Why:
Jonathan, could you take a look please?
You were the one that committed this change.


From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/50830: Building pkgsrc packages broken on Darwin/OS X
Date: Tue, 20 Feb 2018 03:08:42 +0000

 On Thu, Feb 18, 2016 at 07:10:00PM +0000, code@mmayer.net wrote:
  > >Description:
  >
  > Changes made during the commit below break pkgsrc on Darwin in 64
  > bit mode. Packages will build properly (as 64 bit binaries), but
  > they won't install, because they have been mis-labeled as being for
  > "i386" (i.e. 32 bit platforms) while the host machine will be seen
  > as "x86_64" (i.e. 64 bit platform). pkgsrc reports an architecture
  > mismatch and refuses to install the package.
  > 
  > Author: jperkin <jperkin>
  > Date:   Sun Jan 24 16:14:44 2016 +0000
  > 
  >     Attempt to bring sanity to how ABI and MACHINE_ARCH are set.
  > [...]

 Since it's now two years later, I'm guessing this got sorted out, even
 though there's been no followup in the PR? Pretty sure MacOS builds
 have been working for most of the time in between.

 -- 
 David A. Holland
 dholland@netbsd.org

State-Changed-From-To: open->feedback
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Tue, 20 Feb 2018 03:12:56 +0000
State-Changed-Why:
I asked a question


From: Benny Siegert <bsiegert@gmail.com>
To: gnats-bugs@netbsd.org
Cc: jperkin@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org, 
	code@mmayer.net
Subject: Re: pkg/50830: Building pkgsrc packages broken on Darwin/OS X
Date: Tue, 20 Feb 2018 14:44:24 +0000

 > Since it&#39;s now two years later, I&#39;m guessing this got sorted out,
 even
 > though there&#39;s been no followup in the PR? Pretty sure MacOS builds
 > have been working for most of the time in between.

 I used pkgsrc on macOS heavily in both 2016 and 17, and there has never
 been an issue like this. My guess is it is either a local problem, or it
 can be fixed by re-bootstrapping. This PR should be closed.

From: Markus Mayer <code@mmayer.net>
To: gnats-bugs@netbsd.org
Cc: jperkin@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org, 
	Markus Mayer <code@mmayer.net>
Subject: Re: pkg/50830: Building pkgsrc packages broken on Darwin/OS X
Date: Tue, 20 Feb 2018 11:02:20 -0800

 On 20 February 2018 at 06:45, Benny Siegert <bsiegert@gmail.com> wrote:
 > The following reply was made to PR pkg/50830; it has been noted by GNATS.
 >
 > From: Benny Siegert <bsiegert@gmail.com>
 > To: gnats-bugs@netbsd.org
 > Cc: jperkin@netbsd.org, gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org,
 >         code@mmayer.net
 > Subject: Re: pkg/50830: Building pkgsrc packages broken on Darwin/OS X
 > Date: Tue, 20 Feb 2018 14:44:24 +0000
 >
 >  > Since it&#39;s now two years later, I&#39;m guessing this got sorted out,
 >  even
 >  > though there&#39;s been no followup in the PR? Pretty sure MacOS builds
 >  > have been working for most of the time in between.
 >
 >  I used pkgsrc on macOS heavily in both 2016 and 17, and there has never
 >  been an issue like this. My guess is it is either a local problem, or it
 >  can be fixed by re-bootstrapping. This PR should be closed.

 Yes, it was fixed by re-bootstrapping. It had something to do with how
 64 bit builds were handled on pkgs/Darwin. That messed up
 arch-detection with an updated, but not re-bootstrapped pkgsrc tree.

 It can indeed be closed.

 Thanks,
 -Markus

State-Changed-From-To: feedback->closed
State-Changed-By: bsiegert@NetBSD.org
State-Changed-When: Wed, 21 Feb 2018 08:59:16 +0000
State-Changed-Why:
issue was fixed by re-bootsrapping


>Unformatted:

Home	PR Database Search