NetBSD Problem Report #50317

From www@NetBSD.org  Thu Oct  8 21:09:00 2015
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 99591A5858
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  8 Oct 2015 21:09:00 +0000 (UTC)
Message-Id: <20151008210859.0DEEAA65BB@mollari.NetBSD.org>
Date: Thu,  8 Oct 2015 21:08:59 +0000 (UTC)
From: mmayer@mmayer.net
Reply-To: mmayer@mmayer.net
To: gnats-bugs@NetBSD.org
Subject: Determining SDK under Darwin is problematic
X-Send-Pr-Version: www-1.0

>Number:         50317
>Category:       pkg
>Synopsis:       Determining SDK under Darwin is problematic
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    macos-pkg-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 08 21:10:00 +0000 2015
>Closed-Date:    Wed Nov 07 16:53:56 +0000 2018
>Last-Modified:  Wed Nov 07 16:55:00 +0000 2018
>Originator:     Markus Mayer
>Release:        pkgsrc tip as of Oct 8, 2015
>Organization:
>Environment:
Darwin edmonds.xxxx 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64

>Description:
The problem is in mk/platform/Darwin.mk and how it determines the value for OSX_SDK_PATH.

All is well as long as Xcode and OS X "match", meaning the OS X version installed on the machine is the same that Xcode targets as its main/primary target. (It looks like Apple is now only shipping 1 OS X SDK with Xcode. I think they used to ship newest plus previous version in the past.)

In my case, I had Xcode 7 (which was automatically updated), but was still running OS X 10.10 (which needs manual intervention to upgrade). In preparation for said upgrade, I started re-building pkgsrc, so it'll be installed in /opt/pkg rather than /usr/pkg, to be ready for the write-protected /usr directory in El Capitan.

The problem I ran into was that it couldn't determine the SDK, which resulted in the somewhat misleading error:

$ bmake
ERROR: This package has set PKG_FAIL_REASON:
ERROR: libtool-base-2.4.2nb10 requires a working dlopen().
*** Error code 1

Stop.
bmake: stopped in /Users/markus/pkgsrc/devel/libtool-base

And hence, pretty much nothing would build. The reason for the error is that it can't find dlfcn.h, because the OS_SDK_PATH is incorrectly set (i.e. empty).

What is happening is the following (all lines from mk/platform/Darwin.mk):

OSX_VERS!=      sw_vers -productVersion

This is set to 10.10.5, since that's what the machine is running.

Next, it tries this:

OSX_SDK_PATH!= /usr/bin/xcrun --sdk macosx${OSX_VERS} --show-sdk-path 2>/dev/null || echo /nonexistent

That returns an error, since there's no SDK for 10.10.5 on my machine. All it has (due to the automatic Xcode 7 update) is the SDK for 10.11.

Running the command manually proves this:

$ /usr/bin/xcrun --sdk macosx10.10.5 --show-sdk-path
xcodebuild: error: SDK "macosx10.10.5" cannot be located.
xcodebuild: error: SDK "macosx10.10.5" cannot be located.
xcrun: error: unable to lookup item 'Path' in SDK 'macosx10.10.5'
$ /usr/bin/xcrun --sdk macosx10.11 --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk

However, more importantly, this also works:

$ /usr/bin/xcrun --sdk macosx --show-sdk-path
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk

If one does NOT specify the OS X version when querying for the SDK, it'll return the right one!


>How-To-Repeat:
Take any Mac running a newer Xcode (e.g. Xcode 7) and an older OS X (e.g. OS X 10.10) such that the SDK provided under /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/ does NOT match the version of the OS that is running.

pkgsrc will be unable to find the proper SDK and pretty much any and all attempts to build anything (beyond bootstrapping) will fail.
>Fix:
I can't verify that it'll do the right thing for all older OS X versions. From what I've found, it would seem the following change might be sensible:

-OSX_SDK_PATH!= /usr/bin/xcrun --sdk macosx${OSX_VERS} --show-sdk-path 2>/dev/null || echo /nonexistent
+OSX_SDK_PATH!= /usr/bin/xcrun --sdk macosx --show-sdk-path 2>/dev/null || echo /nonexistent

If this doesn't quite work due to older OS X releases, it would be good to check the return value of "/usr/bin/xcrun --sdk macosx${OSX_VERS} ..." and run "/usr/bin/xcrun --sdk macosx ..." if the former failed. I am pretty sure doing this will save others quite a bit of time, as this problem isn't particularly easy to find.

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->macos-pkg-people
Responsible-Changed-By: ryoon@NetBSD.org
Responsible-Changed-When: Thu, 08 Oct 2015 22:13:13 +0000
Responsible-Changed-Why:
OS X problem.


State-Changed-From-To: open->closed
State-Changed-By: schmonz@NetBSD.org
State-Changed-When: Wed, 07 Nov 2018 16:53:56 +0000
State-Changed-Why:
Applied the version-free lookup as a fallback. Thanks for the PR!


From: "Amitai Schleier" <schmonz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/50317 CVS commit: pkgsrc/mk/platform
Date: Wed, 7 Nov 2018 16:50:27 +0000

 Module Name:	pkgsrc
 Committed By:	schmonz
 Date:		Wed Nov  7 16:50:27 UTC 2018

 Modified Files:
 	pkgsrc/mk/platform: Darwin.mk

 Log Message:
 If an SDK matching our OS X version isn't found -- for instance, because
 OS X has been upgraded and Xcode hasn't been -- try again without
 specifying the version. This (1) works better and (2) gives a better
 error message when it doesn't. From Markus Mayer in PR pkg/50317.


 To generate a diff of this commit:
 cvs rdiff -u -r1.88 -r1.89 pkgsrc/mk/platform/Darwin.mk

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.