NetBSD Problem Report #47504

From kre@munnari.OZ.AU  Fri Jan 25 10:12:07 2013
Return-Path: <kre@munnari.OZ.AU>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 058B963C07C
	for <gnats-bugs@gnats.NetBSD.org>; Fri, 25 Jan 2013 10:12:06 +0000 (UTC)
Message-Id: <201301251010.r0PAAgYJ010737@jade.coe.psu.ac.th>
Date: Fri, 25 Jan 2013 17:10:42 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@gnats.NetBSD.org
Subject: fonts/harfbuzz does not build on (standard) NetBSD 5.0 (native X11)
X-Send-Pr-Version: 3.95

>Number:         47504
>Category:       pkg
>Synopsis:       fonts/harfbuzz does not build on (standard) NetBSD 5.0 (native X11)
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jan 25 10:15:00 +0000 2013
>Last-Modified:  Fri Feb 01 04:50:02 +0000 2013
>Originator:     Robert Elz
>Release:        NetBSD 5.1_STABLE   (pkgsrc current 2013-01-25)
>Organization:
	Prince of Songkla University
>Environment:


System: NetBSD jade.coe.psu.ac.th 5.1_STABLE NetBSD 5.1_STABLE (JADE-1.12-20120130) #27: Tue Jan 31 05:20:31 ICT 2012 kre@jade.coe.psu.ac.th:/usr/obj/5/kernels/i386/JADE i386
Architecture: i386
Machine: i386
>Description:
	fonts/harfbuzz seems (after some update within the past couple of
	months) to require a version of graphics/freetype2 that is newer
	than the version that is (was) shipped with NetBSD 5.0 native X11.

	The build system is not detecting that, and the build fails.

>How-To-Repeat:
	I use pkg_comp with NetBSD 5.0 release sets installed, with libkver
	to simulate pure 5.0 (the build system, as shown above, is actually
	something beyond 5.1) and use native X11 from the NetBSD 5.0 release
	sets.

	In that environment, attempting to build fonts/harfbuzz results in...


  CXX    libharfbuzz_la-hb-ft.lo
hb-ft.cc:34:10: error: #include expects "FILENAME" or <FILENAME>
hb-ft.cc: In function 'hb_position_t hb_ft_get_glyph_h_advance(hb_font_t*, void*
, hb_codepoint_t, void*)':
hb-ft.cc:102: error: 'FT_Get_Advance' was not declared in this scope
hb-ft.cc: In function 'hb_position_t hb_ft_get_glyph_v_advance(hb_font_t*, void*
, hb_codepoint_t, void*)':
hb-ft.cc:118: error: 'FT_Get_Advance' was not declared in this scope
gmake[4]: *** [libharfbuzz_la-hb-ft.lo] Error 1

	Line 34 of hb-ft.cc is

#include FT_ADVANCES_H

	and so, obviously, FT_ADVANCES_H is not being defined, the #include
	fails, and everything that's supposed to be defined by that include
	file is missing (leading to the errors on lines 102 and 118).

	Nowhere (in the sandbox system) could I find any references to
	FT_ADVANCES_H leading me to conclude that the version of freetype2
	installed there does not have that include file (or at least doesn't
	define a symbol with which to reference it).

	My workaround for this, which allowed be to build fonts/harfbuzz
	(which is needed for pango, which in turn is needed for lots & lots)
	was to attempt to set
		BUILDLINK_API_DEPENDS.freetype2+= freetype2>=2.4.11
	(as 2.4.11 seems to be the current version, not because that
	version is the lowest one that is actually needed).

	That changed nothing, the build system was still happy to use the
	builtin version of freetype2 instead of the pkgsrc version.
	Manually installing freetype2 in the sandbox changed nothing (which
	is as it should be, pkgsrc is supposed to ignore packages that just
	happen to be installed but are not specifically requested.)

	So, and I am fairly sure this is not the correct solution (so I am
	not including a patch) I added

		USE_BUILTIN.freetype2=no

	to harfbuzz's pkgsrc Makefile.   That did what it was intended to do.

=> Full dependency freetype2>=2.4.7nb2: found freetype2-2.4.11
===> Overriding tools for harfbuzz-0.9.12
===> Extracting for harfbuzz-0.9.12
===> Patching for harfbuzz-0.9.12
=> Applying pkgsrc patches for harfbuzz-0.9.12
===> Creating toolchain wrappers for harfbuzz-0.9.12
ERROR: fontconfig>=2.4.2 fontconfig>=2.1nb2 is not installed; can't buildlink files.
*** Error code 1

	which was the first time (in this current build attempt anyway) that
	harfbuzz had picked pkgsrc's freetype2 as a dependency.

	That something required fontconfig because of this, but fontconfig
	didn't get auto-installed is a problem I think ... it should either
	be a dependency of freetype2, and been installed when pkgsrc added
	that one (from a pre-built binary package) or it should be included
	as a dependency of harfbuzz (directly, or indirectly via some
	buildlink3.mk file).

	I installed it manyally - just...

pkg_comp:5.conf# pkg_add /p/packages/All/fontconfig-2.10.2.tgz

	After that, the build of harfbuzz (0.9.12) succeeded, its binary
	package built with no further problems, or so it seemed.  Unfortunately
	attempting to build devel/pango resulted in ...

  CCLD     pango-querymodules
/pkg_comp/obj/pkgsrc/devel/pango/5/.buildlink/lib/libharfbuzz.so: undefined reference to `FT_Get_Advance'
gmake[4]: *** [pango-querymodules] Error 1
gmake[4]: Leaving directory `/pkg_comp/obj/pkgsrc/devel/pango/5/pango-1.32.5/pango'
gmake[3]: *** [all-recursive] Error 1

	This seems to be because the buildlink for pango managed to
	link to libfreetype.so.6.3.17 in the .buildlink/lib directory
	(which is the version in X11R7/lib) rather than libfreetype.so.6.10.0
	(which did get installed in /usr/pkg/lib - and this is building
	in a new, clean, sandbox, so that install of freetype2 happened as
	a dependency of something in devel/pango - it just wasn't used).

	Adding the line
		USE_BUILTIN.freetype2=no
	to devel/pango's pkgsrc Makefile fixed that, but this cannot be the
	right way, and almost certainly means that every package that uses
	pango is going to need a similar modification (or I would need to
	put that in mk.conf and I am fairly sure that is not the right way.)
	(I am yet to attempt to build any of the packages that need pango.)

	Could someone who understands buildlink3 and the use of the builtin
	versions of packages that are available in the release sets, and
	pkgsrc (with usually newer versions in pkgsrc) take a look at
	fonts/harfbuzz, and implement the correct magic to make it build
	correctly (with all the correct dependencies) on NetBSD 5.0  ??

	Please...

>Fix:
	No correct fix known (yet.)

>Audit-Trail:
From: "OBATA Akio" <obache@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/47504: fonts/harfbuzz does not build on (standard) NetBSD 5.0
 (native X11)
Date: Wed, 30 Jan 2013 20:52:55 +0900

 On Fri, 25 Jan 2013 19:15:00 +0900, <kre@munnari.oz.au> wrote:

 >> Fix:
 > 	No correct fix known (yet.)

 Put PREFER_PKGSRC.freetype2=pkgsrc on your /etc/mk.conf and rebuild whole packages,
 or upgrade your userland to 5.2.

 -- 
 OBATA Akio / obache@NetBSD.org

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/47504: fonts/harfbuzz does not build on (standard) NetBSD 5.0 (native X11)
Date: Wed, 30 Jan 2013 20:58:23 +0700

     Date:        Wed, 30 Jan 2013 11:55:04 +0000 (UTC)
     From:        "OBATA Akio" <obache@NetBSD.org>
     Message-ID:  <20130130115504.30FAF63EBDA@www.NetBSD.org>

   |  Put PREFER_PKGSRC.freetype2=pkgsrc on your /etc/mk.conf

 I thought something like that would probably work, but it really should
 not be needed - pkgsrc is supposed to support NetBSD 5 (including the 5.0
 series), the logic that determines whether freetype2 is included in the
 base system or not, and whether it is an adequate version, should simply
 determine that the pkgsrc version of freetype2 is needed, and include it,
 without my having to explicitly say so.

   | and rebuild whole packages,

 So far, all I have built that is likely to be affected (I think) is
 harfbuzz itself, and pango.  I have yet to start on building all the
 packages that depend upon those (on pango really), so the "rebuild" part
 shouldn't really be (very much) needed.   Otherwise I'd need a way to work
 out what packages need to be rebuilt (clearly it is not everything.)

   |  or upgrade your userland to 5.2.

 For the actual system(s) that I'm using NetBSD 5 on that (or up to 6) is
 actually quite likely - but for building NetBSD 5 packages, that will not
 happen - I always build all packages against the relevant x.0 release
 sets, with no modifications or updates permitted at all (for the purposes
 of package building).  My aim is to guarantee that the binary packages
 will run on any x.y (or x+1.* or later though those matter less to me)
 version of the system (of the correct architecture of course).   I do not
 want to have to have N different sets of packages for NetBSD 5 for 5.0, 5.0.1,
 5.0.2, 5.1, 5.2 ... (and then a whole new set of everything for every new
 NetBSD minor release).

 Thanks,

 kre

From: "OBATA Akio" <obache@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/47504: fonts/harfbuzz does not build on (standard) NetBSD 5.0
 (native X11)
Date: Thu, 31 Jan 2013 20:30:47 +0900

 Your strategy will not work as expected,
 because more than a little binary packages from 5.0.x will not work
 on 5.1.x and 5.2.x due to missing shared libraries, or you must install
 such shared libraries from 5.0.x binary sets.

 -- 
 OBATA Akio / obache@NetBSD.org

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/47504: fonts/harfbuzz does not build on (standard) NetBSD 5.0 (native X11)
Date: Thu, 31 Jan 2013 20:58:36 +0700

     Date:        Thu, 31 Jan 2013 11:35:03 +0000 (UTC)
     From:        "OBATA Akio" <obache@NetBSD.org>
     Message-ID:  <20130131113503.42DE463C07C@www.NetBSD.org>

   |  Your strategy will not work as expected,
   |  because more than a little binary packages from 5.0.x will not work
   |  on 5.1.x and 5.2.x due to missing shared libraries, or you must install
   |  such shared libraries from 5.0.x binary sets.

 In that case we need a compat package so supply the missing libraries, the
 same as is done to allow binaries from older versions to keep on running.

 NetBSD is supposed to be (among other things) much concerned with keeping
 binary compatibility, and generally does a good job - but tends to fail when
 it comes to the X related libraries.

 kre

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/47504: fonts/harfbuzz does not build on (standard) NetBSD
 5.0 (native X11)
Date: Fri, 1 Feb 2013 04:47:54 +0000

 On Thu, Jan 31, 2013 at 02:05:03PM +0000, Robert Elz wrote:
  > NetBSD is supposed to be (among other things) much concerned with keeping
  > binary compatibility, and generally does a good job - but tends to fail when
  > it comes to the X related libraries.

 This is, mostly, because we get a lot of fail shipped down from
 upstream.

 not that this is an excuse.

 I would recommend just adopting 5.1 and higher as your standard,
 really; a fair amount of stuff doesn't build in 5.1's X, but 5.0's is
 even worse.

 As for why builtin X packages are handled so much worse than builtin
 anything else, that's basically because of historical issues affecting
 x11-links.

 -- 
 David A. Holland
 dholland@netbsd.org

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.