NetBSD Problem Report #57517

From tsutsui@ceres.dti.ne.jp  Mon Jul 10 14:30:27 2023
Return-Path: <tsutsui@ceres.dti.ne.jp>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 6C78C1A9241
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 10 Jul 2023 14:30:27 +0000 (UTC)
Message-Id: <202307101430.36AEUK00027916@ceres.dti.ne.jp>
Date: Mon, 10 Jul 2023 23:30:20 +0900 (JST)
From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Reply-To: tsutsui@ceres.dti.ne.jp
To: gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: pkgsrc-2023Q2 inconsistent freetype2 and fontconfig on NetBSD 9.3
X-Send-Pr-Version: 3.95

>Number:         57517
>Category:       pkg
>Synopsis:       pkgsrc-2023Q2 inconsistent freetype2 and fontconfig on NetBSD 9.3
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jul 10 14:35:00 +0000 2023
>Last-Modified:  Wed Jul 10 23:20:01 +0000 2024
>Originator:     Izumi Tsutsui
>Release:        NetBSD 9.3
>Organization:
>Environment:
System: NetBSD optiplex 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 2022  mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/i386/compile/GENERIC i386
Architecture: i386
Machine: i386
>Description:
NetBSD 9.3 has:
 - freetype2 2.10.4 and /usr/X11R7/lib/libfreetype.so.19.0
 - fontconfig 2.13.1 and /usr/X11R7/lib/libfontconfig.so.2.3

pkgsrc-2023Q2 has:
 - freetype2 2.13.0 in pkgsrc/graphics/freetype2/Makefile
   and BUILDLINK_API_DEPENDS in buildlink3.mk is also 2.13.0
  https://github.com/NetBSD/pkgsrc/commit/5c74c638
 - fontconfig 2.14.2 in pkgsrc/fonts/fontconfig/Makefile
   and also requires freetypes>=2.13.0 via buildlink3.mk,
   but BUILDLINK_API_DEPENDS in buildlink3.mk is 2.11.91
  https://github.com/NetBSD/pkgsrc/blob/da7caf4f/fonts/fontconfig/buildlink3.mk#L8

This causes mungled libfreetype2 dependencies, for example,
libcairo.so installed from pkgsrc/graphics/cairo requires:
---
% ldd /usr/pkg/lib/libcairo.so.2.11600.0 
/usr/pkg/lib/libcairo.so.2.11600.0:
        -lpixman-1.2 => /usr/X11R7/lib/libpixman-1.so.2
        -lm.0 => /usr/lib/libm.so.0
        -lc.12 => /usr/lib/libc.so.12
        -lgcc_s.1 => /usr/lib/libgcc_s.so.1
        -lfontconfig.2 => /usr/X11R7/lib/libfontconfig.so.2
        -lexpat.2 => /usr/lib/libexpat.so.2
        -lfreetype.19 => /usr/X11R7/lib/libfreetype.so.19
        -lz.1 => /usr/lib/libz.so.1
        -lbz2.1 => /usr/lib/libbz2.so.1
        -lfreetype.6 => /usr/pkg/lib/libfreetype.so.6
        -lpng16.16 => /usr/pkg/lib/libpng16.so.16
        -lbrotlidec.1 => /usr/pkg/lib/libbrotlidec.so.1
        -lbrotlicommon.1 => /usr/pkg/lib/libbrotlicommon.so.1
        -lpthread.1 => /usr/lib/libpthread.so.1
        -lxcb-shm.0 => /usr/X11R7/lib/libxcb-shm.so.0
        -lxcb.2 => /usr/X11R7/lib/libxcb.so.2
        -lXau.7 => /usr/X11R7/lib/libXau.so.7
        -lXdmcp.7 => /usr/X11R7/lib/libXdmcp.so.7
        -lxcb-render.0 => /usr/X11R7/lib/libxcb-render.so.0
        -lXrender.2 => /usr/X11R7/lib/libXrender.so.2
        -lXext.7 => /usr/X11R7/lib/libXext.so.7
        -lX11.7 => /usr/X11R7/lib/libX11.so.7
        -lrt.1 => /usr/lib/librt.so.1
% 
---

This causes broken rendering of emoji on cairo applications,
at least binaries that pull ruby-cairo:

--- /usr/pkgsrc-2023Q1/graphics/ruby-cairo/work.i386/.extract_makevars.mk      2023-07-
10 23:17:50.040900393 +0900
+++ /usr/pkgsrc-2023Q2/graphics/ruby-cairo/work.i386/.extract_makevars.mk      2023-07-10 23:19:30.906608270 +0900
@@ -7,7 +7,7 @@
 BUILDLINK_PREFIX.dl=	/usr
 BUILDLINK_PREFIX.expat=	/usr
 BUILDLINK_PREFIX.fontconfig=	/usr/X11R7
-BUILDLINK_PREFIX.freetype2=	/usr/X11R7
+BUILDLINK_PREFIX.freetype2=	/usr/pkg
 BUILDLINK_PREFIX.gettext=	/usr
 BUILDLINK_PREFIX.iconv=	/usr
 BUILDLINK_PREFIX.libX11=	/usr/X11R7
 :

>How-To-Repeat:
See above.

>Fix:
No idea.

Bump BUILDLINK_API_DEPENDS in fontconfig/buildlink3.mk to pull
proper pkgsrc version fontconfig for NetBSD 9.3?
(not sure it's acceptable in the release branch)

---
Izumi Tsutsui

>Audit-Trail:
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/57517: inconsistent freetype2 and fontconfig
Date: Sun, 23 Jul 2023 02:50:59 +0000

 On Mon, Jul 10, 2023 at 02:35:00PM +0000, Izumi Tsutsui wrote:
  > NetBSD 9.3 has:
  >  - freetype2 2.10.4 and /usr/X11R7/lib/libfreetype.so.19.0
  >  - fontconfig 2.13.1 and /usr/X11R7/lib/libfontconfig.so.2.3
  > 
  > pkgsrc-2023Q2 has:
  >  - freetype2 2.13.0 in pkgsrc/graphics/freetype2/Makefile
  >    and BUILDLINK_API_DEPENDS in buildlink3.mk is also 2.13.0
  >   https://github.com/NetBSD/pkgsrc/commit/5c74c638
  >  - fontconfig 2.14.2 in pkgsrc/fonts/fontconfig/Makefile
  >    and also requires freetypes>=2.13.0 via buildlink3.mk,
  >    but BUILDLINK_API_DEPENDS in buildlink3.mk is 2.11.91
  >   https://github.com/NetBSD/pkgsrc/blob/da7caf4f/fonts/fontconfig/buildlink3.mk#L8
  > 
  > This causes mungled libfreetype2 dependencies, for example,
  > libcairo.so installed from pkgsrc/graphics/cairo requires:
  > [snip]

 I see the same thing on oldish current and with pkgsrc HEAD. readelf
 -d on libcairo shows

  0x0000000000000001 (NEEDED)             Shared library: [libfreetype.so.6]
  0x0000000000000001 (NEEDED)             Shared library: [libfontconfig.so.2]

 and I have
    libfreetype.so.19 in /usr/X11R7/lib
    libfreetype.so.6 in /usr/pkg/lib
    libfontconfig.so.2 in /usr/X11R7/lib
    libfontconfig.so.1 in /usr/pkg/lib

 that is, it's linked against pkgsrc libfreetype, but base
 libfontconfig.

 The package itself thinks it's using pkgsrc libfontconfig:
    % pkg_info -n cairo | grep fontconfig
    fontconfig>=2.14.1nb1

 which is what's supposed to happen; as soon as we're using pkgsrc
 freetype2, everything that depends on it, which means fontconfig, is
 forced to USE_BUILTIN=no.

 In cairo itself, I get

    % make show-var VARNAME=IS_BUILTIN.fontconfig
    yes
    % make show-var VARNAME=USE_BUILTIN.fontconfig
    no

 but...
    % make show-var VARNAME=BUILDLINK_PREFIX.fontconfig
    /usr/X11R7

 which is all wrong, and predictably the libraries that are actually
 buildlinked are then the wrong ones. How it got that way, IDK, but
 that's the problem.

 Do you see the same thing? I'd expect so...

 -- 
 David A. Holland
 dholland@netbsd.org

From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: gnats-bugs@netbsd.org
Cc: dholland-pbugs@netbsd.org, tsutsui@ceres.dti.ne.jp
Subject: Re: pkg/57517: inconsistent freetype2 and fontconfig
Date: Mon, 24 Jul 2023 04:57:15 +0900

 >  In cairo itself, I get
 >  
 >     % make show-var VARNAME=IS_BUILTIN.fontconfig
 >     yes
 >     % make show-var VARNAME=USE_BUILTIN.fontconfig
 >     no
 >  
 >  but...
 >     % make show-var VARNAME=BUILDLINK_PREFIX.fontconfig
 >     /usr/X11R7
 >  
 >  which is all wrong, and predictably the libraries that are actually
 >  buildlinked are then the wrong ones. How it got that way, IDK, but
 >  that's the problem.
 >  
 >  Do you see the same thing? I'd expect so...

 Here I get:
 ---
 % pwd
 /usr/pkgsrc/graphics/ruby-cairo
 % cat CVS/Tag 
 Tpkgsrc-2023Q2
 % make show-var VARNAME=IS_BUILTIN.fontconfig
 yes
 % make show-var VARNAME=USE_BUILTIN.fontconfig
 no
 % make show-var VARNAME=BUILDLINK_PREFIX.fontconfig
 /usr/X11R7
 % 
 ---

 Same in graphics/cairo.

 ---
 Izumi Tsutsui

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/57517: inconsistent freetype2 and fontconfig
Date: Sat, 25 May 2024 23:43:38 +0000

 On Sun, Jul 23, 2023 at 02:55:02AM +0000, David Holland wrote:
  >  In cairo itself, I get
  >  
  >     % make show-var VARNAME=IS_BUILTIN.fontconfig
  >     yes
  >     % make show-var VARNAME=USE_BUILTIN.fontconfig
  >     no
  >  
  >  but...
  >     % make show-var VARNAME=BUILDLINK_PREFIX.fontconfig
  >     /usr/X11R7
  >  
  >  which is all wrong, and predictably the libraries that are actually
  >  buildlinked are then the wrong ones. How it got that way, IDK, but
  >  that's the problem.
  >  
  >  Do you see the same thing? I'd expect so...

 I just hit this again and had to find it from scratch, but I figured
 out where it's coming from. See what I just posted to tech-pkg.

 -- 
 David A. Holland
 dholland@netbsd.org

From: David Holland <dholland-pkgtech@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/57517: inconsistent freetype2 and fontconfig
Date: Wed, 10 Jul 2024 23:18:29 +0000

 Since there's been no movement, here's a copy of what I posted to
 tech-pkg.

    ------

 From: David Holland <dholland-pkgtech@netbsd.org>
 To: tech-pkg@netbsd.org
 Subject: transitive builtin logic is busted
 Date: Sat, 25 May 2024 23:37:14 +0000

 Today because of another failure I happened to notice that I was
 getting this:

    % make show-var VARNAME=USE_BUILTIN.fontconfig
    no
    % ls -l work/.buildlink/lib/libfontconfig.so
    lrwxr-xr-x  1 dholland  people  31 May 25 02:58 work/.buildlink/lib/libfontconfig.so -> /usr/X11R7/lib/libfontconfig.so

 IOW, it was supposedly not using the builtin fontconfig, but was
 buildlinking its files anyway instead of the pkgsrc version.

 This then causes everything downstream of fontconfig to be linked
 against two versions of freetype2, and things go downhill. (Though the
 native and pkgsrc versions of freetype2 are not very different and
 it's possible that there's no overt fallout or it's rare.)

 This turns out to be caused by the following:

    - fontconfig's builtin.mk sets USE_BUILTIN.fontconfig to yes
      because there's no local reason not to;
    - this causes mk/x11.builtin.mk, when included by fontconfig's
      builtin.mk, to set BUILDLINK_PREFIX.fontconfig to $(X11BASE);
    - the transitivity logic in bsd.buildlink3.mk then sets
      USE_BUILTIN.fontconfig to no because of freetype2;
    - the logic further down in bsd.buildlink3.mk for setting
      BUILDLINK_PREFIX.fontconfig is skipped because it's already
      set;
    - later on the buildlink logic itself symlinks files from the
      wrong prefix, and then that's what's used to build.

 I have no idea when this broke; it could have been some time ago. (I
 do remember a few months back running into a report that something
 seemed to be wrong with the transitivity logic, but I can't remember
 whose report or what it was about, or where I saw it.)

 x11.builtin.mk hasn't been changed in forever and the relevant
 BUILDLINK_PREFIX logic in bsd.buildlink3.mk hasn't changed recently
 either, and I doubt there have been major changes to the ordering of
 things.

 But, it did work once upon a time... anyway, the question is how to
 fix it.

 For the moment I've done this locally, which is enough to get my
 builds to work, but it doesn't seem in any way like the right thing.
 Ideas welcome...

 Index: bsd.buildlink3.mk
 ===================================================================
 RCS file: /cvsroot/pkgsrc/mk/buildlink3/bsd.buildlink3.mk,v
 retrieving revision 1.263
 diff -u -r1.263 bsd.buildlink3.mk
 --- bsd.buildlink3.mk	13 Jan 2024 20:26:47 -0000	1.263
 +++ bsd.buildlink3.mk	25 May 2024 23:34:15 -0000
 @@ -135,6 +135,9 @@
  .      if ${_ok_} != yes
           # not ok for it to be builtin; force it to pkgsrc
           USE_BUILTIN.${_pkg_:S/^-//}:=no
 +         # If the buildlink prefix is already set, clear it or we'll
 +         # buildlink the wrong files. XXX: this is a hack
 +.        undef BUILDLINK_PREFIX.${_pkg_:S/^-//}
           FORCED_PKGSRC+=${_pkg_:S/^-//}
           # using += here fails to evaluate ${_ok_} until after the loop
           FORCED_PKGSRC_REASONS:=${FORCED_PKGSRC_REASONS} ${_pkg_:S/^-//}:${_ok_:S/^no,//}


 -- 
 David A. Holland
 dholland@netbsd.org

Home	PR Database Search