NetBSD Problem Report #42197

From louis@maat.zabrico.com  Mon Oct 19 00:00:13 2009
Return-Path: <louis@maat.zabrico.com>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 764FF63B902
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 19 Oct 2009 00:00:13 +0000 (UTC)
Message-Id: <200910190000.n9J009OZ014427@maat.zabrico.com>
Date: Sun, 18 Oct 2009 20:00:09 -0400 (EDT)
From: louis@zabrico.com
Reply-To: louis@zabrico.com
To: gnats-bugs@gnats.NetBSD.org
Subject: pkg_alternatives portability problems (with patch)
X-Send-Pr-Version: 3.95

>Number:         42197
>Category:       pkg
>Synopsis:       pkg_alternatives portability problem with delimiter character
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    jmmv
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Oct 19 00:05:01 +0000 2009
>Closed-Date:    Wed Jun 13 15:47:19 +0000 2012
>Last-Modified:  Wed Jun 13 15:47:19 +0000 2012
>Originator:     Louis Guillaume
>Release:        NetBSD 5.0_STABLE
>Organization:
	Zabrico
>Environment:
System: NetBSD maat.zabrico.com 5.0_STABLE NetBSD 5.0_STABLE (GENERIC) #8: Sun Oct 11 18:47:13 EDT 2009 louis@maat.zabrico.com:/usr/obj/sys/arch/i386/compile/GENERIC i386
Architecture: i386
Machine: i386
>Description:
On some non-netbsd systems (OS X in particular), when pkg_alternatives uses
`tr' to delimit data in many of its functions, it chokes. This is because a
Latin-1 (0xac) character is used as a delimiter. If the locale is not
ISO-8859-1 or C, we end up with an error: "tr: Illegal byte sequence". 

It was discussed for a while on the mailing lists and the discussion went off
on a tangent about efficiency of shell command constructs etc. The thread is
here:

  http://mail-index.netbsd.org/pkgsrc-users/2008/07/20/msg007671.html


I refactored the pkg_alternatives.sh and wrapper.sh to remove the dependency
on the unusual character. I changed them to use awk for parsing the
configuration files. It works now and is no longer encumbered by the strange
use of a Latin-1 character. Hopefully this improves portability.

>How-To-Repeat:
     o Install pkgtools/pkg_alternatives on Mac OS X (Tiger, Leopard,
       Snow Leopard)
     o ensure the locale is en_US.UTF-8
     o try `pkg_alternatives status ruby18'. Most pkg_alternatives commands
       will fail.

>Fix:

Here is a cvs diff. I bumped the revision in the Makefile. Not sure if that
is the correct practice.

cvs diff: Diffing pkgtools/pkg_alternatives
Index: pkgtools/pkg_alternatives/Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pkg_alternatives/Makefile,v
retrieving revision 1.15
diff -u -r1.15 Makefile
--- pkgtools/pkg_alternatives/Makefile  9 Apr 2009 00:48:12 -0000       1.15
+++ pkgtools/pkg_alternatives/Makefile  18 Oct 2009 23:51:58 -0000
@@ -1,8 +1,8 @@
-# $NetBSD: Makefile,v 1.15 2009/04/09 00:48:12 joerg Exp $
 #

 DISTNAME=      pkg_alternatives-1.4
 CATEGORIES=    pkgtools
+PKGREVISION=   1
 MASTER_SITES=  # empty
 DISTFILES=     # empty

cvs diff: Diffing pkgtools/pkg_alternatives/files
Index: pkgtools/pkg_alternatives/files/pkg_alternatives.sh
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pkg_alternatives/files/pkg_alternatives.sh,v
retrieving revision 1.6
diff -u -r1.6 pkg_alternatives.sh
--- pkgtools/pkg_alternatives/files/pkg_alternatives.sh 15 Jan 2007 02:50:06 -0000      1.6
+++ pkgtools/pkg_alternatives/files/pkg_alternatives.sh 18 Oct 2009 23:51:59 -0000
@@ -56,11 +56,8 @@
     validate_package ${1}
     pkg=${PKG_DBDIR}/${1}*/+ALTERNATIVES

-    set -- $(cat ${pkg} | tr ' ' '?')
-    while [ ${#} -gt 0 ]; do
-        action_auto_wrapper $(echo ${1} | cut -d '?' -f 1)
-        shift
-    done
+    eval "$(awk '{ print "action_auto_wrapper " $1 }' $pkg )"
+
 }

 # -------------------------------------------------------------------------
@@ -161,11 +158,8 @@
     validate_package ${1}
     pkg=${PKG_DBDIR}/${1}*/+ALTERNATIVES

-    set -- $(cat ${pkg} | tr ' ' '?')
-    while [ ${#} -gt 0 ]; do
-        action_manual_wrapper $(echo ${1} | tr '?' ' ')
-        shift
-    done
+    eval "$(awk '{ print "action_manual_wrapper " $0 }' $pkg )"
+
 }

 # -------------------------------------------------------------------------
@@ -229,13 +223,10 @@
 # Each line should follow the semantics expected by action_register_wrapper.
 #
 action_register_package() {
+
     validate_args register ${#} -eq 1
+    eval "$(awk '{ print "action_register_wrapper " $0 }' $1 )"

-    set -- $(cat ${1} | tr ' ' '?')
-    while [ ${#} -gt 0 ]; do
-        action_register_wrapper $(echo ${1} | tr '?' ' ')
-        shift
-    done
 }

 # -------------------------------------------------------------------------
@@ -308,11 +299,10 @@
     validate_package ${1}
     pkg=${PKG_DBDIR}/${1}*/+ALTERNATIVES

-    set -- $(cat ${pkg} | tr ' ' '?')
-    while [ ${#} -gt 0 ]; do
-        action_status_wrapper $(echo ${1} | cut -d '?' -f 1)
-        shift
-    done
+    printf "\n    %s\n\n" "Wrapper alternatives for $1"
+    eval "$(awk '{ print "action_status_wrapper " $1 }' $pkg )"
+    echo
+
 }

 # -------------------------------------------------------------------------
@@ -322,6 +312,7 @@
 # Shows the current status for the given wrapper.
 #
 action_status_wrapper() {
+
     validate_args status ${#} -eq 1
     validate_wrapper ${1} yes

@@ -330,26 +321,28 @@
     sysconf=${Conf_Dir}/${wbase}
     userconf=~/.pkg_alternatives${Prefix}/${wbase}

-    [ $(id -un) = @ROOT_USER@ ] && userconf=
-    alts=$(cat ${userconf} ${sysconf} ${dbconf} 2>/dev/null | grep -v '^#' | \
-           tr ' ' '?')
+    flist=
+    [ -f "$userconf" ] && flist="$userconf"
+    [ -f "$sysconf" ]  && flist="$flist $sysconf"
+    [ -f "$dbconf" ]   && flist="$flist $dbconf"

-    found=
-    for a in ${alts}; do
-        prog=$(echo ${a} | cut -d '?' -f 1)
-        if [ -x ${prog} ]; then
-            found=$(echo ${a} | tr '?' ' ')
-            break
-        fi
-    done
+    [ $(id -un) = @ROOT_USER@ ] && userconf=

-    [ -n "${found}" ] ||
-        err "the wrapper \`${wbase}' exists but has no valid alternatives"
+    awk -v wbase="$wbase" -v prefix="$Prefix" '
+        BEGIN{
+          printf("      %s/%-20s %s ",prefix,wbase,"will actually execute")
+        }
+
+        !/^[ \t]*#/{
+          print
+        }
+
+        END{
+          if ( NR == 0 )
+              print "\n\n        (the wrapper " wbase " exists but has no alternatives.)\n\n"
+      }
+    ' $flist

-    echo "\`${wbase}' points to \`${found}'"
-    for a in $(echo ${alts} | tr ' ' '\n' | sort | uniq); do
-        echo "    candidate: $(echo ${a} | tr '?' ' ')"
-    done
 }

 # -------------------------------------------------------------------------
@@ -360,13 +353,10 @@
 # removes all associated alternatives from their respective wrappers.
 #
 action_unregister_package() {
+
     validate_args unregister ${#} -eq 1
+    eval "$(awk '{ print "action_unregister_wrapper " $0 }' $1 )"

-    set -- $(cat ${1} | tr ' ' '?')
-    while [ ${#} -gt 0 ]; do
-        action_unregister_wrapper $(echo ${1} | tr '?' ' ')
-        shift
-    done
 }

 # -------------------------------------------------------------------------
@@ -428,37 +418,17 @@
 # given wrapper, returns whether it is accepted or ignored.
 #
 filter() {
+
     [ ! -f @CONFDIR@/filter.conf ] && return 0

-    if [ ${Filter_Read} = no ]; then
-        Filter=$(cat @CONFDIR@/filter.conf | grep -v '^#' | tr ' ' '?')
-        Filter_Read=yes
-    fi
-
-    [ -z "${Filter}" ] && return 0
-
-    for f in ${Filter}; do
-        what=$(echo ${f} | cut -d '?' -f 1)
-        case ${what} in
-            accept)
-                name=$(echo ${f} | cut -d '?' -f 2- | tr '?' ' ')
-                if echo ${1} | grep "${name}" >/dev/null; then
-                    info "filter accepts \`${1}'"
-                    return 0
-                fi
-                ;;
-            ignore)
-                name=$(echo ${f} | cut -d '?' -f 2- | tr '?' ' ')
-                if echo ${1} | grep "${name}" >/dev/null; then
-                    info "filter ignores \`${1}'"
-                    return 1
-                fi
-                ;;
-            *)
-                warn "unknown filter type \`${what}'; ignoring"
-                ;;
-        esac
-    done
+    # the second field in filter.conf is a regular expression.
+
+    eval "$(awk '
+      /^[ \t]*#/ || /^[ \t]*$/ { next }
+      !/^accept/ && !/^ignore/ { print "warn No Such Filter Action: " $1 ; next }
+      { action=$1 ; sub ( "^" $1 FS "*" , "" ) }
+      to_check ~ $0 { print "info filter " action "s " to_check }
+    ' to_check="$1" @CONFDIR@/filter.conf)"

     true
 }
@@ -715,4 +685,3 @@

 main "${@}"

-# vim: syntax=sh:expandtab:shiftwidth=4:softtabstop=4
Index: pkgtools/pkg_alternatives/files/wrapper.sh
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pkg_alternatives/files/wrapper.sh,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 wrapper.sh
--- pkgtools/pkg_alternatives/files/wrapper.sh  25 Jan 2005 13:00:46 -0000      1.1.1.1
+++ pkgtools/pkg_alternatives/files/wrapper.sh  18 Oct 2009 23:51:59 -0000
@@ -1,4 +1,4 @@
-#!__SH__
+#!@SH@
 #
 # $NetBSD: wrapper.sh,v 1.1.1.1 2005/01/25 13:00:46 jmmv Exp $
 #
@@ -44,21 +44,20 @@
 else
     userfile=~/.pkg_alternatives/${wrapper}
 fi
-alternatives=$(cat ${userfile} __CONF_FILE__ __DB_FILE__ 2>/dev/null | \
-               grep -v "^#" | tr ' ' '?')

-found=
-for a in ${alternatives}; do
-    prog=$(echo ${a} | cut -d '?' -f 1)
-    if [ -x ${prog} ]; then
-        found=$(echo ${a} | tr '?' ' ')
-       break
-    fi
-done
+flist=
+[ -f "$userfile" ] && flist="$userfile"
+[ -f "__CONF_FILE__" ] && flist="$flist __CONF_FILE__"
+[ -f "__DB_FILE__" ]   && flist="$flist __DB_FILE__"
+
+eval "$(awk '{
+  cmd=$0
+  if ( "test -x " $1 " && echo 1" | getline ) {
+    print "exec " cmd " $@"
+    exit
+  }
+}' $flist )"
+    

-if [ -z "${found}" ]; then
-    echo "${progname}: no alternatives found" 1>&2
-    exit 1
-fi
-
-exec ${found} "${@}"
+echo "${progname}: no alternatives found" >&2
+exit 1



>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->jmmv
Responsible-Changed-By: wiz@NetBSD.org
Responsible-Changed-When: Wed, 21 Oct 2009 14:08:54 +0000
Responsible-Changed-Why:
Over to maintainer.


From: Louis Guillaume <louis@zabrico.com>
To: gnats-bugs@NetBSD.org
Cc: jmmv@NetBSD.org, pkg-manager@NetBSD.org, pkgsrc-bugs@NetBSD.org,
        gnats-admin@NetBSD.org
Subject: Re: pkg/42197 (pkg_alternatives portability problem with delimiter
 character)
Date: Sat, 08 May 2010 14:32:54 -0400

 This is a multi-part message in MIME format.
 --------------080300010105040000040009
 Content-Type: text/plain; charset=ISO-8859-1; format=flowed
 Content-Transfer-Encoding: 7bit


 Attached is a proper patch for this problem. The other one was mangled 
 by coping/pasting...

 Louis

 --------------080300010105040000040009
 Content-Type: application/x-gzip;
  name="pkg_alternatives-pr42197.patch.gz"
 Content-Transfer-Encoding: base64
 Content-Disposition: attachment;
  filename="pkg_alternatives-pr42197.patch.gz"

 H4sICG6r5UsAA3BrZ19hbHRlcm5hdGl2ZXMtcHI0MjE5Ny5wYXRjaADVGO122sbyd3iKiSDB
 BGEkYTs2PvSQxE7re3udHpO2P+JElcUCqkGi2hXYJeoT9SHvzK6EJMAudZo2Xc5Bq9XszOx8
 z575fXbThun1UATBmDdxYjtjwULfEd6M8ebAG7P15V0+KnU+fZQuXvWAKLSh6c54GASCSPHQ
 bT6AI31WCpkIPTbz/CGE+OBe4IO5e1Dqe4MBNCJohPgGm07TaDQeIoVH5j78x/HBMoznYFjt
 faNtHEDDwPGICNfr9QehPYT/ObeI1TTAPGxbRts6UlhL3S409g9004Q6Pg6h2y0BjZkz9vqO
 YPbUca+dIYPKwozVJyTQqSy+++/X9snLk7OLuEmfnjXrL759e3px/uLt2Q+nvRKUGgTLmQAU
 RWXHdQSiwK0xfAQRQpV+v1drCmw+QtbhHUKUY2gMBRjw/hj6gfpKw3EFSt92IhHY89CZTlmI
 WJk7CiRniNSNkFSfkEJjAGYt28tH3kCo137gs1KdZgxPCFplx5lfQ3UB09DzBWibyGhQMSGu
 QgW5h5pWqpcAJVGCMp7sLxpSDeaBKfVg7h9++YqYOH7kjO9QBaH9XSL+JC2s0EA9GH+HHizr
 SDdbULeslo4OQ4oow6njjmDs+QzPEUTjPgyC8TiYgxjhCkNOhedyYDdT5grWh6vbVFAhG3oc
 vTE9xi5iK619TJS7U4MFnaugeCccckghE82wX8DcQoKrxDMZmiTBzbZhfpplrNF8gG18HsW2
 jEPpYNbRUarYL9jDuHBExP+aYCetYgDapU9vT/ilf+lroP2Y4M6nCzRsJGZqW1jXCodrcVIi
 QKY/m6e2LEs/gHrLtPTniZ/2RsGcS690ozBkyK3iUh6Llod4SB82OWPxOHe7YoIw54hFoKXG
 SFW3jOPJJbMtQ7eIW8vUrSy+81vuBv4AzewVPuwTLyQrm185nCVGGHEWSpDfmruryR2NM2QD
 7ya/RykfLWzH62OV4tegA92LN2/e2t/3Ti+68B6ePs2QKmjEyTup6abfYpwn7NG0f6Vm1lfN
 Pps1/Wg8RkschmwKjRlUP5Sr+HqZWSKOvPVLcxiMMTZ01Pwdma5WSalpijEFkVsuACfsEGwe
 WD5h+bWwRbEtd2zYknxNpTYIIr/fSefoGeD5eHQST1z00mkYYGxIndK53ym9AXFzQ2EBt8Xk
 8WiLfkFWCe0iyo3RksZVyJzrbGngrbr81vpf2kvDR2ktJBcxievjxww/C9G/yX1S4778KTW4
 KmY9FCWHKzz+yOHgB8oXCmEl0QnFETQWuROVIJ8arUylGeOSsmcNqmoDjZenX5+dL7L3NJ7t
 aOrtCW8+aVgGxwlousKkS9S6NvfQStHBsZAY3yKnDJXEtFqGLEYnX748bn54B5fi/bNyc53e
 HXtOz08KsKjrHTi/gE4HQ3st/2WJiQKxCsU0dvJy1ZRs8Lku1UJVX5MxPEUfqwkGX2nXqVKl
 KWl5XU0DpI8BMpAaVMquaqsGv7RBsvtcFrv0ycd5EAp8RL73S63oFIoezVzH78tw2IY7LVr7
 /Bn/wJClXGs/V8qFbBJQonPIMjgPXM+hkq2YBMNgQi7qhQjOqajD5VRJvJA2In/7Ki6D/XN1
 XI7G31jJbaD6xdRye9ah3noO9T0Ta7o0+Rdyu46aE1HoczwwQ1WG4AnwUO2uy6akcLR2b+gH
 IesrfWL7LIp5/x08pkjeffXm/DWWf92mAtmlwKnCqCIBRqoAGegri9cSzr5gTj/G6Iu+uxbx
 FUiSczdTWEuua6rMEElaHao2lCtjPljG9V8priuwWFvhu5G5/kDlugSwaBHzkSOy1DS4P9u5
 FMAw5OCeGHEWE5cS/0o2o+E7E3YvCatxX0JMpJ83Tyk9PDphxoNnZcum7LvE4Q8whCk1JMxy
 GSvNZZxcHZkwV7+kaTk/jo+La8oG/yUCUcxuKRDzYQJ5tuEgcwfxaZF/7QdzP/FUELdTpgoR
 srTqseLO84cb2MoTYdxxV8ulctLJo+Nhb+8x7PC9lJDyRgod1IZHYyekRh/FQBeCu2ktUAjg
 aV7O6gksp5ZvlSYswGc3Is3bsvJQxtYk98Q3JWkCTFKBFMF5AL3IHSVODy9koG6r1ut4BeUi
 ieMd+Y1HV1iXaB8k6OseaM800EHToJbtEIHtjph7Db9RcllSlhaQyFxLkIKGtVa2YVl/pCtY
 yZna5rgm05WiF0YyP1BAf27u63tQPzjc11synJdg4niyJO3GMsGVYeZN2tgw+cK56fBRG5WA
 VYZwrtoy58y9vhh19to8GAhc5CKYdvZKZ9tdSacN4T9+E50xcucFtPwVLqHlD3JnaGxx/5yB
 P7KW1877YLbahtHey66dFbltrp5zGO+/cTZJ1fSHim6UH9t27xvbLtXLj7u9b7oyGZehcs7E
 y95JG/ISSU8vWW0aZtPKMfzzZDKD05spVAiFrBL2dMvEImFPN4+kUbExZ1lXTVxv6qqpmVZE
 45IMWfmPhTaZMMRg22Tn9uuzb09tG99OXqbzYq98uRaX0gSvfShrKwkeLT7tRoud6JKTXI7e
 uhv9w070D7vQR7nWk2SjYmja2OeaepLNhqZeLqeABcmtAKsOvQiR7VvK+I7OPg9BIbpYX1O4
 cid4ToNmsmXTBMNtKBoZITXCKGVgkmKGTMjLX1kcKoHL0EgNJcIiKtrWTS+9bjzZLWJUjNOW
 LKnSlfx/LXTaeRWoDkppR6bp9mrnpzSkgfnVUyvZg+Qo2aqiT7KUYk/DZ317vIS2nqD8PzKb
 ZfHQGwAA
 --------------080300010105040000040009--

From: "Jonathan Perkin" <jperkin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42197 CVS commit: pkgsrc/pkgtools/pkg_alternatives
Date: Wed, 13 Jun 2012 15:35:32 +0000

 Module Name:	pkgsrc
 Committed By:	jperkin
 Date:		Wed Jun 13 15:35:32 UTC 2012

 Modified Files:
 	pkgsrc/pkgtools/pkg_alternatives: Makefile
 	pkgsrc/pkgtools/pkg_alternatives/files: pkg_alternatives.sh wrapper.sh

 Log Message:
 Update to 1.6, changes:

  - Use the correct id(1) program.  Fixes PR pkg/40691

  - Avoid unprintable characters, some implementations of tr(1) have issues
    with them.  Instead go for what hopefully will be a unique string, as
    well as making use of shell parameter substitution.  Fixes PR pkg/42197

 These changes combined make pkg_alternatives work correctly on Solaris.


 To generate a diff of this commit:
 cvs rdiff -u -r1.17 -r1.18 pkgsrc/pkgtools/pkg_alternatives/Makefile
 cvs rdiff -u -r1.6 -r1.7 \
     pkgsrc/pkgtools/pkg_alternatives/files/pkg_alternatives.sh
 cvs rdiff -u -r1.1.1.1 -r1.2 \
     pkgsrc/pkgtools/pkg_alternatives/files/wrapper.sh

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: jperkin@NetBSD.org
State-Changed-When: Wed, 13 Jun 2012 15:47:19 +0000
State-Changed-Why:
Fixed.  I went with a simpler change, but thank you for your contribution.


>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.