NetBSD Problem Report #36257
From kre@munnari.OZ.AU Wed May 2 07:05:44 2007
Return-Path: <kre@munnari.OZ.AU>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by narn.NetBSD.org (Postfix) with ESMTP id 8457E63B853
for <gnats-bugs@gnats.NetBSD.org>; Wed, 2 May 2007 07:05:44 +0000 (UTC)
Message-Id: <200705020705.l4275e77018566@jade.coe.psu.ac.th>
Date: Wed, 2 May 2007 14:05:40 +0700 (ICT)
From: kre@munnari.OZ.AU
To: gnats-bugs@NetBSD.org
Subject: games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0]
X-Send-Pr-Version: 3.95
>Number: 36257
>Category: pkg
>Synopsis: games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0]
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: pkg-manager
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed May 02 07:10:00 +0000 2007
>Closed-Date: Sat May 12 05:11:00 +0000 2007
>Last-Modified: Sat May 12 05:11:00 +0000 2007
>Originator: Robert Elz
>Release: NetBSD 3.99.15 (pkgsrc current within past 12 hours)
>Organization:
Prince of Songkla University
>Environment:
System: NetBSD jade.coe.psu.ac.th 3.99.15 NetBSD 3.99.15 (GENERIC-1.696-20060125) #8: Wed Jan 25 04:59:39 ICT 2006 kre@jade.coe.psu.ac.th:/usr/obj/current/kernels/JADE_ASUS i386
Architecture: i386
Machine: i386
>Description:
I use pkg_comp and libkver with NetBSD 3.0 (binary sets, and
sources) installed in the sandbox.
Attempting to build games/wesnoth fails with mggfmt syntax
errors.
>How-To-Repeat:
Attempt to build games/wesnoth on a (more or less) virgin 3.0
Expect to see ...
:r! cat /dev/tty
===> Building for wesnoth-1.2.4
/usr/bin/make all-recursive
Making all in po
Making all in wesnoth
test ! -f ./wesnoth.pot || test -z "af.gmo bg.gmo ca.gmo ca_ES@valencia.gmo cs.
gmo da.gmo de.gmo el.gmo en_GB.gmo es.gmo eo.gmo et.gmo eu.gmo fi.gmo fr.gmo gl_
ES.gmo he.gmo hu.gmo id.gmo it.gmo ja.gmo ko.gmo la.gmo nl.gmo nb_NO.gmo pl.gmo
pt.gmo pt_BR.gmo ro.gmo ru.gmo sk.gmo sl.gmo sr.gmo sv.gmo tl.gmo tr.gmo zh_CN.g
mo" || /usr/bin/make af.gmo bg.gmo ca.gmo ca_ES@valencia.gmo cs.gmo da.gmo de.gm
o el.gmo en_GB.gmo es.gmo eo.gmo et.gmo eu.gmo fi.gmo fr.gmo gl_ES.gmo he.gmo hu
.gmo id.gmo it.gmo ja.gmo ko.gmo la.gmo nl.gmo nb_NO.gmo pl.gmo pt.gmo pt_BR.gmo
ro.gmo ru.gmo sk.gmo sl.gmo sr.gmo sv.gmo tl.gmo tr.gmo zh_CN.gmo
rm -f af.gmo && /usr/bin/msgfmt -c --statistics -o af.gmo af.po
af.po:10812: keyword "msgid_plural" unknown
af.po:10811: missing `msgstr' section
af.po:10812: parse error
af.po:10819: keyword "msgid_plural" unknown
af.po:10818: missing `msgstr' section
af.po:10819: parse error
af.po:10825: keyword "msgid_plural" unknown
af.po:10824: missing `msgstr' section
af.po:10825: parse error
af.po:10831: keyword "msgid_plural" unknown
af.po:10830: missing `msgstr' section
af.po:10831: parse error
af.po:11636: keyword "msgid_plural" unknown
af.po:11635: missing `msgstr' section
af.po:11636: parse error
af.po:11684: keyword "msgid_plural" unknown
af.po:11683: missing `msgstr' section
af.po:11684: parse error
af.po:12491: keyword "msgid_plural" unknown
af.po:12490: missing `msgstr' section
too many errors, aborting
*** Error code 1
>Fix:
From various mesages I've seen on the lists, I suspect this
(kind of) problem is well known to those who understand what is
happening here, and the fix is most likely trivial, so I'm
not looking into this one at all...
>Release-Note:
>Audit-Trail:
From: "OBATA Akio" <obache@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/36257: games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0]
Date: Sat, 05 May 2007 20:23:26 +0900
On Wed, 02 May 2007 16:10:00 +0900, <kre@munnari.OZ.AU> wrote:
>> Fix:
> From various mesages I've seen on the lists, I suspect this
> (kind of) problem is well known to those who understand what is
> happening here, and the fix is most likely trivial, so I'm
> not looking into this one at all...
Pkgsrc already have a wrapper msgfmt tool, handling 'plural' issue.
But this package's configure script judge the wrapper tool is useless,
and use /usr/bin/msgfmt instead, because the wrapper script doesn't allow
/dev/null as input file.
Following patch should fix this problem.
Index: mk/tools/msgfmt.sh
===================================================================
RCS file: /cvsroot/pkgsrc/mk/tools/msgfmt.sh,v
retrieving revision 1.26
diff -u -r1.26 msgfmt.sh
--- mk/tools/msgfmt.sh 15 Dec 2006 13:15:07 -0000 1.26
+++ mk/tools/msgfmt.sh 5 May 2007 11:06:23 -0000
@@ -143,7 +143,7 @@
# If the input file cannot be found as named, then also search for a file
# with the same name that ends in ".po".
#
-if test ! -f "$pofile"; then
+if test ! -f "$pofile" -a ! -c "$pofile"; then
case "$pofile" in
*.po) popofile=$pofile ;;
*) popofile=${pofile}.po ;;
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257: games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0]
Date: Sat, 05 May 2007 20:18:59 +0700
Date: Sat, 5 May 2007 11:25:02 +0000 (UTC)
From: "OBATA Akio" <obache@NetBSD.org>
Message-ID: <20070505112502.84D3F63B853@narn.NetBSD.org>
| Pkgsrc already have a wrapper msgfmt tool, handling 'plural' issue.
| But this package's configure script judge the wrapper tool is useless,
| and use /usr/bin/msgfmt instead, because the wrapper script doesn't allow
| /dev/null as input file.
|
| Following patch should fix this problem.
Thanks for working out what's happening. I'd actually suggest the
patch below - I don't think the script really cares what kind of file
it is dealing with, just whether the name exists or not, or whether it
needs to ad a ".po" suffix to file the file. So, I suggest using the
test operator that checks for existence, rather than building a list of
all the plausible file types that the input file might be (how about
symlinks, or named pipes, or ...)
I have tested the patch below, and it works fine to build games/wesnoth,
so once either your patch, or mine, is incorporated, this PR can be closed.
kre
From: Robert Elz <kre@munnari.OZ.AU>
To: Thomas Klausner <wiz@NetBSD.org>, gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257: games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0]
Date: Sat, 05 May 2007 21:39:26 +0700
Date: Sat, 5 May 2007 16:27:42 +0200
From: Thomas Klausner <wiz@NetBSD.org>
Message-ID: <20070505142742.GD7984@danbala.tuwien.ac.at>
| Missing patch...
Oops - yes, I make that stupid mistake more times than I care
to count...
Here it is.
kre
(To mk/tools/msgfmt.sh)
--- msgfmt.sh.OLD 2006-12-18 11:41:26.000000000 +0700
+++ msgfmt.sh 2007-05-05 19:52:19.000000000 +0700
@@ -143,12 +143,12 @@
# If the input file cannot be found as named, then also search for a file
# with the same name that ends in ".po".
#
-if test ! -f "$pofile"; then
+if test ! -e "$pofile"; then
case "$pofile" in
*.po) popofile=$pofile ;;
*) popofile=${pofile}.po ;;
esac
- if test ! -f "$popofile"; then
+ if test ! -e "$popofile"; then
echo 1>&2 "$0: error while opening \"$pofile\" for reading: No such file or directory"
exit 1
fi
From: OBATA Akio <obache@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: PR/36257 CVS commit: pkgsrc/mk/tools
Date: Mon, 7 May 2007 09:31:05 +0000 (UTC)
Module Name: pkgsrc
Committed By: obache
Date: Mon May 7 09:31:05 UTC 2007
Modified Files:
pkgsrc/mk/tools: msgfmt.sh
Log Message:
Change input file lookup behavior to close to gnu msgfmt's implementation.
* allow "-" (stdin)
* allow other than regurar file (should fix PR 36257).
* if doesn't exist, try to append extension ".po" and ".pot".
To generate a diff of this commit:
cvs rdiff -r1.26 -r1.27 pkgsrc/mk/tools/msgfmt.sh
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->feedback
State-Changed-By: obache@netbsd.org
State-Changed-When: Wed, 09 May 2007 13:25:18 +0000
State-Changed-Why:
Fix commited. Please confirm again.
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Wed, 09 May 2007 23:07:04 +0700
Date: Wed, 9 May 2007 13:25:20 +0000 (UTC)
From: obache@NetBSD.org
Message-ID: <20070509132520.C851F63B853@narn.NetBSD.org>
| Fix commited. Please confirm again.
Yes, it works with your (better) patch. On NetBSD 3.0 anyway.
However, there are two suggestions I'd make to make it even better.
First, it is never really safe with test to use an unknown string
variable unprotected in a = or != comparison.
That is, in a test like
if test "$pofile" != "-"
if the value of $pofile just happens to be "-e" or something like
that, bad things happen (ie: nothing works the way you want, and
usually the result is a bunch of strange error messages.)
The usual fix is to protect at least the leading edge of the variable value,
so it cannot possibly accidentally become a test operator, I like
to do it like ...
if test "%${pofile}%" != "%-%"
(the trailing '%' accomplishes nothing useful, I use it for aesthetics
only.)
(Note: you don't need, and cannot use, this trick on the operand of
one of the file status test operators, for what I hope are obvious
reasons, so the
-e "$pofile"
needs to stay as it is.)
Second, the test grammar has always been fragile, never really well
defined (or it used not to be, maybe posix has fixed that), so that using
complex test expressions (ones containing logical connectives, -a and -o)
has tended to give different results on one version of test than another
(it all ends up depending upon just what test decides to do first, in any
particular implementation).
To avoid that kind of problem, and especially these days with test being
a built in command in every (Bourne type) shell worth mentioning, it is
generally easier to use shell logical operators (&& and ||) and simply
run multiple test commands. So, instead of
if test "$pofile" != "-" -a ! -e "$pofile"
use
if test "%${pofile}%" != "%-%" && test ! -e "$pofile"
(or something like that, many people just use X$var instead of %$var%,
and others prefer the [ ... ] variant instead of "test" - none of that
matters.)
kre
From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Wed, 9 May 2007 18:16:57 +0200
On Wed, May 09, 2007 at 04:10:04PM +0000, Robert Elz wrote:
> That is, in a test like
>
> if test "$pofile" != "-"
>
> if the value of $pofile just happens to be "-e" or something like
> that, bad things happen (ie: nothing works the way you want, and
> usually the result is a bunch of strange error messages.)
Are you sure?
% test "-e" != "-" && echo ok
ok
> The usual fix is to protect at least the leading edge of the variable value,
> so it cannot possibly accidentally become a test operator, I like
> to do it like ...
>
> if test "%${pofile}%" != "%-%"
Read the specification for test. The operator order is far from trivial,
but most cases of using prefix values serve the purpose of hacking
around Solaris /bin/sh bugs with empty arguments and the like.
> To avoid that kind of problem, and especially these days with test being
> a built in command in every (Bourne type) shell worth mentioning, it is
> generally easier to use shell logical operators (&& and ||) and simply
> run multiple test commands.
Avoiding -a and -o is a good thing, yes. That's different from this
though.
Joerg
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Thu, 10 May 2007 00:47:06 +0700
Date: Wed, 9 May 2007 16:20:04 +0000 (UTC)
From: Joerg Sonnenberger <joerg@britannica.bec.de>
Message-ID: <20070509162004.1101363B853@narn.NetBSD.org>
| Are you sure?
Yes, though it certainly depends upon both the version of test,
and the number of operands - I haven't looked at the modern (in
sh) implementations, but original test was totally heuristically
driven - if it had 3 args, and the middle one was an operator, it
assumed the other two must be operands, which is generally pretty
safe, but starts getting much trickier when there are more operands).
Since it is so trivial to avoid even the possibility of a problem
in a way that works for every version of test, it is generally a
good idea to do that.
| Read the specification for test.
I don't much care (for this purpose, as distinct from writing/correcting
an implementation) what the specification says - what matters is what
the implementations in the wild actually do.
| The operator order is far from trivial,
| but most cases of using prefix values serve the purpose of hacking
| around Solaris /bin/sh bugs with empty arguments and the like.
Solaris test (and solaris /bin/sh) were (at least based upon) the
original versions, back in the days when in cases of ambiguity in the
man pages, the source code ruled...
Working with those "bugs" is one of the things that portable code needs
to handle.
| Avoiding -a and -o is a good thing, yes. That's different from this
| though.
Aside from cases where you can rewrite things to simplify and avoid
a logical operator completely, the only reason to avoid -a and -o
(using && and || instead) is to avoid strangeness in implementations of
test. If you were confident that test whatever -a whatever_else
would have the same effect (always) as test whatever && test whatever_else
then using the former would always be better - even where test is built
into the shell, it means less work, and also allows test to cache stat
results (in expressions like "test -b x -o -c x").
kre
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, pkg-manager@netbsd.org,
gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org, kre@munnari.OZ.AU
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Wed, 9 May 2007 14:52:42 -0400
On May 9, 6:50pm, kre@munnari.OZ.AU (Robert Elz) wrote:
-- Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBS
| Aside from cases where you can rewrite things to simplify and avoid
| a logical operator completely, the only reason to avoid -a and -o
| (using && and || instead) is to avoid strangeness in implementations of
| test. If you were confident that test whatever -a whatever_else
| would have the same effect (always) as test whatever && test whatever_else
| then using the former would always be better - even where test is built
| into the shell, it means less work, and also allows test to cache stat
| results (in expressions like "test -b x -o -c x").
You can always use parentheses:
test \( -b x \) -o \( -c x \)
I always use parentheses with -a and -o and have not encountered
a test implementation that does not work properly.
christos
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Thu, 10 May 2007 03:07:51 +0700
Date: Wed, 9 May 2007 18:55:03 +0000 (UTC)
From: christos@zoulas.com (Christos Zoulas)
Message-ID: <20070509185503.7E26263B853@narn.NetBSD.org>
| You can always use parentheses:
They're not immune from problems either. Consider:
test \( = \)
true or false? Either answer is possible (and defensible) and I'm
pretty sure both have been given by different implementations.
kre
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, pkg-manager@netbsd.org,
gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org, kre@munnari.OZ.AU
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Wed, 9 May 2007 16:20:59 -0400
On May 9, 8:10pm, kre@munnari.OZ.AU (Robert Elz) wrote:
-- Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBS
| The following reply was made to PR pkg/36257; it has been noted by GNATS.
|
| From: Robert Elz <kre@munnari.OZ.AU>
| To: gnats-bugs@NetBSD.org
| Cc:
| Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
| Date: Thu, 10 May 2007 03:07:51 +0700
|
| Date: Wed, 9 May 2007 18:55:03 +0000 (UTC)
| From: christos@zoulas.com (Christos Zoulas)
| Message-ID: <20070509185503.7E26263B853@narn.NetBSD.org>
|
| | You can always use parentheses:
|
| They're not immune from problems either. Consider:
|
| test \( = \)
|
| true or false? Either answer is possible (and defensible) and I'm
| pretty sure both have been given by different implementations.
Sure... By this you mean write:
test \( $a = $b \)
instead:
test \( "$a" = "$b" \)
right? I agree though that parentheses can lose if you are not careful.
christos
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Thu, 10 May 2007 04:44:09 +0700
Date: Wed, 9 May 2007 20:25:04 +0000 (UTC)
From: christos@zoulas.com (Christos Zoulas)
Message-ID: <20070509202504.D213963B853@narn.NetBSD.org>
| Sure... By this you mean write:
No, I meant exactly what I wrote, or if you like
a='('
b=')'
test "$a" = "$b"
for a possibly more realistic usage.
I do know how to quote, and usage mistakes wasn't the point.
kre
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, pkg-manager@netbsd.org,
gnats-admin@netbsd.org, pkgsrc-bugs@netbsd.org, kre@munnari.OZ.AU
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Wed, 9 May 2007 17:50:31 -0400
On May 9, 9:45pm, kre@munnari.OZ.AU (Robert Elz) wrote:
-- Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBS
| No, I meant exactly what I wrote, or if you like
|
| a='('
| b=')'
| test "$a" = "$b"
|
| for a possibly more realistic usage.
|
| I do know how to quote, and usage mistakes wasn't the point.
I did not doubt that :-) I was unclear about what case you had in mind.
Your example makes it perfectly clear...
christos
From: "OBATA Akio" <obache@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Fri, 11 May 2007 00:50:00 +0900
On Thu, 10 May 2007 01:10:04 +0900, Robert Elz <kre@munnari.OZ.AU> wrote:
> First, it is never really safe with test to use an unknown string
> variable unprotected in a = or != comparison.
> That is, in a test like
> if test "$pofile" != "-"
> if the value of $pofile just happens to be "-e" or something like
> that, bad things happen (ie: nothing works the way you want, and
> usually the result is a bunch of strange error messages.)
No, see above lines in script. "-e" is never assigned to $pofile.
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Fri, 11 May 2007 01:41:07 +0700
Date: Thu, 10 May 2007 15:55:02 +0000 (UTC)
From: "OBATA Akio" <obache@NetBSD.org>
Message-ID: <20070510155502.DEF4263B853@narn.NetBSD.org>
| No, see above lines in script. "-e" is never assigned to $pofile.
Yes, it looks as if strings starting with '-' cannot (currently)
be assigned to pofile (I would still guard against the possibility
that later changes to the script might alter that - the script does
not currently use "--" to halt option processing, but someone later
might add that, which might then allow "-e" as a file name, after the --).
But aside from that, it is possible for pofile to be '(' or other
stuff like that which can confuse test implementations.
It is so trivial to guard against all of that, that as a rule of thumb,
I'd simply always protect strings passed to test anywhere it is possible
to do it. The cost is close to zero.
kre
ps: please close this PR now - the problem with games/wesnoth is quite
clearly fixed, it builds just fine. What is best to have in the script
is quite a different issue - and changes there are just as likely to have
an impact on some other package as they are on wesnoth. The problem that
affected wesnoth is unlikely to reappear, and if it does, a new PR can
always be opened.
From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: pkg/36257 (games/wesnoth has msgfmt problems (plurals) [NetBSD 3.0])
Date: Thu, 10 May 2007 22:55:09 +0200
On Thu, May 10, 2007 at 08:35:01PM +0000, Robert Elz wrote:
> ps: please close this PR now - the problem with games/wesnoth is quite
> clearly fixed, it builds just fine. What is best to have in the script
> is quite a different issue - and changes there are just as likely to have
> an impact on some other package as they are on wesnoth.
If you are aware of specific typical issues with older/broken test
builtins and/or programs, please drop a mail to rillig@. We have
discussed a long time ago that the current hard-coded feature-list
should be replaced by proper checks in the bootstrap. Having a list of
things to test for is a good start into that direction :-)
Joerg
State-Changed-From-To: feedback->closed
State-Changed-By: obache@netbsd.org
State-Changed-When: Sat, 12 May 2007 05:11:00 +0000
State-Changed-Why:
Build problem has been fixed.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.