NetBSD Problem Report #42209

From www@NetBSD.org  Wed Oct 21 15:23:54 2009
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id D9A9C63B902
	for <gnats-bugs@gnats.netbsd.org>; Wed, 21 Oct 2009 15:23:53 +0000 (UTC)
Message-Id: <20091021152353.7601963B8B6@www.NetBSD.org>
Date: Wed, 21 Oct 2009 15:23:53 +0000 (UTC)
From: M.Drochner@fz-juelich.de
Reply-To: M.Drochner@fz-juelich.de
To: gnats-bugs@NetBSD.org
Subject: various problems with EBCDIC support in iconv(3)
X-Send-Pr-Version: www-1.0

>Number:         42209
>Category:       lib
>Synopsis:       various problems with EBCDIC support in iconv(3)
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    tnozaki
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 21 15:25:00 +0000 2009
>Closed-Date:    Thu Dec 31 06:49:16 +0000 2009
>Last-Modified:  Thu Dec 31 06:49:16 +0000 2009
>Originator:     Matthias Drochner
>Release:        current
>Organization:
>Environment:
i386/-current
>Description:
The selftests of pkgsrc/libxml2 fail for a couple of reasons:
1. EBCDIC-US is not recognized as charset name
2. newline characters are not translated for most EBCDIC codepages
   (other control characters are appearently affected as well)
3. There is no support for CP1141 (which is CP273 + EUR character)
   (This should be called IBM01141 according to
    http://www.iana.org/assignments/character-sets)

>How-To-Repeat:
for problem 2:
$ printf "\x25"|iconv -f EBCDIC-UK -t ascii|hexdump -C
iconv: iconv(): Illegal byte sequence
$ printf "\n"|iconv -t EBCDIC-UK -f ascii|hexdump -C
00000000  0a                                                |.|
00000001

>Fix:
This seems to fix CP273:
--- share/i18n/csmapper/CP/CP273%UCS.src        b7faa80dfb97da879459d1c88098cc897db4c9c4
+++ share/i18n/csmapper/CP/CP273%UCS.src        84af089511d0acff6677765723d0adb042e6147e
@@ -43,6 +43,7 @@ 0x1F = 0x001F
 0x1D = 0x001D
 0x1E = 0x001E
 0x1F = 0x001F
+0x25 = 0x000A
 0x40 = 0x0020
 0x41 = 0x00A0
 0x42 = 0x00E2
============================================================
--- share/i18n/csmapper/CP/UCS%CP273.src        1569837a5ac966a9760459aa83e4226ac5d4a781
+++ share/i18n/csmapper/CP/UCS%CP273.src        9ef05f6f253612340959f2d1959e53354cec8150
@@ -21,7 +21,7 @@ 0x0009 = 0x09
 0x0007 = 0x07
 0x0008 = 0x08
 0x0009 = 0x09
-0x000A = 0x0A
+0x000A = 0x25
 0x000B = 0x0B
 0x000C = 0x0C
 0x000D = 0x0D

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: lib-bug-people->tnozaki
Responsible-Changed-By: tnozaki@NetBSD.org
Responsible-Changed-When: Thu, 29 Oct 2009 18:51:22 +0000
Responsible-Changed-Why:
i'll fix this.

From: Takehiko NOZAKI <tnozaki@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42209 CVS commit: src
Date: Sat, 14 Nov 2009 01:56:23 +0000

 Module Name:	src
 Committed By:	tnozaki
 Date:		Sat Nov 14 01:56:23 UTC 2009

 Modified Files:
 	src/distrib/sets/lists/base: mi
 	src/share/i18n/csmapper/CP: CP.part CP037%UCS.src CP038%UCS.src
 	    CP273%UCS.src CP274%UCS.src CP275%UCS.src CP277%UCS.src
 	    CP278%UCS.src CP280%UCS.src CP281%UCS.src CP284%UCS.src
 	    CP285%UCS.src CP290%UCS.src CP297%UCS.src CP420%UCS.src
 	    CP423%UCS.src CP500%UCS.src CP851%UCS.src CP868%UCS.src
 	    CP870%UCS.src CP871%UCS.src CP880%UCS.src CP891%UCS.src
 	    CP903%UCS.src CP904%UCS.src CP905%UCS.src CP918%UCS.src
 	    UCS%CP037.src UCS%CP038.src UCS%CP273.src UCS%CP274.src
 	    UCS%CP275.src UCS%CP277.src UCS%CP278.src UCS%CP280.src
 	    UCS%CP281.src UCS%CP284.src UCS%CP285.src UCS%CP290.src
 	    UCS%CP297.src UCS%CP420.src UCS%CP423.src UCS%CP500.src
 	    UCS%CP851.src UCS%CP868.src UCS%CP870.src UCS%CP871.src
 	    UCS%CP880.src UCS%CP891.src UCS%CP903.src UCS%CP904.src
 	    UCS%CP905.src UCS%CP918.src
 	src/share/i18n/esdb/CP: CP.alias CP.part
 Added Files:
 	src/share/i18n/csmapper/CP: CP1140%UCS.src CP1141%UCS.src
 	    CP1142%UCS.src CP1143%UCS.src CP1144%UCS.src CP1145%UCS.src
 	    CP1146%UCS.src CP1147%UCS.src CP1148%UCS.src CP1149%UCS.src
 	    UCS%CP1140.src UCS%CP1141.src UCS%CP1142.src UCS%CP1143.src
 	    UCS%CP1144.src UCS%CP1145.src UCS%CP1146.src UCS%CP1147.src
 	    UCS%CP1148.src UCS%CP1149.src

 Log Message:
 1. PR/42209 various problems with EBCDIC support in iconv(3)
 part3: added CP114[0-9] support.
 2. fix csmapper name separator.


 To generate a diff of this commit:
 cvs rdiff -u -r1.841 -r1.842 src/distrib/sets/lists/base/mi
 cvs rdiff -u -r1.12 -r1.13 src/share/i18n/csmapper/CP/CP.part
 cvs rdiff -u -r1.3 -r1.4 src/share/i18n/csmapper/CP/CP037%UCS.src \
     src/share/i18n/csmapper/CP/CP038%UCS.src \
     src/share/i18n/csmapper/CP/CP273%UCS.src \
     src/share/i18n/csmapper/CP/CP274%UCS.src \
     src/share/i18n/csmapper/CP/CP275%UCS.src \
     src/share/i18n/csmapper/CP/CP277%UCS.src \
     src/share/i18n/csmapper/CP/CP278%UCS.src \
     src/share/i18n/csmapper/CP/CP280%UCS.src \
     src/share/i18n/csmapper/CP/CP281%UCS.src \
     src/share/i18n/csmapper/CP/CP284%UCS.src \
     src/share/i18n/csmapper/CP/CP285%UCS.src \
     src/share/i18n/csmapper/CP/CP290%UCS.src \
     src/share/i18n/csmapper/CP/CP297%UCS.src \
     src/share/i18n/csmapper/CP/CP420%UCS.src \
     src/share/i18n/csmapper/CP/CP423%UCS.src \
     src/share/i18n/csmapper/CP/CP500%UCS.src \
     src/share/i18n/csmapper/CP/CP870%UCS.src \
     src/share/i18n/csmapper/CP/CP871%UCS.src \
     src/share/i18n/csmapper/CP/CP880%UCS.src \
     src/share/i18n/csmapper/CP/CP905%UCS.src \
     src/share/i18n/csmapper/CP/CP918%UCS.src \
     src/share/i18n/csmapper/CP/UCS%CP891.src \
     src/share/i18n/csmapper/CP/UCS%CP903.src \
     src/share/i18n/csmapper/CP/UCS%CP904.src
 cvs rdiff -u -r0 -r1.1 src/share/i18n/csmapper/CP/CP1140%UCS.src \
     src/share/i18n/csmapper/CP/CP1141%UCS.src \
     src/share/i18n/csmapper/CP/CP1142%UCS.src \
     src/share/i18n/csmapper/CP/CP1143%UCS.src \
     src/share/i18n/csmapper/CP/CP1144%UCS.src \
     src/share/i18n/csmapper/CP/CP1145%UCS.src \
     src/share/i18n/csmapper/CP/CP1146%UCS.src \
     src/share/i18n/csmapper/CP/CP1147%UCS.src \
     src/share/i18n/csmapper/CP/CP1148%UCS.src \
     src/share/i18n/csmapper/CP/CP1149%UCS.src \
     src/share/i18n/csmapper/CP/UCS%CP1140.src \
     src/share/i18n/csmapper/CP/UCS%CP1141.src \
     src/share/i18n/csmapper/CP/UCS%CP1142.src \
     src/share/i18n/csmapper/CP/UCS%CP1143.src \
     src/share/i18n/csmapper/CP/UCS%CP1144.src \
     src/share/i18n/csmapper/CP/UCS%CP1145.src \
     src/share/i18n/csmapper/CP/UCS%CP1146.src \
     src/share/i18n/csmapper/CP/UCS%CP1147.src \
     src/share/i18n/csmapper/CP/UCS%CP1148.src \
     src/share/i18n/csmapper/CP/UCS%CP1149.src
 cvs rdiff -u -r1.2 -r1.3 src/share/i18n/csmapper/CP/CP851%UCS.src \
     src/share/i18n/csmapper/CP/CP868%UCS.src \
     src/share/i18n/csmapper/CP/CP891%UCS.src \
     src/share/i18n/csmapper/CP/CP903%UCS.src \
     src/share/i18n/csmapper/CP/CP904%UCS.src
 cvs rdiff -u -r1.5 -r1.6 src/share/i18n/csmapper/CP/UCS%CP037.src \
     src/share/i18n/csmapper/CP/UCS%CP273.src \
     src/share/i18n/csmapper/CP/UCS%CP275.src \
     src/share/i18n/csmapper/CP/UCS%CP277.src \
     src/share/i18n/csmapper/CP/UCS%CP278.src \
     src/share/i18n/csmapper/CP/UCS%CP280.src \
     src/share/i18n/csmapper/CP/UCS%CP281.src \
     src/share/i18n/csmapper/CP/UCS%CP284.src \
     src/share/i18n/csmapper/CP/UCS%CP285.src \
     src/share/i18n/csmapper/CP/UCS%CP290.src \
     src/share/i18n/csmapper/CP/UCS%CP297.src \
     src/share/i18n/csmapper/CP/UCS%CP420.src \
     src/share/i18n/csmapper/CP/UCS%CP423.src \
     src/share/i18n/csmapper/CP/UCS%CP500.src \
     src/share/i18n/csmapper/CP/UCS%CP851.src \
     src/share/i18n/csmapper/CP/UCS%CP868.src \
     src/share/i18n/csmapper/CP/UCS%CP870.src \
     src/share/i18n/csmapper/CP/UCS%CP871.src \
     src/share/i18n/csmapper/CP/UCS%CP880.src \
     src/share/i18n/csmapper/CP/UCS%CP905.src \
     src/share/i18n/csmapper/CP/UCS%CP918.src
 cvs rdiff -u -r1.4 -r1.5 src/share/i18n/csmapper/CP/UCS%CP038.src \
     src/share/i18n/csmapper/CP/UCS%CP274.src
 cvs rdiff -u -r1.13 -r1.14 src/share/i18n/esdb/CP/CP.alias
 cvs rdiff -u -r1.10 -r1.11 src/share/i18n/esdb/CP/CP.part

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: tnozaki@NetBSD.org
State-Changed-When: Sat, 14 Nov 2009 02:09:08 +0000
State-Changed-Why:
patch submitted HEAD, thanks for your PR!

From: Matthias Drochner <drochner@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42209 CVS commit: pkgsrc/textproc/libxml2
Date: Sun, 15 Nov 2009 12:48:25 +0000

 Module Name:	pkgsrc
 Committed By:	drochner
 Date:		Sun Nov 15 12:48:25 UTC 2009

 Modified Files:
 	pkgsrc/textproc/libxml2: distinfo
 	pkgsrc/textproc/libxml2/patches: patch-ag

 Log Message:
 remove most of the hacks which were only to make the selftest
 succeed, now that EBCDIC support in NetBSD-current was improved
 (see PR lib/42209)


 To generate a diff of this commit:
 cvs rdiff -u -r1.74 -r1.75 pkgsrc/textproc/libxml2/distinfo
 cvs rdiff -u -r1.7 -r1.8 pkgsrc/textproc/libxml2/patches/patch-ag

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: feedback->closed
State-Changed-By: tnozaki@NetBSD.org
State-Changed-When: Thu, 31 Dec 2009 06:49:16 +0000
State-Changed-Why:
i think this PR has been closed, still problems?

>Unformatted:

Home	PR Database Search