NetBSD Problem Report #44455

From mlelstv@serpens.de  Mon Jan 24 22:19:33 2011
Return-Path: <mlelstv@serpens.de>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 5185163B873
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 24 Jan 2011 22:19:33 +0000 (UTC)
Message-Id: <201101242219.p0OMJ7Hl029505@serpens.de>
Date: Mon, 24 Jan 2011 23:19:08 +0100 (MET)
From: mlelstv@serpens.de
Reply-To: mlelstv@serpens.de
To: gnats-bugs@gnats.NetBSD.org
Subject: vi doesn't handle unprintable characters
X-Send-Pr-Version: 3.95

>Number:         44455
>Category:       bin
>Synopsis:       vi doesn't handle unprintable characters
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jan 24 22:20:01 +0000 2011
>Closed-Date:    Sun May 01 02:57:49 +0000 2011
>Last-Modified:  Sun May 01 03:00:03 +0000 2011
>Originator:     Michael van Elst
>Release:        NetBSD 5.99.44
>Organization:

>Environment:


System: NetBSD dummy 5.99.44 NetBSD 5.99.44 (DUMMY) #0: Mon Jan 24 17:14:20 CET
2011 mlelstv@henery:/home/netbsd-current/obj.amiga/home/netbsd-current/src/sys/a
rch/amiga/compile/DUMMY amiga
Architecture: m68k
Machine: amiga
>Description:
vi should show nonprintable characters as hexcodes, i.e \xe4 for an a-umlaut,
instead it shows \x00. The characters are inserted correctly, just the
display is bad.

I can repeat this only on big-endian machines (amiga, sparc64) but not on
others (amd64), the problem also exists in netbsd-5.

My guess is that for 8bit locales the first byte of a wchar_t is displayed
instead of the low byte.

>How-To-Repeat:
Clear LC_* and LANG environment variables and run vi and input non-ascii
characters. vi shows these as \x00.

>Fix:


>Release-Note:

>Audit-Trail:
From: Takehiko NOZAKI <takehiko.nozaki@gmail.com>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/44455: vi doesn't handle unprintable characters
Date: Sun, 30 Jan 2011 04:30:25 +0900

 hi,

 > vi should show nonprintable characters as hexcodes, i.e \xe4 for an a-umlaut,
 > instead it shows \x00. The characters are inserted correctly, just the
 > display is bad.
 >
 > I can repeat this only on big-endian machines (amiga, sparc64) but not on
 > others (amd64), the problem also exists in netbsd-5.
 >
 > My guess is that for 8bit locales the first byte of a wchar_t is displayed
 > instead of the low byte.
 >

 you are right, dist/nvi/common/keys.c v_key_name() is wrong.

 please try following patch:
 ftp://ftp.netbsd.org/pub/NetBSD/misc/tnozaki/patch-nvi

 this patch contains several fixes i18n related bugs of nvi-1.81:

 1. fix PR/44455, nonprintable character doesn't show hex-visual under
 big endian.
 2. ":set octal" with nonprintable character causes nbwcurses move error.
 3. moving (big)words by wW/eE/bB can't handle non-ascii characters.
 4. toggle uppercase/lowercase by ~ can't handle non-ascii characters.
 5. don't feed CHAR_T(=wchar_t) to is* function directly.
   is* funcs with over UCHAR_MAX value may cause undefined behavior.
   some ctype implementation(such as FreeBSD) return unexpected
 result(same result as isw* func).
 6. using non-ascii digit character with count/line number of ex/vi
 command may causes unexpeced result.


 very truly yours
 -- 
 Takehiko NOZAKI<takehiko.nozaki@gmail.com>

From: Julian Coleman <jdc@coris.org.uk>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/44455: vi doesn't handle unprintable characters
Date: Sun, 30 Jan 2011 13:43:01 +0000

 Hi,

 >  please try following patch:
 >  ftp://ftp.netbsd.org/pub/NetBSD/misc/tnozaki/patch-nvi

 This patch fixes the problem for me on 5.0_STABLE/sparc64.  (I modified the
 patch slightly for the nvi version on the netbsd-5-0 branch).

 Thanks,

 J

 -- 
   My other computer also runs NetBSD    /        Sailing at Newbiggin
         http://www.netbsd.org/        /   http://www.newbigginsailingclub.org/

From: "Takehiko NOZAKI" <tnozaki@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/44455 CVS commit: src/dist/nvi
Date: Mon, 21 Mar 2011 14:53:04 +0000

 Module Name:	src
 Committed By:	tnozaki
 Date:		Mon Mar 21 14:53:04 UTC 2011

 Modified Files:
 	src/dist/nvi/clib: mkstemp.c
 	src/dist/nvi/common: cut.c cut.h extern.h gs.h key.c key.h msg.c
 	    multibyte.h options.c seq.c util.c
 	src/dist/nvi/ex: ex.c ex_abbrev.c ex_argv.c ex_cscope.c ex_display.c
 	    ex_global.c ex_join.c ex_map.c ex_perl.c ex_shell.c ex_subst.c
 	    ex_tag.c ex_tcl.c ex_txt.c ex_usage.c ex_write.c
 	src/dist/nvi/ip: ip_main.c
 	src/dist/nvi/motif_l: m_vi.c
 	src/dist/nvi/regex: regcomp.c regex2.h
 	src/dist/nvi/vi: getc.c v_increment.c v_match.c v_replace.c v_search.c
 	    v_sentence.c v_txt.c v_ulcase.c v_util.c v_word.c vi.c vi.h

 Log Message:
 1. fix PR/44455, nonprintable character doesn't show hex-visual
    under big endian.
 2. ":set octal" with nonprintable character causes nbwcurses move error.
 3. moving (big)words by wW/eE/bB can't handle non-ascii characters.
 4. toggle uppercase/lowercase by ~ can't handle non-ascii characters.
 5. don't feed CHAR_T(=wchar_t) to is* function directly.
    is* funcs with over UCHAR_MAX value may cause undefined behavior.
    some ctype implementation(such as FreeBSD) return unexpected
    result(same result as isw* func).
 6. using non-ascii digit character with count/line number of ex/vi
    command may causes unexpeced result.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1.1.2 -r1.2 src/dist/nvi/clib/mkstemp.c
 cvs rdiff -u -r1.3 -r1.4 src/dist/nvi/common/cut.c src/dist/nvi/common/cut.h \
     src/dist/nvi/common/extern.h src/dist/nvi/common/msg.c \
     src/dist/nvi/common/util.c
 cvs rdiff -u -r1.4 -r1.5 src/dist/nvi/common/gs.h
 cvs rdiff -u -r1.5 -r1.6 src/dist/nvi/common/key.c \
     src/dist/nvi/common/multibyte.h
 cvs rdiff -u -r1.1.1.2 -r1.2 src/dist/nvi/common/key.h
 cvs rdiff -u -r1.9 -r1.10 src/dist/nvi/common/options.c
 cvs rdiff -u -r1.2 -r1.3 src/dist/nvi/common/seq.c
 cvs rdiff -u -r1.6 -r1.7 src/dist/nvi/ex/ex.c src/dist/nvi/ex/ex_cscope.c
 cvs rdiff -u -r1.1.1.2 -r1.2 src/dist/nvi/ex/ex_abbrev.c \
     src/dist/nvi/ex/ex_map.c src/dist/nvi/ex/ex_perl.c \
     src/dist/nvi/ex/ex_tcl.c
 cvs rdiff -u -r1.3 -r1.4 src/dist/nvi/ex/ex_argv.c \
     src/dist/nvi/ex/ex_global.c src/dist/nvi/ex/ex_subst.c \
     src/dist/nvi/ex/ex_write.c
 cvs rdiff -u -r1.2 -r1.3 src/dist/nvi/ex/ex_display.c \
     src/dist/nvi/ex/ex_join.c
 cvs rdiff -u -r1.4 -r1.5 src/dist/nvi/ex/ex_shell.c
 cvs rdiff -u -r1.8 -r1.9 src/dist/nvi/ex/ex_tag.c
 cvs rdiff -u -r1.5 -r1.6 src/dist/nvi/ex/ex_txt.c src/dist/nvi/ex/ex_usage.c
 cvs rdiff -u -r1.1.1.2 -r1.2 src/dist/nvi/ip/ip_main.c
 cvs rdiff -u -r1.1.1.2 -r1.2 src/dist/nvi/motif_l/m_vi.c
 cvs rdiff -u -r1.5 -r1.6 src/dist/nvi/regex/regcomp.c
 cvs rdiff -u -r1.2 -r1.3 src/dist/nvi/regex/regex2.h
 cvs rdiff -u -r1.2 -r1.3 src/dist/nvi/vi/getc.c src/dist/nvi/vi/v_replace.c \
     src/dist/nvi/vi/v_sentence.c src/dist/nvi/vi/v_ulcase.c \
     src/dist/nvi/vi/v_util.c src/dist/nvi/vi/v_word.c
 cvs rdiff -u -r1.4 -r1.5 src/dist/nvi/vi/v_increment.c \
     src/dist/nvi/vi/v_match.c
 cvs rdiff -u -r1.3 -r1.4 src/dist/nvi/vi/v_search.c src/dist/nvi/vi/vi.c \
     src/dist/nvi/vi/vi.h
 cvs rdiff -u -r1.6 -r1.7 src/dist/nvi/vi/v_txt.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 01 May 2011 02:57:49 +0000
State-Changed-Why:
tnozaki@ fixed it for HEAD. It apparently fixed -5 as well, but the patches
look kind of large for pullup. If someone wants to prepare a patchset for -5,
go for it, but otherwise I guess it won't be fixed there.


From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/44455 (vi doesn't handle unprintable characters)
Date: Sun, 1 May 2011 02:58:25 +0000

 On Sun, May 01, 2011 at 02:57:50AM +0000, dholland@NetBSD.org wrote:
  > Synopsis: vi doesn't handle unprintable characters
  > 
  > State-Changed-From-To: open->closed
  > State-Changed-By: dholland@NetBSD.org
  > State-Changed-When: Sun, 01 May 2011 02:57:49 +0000
  > State-Changed-Why:
  > tnozaki@ fixed it for HEAD. It apparently fixed -5 as well, but the patches
                                              ^^^^^
  > look kind of large for pullup. If someone wants to prepare a patchset for
  > -5, go for it, but otherwise I guess it won't be fixed there.

 er, that should be AFFECTS... gah

 -- 
 David A. Holland
 dholland@netbsd.org

>Unformatted:

Home	PR Database Search