NetBSD Problem Report #44021

From www@NetBSD.org  Sun Oct 31 17:54:23 2010
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 3F10463B942
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 31 Oct 2010 17:54:23 +0000 (UTC)
Message-Id: <20101031175423.03B9063B873@www.NetBSD.org>
Date: Sun, 31 Oct 2010 17:54:22 +0000 (UTC)
From: naruse@airemix.jp
Reply-To: naruse@airemix.jp
To: gnats-bugs@NetBSD.org
Subject: setlocale brekas the history of libedit
X-Send-Pr-Version: www-1.0

>Number:         44021
>Category:       lib
>Synopsis:       setlocale brekas the history of libedit
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    lib-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Oct 31 17:55:00 +0000 2010
>Closed-Date:    Thu Nov 04 16:38:41 +0000 2010
>Last-Modified:  Thu Nov 04 16:38:41 +0000 2010
>Originator:     Yui NARUSE
>Release:        NetBSD 5.99.39
>Organization:
>Environment:
NetBSD kelvena.airemix.net 5.99.39 NetBSD 5.99.39 (GENERIC) #0: Sun Oct 31 00:39:11 JST 2010  naruse@kelvena.airemix.net:/usr/obj/sys/arch/amd64/compile/GENERIC amd64

>Description:
Calling setlocale(LC_CTYPE, "") and use readline and addhistory of libedit,
its history shows broken strings.
>How-To-Repeat:
Reproducible program is:
% cat r.c
#include <stdio.h>
#include <locale.h>
#include <readline/readline.h>
#include <readline/history.h>

int
main(void)
{
    setlocale(LC_CTYPE, "");
    for (;;) {
        char *r = readline("> ");
        if (!r) {
            fprintf(stderr, "\n");
            return 0;
        }
        fprintf(stderr, "%s\n", r);
        add_history(r);
        free(r);
    }
    return 0;
}

Compile it:
% cc -ledit r.c

Run it:
% ./a.out
> 

It shows prompt, so type some string:
% ./a.out
> aaa

Type Enter
% ./a.out
> aaa
aaa
> 

Now push up arrow key and get a strange string:
% ./a.out
> aaa
aaa
> \U+16161
>Fix:

>Release-Note:

>Audit-Trail:
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: lib/44021: setlocale brekas the history of libedit
Date: Mon, 1 Nov 2010 18:09:58 -0400

 On Oct 31,  5:55pm, naruse@airemix.jp (naruse@airemix.jp) wrote:
 -- Subject: lib/44021: setlocale brekas the history of libedit

 | Compile it:
 | % cc -ledit r.c
 | 
 | Run it:
 | % ./a.out
 | > 
 | 
 | It shows prompt, so type some string:
 | % ./a.out
 | > aaa
 | 
 | Type Enter
 | % ./a.out
 | > aaa
 | aaa
 | > 
 | 
 | Now push up arrow key and get a strange string:
 | % ./a.out
 | > aaa
 | aaa
 | > \U+16161

 Can't reproduce it; granted I can't use up arrow, so I did ^P.
 Can you send us:
 $ ident /usr/lib/libedit.so

 christos

From: "NARUSE, Yui" <naruse@airemix.jp>
To: gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/44021: setlocale brekas the history of libedit
Date: Tue, 2 Nov 2010 09:23:19 +0900

 2010/11/2 Christos Zoulas <christos@zoulas.com>:
 > The following reply was made to PR lib/44021; it has been noted by GNATS.
 >
 > From: christos@zoulas.com (Christos Zoulas)
 > To: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org,
 > =A0 =A0 =A0 =A0gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
 > Cc:
 > Subject: Re: lib/44021: setlocale brekas the history of libedit
 > Date: Mon, 1 Nov 2010 18:09:58 -0400
 >
 > =A0On Oct 31, =A05:55pm, naruse@airemix.jp (naruse@airemix.jp) wrote:
 > =A0-- Subject: lib/44021: setlocale brekas the history of libedit
 >
 > =A0| Compile it:
 > =A0| % cc -ledit r.c
 > =A0|
 > =A0| Run it:
 > =A0| % ./a.out
 > =A0| >
 > =A0|
 > =A0| It shows prompt, so type some string:
 > =A0| % ./a.out
 > =A0| > aaa
 > =A0|
 > =A0| Type Enter
 > =A0| % ./a.out
 > =A0| > aaa
 > =A0| aaa
 > =A0| >
 > =A0|
 > =A0| Now push up arrow key and get a strange string:
 > =A0| % ./a.out
 > =A0| > aaa
 > =A0| aaa
 > =A0| > \U+16161
 >
 > =A0Can't reproduce it; granted I can't use up arrow, so I did ^P.

 ^P also reproduced this on my environment.

 > =A0Can you send us:
 > =A0$ ident /usr/lib/libedit.so

 % ident /usr/lib/libedit.so
 /usr/lib/libedit.so:
      $NetBSD: crti.S,v 1.1 2010/08/07 18:01:35 joerg Exp $
      $NetBSD: crtbegin.S,v 1.1 2010/08/07 18:01:34 joerg Exp $
      $NetBSD: history.c,v 1.37 2010/01/03 18:27:10 christos Exp $
      $NetBSD: readline.c,v 1.92 2010/09/16 20:08:51 christos Exp $
      $NetBSD: history.c,v 1.37 2010/01/03 18:27:10 christos Exp $
      $NetBSD: tokenizer.c,v 1.18 2010/01/03 18:27:10 christos Exp $
      $NetBSD: tokenizer.c,v 1.18 2010/01/03 18:27:10 christos Exp $
      $NetBSD: crtend.S,v 1.1 2010/08/07 18:01:34 joerg Exp $
      $NetBSD: crtn.S,v 1.1 2010/08/07 18:01:35 joerg Exp $

 --=20
 NARUSE, Yui
 naruse@airemix.jp

From: enami tsugutomo <tsugutomo.enami@jp.sony.com>
To: Christos Zoulas <christos@zoulas.com>
Cc: <gnats-bugs@NetBSD.org>, <lib-bug-people@NetBSD.org>,
        <gnats-admin@NetBSD.org>, <netbsd-bugs@NetBSD.org>
Subject: Re: lib/44021: setlocale brekas the history of libedit
Date: Tue,  2 Nov 2010 12:19:04 +0900 (JST)

 > | Now push up arrow key and get a strange string:
 > | % ./a.out
 > | > aaa
 > | aaa
 > | > \U+16161
 > 
 > Can't reproduce it; granted I can't use up arrow, so I did ^P.

 You can reproduce it by setting LANG to ja_JP.UTF-8 or en_US.UTF-8
 etc.  And the bug is probably that the add_history() passes `char *'
 while history(..., H_ENTER, ..) expects `Char *'.

 enami.

From: christos@zoulas.com (Christos Zoulas)
To: "NARUSE, Yui" <naruse@airemix.jp>, gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, 
	netbsd-bugs@netbsd.org
Subject: Re: lib/44021: setlocale brekas the history of libedit
Date: Tue, 2 Nov 2010 08:45:15 -0400

 On Nov 2,  9:23am, naruse@airemix.jp ("NARUSE, Yui") wrote:
 -- Subject: Re: lib/44021: setlocale brekas the history of libedit

 | ^P also reproduced this on my environment.
 | 
 | > =A0Can you send us:
 | > =A0$ ident /usr/lib/libedit.so
 | 
 | % ident /usr/lib/libedit.so
 | /usr/lib/libedit.so:
 |      $NetBSD: crti.S,v 1.1 2010/08/07 18:01:35 joerg Exp $
 |      $NetBSD: crtbegin.S,v 1.1 2010/08/07 18:01:34 joerg Exp $
 |      $NetBSD: history.c,v 1.37 2010/01/03 18:27:10 christos Exp $
 |      $NetBSD: readline.c,v 1.92 2010/09/16 20:08:51 christos Exp $
 |      $NetBSD: history.c,v 1.37 2010/01/03 18:27:10 christos Exp $
 |      $NetBSD: tokenizer.c,v 1.18 2010/01/03 18:27:10 christos Exp $
 |      $NetBSD: tokenizer.c,v 1.18 2010/01/03 18:27:10 christos Exp $
 |      $NetBSD: crtend.S,v 1.1 2010/08/07 18:01:34 joerg Exp $
 |      $NetBSD: crtn.S,v 1.1 2010/08/07 18:01:35 joerg Exp $

 I have the same.

 christos

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/44021: setlocale brekas the history of libedit
Date: Thu, 4 Nov 2010 06:27:51 +0000

 On Mon, Nov 01, 2010 at 10:15:04PM +0000, Christos Zoulas wrote:
  >  Can't reproduce it; granted I can't use up arrow, so I did ^P.
  >  Can you send us:
  >  $ ident /usr/lib/libedit.so

 This appears to be the same problem as PR 43998, so try

    % env LC_CTYPE=en_US.UTF-8 /usr/bin/ftp
    ftp> splat
    ?Invalid command.
    ftp> [up]

 I get: \U+C7073[a-grave][(C)]\U+A800[c-cedilla][1/2][inverted-?]

 -- 
 David A. Holland
 dholland@netbsd.org

From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/44021 CVS commit: src/lib/libedit
Date: Thu, 4 Nov 2010 09:53:12 -0400

 Module Name:	src
 Committed By:	christos
 Date:		Thu Nov  4 13:53:12 UTC 2010

 Modified Files:
 	src/lib/libedit: eln.c

 Log Message:
 PR/43998, PR/44021: In narrow history emulation, don't treat UTF-8 character
 sets specially as far as history goes since we always need to do the conversion
 from narrow [history] to wide [editline].


 To generate a diff of this commit:
 cvs rdiff -u -r1.8 -r1.9 src/lib/libedit/eln.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "NARUSE, Yui" <naruse@airemix.jp>
To: gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: PR/44021 CVS commit: src/lib/libedit
Date: Fri, 5 Nov 2010 01:04:50 +0900

 2010/11/4 Christos Zoulas <christos@netbsd.org>:
 > The following reply was made to PR lib/44021; it has been noted by GNATS.
 >
 > From: "Christos Zoulas" <christos@netbsd.org>
 > To: gnats-bugs@gnats.NetBSD.org
 > Cc:
 > Subject: PR/44021 CVS commit: src/lib/libedit
 > Date: Thu, 4 Nov 2010 09:53:12 -0400
 >
 > =A0Module Name: =A0 src
 > =A0Committed By: =A0christos
 > =A0Date: =A0 =A0 =A0 =A0 =A0Thu Nov =A04 13:53:12 UTC 2010
 >
 > =A0Modified Files:
 > =A0 =A0 =A0 =A0src/lib/libedit: eln.c
 >
 > =A0Log Message:
 > =A0PR/43998, PR/44021: In narrow history emulation, don't treat UTF-8 cha=
 racter
 > =A0sets specially as far as history goes since we always need to do the c=
 onversion
 > =A0from narrow [history] to wide [editline].

 I confirmed this fixes the proglem.
 Thank you!

 --=20
 NARUSE, Yui
 naruse@airemix.jp

State-Changed-From-To: open->closed
State-Changed-By: christos@NetBSD.org
State-Changed-When: Thu, 04 Nov 2010 12:38:41 -0400
State-Changed-Why:
submitter verified it is fixed.


>Unformatted:

Home	PR Database Search