NetBSD Problem Report #58955

From www@netbsd.org  Thu Jan  2 23:56:24 2025
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 7A9F21A9238
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  2 Jan 2025 23:56:24 +0000 (UTC)
Message-Id: <20250102235623.1635B1A923A@mollari.NetBSD.org>
Date: Thu,  2 Jan 2025 23:56:23 +0000 (UTC)
From: mrrosen@alumni.cmu.edu
Reply-To: mrrosen@alumni.cmu.edu
To: gnats-bugs@NetBSD.org
Subject: libedit: rl_event_hook not called when output is piped to another process
X-Send-Pr-Version: www-1.0

>Number:         58955
>Category:       lib
>Synopsis:       libedit: rl_event_hook not called when output is piped to another process
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jan 03 00:00:00 +0000 2025
>Last-Modified:  Thu Jan 09 23:40:00 +0000 2025
>Originator:     Michael Rosen
>Release:        
>Organization:
>Environment:
Linux --- 5.15.153.1-microsoft-standard-WSL2+ #11 SMP Mon Apr 29 14:24:57 PDT 2024 x86_64 x86_64 x86_64 GNU/Linux
>Description:
rl_event_hook should be called periodically if set while waiting on characters during blocking readline calls. From the documentation on GNU readline:

"If non-zero, this is the address of a function to call periodically when Readline is waiting for terminal input. By default, this will be called at most ten times a second if there is no keyboard input."

However, if the output of a program using Libedit's readline implementation is piped into another program (ie ./libedit_readline_program_using_rl_event_hook | grep "some output"), the program will not call the rl_event_hook function (which might include some important logic) and the readline call will just block, possibly forever.

The problem arises from the following condition to the rl_event_hook installation in readline (readline.c:467):

if (rl_event_hook && !(e->el_flags & NO_TTY)) {
    el_set(e, EL_GETCFN, _rl_event_read_char);
    used_event_hook = 1;
}

The check for NO_TTY (which if one dives into where thats set comes from tty.c:508, isatty(el->el_outfd), which is false when the output is piped to another process) is the reason the rl_event_hook will not be installed in this case.

Is there a reason the output needs to be a tty? Removing this condition seems to fix the problem but I would assume there was a reason for it to be there in the first place (no comment nor any information in the commit message for the change that introduced it, revision 1.38 of readline.c).

>How-To-Repeat:
Using this program:

#include <stdio.h>
#include <stdlib.h>

#include <editline/readline.h>

int try_rl_event_hook(void) {
    static short hit = 0;

    if (hit++ == 0) {
        fprintf(stderr, "in try_rl_event_hook\n");
    }

    return 0;
}

int main() {
    rl_event_hook = try_rl_event_hook;

    while (1) {
        char *input = readline("prompt> ");
        if (!input) {
            break;
        }
        printf("got input: %s\n", input);
        free(input);
    }

    return 0;
}

If run by itself will print "in try_rl_event_hook" frequently, if run piped into grep or some other process/file will not.
>Fix:
Remove the condition checking for tty:


if (rl_event_hook) {
    el_set(e, EL_GETCFN, _rl_event_read_char);
    used_event_hook = 1;
}

>Audit-Trail:
From: Christos Zoulas <christos@zoulas.com>
To: gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org,
 gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org
Subject: Re: lib/58955: libedit: rl_event_hook not called when output is piped
 to another process
Date: Thu, 9 Jan 2025 16:25:16 -0500

 --Apple-Mail=_2E891251-9880-4BC2-8546-5DC03EA3B102
 Content-Transfer-Encoding: quoted-printable
 Content-Type: text/plain;
 	charset=us-ascii

 I am not sure. How does readline behave? If it works for non-interactive =
 processes we should change our code.

 Best,

 christos=

 --Apple-Mail=_2E891251-9880-4BC2-8546-5DC03EA3B102
 Content-Transfer-Encoding: 7bit
 Content-Disposition: attachment;
 	filename=signature.asc
 Content-Type: application/pgp-signature;
 	name=signature.asc
 Content-Description: Message signed with OpenPGP

 -----BEGIN PGP SIGNATURE-----
 Comment: GPGTools - http://gpgtools.org

 iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCZ4A+vAAKCRBxESqxbLM7
 OmyVAJ9kyL1JNCO2qjELSRdcwxYkwxX7OgCdEGXop0mfw3JOMvytJv+j9NN1nKg=
 =nTPK
 -----END PGP SIGNATURE-----

 --Apple-Mail=_2E891251-9880-4BC2-8546-5DC03EA3B102--

From: Michael Rosen <mrrosen@alumni.cmu.edu>
To: gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/58955: libedit: rl_event_hook not called when output is piped
 to another process
Date: Thu, 9 Jan 2025 15:19:31 -0800

 --0000000000005d8ece062b4e3676
 Content-Type: text/plain; charset="UTF-8"
 Content-Transfer-Encoding: quoted-printable

 >From my tests, readline works the same for interactive and non-interactive
 processes. Just changing the example code to use "#include
 <readline/readline.h>" and linking against GNU readline, rl_event_hook is
 called periodically in either case.

 It's worth noting that GNU readline does rate limit calling rl_event_hook
 (the documentation says it will not be called more than 10 times a second
 so long as no input is given; so for my example I made the short a char to
 not spam the terminal) and that rl_done being set in the rl_event_hook hook
 will cause readline() to return, but those differences were less
 significant/could be overcome in other ways for my usage (ie, the rate was
 unimportant, just that it's called at all, and rl_done behavior can be
 achieved by setting rl_event_hook to NULL; but just mentioning).

 On Thu, Jan 9, 2025, 1:30=E2=80=AFPM Christos Zoulas via gnats <
 gnats-admin@netbsd.org> wrote:

 > The following reply was made to PR lib/58955; it has been noted by GNATS.
 >
 > From: Christos Zoulas <christos@zoulas.com>
 > To: gnats-bugs@netbsd.org
 > Cc: lib-bug-people@netbsd.org,
 >  gnats-admin@netbsd.org,
 >  netbsd-bugs@netbsd.org
 > Subject: Re: lib/58955: libedit: rl_event_hook not called when output is
 > piped
 >  to another process
 > Date: Thu, 9 Jan 2025 16:25:16 -0500
 >
 >  --Apple-Mail=3D_2E891251-9880-4BC2-8546-5DC03EA3B102
 >  Content-Transfer-Encoding: quoted-printable
 >  Content-Type: text/plain;
 >         charset=3Dus-ascii
 >
 >  I am not sure. How does readline behave? If it works for non-interactive=
  =3D
 >  processes we should change our code.
 >
 >  Best,
 >
 >  christos=3D
 >
 >  --Apple-Mail=3D_2E891251-9880-4BC2-8546-5DC03EA3B102
 >  Content-Transfer-Encoding: 7bit
 >  Content-Disposition: attachment;
 >         filename=3Dsignature.asc
 >  Content-Type: application/pgp-signature;
 >         name=3Dsignature.asc
 >  Content-Description: Message signed with OpenPGP
 >
 >  -----BEGIN PGP SIGNATURE-----
 >  Comment: GPGTools - http://gpgtools.org
 >
 >  iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCZ4A+vAAKCRBxESqxbLM7
 >  OmyVAJ9kyL1JNCO2qjELSRdcwxYkwxX7OgCdEGXop0mfw3JOMvytJv+j9NN1nKg=3D
 >  =3DnTPK
 >  -----END PGP SIGNATURE-----
 >
 >  --Apple-Mail=3D_2E891251-9880-4BC2-8546-5DC03EA3B102--
 >
 >

 --0000000000005d8ece062b4e3676
 Content-Type: text/html; charset="UTF-8"
 Content-Transfer-Encoding: quoted-printable

 <div dir=3D"auto"><div>From my tests, readline works the same for interacti=
 ve and non-interactive processes. Just changing the example code to use &qu=
 ot;#include &lt;readline/readline.h&gt;&quot; and linking against GNU readl=
 ine, rl_event_hook is called periodically in either case.</div><div dir=3D"=
 auto"><br></div><div dir=3D"auto">It&#39;s worth noting that GNU readline d=
 oes rate limit calling rl_event_hook (the documentation says it will not be=
  called more than 10 times a second so long as no input is given; so for my=
  example I made the short a char to not spam the terminal) and that rl_done=
  being set in the rl_event_hook hook will cause readline() to return, but t=
 hose differences were less significant/could be overcome in other ways for =
 my usage (ie, the rate was unimportant, just that it&#39;s called at all, a=
 nd rl_done behavior can be achieved by setting rl_event_hook to NULL; but j=
 ust mentioning).<br><br><div class=3D"gmail_quote gmail_quote_container" di=
 r=3D"auto"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Jan 9, 2025, 1:30=
 =E2=80=AFPM Christos Zoulas via gnats &lt;<a href=3D"mailto:gnats-admin@net=
 bsd.org">gnats-admin@netbsd.org</a>&gt; wrote:<br></div><blockquote class=
 =3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
 ing-left:1ex">The following reply was made to PR lib/58955; it has been not=
 ed by GNATS.<br>
 <br>
 From: Christos Zoulas &lt;<a href=3D"mailto:christos@zoulas.com" target=3D"=
 _blank" rel=3D"noreferrer">christos@zoulas.com</a>&gt;<br>
 To: <a href=3D"mailto:gnats-bugs@netbsd.org" target=3D"_blank" rel=3D"noref=
 errer">gnats-bugs@netbsd.org</a><br>
 Cc: <a href=3D"mailto:lib-bug-people@netbsd.org" target=3D"_blank" rel=3D"n=
 oreferrer">lib-bug-people@netbsd.org</a>,<br>
 =C2=A0<a href=3D"mailto:gnats-admin@netbsd.org" target=3D"_blank" rel=3D"no=
 referrer">gnats-admin@netbsd.org</a>,<br>
 =C2=A0<a href=3D"mailto:netbsd-bugs@netbsd.org" target=3D"_blank" rel=3D"no=
 referrer">netbsd-bugs@netbsd.org</a><br>
 Subject: Re: lib/58955: libedit: rl_event_hook not called when output is pi=
 ped<br>
 =C2=A0to another process<br>
 Date: Thu, 9 Jan 2025 16:25:16 -0500<br>
 <br>
 =C2=A0--Apple-Mail=3D_2E891251-9880-4BC2-8546-5DC03EA3B102<br>
 =C2=A0Content-Transfer-Encoding: quoted-printable<br>
 =C2=A0Content-Type: text/plain;<br>
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 charset=3Dus-ascii<br>
 <br>
 =C2=A0I am not sure. How does readline behave? If it works for non-interact=
 ive =3D<br>
 =C2=A0processes we should change our code.<br>
 <br>
 =C2=A0Best,<br>
 <br>
 =C2=A0christos=3D<br>
 <br>
 =C2=A0--Apple-Mail=3D_2E891251-9880-4BC2-8546-5DC03EA3B102<br>
 =C2=A0Content-Transfer-Encoding: 7bit<br>
 =C2=A0Content-Disposition: attachment;<br>
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 filename=3Dsignature.asc<br>
 =C2=A0Content-Type: application/pgp-signature;<br>
 =C2=A0 =C2=A0 =C2=A0 =C2=A0 name=3Dsignature.asc<br>
 =C2=A0Content-Description: Message signed with OpenPGP<br>
 <br>
 =C2=A0-----BEGIN PGP SIGNATURE-----<br>
 =C2=A0Comment: GPGTools - <a href=3D"http://gpgtools.org" rel=3D"noreferrer=
  noreferrer" target=3D"_blank">http://gpgtools.org</a><br>
 <br>
 =C2=A0iF0EARECAB0WIQS+BJlbqPkO0MDBdsRxESqxbLM7OgUCZ4A+vAAKCRBxESqxbLM7<br>
 =C2=A0OmyVAJ9kyL1JNCO2qjELSRdcwxYkwxX7OgCdEGXop0mfw3JOMvytJv+j9NN1nKg=3D<br=
 >
 =C2=A0=3DnTPK<br>
 =C2=A0-----END PGP SIGNATURE-----<br>
 <br>
 =C2=A0--Apple-Mail=3D_2E891251-9880-4BC2-8546-5DC03EA3B102--<br>
 <br>
 </blockquote></div></div></div>

 --0000000000005d8ece062b4e3676--

From: Robert Elz <kre@munnari.OZ.AU>
To: Christos Zoulas <christos@zoulas.com>
Cc: gnats-bugs@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/58955: libedit: rl_event_hook not called when output is piped to another process
Date: Fri, 10 Jan 2025 06:34:49 +0700

     Date:        Thu, 9 Jan 2025 16:25:16 -0500
     From:        Christos Zoulas <christos@zoulas.com>
     Message-ID:  <6008A2D2-6735-46CE-A54A-E82BD66263FF@zoulas.com>

   | If it works for non-interactive processes we should change our code.

 I haven't checked the code for this (yet anyway) but for sh uses at
 least, the definition of interactive (unless forced) is that stdin
 and stderr are both ttys, nothing about stdout, which the PR says is
 what is being tested.

 In general (in sh anyway) libedit works with output redirected, as
 in
 	sh > /some/file
 and piped output should be no different.   I don't use readline emulation,
 nor any hooks, so no actual comment about the issue.

 kre

Home	PR Database Search