NetBSD Problem Report #55836

From mike@fth-devel.net  Tue Dec  1 23:21:25 2020
Return-Path: <mike@fth-devel.net>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 6008B1A9217
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  1 Dec 2020 23:21:25 +0000 (UTC)
Message-Id: <202012012321.0B1NLKUF007748@pumpkin.fth-devel.net>
Date: Wed, 2 Dec 2020 00:21:20 +0100 (CET)
From: Michael Scholz <mike@fth-devel.net>
Reply-To: mike@fth-devel.net
To: gnats-bugs@NetBSD.org
Subject: popen(3)'s type "r+" doesn't work correctly
X-Send-Pr-Version: 3.95

>Number:         55836
>Category:       lib
>Synopsis:       popen(3)'s type "r+" doesn't work correctly
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    lib-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Dec 01 23:25:00 +0000 2020
>Closed-Date:    Sun Jan 21 14:29:28 +0000 2024
>Last-Modified:  Sun Jan 21 14:29:28 +0000 2024
>Originator:     Michael Scholz
>Release:        NetBSD 9.99.76
>Organization:
>Environment:
System: NetBSD pumpkin.fth-devel.net 9.99.76 NetBSD 9.99.76 (PUMPKIN) #0: Mon Nov 30 10:24:54 CET 2020 root@pumpkin.fth-devel.net:/usr/obj/usr/src/sys/arch/amd64/compile/PUMPKIN amd64
Architecture: x86_64
Machine: amd64
>Description:

While I was playing with read/write pipes with popen(command, "r+") I
realized that this doesn't work as expected.  The plus triggers creating
the pipe object with socketpair(2) instead of pipe2(2) in
src/lib/libc/gen/popen.c, but socketpair doesn't seem to work properly.
The test program below, executed with ktrace, shows, after vfork, dup2
and some mmaps, that one read call is made but it never gets something
back and has to be stopped with Ctrl-C or so.

Mike

PS:  This may correspond to PR kern/55690 where problems with C-compiler
option -pipe appeared and the kernel was compiled with options
PIPE_SOCKETPAIR.

>How-To-Repeat:

% cat ./popen_t.c

#include <stdio.h>
#include <stdlib.h>
#include <err.h>

int
main(void)
{
	char		buf[BUFSIZ];
	FILE	       *fp;

	if ((fp = popen("cat", "r+")) == NULL)
		err(EXIT_FAILURE, "popen");

	while (fgets(buf, sizeof(buf), fp) != NULL)
		fputs(buf, stdout);

	if (pclose(fp) == EOF)
		err(EXIT_FAILURE, "pclose");

	return EXIT_SUCCESS;
}

/* end of popen_t.c */

Compile it to -o popen_t and

% ls /bin | ./popen_t

shows only the expected result if the popen type is "r"; set type to
"r+" and no output will appear.

>Fix:

>Release-Note:

>Audit-Trail:
From: Wolfgang Solfrank <Wolfgang@Solfrank.net>
To: gnats-bugs@netbsd.org, lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
 netbsd-bugs@netbsd.org
Cc: 
Subject: Re: lib/55836: popen(3)'s type "r+" doesn't work correctly
Date: Wed, 2 Dec 2020 16:55:44 +0100

 Hi,

 > While I was playing with read/write pipes with popen(command, "r+") I
 > realized that this doesn't work as expected.  The plus triggers creating
 > the pipe object with socketpair(2) instead of pipe2(2) in
 > src/lib/libc/gen/popen.c, but socketpair doesn't seem to work properly.

 This seems to be a problem with your expectations.  The "r+" tells
 popen to open the pipe "for reading and writing" (see popen(3)).
 I.e. with your test program, the forked "cat" gets its stdin *and*
 stdout connected to the one end of the socketpair.  So the cat
 doesn't read the output from the ls command, but sits there waiting
 for your program to write something into fp.

 So this works exactly as designed.

 Ciao,
 Wolfgang
 -- 
 Wolfgang@Solfrank.net				Wolfgang Solfrank

From: Michael Scholz <mike@fth-devel.net>
To: gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/55836: popen(3)'s type "r+" doesn't work correctly
Date: Wed, 2 Dec 2020 23:20:11 +0100 (CET)

 Hi, Wolfgang!

 On Wed, 2 Dec 2020, Wolfgang Solfrank wrote:

 > The following reply was made to PR lib/55836; it has been noted by GNATS.
 >
 > From: Wolfgang Solfrank <Wolfgang@Solfrank.net>
 > To: gnats-bugs@netbsd.org, lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
 > netbsd-bugs@netbsd.org
 > Cc:
 > Subject: Re: lib/55836: popen(3)'s type "r+" doesn't work correctly
 > Date: Wed, 2 Dec 2020 16:55:44 +0100
 >
 > Hi,
 >
 > > While I was playing with read/write pipes with popen(command, "r+") I
 > > realized that this doesn't work as expected.  The plus triggers creating
 > > the pipe object with socketpair(2) instead of pipe2(2) in
 > > src/lib/libc/gen/popen.c, but socketpair doesn't seem to work properly.
 >
 > This seems to be a problem with your expectations.  The "r+" tells
 > popen to open the pipe "for reading and writing" (see popen(3)).
 > I.e. with your test program, the forked "cat" gets its stdin *and*
 > stdout connected to the one end of the socketpair.  So the cat
 > doesn't read the output from the ls command, but sits there waiting
 > for your program to write something into fp.
 >
 > So this works exactly as designed.
 >
 > Ciao,
 > Wolfgang
 > --
 > Wolfgang@Solfrank.net				Wolfgang Solfrank

 Thank you for your input.  According to the man pages "the command's 
 standard input is the same as that of the process that called popen()".

 Okay, if you don't trust this, try that after the pipe is opened:

  	fputs("abcdefg\n", fp);

 fgets(buf, sizeof(buf), fp) returns this string but after that it hangs, 
 no NULL will be returned.

 Mike

From: RVP <rvp@SDF.ORG>
To: Michael Scholz <mike@fth-devel.net>
Cc: gnats-bugs@netbsd.org
Subject: Re: lib/55836: popen(3)'s type "r+" doesn't work correctly
Date: Thu, 3 Dec 2020 06:43:18 +0000 (UTC)

 On Wed, 2 Dec 2020 23:20:11 +0100 (CET), Michael Scholz wrote:

 > According to the man pages "the command's standard input is the same as
 > that of the process that called popen()".

 Only when popen(3) is called with type == "r".

 When popen(3) is called with "r+", as you are doing, then *both* stdin
 and stdout of the command point to the socketpair created by popen(3):

 $ fstat -p $(pgrep -x cat)
 USER     CMD          PID   FD  MOUNT         INUM MODE         SZ|DV R/W
 rvp      cat        24353   wd  /          1222645 drwx------     512 r
 rvp      cat        24353    0* unix stream  0xffff941232bace80 <-> 
 0xffff9412551784c0
 rvp      cat        24353    1* unix stream  0xffff941232bace80 <-> 
 0xffff9412551784c0
 rvp      cat        24353    2  /dev/pts         7 crw--w----   pts/2 rw

 $ fstat -p $(pgrep -x /tmp/foo)
 USER     CMD          PID   FD  MOUNT         INUM MODE         SZ|DV R/W
 rvp      foo        21406   wd  /          1222645 drwx------     512 r
 rvp      foo        21406    0* pipe 0xffff941255818000 <- 0x0 r
 rvp      foo        21406    1  /dev/pts         7 crw--w----   pts/2 rw
 rvp      foo        21406    2  /dev/pts         7 crw--w----   pts/2 rw
 rvp      foo        21406    3* unix stream  0xffff9412551784c0 <-> 
 0xffff941232bace80

 Where, /tmp/foo is your program.

 This is not a bug. Certainly, the popen(3) man-page could elaborate on the
 "r+" case too.

 I think this is the code you are aiming for:

 ---START---
 #include <stdio.h>
 #include <stdlib.h>
 #include <err.h>

 int
 main(void)
 {
          char    buf[BUFSIZ];
          FILE    *fp;

          if ((fp = popen("cat", "r+")) == NULL)
                  err(EXIT_FAILURE, "popen");

          while (fgets(buf, sizeof(buf), stdin) != NULL) {
                  if (fputs(buf, fp) == EOF)
                          err(EXIT_FAILURE, "fputs(cat)");

                  if (fgets(buf, sizeof buf, fp) != NULL)
                          fputs(buf, stdout);
                  else
                          err(EXIT_FAILURE, "fgets(cat)");
          }

          if (pclose(fp) == EOF)
                  err(EXIT_FAILURE, "pclose");

          return EXIT_SUCCESS;
 }
 ---END---


 -RVP

From: Wolfgang Solfrank <Wolfgang@Solfrank.net>
To: Michael Scholz <mike@fth-devel.net>, gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/55836: popen(3)'s type "r+" doesn't work correctly
Date: Thu, 3 Dec 2020 12:48:36 +0100

 Hi,

 > Thank you for your input.  According to the man pages "the command's standard input is the same as that of the process that called popen()".

 As rvp said the man page could stand some improvement.

 > Okay, if you don't trust this, try that after the pipe is opened:
 > 
 >      fputs("abcdefg\n", fp);
 > 
 > fgets(buf, sizeof(buf), fp) returns this string but after that it hangs, no NULL will be returned.

 Of course not.  Since the stdin of cat isn't closed, cat still sits
 there waiting for more input, which it would copy to its stdout.
 And since your programm could still read some data from fp,
 it still waits for more input from that.  In order to get what you
 are trying to achieve, you'd somehow make cat terminate.
 E.g., you could kill the cat (ha!) or to shutdown(2) the output
 side of fp (untested!)

 Ciao,
 Wolfgang
 -- 
 Wolfgang@Solfrank.net				Wolfgang Solfrank

From: Michael Scholz <mike@fth-devel.net>
To: RVP <rvp@SDF.ORG>
Cc: gnats-bugs@netbsd.org
Subject: Re: lib/55836: popen(3)'s type "r+" doesn't work correctly
Date: Thu, 3 Dec 2020 22:55:30 +0100 (CET)

 Hi!

 On Thu, 3 Dec 2020, RVP wrote:

 > On Wed, 2 Dec 2020 23:20:11 +0100 (CET), Michael Scholz wrote:
 >
 >> According to the man pages "the command's standard input is the same as
 >> that of the process that called popen()".
 >
 > Only when popen(3) is called with type == "r".
 >
 > When popen(3) is called with "r+", as you are doing, then *both* stdin
 > and stdout of the command point to the socketpair created by popen(3):
 >
 > $ fstat -p $(pgrep -x cat)
 > USER     CMD          PID   FD  MOUNT         INUM MODE         SZ|DV R/W
 > rvp      cat        24353   wd  /          1222645 drwx------     512 r
 > rvp      cat        24353    0* unix stream  0xffff941232bace80 <-> 
 > 0xffff9412551784c0
 > rvp      cat        24353    1* unix stream  0xffff941232bace80 <-> 
 > 0xffff9412551784c0
 > rvp      cat        24353    2  /dev/pts         7 crw--w----   pts/2 rw
 >
 > $ fstat -p $(pgrep -x /tmp/foo)
 > USER     CMD          PID   FD  MOUNT         INUM MODE         SZ|DV R/W
 > rvp      foo        21406   wd  /          1222645 drwx------     512 r
 > rvp      foo        21406    0* pipe 0xffff941255818000 <- 0x0 r
 > rvp      foo        21406    1  /dev/pts         7 crw--w----   pts/2 rw
 > rvp      foo        21406    2  /dev/pts         7 crw--w----   pts/2 rw
 > rvp      foo        21406    3* unix stream  0xffff9412551784c0 <-> 
 > 0xffff941232bace80
 >
 > Where, /tmp/foo is your program.
 >
 > This is not a bug. Certainly, the popen(3) man-page could elaborate on the
 > "r+" case too.
 >
 > I think this is the code you are aiming for:
 >
 > ---START---
 > #include <stdio.h>
 > #include <stdlib.h>
 > #include <err.h>
 >
 > int
 > main(void)
 > {
 >        char    buf[BUFSIZ];
 >        FILE    *fp;
 >
 >        if ((fp = popen("cat", "r+")) == NULL)
 >                err(EXIT_FAILURE, "popen");
 >
 >        while (fgets(buf, sizeof(buf), stdin) != NULL) {
 >                if (fputs(buf, fp) == EOF)
 >                        err(EXIT_FAILURE, "fputs(cat)");
 >
 >                if (fgets(buf, sizeof buf, fp) != NULL)
 >                        fputs(buf, stdout);
 >                else
 >                        err(EXIT_FAILURE, "fgets(cat)");
 >        }
 >
 >        if (pclose(fp) == EOF)
 >                err(EXIT_FAILURE, "pclose");
 >
 >        return EXIT_SUCCESS;
 > }
 > ---END---
 >
 >
 > -RVP

 Thank you very much.  With your and Wolfgang's clarifications I see now
 how I have to handle this.

 Mike

State-Changed-From-To: open->closed
State-Changed-By: jdolecek@NetBSD.org
State-Changed-When: Sun, 21 Jan 2024 14:29:28 +0000
State-Changed-Why:
The discussion in 2020 seems to have concluded this is not problem
with popen(3) with r+, but rather the program using this.
Thanks for report.


>Unformatted:

Home	PR Database Search