NetBSD Problem Report #18767
Received: (qmail 9961 invoked by uid 605); 22 Oct 2002 16:21:18 -0000
Message-Id: <20021022162116.92ECD11122@narn.netbsd.org>
Date: Tue, 22 Oct 2002 09:21:16 -0700 (PDT)
From: manu@netbsd.org
Sender: gnats-bugs-owner@netbsd.org
Reply-To: manu@netbsd.org
To: gnats-bugs@gnats.netbsd.org
Subject: broken behaviour of pipes in Linux emulated shell scripts
X-Send-Pr-Version: www-1.0
>Number: 18767
>Category: kern
>Synopsis: broken behaviour of pipes in Linux emulated shell scripts
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: jdolecek
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Tue Oct 22 16:22:00 +0000 2002
>Closed-Date: Tue Jan 21 08:31:49 +0000 2003
>Last-Modified: Tue Jan 21 08:31:49 +0000 2003
>Originator: Emmanuel.Dreyfus
>Release: NetBSD-1.6
>Organization:
The NetBSD Project
>Environment:
NetBSD mmX 1.6_STABLE NetBSD 1.6_STABLE (MM) #4: Tue Oct 22 17:56:51 CEST 2002 manu@melancolie:/usr/src/sys/arch/i386/compile/MM i386
>Description:
pipes in shell Linux shell scripts running with COMPAT_LINUX loose
data, and this breaks some install scripts.
How it happens:
bash makes an ioctl(TCGETS) on stdin and adapt its behaviour depending
on the result. If the ioctl succeeds, which is what NetBSD-1.6 does,
then it will read chunk of data with blocksize as the size, and it
will lseek back on stdin to go back to the beginning of the lines.
Of course if stdin is a pipe it will not work, thus producing what we
have on NetBSD-1.6
If the ioctl returns EINVAL, bash will read one char at a time, doing
dozens of read system calls. This is what happens on Linux, and it
works with this.
>How-To-Repeat:
#!/emul/linux/bin/sh
cat /etc/group | while read line ; do echo ">>$line<<"; done
On 1.6/i386 with Linux userland from pkgsrc, I get this
>Fix:
Here is the fix to get the Linux behaviour:
Index: linux_termios.c
===================================================================
RCS file: /cvsroot/syssrc/sys/compat/linux/common/linux_termios.c,v
retrieving revision 1.15
diff -U4 -r1.15 linux_termios.c
--- linux_termios.c 2001/12/19 15:20:16 1.15
+++ linux_termios.c 2002/10/22 16:16:51
@@ -494,8 +494,10 @@
retval[0] = 0;
switch (com) {
case LINUX_TCGETS:
+ if (fp->f_type == DTYPE_PIPE)
+ return EINVAL;
error = (*bsdioctl)(fp, TIOCGETA, (caddr_t)&tmpbts, p);
if (error)
return error;
bsd_termios_to_linux_termios(&tmpbts, &tmplts);
While tracking this problem, I discovered an inconsistency in
Linux fstat emulation: When doing fstat on a pipe, st_mode is 1180
(instead of 1000), and st_blocks, st_dev and st_rdev are set to zero.
Here is a fix to this inconsistency (does not fix anything yet, but
it will make a better emulation)
Index: linux_file64.c
===================================================================
RCS file: /cvsroot/syssrc/sys/compat/linux/common/linux_file64.c,v
retrieving revision 1.18
diff -U4 -r1.18 linux_file64.c
--- linux_file64.c 2002/05/20 06:45:11 1.18
+++ linux_file64.c 2002/10/22 16:16:50
@@ -106,8 +106,14 @@
lsp->lst_ctime = bsp->st_ctime;
#if LINUX_STAT64_HAS_BROKEN_ST_INO
lsp->__lst_ino = (linux_ino_t) bsp->st_ino;
#endif
+ if (lsp->lst_mode & S_IFIFO) {
+ lsp->lst_mode = (S_IFIFO | 0600);
+ lsp->lst_blocks = 0;
+ lsp->lst_dev = 0;
+ lsp->lst_rdev = 0;
+ }
}
/*
* The stat functions below are plain sailing. stat and lstat are handled
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: kern-bug-people->jdolecek
Responsible-Changed-By: jdolecek
Responsible-Changed-When: Tue Oct 22 10:53:36 PDT 2002
Responsible-Changed-Why:
I'll handle this.
From: Jaromir Dolecek <jdolecek@netbsd.org>
To: manu@netbsd.org
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: kern/18767: broken behaviour of pipes in Linux emulated shell scripts
Date: Tue, 22 Oct 2002 20:04:12 +0200 (CEST)
manu@netbsd.org wrote:
> How it happens:
> bash makes an ioctl(TCGETS) on stdin and adapt its behaviour depending
> on the result. If the ioctl succeeds, which is what NetBSD-1.6 does,
AFAICS TIOCGETA (this is what TCGETS gets translted to) fails
for anything but tty. Indeed, your supplied example
returns expected results (every line of /etc/group with >> on start
and << on end) for me on:
> uname -smv
NetBSD NetBSD 1.6G (SARUMAN) #238: Sat Aug 31 08:30:56 CEST 2002 root@:/usr/home/dolecek/soft/netbsd/sys/arch/i386/compile/SARUMAN i386
Both in interactive shell and when I save the example shell script
to file and run it as script.
Can you describe exactly how you run the script with what file
contents to trigger the behaviour?
> While tracking this problem, I discovered an inconsistency in
> Linux fstat emulation: When doing fstat on a pipe, st_mode is 1180
> (instead of 1000), and st_blocks, st_dev and st_rdev are set to zero.
I don't think this is worth fixing; I don't think anything would care.
st_dev & st_rdev should be zero always now too. The mode change should
be done in generic code (if it should be done at all).
Jaromir
--
Jaromir Dolecek <jdolecek@NetBSD.org> http://www.NetBSD.org/
-=- We should be mindful of the potential goal, but as the tantric -=-
-=- Buddhist masters say, ``You may notice during meditation that you -=-
-=- sometimes levitate or glow. Do not let this distract you.'' -=-
From: manu@netbsd.org (Emmanuel Dreyfus)
To: jdolecek@netbsd.org (Jaromir Dolecek)
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: kern/18767: broken behaviour of pipes in Linux emulated shell scripts
Date: Wed, 23 Oct 2002 00:04:06 +0200
> Can you describe exactly how you run the script with what file contents to
> trigger the behaviour?
chmod uog+rx test.sh
./test.sh
I get the broken behaviour for any file. Manuel also told me he was not
able to reproduce it. This is very strange, as the problem is 100%
reproductible on my machines.
> I don't think this is worth fixing; I don't think anything would care.
> st_dev & st_rdev should be zero always now too. The mode change should be
> done in generic code (if it should be done at all).
Well I think it's always worth having a more accurate emulation, some
program could rely on this particular behaviour one day.
--
Emmanuel Dreyfus.
NetBSD, parceque je le vaux bien.
manu@netbsd.org
State-Changed-From-To: open->feedback
State-Changed-By: jdolecek
State-Changed-When: Mon Dec 2 14:20:34 PST 2002
State-Changed-Why:
I am unable to repeat the problem. Can you describe exact
steps to trigger the problem? What kind of tty do you use?
What kind of linux libs do you have installed?
From: manu@netbsd.org (Emmanuel Dreyfus)
To: jdolecek@netbsd.org
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/18767
Date: Wed, 4 Dec 2002 00:29:09 +0100
> I am unable to repeat the problem.=20
Yes, I did not committed the fix yet, because I had trouble to find
people that suffer the same problem. However, I had a stock 1.6 system
where it was impossible to use Linux emulation because of this. It has
been running fine with the proposed patch for months.=20
> Can you describe exact steps to trigger the problem?=20
I just run=20
cat >> test.sh << EOF
#!/emul/linux/bin/sh
cat /etc/group | while read line ; do echo "%>>$line<<"; done
EOF
chmod uog+rx test.sh
./test.sh
This produces:
%>>wheel:*:0:root<<
%>>:<<
%>>:45:<<
> What kind of tty do you use?
ttyp0, through ssh.
> What kind of linux libs do you have installed?
suse libs from pkgsrc.
--=20
Emmanuel Dreyfus.
Si la reponse est NT, c'est peut =EAtre=20
que vous n'avez pas compris la question.
manu@netbsd.org
From: Jaromir Dolecek <jdolecek@netbsd.org>
To: Emmanuel Dreyfus <manu@netbsd.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/18767
Date: Wed, 4 Dec 2002 08:35:35 +0100 (CET)
Point is that the 'fix' is not necessary at all, since the ioctl
fails for DTYPE_PIPE now even without that.
Jaromir
Emmanuel Dreyfus wrote:
[ Charset ISO-8859-1 unsupported, converting... ]
> > I am unable to repeat the problem.
>
> Yes, I did not committed the fix yet, because I had trouble to find
> people that suffer the same problem. However, I had a stock 1.6 system
> where it was impossible to use Linux emulation because of this. It has
> been running fine with the proposed patch for months.
>
> > Can you describe exact steps to trigger the problem?
>
> I just run
> cat >> test.sh << EOF
> #!/emul/linux/bin/sh
> cat /etc/group | while read line ; do echo "%>>$line<<"; done
> EOF
> chmod uog+rx test.sh
> ./test.sh
>
> This produces:
> %>>wheel:*:0:root<<
> %>>:<<
> %>>:45:<<
>
> > What kind of tty do you use?
>
> ttyp0, through ssh.
>
> > What kind of linux libs do you have installed?
>
> suse libs from pkgsrc.
>
>
> --
> Emmanuel Dreyfus.
> Si la reponse est NT, c'est peut _tre
> que vous n'avez pas compris la question.
> manu@netbsd.org
>
--
Jaromir Dolecek <jdolecek@NetBSD.org> http://www.NetBSD.org/
-=- We should be mindful of the potential goal, but as the tantric -=-
-=- Buddhist masters say, ``You may notice during meditation that you -=-
-=- sometimes levitate or glow. Do not let this distract you.'' -=-
From: manu@netbsd.org (Emmanuel Dreyfus)
To: jdolecek@netbsd.org (Jaromir Dolecek)
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/18767
Date: Wed, 4 Dec 2002 09:14:03 +0100
> Point is that the 'fix' is not necessary at all, since the ioctl
> fails for DTYPE_PIPE now even without that.
How old is this behavior? ie: was it like that in 1.6?
--=20
Emmanuel Dreyfus.
Windows 95, Windows 98, Windows 2000, Windows Millenium
D=E9sormais, les OS portent la date de peremption dans leurs noms.
manu@netbsd.org
From: Jaromir Dolecek <jdolecek@netbsd.org>
To: Emmanuel Dreyfus <manu@netbsd.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: kern/18767
Date: Thu, 5 Dec 2002 16:23:34 +0100 (CET)
Emmanuel Dreyfus wrote:
> > Point is that the 'fix' is not necessary at all, since the ioctl
> > fails for DTYPE_PIPE now even without that.
>
> How old is this behavior? ie: was it like that in 1.6?
I think so.
Can you run ktrace -i on the shell script, and send me the
kdump output? That should clearly say if the ioctl call
succeeds or not. For me, it appears to fail according
to ktrace.
Jaromir
--
Jaromir Dolecek <jdolecek@NetBSD.org> http://www.NetBSD.org/
-=- We should be mindful of the potential goal, but as the tantric -=-
-=- Buddhist masters say, ``You may notice during meditation that you -=-
-=- sometimes levitate or glow. Do not let this distract you.'' -=-
From: Jaromir Dolecek <jdolecek@netbsd.org>
To: gnats-bugs@gnats.netbsd.org
Cc:
Subject: pr/18767 CVS commit: syssrc/sys/kern
Date: Thu, 5 Dec 2002 18:30:55 +0200 (EET)
Module Name: syssrc
Committed By: jdolecek
Date: Thu Dec 5 16:30:55 UTC 2002
Modified Files:
syssrc/sys/kern: sys_pipe.c
Log Message:
pipe_stat(): add S_IRUSR and S_IWUSR to mode; this is what Linux does,
and seems like generally sensible (more sensible than not doing so), so done
in generic code rather than compat glue only
Change proposed in PR kern/18767 by Emmanuel Dreyfus.
To generate a diff of this commit:
cvs rdiff -r1.31 -r1.32 syssrc/sys/kern/sys_pipe.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: feedback->closed
State-Changed-By: jdolecek
State-Changed-When: Tue Jan 21 00:31:01 PST 2003
State-Changed-Why:
Cannot repeat on -current. Whatever it was, it appears to be fixed.
Please mail if this problem occurs for you again.
>Unformatted:
>>wheel:*:0:root<<
>>:<<
>>:45:<<
Which is completely broken.
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.