NetBSD Problem Report #33693
From woods@building.weird.com Sat Jun 10 20:15:36 2006
Return-Path: <woods@building.weird.com>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by narn.NetBSD.org (Postfix) with ESMTP id 4012363BA5F
for <gnats-bugs@gnats.netbsd.org>; Sat, 10 Jun 2006 20:15:36 +0000 (UTC)
Message-Id: <m1Fp8mO-002IhbC@building.weird.com>
Date: Sat, 10 Jun 2006 15:05:52 -0400 (EDT)
From: "Greg A. Woods" <woods@planix.com>
Sender: "Greg A. Woods" <woods@building.weird.com>
Reply-To: "Greg A. Woods" <woods@planix.com>
To: gnats-bugs@NetBSD.org
Subject: pdksh in 2.0 and newer violates de facto standard Ksh startup behaviour
X-Send-Pr-Version: 3.95
>Number: 33693
>Category: bin
>Synopsis: pdksh in 2.0 and newer violates de facto standard Ksh startup behaviour
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: bin-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Jun 10 20:20:00 +0000 2006
>Last-Modified: Tue Jul 17 04:43:23 +0000 2007
>Originator: Greg A. Woods
>Release: NetBSD 2.0 and all subsequent releases
>Organization:
Planix, Inc.; Toronto, Ontario; Canada
>Environment:
System: NetBSD
Architecture: all
Machine: all
>Description:
Pdksh on all recent releases of NetBSD violate de facto standard
Ksh compatibility with respect to how $ENV works. Every other
ksh and ksh-compatible shell, including pdksh on 1.6 and
earlier, does not, by default, do anything special if $ENV is
not set.
The 1989 KSH book, as well as Usenet folklore going back as much
as 14 years, says that Ksh will only source ~/.profile if it's
an interactive shell and that it'll only source the file pointed
to by $ENV IFF that variable is set (and the file exits).
This was the behaviour of /bin/ksh on NetBSD since /bin/ksh was
added. The current behaviour (since 2.0) breaks rshd, sshd, and
perhaps all other remote execution facilities when the user's
shell is /bin/ksh.
Note also that the NetBSD ksh(1) manual does not say that
/bin/ksh was built with some magic value for DEFAULT_ENV leaving
this aberrant behaviour completely undocumented too (though it
does hint that such a feature is possible in non-standard
compiles).
Note that the "configure" script used for configuring and
building pdksh on systems other than NetBSD says:
NOTE: This is a non-standard feature
(ie, at&t ksh has no default environment)."
As an example of the issue I've noticed an apparent problem with
sftp and scp to _some_ NetBSD systems for a couple of years now
but never was able to debug the issue since the error message
seemed to be related only to OpenSSH on NetBSD and it seemed to
be deep in the internal protocol. My work-around in the past
was to install SSH-2 on the target systems and forget about it
(or to do the transfer in the opposite direction). Not sure why
that works yet, but it does (sshd2 is probably smart enough not
to use the user's $SHELL for sftp).
However with recent installs of 3.0_STABLE in a customer
environment where "rsh" is still used extensively (only on a
physically separate and private administrative network, of
course), similar problems suddenly appeared with all "rsh" jobs
being executed on the upgraded and new machines.
Most importantly note that it is difficult (to the point of
being practically impossible) with rshd (and perhaps with sshd
as well) to pre-set $ENV to a null value in order to counter-act
this aberrant behaviour.
Note also that those who want $ENV to be set and used for their
interactive sessions can trivially follow the nearly decades old
advice (replicated below) to achieve this goal. It might look
like magic, but it's been used successfully by millions for many
years now.
Finally I'll note that I can't even find any discussion of this
rather critical change anywhere, at least not with Google.
>How-To-Repeat:
On a target system that's 2.0 or newer set up a normal ~/.kshrc
using the de facto standard Ksh trick for setting the ENV
variable only in interactive shells (put the following in
~/.profile, and then create a ~/.kshrc file which includes
commands to be run for _interactive_ sessions, such as showing
$HOME with "cd"):
export ENVFILE=$HOME/.kshrc
if [ -n "$ENVFILE" -a -r "$ENVFILE" ] ; then
#
# This magic expression prevents a non-interactive Ksh
# instance from even trying to open and read any file.
#
# The original Korn & Bolsky KSH book [1989] suggests....
#
#export ENV='${ENVFILE[(_$-=0)+(_=1)-_${-%%*i*}]}'
#
# However the expression below is better as it handles the
# empty $- case, which the one above fails on (0+1-1 = 0).
#
# According to a post by Jon H. LaBadie to comp.unix.questions
# on 1989/12/06 (with the Message-ID <826@jonlab.UUCP>) the
# one below is what David Korn actually suggested to him. I'm
# reasonably certain I read a post by Korn suggesting this
# expression as well, but I may simply have read Jon's post.
#
# It does work with all the versions of Ksh I've run across,
# including ksh-85 and the one AIX-3.2, as well as pdksh.
#
export ENV='${ENVFILE[(_$-=1)+(_=0)-(_$-!=_${-%%*i*})]}'
# The following explanation is derived from a Usenet post by
# David Young to comp.sys.apollo (of all places) 1992/10/21:
#
# The idea behind this scheme is to set up an array (ENVFILE)
# whose first element is the file we want executed at startup,
# and whose second element is empty. Note that any string
# variable can be accessed as an array with one element in the
# first (zero) position and any attempt to access other
# elements will result in an empty value.
#
# $ export ENVFILE='~/.envfile'
# $ echo $ENVFILE[0]
# ~/.envfile
# $ echo $ENVFILE[1]
#
# $
#
# So, we want to expand $ENV, when it is referenced by the
# shell during startup, to ENVFILE[0] for interactive shells
# and to ENVFILE[1] for non-interactive shells. To do this we
# need an expression that will evaluate to "0" for an
# interactive shell but which will evaluate to "1" for a
# non-interactive shell.
#
# Keep in mind the two environment variables ENV and PS1 are
# special in Ksh insofar as they get evaluated when they are
# used! So if we assign the value of ENV using single quotes
# as a fixed string it will only be evaluated when it is used
# by the shell.
#
# Also remember that arithmetic expressions are evaluated
# inside array index references (i.e. inside the square
# brackets), but only after '$' variable refs are expanded.
#
# The flags variable ($-) is the key to forming our magic
# expression. If the shell is interactive, the flags will
# contain an 'i'. The expression used as the index consists
# of three parts that are combined using normal arithmetic
# operators to form the final index value:
#
# (_$-=1) + (_=0) - (_$- != _${-%%*i*})
#
# (_$-=1) This creates a variable named "_BLAH", where
# "BLAH" is the value of the variable "$-", and
# assigns it the value "1". The expression has
# the result "1" as well.
#
# (_=0) This creates a variable named "_" and assigns
# it the value "0". The expression has the result
# "0" as well.
#
# (_$- != _${-%%*i*}) This compares the value of _BLAH with
# the value of another variable that
# will either have the same name (and
# thus the same value), or the name "_".
#
# For an interactive shell these variables will be guaranteed
# to have different names, i.e. the first variable will have
# the name of the variable created in the first expression
# ("_BLAH") and the second will have name of the variable
# created in the second expression ("_"), respectively. Since
# the first variable has been assigned the value "1" and the
# second one has the value "0", result of this last expression
# will always be 1 for any interactive shell.
#
# For a non-interactive shell this last expression will have
# the value 0 because both parameters will have the same name
# (there was no "i" to match when forming the name of the
# second var, regardless of whether $- is empty or not).
#
# So for an interactive shell, e.g. when $- is "i", the index is:
#
# (_i=1) + (_=0) - (_i != _)
# 1 + 0 - (1 != 0)
# i.e. 0, thus $ENV[0], for the first and only element
#
# and for a non-interactive shell (e.g. $i is "BLAH") the index is:
#
# (_BLAH=1) + (_=0) - (_BLAH != _BLAH)
# 1 + 0 - (0 != 0)
# i.e. 1, thus $ENV[1], for the second, empty, element
#
# or for a non-interactive shell where $i is "", the index is:
#
# (_=1) + (_=0) - (_ != _)
# 1 + 0 - (0 != 0)
# i.e. 1, thus $ENV[1], for the second, empty, element
#
# Remember class, the test is tomorrow!
fi
Then try using rsh or similar to a 2.0 or newer system and see
that it spits out strange error messages:
$ rsh 192.168.0.220 uname -srm
/home/woods
ksh: No controlling tty (open /dev/tty: Device not configured)
ksh: Can't find tty file descriptor
NetBSD 3.0_STABLE i386
ksh: warning: won't have full job control
See that on on 1.6 and prior target systems it still works
correctly (i.e. only the output from the command is shown):
$ rsh 192.168.0.251 uname -srm
NetBSD 1.6.2_STABLE alpha
See that similarly sftp is also now _completely_ broken and
unusable:
$ sftp whome.planix.com
Keyboard-interactive:
woods@PLANIX.COM's Password:
Keyboard-interactive:
Warning: ssh_packet_wrapper_input: invalid packet received: len 796210483 closing the offending input channel.
Thanks to Andreas Wrede who thought to explore other
representations of that weird value we see that it is 2F75352F
in hex and thus it corresponds to the ASCII string "/u5/", which
just so happens to be the first few characters in my $HOME on
that machine, a string which will be printed initially by my
_interactive_ ~/.kshrc.
>Fix:
revert the following change. ksh should _not_ ever have a
default ENV in a base system install, _especially_ when it is
installed as /bin/ksh!
----------------------------
revision 1.6
date: 2002/11/11 19:03:57; author: jdolecek; state: Exp; lines: +2 -2
default to "$HOME/.kshrc" if ENV is not set
----------------------------
>Release-Note:
>Audit-Trail:
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.