NetBSD Problem Report #51385

From mm_lists@pulsar-zone.net  Tue Aug  2 02:39:47 2016
Return-Path: <mm_lists@pulsar-zone.net>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id A85F67A288
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  2 Aug 2016 02:39:47 +0000 (UTC)
Message-Id: <201608020239.u722dhwl017778@ginseng.pulsar-zone.net>
Date: Mon, 1 Aug 2016 22:39:43 -0400
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Subject: NetBSD-7/amd64 tset(1) requires subsequent stty erase '^?'

>Number:         51385
>Category:       bin
>Synopsis:       NetBSD-7/amd64 tset(1) requires subsequent stty erase '^?'
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    mlelstv
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Aug 02 02:40:00 +0000 2016
>Last-Modified:  Wed Aug 17 07:15:01 +0000 2016
>Originator:     Matthew Mondor
>Release:        NetBSD 7.0_STABLE
>Organization:
>Environment:
System: NetBSD ninja.xisop 7.0_STABLE NetBSD 7.0_STABLE (GENERIC_MM) #0: Thu Jul 28 22:49:47 EDT 2016 root@ninja.xisop:/usr/obj/sys/arch/amd64/compile/GENERIC_MM amd64
Architecture: x86_64
Machine: amd64
>Description:

After an update to NetBSD-7 from NetBSD-6, I noticed that in some
applications like vi(1) and vim(1) the backspace stopped working and
issued instances of ^? instead.  Only as the root user, with the same
editor configuration, and in all of: the native text console
(wsdisplay0 at radeondrmkmsfb0 in this case), xterm and urxvt
(rxvt-unicode).  On the other hand, if typing ^H (control-h) instead of
backspace, it works.

This does not occur with other users, and I then noticed that the
difference is that only the root user automatically ran tset(1)
in .profile.  Running tset(1) at any of the aforementioned terminals,
under any user, reproduces the issue.  If then running the
	stty erase '^?'
command, the normal behaviour is restored.

I'm not too familiar with tset(1) and terminfo(3) or the way they
exchange that information, but it would appear that the backspace
character is not being reported correctly on NetBSD-7.  The reset(1)
command also triggers the issue.

I could not yet test after a fresh install, although terminfo related
databases appear to have been updated during the upgrade (and although
old termcap and terminfo.db files which remained were renamed as a
test, they do not appear to interfere):

# ls -l /usr/share/misc/term*
-r--r--r--  1 root  wheel  2314240 Feb  2  2012 /usr/share/misc/termcap.db.old
-r--r--r--  1 root  wheel   555545 Feb  2  2012 /usr/share/misc/termcap.old
-r--r--r--  1 root  wheel  1000215 Jul 28 21:37 /usr/share/misc/terminfo
-r--r--r--  1 root  wheel  1457428 Jul 28 21:37 /usr/share/misc/terminfo.cdb
-rw-r--r--  1 root  wheel  2895872 Jun 14  2012 /usr/share/misc/terminfo.db.old

The entries in /etc/wscons.conf are configured with emulation vt100.
The entries in /etc/ttys are configured with the type wsvt25.

NOTE
====

The urxvt package I have is still for NetBSD-6.  After moving the
termcap files, native X11's xterm works properly after running tset,
which indicates that old termcap files may interfere even though xterm
is linked with terminfo.  However, the non-X11 native console still has
the same issue, like urxvt.

>How-To-Repeat:

$ tset
Erase set to backspace.                                                 
$ vi /tmp/tmp.txt
iiiiiiiiii[backspace][backspace][backspace][ESC]q!
$ stty erase '^?'
$ vi /tmp/tmp.txt
iiiiiiiiii[backspace][backspace][backspace][ESC]q!

>Fix:

Unclear at current time:

- postinstall may need to delete or move away obsolete possibly
  conflicting termcap/terminfo files.
- This seems to not be enough to solve the issue in the native console.

-- 
Matt

>Release-Note:

>Audit-Trail:
From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty
 erase '^?'
Date: Mon, 1 Aug 2016 22:46:51 -0400

 On Tue,  2 Aug 2016 02:40:00 +0000 (UTC)
 Matthew Mondor <mm_lists@pulsar-zone.net> wrote:

 > >Number:         51385

 May be a duplicate of misc/50222, which I have just found now (my
 pre-PR queries had not matched it unfortunately).

 -- 
 Matt

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty
 erase '^?'
Date: Mon, 1 Aug 2016 23:10:28 -0400

 For urxvt, adding in my ~/.Xdefaults:

 URxvt.backspacekey: ^H

 Where ^H is an actual backspace (i.e. entering with ^V^H in vi)
 mitigates the issue.

 -- 
 Matt

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty
 erase '^?'
Date: Mon, 1 Aug 2016 23:57:43 -0400

 For the console, the following also appears to mitigate the issue:

 wsconsctl -w map+="keycode 14 = BackSpace"

 -- 
 Matt

Responsible-Changed-From-To: bin-bug-people->mlelstv
Responsible-Changed-By: mlelstv@NetBSD.org
Responsible-Changed-When: Tue, 02 Aug 2016 05:48:17 +0000
Responsible-Changed-Why:
prevent breaking things
.


From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty erase
 '^?'
Date: Tue, 2 Aug 2016 06:06:15 +0000

 On Tue, Aug 02, 2016 at 02:40:00AM +0000, Matthew Mondor wrote:
  > This does not occur with other users, and I then noticed that the
  > difference is that only the root user automatically ran tset(1)
  > in .profile.  Running tset(1) at any of the aforementioned terminals,
  > under any user, reproduces the issue.  If then running the
  > 	stty erase '^?'
  > command, the normal behaviour is restored.

 Don't use tset; in the modern world it serves no purpose and then not
 infrequently also mangles settings you want left alone :-/

 (also it's high time the ^H/^? thing got fixed for real)

 -- 
 David A. Holland
 dholland@netbsd.org

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty erase '^?'
Date: Tue, 2 Aug 2016 06:08:18 +0000 (UTC)

 mm_lists@pulsar-zone.net (Matthew Mondor) writes:

 >After an update to NetBSD-7 from NetBSD-6, I noticed that in some
 >applications like vi(1) and vim(1) the backspace stopped working and
 >issued instances of ^? instead.  Only as the root user, with the same
 >editor configuration, and in all of: the native text console
 >(wsdisplay0 at radeondrmkmsfb0 in this case), xterm and urxvt
 >(rxvt-unicode).  On the other hand, if typing ^H (control-h) instead of
 >backspace, it works.

 tset got broken much earlier when we replaced termcap with terminfo.
 It then ignored the terminfo definitions for the backspace key. This
 was fixed for NetBSD-7, it now follows the terminfo and behaves again
 like e.g. FreeBSD.

 The issue with what character (BS or DEL) is used by the backspace key
 is very old and we also do not use it consistently. This is in particular
 an issue with people using Linux as their user interface because
 Linux tried to end this by declaring DEL to be the one and only
 right setting. They even patched xterm once to default to use DEL
 together with a patched termcap entry. Of course this doesn't really
 work unless the whole world takes the patches.


 >I'm not too familiar with tset(1) and terminfo(3) or the way they
 >exchange that information, but it would appear that the backspace
 >character is not being reported correctly on NetBSD-7.  The reset(1)
 >command also triggers the issue.

 terminfo says that the backspace key for xterm ($TERM) does emit BS and 
 tset configures the tty layer to interpret BS as the erase key.

 For local ttys (e.g. console) $TERM should match the terminal
 and tset is the right thing to use. But for remote sessions
 (e.g. ssh) $TERM doesn't often match the terminal (in particular
 not for an X terminal emulator running on Linux). ssh will
 also pass through tty layer settings, so using tset here doesn't
 make sense. That's why it was removed from the default .cshrc
 files. Few people use real terminals, most use ssh from another
 system.

 If anything, postinstall should remove the tset call in .cshrc
 when upgrading. But that might be possible for the root account,
 but not for arbitrary rc files from user accounts.

 -- 
 -- 
                                 Michael van Elst
 Internet: mlelstv@serpens.de
                                 "A potential Snark may lurk in every tree."

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty erase '^?'
Date: Tue, 2 Aug 2016 06:11:05 +0000 (UTC)

 mm_lists@pulsar-zone.net (Matthew Mondor) writes:

 > For urxvt, adding in my ~/.Xdefaults:
 > 
 > URxvt.backspacekey: ^H
 > 
 > Where ^H is an actual backspace (i.e. entering with ^V^H in vi)
 > mitigates the issue.


 Either this, or create a new terminfo entry (!= xterm) that declares
 DEL to be the backspace key. This probably requires you to adapt
 the TERM variable on login.

 -- 
 -- 
                                 Michael van Elst
 Internet: mlelstv@serpens.de
                                 "A potential Snark may lurk in every tree."

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty erase '^?'
Date: Tue, 2 Aug 2016 06:22:34 +0000 (UTC)

 mm_lists@pulsar-zone.net (Matthew Mondor) writes:

 > For the console, the following also appears to mitigate the issue:
 > 
 > wsconsctl -w map+="keycode 14 = BackSpace"

 This depends. We unfortunately have different default keymaps
 for PS/2 keyboards (where backspace = DEL) and USB keyboards
 (where backspace = BS).

 The wscons terminals usually use TERM=wsvt25 which declares
 that the backspace key uses BS. So tset will configure erase=^H.

 So, either change the PS/2 default or adapt the wscons keymap as
 you describe (you can put something like this into wscons.conf).

 -- 
 -- 
                                 Michael van Elst
 Internet: mlelstv@serpens.de
                                 "A potential Snark may lurk in every tree."

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty
 erase '^?'
Date: Tue, 2 Aug 2016 04:05:25 -0400

 On Tue,  2 Aug 2016 06:10:02 +0000 (UTC)
 mlelstv@serpens.de (Michael van Elst) wrote:

 >  terminfo says that the backspace key for xterm ($TERM) does emit BS
 > and tset configures the tty layer to interpret BS as the erase key.
 >  
 >  For local ttys (e.g. console) $TERM should match the terminal
 >  and tset is the right thing to use. But for remote sessions
 >  (e.g. ssh) $TERM doesn't often match the terminal (in particular
 >  not for an X terminal emulator running on Linux). ssh will
 >  also pass through tty layer settings, so using tset here doesn't
 >  make sense. That's why it was removed from the default .cshrc
 >  files. Few people use real terminals, most use ssh from another
 >  system.
 >  
 >  If anything, postinstall should remove the tset call in .cshrc
 >  when upgrading. But that might be possible for the root account,
 >  but not for arbitrary rc files from user accounts.

 I understand this.  I use local terminals as much as remote ones, since
 I also use NetBSD as development workstation and desktop as well as
 firewalls and servers (and rarely use Linux, except to
 setup/maintain/configure it for others).

 I remember that when using actual serial terminals to NetBSD systems,
 tset was indeed necessary, and in the past, when using urxvt (or
 previously, aterm) to ssh on some remote systems.  It makes sense that
 it's not always necessary (and I haven't used it in my user shell
 configuration in a while).  At rare occasions, I also had to set custom
 non-default TERM settings, generally over ssh from xterm or urxvt so
 that [n]curses applications on a remote system I connected to worked
 (and remote tset wasn't enough).

 An issue with completely avoiding tset is that the reset command is
 often the simplest way to recover a terminal that has been rendered in
 an unusable state by i.e. inadvertently printing some binary, or broken
 or crashing software changing its settings and leaving the terminal in
 an inconsistent state.

 I noticed that I used to set vt220 in /etc/ttys (and consequently
 default local TERM for ttyEx), and wondered if my switch back to wsvt25
 wasn't responsible for my recent tset issues on NetBSD-7, but I tested
 with vt220 with the same results, and noticed in the terminfo database
 that wsvt25 mostly fallsback to vs220 anyway including for kbs and dch1.

 If I understand, you are saying that there is nothing we can fix, are
 recommending that I create an alternate terminfo vt220 or wsvt25 entry
 variant, as well as an rxvt variant, or that I explicitely configure
 their backspace independently (wscons and urxvt), and that xterm has
 been broken by freedesktop.  Yet I don't remember needing to do this
 since NetBSD 1.5.  So every NetBSD user with a PC and PS/2 keyboard
 must now have a special custom configuration, or completely avoid
 tset/reset.  But at least you didn't tell me to strictly stick to
 hjklx/ctrl-h :)

 A good reason for having kept PS/2 was for debugging and reliability,
 as the bootloader and DDB input often had issues with USB (I'm not sure
 if that's still often the case).

 I can manage with a custom wscons and urxvt configuration.  I also
 could help to provide man page or documentation patches to better
 document this issue and offer configuration examples.  On the other
 hand I'm not convinced that it's the best possible solution...

 If PS/2 IBM-style keyboards are sending different codes, couldn't it not
 be considered one of the architecture-intimate pckbd(4)'s role to
 possibly help (i.e. via lower level layout/code mappings rather than
 encodings), perhaps eventually configurable via a potential pckbdctl(8)
 or similar?

 -- 
 Matt

From: Michael van Elst <mlelstv@serpens.de>
To: gnats-bugs@NetBSD.org
Cc: mlelstv@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
        Matthew Mondor <mm_lists@pulsar-zone.net>
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty erase
 '^?'
Date: Tue, 2 Aug 2016 13:26:18 +0200

 On Tue, Aug 02, 2016 at 08:10:01AM +0000, Matthew Mondor wrote:
 > 
 >  If I understand, you are saying that there is nothing we can fix, are
 >  recommending that I create an alternate terminfo vt220 or wsvt25 entry
 >  variant, as well as an rxvt variant, or that I explicitely configure
 >  their backspace independently (wscons and urxvt), and that xterm has
 >  been broken by freedesktop.

 I am saying that you shouldn't run tset for all these terminals.

 xterm uses ^h by default. So it's fine to run tset as is (unless you
 use some hacked Linux xterm).

 wsvt25/vt220 is fine with a USB keyboard, but for a PS/2 keyboard
 you should configure wscons accordingly.

 rxvt is strange, terminfo says it is using ^h (and shift-backspace is
 ^?), but apparently that's wrong. In any case nobody needs tset here.



 >  Yet I don't remember needing to do this
 >  since NetBSD 1.5.

 I remember needing to do this since about NetBSD 5.


 >  So every NetBSD user with a PC and PS/2 keyboard
 >  must now have a special custom configuration, or completely avoid
 >  tset/reset.

 Actually, every NetBSD user with a USB keyboard had to.

 I wouldn't mind changing the PS/2 keymap to match the USB keymap.
 But that probably ends like an editor war....never.


 >  A good reason for having kept PS/2 was for debugging and reliability,
 >  as the bootloader and DDB input often had issues with USB (I'm not sure
 >  if that's still often the case).

 They still have. In particular with the server that has PS/2 connectors
 for keyboard and mouse but internally runs a PS/2->USB adapter.



 >  If PS/2 IBM-style keyboards are sending different codes, couldn't it not
 >  be considered one of the architecture-intimate pckbd(4)'s role to
 >  possibly help (i.e. via lower level layout/code mappings rather than
 >  encodings), perhaps eventually configurable via a potential pckbdctl(8)
 >  or similar?

 The key codes are fine and standardized, the mappings of course depend
 on the various layouts. For the MF2 keyboard you have the Backspace
 key (keycode 14) and the Delete key (keycode 211). The keyboard itself
 transmits 1-3 byte scan codes for pressing and releasing keys, the
 pckbd driver generates the keycodes from that. It also provides
 a map for wscons to translate this into key symbols.


 US (default for most things):
 /*  pos      command            normal          shifted */
     KC(14),  KS_Cmd_ResetEmul,  KS_Delete,      
 	KC(211),                    KS_Delete,

 GR:
 /*  pos      normal             shifted         altgr        shift-altgr */
     KC(14),  KS_Delete,         KS_BackSpace,



 A USB keyboard with keycode 42 (Backspace) and keycode 76 (Delete)
 (transmitted as is by the keyboard) would use the following maps:

 US (default for most things):
 /*  pos      command            normal          shifted */
     KC(42),                     KS_BackSpace,
     KC(76),                     KS_Delete,

 Colemak:
 /*  pos      command            normal          shifted */
     KC(57),                     KS_BackSpace,

 KC(57) is regularly the CapsLock key on the left side. So here
 you have two Backspace keys :)

 There is also some magic for Apple keyboards. If I understand this
 correctly, the Apple function key will make the Backspace key act
 like the Delete key.



 And then have a look what the tty layer and getty have to say about
 the topic and why bash and tcsh try to be agnostic.


 For now tset shouldn't be used (it got removed from the root .cshrc
 templates) unless you know how and why to use it.



 Greetings,
 -- 
                                 Michael van Elst
 Internet: mlelstv@serpens.de
                                 "A potential Snark may lurk in every tree."

From: Matthew Mondor <mm_lists@pulsar-zone.net>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51385: NetBSD-7/amd64 tset(1) requires subsequent stty
 erase '^?'
Date: Wed, 17 Aug 2016 03:14:44 -0400

 Another issue I noticed is that at a console getty, after a failed
 login attempt the backspace becomes broken, with either USB or PS2
 keyboard.

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.