NetBSD Problem Report #56496
From hauke@Espresso.Rhein-Neckar.DE Sat Nov 13 13:31:59 2021
Return-Path: <hauke@Espresso.Rhein-Neckar.DE>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 40D941A923A
for <gnats-bugs@gnats.NetBSD.org>; Sat, 13 Nov 2021 13:31:59 +0000 (UTC)
Message-Id: <202111131221.1ADCLF92000990@flatpack.causeuse.org>
Date: Sat, 13 Nov 2021 13:21:15 +0100 (CET)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Reply-To: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@NetBSD.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: etcupdate(8) merge formatting issue
X-Send-Pr-Version: 3.95
>Number: 56496
>Category: bin
>Synopsis: etcupdate(8) merge formatting issue
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: bin-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat Nov 13 13:35:00 +0000 2021
>Last-Modified: Sat Nov 20 14:55:01 +0000 2021
>Originator: Hauke Fath
>Release: NetBSD 9.99.92
>Organization:
Falling Raindrops
>Environment:
System: NetBSD flatpack 9.99.92 NetBSD 9.99.92 (BLACKBOX-$Revision: 1.85 $) #5: Sat Nov 13 00:03:46 CET 2021 hauke@flatpack:/u/scratch/netbsd-build-objects/developer/amd64/sys/arch/amd64/compile/NBLACKBOX amd64
Architecture: x86_64
Machine: amd64
>Description:
The -current etcupdate(8) file merge has a formatting issue:
In a merge session, the right hand column expands whitespace
(tabs, presumably) incorrectly such that even on a 160 col
xterm most of the lines wrap around.
This makes the merge mode almost unusable.
>How-To-Repeat:
Run etcupdate after a base update, and attempt to merge local
config file state with updates.
Find that even in a wide terminal, most of the right-hand side
will wrap around.
>Fix:
Yes, please.
>Audit-Trail:
From: Valery Ushakov <uwe@stderr.spb.ru>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 02:16:33 +0300
On Sat, Nov 13, 2021 at 13:35:00 +0000, Hauke Fath wrote:
> The -current etcupdate(8) file merge has a formatting issue:
> In a merge session, the right hand column expands whitespace
> (tabs, presumably) incorrectly such that even on a 160 col
> xterm most of the lines wrap around.
etcupdate just calls sdiff(1) to do the merge. And i"m pretty sure we
haven't updated diffutils in quite a while...
You can easily try to reproduce it with something like:
sdiff -w $(stty size | cut -d' ' -f 2) -s -a -o merged old new
Are you sure you have correct cols value in stty? (I see that you use
amd64, my first hunch was arm with the "unified" display/serial
console, doing the merge on a serial in a smaller term - you'd get
cols from the wide(r) display there).
-uwe
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 07:57:12 +0000 (UTC)
On Sat, Nov 13, 2021, Hauke Fath wrote:
> The -current etcupdate(8) file merge has a formatting issue:
> In a merge session, the right hand column expands whitespace
> (tabs, presumably) incorrectly such that even on a 160 col
> xterm most of the lines wrap around.
>
Does `stty -oxtabs' or equivalently, setting `XTerm*ttyModes: tabs'
in ~/.Xresources fix the issue?
-RVP
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 12:06:03 +0100
On Sun, 14 Nov 2021 23:20:01 +0000 (UTC), Valery Ushakov wrote:
> etcupdate just calls sdiff(1) to do the merge. And i"m pretty sure we
> haven't updated diffutils in quite a while...
> =20
> You can easily try to reproduce it with something like:
> =20
> sdiff -w $(stty size | cut -d' ' -f 2) -s -a -o merged old new
Indeed, this shows the same problem.
> Are you sure you have correct cols value in stty?
Yes, I checked. (It's an xterm, on two different X servers, for good=20
measure).
Same thing on -9, btw.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 12:17:35 +0100
On Mon, 15 Nov 2021 08:00:03 +0000 (UTC), RVP wrote:
> Does `stty -oxtabs' or equivalently, setting `XTerm*ttyModes: tabs'
> in ~/.Xresources fix the issue?
No, doesn't make a difference.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Valery Ushakov <uwe@stderr.spb.ru>
To: gnats-bugs@netbsd.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 16:25:22 +0300
On Mon, Nov 15, 2021 at 11:10:01 +0000, Hauke Fath wrote:
> On Sun, 14 Nov 2021 23:20:01 +0000 (UTC), Valery Ushakov wrote:
> > etcupdate just calls sdiff(1) to do the merge. And i"m pretty sure we
> > haven't updated diffutils in quite a while...
> >
> > You can easily try to reproduce it with something like:
> >
> > sdiff -w $(stty size | cut -d' ' -f 2) -s -a -o merged old new
>
> Indeed, this shows the same problem.
Do you have a minimized sample old/new input that reproduces it?
-uwe
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: Valery Ushakov <uwe@stderr.spb.ru>
Cc: gnats-bugs@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 16:37:50 +0100
On Mon, 15 Nov 2021 16:25:22 +0300, Valery Ushakov wrote:
> Do you have a minimized sample old/new input that reproduces it?
<ftp://ftp.causeuse.org/pub/NetBSD/pr-bin-56496.tar> has the first=20
dozen lines of inetd.conf, and a screenshot gif that shows the problem.
I noticed that a plain 'diff -u' will also expand whitespace in the=20
same excessive way.
On a=20
% fgrep "Build date" /etc/release=20
Build date Tue Jun 29 23:19:48 UTC 2021
%
netbsd-9 installation, output is fine; it looks like the relevant=20
change has been pulled up since.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 23:39:46 +0700
Date: Mon, 15 Nov 2021 15:40:02 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211115154002.2F87F1A923A@mollari.NetBSD.org>
| I noticed that a plain 'diff -u' will also expand whitespace in the
I can confirm this in HEAD.
I think this might be a tty driver issue though, as "stty oxtabs" looks
to correct the problem.
Further, it looks to be unrelated to anything to do with any diff
variant, as
printf "%70saaaaaaaaaaaaaaaaaa\tbb\n", x
also shows the same problem. The first 'b' shows up in the rightmost column.
kre
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 17:47:40 +0100
Not sure, if I run that in script and hexdump the result I get:
00000110 2e 30 2e 0d 0a 5b 2f 74 6d 70 5d 20 6d 61 72 74 |.0...[/tmp] mart|
00000120 69 6e 40 77 68 6f 65 76 65 72 2d 62 72 69 6e 67 |in@whoever-bring|
00000130 73 2d 74 68 65 2d 6e 69 67 68 74 20 3e 20 70 72 |s-the-night > pr|
00000140 69 6e 74 66 20 22 25 37 30 73 61 61 61 61 61 61 |intf "%70saaaaaa|
00000150 61 61 61 61 61 61 61 61 61 61 61 61 5c 74 62 62 |aaaaaaaaaaaa\tbb|
00000160 5c 6e 22 2c 20 20 08 78 0d 0d 0a 20 20 20 20 20 |\n", .x... |
00000170 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 | |
*
000001b0 78 61 61 61 61 61 61 61 61 61 61 61 61 61 61 61 |xaaaaaaaaaaaaaaa|
000001c0 61 61 61 09 62 62 0d 0a 2c 5b 2f 74 6d 70 5d 20 |aaa.bb..,[/tmp] |
(and the first b did not appear on the rightmost column for me)
Martin
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 23:52:24 +0700 (ICT)
Actually, this might be an xterm bug. I was testing HEAD in a DomU, then
I shut it down, and ran the same test in the underlying xterm, from a
MUCH older system (the printf variant of this, that's easier, needs no
additional data).
Same issue, with -oxtabs the first 'b' is at the right margin (the 2nd one
wraps), with oxtabs, all is fine.
Note this is an OLD xterm ...
-r-xr-xr-x 1 root wheel 505142 Nov 30 2013 /usr/X11R7/bin/xterm
Perhaps there was an old issue, which was fixed, then the fix got lost
in a recent update -- it appears as if xterm was updated in July.
kre
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 23:55:00 +0700 (ICT)
The hexdump output is good of course, printf isn't broken (or
not for something as simple as that).
What's the age of the xterm binary you're using, and what terminal
width did you use?
kre
From: Valery Ushakov <uwe@stderr.spb.ru>
To: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Cc: gnats-bugs@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 20:55:32 +0300
On Mon, Nov 15, 2021 at 16:37:50 +0100, Hauke Fath wrote:
> On a
>
> % fgrep "Build date" /etc/release
> Build date Tue Jun 29 23:19:48 UTC 2021
> %
>
> netbsd-9 installation, output is fine; it looks like the relevant
> change has been pulled up since.
I don't have a current current handy, unfortunately. On a current
from May I don't see the problem.
> I noticed that a plain 'diff -u' will also expand whitespace in the
> same excessive way.
This seems to strongly hint at problems with tab settings, as diff,
afaik, doesn't do any processing on its input.
Do you see the bug with wscons? With some different terminal
emulator? Does something like
printf 'x\tx\tx\tx\tx\tx\tx\tx\tx\tx\tx\tx\tx\n'
(with suitably large number of repetitions) shows the expected ecenly
spaced output?
-uwe
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, Robert Elz <kre@munnari.OZ.AU>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 19:00:50 +0100
At 16:55 Uhr +0000 15.11.2021, Robert Elz wrote:
> What's the age of the xterm binary you're using, and what terminal
> width did you use?
Builds from netbsd-9 Sat Nov 13 14:43:24 UTC 2021 (problem shows) and from
Tue Jun 29 23:19:48 UTC 2021 (problem does _not_ show), and -current from
Fri Nov 12 22:25:53 UTC 2021 (problem shows).
Width is generally between 140 and 160 cols.
Cheerio,
Hauke
--
"It's never straight up and down" (DEVO)
From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 10:21:48 -0800 (PST)
On Mon, 15 Nov 2021, Hauke Fath wrote:
> The following reply was made to PR bin/56496; it has been noted by GNATS.
>
> From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
> To: gnats-bugs@netbsd.org
> Cc: gnats-admin@netbsd.org, Robert Elz <kre@munnari.OZ.AU>
> Subject: Re: bin/56496: etcupdate(8) merge formatting issue
> Date: Mon, 15 Nov 2021 19:00:50 +0100
>
> At 16:55 Uhr +0000 15.11.2021, Robert Elz wrote:
> > What's the age of the xterm binary you're using, and what terminal
> > width did you use?
>
> Builds from netbsd-9 Sat Nov 13 14:43:24 UTC 2021 (problem shows) and from
> Tue Jun 29 23:19:48 UTC 2021 (problem does _not_ show), and -current from
> Fri Nov 12 22:25:53 UTC 2021 (problem shows).
Seems consistent with me, where -current built from sources dated
2021-09-30 at 18:10:00 UTC does _not_ show any problem with Robert's
printf-based test with width=150. In case it's meaningful:
speedy:TOOLS {157} xterm -version
XTerm(368)
speedy:TOOLS {158}
+--------------------+--------------------------+----------------------+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses: |
| (Retired) | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org |
| & Network Engineer | | pgoyette99@gmail.com |
+--------------------+--------------------------+----------------------+
From: Robert Elz <kre@munnari.OZ.AU>
To: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Cc: gnats-bugs@netbsd.org, gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 01:39:59 +0700
Date: Mon, 15 Nov 2021 19:00:50 +0100
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <l03102800ddb84eb2cc5a@q840.causeuse.org>
| Width is generally between 140 and 160 cols.
Are you creating a window that wide (xterm -geom ...) or are you
stretching a "normal" (80 column) window after it is created?
Actually forget that, I doubt that's (directly) the problem either.
Next possibility, are you displaying directly into the xterm
from an application running on the same system (ie: create the
xterm then in it immediately (ignoring window resizing here)
run sdiff (or whatever), or are you connecting elsewhere and
displaying to it via ssh or similar?
Where I detected the problem, I was on an 80 column window,
ssh to another system, then stretched the window wider (and
then used the xen console command to attach to a DomU initially
for testing the -current system - but that's not relevant as
the same issue was reproduced after that was shut down).
kre
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: Robert Elz <kre@munnari.OZ.AU>
Cc: gnats-bugs@netbsd.org, gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 20:38:14 +0100
On Tue, 16 Nov 2021 01:39:59 +0700, Robert Elz wrote:
> Next possibility, are you displaying directly into the xterm
> from an application running on the same system (ie: create the
> xterm then in it immediately (ignoring window resizing here)
> run sdiff (or whatever), or are you connecting elsewhere and
> displaying to it via ssh or similar?
I did the former, so far.=20
When I open a local (macos x 10.13 xquartz) xterm instead, then ssh to=20
the NetBSD machine, the sdiff incantation displays correctly.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 22:04:31 +0000 (UTC)
In an 80-column xterm window try this command with both
`stty -oxtabs' and `stty oxtabs':
printf 'a\ta\ta\ta\ta\ta\ta\ta\ta\ta\ta\ta\n'
With the kernel expanding tabs with `oxtabs', you get `a' evenly
spaced at every 8 cols:
$ printf 'a\ta\ta\ta\ta\ta\ta\ta\ta\ta\ta\ta\n'
a a a a a a a a a a
a a
$
With `-oxtabs' ie. xterm handling tabs, you get this:
$ printf 'a\ta\ta\ta\ta\ta\ta\ta\ta\ta\ta\ta\n'
a a a a a a a a a a a
a
$
But, according to the xterm FAQ[1], this is how a real VT100 behaves.
I recommend you just set `oxtabs' (but, that causes other odd
display issues if a tab char. is embedded within escape sequences...)
FWIW I can't reproduce this at all (-HEAD from Thu Nov 11).
-RVP
[1] https://invisible-island.net/xterm/xterm.faq.html#vt100_wrapping
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 00:03:49 +0100
On Mon, 15 Nov 2021 22:05:02 +0000 (UTC), RVP wrote:
> In an 80-column xterm window try this command with both
> `stty -oxtabs' and `stty oxtabs':
> [...]
Confirmed.
> But, according to the xterm FAQ[1], this is how a real VT100 behaves.
Well, all I can say is that these etcupdate(8) merges have "always"=20
been formatted reasonably, up to and including the netbsd-9 Tue Jun 29=20
23:19:48 UTC 2021 build.
Cheerio,
Hauke
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Mon, 15 Nov 2021 23:24:13 +0000 (UTC)
On Mon, 15 Nov 2021, Hauke Fath wrote:
> Well, all I can say is that these etcupdate(8) merges have "always"=20
> been formatted reasonably, up to and including the netbsd-9 Tue Jun 29=20
> 23:19:48 UTC 2021 build.
>
Post the xterm command line you use and also any xterm-related .Xresource
settings.
BTW, which shell?
-RVP
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 06:56:45 +0700
Date: Mon, 15 Nov 2021 23:05:01 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211115230501.ABCF21A923A@mollari.NetBSD.org>
| Well, all I can say is that these etcupdate(8) merges have "always"
| been formatted reasonably, up to and including the netbsd-9 Tue Jun 29
| 23:19:48 UTC 2021 build.
The underlying issue has nothing to do with etcupdate or sdiff, that
much is clear. There's something else going on, which might be
related to xterm settings (perhaps how it is configured in the resources
database, or something like that).
It appears that sometimes, somehow, the xterm gets tabs set up to
column 80, and no more after that (which results in the next one appearing
to go to just before the final column as RVP indicated vt100's do .. I
no longer remember those things at all).
Certainly setting oxtabs while doing this is a fairly simple workaround,
but there is something else happening.
Hauke, try a full reset from the middle button (by default anyway) menu
after resizing the xterm, see if that makes any difference.
kre
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 00:15:44 +0000 (UTC)
On Tue, 16 Nov 2021, Robert Elz wrote:
> The underlying issue has nothing to do with etcupdate or sdiff, that
> much is clear.
>
Eh... I'm beginning to reconsider now... If you look at that screenshot,
in the second changeset, sdiff seems to be "overcompensating" and adding
a lot more spaces than is warranted.
Hauke, can you post/email your sdiff binary amd the outputs of a: `locale'
and `infocmp'?
-RVP
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 13:13:34 +0700
Date: Tue, 16 Nov 2021 00:20:01 +0000 (UTC)
From: RVP <rvp@SDF.ORG>
Message-ID: <20211116002001.8A3331A923A@mollari.NetBSD.org>
| Eh... I'm beginning to reconsider now... If you look at that screenshot,
| in the second changeset, sdiff seems to be "overcompensating" and adding
| a lot more spaces than is warranted.
You're assuming it is sdiff doing that, and from Hauke's messages, diff as
well - I doubt either of those is expanding tabs (though for sdiff, at least
on the left side, it might make some sense).
But I have seen this happen with printf sending test, and a tab at a position
beyond column 80. Definitely no diff related commands involved, and printf
is most certainly not trying to expand tabs.
That stty oxtabs makes a difference is also revealing.
| Hauke, can you post/email your sdiff binary amd the outputs of a: `locale'
| and `infocmp'?
You can if you want, but none of that will reveal anything useful.
kre
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 06:42:08 +0000 (UTC)
On Tue, 16 Nov 2021, Robert Elz wrote:
> You're assuming it is sdiff doing that, and from Hauke's messages, diff as
> well - I doubt either of those is expanding tabs (though for sdiff, at least
> on the left side, it might make some sense).
>
> But I have seen this happen with printf sending test, and a tab at a position
> beyond column 80. Definitely no diff related commands involved, and printf
> is most certainly not trying to expand tabs.
>
> That stty oxtabs makes a difference is also revealing.
>
> | Hauke, can you post/email your sdiff binary amd the outputs of a: `locale'
> | and `infocmp'?
>
> You can if you want, but none of that will reveal anything useful.
>
You're right. It's xterm after all--I can reproduce that screenshot
exactly:
1. Start an 80x25 xterm:
$ xterm -g 80x25
That dimension is important.
2. Set explicit 8-column tab stops (using tabs(1) for instance):
$ tabs
Now tabs are set at 0, 8, 16, ... 80 (_only_)
3. Resize xterm to make it wider. Last tab position is still only
at 80.
4. Run our familiar sdiff cmd. line.
Now in the second changeset (the RHS one), the if a tab char.
is encountered past col 80 (very likely), and if `-oxtabs' is
set, xterm will advance to the next tab position on that line:
(which defaults to) the _last column_ on that line. The rest of
the RHS line will get printed from there and will follow the
same tab-columnation pattern.
Solution:
Don't set explicit tab positions (wherever it is being set, however
it is being set--tabs(1), tset(1), ...).
Or, "Do Full Reset" in xterm--as kre@ suggested, or run `reset' after
resizing.
-RVP
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 11:14:51 +0100
On Tue, Nov 16, 2021 at 06:45:02AM +0000, RVP wrote:
> Solution:
>
> Don't set explicit tab positions (wherever it is being set, however
> it is being set--tabs(1), tset(1), ...).
... which makes Hauke's shell startup script(s) a likely culprit, and explains
why it is hard or impossible to reproduce for others.
Martin
From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 23:11:57 +0100
On Tue, 16 Nov 2021 10:15:01 +0000 (UTC), Martin Husemann wrote:
> > Don't set explicit tab positions (wherever it is being set, however
> > it is being set--tabs(1), tset(1), ...).
> =20
> ... which makes Hauke's shell startup script(s) a likely culprit,=20
> and explains why it is hard or impossible to reproduce for others.
Unfortunately, no:
I use tcsh, and while csh.{cshrc,login} have historically grown=20
lengthy, I don't set tabs anywhere.
And an xterm started from a newly created unprivileged account using=20
/bin/sh shows the exact same problem.
--=20
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut f=FCr Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344
From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 23:12:53 +0100
On Mon, 15 Nov 2021 16:40:02 +0000 (UTC), Robert Elz wrote:
> Further, it looks to be unrelated to anything to do with any diff
> variant, as
> =20
> =09printf "%70saaaaaaaaaaaaaaaaaa\tbb\n", x
> =20
> also shows the same problem. The first 'b' shows up in the=20
> rightmost column.
This is the simplest test case that we have, so far.
--=20
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut f=FCr Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344
From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Tue, 16 Nov 2021 14:21:41 -0800 (PST)
On Tue, 16 Nov 2021, Hauke Fath wrote:
> > ... which makes Hauke's shell startup script(s) a likely culprit,=20
> > and explains why it is hard or impossible to reproduce for others.
>
> Unfortunately, no:
>
> I use tcsh, and while csh.{cshrc,login} have historically grown=20
> lengthy, I don't set tabs anywhere.
I also considered this, and checked the .cshrc files to ensure that
I was not setting any terminal characteristics. (FWIW, I have also
been able under some situations to reproduce the behaviour, and I
also use tcsh as my shell.)
+--------------------+--------------------------+----------------------+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses: |
| (Retired) | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org |
| & Network Engineer | | pgoyette99@gmail.com |
+--------------------+--------------------------+----------------------+
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 00:40:54 +0000 (UTC)
On Tue, 16 Nov 2021, Hauke Fath wrote:
> Unfortunately, no:
>
> I use tcsh, and while csh.{cshrc,login} have historically grown=20
> lengthy, I don't set tabs anywhere.
>
> And an xterm started from a newly created unprivileged account using=20
> /bin/sh shows the exact same problem.
>
Can't reproduce this at all (apart from using tset or tabs) on my setup.
Can you compile xterm with `--enable-trace' and email the
Trace-{parent,child}.out files that result after running kre's printf
test?
Do you see the same behaviour with another terminal emulator, or a newer
version of xterm (370) ?
-RVP
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 16:07:52 +0100
On Wed, 17 Nov 2021 00:45:01 +0000 (UTC), RVP wrote:
> Do you see the same behaviour with another terminal emulator, or a newer
> version of xterm (370) ?
Reproducible on an Aug 9 netbsd-9 build with modular X11; the xterm was=20
built on Oct 6 and is version 368. The machine is updating pkgsrc as we=20
speak.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 16:23:47 +0100
On Wed, 17 Nov 2021 15:10:01 +0000 (UTC), Hauke Fath wrote:
> Reproducible on an Aug 9 netbsd-9 build with modular X11; the xterm was=
=3D20
> built on Oct 6 and is version 368. The machine is updating pkgsrc as we=
=3D20
> speak.
And on both the modular and the native xterm, the issue goes away when=20
issuing a reset(1) after the re-size - permanently, even after further=20
re-sizes, AFAICS.
--=20
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut f=FCr Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344
From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 07:37:40 -0800 (PST)
On Wed, 17 Nov 2021, Hauke Fath wrote:
> > Do you see the same behaviour with another terminal emulator, or a newer
> > version of xterm (370) ?
>
> Reproducible on an Aug 9 netbsd-9 build with modular X11; the xterm was
> built on Oct 6 and is version 368. The machine is updating pkgsrc as we
> speak.
When I reproduced it, it was on in-tree xsrc, also with xterm 368
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 22:47:11 +0700
Date: Wed, 17 Nov 2021 15:10:01 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211117151001.AA3961A923A@mollari.NetBSD.org>
| On Wed, 17 Nov 2021 00:45:01 +0000 (UTC), RVP wrote:
| > Do you see the same behaviour with another terminal emulator, or a newer
| > version of xterm (370) ?
I believe xterm is acting as it was designed (and has acted forever) - as
poorly designed (or perhaps, just unexpectedly peculiar for modern usages)
as this particular feature might be.
| Reproducible on an Aug 9 netbsd-9 build with modular X11; the xterm was
| built on Oct 6 and is version 368. The machine is updating pkgsrc as we
| speak.
I doubt that checking different xterm versions will make any difference
at all - testing other terminal emulators might be interesting (though
many copy xterm in order to remain compatible).
What is more relevant here is to discover what else is different between
these versions, or perhaps, how you're testing them. Something is causing
the xterm tabs to be set in some cases, and not others. Discovering what
that is, and if appropriate, altering that, is about all that is left to
discover about this PR (which clearly is not really about etcupdate, that
just happens to be able to trigger the issue).
And, from your more recent e-mail:
| And on both the modular and the native xterm, the issue goes away when
| issuing a reset(1) after the re-size - permanently, even after further
| re-sizes, AFAICS.
Not permanently, reset clears the xterm's programmed tabs settings, after
which it treats \t as (to past the next multiple of 8 column number) as
expected - which is independant of terminal width.
But if something happens after which sets the tabs again, it will go back
to operating as it has been doing. This is how it is designed to work.
So, see if you can work out what is setting the tabs in the cases where
they're being set. Something is.
For that, the easiest way would be to use the printf test after stretching
an xterm window, created in various different ways, and see which of them
are affected. If all, then it is likely something in the X resources.
Otherwise, the difference in how they're created might reveal the source
of the setting.
kre
From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@netbsd.org, gnats-admin@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 19:38:07 +0100
On 11/17/21 4:50 PM, Robert Elz wrote:
> And, from your more recent e-mail:
>
> | And on both the modular and the native xterm, the issue goes away when
> | issuing a reset(1) after the re-size - permanently, even after further
> | re-sizes, AFAICS.
>
> Not permanently, reset clears the xterm's programmed tabs settings, after
> which it treats \t as (to past the next multiple of 8 column number) as
> expected - which is independant of terminal width.
>
> But if something happens after which sets the tabs again, it will go back
> to operating as it has been doing.
Sorry, that was overly general. What I was trying to express is that the
effect of reset(1) survives any further xterm resizes,
--
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut für Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344
From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@netbsd.org, gnats-admin@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 19:40:18 +0100
On 11/17/21 4:50 PM, Robert Elz wrote:
> the easiest way would be to use the printf test after stretching
> an xterm window, created in various different ways, and see which of them
> are affected. If all, then it is likely something in the X resources.
There wasn't any change in the X resources. It was simply 'update -9
from a June build, run etcupdate, and notice negative change in formatting'.
--
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email Institut für Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 03:20:06 +0700
Date: Wed, 17 Nov 2021 18:45:01 +0000 (UTC)
From: Hauke Fath <hf=40spg.tu-darmstadt.de>
Message-ID: <20211117184501.C31D61A923A=40mollari.NetBSD.org>
=7C There wasn't any change in the X resources. It was simply 'update =
-9=20
=7C from a June build, run etcupdate, and notice negative change in fo=
rmatting'
I'm still not sure how you're actually doing this testing, it would help
if you were clearer (I mean, the actual steps you take, rather than which=
version you're trying).
That is, are you taking a CD/DVD/USB-stick (or similar, including netboot=
ing)
to an empty system (nothing more than firmware and blank storage) and boo=
ting
it and continuing from there, or ??? I suspect not that, as you're upda=
ting,
not installing, but how exactly? This might matter - including what the=
previous system was. It might also matter whether or not you have reboo=
ted
after the update before running etcupdate. The actual sequence of event=
s
when you're testing the different versions (do these tests all start from=
the
same exact starting environment - system in identical state, etc?)
Further, I wasn't suggesting that you are deliberately changing anything
yourself, but perhaps something in the system, after the updates, could
be different. There might be many differences - different versions of t=
he
termlib database, different system startup scripts, different almost
anything (including the default Xresources files).
Something is configuring the xterm tabs in some cases, and not others. S=
ince
some of us can (occasionally) see this happen, and others cannot, it is m=
ost
likely something in the local setup that we have - even though that somet=
hing
might not be obvious.
It might depend upon some (seemingly unrelated) X resource setting for ex=
ample.
kre
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 20:28:35 +0000 (UTC)
On Wed, 17 Nov 2021, Robert Elz wrote:
> What is more relevant here is to discover what else is different between
> these versions, or perhaps, how you're testing them. Something is causing
> the xterm tabs to be set in some cases, and not others. Discovering what
> that is, and if appropriate, altering that, is about all that is left to
> discover about this PR (which clearly is not really about etcupdate, that
> just happens to be able to trigger the issue).
>
> [...]
>
> So, see if you can work out what is setting the tabs in the cases where
> they're being set. Something is.
>
Martin's script trick is a good one here if recompiling with
`--enable-trace' is too cumbersome. Hauke sould try something like:
env SHELL=/bin/csh xterm -g 80x25 -ls -e /usr/bin/script
then see who's sending '\r\e[3g' followed by '\eH' at each of the tab
positions. But, one has to explicitly set rows and lines with stty after
every window resize because script doesn't trap SIGWINCH.
-RVP
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 21:54:00 +0100
On Wed, 17 Nov 2021 20:25:01 +0000 (UTC), Robert Elz wrote:
> I'm still not sure how you're actually doing this testing, it would help
> if you were clearer (I mean, the actual steps you take, rather than whic=
h
> version you're trying).
Okay... My regular update workflow is
Bring machine to single user
install (selected) tarballs from binary/sets (I have a script for that)
reboot
log in
open xterm, pull it wide to 140 or 160 cols for merges
su to root
run etcupdate -al -w <width> -s /path/to/etc_sets
reboot
The machines in question would all have the same /etc/csh.* files, I=20
use tcsh(1). So the environment is basically identical.
More version dances: My workplace workstation runs netbsd-9 userland=20
from late June, a -current kernel, and pkgsrc xterm v.369 (late Oct).=20
It does not show the tab expansion issue; I am sure when I update -9=20
userland, it will.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 22:48:34 +0100
On Wed, 17 Nov 2021 20:30:02 +0000 (UTC), RVP wrote:
> try something like:
> =20
> env SHELL=3D/bin/csh xterm -g 80x25 -ls -e /usr/bin/script
> =20
> then see who's sending '\r\e[3g' followed by '\eH' at each of the tab
> positions.=20
I did that. Unfortunately, the problem then does not show up (maybe=20
because or the explicit 'stty cols rows'?). In the typescript file,=20
neither the printf ... nor the sdiff ... output lines show the above=20
character sequences.
Same when I open 'xterm -g 140x25'.
> But, one has to explicitly set rows and lines with stty after
> every window resize because script doesn't trap SIGWINCH.
Did that.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 06:23:18 +0700
Date: Wed, 17 Nov 2021 21:50:01 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211117215001.E9F291A923A@mollari.NetBSD.org>
| I did that. Unfortunately, the problem then does not show up (maybe
| because or the explicit 'stty cols rows'?).
Or maybe because tcsh isn't being a login shell when started that way.
| In the typescript file, neither the printf ... nor the sdiff ... output
| lines show the above character sequences.
No, they wouldn't, they're not the source. Something else would be, early
in the xterm's lifetime, and most probably only once.
There's also no point searching for the tab setup if the problem doesn't
happen - in that case, nothing will have set the xterm tabs, and it all works.
| Same when I open 'xterm -g 140x25'.
If you don't resize the terminal, then unless the tabs we set to
a very unusual set of values (which isn't going to happen by accident)
there should be no problem, setting every 8th column (the likely
candidate) is going to be every 8th column up to the current terminal
width.
kre
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Wed, 17 Nov 2021 23:56:44 +0000 (UTC)
On Wed, 17 Nov 2021, Hauke Fath wrote:
> I did that. Unfortunately, the problem then does not show up (maybe=20
>
A better method is with `tmux -lvv' instead of script. Try that.
-RVP
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 10:28:05 +0100
On Wed, 17 Nov 2021 23:25:01 +0000 (UTC), Robert Elz wrote:
> | I did that. Unfortunately, the problem then does not show up (maybe
> | because or the explicit 'stty cols rows'?).
> =20
> Or maybe because tcsh isn't being a login shell when started that way.
The 'xterm -ls' would see to that, no?
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@NetBSD.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 10:32:53 +0100
On Thu, 18 Nov 2021 00:00:03 +0000 (UTC), RVP wrote:
> On Wed, 17 Nov 2021, Hauke Fath wrote:
> =20
> > I did that. Unfortunately, the problem then does not show up (maybe=3D=
20
> >
> =20
> A better method is with `tmux -lvv' instead of script. Try that.
Same 'no show', both with '-g 140x25', and with '-g 80x25'', then=20
resized.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 10:34:52 +0000 (UTC)
On Thu, 18 Nov 2021, Hauke Fath wrote:
> Same 'no show', both with '-g 140x25', and with '-g 80x25'', then=20
> resized.
>
You wrote:
> Okay... My regular update workflow is
> [...]
> open xterm, pull it wide to 140 or 160 cols for merges
> su to root
> run etcupdate -al -w <width> -s /path/to/etc_sets
>
Do root's startup scripts run tabs or tset?
Otherwise, it's either compile with `--enable-trace' or ktrace it:
env SHELL=/usr/pkg/bin/tcsh ktrace -di xterm -g 80x25 -ls
Email the ktrace.out file IFF the trace captures the tab issue.
And make *sure* if you do run a su(1) while under ktrace, to set
a bogus password beforehand, otherwise I *will* be able to see
your real password.
-RVP
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 18:07:41 +0700
Date: Thu, 18 Nov 2021 09:30:02 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211118093002.460C11A923A@mollari.NetBSD.org>
| The 'xterm -ls' would see to that, no?
It would if the xterm was running tcsh directly, but not if it is running
it via script. I don't believe script has a way to pass along that hack.
Not sure if tmux can or not.
kre
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@NetBSD.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 12:30:23 +0100
On Thu, 18 Nov 2021 10:35:01 +0000 (UTC), RVP wrote:
> Do root's startup scripts run tabs or tset?
I have this, since ~forever:
% fgrep -2 tset /etc/csh.*
/etc/csh.login-if (${?prompt}) then
/etc/csh.login- if (${OSTYPE} !=3D 'darwin') then
/etc/csh.login: eval `tset -Q -s -m 'dialup:?vt100' -m=20
'unknown:?vt100' -m 'network:?xterm'`
/etc/csh.login- endif
/etc/csh.login- if (${OSTYPE} =3D=3D 'NetBSD' || ${OSTYPE} =3D=3D 'darwi=
n')=20
then
%
and commenting out the 'eval ...' line makes the formatting issue=20
disappear.=20
No dealing with tabs, AFAICS.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 20:46:14 +0700
Date: Thu, 18 Nov 2021 11:35:02 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211118113502.45FD71A923A@mollari.NetBSD.org>
| % fgrep -2 tset /etc/csh.*
| /etc/csh.login-if (${?prompt}) then
| /etc/csh.login- if (${OSTYPE} !=3D 'darwin') then
| /etc/csh.login: eval `tset -Q -s -m 'dialup:?vt100' -m=20
that would do it
| No dealing with tabs, AFAICS.
see tset(1) ... setting the tabs is something
it does.
for the variable performance, it would make all
the difference which order you used to widen the
xterm and become root
widen first, no problems, root first, bad tabs
after widening
Anyway, this is all solved now.
kre
From: Paul Goyette <paul@whooppee.com>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 06:50:38 -0800 (PST)
On Thu, 18 Nov 2021, Hauke Fath wrote:
> The following reply was made to PR bin/56496; it has been noted by GNATS.
>
> From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
> To: gnats-bugs@netbsd.org
> Cc: gnats-admin@NetBSD.org
> Subject: Re: bin/56496: etcupdate(8) merge formatting issue
> Date: Thu, 18 Nov 2021 12:30:23 +0100
>
> On Thu, 18 Nov 2021 10:35:01 +0000 (UTC), RVP wrote:
> > Do root's startup scripts run tabs or tset?
>
> I have this, since ~forever:
>
> % fgrep -2 tset /etc/csh.*
> /etc/csh.login-if (${?prompt}) then
> /etc/csh.login- if (${OSTYPE} !=3D 'darwin') then
> /etc/csh.login: eval `tset -Q -s -m 'dialup:?vt100' -m=20
> 'unknown:?vt100' -m 'network:?xterm'`
> /etc/csh.login- endif
> /etc/csh.login- if (${OSTYPE} =3D=3D 'NetBSD' || ${OSTYPE} =3D=3D 'darwi=
> n')=20
> then
> %
On the one target which I have reproduced the wrong formatting, I
also have an invocation of tset in my ~/.login, albeit somewhat
simpler:
eval `tset -s -m 'network:?xterm'`
This is only on my old 6.1.5 system. It has been removed (somewhere
along the river of time) from my main system.
+--------------------+--------------------------+----------------------+
| Paul Goyette | PGP Key fingerprint: | E-mail addresses: |
| (Retired) | FA29 0E3B 35AF E8AE 6651 | paul@whooppee.com |
| Software Developer | 0786 F758 55DE 53BA 7731 | pgoyette@netbsd.org |
| & Network Engineer | | pgoyette99@gmail.com |
+--------------------+--------------------------+----------------------+
From: Hauke Fath <hf@spg.tu-darmstadt.de>
To: gnats-bugs@netbsd.org, gnats-admin@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 22:40:45 +0100
On 11/18/21 2:50 PM, Robert Elz wrote:
> that would do it
> =20
> | No dealing with tabs, AFAICS.
> =20
> see tset(1) ... setting the tabs is something
> it does.
Even when all 'tset -Qs' is supposed to do is produce shell-specific=20
command strings for setting TERM?
> for the variable performance, it would make all
> the difference which order you used to widen the
> xterm and become root
Except I ran all the printf... and sdiff... tests as a regular user. No=20
su(1) involved.
> widen first, no problems, root first, bad tabs
> after widening
Not what I see here, no.
> Anyway, this is all solved now.
Well, something changed for the worse in base after last June.
--=20
The ASCII Ribbon Campaign Hauke Fath
() No HTML/RTF in email=09 Institut f=FCr Nachrichtentechnik
/\ No Word docs in email TU Darmstadt
Respect for open standards Ruf +49-6151-16-21344
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 22:59:51 +0100
On Thu, 18 Nov 2021 14:55:01 +0000 (UTC), Paul Goyette wrote:
> On the one target which I have reproduced the wrong formatting, I
> also have an invocation of tset in my ~/.login, albeit somewhat
> simpler:
> =20
> =09eval `tset -s -m 'network:?xterm'`
I ended up checking if TERM is set and non-empty, and only doing the=20
tset dance if it isn't. Since xterm(1) sets TERM, that takes care of=20
things.
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Thu, 18 Nov 2021 22:38:07 +0000 (UTC)
On Thu, 18 Nov 2021, Hauke Fath wrote:
> Even when all 'tset -Qs' is supposed to do is produce shell-specific=20
> command strings for setting TERM?
>
Unfortunately, yes. One has to do `tset -Qs -` to make tset not
send the term init sequences. You can check this by running both
command variants under script.
> Except I ran all the printf... and sdiff... tests as a regular user. No=20
> su(1) involved.
>
If that tset command is in /etc/csh.login, then all (login-shell) uses of
csh would show this issue--provided `-oxtabs' is _also_ set. In some
instances it is not, and that would also account for why some folks see
this issue and others do not.
And, since this tab-setting stuff is an xterm feature, once set, will
propagate to affect other command run in that session--sh for instance
will appear to suffer this issue as well even when its .rc scripts are
not running tset at all.
Hope this helps...
-RVP
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, RVP <rvp@SDF.ORG>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Fri, 19 Nov 2021 12:31:43 +0100
On Thu, 18 Nov 2021 22:40:01 +0000 (UTC), RVP wrote:
> On Thu, 18 Nov 2021, Hauke Fath wrote:
> =20
> > Even when all 'tset -Qs' is supposed to do is produce shell-specific=
=3D20
> > command strings for setting TERM?
> =20
> Unfortunately, yes. One has to do `tset -Qs -` to make tset not
> send the term init sequences. You can check this by running both
> command variants under script.
That would be great, except for whatever reason `tset -Qs -` output=20
prepends the terminal string:
% echo `tset -Qs`
set noglob; setenv TERM xterm; unset noglob;
% echo `tset -Qs -`
xterm set noglob; setenv TERM xterm; unset noglob;
% eval `tset -Qs -`
xterm: No absolute path found for shell: set
%=20
> If that tset command is in /etc/csh.login, then all (login-shell) uses o=
f
> csh would show this issue--provided `-oxtabs' is _also_ set.=20
Which the tset command does, for whatever reason.
> In some
> instances it is not, and that would also account for why some folks see
> this issue and others do not.
> =20
> And, since this tab-setting stuff is an xterm feature, once set, will
> propagate to affect other command run in that session--sh for instance
> will appear to suffer this issue as well even when its .rc scripts are
> not running tset at all.
Would it make sense to follow the eval `tset ...` with an 'stty oxtabs'=20
for (at least) TERM =3D=3D xterm, or would I incur any more side-effects?
> Hope this helps...
It did. A lot.
Thanks,
Hauke
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Valery Ushakov <uwe@stderr.spb.ru>
To: gnats-bugs@netbsd.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Fri, 19 Nov 2021 16:40:02 +0300
On Thu, Nov 18, 2021 at 21:45:01 +0000, Hauke Fath wrote:
> > see tset(1) ... setting the tabs is something
> > it does.
>
> Even when all 'tset -Qs' is supposed to do is produce shell-specific
> command strings for setting TERM?
TERM is not describing something set in stone. You can can have a
termcap entry (i.e. the value that TERM points to) that, say,
describes vt220 with application cursor keys and have the init
sequence that turns that feature on. Or you can setup the entry to
describe normal cursor keys and have the init sequence that turns that
off. Now, I no longer remember if application keys are the default on
vt220 (probably not), but if the vt220 entry is set up with
application keys and you don't output the init sequence, you will not
see the application key sequences on key presses. Ditto for other
features, like auto wrap and what not. You can actually see in the
database entries like vt200-w for 132 columns wide mode, etc.
So it's not really that unlogical a choice for tset to re/init the
termial. It's like: "Ok, I have put your device in the specicic known
state w.r.t. its possible configurations, that is described by the
termcap entry $TERM".
On Fri, Nov 19, 2021 at 11:35:01 +0000, Hauke Fath wrote:
> > Unfortunately, yes. One has to do `tset -Qs -` to make tset not
> > send the term init sequences.
>
> That would be great, except for whatever reason `tset -Qs -` output
> prepends the terminal string:
>
[...]
> % echo `tset -Qs -`
> xterm set noglob; setenv TERM xterm; unset noglob;
echo `tset -Qs - | sed 1d` ?
-uwe
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Sat, 20 Nov 2021 00:01:45 +0700
Date: Fri, 19 Nov 2021 11:35:01 +0000 (UTC)
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Message-ID: <20211119113502.0A5581A923A@mollari.NetBSD.org>
| That would be great, except for whatever reason `tset -Qs -` output=20
| prepends the terminal string:
You really shoukd read tset(1)...
The options are as follows:
- The terminal type is displayed to the standard output, and the
terminal is not initialized in any way.
-I Do not send the terminal or tab initialization strings to the
terminal.
that's not the complete set obviously.
Note the: "The terminal type is displayed to the standard output"
This allows (in sh) TERM=$(tset -) or similar usage in any other
context.
So try:
eval `tset -QIs`
or whatever the csh syntax is that works.
kre
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: Valery Ushakov <uwe@stderr.spb.ru>
Cc: gnats-bugs@netbsd.org
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Fri, 19 Nov 2021 18:27:39 +0100
On Fri, 19 Nov 2021 16:40:02 +0300, Valery Ushakov wrote:
> [...] it's not really that unlogical a choice for tset to re/init the
> termial. It's like: "Ok, I have put your device in the specicic known
> state w.r.t. its possible configurations, that is described by the
> termcap entry $TERM".
Thanks, that helped...
>> % echo `tset -Qs -`
>> xterm set noglob; setenv TERM xterm; unset noglob;
>=20
> echo `tset -Qs - | sed 1d` ?
... as did this, nicely.
Cheerio,
Hauke
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, Robert Elz <kre@munnari.OZ.AU>
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Fri, 19 Nov 2021 18:36:39 +0100
On Fri, 19 Nov 2021 17:05:01 +0000 (UTC), Robert Elz wrote:
> You really shoukd read tset(1)...
Point taken.=20
I find that reading man pages is a skill that needs constant practice,=20
or it will dull. So easy to overlook crucial details.
> So try:
> =09eval `tset -QIs`
Works nicely, thanks.
Cheerio,
Hauke
--=20
Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Linn=E9weg 7
64342 Seeheim-Jugenheim
Germany
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/56496: etcupdate(8) merge formatting issue
Date: Fri, 19 Nov 2021 22:15:33 +0000 (UTC)
On Fri, 19 Nov 2021, Hauke Fath wrote:
> On Thu, 18 Nov 2021 22:40:01 +0000 (UTC), RVP wrote:
>
>> If that tset command is in /etc/csh.login, then all (login-shell) uses of
>> csh would show this issue--provided `-oxtabs' is _also_ set.
>
> Which the tset command does, for whatever reason.
>
> [...]
>
> Would it make sense to follow the eval `tset ...` with an 'stty oxtabs'
> for (at least) TERM == xterm, or would I incur any more side-effects?
>
As an FYI,
a) getty also sets `-oxtabs' for virtual terminals, and,
b) xterm, by default, tries to copy the settings of whatever terminal it
was started from.
The combination of a) and b) can have surprising effects.
Try this first:
$ stty oxtabs; xterm; stty -oxtabs; xterm
Do a `stty -a | fgrep oxtabs' in each xterm instance and see how xterm
copies the current terminal setting.
Now, starting from the top (or, bottom):
1. When a TTY is opened, the kernel applies some default settings
which can be seen in
/usr/include/sys/ttydefaults.h
For output, the NetBSD kernel sets `oxtabs' (see: TTYDEF_OFLAG);
FreeBSD sets `tab0' aka. `-oxtabs'.
Linux also has a <sys/ttydefaults.h> file, but, that is just
there for compatibility's sake--the actual settings are hard-coded
in the driver (`tab0').
2. getty opens, say, /dev/ttyE0 and sets `-oxtabs' as per /etc/gettytab
3. In ~/.xinitrc you have something like:
...
xterm -g 100x35 -fn 10x20 -fg Ivory -bg Black ... &
exec evilwm
Here that xterm will inherit `-oxtabs' from /dev/ttyE0, but, another
xterm started in a different way may not: my WM is evilwm and you
can create a new xterm window any time using Ctrl-Alt-Enter. To do
that evilwm does this:
39 void spawn(const char *const cmd[]) {
...
45 if (!(pid = fork())) {
46 // Put first fork in a new session
47 setsid();
48 switch (fork()) {
...
54 case 0: execvp("xterm" ...); break;
55 default: _exit(0);
That setsid() loses whatever terminal evilwm had and xterm will
therefore inherit the default applied by the kernel which is
`oxtabs'. So, now I have 2 xterm windows running side-by-side,
one with `-oxtabs' set and the other with `oxtabs' and programs
behaving in subtly different ways in each as a consequence. (And
the same program behaving differently on FreeBSD and Linux compared
with NetBSD.) This bit me some time ago, so now I nail down tab
behaviour using:
XTerm*ttyModes: tabs
in ~/.Xresources to always set `-oxtabs'
You can work out what will happen in other situations:
- xterm started with or without a login shell
(login shell runs `tset -s ...'; normal shell doesn't);
- user starts X via xinit instead of startx
(so no initial TTY)
- etc ...
-RVP
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/56496 CVS commit: src/sys/sys
Date: Sat, 20 Nov 2021 09:52:22 -0500
Module Name: src
Committed By: christos
Date: Sat Nov 20 14:52:22 UTC 2021
Modified Files:
src/sys/sys: ttydefaults.h
Log Message:
PR/56496: Hauke Fath: etcupdate(8) merge formatting issue
Follow suit and turn off OXTABS by default
FreeBSD did that in 1994:
r3505 | wollman | 1994-10-10 20:16:28 -0400 (Mon, 10 Oct 1994) | 5 lines
Turn off OXTABS by default. Inspection of systems here finds no commercial
systems with it on by default (or the equivalent flag) and terminal control
sequences confuse it greatly. (Try running `ls' under bash in an XTerm,
for instance.)
OpenBSD did that in 2019:
date: 2019/03/12 11:01:25; author: nicm; state: Exp; lines: +2 -2;
commitid: XOmQAZHjspUKWzDx;
Almost all terminals now support hardware tabs so default to OXTABS off.
This makes three changes: adds the ht capability to the standard lines
in gettytab(5); removes OXTABS from TTYDEF_OFLAG in ttydefaults.h (the
defaults used by pty(4) - diff from martijn); and only sets OXTABS on
terminals which lack hts and tbc in tset(1) (from Thomas Dickey
upstream).
Addresses problems reported by tedu.
ok millert
To generate a diff of this commit:
cvs rdiff -u -r1.16 -r1.17 src/sys/sys/ttydefaults.h
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home