Re: Mismatch with st terminfo files

[Thomas Dickey]

On Wed, Nov 06, 2019 at 10:35:13PM +0100, Sebastian J. Bronner wrote:
> Hi Thomas,
> 
> thanks for your input so far.
> 
> > > Using st 0.8.2 with TERM=st-256color and the terminfo database from
> > > ncurses 6.1, I get problems with two applications in tmux:
> > >
> > > * neovim: tmux crashes on nvim start
> > > * w3m: keypresses aren't registered
> 
> I can confirm that starting with the ncurses terminfo database from
> 2019-06-09 and all later versions the tmux crash is gone. The relevant
> change was the addition of Ms=....
> 
> > > When using the terminfo file supplied with st [1], all is well: vim causes
> > > no crashes, and w3m gets its keystrokes.
> > 
> > That's interesting, since st provides a corrupt file :-)
> 
> I have submitted a patch to st that fixes the bad Se and Ss
> capabilities, which has been accepted and applied in git:
> 
> https://git.suckless.org/st/commit/83866428de031300eab03fbb116bcf7d2b1d4f60.html
> 
> > Essentially the problem was that they added capabilities which are
> > supposed to have values, I made one fix but overlooked another.
> > (The tic/infocmp programs didn't handle the case well when st's
> > terminal entry said a value was a boolean and another entry said it was
> > a string).
> > 
> > The most recent terminal database at the webpage fixes that (by
> > eliminating the defects from st' terminfo file)
> > 
> > https://invisible-island.net/ncurses/ncurses.html#download_database
> > 
> > (subsequent database updates aren't _there_ since they won't work
> > with the vanilla ncurses 6.1 - long story).
> 
> There are further differences between the current ncurses and the
> current st upstream terminfo entries for st and st-256color, which may
> warrant further patches against st upstream. However, in order to submit
> patches, I would like to understand the differences better. Would you
> give me feedback on the diff-formatted list below?
> 
> --- st.st-0.8.2+20+g8386642     2019-11-06 16:25:44.951353958 +0100
> +++ st.ncurses-20191102 2019-11-06 16:27:00.506141009 +0100
>  st| simpleterm,
> +       ccc,
> +       
> initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
> +       oc=\E]104\007,
> # These seem to be meant for st-256color. That is where the developers
> # of st put them, anyway. However, they are placed here, in st, and
> # st-256color has these explicitly removed.

hmm - for that, I'd check the history (st and ncurses).

> #
> -       Tc,
> # What is this? I see that it was intentionally removed in the
> # 2018-01-04 release.

ncurses 6.1 supports direct-colors; that's documented.

"Tc" never had a documented meaning with ncurses.

Since some people have an expectation about how it affects the display,
(and that differs from ncurses, i.e., expecting ncurses to magically
decide when/how to use direct-colors rather than using the terminal database)
I removed it.

> +       XT,
> # tic states that XT is expected with kmous. But with XT tic talks about
> # two other unmet expectations: 'Expected orig_colors capability with XT
> # to have 39/49 parameters' and '"tsl" capability is redundant, given
> # XT'.  However, orig_colors does not seem to have the expected
> # parameters, and if tsl is removed, the titlebar does not get updated.
> # This leaves me thinking that XT might specify more than st really
> # supports.
> #
> +       dsl=\E]0;\007,
> +       TS=\E]0;,
> # These are pulled in from xterm+sl. Can the titlebar "status line" even
> # be disabled (tsl)? TS is supposed to be a replacement for tsl when no
> # parameter is needed, which seems to be exactly what is needed with st.
> # But using it without tsl results in the titlebar not being updated. My
> # guess is that st can't do this.
> #
> -       flash=\E[?5h$<80/>\E[?5l,
> +       flash=\E[?5h$<100/>\E[?5l,
> # Upstream says 80 columns, here 100 are specified. Is something wrong

that 80 is a delay time (milliseconds).

st added the 80 in
        commit 1b514048b254827f1a83c05e1b2a2428bf31924e
        Author: Ivan Delalande <address@hidden>
        Date:   Sun Feb 15 00:34:03 2015 +0100

I used 100ms here:

# 2016-11-26
...
#       + add 0.1sec mandatory delay to flash capabilities using the VT100
#         reverse-video control -TD

and since the previous text for that had no delay, I probably didn't
notice the 80 indicated above.  (A consistent delay helps, though...)

> # with 80?
> #
> -       kcbt=\E[Z,

kcbt is usually shift-tab

> -       kent=\EOM,

this is the Enter key on the numeric keypad, which (vt100) can send
an escape sequence when it's in application-mode.

> # I don't understand their function from reading terminfo(5). What are
> # they for?
> #
> +       rin=\E[%p1%dT,
> # According to docs, st does not support scrolling. Can this capability
> # be right, then? It is pulled in from ecma+index.

st.c has this:

        case 'T': /* SD -- Scroll <n> line down */
                DEFAULT(csiescseq.arg[0], 1);
                tscrolldown(term.top, csiescseq.arg[0]);
                break;

and I probably tested it with vttest:

        Menu 11.7: Miscellaneous ISO-6429 (ECMA-48) Tests

(I recall that some terminal programs which have a case for SD/SU
don't do well on the test).

> #
> -       
> sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
> +       
> sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p5%t;2%;%?%p7%t;8%;m,
> # Am I assuming correctly that this change fixes tic's complaints
> # 'checking sgr(dim) "\e(B\e[0m"; compare to "\e[2m"; unmatched "2m"'
> # and "enter_dim_mode differs from sgr(5)"?

probably.  In my check-report, I don't see those warnings.
When I fix those, I start by checking if the source-code has
support for the feature (saves a lot of time), and test it with
tack, etc., if there's any doubt (takes a lot of time, anyway).

> #
> +       u6=\E[%i%d;%dR,
> +       u7=\E[6n,
> +       u8=\E[?1;2c,
> +       u9=\E[c,
> # What are these?

Eric Raymond defined those a while back for tack.
There's a comment at the top of terminfo.src which describes them.

        # INTERPRETATION OF USER CAPABILITIES

> +       kDN3=\E[1;3B,
> +       kDN5=\E[1;5B,
> +       kLFT3=\E[1;3D,
> +       kLFT5=\E[1;5D,
> +       kNXT3=\E[6;3~,
> +       kNXT5=\E[6;5~,
> +       kPRV3=\E[5;3~,
> +       kPRV5=\E[5;5~,
> +       kRIT3=\E[1;3C,
> +       kRIT5=\E[1;5C,
> +       kUP3=\E[1;3A,
> +       kUP5=\E[1;5A,
> # And these?

There's a summary in terminfo.src:

        #### XTERM Extensions:

The uppercase is a convention from terminfo hinting that it's a shifted
key.  I extended that by adding a number which corresponds to xterm's
coding of shift, alt, control and meta.  You may find this helpful:

        https://invisible-island.net/xterm/xterm-function-keys.html

> #
>  st-256color| simpleterm with 256 colors,
> -       ccc,
> +       ccc@,
> -       
> initc=\E]4;%p1%d;rgb\:%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\\,
> +       initc@,
> -       oc=\E]104\007,
> +       oc@,
> # These seem to be intended for use here. However, they are being
> # removed. What am I missing?

st added those in

        commit b331da550b290d54592b7ba11242c92f5a303a48
        Author: Marc André Tanner <address@hidden>
        Date:   Mon May 15 12:45:41 2017 +0200

When I made the st-0.7 entry, I added the features that differed from st-0.6,
but didn't consider moving some of those to st-256color -- you're reminding
me here that is from the original version, and could be updated.

I do look at the terminfo files which some developers have in their
sources, but those are only a starting point for investigating...

> #
> -       pairs#0x7fff,
> +       pairs#0x10000,
> # This might cause attempted access to uninitialized memory, no?

no - only if there's a bug :-)

If an application initializes a pair at a large number, skipping slots,
ncurses fills in the intervening slots.

The practical limit for indexed color appears to be 64k (256 * 256),
and some applications try to use the whole range.

RGB (direct color) doesn't do that, since the palette isn't modifiable.
That's intentional -- no point in allocating several megabytes for a table
that no one wants to customize.

> -       
> setb=\E[4%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
> -       
> setf=\E[3%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,

They're for non-ANSI terminals, and a nuisance :-)

All of the indexed color terminals you're likely to see are extensions
of ECMA-48/ANSI 8-colors.

> # Why removed?
> #
> 
> > > I was wondering: What speaks against allowing the various packagers of
> > > st (debian stterm) to provide the current terminfo files as part of
> > 
> > That would be a bad idea, since st's developers haven't fixed their
> > terminfo bug (more than two years).  If you do that, there's no point
> > in discussing the results on this list.
> 
> My suggestion was backwards, I apologize. I have since realized that a
> central database of terminfo entries is vital for remote sessions. If
> each terminal package were to provide its own terminfo entries, you
> would have to install every terminal you expected users to use when
> connecting to a machine via ssh.
> 
> I appreciate your explanation, despite being faced with this exact bad
> idea in the past, as I have since discovered.
> 
> Sincerely,
> Sebastian

-- 
Thomas E. Dickey <address@hidden>
https://invisible-island.net
ftp://ftp.invisible-island.net

signature.asc
Description: PGP signature