Re: [PATCH 4/9] man/clear.1: Migrate macro usage conventions.

[Ingo Schwarze]

Hi Thomas & Branden,

Thomas Dickey wrote on Sun, Oct 01, 2023 at 10:32:25AM -0400:

> See attached

I see.  Using that file, i'm indeed able to confirm that there
is a small swarm of bugs in mandoc that can be reproduced with
mandoc-current, not only with released versions of mandoc.  I do not
know why, at some point, Branden had the impression that some version
of mandoc worked with that file with no issues.

To reproduce the bugs, two conditions need to be fulfilled:

 1. The input file uses a multi-line conditional request
    inside a macro definition request
    inside a multi-line conditional request.

 2. The condition of the inner conditional request
    uses the special escape sequence "\\" at the beginning of the
    condition or immediately after an initial negation operator.

In that case, mandoc misparses a line like

> .  if \\n(mE \{\

as a string comparison request comparing the strings "" and "n(mE ",
treats the backslash of the "\{" as the byte ending the imagined
string comparison operator, and treats the string "{" as a single-line
textual body of the conditional request, missing the fact that there
is an escape sequence starting a multi-line conditional scope.
That causes the subsequent

  ERROR: skipping unknown macro: .\}

because it sees two conditional block end markers but only one such
block opening.

The other warning,

  ERROR: skipping end of block that is not open: ..

is harmless but quite ugly and misleading, so it should probably also
be fixed.  It is due to the fact that when parsing a conditional block
with a false condition, mandoc only parses for requests that have
structural effects (like conditionals and loops) but disables most
non-structural effects of most other requests and macros.  For example,
under .if 0, the code that causes .de to begin a macro definition
is skipped, which is indeed a desired effect because Branden used
the outer .if request specifically to disable the macro definition.
However, the expectation to find an end request for the .de is set
up inside the same code, so it is also skipped, and in the end,
mandoc thinks the .. comes as a surprise.

Fixing that probably requires treating macro definition requests
as structural requests, still disabling the actual definition of
the macro, but not skipping the setup of the expectation of the end
macro, just like finding a multi-line conditional request inside a
conditional request with a false condition doesn't let the body of
the inner conditional take effect, whatever the value of its condition
may be, but still sets up the expectation of the \} end marker.

While making macro definition requests structural, i should probably
also disable structural parsing inside definition blocks that
are disabled by an enclosing conditional with a false condition.
When a definition actually takes effect, structural parsing is
already disabled.  That's required because defining a macro than
only opens but does not close a coniditional request blocks is valid,
see the example at the end of this message, but when the definition
isn't even taking effect, structural parsing makes no sense, either.

To summarize, this results in this todo-list for me:

 1. Make macro definition requests structural.
 2. Disable structural parsing inside if an enclosing condition is false.
 3. Expect a macro definition end macro even if the definition is
    disabled by an enclosing false condition.
 4. Consider what to do about leading escape sequences
    in non-numerical conditionals.

The joys of the low-level roff(7) language!

I can't give an ETA, but i think i should eventually fix all this.
The difficulty looks moderate (neither easy nor excessively hard) and
the priority moderate, too, because this is rather severe misparsing
of low-level stuff that is already considered somewhat supported.

All that said, for the time being, my impression is that mandoc(1)
is not actually misformatting the page you sent.  It is merely
reporting bogus errors about the low-level stuff that Branden
added at the beginning, but it appears to in effect ignore that
low-level stuff as intended.  Or do you see evidence of
misformatting?

I still have my doubts about the wisdom of going to such lengths
for supporting the old Kernighan troff in 2023, especially when that
implies risking havoc with more modern formatters.  I mean, Kernighan
troff has been mostly obsolete, unmaintained, and virtually unused
(as far as i'm aware) for at least two, or more likely closer to
three descades now...
Does your library have users you are communicating with who
actually need Kernighan troff compatibility for their practical
purposes?

Yours,
  Ingo


.TH TEST 1 2023-10-02
.SH NAME
test \- test
.SH DESCRIPTION
.\" On the following line, try "if 0", too.
.if 1 \{\
.de sbl
.if (\\n[flag]) \\{start block
..
.\}
.nr flag 1
.sbl
additional text\}
.br
flag off:
.nr flag 0
.sbl
end block\}
end of file