Re: RFC: Adding syntax highlighting to the official documentation

[Jean Abou Samra]

Hi Mats,

Thanks for your thoughts!


Le 31/01/2022 à 09:19, Mats Bengtsson a écrit :
>     Thanks for your work on this! A couple of comments (sorry, I haven't
>     taken the time to look and comment on your proposal earlier):
>
>     - I cannot see that you explain to the user what syntax classes the
>     different colors correspond to. It could of course be seen as a
>     pedagogical trick to let the readers figure out the system themselves,
>     and thereby learning more about the syntax, but I guess that many
>     readers would just get confused or perhaps irritated if they try to see
>     a pattern and don't really get it. Such an explanation could be done in
>     two ways, either in the LM by introducing each color in parallel with
>     the introduction of the corresponding syntactic class, or as a brief
>     list aimed at the readers who already know the syntax. Ideally, we
>     could have both of these.



Will do.



>     - On a page like
>     [1]http://abou-samra.fr/new-highlighting-demo/notation/the-offset-comma
>     nd.html, there are boxes where the syntax is defined, but in those
>     there's no syntax highlighting, in contrast to the full examples. How
>     difficult would it be to add syntax highlighning also within these
>     blocks, generated from
>     @example ... @end example in the .texi code?



You touch a tricky point here. We use @example for a
number of purposes. One is to demonstrate syntax,
as in the page you mention. Another is to give
examples that would require too much boilerplate
code to show something useful when actually
compiled, as is often the case in the section about
spacing, e.g.:
http://abou-samra.fr/new-highlighting-demo/notation/the-paper-block.html

And then, we occasionally use it for something different
entirely, for instance hyperlinks as in:

http://lilypond.org/doc/v2.22/Documentation/notation/replacing-the-notation-font

Because @example can contain arbitrary Texinfo
commands, it's not possible in general to predict
what the printed text will look like from the source.
Highlighting examples with special commands wouldn't
be desirable either -- you don't want the URL
to be highlighted in the hyperlink case. In
the case of syntax explanations, there can be
extra elements messing up the highlighting,
such as the brackets in

[-]\offset property offsets item

which are going to be interpreted as explicit
beams. But the absence of @-commands cannot
guarantee that the snippet is LilyPond code
either -- think of
http://lilypond.org/doc/v2.22/Documentation/usage/invoking-convert_002dly

So I think we would want this to be explicit in
the Texinfo source. My idea for this would be
@lilypond[verbatimonly,quote] as a counterpart
to @lilypond[verbatim,quote]; that would also
allow to get rid of escaping @ { } with @@ @{ @}
inside of @example, by letting @lilypond[verbatimonly,...]
interpret the content as @verbatim does, as
opposed to @example. The main difference is that
@verbatim does not interpret @-commands, on
contrary to @example. The reason why we are currently
seldom using @verbatim is that we have a texi2html
customization to change the styling of @example,
making it as if enclosed in @quotation --
as you can notice from the bad styling of
http://abou-samra.fr/new-highlighting-demo/notation/table-of-contents.html
which uses @verbatim. So this would kill two
birds with one stone: allow syntax highlighting,
and save escaping @{} while not requiring the
slightly cumbersome option of enclosing the
@verbatim ... @end verbatim in a
@quotation ... @end quotation.

So that is doable, but as you can see there is
a fair bit of work to invest, especially for
changing many instances of @example to
@lilypond[verbatimonly,quote] ...

$ git grep "@example" | wc -l
4626

... and I currently have a hundred higher
priorities (including syntax highlighting-related
ones, such as: fixing the Scheme lexer in
Pygments upstream to recognize +inf.0 et al.,
or adding an option to LilyPond proper for outputting
lists of builtins so an external tool does not
have to poke with undocumented internals, and
to allow other tools to reuse these lists).

Best,
Jean