Re: Broken link at http://www.gnu.org/software/texinfo/

[John Mandereau]

On 2008/07/07 19:58 -0500, Karl Berry wrote: 
> On the other hand, my experience with makeinfo is showing me that
> without some consideration of backward compatibility as you go along, it
> is too hard to just "tack on" later.  You end up with what makeinfo is
> now -- a collection of warts.  Anyway.  This is the kind of thing that
> becomes clearer in the actual development than in talking about it,
> IMHO.

I'd like to put all backward compatibility warts in the parser, so clean
formatting rules can be written. I'm not sure it's at all possible, I
may be dreaming :-)


> a Texinfo parser that builds data trees that represent Texinfo
>     documents, to play with for managing translations. Even if I (or we)
> 
> Sorry, I don't see how it's relevant to translations.  You're thinking
> of some gettext-like thing that operates on a strings in the document?

No, I'm thinking about "linking" the same nodes in different languages,
so that e.g. a cross-reference to a node untranslated in some language
can be replaced by a cross-reference to the same node in a different
language.  Other applications are making statistics about which nodes
are translated, or looking for nodes that exist in one lanugage but not
other (if you allow translators to write new documentation in their own
language).


> I was merely imagining a human-written (say) German source document; I
> shouldn't have called it a "translation" specifically.  Support just for
> that is woefully lacking.

I'm not sure what you mean exactly.  LilyPond manuals are already
partially translated in German; all issues with makeinfo and texinfo.tex
I'm aware of have been reported and fixed, I don't see which support for
German is lacking.  I think European languages already have decent
support in Texinfo, even if some improvements would be cool.


>   Let alone (say) Chinese or Arabic, even
> though all those languages and many more are reasonably well-supported
> in TeX and in modern systems in general.

I haven't tested Texinfo with non-European languages, but issues with
documents in Chinese have already been reported on this list.  I
wouldn't be surprised if a lot of things need to be added or rewritten
(even in a rewritten makeinfo :-p) to support non-European languages.
This support is a challenge for Texinfo, but I'm not ready to deal with
it right now, The challenge I'm trying to take is already big enough.


> a parser that can be used in these situations, which is far better than
>     using Python regular expressions :-p
> 
> If you're talking about XML-ish type trees, there is always makeinfo
> --xml, which is basically an XML representation of the Texinfo source.
> But you probably already knew that.

I knew it exists, but have never investigated so much in this direction
for writing my Python scripts.  I'm not sure this would be valuable to
use for a newly written formatter that takes an XML representation of a
Texinfo source as input; this would drastically reduce the cost of
rewriting makeinfo, but this involves two parsing stages, and requires
external libraries for parsing XML.  I'm not familiar with using XML,
does have somebody thoughts about this?


>     1) write a parser
> 
> Yes.  And this alone is a huge job.

I'm sure I'm going to have headaches with this, and after this I expect
to have fun at writing output backends :-)


>   Heck, just *tokenizing* is not
> simple.  It cannot be done anything like a traditional tokenizer/parser,
> since the tokens vary in context.  For instance, @ is not always an
> escape (@verbatim ...).

I think it might be a good idea to create a Flex rule that catches a
whole @ignore or @verbatim block in one go, but this probably involves
some trickery in the parser to catch @ignore and @verbatim blocks which
don't end properly.


> And there is no way to parse bigger constructs without actually
> recognizing/executing some code, because of the verdamnt @macro.

This means @macro definitions and calls should be interpreted while
parsing, which makes sense.

You discouraged me at trying a parser generator, but as you have read
above I'm looking at using Flex to generate a lexer.

Patrice, Karl, thanks a lot for your hints, unless important ideas are
proposed or major issues are raised, I'll start heavily working at
writing code next week, and I'd like to make my code publicly available
at the end of July, whatever its state -- I'd like to achieve a partial
parser, and a minimal text backend that basically ignores all Texinfo
commands.

Cheers,
John