Re: bad makeinfo slowdown

[Patrice Dumas]

On Sun, Nov 07, 2010 at 10:34:07AM -0800, Per Bothner wrote:
> 
> I hate to point out the obvious: Going back to the old version
> may be the right decision.  I'm sure you've considered it, and
> hate the idea.  Presumably there were structural problem with the old code.
> If so, one could use the Perl version as a prototype for a C version, or
> perhaps a hybrid version that re-uses much of the old code, but whose logic
> is like the Perl version.  

This could be possible.  The new Parser code is much more modular than
the texi2any code.  The Parser itself isn't that big and following
the logic in C and using the same representation of the tree could 
be doable.  Then there are the converters that walk the tree (2 of them
being used by the parser itself for node normalization and error messages
output), they are not complicated, but use perl features, for instance
multibyte/unicode support, and it is not clear that it is easy to have the
same in C.

> Unless the slowdown is due to the structure of
> the new code, rather than using Perl 

A part, indeed, the texi2html/texi2any code is a mess, but the new Parser 
is very clean in my opinion, so, I think that the slowness that will remain 
will be mostly perl.

> - in which case the job is
> easier - stay
> with the old logic! Though I assume there was dome near-fatal problem
> with the old logic?

I haven't studied the old code, but, as far as I have understood, 
the problem was that parsing and outputting weren't very well 
separated, and also that the code for the different output formats
were mixed, and last that it was not extensible.

> A 50x slowdown is really bad, considering that the old code worked
> pretty well for most of us.  You need a pretty good reason to replace
> a working version with one that is 50x slower.  Are there strategic
> plans for a next-generation makeinfo that needs to be able to do
> something the old one couldn't be make to do? 

When developping texi2html to become texi2any to eventually replace
makeinfo in C, I tried to systematically do the right output, based
on what makeinfo did, but also on what was right.  In the process I
did a test suite that shows what the output should be.  It happens
that makeinfo has a lot of bugs, in that it leads to wrong output.
Many tests are not real life manuals, but cases especially crafted
to track all and every corner case, and for many cases, maybe it is
not so bad that the result isn't perfect, so a lot of these bugs may 
not really need to be fixed, but a lot would certainly remain.

Other than those bugs, the most problematic issue was that of the
customization, which is not possible with makeinfo in C but is rather
easy with texi2any.  It is what triggered the switch, with lilypond
wanting a customized HTML that is easily done with texi2any and 
cannot be done with makeinfo in C.

The customization will certainly be very different with the new Parser,
but still certainly easy to do (it should even be easier).

Another feature that is more advanced in the perl versions is
the support for other languages and other encodings, including
east asian languages.  Maybe makeinfo in C could be modified to 
have the same support, though, it is hard for me to tell.

> Sometimes you have to be willing to say "this was a failed experiment".

There is nothing irreversible done for now.  makeinfo in C is still 
there and is still built.  When the Parser is done, I think that 
texi2html should still be shipped, because it can use older perl
and the Parser will certainly not be completly compatible.

Karl is the one who can give directions, I cannot say for him,
but, in the end, I think that the real issue is really who is
willing to do more experiments, that is is there somebody to do
a C implementation that has the features the perl implementations
have?  Until that time, the C makeinfo may be used for those who
wnat speed and the perl one for those who want some features.

-- 
Pat