Re: texi2any is too slow because of Unicode::Collate

[Gavin Smith]

On Sun, Feb 12, 2023 at 12:04:16PM +0100, pertusus@free.fr wrote:
> On Sun, Feb 12, 2023 at 09:47:10AM +0000, Gavin Smith wrote:
> > 
> > With TEXINFO_XS=omit, it now takes 52 seconds.  I tested Texinfo
> > 7.0.2 and it only took 42 seconds.
> > 
> > (I thought this was still longer than I remembered, so tested
> > with Texinfo 6.8, where it came out 23 seconds without XS modules.
> > This appears to be a major performance regression which I will need to
> > investigate, although it is unrelated to the Unicode::Collate issue.)
> 
> If you are comparing with the devel version it may simply be because I
> recently disabled the XS versions overrides of _merge_text and
> _abort_empty_line because they need significant changes for the source
> marks.  I do not think that we should do it, nor that we should take
> time to optimize the perl parser, it is only there to ease development,
> and even for this purpose it becomes less relevant as times goes by.

No, I was comparing Texinfo 7.0.2 and 6.8 with XS modules disabled.
A change from 23 seconds to 42 seconds is a big problem.

At the moment I'm working through the git history to try to find the
point where the performance got worse.  NYTProf showed that a lot of
the time was spent in _process_remaining_on_line, but I couldn't see
from the line-by-line report where the time was going.

The XS overrides of those two functions is a different question.  I think
it's okay for them to be disabled or removed as it is a maintenance burden
and they should have been only been used occasionally (for some reason,
the XS parser couldn't load but these XS overrides could).