Re: Post release texi2any performance regression

[Gavin Smith]

On Sat, Nov 04, 2023 at 04:06:06PM +0000, Gavin Smith wrote:
> > > Applying this patch slows the program down by about 3% now.  However,
> > > it could still be useful for investigating why development texi2any is
> > > still significantly slower (about 5%) than texi2any 7.1.  It seems that
> > > even if the new XS structuring/other modules speed the program up 
> > > slightly,
> > > there must be slowdown somewhere else.
> > 
> > To me the most likely reason would be that simple_parser has been
> > removed, such that calls of gdt/pgdt from perl are slower.
> 
> Thanks, I'll try to investigate this.

After comparing results from NYTProf for Texinfo 7.1 and the master
branch with the patch I posted, the remaining possibility that I suspect,
but which I haven't been able to investigate yet due to being stymied
by other problems, is the code dealing with "untranslated" strings
in Plaintext.pm:

    } else {
      my $tree;
      if ($element->{'extra'}
          and $element->{'extra'}->{'translation_context'}) {
        $tree = $self->pgdt($element->{'extra'}->{'translation_context'},
                            $element->{'text'});
      } else {
        $tree = $self->gdt($element->{'text'});
      }
      my $converted = _convert($self, $tree);
      return $converted;
    }

For the manual I was testing (Emacs Lisp manual), _convert was not
run at all with the 7.1 release, but was run 2519 times with the master
branch with the patch.  I could see this early by NYTProf's caller
information for _convert:

7.1:

#   2871 times (56.0ms+-56.0ms) by Texinfo::Convert::Plaintext::_convert at 
line 3387, avg 0s/call
#   2007 times (19.5ms+-19.5ms) by Texinfo::Convert::Plaintext::_convert at 
line 2154, avg 0s/call

Patched master:

#   2871 times (53.9ms+-53.9ms) by Texinfo::Convert::Plaintext::_convert at 
line 3393, avg 0s/call
#   2519 times (43.4ms+-43.4ms) by Texinfo::Convert::Plaintext::_convert at 
line 1859, avg 0s/call
#   2007 times (30.9ms+-30.9ms) by Texinfo::Convert::Plaintext::_convert at 
line 2163, avg 0s/call

It is comparable due to the same number of calls at different sites, and the
extra call site is easy to spot.

Indeed, commenting out those lines of code (although it makes the output
incorrect) appeared to restore the run time to that of 7.1.

This may have been due to a fairly recent change, but the lack of a
stable, satisfactory reference point ever since the 7.1 release makes
this much, much more time-consuming to investigate.  While one problem is
fixed, another may be inadvertently introduced.

I'm hopeful that this will be it, and we can get such a stable reference
point in the near future.  This is my number one priority for Texinfo
development at the moment.