Re: Should we land Lisp reader optimizations?

[Eli Zaretskii]

> From: Ken Raeburn <address@hidden>
> Date: Tue, 20 Jun 2017 06:12:05 -0400
> Cc: address@hidden
> 
> I implemented the benchmark “read all Lisp forms from a set of .elc files 
> previously loaded into buffers” as mentioned in my earlier email.  I used all 
> the .elc files installed by the build, sorted by name.  The test iterated 
> over the whole set of buffers 10 times, after an untimed initial pass.
> 
> With unchanged master sources (63ec338) it took 327s and 920 GC passes.
> 
> I added the change to short-circuit the recursive processing of certain types 
> (#3); 171s.
> 
> I added the change to mutate the placeholder for a cons object instead of 
> doing recursive substitution (#2); 168s.  (I think it did better for me 
> before but maybe it’s specific to dumped.elc.)
> 
> I added the getc_unlocked change (#1), mainly because other patches I was 
> testing updated the same code and I wanted to get numbers quickly without 
> manually updating the patches; 171s.  (The test is reading from buffers, not 
> files, so this is probably just some random run-to-run variability.)
> 
> I then pulled in the change to replace the read_objects list with two hash 
> tables (part of #4); this brought the run time down to 33.4s, despite an 
> increase to 1049 GC passes.
> 
> I added the iteration change (#5) and replaced seen_list with a hash table 
> (other part of #4); no change.  This is expected from #5, and the seen_list 
> change probably requires more #N# values in an expression for it to be 
> significant.
> 
> I added the symbol interning change (#8); the run time is down to 24.6s, with 
> 631 GC passes, and the speedup appears to be mostly from reducing the GC 
> passes.  Overall improvement: 13x reduction in run time, 31% reduction in GC 
> passes.  Remember this is just for parsing the Lisp expressions, not for 
> evaluating or for reading the bits off the disk.

How much faster does it make reading a large .elc file from disk?

In any case, a 13x speedup sounds very impressive, so I think we want
this on master as soon as you can do it.

What do others think?

Thanks.