Re: Should we land Lisp reader optimizations?

[Ken Raeburn]

I implemented the benchmark “read all Lisp forms from a set of .elc files 
previously loaded into buffers” as mentioned in my earlier email.  I used all 
the .elc files installed by the build, sorted by name.  The test iterated over 
the whole set of buffers 10 times, after an untimed initial pass.

With unchanged master sources (63ec338) it took 327s and 920 GC passes.

I added the change to short-circuit the recursive processing of certain types 
(#3); 171s.

I added the change to mutate the placeholder for a cons object instead of doing 
recursive substitution (#2); 168s.  (I think it did better for me before but 
maybe it’s specific to dumped.elc.)

I added the getc_unlocked change (#1), mainly because other patches I was 
testing updated the same code and I wanted to get numbers quickly without 
manually updating the patches; 171s.  (The test is reading from buffers, not 
files, so this is probably just some random run-to-run variability.)

I then pulled in the change to replace the read_objects list with two hash 
tables (part of #4); this brought the run time down to 33.4s, despite an 
increase to 1049 GC passes.

I added the iteration change (#5) and replaced seen_list with a hash table 
(other part of #4); no change.  This is expected from #5, and the seen_list 
change probably requires more #N# values in an expression for it to be 
significant.

I added the symbol interning change (#8); the run time is down to 24.6s, with 
631 GC passes, and the speedup appears to be mostly from reducing the GC 
passes.  Overall improvement: 13x reduction in run time, 31% reduction in GC 
passes.  Remember this is just for parsing the Lisp expressions, not for 
evaluating or for reading the bits off the disk.

The #N# use for symbols will take some tweaking to make it get used for normal 
byte compilation, and then a full bootstrap, so I’ll try testing that later in 
the week.

I guess that gives a pretty clear indication which changes I should look at 
first.

Ken