Re: [O] How to make agenda generation faster

[Adam Porter]

Hi Marcin,

My feedback is: there be dragons.  ;)  The Agenda code is very
complicated and hard to follow, and it's hard to optimize something that
is hard to understand.

In the long run, to get significant speed improvements, I think it may
be necessary to reimplement the Agenda.  However, due to the nature of
it (i.e. regexp searches through buffers to find entries), I don't know
how much faster it can be made.  I don't mean that I doubt it can be--I
mean that, truly, I don't know, because it's hard to understand the flow
of the code.

I think that it is already fairly well optimized, given its limitations.
However, an example of a potential improvement would be to refactor it
to work with lexical-binding enabled (which didn't exist when it was
first created); I can't say how much of an improvement it would make,
but my understanding is that code that runs with lexical-binding enabled
is generally faster.  But doing that would be a non-trivial project, I
think, requiring the fixing of many inevitable regressions in the
process.

If you haven't seen them already, you may find my org-ql and
org-ql-agenda code useful.  org-ql-agenda presents an Agenda-like
buffer.  N.B. It does *not* implement most of the Agenda features, but
it does emulate an Org Agenda buffer by setting the appropriate text
properties on entries and formatting them in a similar way.

It's built on org-ql, which provides per-buffer query caching, which
means that generating an org-ql-agenda view for Org buffers that haven't
changed since the last view was generated is very fast.  It's also
written in a more functional way, which I think is easier to follow and
modify.  Performance of uncached queries/buffers depends on the
query--some are relatively fast, while others are slower than the "real"
Org Agenda.  I think there is significant potential for optimizations,
and I'm hoping to implement some in the future.  Your feedback would be
appreciated!

https://github.com/alphapapa/org-ql