Re: [O] speeding up org (was How could I mix COMMENT and TODO?)

[Max Mikhanosha]

At Fri, 6 Apr 2012 18:40:43 +0530,
Rustom Mody wrote:
> 
> François wrote:
>  
> 
>     In my very first tries with Org, a few months ago, I put all Org files
>     into the agenda, to discover that Org was very, very slow.  So, I
>     changed it all and collected all agenda and TODO into three files only,
>     holding lots of links to all other Org files where the information
>     really was.  Org recovered all its speed.  And besides, to repair the
>     lost search capabilities, I kludged M-x rgrep so it could search all Org
>     files and "reveal" contents when visiting hits.  Well, the "reveal" does
>     not always work, but yet, the quicker search is constantly useful to me.
>    
>     Currently, having put TODOs back in their proper Org files and declaring
>     them as agenda files, 38 agenda files are taken out from 360 Org files.
>     Even if slightly less speedy than 3 agenda files, this is still very
>     bearable: Org does not crawl.  The way Org handles "org-agenda-files" as
>     a string naming a file is really convenient to me, it eases the writing
>     of external programs acting on them all.  All in all, very satisfactory!
> 
> Hi François,
> 
> I made the suggestion that ragel should/could be part of emacs:
> http://lists.gnu.org/archive/html/emacs-orgmode/2012-03/msg00864.html
> 
> Summarised by saying that if ragel is integrated into elisp, org code could 
> become both significantly faster and more readable.
> 
> That most sluggish elisp code may be so due to regular expression code, is 
> discussed here:
> http://lists.gnu.org/archive/html/emacs-devel/2010-04/msg01202.html
> 
> [In all fairness this is all a bit OT for an org list and should really be on 
> an emacs devel list]

Experiencing slow generation of agenda, was the reason for writing
sticky agendas branch. It takes more of a "fix the symptom" approach,
instead speeding up agenda generation, it caches the agenda buffer
itself and lets user refresh it, while also allowing for multiple
agenda buffers, so that a C-c a / search buffer, does not discard your
C-c a a one.

My initial idea was to wrap all the scanner functions (like
org-todo-list or org-tags-view), with caches, but after a day or so it
was obvious that its not a "a few days" task.

But if someone takes up task of speeding up agenda generation, IMHO
the idea of re-factoring the scanning functions to avoid re-scanning
.org files that had not changed, will have best chance of producing
results.

Regards,
  Max