Re: [O] How to improve Org startup time?

[Sebastien Vauban]

Hi Nick,

Nick Dokos wrote:
> Sebastien Vauban <address@hidden> wrote:
>> >> This may have something to do with my big amount of Org files in
>> >> `org-agenda-files': 36 at this point. But is that so big??
>> >
>> > I don't think so.
>>
>> I'm sure it is, as I wrote (removed from the joined log) a "message" call
>> before and after that line, and those time-stamps were 16 seconds one from 
>> the
>> other.
>
> Small misunderstanding here: the "I don't think so" was a reply
> to the "is that so big?" question.

It was already late for me it seems. Yes, I misunderstood your point. Anyway,
I don't mind getting reminded of things which could have escaped me!

>> > Can you get a profile?
>>
>> You're right I should use real tools for that...
>>
>> > M-x elp-instrument-package RET org-agenda RET
>> > M-x org-agenda-to-appt
>> > M-x elp-results
>> >
>> > In my (admittedly unchallenging, run-of-the-mill) setup, I get
>> > (with everything already loaded):
>> >
>> > org-agenda-to-appt                                            1           
>> > 0.053846964   0.053846964
>> > org-agenda-prepare-buffers                                    1           
>> > 0.028483817   0.028483817
>> > org-agenda-get-day-entries                                    8           
>> > 0.024222044   0.0030277555
>> > org-agenda-get-scheduled                                      8           
>> > 0.0154506449  0.0019313306
>> > org-agenda-get-timestamps                                     8           
>> > 0.004179949   0.0005224936
>> > org-agenda-skip                                               184         
>> > 0.0027937810  1.518...e-05
>> > org-agenda-files                                              2           
>> > 0.001386068   0.000693034
>> > org-agenda-get-deadlines                                      8           
>> > 0.001288303   0.0001610378
>> > org-agenda-format-item                                        22          
>> > 0.0012140690  5.518...e-05
>> > org-agenda-get-blocks                                         8           
>> > 0.0007851970  9.814...e-05
>> > org-agenda-new-marker                                         44          
>> > 0.0006114019  1.389...e-05
>> > org-agenda-skip-eval                                          368         
>> > 0.0001511950  4.108...e-07
>> > org-agenda-todayp                                             16          
>> > 0.0001342800  8.392...e-06
>> > org-agenda-fix-displayed-tags                                 22          
>> > 7.2914e-05    3.314...e-06
>> > org-agenda-get-category-icon                                  22          
>> > 1.8382e-05    8.355...e-07
>> > org-agenda-time-of-day-to-ampm-maybe                          6           
>> > 3.347e-06     5.578...e-07
>> >
>> > A factor of 300: maybe it's real, but let's make sure first.
>>
>> Here it is. I don't know how to interpret that difference, tho.
>>
>> org-agenda-to-appt                                            1           
>> 19.673        19.673
>> org-agenda-prepare-buffers                                    1           
>> 18.86         18.86
>> org-agenda-get-day-entries                                    36          
>> 0.7970000000  0.0221388888
>> org-agenda-files                                              37          
>> 0.5440000000  0.0147027027
>> org-agenda-get-scheduled                                      36          
>> 0.5150000000  0.0143055555
>> org-agenda-get-deadlines                                      36          
>> 0.1580000000  0.0043888888
>> org-agenda-skip                                               612         
>> 0.1410000000  0.0002303921
>> org-agenda-get-timestamps                                     36          
>> 0.047         0.0013055555
>> org-agenda-get-blocks                                         36          
>> 0.046         0.0012777777
>> org-agenda-format-item                                        42          
>> 0.031         0.0007380952
>> org-agenda-skip-eval                                          1204        
>> 0.016         1.32...e-005
>> org-agenda-fix-displayed-tags                                 42          
>> 0.0           0.0
>> org-agenda-todayp                                             72          
>> 0.0           0.0
>> org-agenda-new-marker                                         89          
>> 0.0           0.0
>> org-agenda-deadline-face                                      2           
>> 0.0           0.0
>> org-agenda-get-category-icon                                  42          
>> 0.0           0.0
>
> Well, you have a bigger agenda by a factor of 4-5

Well, I'm still under the floating line, for what concerns GTD (in the sense
of getting things _DONE_).

> and I guess a slower machine, but it all takes less than a second except for
> one thing: the big difference seems to be org-agend-prepare-buffers which
> opens the files, reads them in and gets the buffers ready.

Well, I did have a slow machine (6-year old, 2 GB RAM and plain old HD) up to
last month. Now, I do have a brand new Asus laptop with 4 GB RAM and 256 GB
SSD... It can't be that slow. Not possible...

I have to admit working most of the time without the laptop being powered,
which I know (from observation) is 2 to 3 times slower. But, if I do that,
it's because:

- I want to save my battery life, by making full charges and discharges, hence
  being unpowered is how I work 5 hours out of 7 (charging takes 2 hours).

- I want to be in a situation similar to the one of my colleagues, who are
  using Org (because of me) and don't especially have a machine which is that
  new.

> Once the files are open however, they should be cached, so doing it again
> should not take very long at all: is that the case?

Yes, it is the case.

> If so, my guess is that either you have a very slow disk or you have
> disk problems.

I can't imagine how, but?

> Were things fast at some point in the past?

No, it always has been (too) slow for me, hence my efforts to look at how to
separate different Org functions (fontifying a buffer + Org search support,
from generating an agenda, among others) at different points in time.

For me, loading Org when opening an Org file should not trigger the agenda
stuff (requiring org-agenda and the like). This latter should be triggered by
the autoloads when pressing C-c a, for example.

> In that case, I would worry about the disk. Also, what happens if you copy
> your agenda files to some other machine and try it there? If that's fast,
> then again I would worry about this disk.
>
> An anecdote to illustrate: at one point, it would take me a couple of
> minutes to log in whereas previously it was a few seconds.  The kernel
> log contained a few disk errors. The disk errors apparently caused multiple
> retries, both in the kernel but also (and most probably more
> time-consumingly) in the disk firmware. The errors all happened to be in
> the file that contained my desktop background image. When I changed
> backgrounds (leaving the old file in place so that the bad spots would
> not be used by some other file), my login time went back to a few
> seconds. I replaced the disk and (knock on wood) the problem has not
> reappeared.
>
> So check your kernel logs and maybe run a disk diagnostic program as well.
> If there are errors, try backing up the agenda files (copying them
> should take a long time if my suspicion is correct): do the equivalent
> of
>
>       mv file1.org file1.org.bak
>       # the following might take a long time
>       cp file1.org.bak file1.org
>
> so that the backups will still occupy the error loci. If the disk is
> not too far gone, that might restore the speed of the agenda, but if
> that's the case, don't wait too long to replace the disk: once a disk
> goes a little bad, it tends to go a *lot* bad pretty fast.

I'm on Windows 8, so I think I have less control on such log events than you
do. Tho, I looked in the Event Viewer (tabs "System" and "Hardware Events"),
but did not find anything suspect at this point.

Best regards,
  Seb

--
Sebastien Vauban