Re: Speed-up compiling

[Patrick Horgan]

Michael Käppler wrote:

Hi all,
I'm "suffering" from enormous compiling durations on large files. With "large" I mean a file with about 250 measures and seven staves per system. The last time I compiled the file completely (without using showLastLength) I did it overnight, since after one and a half hour Lilypond still was "Preformatting graphical elements..." (Don't know what's the correct term in the english version)
I know that's my laptop is way too old to do such complex task quickly - (Athlon XP 2600, 256MB Ram, OpenSUSE 11.1) but are there general suggestions which help to speed up the compiling process?

The one thing you could do is greatly increase your ram to 1GB.  You'll think you have a new computer for $50-$150.  Here's why.

Here's the problem.

You're thrashing to disk, which means the amount of memory used by lilypond is far larger than the available real memory and your system is spending more time moving information between disk and memory than it is actually running your programs.  Modern operating systems support virtual memory which means that you can use a larger amount of pretend memory than the existing real hardware memory.  As much of the virtual memory will be in the hardware memory as possible including the stuff you're using the most, and the part that doesn't fit will be located in a special area of your hard drive.  That means that when lilypond or another application wants to use more than 256MB of memory to speed things up, some of the stuff that should be in hardware memory for speed gets pushed out to the disk.  When lilypond tries to access the part of memory that's actually on disk, it has to move something else from hardware memory to disk to make room, and then read the part you're trying to access from disk back into hardware memory. 

Disks are WAY SLOWER than memory.  A common disk these days might take 10ms (.010 seconds), on average, to access a bit of data.  Memory would likely take 10ns (.000 000 010 seconds) to access a bit of data.  That means that getting the data out of memory is a MILLION times faster.  This is, I'm sure, slowing down many other things for you, not just lilypond.  256MB or even 512MB is not enough for current software on current operating systems.  The truth is even worse than I just explained, because you never get to use the whole 256MB anyway.  Some of that hardware memory is reserved by the OS for the kernel.  It never gets swapped out.  It's locked into physical memory.  On my Ubuntu box right now it's 45MB, although less would be reserved if there was less available memory--I have 2GB physical memory. Be thankful you're not using Microsoft Windows.  It uses up a LOT more memory than Linux just to present you with an interface you can use to run programs.  Your system would die trying to start up.

Here's the fix.

Without buying a new computer, you can buy new RAM (Random Access Memory) and you'll THINK that you have a new computer.  It will make you amazed at the difference!  You will dance around and shout Huzzah!  Woman will hold up their babies to be kissed by you.  You will get a promotion and a humanitarian award.  Yes!  It makes that big a difference!  To find out what kind of RAM you need you can go to the online memory sites of memory vendors like Kingston or Crucial who have tools you can use to look up what kind of memory your computer takes.  Your laptop with Athlon XP 2600, is likely to max out at 1GB-1 2GB.  If it supports 1 1/2 GB don't do it, just get 1GB.  Memory likes to be in amounts that are powers of two (1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, etc.)   1 1/2GB is not any of these and while your hardware may have special circuitry to support that strange amount, it will actually slow you down!

You'll be taking out the 256MB chip and throwing it away.  Depending on the hardware you'll support a maximum size CHIP of either 512MB or 1GB.  Two 512MB chips supply as much memory as one 1GB chip.  If you support the 1GB chip, just buy one of those and for around $50 you'll be happy.  If the maximum size you support is 512MB, you'll have to buy two of those and it will probably cost you $60-$120.  It's possible that your hardware will support expanding to 2GB with two 1GB chips for around $100-$120.  That's up to you, but you wont see a dramatic change for most applications after 1GB.  2GB will let you run more than one memory hungry thing at a time and some rare applications can use the extra memory by themselves (including lilypond with a large enough input file).  It's up to you.

Go to this CNET video for more tips about how to expand RAM.  If you search around there's lots of videos and tutorials about how to do it.  If you have any more questions just ask.

Best regards,
Patrick

p.s. Memory uses a lot less energy that spinning a disk, so if you have power saving on which stops spinning your drive when you aren't using it, you will save much energy and get much longer battery life, and help the environment.  Huzzah!  Spinning drives are the enemy of long battery life.