Re: Grow heap aggressively during music interpretation (issue 561390043 by address@hidden)

[jonas . hahnfeld]

On 2020/02/04 22:23:45, hanwenn wrote:
> On Tue, Feb 4, 2020 at 7:50 PM <mailto:address@hidden>
wrote:
> 
> > On 2020/02/02 22:33:50, hanwenn wrote:
> > > For testing, use
> > >
> > > https://github.com/hanwen/lilypond/tree/guile22-experiment
> >
> > So I ran this with the large example and I see
> > GC Warning: Failed to expand heap by 30635458560 bytes
> > a few times (that's 30 GB, my laptop only has 8 GB!!) and finally
> > warning: g_spawn_sync failed (0): gs: Failed to fork (Cannot
allocate
> > memory)
> > Maybe exponential growth is not the best choice here? At least my
system
> > can't handle this score anymore.
> >
> > Also most of the 'speedup' by this patch seems to come from segments
> > _after_ music interpretation (only 10% improvement in music
> > interpretation, but around 2x for the total time). I think the later
> > parts just benefit from the enormous heap because GC doesn't need to
run
> > at all? But that's not really the idea of this patch, is it?
> >
> >
> In part, it actually is. If you have a lot of RAM, you should use it
> instead of trying to waste CPU cycles recycling it.
> 
> do you have a pointer to the score for testing?

I posted this above:
https://lists.gnu.org/archive/html/lilypond-user/2016-11/msg00700.html

> Can you do some timings with different values for GC_MAXIMUM_HEAP_SIZE
, eg.
> 
>  GC_MAXIMUM_HEAP_SIZE=100M

Not sure if you actually mean GC_MAXIMUM_HEAP_SIZE or rather
GC_INITIAL_HEAP_SIZE?

guile22-experiment w/o the growing added in this patch:
default: ~2m33s (gc time taken: 153.316484488)
GC_MAXIMUM_HEAP_SIZE=100M -> "GC Warning: Out of Memory!  Trying to
continue..." (kind of makes sense because it needs ~900MB of RAM)
GC_INITIAL_HEAP_SIZE=100M: 2m30s (gc time taken: 2.189005726; can't
believe this)
GC_INITIAL_HEAP_SIZE=900M: 0m59s (gc time taken: 8.895736892)

> Also, it would be interesting to GC_PRINT_STATS=1 and see how much
time is
> spent in GC, and how much a typical GC runs reclaim across the
process.

Um, that outputs a lot. Anything in particular?

https://codereview.appspot.com/561390043/