Re: Grow heap aggressively during music interpretation (issue 561390043 by address@hidden)

[Han-Wen Nienhuys]

On Fri, Feb 7, 2020 at 7:27 PM David Kastrup <address@hidden> wrote:

> Han-Wen Nienhuys <address@hidden> writes:
>
> > Single threaded, the numbers make more sense:
> >
> >
> > MAX=INIT=2G
> > gc time taken: 1.843968805
> > User time (seconds): 49.36
> >
> > MAX=INIT=1G
> > gc time taken: 3.291264925
> > User time (seconds): 51.74
> >
> > MAX=INIT=800M
> > gc time taken: 5.760042906
> > User time (seconds): 54.62
> >
> > 500M
> > gc time taken: 17.921457247
> > User time (seconds): 68.24
> >
> >
> > It's interesting to note that the multithreaded collector doesn't really
> > help.
>
> I think that this may be due to both/either our use of mark hooks and of
> finalisers for calling destructors.  Either may cause serialisation.
> Another serialisation is because Guile itself switches BGC to Java mode
> where finalised objects can no longer be marked (or something like that:
> the exact semantics I do not remember).  And of course the C++ free
> store still has to do its full job.


Interesting. I'll try to put this somewhere as a comment.



> That's one reason why I think it may be a good idea to "punt" a bit on
> the encoding stuff by keeping it basically in 8-bit domain: it may give
> us an operative LilyPond for experimenting with other Guilev2 aspects,
> like making a custom allocator for Scheme-containing data structures
> instead of being in a state where only parts are operative.
>
>
It looks like we'll rather have to punt on the GC scaling. Jonas' run of
the carver score exploded so spectacularly, because the GC stats from libgc
don't mean what I thought they meant, which means that the scaling was
overly aggressive.

It seems impossible to scale up the heap automatically for now.  We should
instead print a message directing people to set

 GC_INITIAL_HEAPSIZE=xxx

in the environment. For more info, see also
https://github.com/ivmai/bdwgc/issues/304. Ivan is open to extending the
API, but until that percolates through all the distributions ...

-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen