Re: Grow heap aggressively during music interpretation (issue 561390043 by address@hidden)

[Han-Wen Nienhuys]

On Sun, Feb 9, 2020 at 10:25 AM <address@hidden> wrote:

> The approach sounds good to me. I can confirm that this works on my
> system and the runtime is pretty good. Please readd the autoconf logic
> to detect bdw-gc.
>

will do.


> FWIW I tried to research on the internals of GC. I found the following
> statement that decides whether to collect or just expand the heap:
> https://github.com/ivmai/bdwgc/blob/v8.0.4/alloc.c#L1435 We are hit by
> the case
> (GC_fo_entries > (last_fo_entries + 500) && (last_bytes_finalized |
> GC_bytes_finalized) != 0)
> Removing this case gets me to a state very close to this patch, both in
> terms of performance and memory usage.
>
> If I understand the approach correctly, GC_fo_entries counts the number
> of outstanding finalizations. It seems like Guile is making good use of
> this functionality, so GC tries to reclaim that memory first instead of
> just expanding the heap. Please note that the number 500 is hard-coded,
> so we have no means to influence it via environment variables or APIs. I
> guess something like that could be added, but it would again take some
> time to bubble through all distributions that we care about.
>

Aha! That explains why we see such poor performance, because we rely on
finalizers a lot. If we were to move away from finalizers and let bdwgc
handle all of the memory management, then that it would also become a
non-issue?

TL;DR: For now it's probably easiest to scale the heap as this patch
> does.




>
> https://codereview.appspot.com/561390043/diff/567180043/lily/smobs.cc#newcode23
> lily/smobs.cc:23
> <https://codereview.appspot.com/561390043/diff/567180043/lily/smobs.cc#newcode23lily/smobs.cc:23>:
> #include <gc/gc.h>
> You probably want this to be guarded by GUILEV2?
>
>
yep.


-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen