Re: out-of-control GC

[Kjetil Matheussen]

> address@hidden:
>

> I've been experiencing problems with guile GC for a while now, that I've
> mostly been able to ignore, but that are becoming debilitating, and I need
> help, or need to find some solution.
>
> Background: I have a C++ library that has guile bindings; it is wrapped up
> as a guile module. It manages a lot of data, anywhere from a few gigabytes,
> to a over a hundred gigabytes.  Almost all this RAM usage should be
> invisible, opaque to guile: so, for example,  while my library is managing
> 5 or 10 GBytes, guile itself is managing maybe 20 or 50 or 100MBytes,
> according to (gc-stats). This is normal and expected. Some of my programs
> have guile use considerably more than this, in the multi-GByte range
>
> However, the larger the RAM usage, the more frequently guile GC seems to
> run, and the longer it takes, so that by the time my dataset is 100GBytes,
> guile is using maybe 1GBytes, and is spending 99% of its time in GC. My app
> slows to a crawl, and nothing gets done.  I need to get GC under control,
> because it is running pointlessly, because its clear that (almost) nothing
> can be reclaimed, (usually) nothing is being reclaimed.
>
>
Hi Lineas,

This is probably not the reason, but could it be that your program has a
lot of global/static data,
or that you are dynamically loading more and more libraries that has
global/static data?
(global/static data are scanned by bdw-gc, unless explicitly prevented from
doing so)

To test this, you can add a "static_roots_callback" before calling GC_INIT
when your
program starts up. Here's an example of a "static_roots_callback":

https://github.com/kmatheussen/radium/blob/master/Qt/Qt_Main.cpp#L3028
(change "#if 0" at line 3094 to "#if 1" to print out info every time you
collect garbage)

It's initialized like this:

int main(){
   GC_register_has_static_roots_callback(gc_has_static_roots_func);
   GC_INIT();
   ...
}


Also, maybe you should ask on the bdw-gc mailing list too. The question is
probably more relevant there if it's a gc tuning issue (which it might not
be though).