Re: out-of-control GC

[Hans Åberg]

> On 9 Sep 2017, at 21:47, Linas Vepstas <address@hidden> wrote:
> 
> I've been experiencing problems with guile GC for a while now, that I've
> mostly been able to ignore, but that are becoming debilitating, and I need
> help, or need to find some solution.
> 
> Background: I have a C++ library that has guile bindings; it is wrapped up
> as a guile module. It manages a lot of data, anywhere from a few gigabytes,
> to a over a hundred gigabytes.  Almost all this RAM usage should be
> invisible, opaque to guile: so, for example,  while my library is managing
> 5 or 10 GBytes, guile itself is managing maybe 20 or 50 or 100MBytes,
> according to (gc-stats). This is normal and expected. Some of my programs
> have guile use considerably more than this, in the multi-GByte range
> 
> However, the larger the RAM usage, the more frequently guile GC seems to
> run, and the longer it takes, so that by the time my dataset is 100GBytes,
> guile is using maybe 1GBytes, and is spending 99% of its time in GC. My app
> slows to a crawl, and nothing gets done.  I need to get GC under control,
> because it is running pointlessly, because its clear that (almost) nothing
> can be reclaimed, (usually) nothing is being reclaimed.

Does it have to do with this issue [1], that the Boehm GC initially uses a 
small amount of heap. One can use say
  export GC_INITIAL_HEAP_SIZE=50G
on the shell command line before running the program.

1. https://github.com/Macaulay2/M2/issues/500