Re: Grow heap aggressively during music interpretation (issue 561390043 by address@hidden)

[hanwenn]

On 2020/02/01 21:59:14, dak wrote:
> On 2020/02/01 21:23:51, hanwenn wrote:
> > On 2020/02/01 20:59:06, dak wrote:
> > > On 2020/02/01 20:49:09, hanwenn wrote:
> > > > On 2020/02/01 11:00:41, hahnjo wrote:
> > > > >
> > > >
> > >
> >
>
https://codereview.appspot.com/561390043/diff/557260051/lily/include/score-engraver.hh
> > > > > File lily/include/score-engraver.hh (right):
> > > > > 
> > > > >
> > > >
> > >
> >
>
https://codereview.appspot.com/561390043/diff/557260051/lily/include/score-engraver.hh#newcode26
> > > > > lily/include/score-engraver.hh:26: #include <gc.h>
> > > > > This effectively adds a dependency on libgc which you should
search for
> at
> > > > > configure - it is not necessarily installed in a default
location.
> > > > 
> > > > I've added a TODO for now. One complication is that libgc
doesn't come
> with
> > > > pkg-config.
> > > 
> > > It seems like a bad idea to change the bgc parameters behind
Guile's back. 
> > 
> > Can you give a specific reason why? 
> > 
> > >  Does
> > > Guile not have any GC hooks of its own that one could use for
increasing the
> > > heap size or its behavior with regard to requesting memory?
> > 
> > I guess you could allocate a large block of memory, and then
deallocate it
> > again.
> > 
> > The whole idea of GUILE moving to libgc is that it gets out of the
business of
> > trying
> > to do GC. Why would they insert themselves into that game again?
> 
> From the Guile-2.2 manual:
> 
>  -- C Function: void * scm_malloc (size_t SIZE)
>  -- C Function: void * scm_calloc (size_t SIZE)
>  [...]
> 
>      These functions will (indirectly) call
>      ‘scm_gc_register_allocation’.
> 
>  -- C Function: void scm_gc_register_allocation (size_t SIZE)
>      Informs the garbage collector that SIZE bytes have been
allocated,
>      which the collector would otherwise not have known about.
> 
>      In general, Scheme will decide to collect garbage only after some
>      amount of memory has been allocated.  Calling this function will
>      make the Scheme garbage collector know about more allocation, and
>      thus run more often (as appropriate).
> 
>      It is especially important to call this function when large
>      unmanaged allocations, like images, may be freed by small Scheme
>      allocations, like foreign objects.
> 
> So it would appear that Guile does have an interface for the notion of
getting
> in memory pressure, and we incidentally also call those hooks when we
are
> allocating smob structures.

that is a hook, but it's not the one we need. What we want is a way to
make BDW/Guile expand the heap even if it thinks it's not necessary.
scm_register_allocation is meant to increase the GC frequency, but we
want the opposite.

We could do scm_gc_malloc + scm_gc_free, but 

1) it's doing more work than we need

2) there is a risk that the GC library will treat our large allocations
especially (e.g returning to the O/S on GC_free) 

I think asking libgc to expand the heap is the best way to ask for a
larger heap.

https://codereview.appspot.com/561390043/