Re: Grow heap aggressively during music interpretation (issue 561390043 by address@hidden)

[Han-Wen Nienhuys]

On Wed, Feb 5, 2020 at 12:33 PM <address@hidden> wrote:

> > Can you do some timings with different values for GC_MAXIMUM_HEAP_SIZE
> , eg.
> >
> >  GC_MAXIMUM_HEAP_SIZE=100M
>
> Not sure if you actually mean GC_MAXIMUM_HEAP_SIZE or rather
> GC_INITIAL_HEAP_SIZE?
>
>
I mean maximum, it sets the total amount of memory. If you set the initial
heap size, it will still grow the heap, but from a larger starting point. I
am wondering

* what the amount of RAM is this score needs as a minimum,
* how much over the minimum you need to be to get decent runtime.

this could give us hints on how aggressive we should scale up the heap.

guile22-experiment w/o the growing added in this patch:
> default: ~2m33s (gc time taken: 153.316484488)
> GC_MAXIMUM_HEAP_SIZE=100M -> "GC Warning: Out of Memory!  Trying to
> continue..." (kind of makes sense because it needs ~900MB of RAM)
> GC_INITIAL_HEAP_SIZE=100M: 2m30s (gc time taken: 2.189005726; can't
> believe this)
> GC_INITIAL_HEAP_SIZE=900M: 0m59s (gc time taken: 8.895736892)
>



> > Also, it would be interesting to GC_PRINT_STATS=1 and see how much
> time is
> > spent in GC, and how much a typical GC runs reclaim across the
> process.
>
> Um, that outputs a lot. Anything in particular?
>

when you see things like:

World-stopped marking took 187 msecs (77 in average)
In-use heap: 51% (40071 KiB pointers + 6357 KiB other)

it means it took 187ms for GC, and then reclaimed 49% of the memory.

If you often see higher percentages, we'll spend more CPU in marking memory
without getting fresh memory.

I tried to run the carver score, but it needs updating and,

 [hanwen@localhost lilypond]$ ./out/bin/convert-ly  carver/*ly
convert-ly (GNU LilyPond) 2.21.0

Traceback (most recent call last):
  File "./out/bin/convert-ly", line 413, in <module>
    main ()
  File "./out/bin/convert-ly", line 387, in main
    f = f.decode (sys.stdin.encoding or "utf-8")
AttributeError: 'str' object has no attribute 'decode'
[hanwen@localhost lilypond]$ python2 ./out/bin/convert-ly  carver/*ly
Traceback (most recent call last):
  File "./out/bin/convert-ly", line 59, in <module>
    import lilylib as ly
  File
"/home/hanwen/vc/lilypond/out/lib/lilypond/current/python/lilylib.py", line
216
    print('command failed:', cmd, file=sys.stderr)
                                      ^
SyntaxError: invalid syntax

Is there still work left for the python3 conversion?

> https://codereview.appspot.com/561390043/
>


-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen