bug#45705: [feature/native-comp] Excessive memory consumption on windows 10

[Eli Zaretskii]

> From: Andrea Corallo <akrl@sdf.org>
> Cc: edouard.debry@gmail.com, 45705@debbugs.gnu.org
> Date: Fri, 08 Jan 2021 22:02:38 +0000
> 
> I've compiled current native-comp with and without --with-nativecomp
> repeating the experiment with and without X.  These are the data-points:
> 
>   |         | --without-x | --without-x --with-nativecomp |      |
>   |---------+-------------+-------------------------------+------|
>   | -Q      | 49M         | 92M                           | 1.9x |
>   | my-conf | 92M         | 179M                          | 1.9x |
>   
>   
>   |         |      | --with-nativecomp |      |
>   |---------+------+-------------------+------|
>   | -Q      | 536M | 756M              | 1.4x |
>   | my-conf | 585M | 1453M             | 2.4x |
> 
> So yes shared are using considerably more memory, I think this is
> expected as also the file footprint suggests native code is less dense
> that byte-code (actually with a quite similar relative results).

Thanks for the data points.

What about memory usage when there's a background compilation of Lisp
going on?  GCC is known to be a memory hog in some cases, so I wonder
what happens in this case with libgccjit.

(Do we allow multiple async compilations, btw? if so, how many
concurrent compilations can be running, and how do we or the user
control that?)

Also, what are the numbers for a session that has been running for
several days?  I understand that it would be hard for you to collect
such numbers about all the configurations, but could you show the
growth of the configuration you are routinely using, which I presume
is --with-x --with-nativecomp and with your config?  As your numbers
above show, it starts at 1.5 GiB, but what is the footprint after a
day or a week?

> Indeed *with use the delta should decay as data are the same and there's
> no difference in its representation*, so this picture should be more on
> the worst case side than on the average.

That's why I asked to see the memory footprint after the session has
run for some time, yes.

> Also if we want to see a positive side, multiple Emacs sessions will
> share the majority the pages allocated for the shared libraries.

Assuming those sessions run the same binary of Emacs, yes.  Otherwise,
only some of them will be shared, the ones that are in common public
directories, and even that only if the running Emacsen are of the same
version.  Most of the shared code is in the pdumper file, btw, which
is outside of the picture for the purposes of this discussion.

Thanks.