Re: Opportunistic GC

[Eli Zaretskii]

> From: Pip Cet <pipcet@gmail.com>
> Date: Wed, 10 Mar 2021 20:21:16 +0000
> Cc: Stefan Monnier <monnier@iro.umontreal.ca>, emacs-devel@gnu.org
> 
> > > > What happens when, on a 8GB machine, we have an Emacs session with a
> > > > 5GB memory footprint, and you fork and start marking (which triggers
> > > > copy-on-write)?
> > >
> > > That is, indeed, one of the cases in which we have to use the
> > > synchronous GC or suffer the consequences.
> >
> > How do we know if/when we are in that case?  Because if we err, we
> > have a meeting with the OOM killer and a swift death.
> 
> Yes, but there's a good chance we would have had that anyway, since
> it's "only" a factor of two, at most.

I don't see how that helps to get out of the conundrum.  The
synchronous GC doesn't enlarge the memory footprint of the Emacs
process, so if we had enough memory when we started GC, there are good
chances we will have it throughout the GC.  By contrast, when you
fork, you enlarge the memory pressure in the system, and the amount of
the increment is unknown in advance (because it's determined by the
degree of nesting of the Lisp data you have in the session at that
moment).  You could enlarge the memory enough to cause OOM killer to
kick in.

So it's important to know when not to attempt the fork.  What is the
algorithm for that?  If you want to use the fraction of the memory the
Emacs process takes, and set the threshold at 50%, then we now need to
discuss how to know that fraction reliably, what with some of the
memory coming from sbrk and some from mmap -- this is why we all but
eliminated memory-limit as not being very useful, and you now want to
use the same technique to base a critical decision.

> If you're arguing that there is a strong case that synchronous GC
> should remain the default, I'm inclined to agree

No, I'm not arguing that yet, not until I see the implementation.  I'm
just thinking aloud about some pesky problems we'd need to solve on
the way.

> > > > Also, I quite frequently need to run Emacs on a system where I'm
> > > > forbidden to run more than 2 processes simultaneously ("make -j3"
> > > > aborts with an error message), and you propose to take those two
> > > > processes with 2 copies of Emacs?
> > >
> > > That is, indeed, one of the cases in which we have to use synchronous GC.
> >
> > Again, how do we know we are in that case?
> 
> I assume you don't have a reliable getrlimit that lets you know?

I don't know -- does getrlimit always report this stuff, with all the
virtualization that goes on nowadays?