Re: GC and stack marking

[Stefan Monnier]

> The short version of the question is: is it possible that a Lisp
> object which is no longer referenced by anything won't be GC'ed
> because it is marked by mark_stack due to some kind of coincidence?

Yes, of course, it's what makes a conservative marking conservative.

> So the huge hash-table gets dumped into the emacs executable, and

That's bad luck, indeed.

> causes all kinds of trouble in the dumped Emacs.

But it shouldn't cause any trouble (other than extra memory use).

> If this can legitimately happen, then how can we make sure this
> hash-table indeed gets GC'd before we dump Emacs?

First we should make sure that even if this table is not GC'd, Emacs
behaves correctly.  Otherwise, we probably have a bug that can appear in
other situations.

As for ensuring that the table gets' GC'd, there are 2 approaches:
- provide a low-level "free-this-table" function which loadup.el could
  use.  This is dangerous, since it basically says "trust me there are
  no other references to this object".  Even implementing this function
  can be tricky; it would probably be easier to provide it as
  a C function only.
- find where the spurious "reference" is coming from and add code to set
  this reference to some other value (e.g. it might be some variable
  left uninitialized, or a dead variable which we could explicitly set
  back to NULL or something), or to mark this memory location as "not
  a pointer" (like GCPRO but reversed: we'd do a NEGATIVE_GCPRO on the
  var (presumably of a type like int or float)).

The Boehm's GC has developed ways to do this second option
automatically: if during a GC, a memory cell is found to "point to"
unallocated memory, then it is assumed to be of non-pointer type and
this fact is recorded somewhere so that if in subsequent GC's this cell
ends up "pointing" to allocated memory that won't be considered as an
actual pointer.  This can be very important when you get close to using
the whole address space, in which case most addresses are allocated, so
that many/most ints and floats end up spuriously pointing "somewhere".

This doesn't work for us, tho, because we don't know when a stack
location is reused for some other purpose (i.e. when it changes type),
and more importantly because we have the Lisp_Object type which is
a memory cell which can sometimes contain integers and
sometimes pointers.  OTOH, we are only conservative w.r.t stack
scanning, so we're only subject to spurious pointers coming from the
stack, not from the rest of the heap.  And furthermore we have the great
advantage that, as an interactive application, our stack regularly comes
back to "almost empty" (and since we do "opportunistic GC" during idle
time, we often GC at the very moment the stack is almost empty).


        Stefan