Re: BDW-GC-Guile incompatibilities

[Neil Jerram]

Hi Ludo,

Sorry for my delay in following up here...

address@hidden (Ludovic Courtès) writes:

> Hi,
>
> "Neil Jerram" <address@hidden> writes:
>
>> 2009/1/5 Ludovic Courtès <address@hidden>:
>
>>>  1. The lack of `gc-live-object-stats'.
>>
>> I doubt it's a stopper, but current `gc-live-object-stats' is quite
>> nice.  Doesn't libgc have a lightweight object-collected hook that
>> would allow us to implement this?  Or is it the problem that would be
>> too bad for performance?
>
> We'd need to write something similar to `scm_i_card_statistics ()',
> which basically traverses a card and accumulates live object info in a
> hash table.  The difficulty is that, while Guile's GC is only able to
> scan the cell heap (which is sufficient), BDW-GC doesn't know what a
> cell is, so we'd have to find tricks to somehow not traverse the whole
> heap but only cells.

I was actually thinking of decrementing an object type counter when an
object is collected - and having looked more at the docs I see that
that would mean using GC_register_finalizer ().  However I also see
from [1] that that (having most SCM objects being finalizable) would
be very bad for the effectiveness of the GC - because of the point
that in a chain of finalizable objects, only one object (the head) is
finalized in each collection.

[1] http://www.hpl.hp.com/personal/Hans_Boehm/gc/finalization.html

> I can look into it into more details to see whether/how that can be done
> before we settle on a decision.

OK.

> It'd be a potential source of incompatibility if `gc-live-object-stats'
> behaved differently.  For instance, `gc.test' in `master' uses it to see
> whether a given object was reclaimed, so the same test was rewritten in
> the BDW-GC branch.

Ultimately I think `gc-live-object-stats' can be sacrificed if need
be.  If there are specific application needs for part of the
information that it provides, I would guess that they are actually for
specific object types only and could be met by using a guardian.

>>>  3. Different behavior of weak hash tables, see
>>>     http://lists.gnu.org/archive/html/guile-devel/2008-11/msg00015.html .
>>>     This can be fixed, but I'm unclear whether it's worth it (comments
>>>     welcome!).
>>
>> IIUC, your example to the GC list can be fixed (in the application) by
>> using a (BGC) doubly weak hash table instead.
>
> Yes, but that's a visible change for the application.

Yes, but at least it is a change that we can document, and we can
precisely advise application authors what they need to do if they see
this problem.

>> I wonder what is the simplest example of a current Guile weak-key hash
>> table that can't be mapped to a BGC doubly weak one?
>
> The test that led me to find it is "Unused modules are removed" in
> `gc.test'.  The culprit is `set-module-eval-closure!', which creates a
> circular reference between CLOSURE and MODULE via the procedure property
> weak-key hash table.
>
> Here are the details (from an old message I never posted since I
> understood what happens, in the meantime...):
>
> Basically, `make-module' invokes a record constructor, after which it
> does:
>
>   (set-module-eval-closure! module (standard-eval-closure module))
>
> Note that `standard-eval-closure' returns a fresh SMOB that points to
> MODULE.
>
> In turn, `set-module-eval-closure!' invokes the record setter and then
> does this:
>
>   (set-procedure-property! closure 'module module)
>
> Here, CLOSURE is actually the SMOB returned by `standard-eval-closure'.
> Procedure properties of "real" closures are stored in the closure itself
> (see `SCM_SETPROCPROPS ()').  But for non-closure objects such as subrs
> or SMOBs, an indirection is used: a phony closure is created and
> associated with the subr/SMOB via a weakly-keyed hash table (see
> `scm_stand_in_scm_proc ()').  Still here?

Yes!

> So, in the case of `make-module', the object graph is as follows:
>
>
>     .---------------------------------.
>     |                                 |
>     V                                 |
> .--------.       .--------------.  .----------------------------.
> | module |<----->| eval-closure |  | closure                    |
> `--------'       `--------------'  | (from                      |
>                          ^         |  `scm_stand_in_scm_proc ())|
>                          :         `----------------------------'
>                          :                  ^                          
>                          :                  :                          
>                          :..................'                         
>                          :                    
>                .---------------------.
>                | scm_stand_in_procs  |
>                | (weak-key hash tab.)|
>                `---------------------'
>
> (Dotted arrows denote weak references, while solid arrows denote
> "regular" references.)
>
> In the BDW-GC-based Guile, the reference from the phony closure to the
> module prevents them from being collected.

Thanks for the detailed description.  But this case could also be solved
by making scm_stand_in_procs doubly weak, can't it?

(Also, I wonder if it could be solved by getting rid of the stand-in
thing.  Why does eval-closure need 2 references to module?  Sounds
like it might be a case of design rot.)

Finally, another concern from [1] is "Cycles involving one or more
finalizable objects are never finalized."  Given that, can our
guardians still work?  You mentioned in another email about "possible
guardian glitches", is this what you had in mind?

Regards,
        Neil