managing external memory

[Tom Schouten]

hi all,

i'm integrating guile in my 'pdp' project
(http://zwizwa.fartit.com/pd/pdp) and i've run into a problem where i
don't really understand how to properly tame the garbage collector.

basicly, libpdp has it's own reference count and reuse pool based memory
manager which is used to allocate and reuse memory buffers for storing
video image frames. data packets are referenced by a positive integer and
the manager is a global object.

what i want to do with the guile extensions is to have the entire system,
which is implemented as a forth-like script language language, to be
available from scheme in a straightforward way, trading the tight
real-time and low memory usage operation mode for a more convenient gc'd
scheme implementation.


the basic idea is that i don't need the system to behave in a real-time
environment when it is accessed from scheme, but i do want to have garbage
collector to behave properly.

right now, i use this interface:


/* packet wrapper */
typedef struct
{
    int packet;
    SCM update_func;
} pdp_scm_t;

static SCM mark_pdp (SCM pdp_smob)
{
    /* Mark the image's name and update function.  */
    pdp_scm_t *wrapper = (pdp_scm_t *) SCM_SMOB_DATA (pdp_smob);
    scm_gc_mark (wrapper->update_func);
    return SCM_BOOL_F;
}

static size_t free_pdp (SCM pdp_smob)
{
    pdp_scm_t *wrapper = (pdp_scm_t *) SCM_SMOB_DATA (pdp_smob);
    pdp_packet_unregister(wrapper->packet);
    scm_must_free(wrapper);
    return sizeof(pdp_scm_t);
}

scm_t_bits pdp_tag = scm_make_smob_type ("pdp", sizeof(pdp_scm_t));
scm_set_smob_free (pdp_tag, free_pdp);
scm_set_smob_mark (pdp_tag, mark_pdp);



the problem with this approach is that guile doesn't know the real
size of the object, but just the smob size. this can make the system run
out of memory if the gc is not called manually from time to time.

obviously i'm doing something terribly wrong, so i wondered if there is a
way to tell the gc how much 'external memory' a smob refers to, without
changing the way the memory allocation in libpdp works.

maybe it is better to let guile do all the memory management, but my
question then would be: will used memory be moved by the gc? in pdp, it is
required a packet's buffer will not be moved as long as it is registered,
and there's no way i can change that.

maybe someone can shed some light on this?

regards,
tom