Re: Guile Binary Format 0.0

[Dirk Herrmann]

On Sat, 3 Feb 2001, Keisuke Nishida wrote:

> I have commited my initial version of a binary loader.

Thanks you for providing it.  However, I have some comments:

* IMO, storing scm_tc* typecodes to identify objects is problematic, since
smob typecodes are generated dynamically and can change between different
runs of guile, for example if shared libraries are loaded in different
orders.  If we want to avoid this, we will probably have to use type
names.  While this can also break if two different smobs have the same
name, it is less likely.  Maybe there is even a mechanism that goes
beyound smob names...

* A lot of the complexity of the storing/reloading mechanism seems to come
from the fact that your binary format requires to have a list of all
objects before an object is actually being written, because you have your
'meta section'.  If the information from the meta section was placed in
between the object data 'on demand', an initial marking phase was
unnecessary, the same would hold for a final storing phase.

For example, writing out some type could be done in a single function that
has three parts: 

- in the first part all referenced non-immediate objects are
  declared.  For this purpose there exists a function similar to your
  scm_dump_mark function.  This function checks, whether the object that
  is to be declared has been declared before.  If so, then nothing is
  done.  If not, the dumping function creates a hash-table entry where for
  the cell of the object an integer index is created.  Further, to the
  binary output file a tag 'declare non immediate object' is written.
- in the second part, the object data itself is written.  This starts with
  the number of the tag that belongs to this object.  For example, if this
  object was declared as the third object, then the object data starts
  with the information 'now comes the data for the third declared
  object'.  Then, the object type is written, followed by its data.  All
  non-immediate scheme objects are written as references like 'the n-th
  declared object'.
- in the third part, all referenced objects are written.  The function
  that takes care of this has to make sure that no object is written
  twice.

Example:

> scm_bits_t smob_foo;
>
> struct foo {
>   int x, y;
>   SCM bar, baz;
> }
> 
> #define FOO_DATA(x)  ((struct foo *) SCM_SMOB_DATA (x))

SCM
foo_dump (SCM obj, SCM dstate)
{
  /* declared referenced objects and write 'declaration' tags
     into the output file */
  scm_dump_object_declaration (FOO_DATA (obj)->bar, dstate);
  scm_dump_object_declaration (FOO_DATA (obj)->baz, dstate);

  /* write out 'this is the n-th declared object' followed by the
     object's type. */
  scm_dump_object_header (obj, dstate);

  /* write out the object's data.  The scm_dump_object_reference calls
     write something like 'place the n-th declared obj here' */
  scm_dump_word (FOO_DATA (obj)->x, dstate);
  scm_dump_word (FOO_DATA (obj)->y, dstate);
  scm_dump_object_reference (FOO_DATA (obj)->bar, dstate);
  scm_dump_object_reference (FOO_DATA (obj)->baz, dstate);

  /* write out the referenced objects */
  scm_dump_object (FOO_DATA (obj)->bar);
  scm_dump_object (FOO_DATA (obj)->baz);
}

When reading, the mapping between declaration tags and actual object cells
is easily restored.  When an object shall read itself, it _knows_ at which
places it has to expect an object reference.  It then reads the object
reference, tells the undump routine where the actual cell address should
be placed, and temporarily initializes the object with #f:

/* The function scm_undump_object_reference reads an object reference from
the input file.  If it is an immediate, it is returned.  If it is a
reference to a non-immediate that has already been read, it returns the
cell pointer to that object.  Otherwise, it adds the given target address
to a hash table that tells, which memory addresses have to be updated as
soon as the corresponding object's cell address is known, and returns #f.
*/

extern SCM scm_undump_object_reference (SCM *, SCM);

SCM
foo_undump (SCM dstate)
{
  struct foo *p = scm_must_malloc (sizeof (struct foo), "foo_undump_alloc");

  p->x   = scm_undump_word (dstate);
  p->y   = scm_undump_word (dstate);
  p->bar = scm_undump_object_reference (&p->bar, dstate);
  p->baz = scm_undump_object_reference (&p->baz, dstate);

  return SCM_NEWSMOB (scm_tc16_foo, p);
}

Best regards,
Dirk Herrmann