Re: Guile Binary Format 0.0

[Keisuke Nishida]

At Sat, 3 Feb 2001 13:18:42 +0100 (MET),
Dirk Herrmann wrote:
> 
> > I have commited my initial version of a binary loader.
> 
> Thanks you for providing it.  However, I have some comments:

Thanks for your comment.

> * IMO, storing scm_tc* typecodes to identify objects is problematic, since
> smob typecodes are generated dynamically and can change between different
> runs of guile, for example if shared libraries are loaded in different
> orders.  If we want to avoid this, we will probably have to use type
> names.  While this can also break if two different smobs have the same
> name, it is less likely.  Maybe there is even a mechanism that goes
> beyound smob names...

Oh, right.  I'll use their names and detect name duplication.

> * A lot of the complexity of the storing/reloading mechanism seems to come
> from the fact that your binary format requires to have a list of all
> objects before an object is actually being written, because you have your
> 'meta section'.  If the information from the meta section was placed in
> between the object data 'on demand', an initial marking phase was
> unnecessary, the same would hold for a final storing phase.

This is more or less what I did before.  While it works, my current
approach has advantages that it consumes less disk space and
(possibly) less loading time.

In my approach, I don't need to store a tag at the beginning of each
object.  If the number of objects stored is very large, this may be
significant.  While we have huge disks, I guess less disk space is
good for fast loading.

Moreover, we can avoid load-time dispatch due to the meta section
(It's not "meta", I guess...)  Since objects of the same type are
sequentially placed, we can reconstruct them in a small loop (see
scm_undump_alloc).  I guess this is also good for fast loading.

In any event, with my current "reconstruct objects as loaded"
approach, there is no reason to object these advantages, though
I'm not sure how significant they are.

BTW, there is another dumpping approach ("dump memory image and
malloc at once"), but I guess the current approach is simpler
and is sufficiently good.

> SCM
> foo_dump (SCM obj, SCM dstate)
> {
>   /* declared referenced objects and write 'declaration' tags
>      into the output file */
>   scm_dump_object_declaration (FOO_DATA (obj)->bar, dstate);
>   scm_dump_object_declaration (FOO_DATA (obj)->baz, dstate);
> 
>   /* write out 'this is the n-th declared object' followed by the
>      object's type. */
>   scm_dump_object_header (obj, dstate);
> 
>   /* write out the object's data.  The scm_dump_object_reference calls
>      write something like 'place the n-th declared obj here' */
>   scm_dump_word (FOO_DATA (obj)->x, dstate);
>   scm_dump_word (FOO_DATA (obj)->y, dstate);
>   scm_dump_object_reference (FOO_DATA (obj)->bar, dstate);
>   scm_dump_object_reference (FOO_DATA (obj)->baz, dstate);
> 
>   /* write out the referenced objects */
>   scm_dump_object (FOO_DATA (obj)->bar);
>   scm_dump_object (FOO_DATA (obj)->baz);
> }

The idea behind this is good, I think.  We can probably cache dump
images, reorganize them later on, and write them into a file.  Thus,
we should be able to simply write

SCM
foo_dump (SCM obj, SCM dstate)
{
  struct foo *p = FOO_DATA (obj);
  scm_dump_word (p->x, dstate);
  scm_dump_word (p->y, dstate);
  scm_dump_object (p->bar, dstate);
  scm_dump_object (p->baz, dstate);
}

> SCM
> foo_undump (SCM dstate)
> {
>   struct foo *p = scm_must_malloc (sizeof (struct foo), "foo_undump_alloc");
> 
>   p->x   = scm_undump_word (dstate);
>   p->y   = scm_undump_word (dstate);
>   p->bar = scm_undump_object_reference (&p->bar, dstate);
>   p->baz = scm_undump_object_reference (&p->baz, dstate);
> 
>   return SCM_NEWSMOB (scm_tc16_foo, p);
> }

Looks nice.  I didn't think of that.  In order to maintain symmetry,
probably the following is better?

SCM
foo_undump (SCM dstate)
{
  struct foo *p = scm_must_malloc (sizeof (struct foo), "foo_undump");
  scm_undump_word (&p->x, dstate);
  scm_undump_word (&p->y, dstate);
  scm_undump_object (&p->bar, dstate);
  scm_undump_object (&p->baz, dstate);
  SCM_RETURN_NEWSMOB (scm_tc16_foo, p);
}

Kei