bug#31878: Module autoloading is not thread safe

[Mark H Weaver]

I wrote:

> I thought about how to fix this thread-safety problem a long time ago,
> and came up with a rough outline of a solution.  The idea is that the
> module should not be added to the global module table until the module
> has finished loading.  While the module is being loaded, it would be
> made visible only to the loading thread, and to any other threads
> spawned during the loading process, by adding the module to a local list
> of modules-being-loaded referenced by a fluid variable.  If any other
> threads attempt to access the module, it would not be found in the
> global module table, and thus trigger an auto-load, which would wait for
> the lock to be released before proceeding.

I forgot to mention an important aspect of the proposed auto-load
locking here.  It would not be a global lock, but rather a lock specific
to the module being loaded.  So, I guess we would need a global table of
locks for modules-being-loaded.

Since Guile (unfortunately) allows cyclic module dependencies, we would
need a mechanism to avoid deadlocks in case modules A and B both import
each other, and two threads concurrently attempt to load those modules.

The first idea that comes to mind is to also have a global structure
storing a partial order on the modules currently being loaded.  If,
while module A is being loaded, there's an attempt to auto-load module
B, then an entry (A < B) would added to the partial order.  The partial
order would not allow cycles to be introduced, reporting an error in
that case.  In case a cycle would be introduced when adding (A < B),
then the thread would simply be given access to the partially-loaded
module B, by adding B to its local list of modules-being-loaded.

Comments and suggestions welcome,

        Mark