[bug#54539] [PATCH 0/6] Start breaking up import cycles

[Ludovic Courtès]

Maxime Devos <maximedevos@telenet.be> skribis:

> Ludovic Courtès schreef op di 19-04-2022 om 11:17 [+0200]:
>> If you follow the logic, breaking up import cycles would mean, in the
>> end, having one file per package.
>
> Not necessarily, (gnu packages minetest) has multiple packages
> (minetest and some of its mods) but it doesn't cause any cycles (no
> other module, except sort-of (guix build-system minetest), imports it.)
>
> That one appears to be, at least currently, a bit of a special case
> though.

I think so.  All the historical package modules started that way.

>> But would that be enough?  Probably not, because low-level packages
>> are bound to depend on high-level packages—e.g., glibc depends on
>> Python, some other low-level tool might depend on Pandoc (GHC),
>> librsvg depends on Rust, and so on.
>>
>> IOW, since the graph of build dependency really is a graph, and not a
>> tree, there’ll always be import cycles.
>
> The graph of build dependencies (in terms of derivations) is a tree,

It’s a directed acyclic graph (DAG), not a tree.

> the build daemon doesn't allow cyclic derivations.  So I think that by
> letting the module graph be a coarser version of the derivation graph
> but still a tree (except for the bootstrap packages gcc, sed, ... whose
> modules may import each other).

I thought so, but came to the conclusion that it’s hardly feasible in
practice.

>> (guix self), the module that ‘guix pull’ uses, already automatically
>> splits package modules into two groups.  It’s not as modular as we’d
>> like, but it’s a start.  What would be useful is to come up with metrics
>> and tools to reduce the closure of the “guix-packages-base” group.
>> 
>> WDYT?
>
> Maybe:
>
>   a tool that determines a minimal set of (importing module ->
>   imported module tuples) that needs to be lazified to reduce the
>   closure size (in number of modules) in guix-packages-base by N

Currently ‘source-module-closure’ considers #:autoloaded modules as part
of the closure; we could change that though and indeed, that might prove
helpful in this case.

> and:
>
>   extend "guix style" to perform these changes
>
> Maybe the ‘number of imports lazified -> number of modules in guix-
> packages-base’ function has some sweet spot somewhere.

Could be.

> I think it would be easier though to work our way up before going to
> "guix pull" -- first "hello", then "util-linux, then "guile-avahi",
> then "guile-ssh", then "sqlite" ... and only eventually guix itself.
>
> Also, even if the closure of "guix-packages-base" cannot be reduced,
> making it (mostly) a tree would allow splitting the group into multiple
> parts (see ‘Faster "guix pull" by incremental compilation and non-
> circular modules?’).
>
> Alternative:
>
>   * make _all_ package module imports lazy -- #:autoload everything!
>
>     guix-packages-base might then need to be set manually though ...

I don’t know, having spent some time on this, I feel like there’s no
easy solution.  But it could be that using autoloads at least in the
right places would help shrink ‘guix-packages-base’.  Worth a try!

Ludo’.