bug#48682: recursive import between (gnu packages chez) and (gnu packages racket)

[Philip McGrath]

Hi, I've been working on Racket packaging. I haven't looked into this 
much yet---hopefully I can tomorrow---but here are a few quick thoughts.

On 5/26/21 2:54 PM, raingloom wrote:
> I also haven't found any other mention of chez-scheme in racket.scm
> other than the line that imports it, which is weird, given that Racket

This was my doing. When I added `(gnu packages racket)` in [1], I 
adapted the patch from a racket.scm I'd been using to experiment ([2], 
also some branches of [3]): the changes that actually use (gnu packages 
chez) aren't ready to go yet, but apparently I left the import line in.


On 5/26/21 2:54 PM, raingloom wrote:
> other than the line that imports it, which is weird, given that Racket
> is now built on Chez, so I'd expect it to use it as an input. I guess
> it's a bundled version? In any case I don't think I should just remove
> the import, because it will be needed eventually, so this issue needs
> to be fixed by then.

Racket uses a fork of Chez Scheme: I described the situation in [4]. 
Racket won't have upstream Chez as an input in the foreseeable, but, 
from a packaging perspective, building Racket's Chez fork uses many of 
the same pieces as building upstream Chez. I think it may make sense to 
refactor out some of the common infrastructure, so both `(gnu packages 
racket)` and `(gnu packages chez)` could depend on some low-level 
module, but I'm not sure what the Guix-preferred way to organize things 
would be. (I'm mostly a Racketeer using Racket packaging to learn more 
about Guix.)

Here are some ways `(gnu packages chez)` and `(gnu packages racket)` are 
intertwined, in no particular order:

  - `racket-minimal` should use the `nanopass` and `stex` origins
     as `chez-scheme` does. (But it would be easiest to do this
     once I've changed `racket-minimal` to build from the Git source,
     rather than the bootstrapped tarball: I hope that will be soon.)
     Some more of my not-quite-current thoughts on that at [5].

  - The `chez-scheme` phases `unpack-nanopass+stex`, `configure`,
    `prepare-stex`, and `install-doc` should be shared with Racket.
    I think it would be better to put them in a build-side module and
    actually share them, rather than to do tricky things to extract
    their s-expression representation from
    `(package-arguments chez-scheme)`. On the other hand, I think a
    build system would be overkill: it would only build vanilla Chez
    and Racket-flavored Chez.

  - Racket-flavored Chez has added some backends that vanilla Chez
    doesn't support (and can't be readily ported, unless upstream Chez
    eventually adopts Racket's changes to generating backends).
    In particular, Racket-flavored Chez adds support for threading on
    ARMv6 and support for aarch64 (which vanilla Chez doesn't support at
    all). I haven't thought at all deeply about this, but it might make
    sense for the default `chez-scheme` package on those architectures
    to be Racket-flavored Chez.

  - We may in fact want to use Racket to bootstrap vanilla Chez.

    Chez has the usual problem with self-hosting compilers that you
    need the old version to compile the new version. Specifically, you
    need "bootfiles", which encapsulate the Scheme-implemented portion
    of Chez, compiled to machine code. Thus, (a) they are platform-
    specific and (b) they need to agree precisely with the C-implemented
    part of Chez on the layout of datatypes and such.

    The vanilla Chez Git repository [6] has bootfiles for i686, x86_64,
    and (non-threaded) ARMv6. Once you've done a native build for one of
    those platforms, you can use it to cross-compile for any platform
    Chez supports (ppc32, various BSDs, etc.). But these are binary
    blobs. From a "trusting trust" perspective, it's especially striking
    if you consider that Chez Scheme was non-free software from 1984 to
    2016, and the first libre release likewise needed the bootfiles of
    its ancestors. (However, building bootfiles is reproducible:
    Chez in fact builds them twice and errors if they aren't
    byte-for-byte identical.)

    But Racket is able to simulate enough of Chez to (slowly)
    compile the Chez compiler and generate bootfiles, providing a path
    to Chez from just a C compiler. Racket does its whole bootstrapping
    process regularly in CI, and I'm working on getting the Guix package
    to do likewise.

    Unfortunately, the Racket fork has diverged enough from vanilla Chez
    that the current Racket "cs-bootstrap" package [7] can't build
    vanilla Chez, but I hope that the solution is just a matter of
    walking through the Git history to find the right older version of
    the bootstrapping package, before the Racket fork's `#!base-rtd`
    gained a vector of ancestors rather than a parent and a few such
    things. But I'd like to be able to build Racket packages with Guix
    before I try that :)

Ok, that got a bit long.

I don't know where the cycle came from with `emacs-geiser-racket`, but I 
think it would be reasonable either to do some refactoring to `(gnu 
packages chez)` and `(gnu packages racket)`, or to just remove the 
import for now to un-break things and figure out the rest later.

-Philip

[1]: https://issues.guix.gnu.org/47829
[2]: https://gitlab.com/philip1/guix-racket-experiment
[3]: https://gitlab.com/philip1/guix-patches/
[4]: https://issues.guix.gnu.org/46865#3
[5]: https://issues.guix.gnu.org/47153#2
[6]: https://github.com/cisco/ChezScheme
[7]: 
https://github.com/racket/racket/tree/master/racket/src/ChezScheme/rktboot