Re: Hosting a GuixSD server on commodity hosting platforms, a journey

[Christopher Allan Webber]

Christopher Allan Webber writes:

> Relatedly!  User dvc in #guix on freenode suggests looking at
> https://www.vultr.com/ which looks quite affordable and hey!  It has a
> "custom ISO" option.  If we can convert our USB boot stick thingy
> (presumably via xorriso) we could try generating a base server image
> from there.  I'd prefer to have a workflow where I go from handing off
> something made with "guix system vm-image" to some API, but maybe in the
> meanwhile Vultr would be a lower barrier to entry.

I decided to explore this a bit more today, figuring converting Guix's
USB image to .iso couldn't be that hard.  Well, turns out it's another
rabbit hole, and I didn't reach the end of this one yet either!  But I
did make some progress, and thought I'd document what I did.  (Probably
the loopback step can be skipped if we get to making a proper Guix
command out of this using some gexp.  But I decided starting with the
USB image was the path of least resistance.)

Let's start up a little environment:

  $ guix environment --ad-hoc xorriso parted

So the key tool here is xorriso, a rather featureful program for ISO
9960 filesystems (read: data CD-ROMs).  But before we use that, we need
to fetch the usb image (or generate it) and mount it.  Assume you've got
the USB image handy (and uncompressed); then we want to mount it via a
loopback device, but we need to get the right offset to find out what
byte the root partition starts from.  Let's find out where that is.
  
  # We need to find out how many bytes the offset is
  address@hidden:/tmp$ parted guixsd-usb-install-0.11.0.x86_64-linux 
  # This switches it to bytes output
  (parted) unit B
  (parted) print
  Model:  (file)
  Disk /tmp/guixsd-usb-install-0.11.0.x86_64-linux: 943718400B
  Sector size (logical/physical): 512B/512B
  Partition Table: msdos
  Disk Flags: 
  
  Number  Start     End         Size        Type     File system  Flags
   1      1048576B  934281727B  933233152B  primary  ext4         boot
  
  (parted) quit

Ok, so we can take that "start" number (be sure to strip the trailing
'B'):

  $ sudo mount -o ro,loop,offset=1048576 guixsd-usb-install-0.11.0.x86_64-linux 
/mnt/tmp

Now we can write it to the .iso image, packaging up all those files that
appear in the usb image:

  # flags: verbose, Rock Ridge filesystem info, output path, input directory
  $ sudo xorrisofs -v -R -o ./guixsd-usb-install-0.11.0.x86_64-linux.iso 
/mnt/tmp/
  GNU xorriso 1.4.6 : RockRidge filesystem manipulator, libburnia project.
  
  Drive current: -outdev 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso'
  Media current: stdio file, overwriteable
  Media status : is blank
  Media summary: 0 sessions, 0 data blocks, 0 data,  242g free
  xorriso : UPDATE : 27100 files added in 1 seconds
  xorriso : UPDATE : 47400 files added in 2 seconds
  xorriso : UPDATE : 50200 files added in 3 seconds
  [.....]
  Written to medium : 351969 sectors at LBA 0
  Writing to 'stdio:./guixsd-usb-install-0.11.0.x86_64-linux.iso' completed 
successfully.

Except, oops!  This won't boot.  We need to put GRUB on it.

Well here's where I'm stuck for tonight.  I don't know exactly what's
needed; maybe either the -b flag, or --grub2-mbr, or some combination.
The man page is a little bit overwhelming (I mean, xorrisofs is clearly
featureful, credit there):

  https://www.gnu.org/software/xorriso/man_1_xorrisofs.html

But how do I generate the right GRUB stuff to put there?  Can I pull it
off the USB image?  Generate it separately?

This web page is very long but appears to have the appropriate info (and
unfortunately requires running arbitrary javascript to even render):

  http://lukeluo.blogspot.com/2013/06/grub-how-to-2-make-boot-able-iso-with.html

... so I guess the next steps are following roughly what's described at
the bottom of the page?

I wonder how other distros do this step.

The light at the end of this particular rabbit hole feels close, but not
there yet!  As always, external input from those who are more
knowledgable: more than welcome.

 - Chris