Re: Guix Docker image inflation

[zimoun]

Dear Stephen, again :-)

On Sun, 31 May 2020 at 20:30, Stephen Scheck <singularsyntax@gmail.com> wrote:

>> No, it is how Docker is designed.  Maybe the terminology "layer" is
>> not the Docker one but when the images are chained, one cannot remove
>> the data of the previous layer of the total image.
>
> I'm not disagreeing with that, but IF any of the store files resulting from 
> `guix pull`
> are ephemeral (i.e. intermediate build results not anchored to a profile) AND 
> guix
> GC worked inside the container, my approach might still work - yes there 
> would be
> image and layers growth but it might be small enough not to care between 
> periodic image
> rebases. But I'm starting to doubt that, or at least it is difficult to 
> quantify with the
> GC issues.

Currently, if I read correctly, your images are chained with something like,

--8<---------------cut here---------------start------------->8---
GUIX_PATH=/root/.config/guix/current/bin
$GUIX_PATH/guix pull --branch=$CI_COMMIT_REF_NAME--fallback
/root/.config/guix/current/bin/guix gc --delete-generations
/root/.config/guix/current/bin/guix gc --collect-garbage
/root/.config/guix/current/bin/guix gc --optimize
docker commit
--8<---------------cut here---------------end--------------->8---

and instead you should do something like

--8<---------------cut here---------------start------------->8---
GUIX_PATH=/root/.config/guix/current/bin
$GUIX_PATH/guix pull --branch=$CI_COMMIT_REF_NAME--fallback
/root/.config/guix/current/bin/guix pull -d
/root/.config/guix/current/bin/guix package -d
/root/.config/guix/current/bin/guix gc
docker commit
docker export | docker import
--8<---------------cut here---------------end--------------->8---

Maybe the explosion of size would be slower.  If you do, please report
here the number after say 12 generations; I am really interesting. ;-)


>> Because if you run Guix outside an Docker container, you will not have
>> the issue.  The main issue is how the Docker "filesystem" is designed.
>
> Actually, there might be another way around this, still avoiding the need for 
> a custom Runner,
> for example mounting /var/guix and /gnu/store into the container instead of 
> belonging to it. If
> done that way, layer accumulation wouldn't be an issue, and maybe GC between 
> layers neither.

Yes, it is one solution.
All the question seems to be:
 - what is the purpose of such Docker image?  Which usage?
 - what infrastructure do you have at hand to build the Docker images?


Thank you for raising all this Docker image production question. :-)

All the best,
simon