Re: packaging go-ethereum, and ultimately bee (of ethswarm.org)

[Bengt Richter]

On +2021-09-03 08:16:00 +0000, Attila Lendvai wrote:
> 
> [with some reorderings]
> 
> > I hope my reply has been valueable. It seems like this discussion
> > is going to end with ‘agreeing to disagree’ and I think I said all
> > I've got to say about the philosophical part of the matter, so I
> > don't think I'll sending further replies on the philosophical parts.
> > But if you have any technical questions, feel free to ask me.
> 
> 
> ok, noted. your feedback is much appreciated, and i don't mean to
> argue against the guix philosopy. i'm converted after all! :)
> 
> so, i'll try to focus on only pointing out perspectives that you may
> not be aware of, and proposing solutions.
> 
> 
> > > this mechanism is part of how go builds are reproducible.
> > > building go-ethereum locally should yield the same binary that they
> > > have released.
> >
> > FWIW, ‘reproducibility’ in guix means that, if the build is repeated on 
> > another
> > machine, with the same architecture and guix version, the binaries will be
> > exactly the same. Bit-for-bit reproducibility isn't guaranteed when the 
> > builds
> > are performed on different distro's.
> 
> 
> thank you for clarifying this! i'll use clearer language when talking
> about reproducibility.
> 
> 
> > the loader to be at /lib/lib(64)/ld-linux.[...].so.[...], but that doesn't 
> > even
> > exist on Guix, so binaries created on another distro cannot be run on guix,
> 
> 
> FTR, the official binary seems to start up fine on my guix, and begins
> syncing with some peers:
> 
> $ ~/.guix-profile/lib/ld-linux-x86-64.so.2 
> ./geth-linux-amd64-1.10.8-26675454/geth --syncmode light
> 
> it's not statically linked:
> 
> $ ldd ./geth-linux-amd64-1.10.8-26675454/geth
>       linux-vdso.so.1 (0x00007fff26da6000)
>       libpthread.so.0 => 
> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/libpthread.so.0 
> (0x00007f86794af000)
>       libm.so.6 => 
> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/libm.so.6 
> (0x00007f867936e000)
>       librt.so.1 => 
> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/librt.so.1 
> (0x00007f8679364000)
>       libc.so.6 => 
> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/libc.so.6 
> (0x00007f86791a7000)
>       /lib64/ld-linux-x86-64.so.2 => 
> /gnu/store/ksy2b6fwfmz40gjajvspl87ia4vsfzj7-glibc-2.31/lib/ld-linux-x86-64.so.2
>  (0x00007f86794d2000)
> 
> will this have any pitfalls? is there anything i'm not aware of?

For the paranoid, from man ldd, red emphasis mine (if it worked)
--8<---------------cut here---------------start------------->8---
   Security
       Be  aware that in some circumstances (e.g., where the program specifies 
an ELF interpreter other than ld-linux.so), some versions of ldd may attempt
       to obtain the dependency information by attempting to directly execute 
the program, which may lead to the execution of whatever code is  defined  in
       the  program's ELF interpreter, and perhaps to execution of the program 
itself.  (In glibc versions before 2.27, the upstream ldd implementation did
       this for example, although most distributions provided a modified 
version that did not.)


       Thus, you should never employ ldd on an untrusted executable, since this 
may result in the execution of arbitrary code.  A  safer  alternative  when
       dealing with untrusted executables is:

           $ objdump -p /path/to/program | grep NEEDED

       Note,  however,  that  this alternative shows only the direct 
dependencies of the executable, while ldd shows the entire dependency tree of 
the exe‐
       cutable.
--8<---------------cut here---------------end--------------->8---

> 
> if not, then i propose adding a package that downloads the official
> release, authenticates it by checking its signature, and sets up a
> wrapper script for it to be able to start it by its proper name.
> 
> so, maybe we should add two packages? one that fetches the official
> release binary, and one that builds it from source (and doesn't
> strictly follow the versions in the project's go.mod file)?
> 
> i, as a user, would be more relaxed running the authenticated official
> binary release. maybe this is misguided, and in that case i'm happy to
> be corrected.
> 
> 
> > > building go-ethereum locally should yield the same binary that they
> > > have released.
> >
> > ... why should it?
> 
> 
> so that, ideally, we are able to use the trusted guix intrastructure
> to rebuild the same binary locally that the project has
> released... and with that double-check also the release process of the
> upstream.
> 
> 
> > > > Another problem is: if a go package has many (transitive) dependencies, 
> > > > how do
> > > > we check that it doesn't contain any malware or non-free components? 
> > > > That needs
> > >
> > > go-ethereum is a software that handles ethereum wallets with several
> > > zeroes. they are probably equally worried about this if not more.
> >
> > I'm sure the go-ethereum peope wouldn't mind if we double-check that no 
> > malware
> > slipt through their review process.
> 
> [...]
> 
> > > for projects like go-ethereum, it's not an option for the packager to
> > > make decisions about the version of any of its dependencies.
> >
> > Why isn't this an option? Choosing different versions from upstream is 
> > already
> > done in guix, see e.g. https://issues.guix.gnu.org/50217 (ok that's not 
> > merged
> > yet, not the best example --- note, editing the version requirement was
> > on my advice, see https://logs.guix.gnu.org/guix/2021-08-26.log).
> 
> 
> extra auditing on top of the official release is certainly welcome by
> each player. but, realisticly, i doubt that the hundreds of guix go
> packages will get comparable amount of security-auditing-attention any
> time soon. let alone -- and this is of key importance here -- auditing
> any possible cross-interactions of altered versions of the
> dependencies! who else is qualified to do that better than the
> developers themselves?
> 
> for a program like the ethereum client, any such semantic surprises
> may be disasterous. (losing money in various ways. e.g. their client
> forking off of the consensus of the ethereum network, and if e.g. they
> are paricipating in staking, then they will suffer penalties for not
> adhering to the rules of the network, etc.)
> 
> this is why i would prefer to have a solution that somehow reproduces,
> compares, and runs the same binary as the officially released one.
> 
> in the follwig days i'll play with reproducing the geth release binary
> on my guix, and also planning to mock up a binary downloader package,
> and see how it goes. i'll report back with anything worth mentioning.
> 
> 
> > I see a third viable option (3): treat the "go.sum" as a mere
> > ‘friendly suggestion’, and just use the latest version when feasible.
> > Again, I don't see much difference with, say, haskell, python, ruby,
> > guile, java ... packages.
> 
> 
> my point here is not that all go projects should be packaged like this
> (i.e. with exacly pinned dependencies), but that applications like
> geth should be, regardless of the language they are written in.
> 
> in case of a random image viewer written in go, i'm all in for a
> relaxed handling of dependencies.
> 
> 
> > Have there been any problems in practice with just using the latest
> > version (updating the version currently in guix where applicable)?
> 
> 
> not that i'm aware of, but the first such identified issue could turn
> out to be very expensive.
> 
> - attila
> 
> 

-- 
Regards,
Bengt Richter