bug#54097: go importer does not honor multi-module repositories

[raingloom]

On Mon, 21 Feb 2022 23:43:54 +0100
Björn Höfling <bjoern.hoefling@bjoernhoefling.de> wrote:

> Go usually has the 1 repository=1 module convention. However, it
> is also allowed that one repository contains multiple go modules.
> 
> If repository "foo" contains only one module, then versions are tagged
> "v1.2.3".
> 
> However, if the repository "foo" contains modules "bar" and "baz",
> each in a sub-directory of "foo", the versions will be tagged with
> their respective prefix, i.e.:
> 
> foo/v1.2.3
> bar/v4.5.6
> 
> See here:
> 
> https://github.com/golang/go/wiki/Modules#publishing-a-release
> https://github.com/golang/go/wiki/Modules#faqs--multi-module-repositories
> https://stackoverflow.com/questions/64701064/golang-separate-versioning-of-multiple-modules
> 
> However, our go-importer does not honor this. The Google Cloud API
> modules are structured into sub-modules, but our importer searches the
> wrong tag and raises an exception:
> 
> $ ./pre-inst-env guix import go cloud.google.com/go/storage
> URL FOR VERSIONS:
> https://proxy.golang.org/cloud.google.com/go/storage/@v/list
> FETCH_GO_MOD:
> https://proxy.golang.org/cloud.google.com/go/storage/@v/v1.21.0.mod
> Backtrace: In ice-9/boot-9.scm: 1752:10 17 (with-exception-handler _
> _ #:unwind? _ # _) In unknown file:
>           16 (apply-smob/0 #<thunk 7f5b3d24f0c0>)
> In ice-9/boot-9.scm:
>     724:2 15 (call-with-prompt _ _ #<procedure
> default-prompt-handle?>) In ice-9/eval.scm:
>     619:8 14 (_ #(#(#<directory (guile-user) 7f5b3d255c80>)))
> In guix/ui.scm:
>    2209:7 13 (run-guix . _)
>   2172:10 12 (run-guix-command _ . _)
> In guix/scripts/import.scm:
>    124:11 11 (guix-import . _)
> In ice-9/boot-9.scm:
>   1752:10 10 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/scripts/import/go.scm:
>    116:29  9 (_)
> In ice-9/exceptions.scm:
>    406:15  8 (go-module->guix-package* . _)
> In ice-9/boot-9.scm:
>   1752:10  7 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/import/go.scm:
>    525:18  6 (go-module->guix-package "cloud.google.com/go/storage" #
> ?) In guix/git.scm:
>     277:4  5 (update-cached-checkout _ #:ref _ #:recursive? _ # _ # _
> ?) 266:18  4 (resolve _)
> In git/reference.scm:
>      60:8  3 (_ _ _)
> In git/bindings.scm:
>      77:2  2 (raise-git-error _)
> In ice-9/boot-9.scm:
>   1685:16  1 (raise-exception _ #:continuable? _)
>   1683:16  0 (raise-exception _ #:continuable? _)
> 
> ice-9/boot-9.scm:1683:16: In procedure raise-exception:
> Git error: reference 'refs/tags/v1.21.0' not found
> 
> 
> The correct git reference to look for is:
> 
> refs/tags/storage/v1.21.0
> 
> Björn
> 

I think this has been mentioned before, but I really think we should
use Go's built-in tooling to extract this info, instead of
reimplementing it in Guile. There is already some massive ineffeciency
introduced by the importer cloning whole repos, whereas `go get` is
smart enough to only download `go.mod` files.
Just my 2c.