Re: Bug? Importing (gnu rest ...) fails due to lack of patch files on build side.

[David Dashyan]

Hello, Maxime!

> No, it is not supposed to work!  Only guix/build/... and
> gnu/build/utils/...  (and maybe others I forgot) are supposed to be
> imported, so source-module-closure excludes other modules.  The reason
> is that gnu/packages/*.scm are supposed to be changeable without
> causing any change in derivation hash (and therefore resulting in
> rebuilds).
Ah, great! I'm glad I got definitive answer to this question.  But I
don't get the last sentence.  Changeable when?  You mean that by
including (gnu package ...) in inputs of a derivation change in
gnu/package/*scm files will cause change of derivation hash?  But why it
is a problem for service derivations?  As I understand it won't cause
change in other package derivations hashes propagating down the graph.
Service derivation sits on the leaf of dependency graph.  Or I missing
something?

>> Some context to why am I doing it in the first place.  I found this
>> out while I was implementing "guix-cloud-init" service.
>
> Instead of importing (gnu packages ...), you could use
> (with-extensions (list guix) ...), where guix is the guix package.
> Then there's no need for with-imported-modules IIUC.  That package
> isn't always the newest though (it is primarily used for the
> guix-daemon, for cuirass, emacs-guix and some others), so consider
> using an inferior or something.
Thanks! I'll try it.  I havent thought of that.


>> Some context to why am I doing it in the first place.  I found this
>> out while I was implementing "guix-cloud-init" service.  The service
>> to simplify guix deployment to digialocean, aws and others.
>> Cloud-init is widely used way of configuring cloud vm instances and
>> it works by providing metadata via mounted storage or link-local
>> address http endpoint.  This includes ip addresses, devices, ssh keys
>> etc...  So given you have a guix image with guix-cloud-init service
>> you can upload to your cloud and it will "just work". :)
>
> Do you know about "guix deploy"?
>
Yes.  I think ssh deploy is great, but I don't really like how
digitalocean deployment works, particularly the idea of mutating debian
instance by running big chunk of shell code and leaving all the debian
stuff laying around.  Especially when you have all the info needed for
system configuration provided to you.  I think creating with `guix
system image`, uploading it to digitalocean/aws/etc and only then use
`guix deploy` is much cleaner and robust way.  There are also use cases
when guix deploy just won't work.  E.g. vm instance might be teleported
to other location with other network, ssh keys, network devices.  By
using metadata services instance can automatically reconfigure itself
after migration.

example metadata structure:

(define %test-metadata-json-string
  "{
  \"droplet_id\": 213129999,
  \"hostname\": \"test-test\",
  \"vendor_data\": \"Content-Type: loadsof/randomstuff\"
  \"public_keys\": [
    \"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCUnbO+g3Gc8zaxCLfpVegABaCyrKQwq3l7D\
asMqcVXdFFdDkaR26RfZkS13T0XQuxeM+2o7HGwFebkts56vWe/FLDMdNjAvViNo8hlX6bUaR+IPRf3\
KJSyy0aS7Ht7qtWHdDcuxtaDS8YebOw3GE9AG1P+nQMVRR12TesJnuq632gLdv/UlzIH32Qk7rMR3/9\
oDIbO24kFx9dTU/5Mdj2b+GQzfstux3uLMvlNxztRLTqFNSwQcc09vMAS0WZUkB4trT/WlkoDka/L46\
7k5gA0KjHtIprnP73oKlMBLu7NFq+UANUqncUzGbb99XbXQDFQYRiDdYCMX6wrW8mRL+Cz\"
  ],
  \"auth_key\": \"d873f96dbd087d05a4c5490379a925b5\",
  \"region\": \"fra1\",
  \"interfaces\": {
    \"private\": [
      {
        \"ipv4\": {
          \"ip_address\": \"10.114.0.2\",
          \"netmask\": \"255.255.240.0\",
          \"gateway\": \"0.0.0.0\"
        },
        \"mac\": \"de:83:28:c9:fe:92\",
        \"type\": \"private\"
      }
    ],
    \"public\": [
      {
        \"ipv4\": {
          \"ip_address\": \"207.188.191.111\",
          \"netmask\": \"255.255.240.0\",
          \"gateway\": \"207.155.240.1\"
        },
        \"ipv6\": {
          \"ip_address\": \"2848:F0F:5171:2F45:58B5:3A88:6062:AB3F\",
          \"CIDR\": 64,
          \"gateway\": \"2A03:B0C0:0003:00D0:0000:0000:0000:0001\"
        },
        \"anchor_ipv4\": {
          \"ip_address\": \"10.19.0.5\",
          \"netmask\": \"255.255.0.0\",
          \"gateway\": \"10.19.0.1\"
        },
        \"mac\": \"12:BE:48:43:D9:0D\",
        \"type\": \"public\"
      }
    ]
  },
  \"floating_ip\": {
    \"ipv4\": {
      \"active\": false
    }
  },
  \"dns\": {
    \"nameservers\": [
        \"67.207.67.2\",
        \"67.207.67.3\"
    ]
  },
  \"tags\": [
    \"some\",
    \"test\",
    \"tags\"
  ],
  \"features\": {
    \"dhcp_enabled\": false
  },
  \"modify_index\": 88992338
}")

It usually either provided on 169.244.169.244/metadata/v1.json lladdr
endpoint and/or inside mounted /dev/vdb storage.

Plus cloud-init is kind of standard nowadays. So it will provide
integration with other "clouds" for free.

>> So my thinking was that I will write shepherd-root-service extension
>> that will query metadata import (gnu services ...) modules and run them
>> with values gotten from metadata.
> Not sure what you're trying to do there. ‘query metadata import’ and
> ‘run them with values from metadata’ is a bit vague.

Sorry forgot the comma here. It should have been 
>> "..will query metadata#{,}# import (gnu services ...) modules and..."

>> To be honest it feels a bit hackish
>> to me, but I didn't find any other ways to make such service and reuse
>> existing code.
>> 
>> Comments are much appreciated!
>> 
>> P.S. Side note to macrologysts out there...
>> If I embed select? into with-imported-modules form like so:
>> 
>> (with-imported-modules
>>     `(((guix config) => ,(make-config.scm))
>>       ,@(source-module-closure
>>          '((gnu packages base))
>>          #:select? (match-lambda
>>                      (('guix  'config)   #f)
>>                      (('guix  rest ...)  #t)
>>                      (('gnu   rest ...)  #t)
>>                      (_ #f))))
>>   #~(pk 'hello))
>
> I wonder what would happen if you replace #t with
> (and (list rest ...) #t)?  
Same result...

-- 
David aka zzappie