Re: Help packaging R Quarto Cli

[Sébastien Rey-Coyrehourcq]

Wojtek Kosior <koszko@koszko.org> writes:


> Hello again
>
> I am impressed with your determination :)
>

thx :D

> I see most packages in Guix that use `(method git-fetch)` also use
> `(file-name (git-file-name name version))` in `(origin)`.
>
> Note that this only affects the name under which the git checkout is
> saved. Whether the source checkout directory under `/gnu/store` is
> named `8479xfpn9hp2b3kc9d3596kpncan9d8w-rust-deno-test-util-0.1.0.tar.gz/`
> or something else should not be relevant to the build.
>

Yes, you’re right i correct this thing, but this is weird in all case, here i 
clone all deno repository (at tag v1.25.2), and this is renamed as 
`deno-test-util-checkout-0-1-0' because i’m only interested by this folder

<https://git.sr.ht/~reyman/rust-channel/commit/00e9ef93bbe5a853b8a196305b6d86451c036156>

I found only one crate that use this method `git-fetch' in the list :
<https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/crates-io.scm#n1576>

So i suppose this is possible to reuse as a dependency in another build…

This is an intuition but i think i have a problem of ordering in my way to do 
things :

When i read the `cargo-build-system' sources here ( 
<https://github.com/ecbrown/guix/blob/master/guix/build/cargo-build-system.scm#L76>
 ) i see that the build input wait for an /gnu/store link that contain a 
`.crate' or the equivalent as `tar.gz'

Or in my case, the /gnu/store link called by cargo contain only sources, 
because the `deno-util' need to be built before creating .crate into the 
gnu/store. Perhaps i could cheating the system by providing myself a 
`gnu/store/xxxx.tar.gz' that contain this sources.

Inside my `rust-deno-test-util-1-0-0.scm` file i could  `(invoke tar ...)' 
command to generate a tar.gz at the root of the this folder. Do you think that 
could work ?

Hum, not sure … the real problem is the fact that guix don’t build the 
derivation completely? (creating and storing a .crate for deno-test-util-0-1-0 
into /gnu/store) when i run build of main rust-deno.scm … only the source are 
present, and not the package resulting from the build of this derivation.

Another way is to first compile deno-test-util.scm, that install the 
corresponding crate into /gnu/store, and after that i give this local path to 
the inputs of my main rust-deno.scm . But i don’t know how to give this path to 
my cargo build system input . How do you reference/call/reuse a package that 
already exist in your /gnu/store in a .scm file ?

> Although I doubt I will be able to help with no knowledge of Rust, I’ll
> at least suggest that you attach the full rust-deno build logs here.
> This will perhaps at least enable others to help (the time needed to
> build it all does scare away!).
>
> Btw, it seems awkward to be required to provide the test_util package
> as a separate crate. I’d rather imagine Deno devs somehow using the
> test_util sources already in the deno tree.
>

Yeah, but now it’s packaged and the package compile, i try one more time like 
that :D

> Have you tried fiddling around inside the build container? If not, you
> may want to look at this manual node[1]. Perhaps after running some of
> the build commands manually you’ll get better ideas?

Thanks i will look on that
>
> At the end, in the worst case, you could resort to disabling tests for
> the package. Nobody likes this “solution” but sometimes it is the only
> way to go ¯\_(ツ)_/¯

Ahah right, i probably do that if i want to finish in 2022 :D

Best,
Sébastien Rey-Coyrehourcq

>
> Good luck,
> Wojtek
>
> [1] <https://guix.gnu.org/manual/en/html_node/Debugging-Build-Failures.html>
>
> – (sig_start)
> website: <https://koszko.org/koszko.html>
> PGP: <https://koszko.org/key.gpg>
> fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
>
> Meet Kraków saints!           #24: blessed Ludwik Pius Bartosik
> Poznaj świętych krakowskich!  #24: błogosławiony Ludwik Pius Bartosik
> <https://pl.wikipedia.org/wiki/Ludwik_Pius_Bartosik>
> – (sig_end)
>
>
> On Wed, 14 Dec 2022 11:30:16 +0100
> Sébastien Rey-Coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> wrote:
>
>> Hi,
>>
>> After two weeks of suffering and pain with this complex rust packaging, and
>> thanks to people on the libera #guix chat, my “rust packaging adventure” is
>> near ending… at least with deno, i see quarto after that…
>>
>> Everything compile by parts, and i need a final help to merge things.
>>
>> I only have a problem at `testing-phase’, with a local rust package defined 
>> on Deno `/test_util’ folder. This package is not published at all on 
>> crates.io and it’s needed at test-phase…
>>
>> This nested crate package is not detected/compiled by actual 
>> `cargo-build-system’, so i decide to package it myself , as 
>> `rust-deno-test-util-0.1.0’
>>
>> I tested and this crate compile well, with two derivation :
>>
>> • `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/’
>> • `/gnu/store/zp5flzykz7y5n35kqrlryqkynvrvcw3z-rust-deno-test-util-0.1.0.drv’
>>
>> `/gnu/store/ma04jfp0f33kf38cdn66qai60nhqxx7d-rust-deno-test-util-0.1.0/share/cargo/registry/’
>>   contain the test_util-0.1.0.crate
>>
>> So … i added this *.scm* to my *deno/deno-dep* list of package in the 
>> *rust-channel* needed by deno to compile :
>> <https://git.sr.ht/~reyman/rust-channel/tree/543fee4cc3626ae27da5b41a7884cecb71b9dcf8/item/deno-dep/rust-deno-test-util-0-1-0.scm>
>>
>> *My problem is simple?* i added this crate as a dependency of my main 
>> `rust-deno.scm’ :
>> • `#:use-module (deno-dep rust-deno-test-util-0-1-0)’
>> • `#:cargo-development-inputs `(((“rust-deno-test-util”, 
>> rust-deno-test-util-0.1.0) … )’
>>
>> BUT BUT BUT … rust-deno-test-util is not found and not added to 
>> `/guix-vendor’
>> during deno crate retrieval, so there is a problem somewhere during packaging
>> retrieval from local `/gnu/store’. I found in the log that
>>
>> `/gnu/store/8479xfpn9hp2b3kc9d3596kpncan9d8w-rust-deno-test-util-0.1.0.tar.gz/’
>> contain the sources and not a `tar.gz’ like others crates. I suppose this is
>> part of the problem, but i don’t know how to solve that.
>>
>> See by yourself using :
>>
>> guix time-machine -C channels.scm – build -f rust-deno.scm
>>
>> that return :
>>
>> error: no matching package named `deno_test_util` found
>>
>> **Info to reproduce**
>>
>> • All these package needed by Deno are defined into my-rust channel here : 
>> <https://git.sr.ht/~reyman/rust-channel>
>> • The building and channel scm needed to build deno are here : 
>> <https://git.sr.ht/~reyman/build-deno-guix>
>> • /Final Warning :/ Deno build (outside rust build) take at least 2 hours on 
>> my machine, rusty-v8 is huge and take itself ~30 to 45 minutes.
>>
>> Any *final* help appreciated !
>>
>> Best ,
>> Sébastien Rey-Coyrehourcq
>>
>>
>> Sébastien Rey-Coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> writes:
>>
>> >
>> > Hi,
>> >
>> > Thanks a lot, that helps me to make one more step :)
>> >
>> > I set :
>> >
>> > ┌────
>> > │     (arguments
>> > │         `(#:phases
>> > │            (modify-phases %standard-phases
>> > │          (add-before ’configure ’set-source
>> > │            (lambda _
>> > │              (setenv “V8_FROM_SOURCE” “1”)
>> > │              (setenv “RUST_BACKTRACE” “1”)
>> > │              (setenv “CLANG_BASE_PATH” (getenv “CMAKE_PREFIX_PATH”))
>> > │              #t)))
>> > └────
>> >
>> > I also try C_INCLUDE_PATH
>> >
>> > and native input :
>> >
>> > ┌────
>> > │  (native-inputs (list ninja gn clang-toolchain ccache clang))
>> > └────
>> >
>> > I have now another problem during compilation of v8, i don’t understand 
>> > why “clang” path is not well recognized by the buildscript, test is 
>> > defined here :
>> >
>> > <https://github.com/denoland/rusty_v8/blob/main/build.rs#L517>
>> >
>> > The backtrace :
>> >
>> > error: failed to run custom build command for `v8 v0.49.0`
>> >
>> > Caused by:
>> >   process didn’t exit successfully: 
>> > `/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/v8-bbb68ec56db1d802/build-script-build`
>> >  (exit status: 101)
>> >   — stdout
>> >   cargo:rerun-if-changed=.gn
>> >   cargo:rerun-if-changed=BUILD.gn
>> >   cargo:rerun-if-changed=src/binding.cc
>> >   cargo:rerun-if-env-changed=CCACHE
>> >   cargo:rerun-if-env-changed=CLANG_BASE_PATH
>> >   cargo:rerun-if-env-changed=DENO_TRYBUILD
>> >   cargo:rerun-if-env-changed=DOCS_RS
>> >   cargo:rerun-if-env-changed=GN
>> >   cargo:rerun-if-env-changed=GN_ARGS
>> >   cargo:rerun-if-env-changed=HOST
>> >   cargo:rerun-if-env-changed=NINJA
>> >   cargo:rerun-if-env-changed=OUT_DIR
>> >   cargo:rerun-if-env-changed=RUSTY_V8_ARCHIVE
>> >   cargo:rerun-if-env-changed=RUSTY_V8_MIRROR
>> >   cargo:rerun-if-env-changed=SCCACHE
>> >   cargo:rerun-if-env-changed=V8_FORCE_DEBUG
>> >   cargo:rerun-if-env-changed=V8_FROM_SOURCE
>> >   cargo:rustc-link-lib=static=rusty_v8
>> >   using Chromiums clang
>> >   clang_base_path 
>> > /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/clang
>> >
>> > — stderr
>> > thread ’main’ panicked at ’called `Result::unwrap()` on an `Err` value: Os 
>> > { code: 2, kind: NotFound, message: “No such file or directory” }’, 
>> > /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/guix-vendor/rust-v8-0.49.0.tar.gz/build.rs:537:6
>> > stack backtrace:
>> >    0: rust_begin_unwind
>> >    1: core::panicking::panic_fmt
>> >    2: core::result::unwrap_failed
>> >    3: core::result::Result<T,E>::unwrap
>> >    4: build_script_build::clang_download
>> >    5: build_script_build::build_v8
>> >    6: build_script_build::main
>> >    7: core::ops::function::FnOnce::call_once
>> > note: Some details are omitted, run with `RUST_BACKTRACE=full` for a 
>> > verbose backtrace.
>> >
>> > Best,
>> >
>> >
>> > Wojtek Kosior <koszko@koszko.org> writes:
>> >
>> >>> Hi,
>> >>>
>> >>> You’re both right, seems there is a flag to skip binary downloading and
>> >>> compile the V8 lib.
>> >>>
>> >>> […]
>> >>
>> >> Good to see you found it :)
>> >>
>> >>> So, my packaging friend :), what’s the best way to push an “export
>> >>> V8_FROM_SOURCE=1” or something like that into my rust-deno.scm ?
>> >>>
>> >>> Best,
>> >>>
>> >>> SR
>> >>
>> >> When I first read your question, I did not know the exact function. But
>> >> I knew where to look. So I thought I’d better share my way of learning
>> >> rather than just the solution ;)
>> >>
>> >> I started with
>> >>
>> >>     grep -R ’export’ 
>> >> ~/.config/guix/current/share/guile/site/3.0/gnu/packages/ | less
>> >>
>> >> That showed a lot of results. I noticed a line like this
>> >>
>> >>> /home/urz/.config/guix/current/share/guile/site/3.0/gnu/packages/engineering.scm:
>> >>>              (setenv “HOME” (getcwd)) ;; cmake needs this to export 
>> >>> modules
>> >>
>> >> Thought it might be the thing I was looking for, so I did
>> >>
>> >>     less 
>> >> /home/urz/.config/guix/current/share/guile/site/3.0/gnu/packages/engineering.scm
>> >>
>> >> and navigated to this line. This seems to be it. The (setenv) function.
>> >> Can be used as from a modified packaging phase function (as you can see
>> >> in engineering.scm).
>> >>
>> >> Hope I helped. Good luck once again!
>> >>
>> >> W.
>> >>
>> >> P.S. Make sure you know some ’less’ shortcuts if you’re going to do
>> >>      things this way. It cad speed things up ^^
>> >>
>> >> – (sig_start)
>> >> website: <https://koszko.org/koszko.html>
>> >> PGP: <https://koszko.org/key.gpg>
>> >> fingerprint: E972 7060 E3C5 637C 8A4F  4B42 4BC5 221C 5A79 FD1A
>> >>
>> >> Meet Kraków saints!           #12: saint Jadwiga Andegaweńska
>> >> Poznaj świętych krakowskich!  #12: święta Jadwiga Andegaweńska
>> >> <https://pl.wikipedia.org/wiki/Jadwiga_Andegaweńska>
>> >> – (sig_end)
>> >>
>> >>
>> >> On Wed, 16 Nov 2022 21:38:47 +0100
>> >> Sebastien Rey-Coyrehourcq <sebastien.rey-coyrehourcq@univ-rouen.fr> wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> You’re both right, seems there is a flag to skip binary downloading and
>> >>> compile the V8 lib.
>> >>>
>> >>>  From the githubpage (<https://github.com/denoland/rusty_v8>) : “V8 is
>> >>> very large and takes a long time to compile. Many users will prefer to
>> >>> use a prebuilt version of V8. We publish static libs for every version
>> >>> of rusty v8 on Github <https://github.com/denoland/rusty_v8/releases>.
>> >>>
>> >>> Binaries builds are turned on by default: |cargo build| will initiate a
>> >>> download from github to get the static lib. To disable this build using
>> >>> the |V8_FROM_SOURCE| environmental variable.
>> >>>
>> >>> When making changes to rusty_v8 itself, it should be tested by build
>> >>> from source. The CI always builds from source”
>> >>>
>> >>> So, my packaging friend :), what’s the best way to push an “export
>> >>> V8_FROM_SOURCE=1” or something like that into my rust-deno.scm ?
>> >>>
>> >>> Best,
>> >>>
>> >>> SR
>> >>>
>> >>> Le 15/11/2022 à 08:58, Efraim Flashner a écrit :
>> >>> > On Mon, Nov 14, 2022 at 11:30:47PM +0100, Sébastien Rey-Coyrehourcq 
>> >>> > wrote:
>> >>> >> Hi,
>> >>> >>
>> >>> >> After some day of packaging rust crate, i progress and deno start to
>> >>> >> compile … but after 1min i have this error when cargo start compiling
>> >>> >> *rust-v8-0.49* . Any rust + guix help appreciated.
>> >>> >>
>> >>> >> I push the channel to reproduce the problem here :
>> >>> >>
>> >>> >> The rust scm repo : git.sr.ht:~reyman/rust-channel
>> >>> >> Channel info to put into *channels.scm* 
>> >>> >> :<https://paste.debian.net/1260722>
>> >>> >> The *rust-deno.scm* file to build :<https://paste.debian.net/1260723>
>> >>> >> The command : guix time-machine -C channels.scm – build -f 
>> >>> >> rust-deno.scm
>> >>> >>
>> >>> >> And the rust error :
>> >>> >>
>> >>> >> —
>> >>> >>
>> >>> >> error: failed to run custom build command for `v8 v0.49.0`
>> >>> >>
>> >>> >> Caused by:
>> >>> >>    process didn’t exit successfully: 
>> >>> >> `/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/v8-bbb68ec56db1d802/build-script-build`
>> >>> >>  (exit status: 101)
>> >>> >>    — stdout
>> >>> >>    cargo:rerun-if-changed=.gn
>> >>> >>    cargo:rerun-if-changed=BUILD.gn
>> >>> >>    cargo:rerun-if-changed=src/binding.cc
>> >>> >>    cargo:rerun-if-env-changed=CCACHE
>> >>> >>    cargo:rerun-if-env-changed=CLANG_BASE_PATH
>> >>> >>    cargo:rerun-if-env-changed=DENO_TRYBUILD
>> >>> >>    cargo:rerun-if-env-changed=DOCS_RS
>> >>> >>    cargo:rerun-if-env-changed=GN
>> >>> >>    cargo:rerun-if-env-changed=GN_ARGS
>> >>> >>    cargo:rerun-if-env-changed=HOST
>> >>> >>    cargo:rerun-if-env-changed=NINJA
>> >>> >>    cargo:rerun-if-env-changed=OUT_DIR
>> >>> >>    cargo:rerun-if-env-changed=RUSTY_V8_ARCHIVE
>> >>> >>    cargo:rerun-if-env-changed=RUSTY_V8_MIRROR
>> >>> >>    cargo:rerun-if-env-changed=SCCACHE
>> >>> >>    cargo:rerun-if-env-changed=V8_FORCE_DEBUG
>> >>> >>    cargo:rerun-if-env-changed=V8_FROM_SOURCE
>> >>> >>    cargo:rustc-link-lib=static=rusty_v8
>> >>> >>    download lockfile: 
>> >>> >> “/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/build/lib_download.fslock”
>> >>> >>    static lib 
>> >>> >> URL:<https://github.com/denoland/rusty_v8/releases/download/v0.49.0/librusty_v8_release_x86_64-unknown-linux-gnu.a>
>> >>> >>    
>> >>> >> cargo:rustc-link-search=/tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/target/release/gn_out/obj
>> >>> >>    
>> >>> >> Downloading<https://github.com/denoland/rusty_v8/releases/download/v0.49.0/librusty_v8_release_x86_64-unknown-linux-gnu.a>
>> >>> >>    Python downloader failed, trying with curl.
>> >>> > Looks like you need to patch rust-v8-0.49 to not try to download
>> >>> > librusty_v8_release… but instead you’ll have to build it from source
>> >>> > and let it know where to find it.
>> >>> >
>> >>> >>    — stderr
>> >>> >>    thread ’main’ panicked at ’called `Result::unwrap()` on an `Err` 
>> >>> >> value:
>> >>> >> Os { code: 2, kind: NotFound, message: “No such file or directory” }’,
>> >>> >> /tmp/guix-build-rust-deno-1.25.2.drv-0/deno-1.25.2/guix-vendor/rust-v8-0.49.0.tar.gz/build.rs:405:10
>> >>> >>    note: run with `RUST_BACKTRACE=1` environment variable to display 
>> >>> >> a backtrace
>> >>> >> warning: build failed, waiting for other jobs to finish…
>> >>> >> error: build failed
>> >>> >> error: in phase ’build’: uncaught exception:
>> >>> >> %exception #<&invoke-error program: “cargo” arguments: (“build” 
>> >>> >> “–release”) exit-status: 101 term-signal: #f stop-signal: #f>
>> >>> >> phase `build’ failed after 105.5 seconds
>> >>> >> command “cargo” “build” “–release” failed with status 101
>> >>> >> builder for 
>> >>> >> `/gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv’ 
>> >>> >> failed with exit code 1
>> >>> >> la compilation de 
>> >>> >> /gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv a 
>> >>> >> échoué
>> >>> >> Vous trouverez le journal de compilation dans « 
>> >>> >> /var/log/guix/drvs/g4/m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv.gz
>> >>> >>  ».
>> >>> >> guix build: erreur : build of 
>> >>> >> `/gnu/store/g4m5c558l1q4g1kggzg2v9vkw352nnaj-rust-deno-1.25.2.drv’ 
>> >>> >> failed
>> >>> >
>> >>
>> >>
>> >
>
>

signature.asc
Description: PGP signature