help-guix
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: How can I change the Makeconf file of R to not use hardcoded, wrong


From: Moritz Lell
Subject: Re: How can I change the Makeconf file of R to not use hardcoded, wrong include paths when compiling a package with Rcpp
Date: Thu, 16 Jan 2020 01:35:02 +0100
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.3.1

Hi!

thank you for your suggestions... they help me understand the "spirit"
of this program, it is still quite new for me.

While you're right that a manifest of R and all packages is the best
way, I will still need R to compile something from time to time, for
example C++ code from within my project.

I have already found out some knobs in the R installation but I'm not
done yet. I need someone with more experience with C++ linking and
including flags to help me for this to work.

In the [R-Admin
manual](https://cran.r-project.org/doc/manuals/r-release/R-admin.pdf),
B.3.3 (p.53) they explain that they set default flags if certain
environment variables are not set.

The Nix people seemingly had similar problems and they've come up with
some shims where they set more environment variables than we do.

https://github.com/NixOS/nixpkgs/blob/eb65d1dae626f4b149566c4cbccdad7ec24af189/pkgs/applications/science/math/R/default.nix#L64

>From there:

-- begin ---------------------------------------------------------------

preConfigure = ''
    configureFlagsArray=(
      --disable-lto
[...]
      R_SHELL="${stdenv.shell}"
  '' + stdenv.lib.optionalString stdenv.isDarwin ''
      --without-tcltk
      --without-aqua
      --disable-R-framework
      OBJC="clang"
      CPPFLAGS="-isystem ${libcxx}/include/c++/v1"   # <<<----
      LDFLAGS="-L${libcxx}/lib"                      # <<<----
  '' + ''
    )
    echo >>etc/Renviron.in "TCLLIBPATH=${tk}/lib"    # <<<----
    echo >>etc/Renviron.in "TZDIR=${tzdata}/share/zoneinfo"
  '';
-- end -----------------------------------------------------------------


I tried to translate this into R with a patch versus the guix git repo,
commit 40b1cee620a55bf1fc5d8d897ed1ec147b2535c8 that I have attached to
this mail. However, I think I got some include paths wrong because I get
errors when starting R:

--- begin --------------------------------------------------------------

$ LANG=en_US.UTF-8
/gnu/store/3k62212z7ljmqaqa2vf2nfvjn2rw7q15-r-minimal-3.6.2/bin/R

R version 3.6.2 (2019-12-12) -- "Dark and Stormy Night"
[...]
Error: package or namespace load failed for ‘utils’:
 .onLoad failed in loadNamespace() for 'utils', details:
  call: switch(os, Linux = if (file.exists("/etc/os-release")) {
  error: EXPR must be a length 1 vector
Error: package or namespace load failed for ‘stats’:
 .onLoad failed in loadNamespace() for 'utils', details:
  call: switch(os, Linux = if (file.exists("/etc/os-release")) {
  error: EXPR must be a length 1 vector
During startup - Warning messages:
1: package ‘utils’ in options("defaultPackages") was not found
2: package ‘stats’ in options("defaultPackages") was not found
>

--- end ----------------------------------------------------------------




Are the paths that I set in the attached commit correct?


guix describe

  guix 4aea90b
    Repository-URL: https://git.savannah.gnu.org/git/guix.git
    Branch: master
    Commit: 4aea90b1876179aab8d603a42533a6bdf97ccd3c


Best,
Moritz




On 14.01.20 20:00, zimoun wrote:
> Hi Moritz,
> 
> Hum? complicated issue... so it is more questions than concrete help. Sorry.
> 
> Currently, there is no easy path to achieve your goal, if I understand
> correctly.
> But that is an interesting question for improving the tools. :-)
> 
> 
> On Tue, 14 Jan 2020 at 02:51, Moritz Lell <address@hidden> wrote:
> 
>> to reproduce a scientific project, I need a setup with an old R version
>> (3.4.3), together with other, newer libraries. I created a profile, the
>> manifest and package definitions are below. However, when loading the
>> profile I cannot compile R packages with this version:
> 
> How many packages are required by your scientific project?
> Only R?
> 
> 
>> The last lines of the output of `install.packages("Rcpp_1.0.3.tar.gz",
>> repos = NULL)` (I have to manually download/use the miniCRAN package
>> because of SSL errors while downloading via R but that is another question)
> 
> I am not sure that mixing packages installed by Guix and other by R
> itself do help to clean an unreproducible project, IMHO.
> To me, packaging all your needs with Guix seems better. You should end
> up with your own package definitions living in a Git repo (channel)
> and a manifest file describing how to rebuild everything.
> 
> However, the issue is: you use an old R version with recent packages.
> Not usual. ;-)
> 
> The naive solution is to re-write the dependency graph by replacing
> the current r-minimal@3.6.2 with the old one r-minimal@3.4.3. But it
> does not work. Sadly! The implicit inputs cannot be re-written on the
> fly with the '--with-inputs' option.
> 
> 
> Now, I list some steps that you have already done just in case others
> want to reproduce and/or help. :-)
> 
> (Note the Git commit commands are launched where the Git repo was
> cloned. And the Guix command are launched in /tmp/t.)
> 
> 
> 1. Find the version 3.4.3 in Guix:
> 
>      git log --oneline | grep r-minimal
> 
> --8<---------------cut here---------------start------------->8---
> d22546cf80 gnu: r-minimal: Update to 3.5.0.
> 3146aab7a8 gnu: r-minimal: Add tcl/tk support.
> c43ea99b39 gnu: r-minimal: Update to 3.4.4.
> a768e41523 gnu: r-minimal, r: Update to 3.4.4.
> 9a7b578463 gnu: r-minimal: Add bash to inputs.
> 0fa4702c04 gnu: r-minimal: Respect SOURCE_DATE_EPOCH in srcfile.
> cbe1314a7e gnu: Update r-minimal to 3.4.3.
> 1344deb937 gnu: r-minimal: Update to 3.4.2.
> ebbb6301a3 gnu: r-minimal: Update to 3.4.1.
> 174fbd5f3b gnu: r-minimal: Remove timestamp from man page.
> bd3a184613 gnu: r-minimal: Do not compress serialized files.
> a8cd352304 gnu: r-minimal: Work around failure to embed reference to "which".
> 269504a797 gnu: python-pbr-minimal: Update to 3.0.1
> a71d769d1e gnu: r-minimal: Update to 3.4.0.
> 60c9190e21 gnu: r-minimal: Fix remaining reproducibility problems.
> 2d7c4ae3ee gnu: r: Rename to r-minimal.
> --8<---------------cut here---------------end--------------->8---
> 
> 
> Well, we need any commit before the "Update to 3.4.4." (a768e41523),
> so for example the parent:
> 
>   git log --pretty=%P -n 1 a768e41523
> 
> is the commit: 38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd.
> 
> 
> 2. Try the time machine with the commit
> 38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd.
> Possible resources consuming... everything will be compiled as it was
> in Guix at the time (March 2018) so some substitutes are not available
> any more.
> 
>    guix time-machine --commit=38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd \
>       -- environment --container --ad-hoc r-minimal -- R --version
> 
> 
> 3. In the meantime, let compile the version 3.4.3 with the current
> Guix (commit 2aeca24).
> 
>   git show 
> 38ef437b0c08d8568d6cb1cb0ff1f5c9a66336dd:gnu/packages/statistics.scm
> \
>            > /tmp/t/old-statistics.scm
>   # edit the module name
>   guix build -L . r-minimal@3.4.3
> 
> and the output is: 
> /gnu/store/3wagrrl9gzb5p1an9i5xbm9kp3km4zim-r-minimal-3.4.3.
> 
> 
> Compared to your output
> gnu/store/d64gfblqc5b9gmwj3rbl8nx3zwbf973b-r-minimal-3.4.3, one
> difference is the tzdata package which is different, I guess.
> 
> 
> 4. Speaking about reproducibility, all the toolchain matters. :-)
> 
>   guix time-machine --commit=14c272d5ce --  build -L . r-minimal@3.4.3
> 
> outputs /gnu/store/n0py7ixdqkim1lx9iis6xlr2mn7l897f-r-minimal-3.4.3.
> 
> 
> So for example compare:
> 
>     guix time-machine --commit=2aeca24 \
>       -- environment --ad-hoc gcc-toolchain -- gcc --version
> 
>     guix time-machine --commit=14c272d5ce \
>       -- environment --ad-hoc gcc-toolchain -- gcc --version
> 
> And let aside the grafting system. ;-)
> 
> 
> To be concrete, compare
> 
>   # GCC 8.3.0
>   guix time-machine --commit=14c272d5ce --  build -L . r-minimal@3.4.3
> 
> # -> /gnu/store/n0py7ixdqkim1lx9iis6xlr2mn7l897f-r-minimal-3.4.3
> 
>   guix time-machine --commit=14c272d5ce \
>     --  environment --container -L . --ad-hoc r-minimal@3.4.3 -- R --version
> 
> 
>   # GCC 9.2.0
>   guix time-machine --commit=2aeca24 --  build -L . r-minimal@3.4.3
> 
> # -> /gnu/store/3wagrrl9gzb5p1an9i5xbm9kp3km4zim-r-minimal-3.4.3
> 
>   guix time-machine --commit=2aeca24 \
>     --  environment --container -L . --ad-hoc r-minimal@3.4.3 -- R --version
> 
> 
> Well, my point is just to be clear about what needs to be fixed to end
> up with something reproducible.
> 
> 
> 
>> The problem lies in the file
>> /gnu/store/d64gfblqc5b9gmwj3rbl8nx3zwbf973b-r-minimal-3.4.3/etc/Makeconf
> 
> You mean:
> /gnu/store/d64gfblqc5b9gmwj3rbl8nx3zwbf973b-r-minimal-3.4.3/lib/R/etc/Makeconf
> ?
> 
> 
>> CPPFLAGS = -I/usr/local/include
> 
> Hum? need more investigations... I have no idea...
> 
> 
>> CPPFLAGS is set to the wrong, hardcoded value, but for example FLIBS is
>> set correctly. How can I install R such that CPPFLAGS refers to the
>> /lib/include directory of my profile or doesn't refer to anything?
> 
> It appears to me odd that it is hard coded...
> 
> 
>> I can define the PKG_CXXFLAGS environment variable that can be used to
>> insert additional header locations for R compilation, but unfortunately
>> they are inserted after the R default locations. But I can use it if the
>> R default is set to nothing. I've also tested setting CPLUS_INCLUDE_PATH
>> and PKG_CPPFLAGS, to no avail.
> 
> Instead, I propose to overwite the r-rcpp package by adding an
> explicit reference to r-minimal and so to be able to use the option
> '--with-input'.
> 
> Something like:
> 
> --8<---------------cut here---------------start------------->8---
> (define-module (explicit-pkgs)
>   #:use-module ((guix licenses) #:prefix license:)
>   #:use-module (guix packages)
>   #:use-module ((gnu packages cran) #:select (r-rcpp))
>   #:use-module (old-statistics))        ;need the symbol r-minimal
> ;;; #:use-module (gnu packages statistics) ;or here
> 
> 
> (define-public r-rcpp-explicit
>   (package
>     (inherit r-rcpp)
>     (version "mine")
>     (inputs
>      `(("r-minimal" ,r-minimal)))))
> --8<---------------cut here---------------end--------------->8---
> 
> Then,
> 
>     guix environment --container \
>        -L . --with-input=r-minimal=r-minimal@3.4.3 \
>       --ad-hoc r-minimal r-rcpp@mine \
>       -- R
> 
> should spawn a R prompt using the version 3.4.3 with Rcpp inside.
> 
> But it is not working properly:
> 
> --8<---------------cut here---------------start------------->8---
>> library(Rcpp)
> Error in readRDS(pfile) :
>   cannot read workspace version 3 written by R 3.6.2; need R 3.5.0 or newer
> --8<---------------cut here---------------end--------------->8---
> 
> I do not know why... need more investigations. :-)
> 
> 
> I hope the strategy makes sense.
> 
> 
> Well,
>  a) I do not know if it the good strategy; it is what I am thinking of;
>  b) something easier is maybe possible;
>  c) today, there is a discussion on IRC to see how it should be
> possible to add "something" for changing the R version (e.g., see how
> Python packages do with 2 vs 3).
> 
> What do you think?
> 
> 
> All the best,
> simon
> 

Attachment: 0001-gnu-r-minimal-Prevent-absolute-paths-in-lib-R-etc-Ma.patch
Description: Text Data


reply via email to

[Prev in Thread] Current Thread [Next in Thread]