Re: Multiple versions of Org in load-path problem

[Sharon Kimble]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Tim Cross <theophilusx@gmail.com> writes:

> David Masterson <dsmasterson@gmail.com> writes:
>
>> Tim Cross <theophilusx@gmail.com> writes:
>>
>>> David Masterson <dsmasterson@gmail.com> writes:
>>>
>>>> "Michel Schinz" <michel@schi.nz> writes:
>>>>
>>>>> Just for the record, I also ran into problems when installing Org 9.6
>>>>> using Emacs' package system on top of an older version that came with
>>>>> Emacs. If I tried to install it as usual (M-x list-packages, then
>>>>> install the package from there), I had errors during compilation related
>>>>> to `org-assert-version`, and then if I restarted Emacs, I would get a
>>>>> fatal error in an unrelated package.
>>>>>
>>>>> I managed to solve that problem by:
>>>>> 1. uninstalling Org 9.6 and exiting Emacs,
>>>>> 2. starting Emacs with -q,
>>>>> 3. installing Org 9.6 from there (using M-x list-packages as usual),
>>>>> 4. restarting Emacs.
>>>>
>>>> Interesting!  I tried this (essentially) and it worked for my case.  In
>>>> my case, I had a built-in Org-9.3 and I was trying to use list-packages
>>>> to install Org-9.6. I checked that using -q still added Org-9.3 to
>>>> the
>>>> load-path, but, since Org wasn't loaded, the install via list-packages
>>>> worked.
>>>>
>>>> The question is what's the proper way of doing this without '-q'?

This is what is in my 'init.el' file -

=====
;;; init.el --- sharon's config -*- eval: (read-only-mode 1) -*-
;; Make sure that Git version of Org mode is being loaded instead of the 
built-in version.
;;; (add-to-list 'load-path (expand-file-name 
"/home/boudiccas/.emacs.d/elpa/org-9.5.5"))
(add-to-list 'load-path (expand-file-name 
"/home/boudiccas/.emacs.d/elpa/org-9.6"))
(add-to-list 'load-path (expand-file-name 
"/home/boudiccas/git/org-contrib/lisp"))

(require 'package)
(setq package-enable-at-startup nil)
(package-initialize)

(require 'ob-tangle)
(org-babel-load-file "/home/boudiccas/.emacs.d/config22-2.org")    
=====

And the first thing that is loaded is org-9.6 *before* anything else! And you 
could equally have org-mode loading from a git repository on your hard drive, 
just by changing the filepath.

Super-easy without having to faff about with 'emacs -q' etc.

And as you can see I'm using org-9.6 from elpa, and I didn't move over to it 
until this morning, so far so good.  

Thanks
  Sharon.  
  
>>
>> [...]
>>
>>> I don't think there is any safe way to install an updated version of
>>> org-mode other than
>>>
>>> 1. Use the -q approach outlined above
>>
>> Thinking about it, this only works if Org is in elpa as melpa (etc.) are
>> not added to package-archives.  You'd have to do some handwritten elisp
>> out of *scratch* to setup package-archives if Org-9.6 was still coming
>> out of melpa. That's why this can only be labeled as a hack and not a
>> solution.
>>
>
> Well, yes, if your going to use this technique to load a package which
> is not in the default package archives you would need to add that
> archive first. People who use this technique often just have a
> 'update.el' file which they load/evaluate when starting Emacs with -q. 
>
>
>>> 2. Craft your init.el file such that org functionality is only loaded
>>> when explicitly requested and always update as the first action after
>>> starting emacs.
>>
>> In this case, something happened in package-install when trying to
>> install Org-9.6 with a built-in Org-9.3.  During the compilation check
>> (.el -> .elc) many files failed because the new 'org-assert-version'
>> macro was not defined.  Sort of like, after package-install started
>> working on Org-9.6, org-macs.el (where org-assert-version should be) got
>> loaded *before* the new load-path had been set causing it to load the
>> old one from 9.3.  Thereafter, everything went awry. 
>>
>
> You must have some custom code in your init.el or are using something
> like use-package as package.el doesn't try to install or upgrade
> packages during init by default.
>
> Where people can come undone is when they are using use-package and set
> the :ensure t option. In this case, use-package will not know about the
> bundled version and will attempt to install org from ELPA. If use
> package runs after org has already been loaded (possibly because some
> other package has been loaded which depends on/requires org mode and has
> loaded the bundled version) then things will break because you will end
> up with a mixed version install. This is why I always ensure org is the very
> first use-package in my init.el and it comes before any other code which
> loads or initialises anything.  
>
>>> The first approach is actually the easiest. The second is hard to get
>>> right and very fragile because packages like use-package and more
>>> specifically, other packages with leverage off org functionality, make it
>>> impossible to reliably know exactly when org is loaded.
>>
>> Using ':after" in use-package is supposed to help that, but I'm not sure
>> it is reliable.  Packages are often incomplete about what other packages
>> it depends on.
>>
>
> You cannot rely on :after. The problem is, other packages may require
> org functionality and will load org when they are loaded. This can be
> very subtle as there are a lot of packages which only make very small
> use of org mode, but even that requires that org mode is loaded. 
>
>>> An approach used by many 'canned' distributions is to postpone package
>>> updates. You have a function you run to check for updates which
>>> generates a list of packages to update and writes that list to a
>>> file. Each time emacs is started, it looks for this update list and if
>>> it finds it, it installs packages updates at the very beginning of the
>>> init process (before any of your other init.el code or custom
>>> blocks). The process also looks for org in the list of packages to
>>> update and if it is found, updates it first. 
>>
>> Probably doesn't work in this case as you would need to be able to use
>> package.el suggesting that the load-path has been updated for all
>> built-ins already.  The thing I note is that the load-path has already
>> been updated for built-ins at the beginning of
>> '~/.emacs.d/early-init.el', but the libraries haven't been loaded yet
>> (unless needed). That's okay if the newer version of a package then
>> cleanly replaces all the files in the old version.
>>
>
> It does work. Distributions like Sapcemacs use this approach and do not
> have the mixed versions issue. I've not looked closely at their
> implementation, but they well might do all this work as part of the
> early-init.el file.
>
>
>>> I don't think there is a safe way to load org mode after the init
>>> process i.e. after booting emacs by M-x package-update.
>>
>> Where is package-update called in the boot process of emacs?  I don't
>> see package-update in Emacs v2.7.
>>
>
> Sorry, I wasn't clear.
>
> What I meant is that there is no clean and reliable way of running the
> package update process after emacs has initialised. The package.el API
> has changed, but previously, you would run M-x package-update-packages
> to update installed packages. Now it is M-x package-update-all. With
> many packages, you can run this command and update your installed
> packages. However, more complex packages, like org-mode, cannot handle
> this due to potential mixed version problems - a problem which has
> existed for many years. 
>
> By default, there is no automated package update setting (unless that
> has been added in recent versions). There are some add-on packages which
> can do this and the elisp to make this happen is trivial (but much
> harder to do it and make it robust). 
>
>
>>> I've had good success using straight.el. I had to be careful regarding
>>> how I structured my init.el file (ensuring any straight stuff happens
>>> first and the first use package stanza is for org. The main reason
>>> straight works well for me is that my work flow is to do a M-x
>>> straight-pull-all when I want to update my packages. This does a git
>>> pull for all the sources, but does not do any build/install. This occurs
>>> when I next start Emacs and because I have all the straight stuff at the
>>> start and because org mode is the first straight-use-package, the update
>>> and install happens before any other org functionality is loaded,
>>> avoiding mixed version issues.
>>
>> Where do you get straight.el?  I don't see it in [m]elpa.
>
> No, to use straight.el, you add some code to your init.el which will
> retrieve the current straight.el file from the github
> repository. Straight.el does not use the ELPA/NONGNU ELPA/MELPA archives
> - at least not directly. It does use information from those archives to
> build recipes which it sues to obtain the code. Essentially, straight.el
> clones the package source repositories and builds it locally. It give a
> lot more control and is really great if you want to modify the code
> (basically, it creates a local fork). Of course, with all that extra
> power, there is also additional responsibility placed on the user. You
> may need to tell it which versions you want to use, which ones to 'pin'
> to, which branches or fix merge conflicts when you have made changes and
> want to update from the upstream soruces etc. I've been using it for
> about 12 months and find it really good. YMMV.

- -- 
A taste of linux = http://www.sharons.org.uk
TGmeds = http://www.tgmeds.org.uk
DrugFacts = https://www.drugfacts.org.uk
Debian 11.5, fluxbox 1.3.7, emacs 29.0.50, org 9.6 
-----BEGIN PGP SIGNATURE-----

iQJRBAEBCgA7FiEELSc/6QwVBIYugJDbNoGAGQr4g1sFAmOTWQodHGJvdWRpY2Nh
c0Bza2ltYmxlMDkucGx1cy5jb20ACgkQNoGAGQr4g1sBchAAkByRu8RZ6EV9WkAi
tbYOMCW9+rBQ9Oib2i6pj4XuGC1opjjjgMr84y2+UfgFm1bbcFo6bv1hi33UFvM2
9D8ORzSPW/wcPjGILcwHS6w1hbwkflB8qwRDwL7TxUFuZqCZ3mr9q7aLZR4ZVuf9
3WFzUHd30O61H6rLKrJWRPJgB2zYcUS4pbsiurOTK35sYoiogAshoV4akTNxQefV
TmLFcoHmJvK0M+L/5BhPJblvrG0jkudUyMCYPeAHGxDfDpXiW/aFdYJuibqhk4mm
nblCIe1EFBa5Lxu3m0lTkY7X9I/XtS1cf92gkj7qA9Y/ZYtVfWO54FyDA5sJFOZ2
aHcQrGS2vqoU1HNYv9u0CVj+/QJQbUSGrI60lrwIikOe+Xt+iHVlyx6dmmgKHLzt
YHuZVKwMZxOEJtUjVwHc23+MzDmHc1WbJ8o5yOomFnYivFg/P0GQpY0o9O3CDJCV
yV/ux7qPSPlyQZcJqKVHjrg05y/t76i5CPsiYAeUUQPa8jdU3F5SHxtpmuf9KPgQ
+5EmUYkYodcNyss93YAuPEamKI491nLuWkc+F4t+fi24QS/t3FJaemkfW+UIIBuk
gpg8/fgMkOdrs8E4IEwb7wZkCEHl52J0WypsGv9Of+vcrb5YXdPk7x/dJoHUmADm
SremPednv0xBoZCKg12orlIQzIM=
=cfTm
-----END PGP SIGNATURE-----