bug#62720: 29.0.60; Not easy at all to upgrade :core packages like Eglot

[João Távora]

Philip Kaludercic <philipk@posteo.net> writes:

> João Távora <joaotavora@gmail.com> writes:
>>> Philip presented such a safe modification, and we are in the final
>>> stages of discussing its details, before it will be installed.  So
>>> yes, it is possible.
>
> I might have missed a message, what was the last state here?

Eli is talking about an interaction between you two, so you should be
able to figure out.

>> As I've explained to Philip, the big drawback of that -- undoubtedly
>> safe -- modification is that it is not compatible to user's
>> configurations that have a (use-package 'eglot) or a
>> (package-install 'eglot) in them.
>
> Again: Are we sure about this?  After all, the package is installed
> (which I think is the main thing), it just might not have the most
> recent version.  Calling this "not compatible" seems excessive.

I don't think we should look at this in legalese and disguise the fact
that the same form in Emacs 26, 27 and 28 has very different results.
In those versions (use-package 'eglot :ensure t) brings you always the
latest and greatest.  In Emacs 29, supposedly the more advanced version,
it will bring you a different, older, less powerful, and one that might
even be incompatible with the remainder of your configuration if you
aren't aware of this obscure quirk.

Furthermore the problem continues to worsen with time as packages
evolve.

This is incompatibility if there ever was such a thing.  So the "package
'eglot' is already installed" shown to the user, while technically
correct, is completely misleading.

The patch you're preparing, last I looked at it, doesn't solve this.  It
does makes Eglot slightly easier to upgrade, at least if the user is
aware of this quirky situation.  But it still doesn't fix the
use-package case, for example or the non-interactive config case.

Eli said something like this patch could be acceptable for master and
maybe 29.2.  

----------8<----------8<----------8<----------8<----------8<----------8<
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -652,6 +652,9 @@ package--builtins
 name (a symbol) and DESC is a `package--bi-desc' structure.")
 (put 'package--builtins 'risky-local-variable t)
 
+(defvar package--safely-upgradeable-builtins '(eglot use-package)
+  "See bug#62720 for longest docstring ever.")
+
 (defvar package-alist nil
   "Alist of all packages available for activation.
 Each element has the form (PKG . DESCS), where PKG is a package
@@ -2201,14 +2204,19 @@ package-install
      (package--archives-initialize)
      (list (intern (completing-read
                     "Install package: "
+                    (append
                      (delq nil
                            (mapcar (lambda (elt)
                                      (unless (package-installed-p (car elt))
                                        (symbol-name (car elt))))
                                    package-archive-contents))
+                     package--safely-upgradeable-builtins)
                     nil t))
            nil)))
   (package--archives-initialize)
+  (when-let ((desc (and (memq pkg package--safely-upgradeable-builtins)
+                        (cadr (assoc pkg package-archive-contents)))))
+    (setq pkg desc))
   (add-hook 'post-command-hook #'package-menu--post-refresh)
   (let ((name (if (package-desc-p pkg)
                   (package-desc-name pkg)
---------->8---------->8---------->8---------->8---------->8---------->8

1. Node code complexity.  It's 6 lines of trivial code.
2. Absolutely no risk of "silent installation of software that wasn't ever
   installed before"
3. Fixes all the aforementioned issues

Doesn't fix package-update, but that's minor in comparison, I didn't
want to add bloat this enormous patch with 2 more lines.

It is based on a hardcoded list.  Dmitry has suggested other ways to
whitelist.  There are infinite ways, of course.  Other have suggested
blacklists.

João