bug#48995: 28.0.50; Within eshell, company completion for cd-less directory-changing wipes out all but the most recent subdirectory

[Basil L. Contovounesios]

Brian Leung via "Bug reports for GNU Emacs, the Swiss army knife of text
editors" <bug-gnu-emacs@gnu.org> writes:

> With an init file like
>
>> (require 'company)
>> (global-company-mode 1)
>
> I run
>
>> mkdir ~/apple ~/apple/ball ~/apple/ball/caterpillar ~/apple/ball/catnip
>> emacs -q -l above-init.el
>
> and start Eshell in the home directory. At the Eshell prompt, I type
>
>> apple/ball/cat
>
> and am presented with two candidates for completion in the Company
> popup. Choosing either one will complete with that subdirectory alone
> (for example, "catnip/" instead of something like the desired
> "/home/alice/apple/ball/catnip/").
>
> This issue appears only in Emacs 28; if I perform the same experiment
> with the latest rev of origin/emacs-27, the completion correctly
> completes with the full path.

I suspect this is the same issue as the following, so I'm CCing Stefan.

https://lists.gnu.org/r/emacs-devel/2021-05/msg01220.html
https://lists.gnu.org/r/emacs-devel/2021-06/msg00254.html

I'm not familiar with Company, but in the case of Ivy, the problem boils
down to the following change in behaviour, starting with this change:

* lisp/eshell/em-cmpl.el: Try and fix bug#41423
82c76e3aeb 2021-01-31 19:27:10 -0500
https://git.sv.gnu.org/cgit/emacs.git/commit/?id=82c76e3aeb2465d1d1e66eae5db13ba53e38ed84

0. emacs -Q
1. Evaluate:

(let* (dir (clean (lambda () (when dir (delete-directory dir t)))))
  ;; Clean up temporary directory on exit.
  (add-hook 'kill-emacs-hook clean)
  ;; Create temporary parent directory.
  (setq dir (make-temp-file "ivy-" t))
  ;; Create 'bin' subdirectory.
  (make-directory (expand-file-name "bin" dir))
  ;; Start `eshell' in parent directory.
  (let ((default-directory dir))
    (eshell))
  ;; Debug in-buffer completion.
  (setq-local completion-in-region-function
              (lambda (beg end table &optional pred)
                (let* ((str (buffer-substring beg end))
                       (md (completion-metadata str table pred))
                       (all (completion-all-completions
                             str table pred (- end beg) md))
                       (standard-output (messages-buffer))
                       (inhibit-read-only t))
                  (terpri nil t)
                  (pp md)
                  (pp all)))))

2. C-h e
3. ./bi TAB

This used to yield:

  (metadata
   (cycle-sort-function . #[...])
   (category . file)
   (completion--unquote-requote . t))
  (#("bin/"
     0 1 (face completions-common-part)
     1 2 (face completions-common-part arg-end t
               rear-nonsticky (arg-end arg-begin))
     2 3 (face (completions-first-difference)))
   . 2)

but now yields:

  (metadata (completion--unquote-requote . t))
  (#("bin/" 0 4 (face (completions-common-part)))
   . 0)

So even without Company/Ivy, there is some loss of information and the
highlighting in *Completions* is inaccurate.

Thanks,

-- 
Basil