bug#42149: Substring and flex completion ignore implicit trailing ‘any’

[João Távora]

Dario Gjorgjevski <dario.gjorgjevski@gmail.com> writes:

> Hi,
>
> I have found out that substring and flex completion ignore the implicit
> trailing ‘any’ introduced by ‘completion-pcm--optimize-pattern’.  This
> is evident from the examples shown next.
>
> My Emacs version is 28.0.50, built on 2020-07-01 from commit e98ddd6fc1.
>
> Example 1
> =========
>
>   (completion-substring-all-completions "f" (list "f") nil 1)
>
> and
>
>   (completion-flex-all-completions "f" (list "f") nil 1)
>
> both result in
>
>   (#("f" 0 1 (face completions-common-part completion-score 0.0)) . 0)
>
> whereas I would expect a completion score of 1.

Is it the fact that the completion-score is 0 that causes the earlier
problems you experienced?

> Example 2
> =========
>
>   (completion-substring-all-completions "fo" (list "fo") nil 1)
>
> results in
>
>   (#("fo" 0 1 (face completions-common-part completion-score 0.5) 1 2
>     (face (completions-first-difference completions-common-part))) . 0)
>
> whereas I would again expect a completion score of 1.
>
> Proposed Solution
> =================
>
> I propose that we make the implicit trailing ‘any’ explicit in
> ‘completion-substring--all-completions’.
>
> diff --git a/lisp/minibuffer.el b/lisp/minibuffer.el
> index d2c3f9045e..a598b1d1fd 100644
> --- a/lisp/minibuffer.el
> +++ b/lisp/minibuffer.el
> @@ -3585,10 +3585,12 @@ that is non-nil."
>           (pattern (if (not (stringp (car basic-pattern)))
>                        basic-pattern
>                      (cons 'prefix basic-pattern)))
> -         (pattern (completion-pcm--optimize-pattern
> -                   (if transform-pattern-fn
> -                       (funcall transform-pattern-fn pattern)
> -                     pattern)))
> +         (pattern (append
> +                   (completion-pcm--optimize-pattern
> +                    (if transform-pattern-fn
> +                        (funcall transform-pattern-fn pattern)
> +                      pattern))
> +                   '(any)))             ; make implicit `any' explicit
>           (all (completion-pcm--all-completions prefix pattern table pred)))
>      (list all pattern prefix suffix (car bounds))))
>  
>
>
> This fixes the problem and seems to perform well from my testing.
> However, I have no idea if I am overlooking something, so please let me
> know.

You analysis is good and it does reveal a quirk somewhere.  However, for
now, completion scores are relative things, i.e. they an absolute value
has no meaning by its own.

I can therefore understand that "t" disappears from your completion list
(goes to the very end) given that something else has non-zero score,
like "footrix".

But does the problem also manifest itself with two-character
completions?  I.e. is the 0.5 perfect match for "fo" (in the example you
gave) ever surpassed by another, presumably less good, match?

So I'd have to look better.  Either

- The bug is where you say it is, and the fix should go where you say it
  does.

- Some numeric adjust should be made to the algorithm
  completion-pcm--hilit-commonality.