bug#41544: 26.3; Possible incorrect results from color-distance

[Mattias Engdegård]

4 juni 2020 kl. 16.07 skrev Eli Zaretskii <eliz@gnu.org>:

> Let's please discuss each problem in detail (I tried to understand
> them from the log message you posted, but couldn't find rationale
> there).
> 
> And in any case, I will prefer solutions that fix any problems
> locally, not changes in low-level stuff used in many other places,
> because the latter run the risk of introducing new bugs.  As the
> problems are quite minor, AFAICT, solving them in unsafe ways is
> something to be avoided.

I think we agree. There should be nothing unsafe here other than the code being 
replaced, but code should be scrutinised.

> You mean, the component that is 1.0393?  What bad decisions caused
> that, what problems does this small deviation causes in itself?

Yes, this is as good a place to start as any, and the fix for this is a good 
change on its own. It goes something like this:

1. color-name-to-rgb calls (color-values "#ffffffffffff") as a means to get the 
range of the colour values.
2. With TERM=xterm-color, there are 8 colours. These are assumed to be the 8 
first of xterm-standard-colors (xterm.el).
3. The colour closest to "#ffffffffffff" is "white", with the values (229 229 
229), or translated to 16 bit/channel, (#e5e5 #e5e5 #e5e5) which color-values 
returns.
4. "blue" has the values (0 0 238), or (0 0 #xeeee).
5. Thus color-name-to-rgb returns #xeeee/#xe5e5 for the blue channel, or 1.039, 
which is a clear bug.

The main problem is trusting "#ffffffffffff" to match a colour with the maximum 
range. It could be argued that xterm.el shouldn't use subdued colours when only 
8 are present; I didn't go far back in XTerm history to find out. Modern XTerm 
has default colours 0-7 that correspond to the assumptions of Emacs.

Since we already document that the colour channel maximum is either 65535 or 
65280 depending on platform, taking the very roundabout way of trying to match 
a sufficiently white colour and using its components is demonstrably unsafe and 
error-prone, as well as unnecessarily slow. Hence color-component-max in the 
patch.

This also fixes a different problem: if the display hasn't been initialised 
fully, such as when running in batch mode, then (color-values "#ffffffffffff") 
returns nil, and as we shall see later, it may be useful to be able to call 
color-name-to-rgb at this stage without crashing.