[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#58168: string-lessp glitches and inconsistencies
From: |
Mattias Engdegård |
Subject: |
bug#58168: string-lessp glitches and inconsistencies |
Date: |
Thu, 29 Sep 2022 18:24:04 +0200 |
We really want string< to be consistent with string= and itself since this is
fundamental for string ordering in searching and sorting applications.
This means that for any pair of strings A and B, we should either have A<B, B<A
or A=B.
Unfortunately:
(let* ((a "ü")
(b "\xfc"))
(list (string= a b)
(string< a b)
(string< b a)))
=> (nil nil nil)
because string< considers the unibyte raw byte 0xFC and the multibyte char
U+00FC to be the same, but string= thinks they are different.
We also distinguish raw bytes by multibyte-ness:
(let* ((u "\x80")
(m (string-to-multibyte u)))
(list (string= u m)
(string< u m)
(string< m u)))
=> (nil t nil)
but this is a minor annoyance that we can live with: we strongly want string=
to remain consistent with `equal` for strings.
So, what can be done? The current string< implementation uses the character
order
ASCII < ub raw 80..FF = mb U+0080..U+00FF < U+0100..10FFFF < mb raw 80..FF
in conflict with string= which unifies unibyte and multibyte ASCII but not raw
bytes and Latin-1.
It suggests the following alternative collation orders:
A. ASCII < ub raw 80..FF < mb U+0080..10FFFF < mb raw 80..FF
which puts all non-ASCII multibyte chars after unibyte.
B. ASCII < ub raw 80..FF < mb raw 80..FF < mb U+0080..10FFFF
which inserts multibyte raw bytes after the unibyte ones, permitting any ub-ub
and mb-mb comparisons to be made using memcmp, and a slow decoding loop only
required for unibyte against non-ASCII multibyte strings.
C. ASCII < mb U+0080..10FFFF < mb raw 80..FF < ub raw 80..FF
which instead moves unibyte raw bytes to after the multibyte raw range. This
has the same memcmp benefit as alternative B, but may be slightly faster for
ub-mb comparisons since only unibyte 80..FF need to be remapped.
Any particular preference? Otherwise, I'll go with B or C, depending on what
the resulting code looks like.
- bug#58168: string-lessp glitches and inconsistencies,
Mattias Engdegård <=