Re: test-org-table/sort-lines: Failing test on macOS

[Max Nikulin]

On 21/11/2022 10:15, Ihor Radchenko wrote:
> Max Nikulin writes:
>
> > > However, I am not sure if ignoring locale is something we really want.
> > > WDYT?
> >
> > I think we should keep `string-collate-lessp' in the
> > `org-table-sort-lines' implementation. Users expect sorting accordingly
> > to their locales. However it is better to add a warning to
> > `org-table-sort-lines' docstring and to the manual that caseless sort
> > depends on its implementation in libc, so currently it does not work in
> > clang/llvm and so e.g. on MacOS.
>
> Sounds reasonable.
>
> Note that not only `org-table-sort-lines' is using
> `string-collate-lessp'. The full list of functions potentially affected
> by libc sorting is:
>
> 1. Bibliography order in `org-cite-basic-export-bibliography'
>     (via org-cite-basic--sort-keys -> org-cite-basic--field-less-p)
> 3. `org-table-sort-lines'
Confirmed.

> 2. `org-sort-list'
> 5. `org-sort-entries'
`downcase' is used, not proper case folding, so a potential issue

> 4. `org-set-tags' (tag order), when `org-tags-sort-function' is set to
>     "Alphabetical" or "Reverse alphabetical".

IGNORE-CASE argument is not used, perhaps `downcase' is hidden in the code.

> 6. Agenda sorting, when alphabetical sorting is involved

`string-lessp' and `downcase' so even more severe locale-related issues 
might be expected.

> 7. `org-map-entries'

Unsure which predicate is used.

> I am not 100% sure where we should add the information to
> docstring/manual and where we should not.

If footnotes in the manual had fixed labels then I would suggest 
reference the same footnote in the manual and in the docstrings. 
Perhaps, a new subsection should be added to info "(org) Miscellaneous" 
and "see info node ..." should be added to all involved docstrings.

> > Concerning the test, I would split the current testcase into 2 parts
> > depending on WITH-CASE argument, check if caseless collation is
> > available and skip the related test otherwise.
>
> How can we check the availability?

(string-collate-lessp "a" "B" "C" t)

> Indeed. Also, see https://nullprogram.com/blog/2014/06/13/. However,
> what we are concerned about here is consistency. Not the pitfalls per
> se.

Achieving consistency across Org code requires additional efforts.