bug#57531: 28.1; Character encoding missing for "eo"

[Jonathan Reeve]

> I believe this is due to the fact that the text was saved in UTF-8,
> and Emacs was trying to decode it as if it were in Latin-3.

That’s the problem. Emacs should decode UTF-8 as UTF-8, not Latin-3. The fact 
that it assumes a UTF-8 locale is in fact a Latin-3 locale, without any 
reasoning for that, is a problem.

> Using the prefer-coding-system customization should fix that.

The user shouldn’t have to customize an encoding system to have a UTF-8 locale 
be interpreted as UTF-8. A UTF-8 locale should be encoded as such, without 
needing to be told otherwise.

> I disagree.  I think your system doesn’t tell Emacs enough to guess
> correctly.

It does, though. The locale data is already there, which is why I only have 
this problem in Emacs, and nowhere else on my whole system. The problem is in 
this line from `locale-language-names'. Here’s what it says:

`("eo" . "Esperanto")'

Here’s what it should say:

`("eo" "Esperanto" utf-8)'

> There’s no evidence of “eo” being a UTF-8 locale, except what we see
> in glibc.  Which is just one library on just one OS.

The evidence is everywhere, in fact, across my whole system, and every other 
system I’ve used. And glibc is not just one library on one OS: it’s the 
reference data for locales across any UNIX-like or POSIX system.

Show me any other library on any other OS that has locale data that suggests 
that “eo” is anything other than UTF-8. In particular, show me a library that 
shows that the eo locale should be encoded as latin-3.

> Emacs cannot know the system character set unless the system tells
> that.  The way to tell that is via the locale’s codeset.  If that is
> impossible, the next best is for you to tell that to Emacs in your
> init file.  I don’t understand why you insist on not using the
> solution I proposed.

The system says that it’s a UTF-8 locale. It’s interpreted as a UTF-8 locale by 
every other program except for emacs. It’s only emacs that incorrectly assumes 
latin-3, and for no reason, as far as I can tell. That’s because it’s getting 
its locale/encoding information from `locale-language-names', which is 
incorrect, or at least incomplete.

> Please try the solution I proposed, and if it doesn’t work, let’s see
> what else is needed.  If you keep insisting on defaulting Esperanto to
> UTF-8, I see now way to make any progress here.

You’re not proposing a solution, you’re proposing a workaround. Any other user 
with the “eo” locale will have this same problem, and they shouldn’t be 
expected to find this email thread, in order to find a special hack to have 
their system work as expected.

There’s no reason at all why Esperanto should be encoded in latin-3. It never 
has been, as far as I can tell, and it never well be, in latin-3, with the eo 
locale. If you can find any good reason why it should be in latin-3, I’m all 
ears, but as far as I can tell, this is a mistake.

Please keep in mind that /I’m trying to help improve emacs,/ by submitting a 
bug report about behavior in emacs that is incorrect and potentially causing 
problems for many users. Language like “I see now way to make any progress 
here” doesn’t extend any courtesy, any effort towards understanding this 
problem, or even any effort towards giving it the benefit of the doubt. It 
makes the bug reporting process unnecessarily adversarial, and, quite frankly, 
feels unprofessional.