Re: 23.0.60; [nxml] BOM and utf-8

[Stephen J. Turnbull]

David Kastrup writes:

 > I am not interested in the "goal of Unicode" but in that of Emacs.
 > Unicode is about text files.  But Emacs communicates via byte streams
 > and those are not necessarily text, or necessarily all text.

Some Emacs files *are* text, and getting them to behave correctly will
require understanding "the goals of Unicode".  Since Unicode is now
the underlying representation of multibyte buffers, you don't have a
choice about this.  Cf. Thomas Morgan's recent post on "disappearing
cursor".

 > > Sure, and Emacs must provide coding systems that preserve them, and
 > > generally use those coding systems by default.  Did anybody say
 > > otherwise?
 > 
 > So what was your point supposed to be?

That Miles could use a BOM-swallowing encoding on input and a non-BOM-
producing encoding on output to enforce his view of Microsoft
conventions on others.  I told Patrick what I thought *Emacs* should
do, but apparently it doesn't do that yet.

 > So forward-char and replace-string should be made to work as
 > expected on non-normalized texts.

Good luck.  I don't know how to do that, and doubt that it is
possible.  I do not think that "as expected" can be well defined,
because for purposes like computing storage requirements composing
characters should be considered characters, while for others like
computing the number of columns occupied by a line they should not.

 > > Binary faithfulness may be incompatible with other user demands, for
 > > example if a user introduces Latin-2 characters into a Latin-9 text.
 > 
 > Why do you think we switched to utf-8 internally and got rid of latin
 > unification?

David, don't you realize that is not a response to what I wrote?

I think it's time to stop this thread until you address the issues
instead of me.