Re: unibyte<->multibyte conversion [Re: Emacs-diffs Digest, Vol 2, Issue 28]

[Stefan Monnier]

> In article <address@hidden>, "Eli Zaretskii" <address@hidden> writes:
> >>  On process reading, if raw-text is used, the process output
> >>  is at first read as a unibyte string, the string is coverted
> >>  to multibyte by string-as-mulitbyte (not by not-yet-existing
> >>  string-to-multibyte), then inserted in a multibyte buffer.
> 
> > Sorry, I don't think I understand the difference.  What will we have
> > in the buffer after process output is converted as you describe in the
> > last paragraph above?
> 
> Ok, here's an example (Latin-1 lang. env.).
> 
> unibyte sequence (hex): 81    81    C0    C0
>                         result of conversion    display in multbyte buffer
> string-as-multibyte:    9E A1 81    C0    C0    \201À\300
> string-make-multibyte:  9E A1 9E A1 81 C0 81 C0 \201\201ÀÀ
> string-to-multibyte:    9E A1 9E A1 C0    C0    \201\201\300\300

I find the terminology and the concepts confusing.
On the other hand, I understand the concept of encoding and decoding.
The following equivalences almost hold:

 (string-as-multibyte str) == (decode-coding-string str 'internal)
 (string-make-multibyte str) == (decode-coding-string str 'default)
 (string-to-multibyte str) == (decode-coding-string str 'raw-text)

I said "almost" because:

1 - there is no `internal' coding-system as of now.  In Emacs-21 we'd
    use `emacs-mule' but for Emacs-22 it would be `utf-8-emacs'.
    I'm still not sure what such a thing is useful for, tho (see
    my other email).

2 - there is no `default' coding-system either.  Or maybe
    locale-coding-system is this default: if your locale is
    latin-1 then that's latin-1.  For non-8-bit locales,
    I don't know what string-make-multibyte does.

3 - when called with a `raw-text' coding-system, decode-coding-string
    returns a unibyte string, which is obviously not what we want here.
    It might make sense for internal operations to return unibyte
    strings for the `raw-text' case, but I was really surprised that
    decode-coding-string would ever return a unibyte string.

I think avoiding string-FOO-multibyte and using decode-coding-string
instead would make things a lot more clear.


-- Stefan