[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: creating unibyte strings
From: |
Eli Zaretskii |
Subject: |
Re: creating unibyte strings |
Date: |
Fri, 22 Mar 2019 17:11:52 +0200 |
> From: Stefan Monnier <address@hidden>
> Date: Fri, 22 Mar 2019 10:23:20 -0400
>
> >> I don't see what's subtle about "unibyte" strings, as long as you
> >> understand that these are strings of *bytes* instead of strings
> >> of *characters* (i.e. they're `int8[]` rather than `w_char_t[]`).
> >
> > That's the subtlety, right there. Handling such "strings" in Emacs
> > Lisp can produce strange and unexpected results for someone who is not
> > aware of the difference and its implications.
>
> But this has nothing to do with the modules API: it's not more tricky
> then when doing it purely in Elisp. Are you seriously suggesting we
> deprecate unibyte strings altogether?
We won't deprecate unibyte strings, but we decided long ago to
minimize their use.
> >> "Multibyte" strings are just as subtle (maybe more so even), yet we
> >> rightly don't hesitate to offer a primitive way to construct them.
> > Because we succeed to hide the subtleties in that case,
> > so the multibyte nature is not really visible on the Lisp level,
> > unless you try very hard to make it so.
>
> Then I don't know what subtleties you're talking about.
> Can you give some examples of the kinds of things you're thinking of?
String concatenation, for one. Regular expression search for another.
And those just the ones I thought about in the first 5 seconds.
> >> > Instead, how about doing that via vectors of byte values?
> >> What's the advantage? That seems even more convoluted: create a Lisp
> >> vector of the right size (i.e. 8x the size of your string on a 64bit
> >> system), loop over your string turning each byte into a Lisp integer
> >> (with the reverted API, this involves allocation of an `emacs_value`
> >> box), then pass that to `concat`?
> > That's one way, but I'm sure I can come up with a simpler one. ;-)
>
> I'm all ears.
Provide an Emacs primitive for that, then at least some of the
awkwardness is gone. And/or use records.
> >> It's probably going to be even less efficient than going through utf-8
> >> and back.
> > I doubt that. It's just an assignment. And it's a rare situation
> > anyway.
>
> Why do you think it's rare?
Because the number of Emacs features that require you to submit a
unibyte string is very small.
> It's pretty common to receive non-utf-8 byte streams from the external world.
> And when you do receive them, it can come at a very fast pace and become
> temporarily anything but rare.
Are you talking about text encoded in some non-UTF-8 encoding? If so,
it should be converted to UTF-8, and that will solve the problem. If
it isn't text, then what common use cases are you talking about?
> 2- the C side string contains text in latin-1, big5, younameit.
> The module API provides nothing convenient. Should we force our
> module to link to C-side coding-system libraries to convert to utf-8
> before passing it on to the Elisp, even though Emacs already has all
> the needed facilities? Really?
Yes, really. Why is that a problem? libiconv exists on every
platform we support, and is easy to use. Moreover, if you just want
to convert a native string into another native string, using Emacs
built-in en/decoding machinery is inconvenient, because it involves
more copying than necessary.
> 3- The C side string contains binary data (say PNG images).
> What does "arrange for it to be UTF-8" even mean?
Nothing, since in this case there's no meaning to "decoding".
- Re: Oddities with dynamic modules, (continued)
- Re: Oddities with dynamic modules, Philipp Stephani, 2019/03/21
- Re: Oddities with dynamic modules, Eli Zaretskii, 2019/03/21
- Re: Oddities with dynamic modules, Philipp Stephani, 2019/03/21
- Re: Oddities with dynamic modules, Eli Zaretskii, 2019/03/21
- Re: Oddities with dynamic modules, Philipp Stephani, 2019/03/21
- creating unibyte strings (was: Oddities with dynamic modules), Stefan Monnier, 2019/03/21
- Re: creating unibyte strings (was: Oddities with dynamic modules), Eli Zaretskii, 2019/03/22
- Re: creating unibyte strings, Stefan Monnier, 2019/03/22
- Re: creating unibyte strings, Eli Zaretskii, 2019/03/22
- Re: creating unibyte strings, Stefan Monnier, 2019/03/22
- Re: creating unibyte strings,
Eli Zaretskii <=
- Re: creating unibyte strings, Stefan Monnier, 2019/03/22
- Re: creating unibyte strings, Eli Zaretskii, 2019/03/22
- Re: creating unibyte strings, Elias Mårtenson, 2019/03/24
- Re: creating unibyte strings, Eli Zaretskii, 2019/03/24
- Re: creating unibyte strings, Elias Mårtenson, 2019/03/24
- Re: creating unibyte strings, Eli Zaretskii, 2019/03/24
- Re: creating unibyte strings, Elias Mårtenson, 2019/03/26
- Re: creating unibyte strings, Stefan Monnier, 2019/03/26
- Re: Oddities with dynamic modules, Eli Zaretskii, 2019/03/22