Re: sqlite3

[Eli Zaretskii]

> From: Lars Ingebrigtsen <larsi@gnus.org>
> Cc: emacs-devel@gnu.org
> Date: Sun, 12 Dec 2021 05:21:48 +0100
> 
> Eli Zaretskii <eliz@gnu.org> writes:
> 
> > A couple of issues that caught my eye while reading the code:
> >
> >  . Should we invoke encode_string_utf_8 with 2 last arguments Qnil,
> >    and signal an error if the result is nil?  That would make sure we
> >    were passed a valid UTF-8 string.  We could also use non-zero 3rd
> >    argument, for speed.
> 
> Let's see...  it's these two?
> 
>    If HANDLE-8-BIT is Qt, encode eight-bit characters into single bytes
>    of the same value, like the usual Emacs encoding does.
> 
>    If HANDLE-OVER-UNI is Qt, encode characters beyond the Unicode
>    range into the same 4 or 5-byte sequence as used by Emacs
>    internally, like the usual Emacs encoding does.
> 
> I think sqlite3 will give us whatever bytes have been stuffed into the
> file, so I think Qt is the most useful values here.  (Unless I'm missing
> something.)  That is, sqlite3 doesn't enforce the charset for the data,
> but it does use the charset when interpreting the data (for instance, if
> you select for 'ó < ź').
> 
> I have not read the documentation in detail on this issue, so if this is
> mistaken, somebody please correct me.

The documentation says "UTF-8", and my reading of that is that
anything outside of that is likely to trigger "undefined behavior".
There's also a question of whether we, the Emacs project, want to
force using only valid UTF-8 in those interfaces.

A related issue is with encoding file names, which are also expected
to be in UTF-8.  I don't know what will happen if a file name includes
eight-bit raw bytes, for example.  At least in the Windows code in
sqlite3, which I did read, the result will probably be a failure to
access the file.

> But to answer your question, a module is currently loaded by absolute
> name, and is typically "/usr/lib/sqlite3/pcre.so" (on this Debian
> system, at least), and comes from the sqlite3-pcre package.  But I think
> there's a lot of different extensions floating around out there, so if
> we decide to keep the facility, adding something PATH-like might be
> nice, and figuring out the .so/DLL thing by itself might also be nice.
> 
> So we'd have (sqlite-load-extension db 'pcre), perhaps, instead of the
> absolute file name.

OK, let's revisit this if and when we decide that
sqlite-load-extension will stay in Emacs.  Right now, it looks there
are several non-trivial issues with it.