Re: Display of undisplayable characters: \U01F3A8 instead of diamond

[Eli Zaretskii]

> From: Akib Azmain Turja <akib@disroot.org>
> Cc: emacs-devel@gnu.org
> Date: Fri, 02 Sep 2022 17:47:39 +0600
> 
> >   > (set-char-table-extra-slot glyphless-char-display 0 'thin-space)
> >   > --8<---------------cut here---------------end--------------->8---
> >
> >   > This shows a single space character.  
> >
> > Could a variant of that display a diamond instead of a space?
> 
> I couldn't find one.  I tried the following, but it didn't work:
> 
> --8<---------------cut here---------------start------------->8---
> (set-char-table-extra-slot glyphless-char-display 0 "�")
> --8<---------------cut here---------------end--------------->8---
> 
> It didn't show the diamond (or, in my case, a character similar to a
> question mark in inverse video), which is the expected behavior
> according to the manual, but it didn't even show the diamond in a box
> (or square brackets), instead just a empty box appeared, which is
> unexpected.

It shouldn't work, indeed.  See the doc string of
glyphless-char-display.

> >   > One way of verifying this is to type
> >
> >   >    C-x 8 RET fffd RET
> >
> >   > in an Emacs session which displays on the Linux console, and see if
> >   > that shows the diamond-like glyph or the \Unnnn thing.  If it's the
> >   > latter, it means the console doesn't support that character, and we
> >   > cannot send its code from Emacs.
> 
> How do Emacs know that determine whether a character is supported or
> not?  Using the encoding used for output?

Emacs uses char-displayable-p for that purpose.

> > but what is so bad about sending a glyph that the terminal can't
> > display?  What bad results does it cause?
> 
> As long as the character can be encoded to the encoding the terminal
> understand, AFAIK it's fine.

That's insufficient.  The Linux console uses UTF-8 encoding, but it
doesn't support all the Unicode characters, although they all can be
encoded by UTF-8.  AFAIR, the Linux console supports at most 256
characters.

> > Or how about fixing it by sending an official code for that diamond glyph?
> 
> I think it's better.

If that is what is wanted, we already have in Emacs the means to set
the display to do it, and I already explained how to set that up.

> But the format of glyphless-char-display should be
> changed to allow something like the following too, which won't put the
> text in a box:
> 
> --8<---------------cut here---------------start------------->8---
> (set-char-table-extra-slot glyphless-char-display 0 (verbatim . "�"))
> --8<---------------cut here---------------end--------------->8---

It won't, because the extra slot is the fallback, the last resort.  So
it cannot use any fancy non-ASCII characters.  Which I also explained
here.