Re: UTF-8 characters in filenames with Lilypond 2.23

[David Wright]

On Tue 24 May 2022 at 20:59:18 (-0700), Curt McDowell wrote:
> I get that same error if running under Docker, if LANG is set to
> en_US.UTF-8, but where en_US.UTF-8 does not exist in the output of
> "locale -a".
> 
>    ./out/bin/lilypond scheme-sandbox /(Lilypond 2.23.0, Guile 2.2.7,
>    Ubuntu 20.04)/
>    (open-input-file "tést.ly")
>    In procedure open-file: No such file or directory: "t??st.ly"
> 
> Changing LANG makes Guile happy
> 
>    export LANG=C.UTF-8
>    ./out/bin/lilypond scheme-sandbox
>    (open-input-file "tést.ly")
>    $1 = #<input: tést.ly 7>
> 
> but strangely it still does not make Lilypond happy
> 
>    ./out/bin/lilypond tést.ly
>    warning: cannot find file: `t??st.ly'
> 
> I don't know why.

It might help for us to see your entire locale, in case something
is overriding LANG. Then again, it might not:

(first, my disclosures)

$ cat /etc/default/locale 
LANG=C.UTF-8
LC_CTYPE="en_GB.UTF-8"
$ 

which generates:

$ locale
LANG=C.UTF-8
LANGUAGE=
LC_CTYPE=en_GB.UTF-8
LC_NUMERIC="C.UTF-8"
LC_TIME="C.UTF-8"
LC_COLLATE="C.UTF-8"
LC_MONETARY="C.UTF-8"
LC_MESSAGES="C.UTF-8"
LC_PAPER="C.UTF-8"
LC_NAME="C.UTF-8"
LC_ADDRESS="C.UTF-8"
LC_TELEPHONE="C.UTF-8"
LC_MEASUREMENT="C.UTF-8"
LC_IDENTIFICATION="C.UTF-8"
LC_ALL=
$ 

Available locales:

$ locale -a
C
C.UTF-8
POSIX
en_GB.utf8
en_US.utf8
$ 

> Lilypond 2.20.0 works in the same environment and seems insensitive to LANG.

Yes, 2.22 also, to anything:

$ LC_ALL=de_DE.UTF-8 lily -s /tmp/wwéstärn.ly 
bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8)
GNU LilyPond 2.22.1
Processing `wwéstärn.ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
Finding the ideal number of pages...
Fitting music on 1 page...
Drawing systems...
Converting to `wwéstärn.pdf'...
Success: compilation successfully completed
$ 

And 2.18.2 LP itself is unfazed, but locale trips up ghostscript:

$ LC_ALL=de_DE.UTF-8 lily -o /tmp/wwéstärn.ly 
bash: warning: setlocale: LC_ALL: cannot change locale (de_DE.UTF-8)
GNU LilyPond 2.18.2
Processing `wwéstärn.ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
Finding the ideal number of pages...
Fitting music on 1 page...
Drawing systems...
Layout output to `wwéstärn.ps'...
Converting to `./wwéstärn.pdf'...
warning: `(gs -q -dSAFER -dDEVICEWIDTHPOINTS=595.28 -dDEVICEHEIGHTPOINTS=841.89 
-dCompatibilityLevel=1.4 -dNOPAUSE -dBATCH -r1200 -sDEVICE=pdfwrite 
-sOutputFile=./wwéstärn.pdf -c.setpdfwrite -fwwéstärn.ps)' failed (256)

fatal error: failed files: "wwéstärn.ly"

$ 

OTOH, one might expect this problem:

$ LC_CTYPE=de_DE.UTF-8 lily /tmp/wwéstärn.ly 
bash: warning: setlocale: LC_CTYPE: cannot change locale (de_DE.UTF-8)
GNU LilyPond 2.23.9 (running Guile 2.2)
warning: cannot find file: `ww??st??rn.ly'
fatal error: failed files: "ww??st??rn.ly"

$ 

but perhaps not this one, as LC_CTYPE should override LANG
(I've made it explicit as well as being my default):

$ LANG=de_DE.UTF-8 LC_CTYPE=en_GB.UTF-8 lily /tmp/wwéstärn.ly 
GNU LilyPond 2.23.9 (running Guile 2.2)
warning: cannot find file: `ww??st??rn.ly'
fatal error: failed files: "ww??st??rn.ly"

$ 

But you should be able to get everything to work with LC_ALL:

$ LC_ALL=en_GB.UTF-8 LANG=de_DE.UTF-8 LC_CTYPE=de_DE.UTF-8 lily 
/tmp/wwéstärn.ly 
GNU LilyPond 2.23.9 (running Guile 2.2)
Processing `wwéstärn.ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
Finding the ideal number of pages...
Fitting music on 1 page...
Drawing systems...
Converting to `wwéstärn.pdf'...
Success: compilation successfully completed
$ 

Cheers,
David.