Re: Ghostscript fails with special characters in filename

[Ian Hulin]

Hi Reinhold,
I've tried your patch for ly:format and "~S" here.

Reinhold Kainhofer wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Am Samstag, 22. August 2009 15:57:51 schrieb Ian Hulin:
> > Hi Patrick, Reinhold,
> >
> > Patrick McCarty wrote:
> > > On 2009-08-21, Ian Hulin wrote:
> > > > Patrick McCarty wrote:
> > > > > Hmm...
> > > > >
> > > > > I just realized why the ~S was being used instead of ~a: the ~S allows
> > > > > double quotes in the filename, but ~a does not.
> > > > >
> > > > > So this would fail with your patch:
> > > > >
> > > > >  $ lilypond \"file\".ly
> > > > >
> > > > > Even though it is an unlikely filename, ly:format cannot accept it
> > > > > using ~a.
> > > > >
> > > > > Now, I'm having trouble finding an equivalent to ~S to use with
> > > > > ly:format; such an option does not seem to exist.  I think we're
> > > > > stuck, unless someone can implement an ~S format specifier for
> > > > > ly:format.
> > > > Just thinking out loud. .
> > > >
> > > >    * These patches work round a guile restriction, which they're
> > > >      planning to fix.
> > > Well, I don't know for certain if the Guile team has fixed *this*
> > > particular restriction, namely that "format" cannot handle wide UTF-8
> > > characters.
> > >
> > > >    * The patches fix 90% of the back-end crashes for pdf and png (do we
> > > >      need another for svg, by the way?)
> > > It's not needed for the SVG backend, since the filename is not being
> > > passed to a command.
> > >
> > > >    * You've identified a rare case which needs a fix to some pretty
> > > >      basic code.
> > > How do we know it's rare?  I don't think it's worth breaking PDF
> > > generation based on an assumption.
> > >
> > > >    * You could add something like
> > > >      (set! filename string-regexp-substitute ("[^-[:alnum:]]" "_"
> > > >      filename)) to defend against this, see attached patch - it's
> > > >      Rheinhold's patch with some extra code to defend against the case
> > > >      you pointed out, changing the file\.pdf to file_.pdf.
> > > The real issue is that double quotes are not escaped, so in your
> > > patch, you should use
> > >
> > >   (filtered-name (string-regexp-substitute "\"" "\\\"" name))
> > >
> > > instead.  If you look at the "escape-string" procedure in
> > > scm/output-socket.scm, you'll see another place where this technique
> > > is used.
> > Looks to me like it's not quite so straightforward.
> > (I'm assuming here we've accepted and are using Reinhold's ly:format
> > upgrade to interpret ~S)
> > Maybe we need to do something like this pseudocode?
> > if (the incoming filename has any " characters)
> > then
> >         escape all the " characters
> >         use the /S directive
> > else
> >         filter out any problem characters
> >         use the /a directive
> > endif
>
> Why would we need this? ~s should always be able to run gs without any 
> problems. If it's not, that's a bug in the implementation of ~s!
I tried your patch, recompiled lily and ran this test with the 2.13.4 
development version

It looks like a filename with quoted characters will cause the 
compilation to hang:

1)

address@hidden:~/Desktop/Development/ExsultateTest$ $devlilypond 
\"Čača-file\".lyGNU LilyPond 2.13.4
Processing `"Čača-file".ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
Solving 1 page-breaking chunks...[1: 1 pages]
Drawing systems...
Layout output to `"Čača-file".ps'...^C
address@hidden:~/Desktop/Development/ExsultateTest$ $devlilypond 
--verbose \"Čača-file\".ly
GNU LilyPond 2.13.4
warning: Relocation: is absolute: argv0=/home/ian/usr/bin/lilypond
PATH=/home/ian/usr/bin (prepend)
Setting PATH to 
/home/ian/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
warning: Relocation: compile datadir=, new 
datadir=/home/ian/usr/share/lilypond//2.13.4
warning: Relocation: framework_prefix=/home/ian/usr/bin/..
Setting INSTALLER_PREFIX to /home/ian/usr/bin/..
PATH=/home/ian/usr/bin/../bin (prepend)
Setting PATH to 
/home/ian/usr/bin/../bin:/home/ian/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
Setting GUILE_MIN_YIELD_1 to 65
Setting GUILE_MIN_YIELD_2 to 65
Setting GUILE_MIN_YIELD_MALLOC to 65
Setting GUILE_INIT_SEGMENT_SIZE_1 to 10485760
Setting GUILE_MAX_SEGMENT_SIZE to 104857600

LILYPOND_DATADIR="/home/ian/usr/share/lilypond/2.13.4"
LOCALEDIR="/home/ian/usr/share/locale"

Effective prefix: "/home/ian/usr/share/lilypond/2.13.4"
PATH="/home/ian/usr/bin/../bin:/home/ian/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
[/home/ian/usr/share/lilypond/2.13.4/scm/lily.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/lily-library.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/file-cache.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-event-classes.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-music-types.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/output-lib.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/c++.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/chord-ignatzek-names.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/chord-entry.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/chord-generic-names.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/stencil.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/markup.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/music-functions.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/part-combiner.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/autochange.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-music-properties.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/beam-settings.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/auto-beam.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/chord-name.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/parser-ly-from-scheme.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/ly-syntax-constructors.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-context-properties.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/translation-functions.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/script.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/midi.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/layout-beam.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/parser-clef.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/layout-slur.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/font.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/encoding.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/flag-styles.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/fret-diagrams.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/harp-pedals.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/predefined-fretboards.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-markup-commands.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-grob-properties.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-grobs.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-grob-interfaces.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/define-stencil-commands.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/titling.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/paper.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/backend-library.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/x11-color.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/tablature.scm]
[/home/ian/usr/share/lilypond/2.13.4/scm/safe-lily.scm]
Initializing FontConfig...
adding font directory: /home/ian/usr/share/lilypond/2.13.4/fonts/otf
adding font directory: /home/ian/usr/share/lilypond/2.13.4/fonts/type1
Building font database.

Processing `"Čača-file".ly'
Parsing...
[/home/ian/usr/share/lilypond/2.13.4/ly/init.ly
 [/home/ian/usr/share/lilypond/2.13.4/ly/declarations-init.ly
  [/home/ian/usr/share/lilypond/2.13.4/ly/markup-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/music-functions-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/toc-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/nederlands.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/drumpitch-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/chord-modifiers-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/script-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/scale-definitions-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/grace-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/midi-init.ly
   [/home/ian/usr/share/lilypond/2.13.4/ly/performer-init.ly]]
  [/home/ian/usr/share/lilypond/2.13.4/ly/paper-defaults-init.ly
   [/home/ian/usr/share/lilypond/2.13.4/ly/titling-init.ly]]
  [/home/ian/usr/share/lilypond/2.13.4/ly/engraver-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/dynamic-scripts-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/spanners-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/property-init.ly]
  [/home/ian/usr/share/lilypond/2.13.4/ly/predefined-fretboards-init.ly]]
 ["Čača-file".ly]
Interpreting music...
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-20.otf]
elapsed time: 0.02 seconds
Element count 31 (spanners 6)
Preprocessing graphical objects...
Grob count 69
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-11.otf]
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-13.otf]
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-14.otf]
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-16.otf]
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-18.otf]
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/emmentaler-23.otf]
Solving 1 page-breaking chunks...
[century_schoolbook_l_serif_3.865234375][1: 1 pages]
Drawing systems...
Element count 51
Layout output to `"Čača-file".ps'...
[/home/ian/usr/share/lilypond/2.13.4/fonts/otf/CenturySchL-Roma.otf][/home/ian/usr/share/lilypond/2.13.4/ps/music-drawing-routines.ps]
[/home/ian/usr/share/lilypond/2.13.4/ps/lilyponddefs.ps]
^C
address@hidden:~/Desktop/Development/ExsultateTest$

2)

The same file unquoted, works
address@hidden:~/Desktop/Development/ExsultateTest$
 $devlilypond Čača-file.ly
GNU LilyPond 2.13.4
Processing `Čača-file.ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
Solving 1 page-breaking chunks...[1: 1 pages]
Drawing systems...
Layout output to `Čača-file.ps'...
Converting to `./Čača-file.pdf'...
address@hidden:~/Desktop/Development/ExsultateTest$

3)
Quoted string without accented characters
address@hidden:~/Desktop/Development/ExsultateTest$ $devlilypond \"file\".ly
GNU LilyPond 2.13.4
Processing `"file".ly'
Parsing...
Interpreting music...
Preprocessing graphical objects...
Solving 1 page-breaking chunks...[1: 1 pages]
Drawing systems...
Layout output to `"file".ps'...^C
address@hidden:~/Desktop/Development/ExsultateTest$

> Furthermore, your pseudocode to filter out any "problem characters" will 
> terribly break makefiles etc. since the output filename will no longer be 
> inputfilename.pdf and thus standard rules like:
> %.pdf: %.ly
>         lilypond $<
> will fail.

Not quite, % and  $< are make directives which get substituted for real 
names when make generates the actual command line.  However that's a 
nit-pick and I accept your main point.

> Lilypond needs to be able to handle all characters that the OS allows in a 
> filename and produce output files that are exactly the same as the input 
> filename. Period.

See test results above

Cheers,

Ian