Re: Fwd: REVIEW REQUEST - user manual generation

[Vincent Belaïche]

Hello,

Answers below,

Le 02/05/2016 21:53, Gavin Smith a écrit :
> On 29 April 2016 at 09:27, Vincent Belaïche <address@hidden> wrote:
>> Attached is an attempt to do that. Unfortunately using \egroup for group
>> end does not work, because whether it is a \toks or a \def you need the
>> the end of the definition terminated by a catcode=2 character.
>>
>> So, I need only one forbidden character in filenames (in addition to
>> backslash) to make it.
>
> The use of \expandafter and groups seems like a good trick to get
> arbitrary filenames. One suggestion is to use "/" as the close group
> character (catcode 2), as that shouldn't appear in filenames.

That would work only if you ensure that the filename is a plain filename
not a file path. This is not the case currently with the -t option. But
that could be easilly achieved by doing something like (not tested):

  tex_cmd="(cd '`dirame "$_run_tex_file_name"`' && $tex_cmd '${escape}input' 
'`basename "$_run_tex_file_name"`')"

> Another idea is to look through the filename that was given for what
> characters are in it.

Indeed, that was what I was considering also, we could check for a few
unlikely characters like < > & or $, and select one of them.

> Either way it's tricky, but I don't see a better way.
>
> I've been working on a patch to avoid the use of absolute paths
> altogether, but it isn't done yet.


Anyway that is not a final answer to the issue of odd filenames. For
instance a filename like '^^20.texi' would be misinterpreted as
' .texi' by the latest patch which I have submitted.

Attached is a novel patch that uses a completely different
approach. Instead of keeping the filename as is, and catcoding special
characters to 11 or 12, it changes the filename through some sed script
in order to escape the special characters (ie replace them by fully
expandable macros).

I have also removed the piece of code that passes explicitely double
quotes to tex in case of a blank found in the filename for the two
following reasons :

- this does not work anyway because of the -recorder option causes some
  I/O error on the recorder file (double quotes not stripped)

- double quote invalidity in filenames is MSW specific. So, this is
  anyway some system dependent stuff : I am not even sure that the
  double-quotes are interpreted by tex engine, maybe this is MSW that
  interpet them.


I did some trial of the new patch on a file with filename
toto$#{}~_^%&^^20.texi and it worked, see log below:

--8<----8<----8<----8<----8<-- begin -->8---->8---->8---->8---->8----
/bin/texi2dvi: Processing toto$#{}~_^%&^^20.texi ...
/bin/texi2dvi: Checking TeX \openout in log support...
/bin/texi2dvi: /bin/texi2dvi: running etex </dev/null '\nonstopmode' '\input' 
'./openout.tex' ...
/bin/texi2dvi: Checking TeX \openout in log support... no
/bin/texi2dvi: Checking TeX recorder support...
/bin/texi2dvi: /bin/texi2dvi: running etex -recorder </dev/null '\nonstopmode' 
'\input' './openout.tex' ...
/bin/texi2dvi: Checking TeX recorder support... yes
/bin/texi2dvi: BIBINPUTS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: BSTINPUTS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: DVIPSHEADERS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: INDEXSTYLE='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: MFINPUTS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: MPINPUTS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: TEXINPUTS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: TFMFONTS='/tmp/efrench/with space:/tmp/efrench/with space/.:'
/bin/texi2dvi: texinfo.tex preloaded as `', version is `20160323' ...
/bin/texi2dvi: Inserting extra commands: \
@finalout
/bin/texi2dvi: Backing up xref files: toto$#{}~_^%&^^20.toc
toto$#{}~_^%&^^20.aux
toto$#{}~_^%&^^20.cp
/bin/texi2dvi: Cycle 1 for ./toto$#{}~_^%&^^20.texi
/bin/texi2dvi: /bin/texi2dvi: Running etex -recorder 
'\begingroup\catcode37=12\def\temp{\def\litpercent{%}}\expandafter\endgroup\temp'
 '\input' './toto$#{}\string~_\string^\litpercent 
&\string^\string^20.t2d/dvi/xtr/toto$#{}\string~_\string^\litpercent 
&\string^\string^20.texi' '\relax' ...
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (MiKTeX 2.9 64-bit)
entering extended mode
(toto$#{}~_^%&^^20.t2d/dvi/xtr/toto$#{}~_^%&^^20.texi
(C:\Nos_Programmes\MinGW\msys\share\texmf\tex\texinfo\texinfo.tex
Loading texinfo [version 2016-03-23.14]: pdf, fonts, markup, glyphs,
page headings, tables, conditionals, indexing, sectioning, toc, environments,
defuns, macros, cross references, insertions,
(C:\Nos_Programmes\MinGW\msys\share\texmf\tex\generic\epsf\epsf.tex
This is `epsf.tex' v2.7.4 <14 February 2011>
) localization, formatting, and turning on texinfo input format.) [1] [2]
(toto$#{}~_^%&^^20.toc) [-1] Chapter 1 (toto$#{}~_^%&^^20.aux) (Index) [1]
[2] )
Output written on toto$#{}~_^%&^^20.dvi (5 pages, 2688 bytes).
Transcript written on toto$#{}~_^%&^^20.log.
/bin/texi2dvi: Original xref files = toto$#{}~_^%&^^20.toc
toto$#{}~_^%&^^20.aux
toto$#{}~_^%&^^20.cp
/bin/texi2dvi: New xref files      = toto$#{}~_^%&^^20.toc
toto$#{}~_^%&^^20.aux
toto$#{}~_^%&^^20.cp
/bin/texi2dvi: Comparing xref file toto$#{}~_^%&^^20.toc ...
/bin/texi2dvi: Comparing xref file toto$#{}~_^%&^^20.aux ...
/bin/texi2dvi: Comparing xref file toto$#{}~_^%&^^20.cp ...
/bin/texi2dvi: Removing /tmp/efrench/with space/toto$#{}~_^%&^^20.t2d
/bin/texi2dvi: done.
/bin/texi2dvi: Removing /tmp/efrench/with space/toto$#{}~_^%&^^20.t2d
--8<----8<----8<----8<----8<--  end  -->8---->8---->8---->8---->8----

So this alternative approach is promissing, please note however :

- I think that the expansion of macroes in filenames after \input is
  implementation dependent. So this solution may not work with all
  implementation of tex. But anyway the filename is changed only when it
  contains strange characters, so even with a tex implementation not
  supporting well expanding macros in filenames, the patch will work for
  filenames made of alphanumeric characters only.

- In think however that expanding macro in filnames, even though not
  documented, is supported by all tex implementation, otherwise a code
  like

   \input\jobname.aux

  without a preceding \expandafter would not work. So using this
  explicit escaping rather than the catcoding does not seem to be a
  limitation in any way.

- it seems that \input already does some catcode setup, the only
  characters for which I needed some escaping stuff are %, ~ and ^. The
  latter is in fact needed only to handle filenames containing some ^^XX
  string (with X an hexadecimal digit). I think I should probably refine
  the sed script in order to replace ^ by \string^ only in the ^^XX
  case, ie minimize changes in the filename, and rely on the tex
  implementation to do the job in the other cases.

- I had to create a \litpercent macro for the case that the filename
  contains the % character.  Do you know whether there is any predefined
  macro in plain TeX already doing that. A similar trick could be done
  if we wanted to accept \ in filenames also (that would of course be
  useful in *nix only, if useful at all).

- I replaced spaces in filename by \space. This does not work with
  MikTeX, but maybe some day there is some tex implementation able to
  handle this properly --- there is no good reason why this should be more
  difficult than handling %. My opinion is anyway that the use of some
  double quote delimiters explicitely passed to tex is not a good idea.

- I have checked whether the LaTeX \input can handle spaces in
  filenames, and it cannot. So this is a deep limitation in the engine.


I suggest we go to this kind of solution for robustifying texi2dvi
w.r.t. to stange filenames. I want to refine it further in order to:

- remove current handling of ~ and make option -~ obsolete (ie ignored)

- do ^ -> \string^ only in the case of ^^XX

I am quite sorry for Stephen but there seems to be no solution to
support spaces in filenames --- at least with MikTeX etex engine.

VBR,
        Vincent

Index: texi2dvi

===================================================================

--- texi2dvi    (revision 7139)

+++ texi2dvi    (working copy)

@@ -889,8 +889,10 @@

   # resetting their catcodes to other on the command line and changing
   # texinfo.tex to initialize everything to plain catcodes.  Maybe someday.
 
-  # append the \input command.
-  tex_cmd="$tex_cmd '${escape}input'"
+  # append the \input + input filename.
+  if test $# -gt 0; then
+    tex_cmd="$tex_cmd '${escape}input' '$1'"
+  fi
 }
 
 # run_tex - Run TeX as "$tex $in_input", taking care of errors and logs.
@@ -918,14 +920,38 @@

         run rm -f "$_run_tex_file_name"
         run ln -s "$in_input"
       fi
-      tex_cmd="$tex_cmd '$_run_tex_file_name'"
       ;;
 
     *)
-      tex_cmd="$tex_cmd '$in_input'"
+      _run_tex_file_name="$in_input"
       ;;
   esac
 
+
+  # Make input filename relative, in order to alleviate risk of spaces
+  # in name
+  _run_tex_cur_dir=`pwd`
+  case "$_run_tex_file_name" in
+    *"$_run_tex_cur_dir"*)
+      # escape sed regexp special characters for then quoting cur dir in sed 
regexp
+      _run_tex_cur_dir=`echo "$_run_tex_cur_dir" | $SED -e 
's/\([.$^\]\)/\\\1/g'`
+      _run_tex_file_name=`echo "$_run_tex_file_name" | $SED -e 
"s<^$_run_tex_cur_dir/<./<1"`
+    ;;
+  esac
+
+  case "$_run_tex_file_name" in
+    *%*)
+      tex_cmd="$tex_cmd 
'${escape}begingroup${escape}catcode37=12${escape}def${escape}temp{${escape}def${escape}litpercent{%}}${escape}expandafter${escape}endgroup${escape}temp'"
+      ;;
+  esac
+  
+  # Escape special TeX characters
+  _run_tex_file_name=`echo "$_run_tex_file_name" \
+| $SED -e 's!\\([~^]\\)!\\\\string\\1!g; s! !\\\\space !g; \
+s!%!\\\\litpercent !g'`
+
+  tex_cmd="$tex_cmd '${escape}input' '$_run_tex_file_name' '${escape}relax'"
+
   verbose "$0: Running $tex_cmd ..."
   if eval "$tex_cmd" >&5; then
     case $out_lang in
@@ -1479,10 +1505,9 @@

     save_batch=$batch
     catcode_special=false
     batch=true
-    make_tex_cmd
+    make_tex_cmd ./openout.tex
     catcode_special=$save_catcode_special
     batch=$save_batch
-    tex_cmd="$tex_cmd ./openout.tex"
     # ./ in case . isn't in path
     verbose "$0: running $tex_cmd ..."
     rm -fr "openout.$2"