Re: Skip filename recoding tests on MS-Windows

[pertusus]

On Mon, Oct 24, 2022 at 07:55:40PM +0300, Eli Zaretskii wrote:
> > Date: Sun, 23 Oct 2022 20:22:29 +0200
> > From: pertusus@free.fr
> > Cc: GavinSmith0123@gmail.com, bug-texinfo@gnu.org
> > 
> > The failure of manual_include_accented_file_name_latin1_explicit_encoding
> > is more surprising to me, as in that case INPUT_FILE_NAME_ENCODING is
> > set to ISO-8859-1, so I do not understand why the test fails, the
> > reverse encoding from UTF-8 to ISO-8859-1 should lead to a path that can
> > be found.  The function where paths are looked for is
> > locate_include_file() in input.c, it could be where something unexpected
> > happens, maybe if stat() on Windows does some kind of conversion.
> > 
> > 
> > Debugging further the 
> > manual_include_accented_file_name_latin1_explicit_encoding
> > test would require showing the string bytes before and after the call to
> > encode_file_name() in end_line.c, and then, if the string bytes seem to
> > match the expected latin1 string with \xEE for î, check if something
> > unexpected happens in locate_include_file, maybe checking what are the
> > values of filename to check if there is indeed one that should lead to
> > stat giving a 0 return value.  
> > 
> > I do not know if it is practical for you to do that Eli?
> > 
> > The texi2any.pl call for that test should be something like:
> > 
> > perl -w ./..//texi2any.pl -c TEXI2HTML --force --conf-dir ./../t/init/ 
> > --conf-dir ./../init --conf-dir ./../ext -I ./formatting/ -I formatting// 
> > -I ./ -I .  --error-limit=1000 -c TEST=1  --output 
> > formatting//out_parser/manual_include_accented_file_name_latin1_explicit_encoding/
> >  --info -c INPUT_FILE_NAME_ENCODING=ISO-8859-1 -D 'needrecodedfilenames 
> > Need recoded file names' 
> > ./formatting//manual_include_accented_file_name_latin1.texi
> 
> I've added debugging printouts and ran the above command.  What I see
> is that encode_file_name doesn't encode the file name at all, so it
> stays in its UTF-8 encoding, and the 'stat' in locate_include_file
> cannot find a file by that name.  The reason encode_file_name doesn't
> encode is that reverse_iconv is zero at entry to encode_file_name, and
> all of input_file_name_encoding, doc_encoding_for_input_file_name, and

input_file_name_encoding should be the string "ISO-8859-1", I checked
that it is the case for me.

> locale_encoding are NULL.  So none of the calls to iconv are made,
> reverse_iconv stays zero, and the file name is returned in its
> original encoding.
> 
> Let me know if you want me to debug this further.

Yes, that'd probably be good, although I have no clue as what could be
wrong.  You could add a printout in tp/Texinfo/XS/parsetex/Parsetexi.pm
like

      } elsif ($key eq 'INPUT_FILE_NAME_ENCODING' and defined($conf->{$key})) {
        print STDERR "III $key $conf->{$key}\n";
        conf_set_input_file_name_encoding ($conf->{$key});

If it is not set, then check earlier in texi2any.pl, maybe l 1111,
after procesing of the command line:
print STDERR "INPUT_FILE_NAME_ENCODING: 
".get_conf('INPUT_FILE_NAME_ENCODING')."\n";

Also a possible location to check would be just before calling the
parser, near l 1364:
print STDERR "INPUT_FILE_NAME_ENCODING: 
$parser_file_options->{'INPUT_FILE_NAME_ENCODING'}\n";


If the INPUT_FILE_NAME_ENCODING is passed to the Parser, then the best
would probably be to check, in input.c at the end of
set_input_file_name_encoding whether input_file_name_encoding is set
to "ISO-8859-1".  If it is the case, I don't know what's happening as
the variable is not used much in the code besides
set_input_file_name_encoding.  If it is not set, maybe a printout in
conf_set_locale_encoding in api.c cound be relevant, or looking at
Parsetexi.c to see if something is strange.

-- 
Pat