Re: idn2 failed to convert

[Simon Josefsson]

Tim Ruehsen <address@hidden> writes:

> Hi,
>
> how can I convince 'idn2 -l' to accept an UTF-8 encoded domain from a file, 
> while the local encoding differs (it is ISO-8859-15) ?

Hi!

The tool assumes data is encoded in the locale charset, use --debug to
make it print what it believes it is using.  So the general answer is to
temporarily change locale.

> file 'xx' contains 碼標準萬國碼.com:
>
> $ CHARSET="utf-8" idn2 -l <xx
> idn2: lookup: string contains a disallowed character

There is no support for the CHARSET variable in idn2.

> $ export LC_ALL="utf-8"; idn2 -l <xx
> bash: warning: setlocale: LC_ALL: cannot change locale (utf-8): No such file 
> or directory

I don't think "utf-8" is a valid locale.  Try:

env LC_ALL="en_US.UTF-8" idn2 -l < xx

> I guess a --encoding or --charset would be helpful or at least a hint/example 
> using CHARSET= in the man page.

That's not implemented today, but I agree it would be a useful
addition.  Patches welcome.

> BTW: I tried that libidn conversion page (http://josefsson.org/idn.php/...) 
> with the above domain and it doesn't seem to work:
>
> Output
>
> $ CHARSET='UTF-8' idn --usestd3asciirules --punycode-encode '.com' 2>&1
> .com-
> $ 

That's the libidn IDNA2003 implementation, it supports CHARSET.  The
libidn2 IDNA2008 implementation doesn't.

/Simon