[Monotone-devel] Updated Issue 154 - Localized output of mtn manpage leads to corrputed characters (monotone)

[code]

Hello,

The following issue has been updated:

154 - Localized output of mtn manpage leads to corrputed characters
Project: monotone
Status: Accepted
Reported by: Thomas Keller
URL: https://code.monotone.ca/p/monotone/issues/154/
Labels:
 Type:Incorrect Behavior
 Priority:Medium

Comments (last first):

# By Richard Levitte, Mar 22, 2011:

Revision 40ca0bb7367dc73f024da6ded659be69cfac1144 contains a solution for groff 
users.  That doesn't solve the issue for those who do not use groff, but it 
won't harm them either.

# By Richard Levitte, Mar 22, 2011:

The conclusion is incorrect.  What happens is that nroff/groff does not 
recognise UTF-8 input.  For groff, the default is iso-8859-1, and all it will 
do, then, is to interpret the UTF-8 characters as a series of iso-8859-1 
characters, and will happily convert them to UTF-8 (because the locale says it 
should), hence the weird display.

With groff, it's possible to have it filter the input with another program, 
preconv.  It does so if you call it with -k or -K, or if the environment 
variable GROFF_ENCODING is set to our input encoding.

The absolutely easiest way to deal with this is to change the current nroff 
call in std_hooks.lua to the following:

    GROFF_ENCODING=`locale charmap` nroff -man -rLL=%d | less -R

I can't quite grasp why 'LANG=de_DE.ISO-8859-1 mtn manpage' wouldn't work for 
you, Thomas.  The following works perfectly for me, and makes sense:

    LANG=sv_SE.ISO-8859-1 mtn manpage

Anyhow, there's a choice.  Either we can modify std_hooks.lua, which is really 
the simplest, or we can modify scr/cmd.cc, which isn't hard, really, it just 
requires a few more lines of code (more to type ;-)).

 Status: Accepted
 Owner: levitte

# By Thomas Keller, Mar 22, 2011:

Steps to reproduce the problem:
-------------------------------

Call `mtn manpage` or `mtn manpage | nroff -man` in a non-UTF8 and non-english 
locale.

Expected result:
----------------

Proper display of characters outside the ASCII range.


Actual results:
---------------

Single UTF8 bytes.


Output of `mtn version --full`:
-------------------------------

0.99.1


Note that this doesn't work either:

   `mtn manpage | nroff -man | less`
   `mtn manpage | nroff -Tutf8 -man | less`
   `LANG=de_DE.ISO-8859-1 mtn manpage`

but this does for some weird reason:

   `LANG=de_DE.ISO-8859-1 mtn manpage | nroff -man | less`


I think the problem is that nroff (groff actually) doesn't really notice that 
it should run in utf8 mode (this should be the default nowadays) and that it 
refuses to do so even if we tell him with -Tutf8 to do so explicitely.



--
Issue: https://code.monotone.ca/p/monotone/issues/154/