Re: [Bug-datamash] Decimal separator

[Assaf Gordon]

Hello Martin,

Thank you for reporting this issue.

> On Jun 9, 2016, at 17:48, Martin Heller <address@hidden> wrote:
> 
> I am not sure if the following is considered a feature or if it's a bug? 
> However, it was unexpected to me that datamash on my system (Windows with 
> default input language set to Danish) expects the decimal separator 
> to be a comma (,) and not dot (.) as most other data processing tools. 

If I understand correctly, the default decimal separator in Denmark's locale is 
a comma (and not a dot).

'datamash' uses a function called 'strtold' to convert numbers [1], and this 
function respects the correct locale.
This is likely the reason for this unexpected behavior.
(The flip-side is, if a program always assumes the decimal separator is a dot, 
then people with locales which use a comma  might be upset the program does not 
respect their locale).

I do not have ready access to a Windows machine, so it will take a bit longer 
for me to troubleshoot it.

In the meantime, may I ask you to try the following commands?
They should set environment variables that will override your default locale, 
and will hopefully make 'datamash' accept dot as decimal separator:

    T:\> set LC_ALL=C
    T:\> set LANG=C
    T:\> set LC_NUMERIC=C
    T:\> cat data.txt | datamash-win32 mean 1

It that works, likely only one of them is needed, not all three.

BTW, setting "LC_ALL=C" is a common advice to override locales on unix machines 
as well, e.g. here:
  
https://www.gnu.org/software/coreutils/faq/coreutils-faq.html#Sort-does-not-sort-in-normal-order_0021


As a side note,
Are you compiling 'datamash' yourself ?
If so, there is a newer version: 1.1.0 available here: 
 http://ftp.gnu.org/gnu/datamash/
(which reminds me I should probably make an updated 'windows' executable for 
download).

Regards,
 - assaf


[1] strtold on windows:
https://msdn.microsoft.com/en-us/library/dn320178.aspx
(see the 'LC_NUMERIC' comment at the bottom).