bug-datamash
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [BUG] fractional bin sizes do not work in some locales (e.g., de_DE.


From: Tim Rice
Subject: Re: [BUG] fractional bin sizes do not work in some locales (e.g., de_DE.UTF-8)
Date: Sat, 25 Jun 2022 04:00:12 +0000

Hey Erik

I suspect the operation parsing code, but have not yet looked at it.

I've been having a look. It comes down to the function `scanner_get_token()` in 
op-scanner.c.

This function returns TOK_FLOAT only when it sees a digit followed by a period. 
TOK_COMMA is handled separately and is never considered for being part of a 
float.

Turning the scanner code into a bunch of special cases for each locale would be 
a nasty piece of work. Let's not head in that direction.

I think we might want something like a function pointer which switches between scanners 
according to what operation is being parsed. We might have something like a 
"bin_scanner_get_token()" function. It would be set as the scanner whenever the 
current operation is `bin`. It knows everything after the ':' delimiter should be a 
float, and takes advantage of locale-aware code to do the right thing.

Using function pointers like this could be a useful first step towards the 
modularization work proposed for v2.0, too. Admitting different scanners on an 
ad hoc basis will make it easier to introduce new ops, or do more sophisticated 
things with the current ops.

Given the complexity of the work, I wonder what you think of defering this 
until after the v1.8 release?

~ Tim



reply via email to

[Prev in Thread] Current Thread [Next in Thread]