Re: RFC: "count_chars" function for locations

[Frank Heckenbach]

This goes back some time, but better late than never ...

Akim Demaille wrote:

> >    void count_chars (const char *s, size_t n)
>
> I understand the need, and agree with the proposal.  However I'm
> concerned by the fact that this gives a meaning to 'columns'.  Currently,
> it is unspecified: it could be just bytes, or actual characters,

... or character width (as Bison uses now in its own scanner).

> it's
> entirely up to the user to give a precise semantics to column.  Another
> issue is the handling of tabulations.  Bison's own location struct
> deals with them to implement them as 'goes to the column which is the
> next multiple of 8'.
> 
> So I'm not sure we should start giving some semantics to 'column'.

Indeed, I hadn't considered these issues (I hate the tab character,
for just this reason and similar ones, and I usually use only
English in source code and comments, so I don't have many non-ASCII
characters there).

Then again, the simple form as given in calc++/scanner.ll (also in
the manual) has the same issues:

  # define YY_USER_ACTION  loc.columns (yyleng);

Actually, I used something like this in my own code until I had
multi-line tokens, when I extended it to support newlines, unaware
of these other issues.

You might want to add at least a comment about these issues in this
and other examples, in the manual, and in location.cc.

BTW, when I added tab handling to my function, I noticed a possible
issue with the position interface. I had to do something like this:

  case '\t':
    end.column += tab_width - end.column % tab_width;
    break;

since the columns() function can only add columns, not query the
current column as needed to compute tabs. It's not a real problem
since column is a public field, but it feels a bit like breaking the
abstraction. You may or may not care about this (just please don't
make column non-public in the future :).

> Maybe something like %code location and %code position.  WDYT?  One
> issue would then be that if someone introduced something new in position
> or location, and we later add the same name, we would break the
> user's code :(

If I understand you right, this would only enable one to put one's
own implementation of such a function in the parser, rather than the
scanner file. Of course, one can't add a function to yy::location in
the scanner file, but one can use a free function and pass the
location object to it, or subclass yy::location with this function
added easily. And the scanner file is where this code belongs
logically more than the parser file (if not the location class in
the first place). So I don't think it's worth the effort to add this
to Bison.

Regards,
Frank