Re: RFC: custom error messages

[Akim Demaille]

Hi Christian,

> Le 17 janv. 2020 à 00:56, Christian Schoenebeck <address@hidden> a écrit :
> 
> On Dienstag, 14. Januar 2020 21:24:59 CET Akim Demaille wrote:
>> Hi Christian,
>> 
>> Thanks for your answer!  I definitely need feedback on these matters.
>> 
>>> Le 14 janv. 2020 à 14:50, Christian Schoenebeck 
> <address@hidden> a écrit :
>>>> I wouldn't call memory management in yacc.c "easy": lots of efforts
>>>> are made to allocate on the call stack, and to avoid malloc.
>>> 
>>> For such a new API function it would be easier, e.g. because you don't
>>> have to take care about breaking existing users' code. So you could e.g.
>>> add an optional allocator argument (where NULL would use Bison's default
>>> allocation) for this new API function, or a macro that could be redefined
>>> by users, etc. So there would be a bunch of options to address this
>>> concern.
>> 
>> I'm looking for answers that apply in all the backends, so I'm trying
>> to stay away from macros.  C++ developers also tend to despise them.
> 
> Yes, but unfair comparison. With C++ you can avoid many macros by using 
> templates and/or lambda expressions, without facing downsides like reduced 
> optimization potential by the compiler or increased memory pressure or 
> extensive amount of code, unsafety, etc.

My point was that I try to avoid CPP macros in Bison, and prefer solutions
that apply in all the case.  

But memory management is a different beast indeed, being almost always
different per language.


>> The current state of my draft looks like this:
>> 
>> /* Put in YYARG at most YYARGN of the expected tokens given
>>   the current YYCTX, and return the number of tokens stored
>>   in YYARG.
>>   If YYARG is null, return the number of expected tokens.  */
>> static int
>> yyexpected_tokens (const yyparse_context_t *yyctx,
>>                   int yyarg[], int yyargn);
>> 
>> WDYT?  It should satisfy those who are ready to pay for a dynamic
>> allocation, and satisfy those who want a bounded one, at compile
>> time.  Anyway, it is always bounded by YYNTOKENS.
> 
> Well, it would be fine with me, but not sure if it would for everyone. 
> Disadvantage might be that user code would always need to allocate
> YYNTOKENS * sizeof(int) for calling this function (e.g. embedded devices).

Well, I don't see much of a future for autocompletion features
in devices where memory is scarce.


> I haven't seen your implementation, but remembering how I implemented it, it 
> is probably not just a single table lookup or anything comparable simple 
> action as efficient as Theta(1), right?

There are two implementations.  The default one is quite simple
and almost O(1).  The real one, which sits on top of LAC, is way
more coslty: it forks and runs a thin parser to explore the possible
shifts.

> I mean you are probably walking the 
> tables, so a function interface taking an index as argument would not be an 
> option for runtime efficiency reason, right?

Right.


>>>> I do not plan to expose an enum for symbol numbers.  What value would
>>>> it bring to give name to these numbers?
>>> 
>>> To make it clear which internal numbers are actually reflecting user's
>>> rules/
>> Please, do not use "rule" when you mean "nonterminal symbol".  It
>> seems to confuse some people.
> 
> I knew I'll get that ticking-off. :) "nonterminal symbol" is too long for 
> always using it, and most people know the semantic of "rule" in the context 
> of 
> grammars. Invent a better, non ambiguous short term as alternative then. ;-)

There must be some confusion here: yes, most people know what a rule
is, and it's definitely not the same as a nonterminal symbol.

exp: exp '+' '0' | '0'

There is one nonterminal (exp), two terminals ('+' and '0') and two
rules ("exp: exp '+' '0'" and "exp: '0'").

There are synonyms:
- token, terminal, terminal symbol, letter
- variable, nonterminal, nonterminal symbol
- rule, production, production rule

but in no universe "exp" is a rule.  It's a symbol.  Make it an NT
if you want it to be short, but don't call it a "rule".


[Bison 4]
> Any chance that this would be released under a more permissive license, e.g. 
> GPL>=2?

No, Bison is GPL v3, period.  Too bad for Apple.

A different question though is the license of the generated code.
So far, all our skeletons have an exception that allows to use
them in non-GPL v3 projects.


>> That would be very helpful, indeed.  But in a way, that's the point
>> of the betas...  It's not automated, but, well, I do count on you
>> guys to really give a shot at the betas and report your mileage.
>> 
>> On this regard Frank's feedback is extremely valuable.  Even when
>> his answer is just "runs fine with my stuff": at least I know someone
>> else tried, and it gives me _some_ confidence in the release.
> 
> The only way to handle QA are automated tests nowadays. That's the only way 
> to 
> go.

Bison's CI is running on Travis.

> You barely find people testing things manually today in regular release 
> cycles, especially when it comes to tools. Among developers you also see the 
> trend that more and more are not running their "own" software manually at 
> all; 
> they just write tests (which run automated on remotes), or even: are just 
> reviewing those submitted by other people.
> 
> Likewise you can be sure I'll get my hands on those planned builtin scanner 
> features once they are there, but I won't be able test every Bison release 
> manually. That's simply not possible.

Well, you guys, who depend on Bison, could just as well pull
a version of it in your CI to see what happens.  If all it takes
is that I push release candidates of Bison on Docker so that CIs use
it in experimental branches, that would be great.

But I certainly cannot set up a CI for projects that use
Bison.  Setting up a CI is already sufficiently time consuming
as it is.


>> However, when I was referring to the complexity of testing Bison,
>> I was thinking about the test suite: I try to always create
>> test cases for the regressions we discover, and for the features
>> I add.  But it's difficult, and hard to predict which combination
>> of features would break.  And it's impossible to try all the possible
>> combinations.
> 
> Yeah, that's what I assumed and hence my idea about some kind of formal 
> interface for automated downstream test cases.

Docker is the closest approximation I see. But downloading and
compiling bison locally would work just as well. 

> That way you would always have 
> feedback about feature set combinations projects are really using, and 
> immediate feedback already even if some yet unreleased change would break 
> something for any downstream project,

That's even more ambitious :)

> It is about saving time as you ran see and being able to concentrate on 
> things 
> that really matter instead of having to write all those tests by yourself.

I need the elementary tests anyway when I implement.  You are right that
the real issue here is the mixture of features.  And the freedom some
users think they are entitled to take on the contract on the generated
code.

> And downstream projects would benefit from probably having to maintain less 
> Bison version checks, and/or being automatically informed very early about 
> upcoming backward incompatibility issues to be handled and/or trying to react 
> by sending a veto email to prevent that. :)

I created the bison-announce mailing list to this end: low trafic, but
people are warned when something is coming.

Cheers!