Odd parser behaviour

[Tim Van Holder]

Given a simple grammar

%token A B C D E F

%%

foo
: bar { YYACCEPT; }
;

bar
: A plus_xyzzy
;

plus_xyzzy
: xyzzy
| plus_xyzzy xyzzy
;

xyzzy
: B
| opt_C opt_D E
;

opt_C
:
| C
;

opt_D
:
| D
;

%%

I would expect this to accept any input starting with A(B|C?D?E)+,
but in practice, it only accepts input starting with A(B|CDE)+,
because for the xyzzy nterm, it always tries to reduce the opt_C.

>From the .output:

state 1

    2 bar: A . plus_xyzzy
    3 plus_xyzzy: . xyzzy
    4           | . plus_xyzzy xyzzy
    5 xyzzy: . B
    6      | . opt_C opt_D E
    7 opt_C: .  [D, E]
    8      | . C

    B  shift, and go to state 4
    C  shift, and go to state 5

    $default  reduce using rule 7 (opt_C)

    plus_xyzzy  go to state 6
    xyzzy       go to state 7
    opt_C       go to state 8

Now it seems to me that bison should be able to treat detect the fact
opt_C should only be reduced if D or E is the lookahead.  The current
behaviour seems counterintuitive - and unlike with S/R or R/R conflicts
there is no diagnostic to draw attention to it.

PS.  The really weird thing is that in the real grammar that showed
this problem, using bison 2.0 (and not 2.1, 2.2 or 2.3) made it behave
as I expected.  For this sample however, bison 2.0 and 2.3 (and 1.35 for
that matter) have the same behaviour.