Re: api.header.include and backward-compatible .y files

[Kaz Kylheku]

On 2020-08-30 05:21, Akim Demaille wrote:
> Hi Adam,
>
> > Le 22 août 2020 à 02:33, Adam Novak <anovak@soe.ucsc.edu> a écrit :
> >
> > Hello,
> >
> > I'm maintaining a .y file at
> > https://github.com/vgteam/raptor/blob/master/src/turtle_parser.y that
> > needs to be backward-compatible with the Bison available in Ubuntu
> > 18.04 (3.0.4), but also work on the latest Bison that our project's
> > Mac users get supplied from Homebrew (3.7.1).
>
> Back in the days, people were *shipping* the generated files.  That
> was awesome, since then maintainers are free from such constraints:
> they use whatever version of their favorite generator is, and are
> free from requiring anything from the user; users don't even need
> to have the generator (Bison in the present case).
>
> It's a pity today we lost this wisdom.

Back in the day, people retained the generated files because
the C language had started to become portable, whereas to get
C from a Yacc grammar, they still had to upload their code
to a Unix box to run the proprietary Yacc program.

Even the person who wrote the program didn't necessarily have
consistent Unix access, not to mention anyone friends to whom
that person might give the code.

People would upload just their .y file to a Unix system, run
yacc and then download the y.tab.[ch] files.

The only valid reasons for having any generated files in version
control or distribution is unavailability of the tool.

The main excuse for the unavailability of the tool is that the
tool is what that project itself is providing. For instance,
the project contains a compiler written in its own language,
which must be bootstrapped by a copy of the compiled compiler.
Today, we try to avoid situations like a build step that requires
using a proprietary machine and tool. Free software is our main
weapon for that.

Bison's user is whoever runs Bison. Bison's user is not that one
who runs the program built with Bison; that is the user's user.
The user's user is not your user.

You cannot assume that your user is just a middleman in
a delivery chain, who can deal with any nuisance that lands his way,
because it's his job. That user may be a free software developer,
like you.

Or, your user may be working on something for his or her own use,
such that the user's user is in fact same individual as the user.

The user's for-own-use project has a grammar file in it which
they have to process through the generator from time to time.

When they rebuild it, they want to use the packaged Bison from
their current GNU/Linux environment. They don't want to use some old
version of Bison that is no longer packaged, because the current
one breaks. Just getting that to build is going to be a hassle.
It will likely have "old-environment dependencies" creating
recursive hassles just to get it to build.

Bison is a programming language. The consumers of programming languages
are programmers. Yet, we broadly value stability of programming
languages. Multiple implementations that adhere to common standards
are also a boon.

Can you imagine some GCC maintainer suggesting that you ship a .o
file built with a specific GCC version because of some problem?

These build problems are not even issues with the "core" of Bison,
which works great. They are "fluff". But the fluff is a pain in the 
butt.

I think the problem is that Bison has a good test suite for the "meat".
But the tests are, by and large, single-file programs that do not
provide coverage of possible program structures and build steps
that surround a grammar file "in the wild".

This is a problem because Bison is a generator that copy and pastes
pieces of the user's code into multiple generated files, and
code is subject to issues of ordering, naming and scope.

Bison "slices and dices" code like a giant, unhygienic macro,
whose expansion strategy is not completely documented in every detail.

Any changes which alter which piece you stick where and in what
order can potentially break things downstream.