Re: Include %token and %union declaration from a shared file

[Alexander Barkov]

Hi again,


On 08/04/2016 10:44 AM, Alexander Barkov wrote:
> Hello Hans,
>
>
> On 07/09/2016 12:53 AM, Hans Åberg wrote:
> > > On 8 Jul 2016, at 14:46, Alexander Barkov <address@hidden> wrote:
> >
> > > We're inclined toward generating the *.yy files with help of some text
> > > pre-processor. Possibly, using some CMake commands...
> > >
> > > It would be great though if bison could collect the grammar definition
> > > from multiple files. Where can I post this feature request?
> >
> > Might it suffice with a Bison command %include <file>, that inserts
> > <file> into the *.yy file?
>
> I'm now back from my vacation.
>
> Can you please give some hints on how to implement this new directive
> %include? I looked into the sources, but have no ideas where to start
> with :)


I made this change:

> diff --git a/src/parse-gram.y b/src/parse-gram.y
> index 5a0b200..e9b8195 100644
> --- a/src/parse-gram.y
> +++ b/src/parse-gram.y
> @@ -147,6 +147,7 @@
>    PERCENT_FLAG            "%<flag>"
>    PERCENT_FILE_PREFIX     "%file-prefix"
>    PERCENT_GLR_PARSER      "%glr-parser"
> +  PERCENT_INCLUDE         "%include"
>    PERCENT_INITIAL_ACTION  "%initial-action"
>    PERCENT_LANGUAGE        "%language"
>    PERCENT_NAME_PREFIX     "%name-prefix"
> @@ -311,6 +312,7 @@ prologue_declaration:
>        nondeterministic_parser = true;
>        glr_parser = true;
>      }
> +| "%include" STRING { }
>  | "%initial-action" "{...}"
>      {
>        muscle_code_grow ("initial_action", translate_code ($2, @2, false), @2);
> @@ -586,6 +588,7 @@ grammar:
>  rules_or_grammar_declaration:
>    rules
>  | grammar_declaration ";"
> +| "%include" STRING { }
>  | error ";"
>      {
>        yyerrok;
> diff --git a/src/scan-gram.l b/src/scan-gram.l
> index a7e9bc4..51654e4 100644
> --- a/src/scan-gram.l
> +++ b/src/scan-gram.l
> @@ -227,6 +227,7 @@ eqopt    ([[:space:]]*=)?
>    "%expect-rr"                      return PERCENT_EXPECT_RR;
>    "%file-prefix"                    return PERCENT_FILE_PREFIX;
>    "%fixed-output-files"             return PERCENT_YACC;
> +  "%include"                        return PERCENT_INCLUDE;
>    "%initial-action"                 return PERCENT_INITIAL_ACTION;
>    "%glr-parser"                     return PERCENT_GLR_PARSER;
>    "%language"                       return PERCENT_LANGUAGE;

Now bison understands directives like this:

%include "test.yy"

in both prologue and grammar sections.


Of course, currently it just skips these directives :)

Am I going to the right direction?



Now I guess I need to implement a file stack.

Whenever an %include directive is found, the new code should:

1. Push gram_in to the stack.
2. Open the file mentioned in %include STRING and assign it to gram_in

Whenever a grammar file ends, it should pop the previous file value to 
gram_in.


Does this sound reasonable?


I've got some questions:

1. Where is end-of-file handled? What is the best place to add
   the "pop" related code?

2. The filename specified in STRING should probably
   be relative to the file specified in the command line,
   unless an absolute path is specified.

So if I do:

   bison ../src/test.yy

and test.yy has this directive:

  %include include.yy

it should actually include ../src/include.yy



Are there any routines in bison to distinguish absolute/relative file
names and/or to compose relative file names?


Are there any means to handle slashes in file names?
I guess to split a file name from a directory name it should
search for '/' on Linux while search search for both '/' and '\\'
on Windows, as Windows actually understands both notations, AFAIK.
Or perhaps, I could just treat both '/' and '\\' as directory part
separator on all platforms for simplicity.

3. Should we bother to catch loops, when test1.yy includes test2.yy,
and then test2.yy includes test1.yy, or even more complex loops with
more than two files involved?
We could leave this to the bison users not to do loops...

Note, for our purposes just one level on %include would be enough.
Another solution would be just to return an error whenever %include
is used in a included file.



Many thanks!