Re: Parsing included files

[Giacinto Cifelli]

Hi Ervin,

according to the lex&yacc manual, there should be a yy_create_buffer called
at the beginning, corresponding to the yy_delete_buffer.
I don't see it in your code.

the example is here:
      https://github.com/mbbill/flexbison/blob/master/flexbison/fb2-3.l

Regards.
Giacinto





On Mon, Aug 17, 2020 at 6:30 PM Ervin Hegedüs <airween@gmail.com> wrote:

> Hi there,
>
> I ran into a problem: the language to be analyzed has a speciel
> expression, eg:
>
> include foo.conf
>
> which means "find and open the file foo.conf, and parse it!".
>
> Here is how I solve that:
>
> in my lexer:
>
> <INITIAL>include[ \t]+[0-9A-Za-z_\/\.\-\*\:]+ { push_sym();
> handle_include((const char *)yytext); return T_INCLUDE_DIRECTIVE_WITH_PATH;
> }
>
> in the parser:
>
> config_include:
>       T_INCLUDE_DIRECTIVE_WITH_PATH    { free($1); }
> ;
>
> the push_sym():
>
> void push_sym() {
>     yylval.s = strdup(yytext);
>     if (yylval.s[strlen(yytext)-1] == '\n') {
>         yylval.s[strlen(yytext)-1] = '\0';
>     }
> }
>
> the handle_include() does a split by space, to get the filename,
> and call open_and_parse(char * fname);
>
> There is a speciel function which opens the input file, and runs
> the parser:
>
> int file_inclusion_level = 0;   // a global variable to count the
> inclusion depth
>
> int open_and_parse(char * fname) {
>
>     FILE *fp;
>     ...
>     YY_BUFFER_STATE bufftemp = NULL;
>
>     ...
>
>     if(fp = fopen(...)) {
>         file_inclusion_level++;
>
>         if (file_inclusion_level == 1) {
>             yy_scan_string(inputbuff);
>         }
>
>         if (file_inclusion_level > 1) {
>             bufftemp = YY_CURRENT_BUFFER;
>             yy_scan_string(inputbuff);
>         }
>
>         yyparse();
>
>         if (file_inclusion_level > 1) {
>             yy_delete_buffer(YY_CURRENT_BUFFER);
>             yy_switch_to_buffer(bufftemp);
>         }
>
>         fclose(fp);
>         file_inclusion_level--;
>         yy_flex_destroy();
>     }
>
>
> When I run the compiled code, I got a fault:
>
> malloc_consolidate(): invalid chunk size
> Aborted
>
>
> There is a "main" config file with name test_inc_01.conf, which hase two
> lines:
>
> include test_inc_02.conf
> include test_inc_03.conf
>
> Both files have 2 lines:
>
> include test_inc_21.conf
> include test_inc_22.conf
>
> and
>
> include test_inc_31.conf
> include test_inc_32.conf
>
> All 4 files have just a simple config line with few tokens. The
> parser works as well when I'm just parsing the files with
> "relevant" content. But if I pass the test_inc_02.conf (which
> have 2 relevant files - see above), the malloc_consolidate()
> comes again.
>
>
> What's the correct way to handle of "include" methods?
>
>
> Thanks,
>
>
>
> a.
>
>
>
>