Re: Multiple lex scanners? How?

[Nicolas Joly]

On Mon, Jan 15, 2007 at 04:52:37PM -0700, Bob Proulx wrote:
> Multiple flex generated scanners are giving me trouble with duplicate
> symbols.  I am using automake-1.10.
> 
> What is the best practice for organizing a program that includes
> multiple lex generated scanners?
> 
> I am using the recommended practice of using defines to rename all of
> the yacc/lex symbols as described in the automake manual
> (e.g. "#define yylex c_lex", "#define yyerror c_error", etc.) for the
> yacc generated parser.  The problem I encounter is that yyin is hard
> to get redefined.  Flex outputs a definition for it before outputing
> my define for it in the top of the scan.l file.
> 
> Basically what I have looks like this:
> 
>   myprog_SOURCES = parse.y scan.l
>   BUILT_SOURCES = parse.c parse.h scan.c
>   AM_YFLAGS = -d
> 
> That is all well and good but then yyin is a global symbol defined by
> the flex generated scanner.  This appears before I can redefine it
> inside the source.
> 
> I guess I can add an AM_CPPFLAGS=-Dyyin=myprogin and list out all of
> the symbols to redefine there.  But that fails to work if I have two
> flex generated scanners in the same directory.  I would rather find a
> more general solution.
> 
> If I add an option to flex to rename the symbols like this then the
> scanner flex generates renames the symbols for me.
> 
>   AM_LFLAGS = -l -Pmyprog
> 
> But in this case ylwrap fails because -P also renames the output file
> to lex.myprog.c and ylwrap fails to find the produced file.  And it
> has the same problem of failing if there are two flex generated
> scanners in the same directory.
> 
> I am about ready to write a custom rule to sed the generated file.  I
> am sure that will work but I must be missing something obvious since
> this is a common enough problem that there must be a standard
> solution that I am missing.

I have a program that use 17 flex scanners (and 17 grammatical
analysers) to detect/parse biological sequence and alignement formats.

I encountered the same problem ... and made the following constructs
that did the trick (at least for me).

AUTOMAKE_OPTIONS = subdir-objects

AM_YFLAGS = -d -p `basename $* | sed 's,y$$,,'`
AM_LFLAGS = -s -P`basename $* | sed 's,l$$,,'` -olex.yy.c
[...]
CLU_SRC = align/clustal.c align/clustaly.y align/clustall.l
CLU_HDR = align/clustal.h
EMB_SRC = sequence/embl.c sequence/embly.y sequence/embll.l
EMB_HDR = sequence/embl.h
FAS_SRC = sequence/fasta.c sequence/fastay.y sequence/fastal.l
FAS_HDR = sequence/fasta.h
[...]

Hope this helps,
Regards.

-- 
Nicolas Joly

Biological Software and Databanks.
Institut Pasteur, Paris.