Re: Question about Lexx & Yacc

[Hans Aberg]

At 22:34 +0100 1-01-12, Axel Kittenberger wrote:
>English is not context-free, right?

Languages that in some sense are "natural" or "intuitive" to humans seem to
be notoriously context sensitive, bacause humans are exteremenylgood at
pattern recognition, recognizing contexts and such.

Also computer programming languages such as C and C++ are context
sensitive, but to a lesser extent.

Working with Bison and Flex, which formally only recognizes context-free
grammars, one has to tweak them using a few context-sensitive senstive
featues available: Most notably, Flex has the ability to turn on/off
start-conditions, giving the chance to pare lexemes in a context sensitive
way. (Example of uses: Source code comments, C-strings, #include
preprocessor directives.)

And one can let Flex return a token based on semantic information. Then one
can let Bison change the style of parsing simply by making duplicates of
the grammar, with suitable modifications. (There is an example of this in
the Bison manual, but I do not remeber where.) -- Bison does not bother
about how it gets the token values, so if one only knows in advance of
Bison compilation time the vbarious possibilities (which later will depend
on semantic information), one simply puts in all the different (static)
grammar rules. Then let Flex dynamically select which rule to use simply by
returning a suitable token value.

Here is an example from
  http://lands.let.kun.nl/TSpublic/tosca/ti_tagman.zip
page 8:
For example, the word "after" can be a preposition (e.g. after the event), an
adverb (e.g. she arrived the morning after) or a subordinating conjunction
(After she had left, we came in).

Now, you cannot get Flex returning all possible token values of the word
"after" at once, and let Bison dermine which one is right. But you can let
Bison set a value (say by using midrules) set a flag telling which context
is right. Then, the Flex parser can use to single out a single token value.

  Hans Aberg