[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Which lexer do people use?
From: |
Akim Demaille |
Subject: |
Re: Which lexer do people use? |
Date: |
Mon, 6 Jul 2020 18:03:07 +0200 |
> Le 6 juil. 2020 à 14:00, Maury Markowitz <maury.markowitz@gmail.com> a écrit :
>
>> On Jul 4, 2020, at 6:46 AM, Christian Schoenebeck
>> <schoenebeck@crudebyte.com> wrote:
>> The only thing that people are missing once in a while on scanner side is
>> unicode support
>
> I am missing the capability to support grammars with optional spaces between
> elements. In BASIC:
>
> FORI=1TO5
>
> is equivalent to:
>
> FOR I=1 TO 5
>
> I have yet to see an elegant way to model this in Flex. This topic comes up
> time to time here, so I don't think Unicode is the "only" thing.
I believe this is a perfect place to use Flex's / operator I mentioned
(similar to what other engines call "lookahead", and write 'r(?=s)').
'r/s'
an 'r' but only if it is followed by an 's'. The text matched by
's' is included when determining whether this rule is the longest
match, but is then returned to the input before the action is
executed. So the action only sees the text matched by 'r'. This
type of pattern is called "trailing context". (There are some
combinations of 'r/s' that flex cannot match correctly. *Note
Limitations::, regarding dangerous trailing context.)
- Re: Parsing a language with optional spaces, (continued)
- Re: Parsing a language with optional spaces, Christian Schoenebeck, 2020/07/14
- Re: Parsing a language with optional spaces, Akim Demaille, 2020/07/18
- Re: Parsing a language with optional spaces, Christian Schoenebeck, 2020/07/20
- Re: Parsing a language with optional spaces, Akim Demaille, 2020/07/26
- Re: Parsing a language with optional spaces, John P. Hartmann, 2020/07/26
- Re: Which lexer do people use?, uxio prego, 2020/07/06
- Re: Which lexer do people use?,
Akim Demaille <=
Re: Which lexer do people use?, Derek Clegg, 2020/07/04
Re: Which lexer do people use?, Hans Åberg, 2020/07/04
Re: Which lexer do people use?, Adrian Vogelsgesang, 2020/07/04