Re: Tokenizing

[Vladimir Kazanov]

Okay, I'll give text properties a try.

Right now my vision for this mode is the following:

- avoid retokenizing undamaged buffer parts at all costs (as a main
feature meant for incremental parsing);

- collect damages and do reparsing only when user stops editing,
similar to the font-lock-mode (js2-mode, nxml-mode...);

- the incremental logic should have two interfaces, the first one
meant for language-specific tokenizing code and a second one - for the
user code, be it code beautifiers or advanced incremental parsers;

- it should be possible to completely replace the font-lock-mode with
this mode, given a concrete language tokenizer;

You said two things basically: 1) I must use text properties, 2) it is
possible to improve text properties interfaces to help the tokenizer.
I suggest the following plan:

1) try to implement the tokenizer using available text property mechanics;

2) see if there are slow-downs or problems, or space for improvements
on the Emacs side.

Does that sound reasonable?

On Sat, Sep 20, 2014 at 4:21 PM, Richard Stallman <address@hidden> wrote:
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Do you plan to reparse continuously to keep the token structure up to
> date all the time?
>
> Given that reparsing is likely to take time, it may not be the best approach.
> It could sometimes make Emacs become nonresponsive while it is reparsing
> a substantial amount of text.
>
> Ideally it should be like Font Lock -- reparsing parts of the buffer
> when time is available.
>
> --
> Dr Richard Stallman
> President, Free Software Foundation
> 51 Franklin St
> Boston MA 02110
> USA
> www.fsf.org  www.gnu.org
> Skype: No way! That's nonfree (freedom-denying) software.
>   Use Ekiga or an ordinary phone call.
>



-- 
Yours sincerely,


Vladimir Kazanov


--
С уважением,

Владимир Казанов