Re: A vision for multiple major modes [was: Re: [Emacs-diffs] widen-limits c331b66:]

[Alan Mackenzie]

Hello, Dmitry.

On Mon, Mar 28, 2016 at 01:59:21AM +0300, Dmitry Gutov wrote:
> On 03/27/2016 03:09 PM, Alan Mackenzie wrote:

> > I can't really see a proof of concept happening.  Either the thing is
> > implemented or it's not.  There's hardly a half way point.

> "Full implementation" would work, too. But it would still be considered 
> a prototype. Point is, we can't really decide whether we'll end up using 
> it, before trying it for a while.

Of course.

> >>> That will require more investigation.  But syntax based searching and
> >>> regexp based searching will certainly be amongst them.

> I don't think that's enough. E.g. (goto-char (match-end n)) is a common 
> idiom for syntax highlighting and indentation code.

Yes, but that match is going to be in the same island, or at the very
least, in the chain of islands containing the current one.

> >> [island-affected primitives]
> > looking-at? Probably.
> > char-after? No.
> > syntax-after? No. (This works only on a single char, hence must be in a
> >               single island, it can't span two.)
> > forward-char? Maybe.  In fact, there's a good argument for not moving
> >               forward when `forward-char' hits an island boundary.

> Being confined to the current island is not what I had in mind. If 
> that's the idea of how such primitives would work, I don't really see a 
> lot of benefit over using narrowing around all those calls.

Are you thinking more that `forward-char' should move from the end of an
island to the beginning of the next island in a chain?

> >> Speaking of the last one, will the buffer positions, as visible to the
> >> submode code, be renumerated, or will they have gaps?

> > It would retain the global buffer position, for consistency with the rest
> > of Emacs.  For example, in narrowed buffers, the position relative to
> > point-min is never used.

> True. I'm more worried about gaps. But they could be treated like 
> whitespace, I suppose.

For example, by `forward-char'?  What primitives were you thinking of,
here?

> >> Depending on your answer to the previous question, even simply inserting
> >> text inside one of the preceding islands might make syntax-ppss-cache
> >> out of date in the current island.

> > That could probably be solved by making positions in that cache relative
> > to the beginning of the island, and things like that.  I think you'd want
> > to make `syntax-propertize--done' island local, too.

> Right. So then, some yet-unknown body of code has to become 
> island-aware, and the improvement is not that seamless anymore.

The way I see it, the super modes would have to be totally aware of the
island mechanism, but the major modes would be largely, it not totally,
unaware of it.  `syntax-ppss' seems more part of the super mode
mechanism.

> >>> It wouldn't use any high level facility like post-command-hook.  The
> >>> mechanism would be more like a buffer local variable, entirely handled
> >>> by the C level, so that such a binding would simply exist.

> >> You didn't answer my question, though. When will the bindings apply?

> > Sorry, an island local binding would be current when point is within that
> > island.  Or, perhaps there could be some variable which would be set to a
> > position to do this job.

> What if var has an island-local binding? And my function has this:

> (setq var 1)
> (goto-char xx) ; where xx is in a different island

> will its value change mid-program? What if xx is in the same island? 
> Will the value change then?

A different island local binding would become current, so yes its value
would thereby change.  Much like if var had a buffer local binding, and
you do (set-buffer "foo"), the value of var you'd just set would no
longer be in the current binding.  if xx is in the same island, the
binding just setq'd would remain the same.

> I don't think either should be true. Then, the "adequate" model would 
> amount to changing them in post-command-hook anyway.

What sort of variables are you thinking about here?  [ Some time later:]
Could it be that it might be better to have "island chain local"
variables rather than merely "island local" ones?  So that the three ruby
lines in your example below would be three islands in a chain, and would
thus all share a set of "island chain local" bindings rather than each
line having its own binding?

> >> I don't know if we'd need the island-beginning/island-end accessors in
> >> your proposal.

> > I think we would.  (narrow-to-region (point-min) (point-max)) is a null
> > operation.  If the major mode has narrowed, and needs to set point-min to
> > "1", it will need to know the island-beginning, somehow.

> What if it just calls widen?

That also sets point-max, which would specifically not be wanted.

> They [ super modes ]'d still have to maintain the island boundaries.
> And the lists of variables to set island-locally, or mode-locally, or
> chunk-locally. That constitutes the majority of the related code
> already.

Yes.  How could it be otherwise?  But with sharp tools, the work could be
easier.

> > It would not be necessary to use widening and narrowing to
> > restrict indentation/fontification code to an island.

> So making it seem, to the Lisp code, like the multiple related islands 
> don't have anything (for some definition of "anything") between them is 
> not a priority?

By "related", I think you mean the same thing I meaan by "chained" - the
three ruby mode lines enclosed in <%...%> in your example below, would be
chained together by the super mode.

By "anything between them", I take it you mean certain primitives (still
to be pinned down precisely) would make it look like the successive
islands in a chain followed on from eachother without "anything between
them".

We could make it a priority.

> That would be the high point of the "islands" idea from my perspective.
> But having only some primitives affected would likely break this use
> case.

> Here's an example of ERB code Vitalie brought up recently:

> <% if foo %>
>    <%= bar(boo) %>
> <% end %>

> The parts of the buffer between %'s would be Ruby islands. The 
> suggestion was to use the value returned by the indentation function in 
> the second island (second line) to indent the "real" buffer contents.

The question seems to be, are we indenting in the super mode or in each
island?  The answer seems to be you'd want to indent in the super mode
here, I think.

> But: neither of the islands contains a newline. If they are combined in 
> the most straightforward way, the Ruby line will be 'if foo bar(boo)', 
> and its indentation would be not what we're looking for. I think the 
> current ways to look for a newline,

>    (progn (skip-syntax-backward " ")
>           (eq (char-before ?\n))

> or

>    (looking-at "\\s-$")

> will fail to work. Also note that neither of the two islands in this 
> example contains the newline in question.

Two?  Don't you mean three, here?  Is this a minor mistake in your
counting or am I missing something important because I don't grok ruby?

> To sum up, hard examples are hard. And yet, the new feature should make 
> some use cases possible that were impossible before, not just allow for 
> easier island-local variables.

> > I don't think having islands start at position 1 is a good idea.  But
> > having the relevant primitives skip the gaps between chained islands, and
> > stop at an island boundary when not chained, when `restrict-to-island' is
> > non-nil, would be a central idea.

> The "not chained" idea would require some explanation.

For example, islands in a shell script which were individual AWK scripts
would not be chained together.  But the three (?two) lines of ruby in the
example above would be chained together.  I think we need to work this
out more precisely.

For example, in the three ruby line scenario above, is there any variable
you can think of for which it would be advantageous to have a binding
for each island rather than a shared binding for the set of three (?two)?

I'm kind of envisioning successive `forward-char's moving from the end
of "<% if foo %>" to the beginning of "<%= bar(boo) %>", or even from
the end of "if foo" to the beginning of "bar(boo)".  How does this sound?
Or am I talking total rubbish?  (It's getting late).

-- 
Alan Mackenzie (Nuremberg, Germany).