Re: locked narrowing in ELisp

[Dmitry Gutov]

On 18.08.2022 09:25, Eli Zaretskii wrote:

> > I guess I didn't understand your concern, sorry. Invisible is handled
> > somewhere on the high level, OK. I did not mistake it for 'intangible'.
>
> The concern is that some parts of the display code will ignore the
> invisible text and some won't.  The display code doesn't expect
> BEGV..ZV restriction to behave that way, it expects these limits to
> affect every part of Emacs.

That's not a fundamental problem. I believe it's possible to come 99.9% 
close (and even provide better usability, say, by highlighting the 
narrowing bounds to indicate that there is more to the buffer, but it's 
hidden), but if not, the implementation can go a little lower level:

The redisplay could repeat the trick I showed with Isearch and wrap most 
of its logic in (with-soft-narrow ...) -- which translates the soft 
narrowing into an actual one temporarily. Then the 'invisible' property 
won't even be required.

> > > > A number of features/commands will indeed need to know the bounds of the
> > > > user-level narrowing (and we'll have a buffer-local variable for that),
> > > > but that's probably it.
> > >
> > > I don't think you realize how widespread is use of low-level
> > > primitives and functions in user-level commands.
> >
> > Commands that do want to obey narrowing, can take the soft-narrowing
> > bounds and apply the narrowing internally.
>
> That would mean all of the commands.  A lot of changes (or a lot of
> breakage).  It doesn't sound like a good solution, at least not better
> than just biting the bullet and introducing 2 different kinds of
> narrowing with some ways of telling Emacs internals which of the two
> to obey at any particular moment.

Two kinds of narrowings is inherently a more complex solution. But it 
might be more backward-compatible, yes. Since it just adds "one more 
thing" on top of the existing system.

> (The latter is actually the tricky
> part, IMO.)

If we just make the "locking" feature to be accessible to Lisp, the 
majority of the code shouldn't care or be aware of it. Just some 
particular features/functions/packages will make use of the locking 
argument, limiting the 'widen' calls in the rest of the Emacs from 
widening past the locked region.

> > I'm likely missing a lot of things, since I don't usually use this
> > feature interactively. All I know is, about once or twice a year, people
> > come and ask to make a certain command ignore narrowing. And nobody
> > every comes and asks to revert those changes.
>
> Making a specific command ignore narrowing is easy.  Your proposal
> implicitly assumes that the number of commands that want to ignore
> narrowing is much larger than the other kind.

That is indeed the key assumption. I'm inclined to believe that not only 
it is larger, but that the set of commands that should ignore user-level 
narrowing also grows faster.

> I don't think it's
> true, and you haven't provided any evidence to that effect.

My observation is that the number of regular users of interactive 
narrowing is not so big. Perhaps limited to a particular subset of 
old-school Emacsers (and some recently-converted sympathizers).

As a result, a lot of problems with commands _not_ ignoring narrowing 
when they should go unreported for years.

E.g. https://debbugs.gnu.org/21262 or 
https://github.com/dgutov/diff-hl/issues/48 (the packages was 3 years 
old at that point, and its main functionality was broken when narrowed) 
https://github.com/dgutov/diff-hl/pull/109 (the corresponding feature 
was 6 year old at that point).

With that in mind, creating any precise statistics doesn't seem 
possible, even if one decides to try.

On the flip side, though, it doesn't seem like diff-hl needs any support 
for user-level narrowing (ignoring it is fine), and company-mode "just 
worked" when I tested it with soft-narrow-to-region.

Same goes for all other packages I maintain.

> Moreover,
> I think it might make sense for some commands to honor or ignore the
> narrowing depending on the use case, and that is not solved with your
> proposal.

Doesn't seem difficult:

(if something
    (with-soft-narrow
     do-the-thing)
  do-the-thing)

or without the macro:

(save-restriction
  (when something
    (let ((bounds (soft-narrow-bounds)))
      (and bounds
           (narrow-to-region (car bounds) (cdr bounds)))))
  do-the-thing)

> > Could someone give a few problem scenarios with this patch? Preferably
> > ones that should be hard to fix. Adapting isearch took about 2 lines.
>
> That's just one command.  Emacs has thousands of them.

True. In the current state it only solves Isearch (obviously) and the 
basic movement/editing commands by using 'invisible', 'read-only' and 
'modification-hooks' properties.

Stefan's complaint should be fixable by changing either 
mark-whole-buffer or push-mark directly (I'm not sure yet) and 
soft-narrow-to-region to update the region bounds just in case it was 
called from a Lisp program.

I was thinking the vast majority of use cases should be solved by 
altering some key functions. But I can't guarantee that, of course.