|
| From: | Jim Porter |
| Subject: | bug#54603: 29.0.50; [PATCH] Eshell's external pipe module interferes with other argument parsing hooks |
| Date: | Thu, 31 Mar 2022 13:58:11 -0700 |
On 3/31/2022 11:26 AM, Sean Whitton wrote:
The call to `eshell-parse-lisp-argument' is meant to handle precisely
your case. It isn't doing atm, and I think it might actually be a bug
over in that function. To see this, with point on the first nonblank
char of each of these lines, do 'M-: (eshell-parse-lisp-argument) RET':
#'foo
'foo
In the first case it successfully parses it and skips point forward, but
in the latter case it does not. But I think it should, right?
I'm not so sure. That would mean "'foo" in "echo 'foo" is treated as a Lisp form, but Eshell expects you to use "#'" (or "(" or "`") to introduce a Lisp form. As I understand it, that's so that typos don't do such surprising things. There's a good chance that a user typing "echo 'foo" actually meant "echo 'foo'".
Because of that, in general, once you've moved point to partway inside a Lisp form, you can't use `eshell-parse-lisp-argument'. Maybe that should be changed? It's a bit surprising that you have to sharp-quote symbols in Eshell if you're not already inside a Lisp form (especially when those symbols aren't necessarily functions). Still, the typo-protection of the current implementation seems like a good feature...
That said, this isn't the only situation where "unbalanced" single quotes can occur in an Eshell command. For example, see this command:
echo $(list "one" "two")(:s'o'x')This creates a list of two strings, and then performs a regexp substitution from "o" to "x", so the output is:
("xne" "twx")
Under Emacs 27/28, this works correctly, but it fails in 29 for the same
reason as the original issue. The extpipe module could account for this
and try to parse argument predicates/modifiers so that it knows when to
stay out of things, but then what about configurations where that module
is disabled? (And for that matter, a third-party Eshell module could
cause conflicts in a similar manner.)
If possible, I think it would be better for `eshell-parse-external-pipeline' to solely focus on finding the external pipe operators ("*|", "*<", and "*>")[1] and then for `eshell-rewrite-external-pipeline' to prepare the command string to pass to sh. This would also have the advantage[2] of making it possible to support a richer set of Eshell features with external pipes, such as the following: ~ $ echo $(message "[%s]" "hi") *| cat zsh:1: command not found: message (If you remove the "*", this outputs "[hi]", and it should be technically possible to make this work with external pipes too, provided it executes the Lisp code before generating the command string for sh.)In this case I would want the whole '$(message ..' construction to go to the external shell. Although extpipe supports some combinations of piping Eshell in and out of the external shell, fundamentally it's more about making it easier to bypass Eshell features than to make complex usage of them. It's also simpler to understand as a user. If you do want more involved combinations of Eshell and external shell commands, you can always do the 'sh -c' wrapping yourself.
From my point of view, since the only difference between using an "Eshell pipe" and an extpipe is that the pipe operator has a "*", I'd expect the commands to work largely the same, except that the extpipe is faster. When I see a command like 'echo $(message "[%s]" "hi") *| cat', I usually think of it as a two-phase operation: expansions/subcommands are expanded first, and then the final command is executed. In that model, the expansions would still be "in Eshell".
On the other hand, maybe there's enough practical use for passing the raw command string to `sh' that there should be a very simple way of invoking it (and without having to mess around with escaping interior quotes, as you would if you used `sh -c' manually). Maybe the parsing would be more robust if it used special sigils for the start/end of the external command? I suppose that's similar to your original proposal of using !! or || to introduce an external pipeline, so maybe it's not feasible to go this route.
Another possibility would be to keep the current behavior (or close to it), but to reconstruct the command to pass to `sh' during Eshell's rewrite phase. I'm not quite sure if that would actually work, but if it did, it would allow other argument parsers to run normally without extpipe needing to know what parsers to try. Perhaps if we kept around the substring that each argument parser consumed, it would be possible to reconstruct the relevant bits for extpipe's purposes?
More generally though, maybe there are really two different use cases?1) Eshell's built-in pipelines are slow because they go through Emacs buffers.
2) It would be convenient to invoke a whole command (or some large part of a command) using `sh' syntax.
For (1), Eshell could opportunistically use external pipelines without any special syntax. It should be possible to tell just by looking at the parsed command form if "foo | bar" connects two external processes on the same host, and then perform the appropriate rewrite to connect the processes efficiently (e.g. using `sh -c'). This would happen after expansion of variables/subcommands, so to the user it would work just like any other Eshell command, but it would be faster.
For (2), we'd need a convenient syntax for forwarding some command string to `sh'. Something like your proposed !! or || syntax, or maybe something to wrap around part of a command? (Or maybe an even something like an interactive `eshell-externalize' function that replaces the selected region with the correct `sh' invocation?)
And finally, sorry for bringing up these issues months after bug#46351. At the time, I didn't really understand the internals of Eshell, so I didn't have anything of substance to say then. Since then I've delved a bit *too* deep into Eshell's internals while trying to prove to myself that my implementation of Lisp function pipelines is sufficiently-flexible. :)
| [Prev in Thread] | Current Thread | [Next in Thread] |