Re: Shell-quoting issue for sshx/scpx on MS Windows

[Jim Porter]

On Fri, May 7, 2021 at 8:26 AM Michael Albinus <michael.albinus@gmx.de> wrote:
>
> Jim Porter <jporterbugs@gmail.com> writes:
>
> > Would it be possible to enable your fix only for the Win32 version? It
> > looks like you can tell them apart by checking `ssh -V'. The Win32
> > version looks like:
> >
> >   OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
> >
> > and the MSYS2 version looks like:
> >
> >   OpenSSH_8.3p1, OpenSSL 1.1.1g  21 Apr 2020
>
> I have resisted for more than 20 years I'm in Tramp development, to
> check version numbers of tools. This is a box of Pandora, because you
> must keep such a check *forever*. You don't know whether somebody still
> uses this special super-old version. Think about the number of ssh
> versions which have appeared last 20 years ...

That makes sense. I wonder if there's another way to detect the
difference between the Win32 version and the MSYS2 version. It looks
like the MSYS2 version works like "normal SSH", so Tramp wouldn't need
any special handling for it. The Win32 version has bad filename
handling and lacks support for ControlMaster (trying to use it with
the Win32 OpenSSH causes Tramp to hang while connecting).

> Instead, I check for features. For example, I check whether "scp -T"
> returns the error message that the "-T" argument is unknown.

It would be nice if we could find a way like this to detect whether
ControlMaster works (Tramp 2.5.0.4 just disables it on Windows, but it
appears to work with the MSYS2 port). Maybe there's something
different in the output that we could detect.

I'm not sure how to go about detecting the differences in how paths
are handled between the two versions though. It seems like the Win32
port's behavior is just a bug; I can't think of any good reason why it
should work differently from the MSYS2 port, which works just fine
with Tramp 2.5.0.4.

If there's no way to detect this automatically, maybe it would make
sense to treat the Win32 port of OpenSSH as a different Tramp method
("wssh" and "wscp", for example)? Or maybe just a flag users can set
to enable/disable Win32 compatibility mode.

> Well, I've debugged it further. It looks, like Emacs
> shell-quote-argument does not work proper on MS Windows, whyever. It
> produces the spurious caret char (^), which I don't understand.

The caret is the escape character for the Windows Console. I've never
been able to fully understand how it works in all cases, but it does
have enough strange behaviors[1] that I just try to avoid it if I can.
In theory though, using caret to escape things should work, but it
seems that the escapes aren't getting transformed in this case,
resulting in errors like "scp: ^/home/jim/a: No such file or
directory". Maybe that has something to do with how scp is getting
invoked by Tramp? That escape is correctly removed before reaching the
server if I run it from a console.

> I have run my test suite almost successfully, testing the file names
>
> --8<---------------cut here---------------start------------->8---
> foo bar baz
> @foo@bar@baz@
> -foo-bar-baz-
> %foo%bar%baz%
> &foo&bar&baz&
> 'foo'bar'baz'
> #foo~bar#baz~
> ;foo;bar;baz;
> (foo)bar(baz)
> [foo]bar[baz]
> {foo}bar{baz}
> --8<---------------cut here---------------end--------------->8---
>
> This are not all special characters available, but not bad for now :-)
> I've committed everything, it would be great if you could verify it on
> your system with MS OpenSSH 8.x and MSYS2 OpenSSH 8.x - the latter I
> haven't installed.

Here are the new results from my manual tests. For the cases I'm
trying, the results are the same as before; specifically, filenames
with double-quotes still fail everywhere. That's consistent with the
test cases you ran above. Strangely, I see different errors when I run
these commands manually in a command prompt. I'm not sure why that's
happening, though; probably for the same reason that the caret was
getting passed to the server in my previous tests...

I'll see if I can get the automated test suites running today or tomorrow.

- Jim

---------- Begin Testing Results ----------

tramp-dev2  ssh-win32-7.7  spaces    FAIL
  scp -p -q -r jim@server:"/home/jim/foo bar.txt" [[dest]]
  scp: /home/jim/foo: No such file or directory

tramp-dev2  ssh-win32-7.7  quotes    FAIL
  scp -p -q -r jim@server:"/home/jim/a ""quoted"" file.txt"
  scp: /home/jim/a: No such file or directory
  scp: quoted: No such file or directory
  scp: file.txt: No such file or directory


tramp-dev2  ssh-win32-8.1  spaces    ok
  scp -p -T -q -r jim@server:"/home/jim/foo bar.txt" [[dest]]

tramp-dev2  ssh-win32-8.1  quotes    FAIL
  scp -p -T -q -r jim@server:"/home/jim/a ""quoted"" file.txt" [[dest]]
  scp: /home/jim/a quoted file.txt: No such file or directory


tramp-dev2  ssh-msys2-8.3  spaces    ok
  scp -p -T -q -r jim@server:"/home/jim/foo bar.txt" [[dest]]

tramp-dev2  ssh-msys2-8.1  quotes    FAIL
  scp -p -T -q -r jim@server:"/home/jim/a ""quoted"" file.txt" [[dest]]
  scp: /home/jim/a quoted file.txt: No such file or directory

---------- End Testing Results ---------

[1] https://stackoverflow.com/questions/11193691/batch-escaping-with-caret