Re: Shell-quoting issue for sshx/scpx on MS Windows

[Michael Albinus]

Jim Porter <jporterbugs@gmail.com> writes:

Hi Jim,

> Ok, I've tested in all the combinations I can, with the results in
> the table below. "tramp-dev" is the latest Git revision, whereas
> "tramp-2.5.0.4" is the version on GNU ELPA. I tested against the Win32
> versions of OpenSSH (both v7.7 and v8.1) as well as the MSYS2 version
> (I have v8.3). The "spaces" test is attempting to open `~/foo
> bar.txt', and "quotes" is attempting to open `~/a "quoted" file.txt'.
> I've also included the scp command, and the error message, if any.

Thanks a lot!

> (I saved logs with `tramp-verbose' set to 6, but they're pretty large
> even compressed, and I'm not sure they'd be that useful anyway. If you
> want me to send them over, just let me know.)

No, I don't need them.

> From my tests, the best version combination is Tramp 2.5.0.4 with the
> MSYS2 port of OpenSSH: both files open correctly. However, the
> development version of Tramp *is* an improvement for anyone with
> Win32-OpenSSH v8.1.
>
> Would it be possible to enable your fix only for the Win32 version? It
> looks like you can tell them apart by checking `ssh -V'. The Win32
> version looks like:
>
>   OpenSSH_for_Windows_7.7p1, LibreSSL 2.6.5
>
> and the MSYS2 version looks like:
>
>   OpenSSH_8.3p1, OpenSSL 1.1.1g  21 Apr 2020

I have resisted for more than 20 years I'm in Tramp development, to
check version numbers of tools. This is a box of Pandora, because you
must keep such a check *forever*. You don't know whether somebody still
uses this special super-old version. Think about the number of ssh
versions which have appeared last 20 years ...

Instead, I check for features. For example, I check whether "scp -T"
returns the error message that the "-T" argument is unknown.

> I mentioned it in the other thread, but I'm starting to think that the
> MSYS2 port of OpenSSH is the best way to use SSH on MS Windows. It
> seems that it even supports the ControlMaster option, so Tramp could
> use it on MS Windows, and only disable that feature when it detects
> "OpenSSH_for_Windows".

Well, I've debugged it further. It looks, like Emacs
shell-quote-argument does not work proper on MS Windows, whyever. It
produces the spurious caret char (^), which I don't understand.

So I have replaced the call to this function by
tramp-unquote-shell-quote-argument, and I have improved that function a
little bit. And voilà, special characters are passed when using the scpx
Tramp method, with MS OpenSSH. I have run my test suite almost
successfully, testing the file names

--8<---------------cut here---------------start------------->8---
foo bar baz
@foo@bar@baz@
-foo-bar-baz-
%foo%bar%baz%
&foo&bar&baz&
'foo'bar'baz'
#foo~bar#baz~
;foo;bar;baz;
(foo)bar(baz)
[foo]bar[baz]
{foo}bar{baz}
--8<---------------cut here---------------end--------------->8---

This are not all special characters available, but not bad for now :-)
I've committed everything, it would be great if you could verify it on
your system with MS OpenSSH 8.x and MSYS2 OpenSSH 8.x - the latter I
haven't installed.

I have also checked that my test suite still works with pscp. It
doesn't, of course, so my recent changes have broken that case :-(
But this has time to fix until we know better how to handle scp, we
*know* that pscp is working.

If you want to use the test suite yourself, it is in the
test/subdirectory. There is a README file, which shall explain what to
do (if not, pls rant).

Since I haven't installed the make tool, I run the following commands instead:

--8<---------------cut here---------------start------------->8---
set REMOTE_TEMPORARY_FILE_DIRECTORY=/scpx:gandalf:tmp
"%ProgramFiles%\Emacs\emacs-28.0.50-snapshot\bin\emacs.exe" -Q -batch -L 
X:\src\tramp\lisp -l X:\src\tramp\test\tramp-tests.el -f 
ert-run-tests-batch-and-exit
--8<---------------cut here---------------end--------------->8---

gandalf is my remote machine, equipped with Fedora 34.

> ---------- Begin Testing Results ----------
>
> tramp-dev  ssh-win32-7.7  spaces    FAIL
>   scp -p -q -r jim@server:"/home/jim/foo bar.txt" [[dest]]
>   scp: /home/jim/foo: No such file or directory
> tramp-dev  ssh-win32-7.7  quotes    FAIL
>   scp -p -q -r jim@server:^"/home/jim/a \^"quoted\^" file.txt^" [[dest]]
>   scp: ^/home/jim/a: No such file or directory
>
> tramp-dev  ssh-win32-8.1  spaces    ok
>   scp -p -T -q -r jim@server:"/home/jim/foo bar.txt" [[dest]]
> tramp-dev  ssh-win32-8.1  quotes    FAIL
>   scp -p -T -q -r jim@server:^"/home/jim/a \^"quoted\^" file.txt^" [[dest]]
>   scp: ^/home/jim/a /^quoted/^ file.txt^: No such file or directory
>
> tramp-dev  ssh-msys2-8.3  spaces    ok
>   scp -p -T -q -r jim@server:"/home/jim/foo bar.txt" [[dest]]
> tramp-dev  ssh-msys2-8.3  quotes    FAIL
>   scp -p -T -q -r jim@server:^"/home/jim/a \^"quoted\^" file.txt^" [[dest]]
>   scp: ^/home/jim/a \^quoted^ file.txt^: No such file or directory
>
>
> tramp-2.5.0.4  ssh-win32-7.7  spaces    FAIL
>   scp -p -q -r jim@server:/home/jim/foo\ bar.txt [[dest]]
>   scp: /home/jim/foo: No such file or directory
> tramp-2.5.0.4  ssh-win32-7.7  quotes    FAIL
>   scp -p -q -r jim@server:/home/jim/a\ \"quoted\"\ file.txt [[dest]]
>   scp: ^/home/jim/a: No such file or directory
>
> tramp-2.5.0.4  ssh-win32-8.1  spaces    FAIL
>   scp -p -q -r jim@server:/home/jim/foo\ bar.txt [[dest]]
>   scp: /home/jim/foo: No such file or directory
> tramp-2.5.0.4  ssh-win32-8.1  quotes    FAIL
>   scp -p -q -r jim@server:/home/jim/a\ \"quoted\"\ file.txt [[dest]]
>   scp: ^/home/jim/a: No such file or directory
>
> tramp-2.5.0.4  ssh-msys2-8.3  spaces    ok
>   scp -p -q -r jim@server:/home/jim/foo\ bar.txt [[dest]]
> tramp-2.5.0.4  ssh-msys2-8.3  quotes    ok
>   scp -p -q -r jim@server:/home/jim/a\ \"quoted\"\ file.txt [[dest]]
>
> ---------- End Testing Results ----------

Great table! If we use the test suite, we have even more results to
compare. However, I don't expect it will work with ssh-win32-7.7, but
who knows.

> - Jim

Best regards, Michael.