bug#53941: 27.2; socks + tor dont work with https

[J.P.]

v4. Include a minimal (hacky[1]) url-gw integration.

I'm now slightly of the opinion that offering no interface whatsoever is
probably too stark an approach. Ignoring the three Tor-related patches
for now, it seems that even without proper url.el integration[2], we can
still try to ensure that for most use cases, no unnecessary hackery need
apply.

Another issue is whether to address the questionable top-level advising
going on with `open-network-streams', since we're already refactoring
all the functions it affects. Assuming users exist who still have
`socks-override-functions' non-nil at load time, would it make sense to
warn them more fervently than would be done for a normal deprecation?
The thinking is that folks may be relying on this for things like
bypassing firewalls at work (and could therefore get dinged more than
usual just for upgrading Emacs).

As a start, I figured we could try and determine exactly why this
(perhaps somewhat ill-considered) top-level advising was ever instituted
in the first place[3]. AFAICT, it was mainly intended to

1. allow libraries calling `open-network-stream' (and unaware of
   `socks-open-network-stream') to proxy transparently[4]

2. guard the tunneled protocol from being accidentally subject to a
   recommencing of the SOCKS dialog

If anyone has better ideas, please share. Thanks.


Notes
~~~~~

[1] The second patch is new and a bit of an ugly hack. It has to do with
    this change from a while back:

      Do not set `url-gateway-method' in `url-https'
      commit 98c58df832975b01287ef749dd5235199d4cd431
      Sun Sep 28 20:00:54 2014 +0200

    which made it impossible for `url-gateway-method' to be respected by
    `url-open-stream' when called by `url-https'. But rather than
    undoing the offending portions out of hand, it might be nicer to
    first figure out how url-proxy.el is supposed to work and maybe get
    it and `url-retrieve-internal' (and `url-https') more in sync and
    sensitive to `url-gateway-method'.

[2] If we do end up with a proper url.el solution, it might then make
    more sense to emphasize the fact that `socks-open-network-stream' is
    really mostly about catering to url-gw (which it is). If that's
    agreeable, we could rename the following like so:

               socks-open-network-stream -> socks-url-open
        socks-open-network-stream-legacy -> socks-open-network-stream
      socks-open-network-stream-function -> socks-url-open-function

[3] A summary of the advice-based behavior triggered by
    `socks-override-functions', assuming `socks-find-route' returns
    non-nil:

    | topmost function invoked  | o-n-s advised | s-o-f | proxied |
    |---------------------------+---------------+-------+---------|
    | socks-open-network-stream | nil           | t     | yes     |
    | socks-open-network-stream | nil           | nil   | yes     |
    | socks-open-network-stream | t             | t     | yes     |
    | socks-open-network-stream | t             | nil   | yes     |
    | open-network-stream       | nil           | t     | no      |
    | open-network-stream       | nil           | nil   | no      |
    | open-network-stream       | t             | nil   | no      |
    | open-network-stream       | t             | t     | yes     |

    o-n-s: open-network-stream
    s-o-f: socks-override-functions

[4] It could be argued that the 2014 commit in [1] converted gw into one
    such library insofar as `url-https' is concerned.

0000-NOT-A-PATCH-v3-v4.diff
Description: Text Data

0001-Simplify-network-stream-opener-in-socks.el.patch
Description: Text Data

0002-lisp-url-url-gw.el-url-open-stream-Honor-socks-gatew.patch
Description: Text Data

0003-Fix-string-encoding-bug-in-socks-tests.patch
Description: Text Data

0004-Add-support-for-SOCKS-4a.patch
Description: Text Data

0005-Support-SOCKS-resolve-extension.patch
Description: Text Data

0006-POC-Demo-SOCKS-resolve-with-HTTPS.patch
Description: Text Data