bug#33780: network-stream.el: network-stream-certificate always returns nil

[Robert Pluim]

Following discussion on emacs-devel on how to do this, latest version
of patch attached. This maintains backwards compatibility for
open-gnutls-stream (assuming I haven't screwed up the checks), and
updates the relevant documentation.

>From 6bdf3d94dc83e79394d109486f68810ef9f4b373 Mon Sep 17 00:00:00 2001
From: Robert Pluim <rpluim@gmail.com>
Date: Fri, 21 Dec 2018 11:58:00 +0100
Subject: [PATCH] Check for client certificates when using GnuTLS
To: emacs-devel@gnu.org

This fixes Bug#33780, and extends the documentation to describe how to
enable use of client certificates.

* lisp/net/network-stream.el (network-stream-certificate): Correct
order of parameters to plist-get.
(network-stream-open-tls): Pass all received parameters to
open-gnutls-stream as plist, not just :nowait.

* lisp/net/gnutls.el (open-gnutls-stream): Change optional nowait arg
to be plist.  Derive nowait and client certificate(s) and keys(s) from
plist (maybe via auth-source) and pass to gnutls-boot-parameters and
gnutls-negotiate.
(network-stream-certificate): Add declare-function form for it.

* doc/misc/auth.texi (Help for users): Describe format to use for
client key/cert specification.

* doc/misc/emacs-gnutls.texi (Help For Developers): Describe usage of
optional plist argument.  Add crossreference to description of
.authinfo format for client key/cert specification.

* etc/NEWS: Describe new client certificate functionality for
  'open-network-stream'.
---
 doc/misc/auth.texi         |  9 ++++++
 doc/misc/emacs-gnutls.texi | 38 ++++++++++++++++++-------
 etc/NEWS                   |  7 +++++
 lisp/net/gnutls.el         | 57 +++++++++++++++++++++++++-------------
 lisp/net/network-stream.el |  4 +--
 5 files changed, 84 insertions(+), 31 deletions(-)

diff --git a/doc/misc/auth.texi b/doc/misc/auth.texi
index 495d9f53e1..ddfeabcba7 100644
--- a/doc/misc/auth.texi
+++ b/doc/misc/auth.texi
@@ -109,6 +109,15 @@ Help for users
 @code{auth-source-search} queries.  You can also use @code{login} and
 @code{account}.
 
+You can also use this file to specify client certificates to use when
+setting up TLS connections.  The format is:
+@example
+machine @var{mymachine} port @var{myport} key @var{key} cert @var{cert}
+@end example
+
+@var{key} and @var{cert} are filenames containing the key and
+certificate to use respectively.
+
 You can use spaces inside a password or other token by surrounding the
 token with either single or double quotes.
 
diff --git a/doc/misc/emacs-gnutls.texi b/doc/misc/emacs-gnutls.texi
index aae583c641..0e2a9764a1 100644
--- a/doc/misc/emacs-gnutls.texi
+++ b/doc/misc/emacs-gnutls.texi
@@ -179,17 +179,35 @@ Help For Developers
 You should not have to use the @file{gnutls.el} functions directly.
 But you can test them with @code{open-gnutls-stream}.
 
-@defun open-gnutls-stream name buffer host service &optional nowait
+@defun open-gnutls-stream name buffer host service &optional parameters
 This function creates a buffer connected to a specific @var{host} and
-@var{service} (port number or service name).  The parameters and their
-syntax are the same as those given to @code{open-network-stream}
-(@pxref{Network,, Network Connections, elisp, The Emacs Lisp Reference
-Manual}).  The connection process is called @var{name} (made unique if
-necessary).  This function returns the connection process.
-
-The @var{nowait} parameter means that the socket should be
-asynchronous, and the connection process will be returned to the
-caller before TLS negotiation has happened.
+@var{service} (port number or service name).  The mandatory arguments
+and their syntax are the same as those given to
+@code{open-network-stream} (@pxref{Network,, Network Connections,
+elisp, The Emacs Lisp Reference Manual}).  The connection process is
+called @var{name} (made unique if necessary).  This function returns
+the connection process.
+
+The optional @var{parameters} argument is a list of keywords and
+values.  The only keywords which currently have any effect are
+@code{:client-certificate} and @code{:nowait}.
+
+Passing @code{:client certificate t} triggers looking up of client
+certificates matching @var{host} and @var{service} using the
+'auth-source' library.  Any resulting client certificates are passed
+down to the lower TLS layers.  The format used by @file{.authinfo} to
+specify the per-server keys is described in @xref{Help for
+users,,auth-source, auth, Emacs auth-source Library}.
+
+Passing @code{:nowait t} means that the socket should be asynchronous,
+and the connection process will be returned to the caller before TLS
+negotiation has happened.
+
+For historical reasons @var{parameters} can also be a symbol, which is
+interpreted the same as passing a list containing @code{:nowait} and
+the value of that symbol.
+
+Example calls:
 
 @lisp
 ;; open a HTTPS connection
diff --git a/etc/NEWS b/etc/NEWS
index 3670ab5bf4..43997f8418 100644
--- a/etc/NEWS
+++ b/etc/NEWS
@@ -199,6 +199,13 @@ issued), you can either set 
'network-security-protocol-checks' to nil,
 or adjust the elements in that variable to only happen on the 'high'
 security level (assuming you use the 'medium' level).
 
++++
+** Native GnuTLS connections can now use client certificates.
+Previously, this support was only available when using the external
+gnutls-cli command.  Call 'open-network-stream' with
+':client-certificate t' to trigger looking up of per-server
+certificates via 'auth-source'.
+
 +++
 ** New function 'fill-polish-nobreak-p', to be used in 
'fill-nobreak-predicate'.
 It blocks line breaking after a one-letter word, also in the case when
diff --git a/lisp/net/gnutls.el b/lisp/net/gnutls.el
index 78ac3fe35b..dae208e926 100644
--- a/lisp/net/gnutls.el
+++ b/lisp/net/gnutls.el
@@ -38,6 +38,9 @@
 (require 'cl-lib)
 (require 'puny)
 
+(declare-function network-stream-certificate "network-stream"
+                  (host service parameters))
+
 (defgroup gnutls nil
   "Emacs interface to the GnuTLS library."
   :version "24.1"
@@ -138,23 +141,25 @@ gnutls-min-prime-bits
                  (integer :tag "Number of bits" 512))
   :group 'gnutls)
 
-(defun open-gnutls-stream (name buffer host service &optional nowait)
+(defun open-gnutls-stream (name buffer host service &optional parameters)
   "Open a SSL/TLS connection for a service to a host.
 Returns a subprocess-object to represent the connection.
 Input and output work as for subprocesses; `delete-process' closes it.
 Args are NAME BUFFER HOST SERVICE.
 NAME is name for process.  It is modified if necessary to make it unique.
 BUFFER is the buffer (or `buffer-name') to associate with the process.
- Process output goes at end of that buffer, unless you specify
- a filter function to handle the output.
- BUFFER may be also nil, meaning that this process is not associated
- with any buffer
-Third arg is name of the host to connect to, or its IP address.
-Fourth arg SERVICE is name of the service desired, or an integer
+Process output goes at end of that buffer, unless you specify a
+filter function to handle the output.  BUFFER may be also nil,
+meaning that this process is not associated with any buffer
+Third arg HOST is the name of the host to connect to, or its IP address.
+Fourth arg SERVICE is the name of the service desired, or an integer
 specifying a port number to connect to.
-Fifth arg NOWAIT (which is optional) means that the socket should
-be opened asynchronously.  The connection process will be
-returned to the caller before TLS negotiation has happened.
+Fifth arg PARAMETERS is an optional list of keyword/value pairs.
+Only :client-certificate and :nowait keywords are recognized, and
+have the same meaning as for `open-network-stream'.
+For historical reasons PARAMETERS can also be a symbol, which is
+interpreted the same as passing a list containing :nowait and the
+value of that symbol.
 
 Usage example:
 
@@ -168,19 +173,33 @@ open-gnutls-stream
 documentation for the specific parameters you can use to open a
 GnuTLS connection, including specifying the credential type,
 trust and key files, and priority string."
-  (let ((process (open-network-stream
-                  name buffer host service
-                  :nowait nowait
-                  :tls-parameters
-                  (and nowait
-                       (cons 'gnutls-x509pki
-                             (gnutls-boot-parameters
-                              :type 'gnutls-x509pki
-                              :hostname (puny-encode-domain host)))))))
+  (let* ((parameters
+          (cond ((symbolp parameters)
+                 (list :nowait parameters))
+                ((not (cl-evenp (length parameters)))
+                 (error "Malformed keyword list"))
+                ((consp parameters)
+                 parameters)
+                (t
+                 (error "Unknown parameter type"))))
+         (cert (network-stream-certificate host service parameters))
+         (keylist (and cert (list cert)))
+         (nowait (plist-get parameters :nowait))
+         (process (open-network-stream
+                   name buffer host service
+                   :nowait nowait
+                   :tls-parameters
+                   (and nowait
+                        (cons 'gnutls-x509pki
+                              (gnutls-boot-parameters
+                               :type 'gnutls-x509pki
+                               :keylist keylist
+                               :hostname (puny-encode-domain host)))))))
     (if nowait
         process
       (gnutls-negotiate :process process
                         :type 'gnutls-x509pki
+                        :keylist keylist
                         :hostname (puny-encode-domain host)))))
 
 (define-error 'gnutls-error "GnuTLS error")
diff --git a/lisp/net/network-stream.el b/lisp/net/network-stream.el
index 98b2033790..1723931c67 100644
--- a/lisp/net/network-stream.el
+++ b/lisp/net/network-stream.el
@@ -196,7 +196,7 @@ open-network-stream
          (car result))))))
 
 (defun network-stream-certificate (host service parameters)
-  (let ((spec (plist-get :client-certificate parameters)))
+  (let ((spec (plist-get parameters :client-certificate)))
     (cond
      ((listp spec)
       ;; Either nil or a list with a key/certificate pair.
@@ -389,7 +389,7 @@ network-stream-open-tls
           (stream
             (if (gnutls-available-p)
                 (open-gnutls-stream name buffer host service
-                                    (plist-get parameters :nowait))
+                                    parameters)
               (require 'tls)
               (open-tls-stream name buffer host service)))
           (eoc (plist-get parameters :end-of-command)))
-- 
2.19.1.816.gcd69ec8cde.dirty