bug#36350: [2.2.5] ‘read-headers’ blocks, thereby breaking web servers

[Dan Frumin]

I believe that `(lookahead-char port)` really blocks when the client has 
finished sending the request and there is no more data from `port` to consume.
If I understand it correctly, then per HTTP/1.1 [1] the request ends with CRLF at the last line, and then comes the message. So I we have read an 
empty string, then we shouldn't proceed with further lookaheads.

Specifically, the following code works out for me:


(define (read-header-line port)
  "Read an HTTP header line, including any continuation lines, and
return the combined string without its final CRLF or LF.  Raise a
'bad-header' exception if the line does not end in CRLF or LF, or if EOF
is reached."
  (format #t "Reading header line now: ")
  (match (%read-line port)
    (((? string? line) . #\newline)
     ;; '%read-line' does not consider #\return a delimiter; so if it's
     ;; there, remove it.  We are more tolerant than the RFC in that we
     ;; tolerate LF-only endings.
     (let ((line (if (string-suffix? "\r" line)
                     (string-drop-right line 1)
                     line)))
       ;; If the next character is a space or tab, then there's at least
       ;; one continuation line.  Read the continuation lines by calling
       ;; 'read-header-line' recursively, and append them to this header
       ;; line, folding the leading spaces and tabs to a single space.
       (if (and (not (string-null? line))
                (space-or-tab? (lookahead-char port)))
           (string-append line " " (string-trim (read-header-line port)
                                                spaces-and-tabs))
           line)))
    ((line . _)                                ;EOF or missing delimiter
     (bad-header 'read-header-line line))))

Moreover, the continuation lines in general have been deprecated: [2].
I have to say I would be in favor of removing support for continuation lines in 
general.

Best regards,
-Dan


[1]: https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html
[2]: https://tools.ietf.org/html/rfc7230#section-3.2.4