bug#54154: 29.0.50; [PATCH] `sieve-manage-getscript' fails if script contains multibyte characters

[Kai Tetzlaff]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> The mail bounced with:
>
>   kai.tetzlaff@t-online.de
>     host mx03.t-online.de [194.25.134.73]
>     SMTP error from remote mail server after initial connection:
>     554 IP=95.216.78.240 - A problem occurred. (Ask your postmaster for help 
> or to contact tosa@rx.t-online.de to clarify.)

Sorry, not sure what is happening there. Using a different From:
address, now (hopefully - using the t-online address was accidental
anyway).

>>>    (with-current-buffer (or buffer (current-buffer))
>>>      (sieve-manage-send (format "GETSCRIPT \"%s\"" name))
>>> +    (set-buffer-multibyte nil)
>>>      (let ((script (sieve-manage-parse-string)))
>>> +      (set-buffer-multibyte t)
>>
>> Changing multibyteness in a buffer like this is (virtually) never the
>> right thing to do -- it usually leads to obscure breakages.
>>
>>> In general, it is also not clear to me why the response (or process)
>>> buffer needs to be multibyte enabled at all as it should only be used
>>> for the line/byte oriented protocol data. But the commit message of
>>> 8e16fb987df9b which introduced the multibyte handling states:
>>>
>>>     commit 8e16fb987df9b80b8328e9dbf80351a5f9d85bbb
>>>     Author: Albert Krewinkel <krewinkel@moltkeplatz.de>
>>>     Date:   2013-06-11 07:32:25 +0000
>>>     ...
>>>         * Enable Multibyte for SieveManage buffers: The parser won't 
>>> properly
>>>           handle umlauts and line endings unless multibyte is turned on in 
>>> the
>>>           process buffer.
>>>     ...
>>>
>>> so this was obviously done on purpose. I contacted Albert about this but
>>> he couldn't remember the details (it's been nearly 10 years).
>>
>> I don't see why this buffer should be multibyte, either.  The
>> communication with the server is done using bytes, not characters.  When
>> we need to have characters, we should decode the data and put it in a
>> multibyte buffer.
>>
>> So can you try to back out that commit and see whether it fixes the
>> problem instead?

Most of the referenced commit was about changes related to STARTTLS
handling. Here's the full commit message:

    lisp/gnus/sievel-manage.el: fully support STARTTLS, fix bit rot
    
    * Make sieve-manage-open work with STARTTLS: shorten stream managing
      functions by using open-protocol-stream to do most of the work.
      Has the nice benefit of enabling STARTTLS.

    * Remove unneeded functions and options: the following functions and
      options are neither in the API, nor called by any other function,
      so they are deleted:
      - sieve-manage-network-p
      - sieve-manage-network-open
      - sieve-manage-starttls-p
      - sieve-manage-starttls-open
      - sieve-manage-forward
      - sieve-manage-streams
      - sieve-manage-stream-alist

      The options could not be applied in a meaningful way anymore; they
      didn't happen to have much effect before.

    * Cosmetic changes and code clean-up

    * Enable Multibyte for SieveManage buffers: The parser won't properly
      handle umlauts and line endings unless multibyte is turned on in the
      process buffer.

    * Wait for capabilities after STARTTLS: following RFC5804, the server
      sends new capabilities after successfully establishing a TLS
      connection with the client.  The client should update the cached list
      of capabilities, but we just ignore the answer for now.

So just reverting it won't work. I will try to undo the parts relevant
to this issue.

For clarification: The original code before Alberts change was using
this macro (which seemingly contains an error in the doc string):

(defmacro sieve-manage-disable-multibyte ()
  "Enable multibyte in the current buffer."
  (unless (featurep 'xemacs)
    '(set-buffer-multibyte nil)))

to *disable* multibyte handling in the response/protocol buffer. If
using `set-buffer-multibyte' is not the right thing, what should be used
instead?