bug#44307: 27.1; UTF-8 parts transferred as 8bit in multipart messages fail to decode

[Alexandre Duret-Lutz]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Alexandre Duret-Lutz <adl@lrde.epita.fr> writes:
>
>> Here is the file.  But see by previous message from today.  Reading this
>> with nndoc will appear bogus with emacs 27.1, but not with emacs 28.
>> However if there is a way to read that mail with nnmaildir, you should
>> see the issue with both versions.
>
> Thanks for the test file (and the analysis of the differences between
> 27.1 and 28 here).  I'll try to work on fix for this in Emacs 27
> tomorrow -- I'm not sure the simple fix you outlined won't have adverse
> effects if the part in question is binary (i.e., an image or the like).

Sorry, you can ignore my simple "fix": I have now found a case that it
breaks.

I was testing mainly on utf-8 messages.  But I've now looked (through
nnmaildir) at a mail containing a part encoded as follow:

Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit

However the rendering was garbled.  Looking ag the output of "C-u g" I
can see that the buffer contains utf-8 code (I'm guessing because that
the windows-1252 representation was decoded when nnmaildir read the mail
from file), and that will make no sense once Gnus attempts to decode
that in windows-1252.

Without my incorrect patch, displaying such an email would appear to
work (by luck?), because when mm-with-part inserts the multibyte buffer
into the unibyte buffer, the utf-8 characters are somehow converted to
their windows-1252 equivalent encoding.

I guess the real issue is that mmaildir does not correctly deal with
multibyte files et may return multibyte buffer, just like nndoc used to.
-- 
Alexandre Duret-Lutz