Re: inc and non-compliant long lines redux

[Andy Bradford]

Thus said Ken Hornstein on Tue, 15 Nov 2022 15:44:59 -0500:

> My reading of the RFC is that the 512 limit is _just_ for responses to
> commands. For the  response to something like RETR it  punts it off to
> RFC 822, which it seems like people are ignoring anyway.

This is also how I interpreted  it. I left the command processing alone.
The  changes  I made  only  kick  in after  the  RETR  command has  been
successfully started.

Also, I  recognize Ralph's statement  that this  is not so  much "people
ignoring" the  RFCs, but really  just one "sender".  In my 23+  years of
using nmh  for email this is  the only "sender" I  have ever encountered
that generates such messages.  So it does seem a bit  drastic to take on
the added risk of "fixing" nmh. :-)

> I used netsec_readline() because it handles the case of stripping line
> endings so we could easily convert CR-LF to Unix style LF-end messages.
> But really we could simply use the netsec_read() and just convert every
> CR-LF pair to a simple LF. 

I originally looked  at netsec_read() as an alternative,  but thought it
would be  easier (maybe not  simpler) to use netsec_readline()  since it
already  had logic  for dealing  with CR/LF.  And the  more I  looked at
using  netsec_read(), the  more it  seemed like  I would  be reinventing
netsec_readline().  Perhaps this  direction was  misguided and  I should
have  stuck  with  netsec_read().  Instead, what  pop_retr()  (and  more
specifically  traverse()) could  do is  after successfully  starting the
RETR command, it  could just switch to using  netsec_read() and handling
any  CR/LF encountered  until  it sees  the  CRLF.CRLF indicator.  There
wouldn't even  be any need for  inc to distinguish between  "header" and
"body"  in this  mode, just  read() data  until we  get CRLF.CRLF,  then
return so DELE and QUIT can be sent.

For  now,  I'm dealing  with  another  synchronization problem  with  my
original  patch  (as  reported  in a  different  email).  Basically  the
CRLF.CRLF is getting flagged early in these lines:

ng> <the outputfile of pdftotext>" and render this text in the browser=
. Optionally the output file can be stored and rendered in the reposit=


The POP server dutifully sends the lines as:

ng> <the outputfile of pdftotext>" and render this text in the browser=
.. Optionally the output file can be stored and rendered in the reposit=

However, the way the  patch reads chunks of data must  be causing the ..
to be split into  two parts, and so it sees the first  one and thinks it
has arrived at  the end of the  RETR command. The message  is written to
disk excluding the dot and everything that follows.

Anyway, I've reverted my changes locally  for now until I can figure out
how  to make  sure  that the  ..  are not  split  while reading  chunks.
I  imagine  that  this  same  challenge will  be  had  even  when  using
netsec_read() instead.

Andy