[Ifile-discuss] Re: Maildir format

[clemens fischer]

"William E. Kempf" <address@hidden>:

> That said, I would think the Maildir mechanism should be useful for
> manipulating files as well.
>
> 1) Read the mail message from $Maildir/cur/$msg and write modifications to
> $Maildir/tmp/$msg.
> 2) Remove $Maildir/cur/$msg.
> 3) Hardlink $Maildir/tmp/$msg to $Maildir/cur/$msg.
> 4) Remove $Maildir/tmp/$msg.

when moving files (messages) between maildir/ subdirectories, you have
to make sure that their names are unique.  it isn't customary to move
files back to tmp/:  arriving files are collected there until they're
safe, then renamed to new/.  MUAs move them from there to cur/ once
they've seen them.  it makes more sense to move files from the
receiving maildir/ to another one, especially since they aren't the
same messages any more!  they will have ifile-user-headers added.

> Any failure should leave you with a valid file in either
> $Maildir/cur/$msg or $Maildir/tmp/$msg (depending on when the
> failure occurred).  No chance for data loss.  And by employing a
> Hardlink instead of doing a file move and/or copy the process should
> be more efficient with less chance for failure.

it's wrong to assume anything about files unless one can track
everything that happened to them.  i'd design the operations like
this:  if a file arrives in new/ (or cur/, for that matter) the
operation has successfully finished, if not, it has to be repeated or
failed entirely.

> * Filtering of mailing lists.  I've discussed this on this list in
> the past.  I expect to read various (user selectable) headers, munge
> the contents in some manner to produce a "unique key" and use this
> key in a mechanism similar to ifile classifications to choose
> folders.  I can give much more detail on this if needed.

this interests me.  i have a system where new messages are filtered
through bogofilter and an awk script that takes care of tagging
mailinglists, HTML and so on.  the reason for this is that i need
simple, standardized headers for the various backends i might hand
messages over to.  currently it's gnus, but it has been procmail+
mutt, procmail+gnus and i even installed maildrop for its superior
qualities.  the mailinglists have been the biggest problem:  i want
one simple header "X-Mailinglist: <name|posting-address>".  i can spot
any mailinglist by its email-address appearing in either To: or Cc:,
but there may also be other names and it may appear on continuation
lines.  there's an RFC mentioning which headers a mailinglist should
have:

RFC 2369                  URLs as Meta-Syntax                  July 1998

3.1. List-Help

      The List-Help field is the most important of the header fields
      described in this document. It would be acceptable for a list
      manager to include only this field, since by definition it SHOULD
      direct the user to complete instructions for all other commands.
      Typically, the URL specified would request the help file, perhaps
      incorporating an HTML form for list commands, for the list, and
      alternatively provide access to an instructive website.

      Examples:

     List-Help: <mailto:address@hidden> (List Instructions)
     List-Help: <mailto:address@hidden>
     List-Help: <mailto:address@hidden> (Info about the list)
     List-Help: <http://www.host.com/list/>, <mailto:address@hidden>
     List-Help: <ftp://ftp.host.com/list.txt> (FTP),
         <mailto:address@hidden>

3.2. List-Unsubscribe

   The List-Unsubscribe field describes the command (preferably using
   mail) to directly unsubscribe the user (removing them from the list).

   Examples:

     List-Unsubscribe: <mailto:address@hidden>
     List-Unsubscribe: (Use this command to get off the list)
         <mailto:address@hidden>
     List-Unsubscribe: <mailto:address@hidden>
     List-Unsubscribe: <http://www.host.com/list.cgi?cmd=unsub&lst=list>,
         <mailto:address@hidden>

but most lists don't honor RFC-2369.  i had to throw big regular
expressions at the raw messages.

> * The above functionalities should be scripted (scripting language
> TBD), and callable by such programs as procmail for initial mail
> delivery.  My personal interest is only for delivery to Maildir, but
> a reusable system would handle the other formats as well.

you could achieve this by careful factoring of tasks.  i currently
have:

1.  SMTP receiver with simple rules (mailfront)
2.  pre-queue filtering for spam and reject/accept patterns
3.  the queue and local delivery system, which is a black box here
4.  mail splitting into folders.

i want spam-rejection before enqueing emails in the faint hope that
spamware will remove my email from their target-lists on receiving 5xx
SMTP codes, but it seems the majority of spampumps is just that:
pumps.  it will deliver to dead addresses for years and never note.

you can see that this system is ad-hoc.  the steps you'd have to
refine is local delivery and splitting, because you'd have to provide
for autoresponders, virus-scanners and whatever people may dream up.
one of the most flexible systems i ever learned about is the qmail
local delivery mechanism, which gives users many of the possibilities
normaly only admins have.

a good scripting language might be lua.org.  it can easily be embedded
later into larger chunks of code writen in C.

> * Mis-delivered mail should simply need to be moved, by any means,
> to the proper location, where a daemon process will relearn how the
> message should have been delivered.  This way, you can use any mail
> client to move the message, and the system will relearn how to
> filter and do so correctly in the future.

then you should generalize this deamon to handle all ifiling as part
of the delivery process.  this would save you from locking issues and
tallying ifiles accuracy.

  clemens