Present Perfect


Picture Gallery
Present Perfect

How do you manage mailing lists?

Filed under: Question,sysadmin — Thomas @ 17:00


Every new year is a time of cleaning. After getting back to Inbox 0, my next target is my mailing list subscriptions.

It must be something psychological, but I cannot bring myself to unsubscribe from some of these mailing lists. I don't check on them daily, but once in a while it's darn useful to search through my local copy of mails on, say, selinux, and find solutions for a problem I'm having.

However, all this mailing list mail brings me a lot of headache. My email client is slow, and I would want it to be fast for the real mail I'm getting (from actual people, needing actual work). It's hard to track the mails that matter - all my list mail gets put into folders automatically with some procmail magic, but it also means that some of the things I should be paying more attention to are just another bold folder in Evolution somewhere down the mail tree. And lastly, the server where I host my mail shared with friends gets too much traffic, and syncing 3 different evolutions over IMAP with it is a big part of the burden.

I vastly prefered the newsreader model of old, and I think the de facto standard of mailing lists really is a mistake. But I'm not sure what to replace it with.

What I want:

  1. have selected mailing list archives be available on my machines, locally
  2. have them synced/updated automatically
  3. have them out of the way of my normal mail usage unless when I need them

I've been considering getting a separate email account just for email lists for this purpose, although I don't look forward much to having to change all my subscriptions, and would first like to hear from other people how this approach works out for them.

There used to be a push towards web-based mailing list subscriptions, but I don't know if anyone is really seriously using that, and I would like to have the option of reading these mailing list archives offline.

How do you separate your 'real' mail from your mailing list mail? How do you handle them?


  1. kmail’s filtering works fine for me — I’ve got a huge number of filters, but only the spamasassin step is slow. For the rest, all mail to mailing lists gets filtered by mailing list id into folders, if it’s something that a reply to me it gets flagged as important, some mails that contian key prhases go into a mailinglist/important folder and finally commit mails are filtered into many folders making it easy to keep track of many projects. I’ve got about 8GB of mail archive currently, and search is still really fast. I’m not sure about the number of mailing lists I’m subscribed to, it must about a hundred.

    Comment by Boudewijn Rempt — 2012-01-02 @ 17:38

  2. I have a gmail account *just* for mailing lists, yes there’s a bit of pain tranferring the subscriptions over (if you belong to several lists on a single server you can often tranfer them en-masse) and setting up automatic labelling per list, but it works for me …

    Just cleared it from 5GB down to 1GB.

    Comment by Andy Burns — 2012-01-02 @ 17:53

  3. I was thinking about moving to gmail as well just for lists, but I want to not lock myself out of possibly switching to gmail for my real people mail too. Also, I have more than 1GB of list mails I’m interested in, and I have found that gmail’s offline sync feature is pretty unreliable…

    Comment by Thomas — 2012-01-02 @ 18:31

  4. Read mailing lists though GMane using a NNTP news reader. Most of the lists you’re interested in are already subscribed, and it is easy to request the GMane admins to add a new subscription.

    Comment by Stephen Kelly — 2012-01-02 @ 17:58

  5. I use gmane.org along with a leafnode server. Then gnus (which is incredibly painful on slow connections due to being single threaded) works absolutely fine for me. (And switch off mailing list delivery, obviously)

    Comment by Rupert Swarbrick — 2012-01-02 @ 18:06

  6. I didn’t know about leafnode. Why do you use leafnode? Only because gnus doesn’t do offline reading?

    Comment by Thomas — 2012-01-02 @ 18:30

  7. I use procmail to filter all of my mail server-side. Each mailing list gets its own folder and I can use subscriptions to only read the lists I’m interested in. I’m almost always online, so I don’t need to sync the contents of emails to my laptop, but I do use mutt’s header cache so that I sync the metadata and don’t need to pull that over each time.

    Comment by brian m. carlson — 2012-01-02 @ 18:07

  8. I do the same with procmail. Yeah, I could use the IMAP subscription feature, but I’m not a big fan of having to do that manually on each client the whole time. Since I now have as a goal to hammer our shared server less, I do need to find a different solution than mail-based.

    Comment by Thomas — 2012-01-02 @ 18:30

  9. Sieve on my dovecot server filters anything I wouldn’t want delivered to my phone into second-tier imap folders.

    Comment by Gabe — 2012-01-02 @ 18:31

  10. I use gmail for everything. I am subscribed to a few of the fedora and redhat lists and for various packages I build. But I never use email off-line, so the sync is not needed.
    Before I switched to gmail I used procmail and mutt for all mail, which pretty much worked the same but at some point I couldn’t be bothered to install it on yet another machine.

    Comment by Christof — 2012-01-02 @ 19:28

  11. Separate mailinglist account, disabled during travel and purged when I come back from hollidays etc :)
    Definitely not arriving on any of my mobile devices.

    Comment by Kris Buytaert — 2012-01-02 @ 19:45

  12. Another +1 to gmane. With my Thunderbird works just fine via NNTP.

    Comment by MatÄ›j Cepl — 2012-01-02 @ 21:07

  13. Since I have my email going through my domain (I’m actually using the free version of Google apps), creating additional accounts and aliases is basically “free”. Right now I have two accounts, one as my primary email and another one that handles mailing lists. Each time I sign up for a mailing list, I’ll give it a new email address, “mailing-list@example.com”, then add that address as an alias to the mailing list account. If I ever wanted to rearrange where the mail is going to, I can just move that alias to a different account. I then use Gmail’s server-side filters to sort the mailing list messages into folders per list.

    Comment by Calvin Walton — 2012-01-02 @ 21:52

  14. I do it exactly like the previous commentator. It works like a charm.

    Comment by David — 2012-01-02 @ 22:13

  15. This is the perfect use-case for notmuch! Subscribe to as many lists as you want and set your “INBOX” tag to ignore anything from a mailing list.

    Comment by foo — 2012-01-03 @ 00:03

  16. I’ve been testing sup+offlineimap for about a month for my work email, and I am quite happy with it. Functionality-wise, it’s quite similar to GMail (including fast search), but with a better editor (you can use your favourite one). In my setup, I get the important email right in my “inbox” space (stuff addressed to me + a few very important internal mailing lists I _have_ to read), and I basically tag the rest according to the mailing list it comes from. I get warned about new emails not in my “inbox” space, but in a way that is not invasive, and I can just have a look at them when I have some time for it.

    The big drawback it has, that is not too annoying for me so far, is that you can’t easily synchronise your emails between different clients/machines (especially the “read” state). This is OK for me because:
    – I have only one computer that I use for email (my laptop)
    – Not reading work emails on my phone turns out to be a life improvement :)

    I still use gmail for my personal emails though (in that case, I prefer to be able to get them on my phone).

    Comment by guijemont — 2012-01-03 @ 15:52

  17. I also just let procmail put list traffic in Maildirs per list. Disk space is cheap and I like having a “local” archive of mailing lists rather than having to rely on dodgy “web” archives in the sky. They’re no use to me when I’m in the sky (on a plane) anyway — which is when I get a lot of my work done!

    In addition to the ‘archival’ filters based on ^List-Id or ^TO_, I also have some more intricate filters that put a copy of very interesting mails in a separate mailbox. A cron job keeps that mailbox mostly empty. Similarly, I have procmail put IMAP flags on some messages it delivers (using a very sick hack in Perl which speaks IMAP directly, rather than using some module for it). My procmail setup is quite sick in general.

    Other than that, I also use Apple’s Mail.app as a mail ‘reader’. It has a concept of “smart folders” which works quite well. Unfortunately, I can’t figure out how to write mail with Mail.app, so I can’t migrate to it completely. Nor do I really feel like being tied to a fruit machine all the time. Mutt will remain my primary mail client for the forseeable future.

    Comment by Philip Paeps — 2012-01-03 @ 16:54

  18. A couple of notes, now that I’ve had time to read what others wrote :)

    Using Google Mail: do you really want Google to know what lists you read? If you’re already letting Google read your private email, all hope is lost for you already, but even just letting them know what lists you subscribe to, gives them an awful lot of information about you. I would not feel comfortable with Google knowing that much. But I don’t feel comfortable with Google knowing *anything*, so I’m statistically irrelevant. In fact, it annoys me a little that I communicate with people who let Google read the email I send them and the email they send me but there isn’t really anything I can do about that.

    IMAP subscriptions: these are (supposed to be) server-side, not client-side, so you don’t have to configure them per machine. Unfortunately, IMAP subscriptions don’t really seem to work most of the time. Mutt completely ignores subscriptions as far as I know, and Apple Mail.app doesn’t seem to be able to understand Dovecot’s subscriptions. It hasn’t annoyed me enough to try to figure out why (I “subscribe” to all my mailboxes anyway).

    List aliases: I used to subscribe with philip+listname@ to mailing lists, but many, many mailing lists have started hiding behind ridiculous “web” front ends that seem to think that “+” is not a valid character in an email address (it is!). Also, this caused mailing list spam which wasn’t caught by bogofilter for some reason to get filed with the mailing list. I prefer for any spam the filter misses to land in my INBOX so I can immediately feed it to train the filter (happens rarely enough these days though). I’m too lazy to edit /etc/aliases on my mailserver every time I subscribe to a new list.

    GMANE and stuff: meh … I don’t like webby things. And as I wrote I really like a local archive of all my mail. Using a header cache and reasonably well-implemented IMAP server, syncing large volumes of mail isn’t too big of a problem. Dovecot is perfectly happy (on a 64bit machine) with mailboxes with hundreds of thousands of messages in them (I have some of those).

    Comment by Philip Paeps — 2012-01-03 @ 19:52

RSS feed for comments on this post. TrackBack URL

Leave a comment