CCE

imapfilter filters my imap

Contents

imapfilter is a Tool that can filter mail from imap mail servers and is configured and programmed using Lua. I use it as the primary/only mail filtering on my Email and News and Information Pipeline management, but until recently it was just a stale file in an old version of the CCE that I edited by hand. I extend it here, to generate the configuration file from org-mode tables using Org Babel.

Lua Functions and Top Matter

These are the scaffolding that makes this work. Some simple little Lua functions that do the mailbox sub-searches and move them around to its destination.

I store an fastmail app password in my .authinfo... If I decide to move this on to my endpoint it should be more-properly secured in vaultwarden and pulled with rbw. this pipe_from stuff is a mess lul.

lua:tangle ~/nix/files/imapfilter.lua
function authinfo_user(server)
   status, user = pipe_from("cat ~/.authinfo | perl -pe \"$/ = ''; " ..
                            "s/\\n/ /g\" | perl -ne 'm/" .. server ..
                            "\\s+login\\s+(\\S+)/; print $1;'")
   return user
end
function authinfo_pass(server)
   status, passwd = pipe_from("cat ~/.authinfo | perl -pe \"$/ = ''; " ..
                              "s/\\n/ /g\" | perl -ne 'm/" .. server ..
                              "\\s+login\\s+\\S+\\s+password\\s+(\\S+)/; print $1;'")
   return passwd
end

Some minutiae..

lua:tangle ~/nix/files/imapfilter.lua
options.timeout = 10;
options.create = false;
options.expunge = true;
options.namespace = false;
options.subscribe = true;

print_results_info just counts the number of mails in a mailbox/search for status output.

lua:tangle ~/nix/files/imapfilter.lua
function print_results_info(mb)
   local total = 0
   for key, value in pairs(mb) do
      total = total + 1
   end
   print("Filtering " .. total .. " mails")
end

And these are the functions which do the work.

lua:tangle ~/nix/files/imapfilter.lua
function file_by_list_id(mb, name, dest)
   results = mb:contain_field("List-Id", name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_destination(mb, name, dest)
   results = mb:contain_to(name) +
      mb:contain_cc(name) +
      mb:contain_bcc(name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_sender(mb, name, dest)
   results = mb:contain_from(name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_subject(mb, subject, dest)
   results = mb:contain_subject(subject)
   results:move_messages(fastmail[dest])
   return mb - results
end

Picking the Mail to filter

And here is how I pick the mail, this is the "root" of the search: By default it just grabs unreads in the inbox, but I have some other common searches commented out in case I want to do something else... I would love to have a better way to change target other than to swap some comments around...

lua:tangle ~/nix/files/imapfilter.lua
fastmail = IMAP {
   server = 'mail.messagingengine.com',
   username = 'rrix@fastmail.com',
   password = authinfo_pass("mail.messagingengine.com"),
   ssl = "auto",
}

-- mb = fastmail.INBOX:select_all()
-- mb = fastmail.newsletters:select_all()
mb = fastmail.INBOX:is_unseen()

print_results_info(mb)

Literate Programming Soup

This function takes a org-mode table, iterates over it, and turns it in to a set of Lua statements which will run the filter rule when evaluated by imapfilter.

lua#+name: imapfilter-from-table:eval arroyo:var tbl=zendo-contacts:tangle no:noweb-ref imapfilter-from-table
local lines = {}
for _, row in ipairs(tbl) do
  table.insert(lines, string.format('mb = file_by_%s(mb, "%s", "%s")', row[1], row[2], row[3]))
end
return table.concat(lines, "\n")

The <<imapfilter-from-table(table-name)>> noweb reference passes the named table to the Lua generator, which produces the file_by_* calls for that table's rules.

NEXT running this on The Wobserver

How do I run this before mbsync? imapnotify? move away from async-forever...?

Filter Tables

These use the Org Babel helpers above to turn the tables in to lua statements that get tangled in to the final imapfilter configuration file.

Most of the stuff that is interesting doesn't get exported to the web for publishing, but some common public lists are kept here for "usage information", i guess.

INPROGRESS Social Crap

There's probably more of these than this.

Predicate Value Destination
sender @angel.co social
sender @digg.com social
sender @e.linkedin.com social
sender @email.app.net social
sender @eventbritemail.com social
sender @facebookmail.com social
sender goodreads.com social
sender google-plus@google.com social
sender h4x@cybre.space social
sender @hs.email.nextdoor.com social
sender info@meetup.com social
sender @instagram.com social
sender @linkedin.com social
sender @meetup.com social
sender @nextdoor.com social
sender oknotify2.com social
sender @pinterest.com social
sender @redditmail.com social
sender @rs.email.nextdoor.com social
sender thegibson@hackers.town social
sender @tripit.com social
sender @twitter.com social
sender @useqwitter.com social
destination fedora-project-3157@pages.plusgoogle.com fedora/ambassadors
sender @youtube.com social
lua:tangle ~/nix/files/imapfilter.lua:noweb yes
<<imapfilter-from-table(social-lists)>>

Fedora Lists

I used to be more active than I am in Fedora.

Predicate Value Destination
destination gitbeefymiracle-members@fedoraproject.org fedora
list_id advisory-board.lists.fedoraproject.org fedora/advisory-board
list_id council.lists.fedoraproject.org fedora/advisory-board
list_id ambassadors.lists.fedoraproject.org fedora/ambassadors
list_id famna-regionals.lists.fedoraproject.org fedora/ambassadors
destination fedora-socialmedia-members@fedoraproject.org fedora/ambassadors
list_id announce.lists.fedoraproject.org fedora/announce
list_id bugzilla.redhat.com fedora/bugs
list_id pkgs.fedoraproject.org fedora/bugs
sender bugzilla@redhat.com fedora/bugs
sender buildsys@fedoraproject.org fedora/buildsys
sender scm-commits@lists.fedoraproject.org fedora/buildsys
sender updates@fedoraproject.org fedora/buildsys
list_id cloud.lists.fedoraproject.org fedora/cloud
list_id design-team.lists.fedoraproject.org fedora/design-team
list_id devel.lists.fedoraproject.org fedora/devel
list_id docs.lists.fedoraproject.org fedora/docs
list_id freemedia.lists.fedoraproject.org fedora/freemedia
list_id fudcon-planning.lists.fedoraproject.org fedora/fudcon-planning
list_id kde.lists.fedoraproject.org fedora/kde
list_id marketing.lists.fedoraproject.org fedora/marketing
list_id news.lists.fedoraproject.org fedora/news
list_id packaging.lists.fedoraproject.org fedora/packaging
list_id .rpmfusion.org fedora/rpmfusion
sender @rpmfusion.org fedora/rpmfusion
list_id summer-coding-discuss.lists.fedoraproject.org fedora/summer-coding
list_id summer-coding.lists.fedoraproject.org fedora/summer-coding
list_id tos.teachingopensource.org fedora/tos
list_id metrics-wg.theopensourceway.org fedora/tosw
list_id tosw.lists.fedorahosted.org fedora/tosw
list_id web-members fedora/websites
list_id websites.lists.fedoraproject.org fedora/websites
destination web-members@fedoraproject.org fedora/websites
sender root@fedoraproject.org fedora/websites

This line-noise exports the table as lua to the imapfilter configuration using the helpers defined above.

lua:tangle ~/nix/files/imapfilter.lua:noweb yes
<<imapfilter-from-table(fedora-lists)>>

Hackerspace Lists

I used to be subscribed to a bunch of these, I'm pretty sure half of these are inactive now, but I keep the filters around Just In Case

Predicate Value Destination
list_id c.h.i.p..bbs.nextthing.cofastmail. CHIP
list_id discuss.lists.acemonstertoys.org hackerspaces/amt
list_id lists.acemonstertoys.org hackerspaces/amt
destination acemonstertoys.org hackerspaces/amt
list_id board.lists.acemonstertoys.org hackerspaces/amt-board
list_id bay-area-reprap.googlegroups.com hackerspaces/bay-area-reprap
list_id dn42 hackerspaces/dn42
list_id discuss.lists.hackerspaces.org hackerspaces/hackerspaces-discuss
list_id heatsynclabs+owner.googlegroups.com hackerspaces/heatsynclabs
list_id heatsynclabs.googlegroups.com hackerspaces/heatsynclabs
destination heatsynclabs+msgappr@googlegroups.com hackerspaces/heatsynclabs
list_id makerbot.googlegroups.com hackerspaces/makerbot
list_id noisebridge-discuss.lists.noisebridge.net hackerspaces/noisebridge
list_id open-access-control.googlegroups.com hackerspaces/oac
list_id phxdata.googlegroups.com hackerspaces/phxdata
list_id sfhackerspace.googlegroups.com hackerspaces/sfhackerspace
list_id nortonimperiallabs.github.com hackerspaces/sfhackerspace
destination Norton Imperial Laboratories hackerspaces/sfhackerspace
list_id spaceapi-devel.lists.hackerspaces.org hackerspaces/spaceapi
list_id sudoroom.org hackerspaces/sudo
destination newsletter@peoplesopen.net hackerspaces/sudo
list_id discuss.synhak.org hackerspaces/synhak
list_id synshop.googlegroups.com hackerspaces/synshop
list_id ultimaker.googlegroups.com hackerspaces/ultimaker
list_id vhs-general.lists.hackspace.ca hackerspaces/vhs
lua:tangle ~/nix/files/imapfilter.lua:noweb yes
<<imapfilter-from-table(hackerspace-lists)>>

KDE Lists

I am mostly idle in KDE land but still have some bugzillas or projects I track

Predicate Value Destination
list_id android-qt.googlegroups.com kde/android-qt
list_id kde-announce.kde.org kde/announce
list_id kde-cvs-announce.kde.org kde/announce
list_id bugsquad.kde.org kde/bugs
sender bugzilla_noreply@kde.org kde/bugs
list_id kde-community.kde.org kde/community
list_id kde-core-devel.kde.org kde/core-devel
list_id kde-devel.kde.org kde/devel
list_id kde-ev-membership.kde.org kde/ev-membership
list_id kde-games-devel.kde.org kde/games
list_id kde-linux.kde.org kde/linux
list_id owncloud.kde.org kde/owncloud
list_id kde-pim.kde.org kde/pim
list_id kdepim-users.kde.org kde/pim-users
list_id plasma-devel.kde.org kde/plasma
list_id kde-promo.kde.org kde/promo
list_id kde-scm-interest.kde.org kde/scm
list_id kde-soc.kde.org kde/soc
list_id kde.mail.kde.org kde/support
list_id kde-usability.kde.org kde/usability
lua:tangle ~/nix/files/imapfilter.lua:noweb yes
<<imapfilter-from-table(kde-lists)>>