The Complete Computer

imapfilter is a Tool that can filter mail from imap mail servers and is configured and programmed using roam:Lua . I use it as the primary/only mail filtering on my Email and News and Information Pipeline management, but until recently it was just a stale file in an old version of the CCE that I edited by hand. I extend it here, to generate the configuration file from org-mode tables using Org Babel .

These are the scaffolding that makes this work. Some simple little Lua functions that do the mailbox sub-searches and move them around to its destination.

I store an fastmail app password in my .authinfo... If I decide to move this on to my endpoint it should be more-properly secured in vaultwarden and pulled with rbw. this pipe_from stuff is a mess lul.

lua source: :tangle ~/nix/files/imapfilter.lua
function authinfo_user(server)
   status, user = pipe_from("cat ~/.authinfo | perl -pe \"$/ = ''; " ..
                            "s/\\n/ /g\" | perl -ne 'm/" .. server ..
                            "\\s+login\\s+(\\S+)/; print $1;'")
   return user
end
function authinfo_pass(server)
   status, passwd = pipe_from("cat ~/.authinfo | perl -pe \"$/ = ''; " ..
                              "s/\\n/ /g\" | perl -ne 'm/" .. server ..
                              "\\s+login\\s+\\S+\\s+password\\s+(\\S+)/; print $1;'")
   return passwd
end

Some minutiae..

lua source: :tangle ~/nix/files/imapfilter.lua
options.timeout = 10;
options.create = false;
options.expunge = true;
options.namespace = false;
options.subscribe = true;

print_results_info just counts the number of mails in a mailbox/search for status output.

lua source: :tangle ~/nix/files/imapfilter.lua
function print_results_info(mb)
   local total = 0
   for key, value in pairs(mb) do
      total = total + 1
   end
   print("Filtering " .. total .. " mails")
end

And these are the functions which do the work.

lua source: :tangle ~/nix/files/imapfilter.lua
function file_by_list_id(mb, name, dest)
   results = mb:contain_field("List-Id", name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_destination(mb, name, dest)
   results = mb:contain_to(name) +
      mb:contain_cc(name) +
      mb:contain_bcc(name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_sender(mb, name, dest)
   results = mb:contain_from(name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_subject(mb, subject, dest)
   results = mb:contain_subject(subject)
   results:move_messages(fastmail[dest])
   return mb - results
end

And here is how I pick the mail, this is the "root" of the search: By default it just grabs unreads in the inbox, but I have some other common searches commented out in case I want to do something else... I would love to have a better way to change target other than to swap some comments around...

lua source: :tangle ~/nix/files/imapfilter.lua
fastmail = IMAP {
   server = 'mail.messagingengine.com',
   username = 'rrix@fastmail.com',
   password = authinfo_pass("mail.messagingengine.com"),
   ssl = "auto",
}

-- mb = fastmail.INBOX:select_all()
-- mb = fastmail.newsletters:select_all()
mb = fastmail.INBOX:is_unseen()

print_results_info(mb)

This function takes a org-mode table, iterates over it, and turns it in to a set of Lua statements which will run the filter rule when evaluated by imapfilter.

 imapfilter-from-tablelua source: :eval arroyo :var tbl=zendo-contacts :tangle no :noweb-ref imapfilter-from-table
local lines = {}
for _, row in ipairs(tbl) do
  table.insert(lines, string.format('mb = file_by_%s(mb, "%s", "%s")', row[1], row[2], row[3]))
end
return table.concat(lines, "\n")

The <<imapfilter-from-table(table-name)>> noweb reference passes the named table to the Lua generator, which produces the file_by_* calls for that table's rules.

How do I run this before mbsync ? imapnotify? move away from async-forever ...?

These use the Org Babel helpers above to turn the tables in to lua statements that get tangled in to the final imapfilter configuration file.

Most of the stuff that is interesting doesn't get exported to the web for publishing, but some common public lists are kept here for "usage information", i guess.

There's probably more of these than this.

| Predicate | Value | Destination | |---------------------------------------------------------------------------| | sender | @angel.co | social | | sender | @digg.com | social | | sender | @e.linkedin.com | social | | sender | @email.app.net | social | | sender | @eventbritemail.com | social | | sender | @facebookmail.com | social | | sender | goodreads.com | social | | sender | google-plus@google.com | social | | sender | h4x@cybre.space | social | | sender | @hs.email.nextdoor.com | social | | sender | info@meetup.com | social | | sender | @instagram.com | social | | sender | @linkedin.com | social | | sender | @meetup.com | social | | sender | @nextdoor.com | social | | sender | oknotify2.com | social | | sender | @pinterest.com | social | | sender | @redditmail.com | social | | sender | @rs.email.nextdoor.com | social | | sender | thegibson@hackers.town | social | | sender | @tripit.com | social | | sender | @twitter.com | social | | sender | @useqwitter.com | social | | destination | fedora-project-3157@pages.plusgoogle.com | fedora/ambassadors | | sender | @youtube.com | social |

lua source: :tangle ~/nix/files/imapfilter.lua :noweb yes
<<imapfilter-from-table(social-lists)>>

I used to be more active than I am in Fedora.

| Predicate | Value | Destination | |------------------------------------------------------------------------------------| | destination | gitbeefymiracle-members@fedoraproject.org | fedora | | list_id | advisory-board.lists.fedoraproject.org | fedora/advisory-board | | list_id | council.lists.fedoraproject.org | fedora/advisory-board | | list_id | ambassadors.lists.fedoraproject.org | fedora/ambassadors | | list_id | famna-regionals.lists.fedoraproject.org | fedora/ambassadors | | destination | fedora-socialmedia-members@fedoraproject.org | fedora/ambassadors | | list_id | announce.lists.fedoraproject.org | fedora/announce | | list_id | bugzilla.redhat.com | fedora/bugs | | list_id | pkgs.fedoraproject.org | fedora/bugs | | sender | bugzilla@redhat.com | fedora/bugs | | sender | buildsys@fedoraproject.org | fedora/buildsys | | sender | scm-commits@lists.fedoraproject.org | fedora/buildsys | | sender | updates@fedoraproject.org | fedora/buildsys | | list_id | cloud.lists.fedoraproject.org | fedora/cloud | | list_id | design-team.lists.fedoraproject.org | fedora/design-team | | list_id | devel.lists.fedoraproject.org | fedora/devel | | list_id | docs.lists.fedoraproject.org | fedora/docs | | list_id | freemedia.lists.fedoraproject.org | fedora/freemedia | | list_id | fudcon-planning.lists.fedoraproject.org | fedora/fudcon-planning | | list_id | kde.lists.fedoraproject.org | fedora/kde | | list_id | marketing.lists.fedoraproject.org | fedora/marketing | | list_id | news.lists.fedoraproject.org | fedora/news | | list_id | packaging.lists.fedoraproject.org | fedora/packaging | | list_id | .rpmfusion.org | fedora/rpmfusion | | sender | @rpmfusion.org | fedora/rpmfusion | | list_id | summer-coding-discuss.lists.fedoraproject.org | fedora/summer-coding | | list_id | summer-coding.lists.fedoraproject.org | fedora/summer-coding | | list_id | tos.teachingopensource.org | fedora/tos | | list_id | metrics-wg.theopensourceway.org | fedora/tosw | | list_id | tosw.lists.fedorahosted.org | fedora/tosw | | list_id | web-members | fedora/websites | | list_id | websites.lists.fedoraproject.org | fedora/websites | | destination | web-members@fedoraproject.org | fedora/websites | | sender | root@fedoraproject.org | fedora/websites |

This line-noise exports the table as lua to the imapfilter configuration using the helpers defined above.

lua source: :tangle ~/nix/files/imapfilter.lua :noweb yes
<<imapfilter-from-table(fedora-lists)>>

I used to be subscribed to a bunch of these, I'm pretty sure half of these are inactive now, but I keep the filters around Just In Case

| Predicate | Value | Destination | |-------------------------------------------------------------------------------------------| | list_id | c.h.i.p..bbs.nextthing.cofastmail. | CHIP | | list_id | discuss.lists.acemonstertoys.org | hackerspaces/amt | | list_id | lists.acemonstertoys.org | hackerspaces/amt | | destination | acemonstertoys.org | hackerspaces/amt | | list_id | board.lists.acemonstertoys.org | hackerspaces/amt-board | | list_id | bay-area-reprap.googlegroups.com | hackerspaces/bay-area-reprap | | list_id | dn42 | hackerspaces/dn42 | | list_id | discuss.lists.hackerspaces.org | hackerspaces/hackerspaces-discuss | | list_id | heatsynclabs+owner.googlegroups.com | hackerspaces/heatsynclabs | | list_id | heatsynclabs.googlegroups.com | hackerspaces/heatsynclabs | | destination | heatsynclabs+msgappr@googlegroups.com | hackerspaces/heatsynclabs | | list_id | makerbot.googlegroups.com | hackerspaces/makerbot | | list_id | noisebridge-discuss.lists.noisebridge.net | hackerspaces/noisebridge | | list_id | open-access-control.googlegroups.com | hackerspaces/oac | | list_id | phxdata.googlegroups.com | hackerspaces/phxdata | | list_id | sfhackerspace.googlegroups.com | hackerspaces/sfhackerspace | | list_id | nortonimperiallabs.github.com | hackerspaces/sfhackerspace | | destination | Norton Imperial Laboratories | hackerspaces/sfhackerspace | | list_id | spaceapi-devel.lists.hackerspaces.org | hackerspaces/spaceapi | | list_id | sudoroom.org | hackerspaces/sudo | | destination | newsletter@peoplesopen.net | hackerspaces/sudo | | list_id | discuss.synhak.org | hackerspaces/synhak | | list_id | synshop.googlegroups.com | hackerspaces/synshop | | list_id | ultimaker.googlegroups.com | hackerspaces/ultimaker | | list_id | vhs-general.lists.hackspace.ca | hackerspaces/vhs |

lua source: :tangle ~/nix/files/imapfilter.lua :noweb yes
<<imapfilter-from-table(hackerspace-lists)>>

I am mostly idle in KDE land but still have some bugzillas or projects I track

| Predicate | Value | Destination | |-----------------------------------------------------------| | list_id | android-qt.googlegroups.com | kde/android-qt | | list_id | kde-announce.kde.org | kde/announce | | list_id | kde-cvs-announce.kde.org | kde/announce | | list_id | bugsquad.kde.org | kde/bugs | | sender | bugzilla_noreply@kde.org | kde/bugs | | list_id | kde-community.kde.org | kde/community | | list_id | kde-core-devel.kde.org | kde/core-devel | | list_id | kde-devel.kde.org | kde/devel | | list_id | kde-ev-membership.kde.org | kde/ev-membership | | list_id | kde-games-devel.kde.org | kde/games | | list_id | kde-linux.kde.org | kde/linux | | list_id | owncloud.kde.org | kde/owncloud | | list_id | kde-pim.kde.org | kde/pim | | list_id | kdepim-users.kde.org | kde/pim-users | | list_id | plasma-devel.kde.org | kde/plasma | | list_id | kde-promo.kde.org | kde/promo | | list_id | kde-scm-interest.kde.org | kde/scm | | list_id | kde-soc.kde.org | kde/soc | | list_id | kde.mail.kde.org | kde/support | | list_id | kde-usability.kde.org | kde/usability |

lua source: :tangle ~/nix/files/imapfilter.lua :noweb yes
<<imapfilter-from-table(kde-lists)>>