The Complete Computer

imapfilter is a Tool that can filter mail from imap mail servers and is configured and programmed using roam:Lua . I use it as the primary/only mail filtering on my Email and News and Information Pipeline management, but until recently it was just a stale file in an old version of the CCE that I edited by hand. I extend it here, to generate the configuration file from org-mode tables using Org Babel .

These are the scaffolding that makes this work. Some simple little Lua functions that do the mailbox sub-searches and move them around to its destination.

I store an fastmail app password in my .authinfo... If I decide to move this on to my endpoint it should be more-properly secured in vaultwarden and pulled with rbw. this pipe_from stuff is a mess lul.

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua
function authinfo_user(server)
   status, user = pipe_from("cat ~/.authinfo | perl -pe \"$/ = ''; " ..
                            "s/\\n/ /g\" | perl -ne 'm/" .. server ..
                            "\\s+login\\s+(\\S+)/; print $1;'")
   return user
end
function authinfo_pass(server)
   status, passwd = pipe_from("cat ~/.authinfo | perl -pe \"$/ = ''; " ..
                              "s/\\n/ /g\" | perl -ne 'm/" .. server ..
                              "\\s+login\\s+\\S+\\s+password\\s+(\\S+)/; print $1;'")
   return passwd
end

Some minutiae..

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua
options.timeout = 10;
options.create = false;
options.expunge = true;
options.namespace = false;
options.subscribe = true;

print_results_info just counts the number of mails in a mailbox/search for status output.

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua
function print_results_info(mb)
   local total = 0
   for key, value in pairs(mb) do
      total = total + 1
   end
   print("Filtering " .. total .. " mails")
end

And these are the functions which do the work.

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua
function file_by_list_id(mb, name, dest)
   results = mb:contain_field("List-Id", name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_destination(mb, name, dest)
   results = mb:contain_to(name) +
      mb:contain_cc(name) +
      mb:contain_bcc(name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_sender(mb, name, dest)
   results = mb:contain_from(name)
   results:move_messages(fastmail[dest])
   return mb - results
end

function file_by_subject(mb, subject, dest)
   results = mb:contain_subject(subject)
   results:move_messages(fastmail[dest])
   return mb - results
end

And here is how I pick the mail, this is the "root" of the search: By default it just grabs unreads in the inbox, but I have some other common searches commented out in case I want to do something else... I would love to have a better way to change target other than to swap some comments around...

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua
fastmail = IMAP {
   server = 'mail.messagingengine.com',
   username = 'rrix@fastmail.com',
   password = authinfo_pass("mail.messagingengine.com"),
   ssl = "auto",
}

-- mb = fastmail.INBOX:select_all()
-- mb = fastmail.newsletters:select_all()
mb = fastmail.INBOX:is_unseen()

print_results_info(mb)

This function takes a org-mode table, iterates over it, and turns it in to a set of Lua statements which will run the filter rule when evaluated by imapfilter.

 imapfilter-from-tableemacs-lisp source: 
(defun cce-literate-imapfilter-from-table (tbl)
  (thread-last
    tbl
    (seq-map
     (lambda (row)
       (format "mb = file_by_%s(mb, \"%s\", \"%s\")"
               (first row)
               (second row)
               (third row))))
    (s-join "\n")))

The code below is call-imapfilter-from-table in the functions under the filter tables. It embeds and ensures the function defined above is evaluated, and then calls it with the table that is passed in as a variable called tbl.

Chaining the functions like this allows me to C-c C-c in the code above to run it in my local session, and then below to <<call()>> that code inline below.

 call-imapfilter-from-tableemacs-lisp source: :var tbl=fedora-lists :noweb yes
<<imapfilter-from-table>>

(cce-literate-imapfilter-from-table tbl)

How do I run this before mbsync ? imapnotify? move away from async-forever ...?

These use the Org Babel helpers above to turn the tables in to lua statements that get tangled in to the final imapfilter configuration file.

Most of the stuff that is interesting doesn't get exported to the web for publishing, but some common public lists are kept here for "usage information", i guess.

There's probably more of these than this.

| Predicate | Value | Destination | |---------------------------------------------------------------------------| | sender | @angel.co | social | | sender | @digg.com | social | | sender | @e.linkedin.com | social | | sender | @email.app.net | social | | sender | @eventbritemail.com | social | | sender | @facebookmail.com | social | | sender | goodreads.com | social | | sender | google-plus@google.com | social | | sender | h4x@cybre.space | social | | sender | @hs.email.nextdoor.com | social | | sender | info@meetup.com | social | | sender | @instagram.com | social | | sender | @linkedin.com | social | | sender | @meetup.com | social | | sender | @nextdoor.com | social | | sender | oknotify2.com | social | | sender | @pinterest.com | social | | sender | @redditmail.com | social | | sender | @rs.email.nextdoor.com | social | | sender | thegibson@hackers.town | social | | sender | @tripit.com | social | | sender | @twitter.com | social | | sender | @useqwitter.com | social | | destination | fedora-project-3157@pages.plusgoogle.com | fedora/ambassadors | | sender | @youtube.com | social |

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua :noweb yes
<<call-imapfilter-from-table(social-lists)>>

I used to be more active than I am in Fedora.

| Predicate | Value | Destination | |------------------------------------------------------------------------------------| | destination | gitbeefymiracle-members@fedoraproject.org | fedora | | list_id | advisory-board.lists.fedoraproject.org | fedora/advisory-board | | list_id | council.lists.fedoraproject.org | fedora/advisory-board | | list_id | ambassadors.lists.fedoraproject.org | fedora/ambassadors | | list_id | famna-regionals.lists.fedoraproject.org | fedora/ambassadors | | destination | fedora-socialmedia-members@fedoraproject.org | fedora/ambassadors | | list_id | announce.lists.fedoraproject.org | fedora/announce | | list_id | bugzilla.redhat.com | fedora/bugs | | list_id | pkgs.fedoraproject.org | fedora/bugs | | sender | bugzilla@redhat.com | fedora/bugs | | sender | buildsys@fedoraproject.org | fedora/buildsys | | sender | scm-commits@lists.fedoraproject.org | fedora/buildsys | | sender | updates@fedoraproject.org | fedora/buildsys | | list_id | cloud.lists.fedoraproject.org | fedora/cloud | | list_id | design-team.lists.fedoraproject.org | fedora/design-team | | list_id | devel.lists.fedoraproject.org | fedora/devel | | list_id | docs.lists.fedoraproject.org | fedora/docs | | list_id | freemedia.lists.fedoraproject.org | fedora/freemedia | | list_id | fudcon-planning.lists.fedoraproject.org | fedora/fudcon-planning | | list_id | kde.lists.fedoraproject.org | fedora/kde | | list_id | marketing.lists.fedoraproject.org | fedora/marketing | | list_id | news.lists.fedoraproject.org | fedora/news | | list_id | packaging.lists.fedoraproject.org | fedora/packaging | | list_id | .rpmfusion.org | fedora/rpmfusion | | sender | @rpmfusion.org | fedora/rpmfusion | | list_id | summer-coding-discuss.lists.fedoraproject.org | fedora/summer-coding | | list_id | summer-coding.lists.fedoraproject.org | fedora/summer-coding | | list_id | tos.teachingopensource.org | fedora/tos | | list_id | metrics-wg.theopensourceway.org | fedora/tosw | | list_id | tosw.lists.fedorahosted.org | fedora/tosw | | list_id | web-members | fedora/websites | | list_id | websites.lists.fedoraproject.org | fedora/websites | | destination | web-members@fedoraproject.org | fedora/websites | | sender | root@fedoraproject.org | fedora/websites |

This line-noise exports the table as lua to the imapfilter configuration using the helpers defined above.

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua :noweb yes
<<call-imapfilter-from-table(fedora-lists)>>

I used to be subscribed to a bunch of these, I'm pretty sure half of these are inactive now, but I keep the filters around Just In Case

| Predicate | Value | Destination | |-------------------------------------------------------------------------------------------| | list_id | c.h.i.p..bbs.nextthing.cofastmail. | CHIP | | list_id | discuss.lists.acemonstertoys.org | hackerspaces/amt | | list_id | lists.acemonstertoys.org | hackerspaces/amt | | destination | acemonstertoys.org | hackerspaces/amt | | list_id | board.lists.acemonstertoys.org | hackerspaces/amt-board | | list_id | bay-area-reprap.googlegroups.com | hackerspaces/bay-area-reprap | | list_id | dn42 | hackerspaces/dn42 | | list_id | discuss.lists.hackerspaces.org | hackerspaces/hackerspaces-discuss | | list_id | heatsynclabs+owner.googlegroups.com | hackerspaces/heatsynclabs | | list_id | heatsynclabs.googlegroups.com | hackerspaces/heatsynclabs | | destination | heatsynclabs+msgappr@googlegroups.com | hackerspaces/heatsynclabs | | list_id | makerbot.googlegroups.com | hackerspaces/makerbot | | list_id | noisebridge-discuss.lists.noisebridge.net | hackerspaces/noisebridge | | list_id | open-access-control.googlegroups.com | hackerspaces/oac | | list_id | phxdata.googlegroups.com | hackerspaces/phxdata | | list_id | sfhackerspace.googlegroups.com | hackerspaces/sfhackerspace | | list_id | nortonimperiallabs.github.com | hackerspaces/sfhackerspace | | destination | Norton Imperial Laboratories | hackerspaces/sfhackerspace | | list_id | spaceapi-devel.lists.hackerspaces.org | hackerspaces/spaceapi | | list_id | sudoroom.org | hackerspaces/sudo | | destination | newsletter@peoplesopen.net | hackerspaces/sudo | | list_id | discuss.synhak.org | hackerspaces/synhak | | list_id | synshop.googlegroups.com | hackerspaces/synshop | | list_id | ultimaker.googlegroups.com | hackerspaces/ultimaker | | list_id | vhs-general.lists.hackspace.ca | hackerspaces/vhs |

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua :noweb yes
<<call-imapfilter-from-table(hackerspace-lists)>>

I am mostly idle in KDE land but still have some bugzillas or projects I track

| Predicate | Value | Destination | |-----------------------------------------------------------| | list_id | android-qt.googlegroups.com | kde/android-qt | | list_id | kde-announce.kde.org | kde/announce | | list_id | kde-cvs-announce.kde.org | kde/announce | | list_id | bugsquad.kde.org | kde/bugs | | sender | bugzilla_noreply@kde.org | kde/bugs | | list_id | kde-community.kde.org | kde/community | | list_id | kde-core-devel.kde.org | kde/core-devel | | list_id | kde-devel.kde.org | kde/devel | | list_id | kde-ev-membership.kde.org | kde/ev-membership | | list_id | kde-games-devel.kde.org | kde/games | | list_id | kde-linux.kde.org | kde/linux | | list_id | owncloud.kde.org | kde/owncloud | | list_id | kde-pim.kde.org | kde/pim | | list_id | kdepim-users.kde.org | kde/pim-users | | list_id | plasma-devel.kde.org | kde/plasma | | list_id | kde-promo.kde.org | kde/promo | | list_id | kde-scm-interest.kde.org | kde/scm | | list_id | kde-soc.kde.org | kde/soc | | list_id | kde.mail.kde.org | kde/support | | list_id | kde-usability.kde.org | kde/usability |

lua source: :tangle ~/arroyo-nix/files/imapfilter.lua :noweb yes
<<call-imapfilter-from-table(kde-lists)>>