______________________________________________
AN IDEA FOR BRINGING PHLOGGING TO THE MASSES
lro
______________________________________________
<2019-01-30 Wed>
Table of Contents
_________________
An idea
The Prototype
. Imports
. Global vars
. Getting email
. Mapping emails to users
. From
. Is that email address in the dictionary?
. Phlog post
. Write phlog post to file
. Update gophermap
. New Phlog entry
. iterating
Conclusion
An idea
=======
I had a what I think might be a good idea to help people get setup
phlogging, and thats an email based phlogging service. Where a user
would email say
[email protected] and the subject is their chosen
title, and the cotents of the emial is the phlog post.
Each user would be identified by their email address, registrations
would probably happen on a webpage, with a captcha to stop spammers
etc.
I would have each user have a user on the server, and use gophernicus
to display the user gopher directory to host the phlog. An email bot
would just watch the inbox of
[email protected] and create the phlog
post, and move it to the user gopher directory, and update the
gophermap.
Then we can have a master page at example.com:70 with the most
recently posted phlogs.
The Prototype
=============
I figured a good course of action was to setup a prototype using my
sdf account. I will leave the web based registration for later as that
seems the most trivial, I am more interested in getting what is
essentially an email bot setup to watch my email and add phlog posts.
For ease of writing and ease of development by others, i'm going to
write it in python. Not a language I've used alot, but this should
make things easier for others to jump in and modify and run the code
themselves.
What my initial implementation does is look for unread emails in a
mailfile, and create a file named with todays date in that users
directory as the phlog post. And updates the gophermap with the new
phlog entry at the top with the subject of the email as a nice little
description/title.
Imports
~~~~~~~
I need to import a few libraries for various things, like reading the
mailbox file, importing json objects from a file, and getting the
date/time.
,----
| #!/usr/bin/env python3
|
| ###################################################################
| # To the extent possible under law, the person who associated CC0 #
| # with phlog_bot.py has waived all copyright and related or #
| # neighboring rights to phlog_bot.py. #
| # #
| #
https://creativecommons.org/publicdomain/zero/1.0/ #
| ###################################################################
|
| # NO WARRANTY!!!!
| # PLEASE DON'T RUN THIS SCRIPT UNLESS YOU HAVE BACKUPS OF YOUR
| # MAILFILE OR ANYTHING ELSE THIS SCRIPT TOUCHES PLEASE.
|
| import mailbox
| import json
| import time
| import datetime
| import os
|
`----
Global vars
~~~~~~~~~~~
Global variables.
,----
| gopher_dir = "/sdf/arpa/gm/l/lro/test"
| userdictionaryfile = "users.json"
| mailboxf = "/mail/lro"
| today = datetime.date.today()
| today_fmt = today.strftime("%Y")+"-"+today.strftime("%m")+"-"+today.strftime("%d")
|
`----
Getting email
~~~~~~~~~~~~~
The first step is to open the email and get the contents, then collect
the unread emails. For this prototype i'm just going to read my sdf
mailspool file at `/mail/lro'.
To get the unread emails I need to get all the mail from the file and
return all the mail that doesn't have the read (R) flag set and mark
all the new mail as read. Which as it turns out is a bit weird to set
the read flag in mail, which is why I have to keep track of the index
into /lromailbox/.
See if you just use `add_flag' on /message/, it doesn't write that
back to /lromailbox/, so to get around that I set the index into
/lromailbox/ to the altered /message/, then when I write /lromailbox/
back to disk the read flag is now set.
And we return a list of all the unread mail.
,----
| def get_unread_emails(mailboxfile):
| unread = []
| if not os.path.isfile(mailboxfile):
| print("Mailboxfile doesn't exist: %s" % (mailboxfile))
| return unread
|
| mailboxfp = mailbox.mbox(mailboxfile)
| ind = 0
| for message in mailboxfp:
| flags = message.get_flags()
| if flags.find('R') == -1:
| unread.append(message)
| message.add_flag('R')
| mailboxfp[ind] = message
| ind = ind + 1
|
| mailboxfp.lock()
| mailboxfp.flush()
| mailboxfp.unlock()
| mailboxfp.close()
| return unread
|
`----
Mapping emails to users
~~~~~~~~~~~~~~~~~~~~~~~
I need to store a dictionary of mappings of email addresses to
usernames. At this point i've chosen to go with JSON so I can read it
in easily from a file on disk.
For the time being this is file is handled manually, but in the
production version will be handled by the registration program.
,----
| {"
[email protected]" : "lrojr", "
[email protected]" : "lrosr"}
`----
,----
| def load_dictionary(dictfile):
| if not os.path.isfile(dictfile):
| return []
|
| usersfile = open(dictfile)
| userdictionary = json.load(usersfile)
| usersfile.close()
| return userdictionary
|
`----
From
~~~~
What we need next is a function that gets the from address from each
unread message and transforms it to lower case so that the output can
be used to search through /userdictionary/.
,----
| def get_from_lower(message):
| From = message.get_from()
| return From.split(" ")[0].lower()
|
`----
Is that email address in the dictionary?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Checks if an email is one of our registered users by trying to get
their username from the dictionary, returning the username if they
are, otherwise returns False.
,----
| def user_in_dict(email, dictionary):
| try:
| return dictionary[email]
| except KeyError:
| return False
|
`----
Phlog post
~~~~~~~~~~
Gets new phlog post text from email.
,----
| def phlog_post_contents(email):
| return email.get_payload()
|
`----
Write phlog post to file
~~~~~~~~~~~~~~~~~~~~~~~~
Writes the phlog post file to disk. /filename/ includes full path to
file.
If file already exists then add a "-x" to the end.
,----
| def write_phlog_post(filename, contents):
| ind = 0
| justfilenamenotpath = filename.split("/")[-1]
| while os.path.isfile(filename):
| if ind == 99:
| break
| filename = filename.split(".")[0].split("_")[0]+"_"+str(ind)+".txt"
| justfilenamenotpath = filename.split("/")[-1]
|
| with open(filename, "w") as phlogfile:
| phlogfile.write(contents)
|
| return justfilenamenotpath
|
`----
Update gophermap
~~~~~~~~~~~~~~~~
Updates the phlog gophermap with the new phlog post.
/gophermap/ is the full path and filename for the gophermap.
,----
| def update_gophermap(gophermap, filename, description):
| entry = "0%s %s" % (description, filename)
|
| with open(gophermap, "r") as gfp:
| buf = gfp.readlines()
|
| with open(gophermap, "w") as gfp:
| for line in buf:
| if "==================================================" in line:
| line = line+"\n"+entry+"\n"
| gfp.write(line)
|
`----
New Phlog entry
~~~~~~~~~~~~~~~
Creates a new phlog entry, and updates the gophermap.
,----
| def new_phlog_entry(message):
| email = get_from_lower(message)
| username = user_in_dict(email, userdictionary)
| if not username:
| print("User: %s not in user dictionary" % (email))
| return
| phlog_title = message['subject']
| phlog_filename = today_fmt + ".txt"
| phlog_contents = phlog_post_contents(message)
| phlog_dir = gopher_dir+"/"+username
| phlog_gophermap = phlog_dir+"/gophermap"
| phlog_filename = write_phlog_post(phlog_dir+"/"+phlog_filename, phlog_contents)
| update_gophermap(phlog_gophermap, phlog_filename, phlog_title+" <"+today_fmt+">")
| update_gophermap(gopher_dir+"/gophermap", username+"/"+phlog_filename, username+" - "+phlog_title+" <"+today_fmt+">")
| print("Added phlog post for %s" % (username))
|
`----
iterating
~~~~~~~~~
The final step is to iterate over all the unread emails and make them
phlog posts.
,----
| userdictionary = load_dictionary(userdictionaryfile)
|
| for message in get_unread_emails(mailboxf):
| new_phlog_entry(message)
|
`----
Conclusion
==========
So thats the initial prototype done, I don't know what I am going to
do with it though, whether I should host a small gopher server and set
up a new email or not. If someone else wants too, bloody go for it!
This could also be used personally by anyone as well, just add a
snippet to the /get_unread_emails/ that checks the subject for ones
that start with [PHLOG_POST] or something and have it watch your mail
account.
If your worried about security you could always GPG sign the emails
and have the dictionary verify each email, but that adds alot of
complexity.
Feature requests welcome!
