Processing Email - Single Script / Multiple Emails

Part 6 in a series of articles on implementing a notification system using Gmail and Line Bot

Greetings. In this article I will be going over the thought process of how I handled multiple emails in a single script.

In most cases there will only be a single email that needs to be processed at any given time. But to allow for more flexibility and to have a single place of control, and unfortunately complexity, I have decided to use a single script.

Some background

The services I receive email from have some well defined static attributes.

• Sending Address
• Subject Line

About once a month they will also send billing information using the same Sending Address which can be ignored by the script.

Because of this; the logic will be something along the lines of.

1. Check the subject line. If it exist in the list of known subject lines process the email.
2. If 1 true. Double check that the sending address is an approved/known sending address.
3. If 1 is false. Log an issue (This should not occur due to the filtering which is being done within Gmail.)
4. If 1 is true, but the sending address is false. Log the message.
5. If both 1 and 2 are true. This is a valid notification email.
   1. Determine which notification is being handled using the Sender's Address.
   2. Apply the correct regex expression(s) to get the information
   3. Send the notification and log its successful handling.
   4. Attach the notified label to the email.

Organization

constants.py

SUB_1 = "something" SUB_2 = "something" SUBJECTS = [SUB_1, SUB_2] FROM_BUS = "<email address of sender>" FROM_KIDZDUO = "<email address of sender>" 

gmail.py

I have intentionally used a somewhat defensive coding style here.

The code look something like this

def handle_each_email(service, message_id, logger) -> tuple: data = notifier = None single_email = get_message(service, message_id, logger) # Check the subject is an expected notification subject line  subject = single_email.get("subject") if subject in constants.SUBJECTS: # workout which notification we are dealing with and use the correct  # regular expression string  sender = single_email.get("from") email_body = single_email.get_content() if sender == constants.FROM_BUS: data = patterns.findMatches(email_body, patterns.BUS_DATA) datetime = patterns.findMatches(email_body, patterns.BUS_DATE_TIME) # Merge data and datetime into a single dictionary  data.update(datetime) notifier = "BUS" elif sender == constants.FROM_KIDZDUO: data = patterns.findMatches(email_body, patterns.KIDZDUO_ENTEREXIT) notifier = "KIDZDUO" else: # This needs to be logged. Means failed to match sender.  # if notifier is None:  logger.warning(f"Failed to process message_id: {message_id} " f"Matched Subject: {subject} " f"Sender not matched: {sender}") pass return notifier, data else: # Not an expected subject line. Ignore this email  logger.info("This is not a notifcation.") logger.info(f"sender: {sender}\n\t") logger.info(f"subject: {subject}") return notifier, data 

The break down

1. First we set data and notifier to None
2. Get the full email
3. Get the subject line
4. Check that the subject line is one of the expect subject strings
5. Get the sending address and email body
6. Check who the sender is:
   1. If it's the bus company. set data using the bus regex and set notifier to "BUS"
   2. If it's Kidsduo. Set the data using the kidzduo regex and set nofifer to "KIDZDUO"
   3. Otherwise log the warning and return notifier and data
7. If item 4 above is false, log an info message and return notifier and data both of which should be None

Main loop

Let's see how we will process this in a batch form if needed.

for message_id in list_of_message_ids: processed = False notifier, data = handle_each_email(service, message_id, logger) # Notifier tells us how the data dict is structured  if notifier == "BUS": logger.debug("Bus") # Your notification / bot code here  processed = True elif notifier == "KIDZDUO": logger.debug("KidsDuo") # Your notification / bot code here  processed = True elif notifier is None and data is not None: logger.warning(f"Subject matched but From was not matched") elif notifier is None and data is None: logger.info(f"Non-Notification email from expected sender") else: # We should not get here. But log it.  logger.warning( f"Something went wrong. Unexpected match." f"Don't know how to handle data." ) if processed: # Mail was processed. Add label so it's not processed again  # Gmail only allows for the shortest time interval of one day.  logger.debug("adding label") add_label_to_message(service, message_id, secrets.LABEL_ID) # End of the program logger.info("Ending cleanly") 

Note that there are several checks that should probably be made. The return of add_label_to_message() for example.

I will be making the code available in GitHub later. Along with the systemd configuration I used to have the script run at select times of the day and week. Watch for the future post on this.