Building an Email Archiving System: Searching and Displaying the Data – Part 4

Jeff Goldstein
Apr. 15, 2019 by Jeff Goldstein

In the fourth installment of this blog series and the third in the coding phase, I’m discussing the first phase of the Archive UI (yes, I just said that). Leveraging the UI, an auditor can search for all activities for a given email address filtered between two dates; that will include the original rcpt_to address events and optionally, the cc and bcc events as well. That includes retrieving the email body and a boatload of other fun facts that SparkPost delivers with each event like subject text, campaign name, IP locations, client type used when opening, and much much more.

Just like previous blogs, the code for this version is available in my github repository. For this version, look at the step 3 folder.  The diagram below depicts what part of the process this code is addressing; in this case, the UI (Viewing Application).

 

* This blog is addressing the process(s) in green

The look, feel and capabilities can and typically are vastly different for each designer. In my case I decided on a few principles before building my UI:

  1. I do not support paging. I’m assuming that an auditor is looking for specific emails and will want to see an exact email along with all of its details. Because of this, there is a configuration setting that will limit the number of entries that can be returned. That means that all of the corresponding data will be displayed on a single page; so be cognizant of the amount of data you will display in the browser tab so you don’t cause memory issues.
  2. Allow for in table filtering in order to limit calls to the database.
  3. Leave reporting for the next phase
  4. While I don’t show all the data that is captured by SparkPost, be sure to pull all of the data so it’s easy to display any data. This practice will also cause a lot of memory to be used by the browser.
  5. Since this is an auditing tool, it probably won’t be used very often so DON’T spend a lot of time building the UI; (Keep It Simple S****d , KISS method of design and development)

So with those simple principles, I have created an easy user interface that starts out with the user entering in an email address, a start date, and an end date. The system will then make a call to the database and compare the number of rows returned to the configuration setting that limits how many rows can be displayed. If the number returned by the database is lower than the limit; it builds the HTML table that is then displayed in the UI. Because I like to cause headaches for myself, I did dirty the waters up a bit by allowing the user to see the data in two different formats. One is just a simple table across the page, and the second is a condensed format that actually shows a little more data. Here are two small samples of the UI:

The one on the left is the wide version while the one on the right is the collapsed version. For the most part, they do show the same data, but I was able to get a few more items into the collapsed version without it looking too busy. The collapsed version uses a fixed iframe to display the HTML body along with the selected metadata for that event while the wide version displays the body and event data in a floating window.

Once displayed, the user can filter the displayed rows by typing into any of the five filters.  I chose to allow for filtering on the subject, injection time, campaign, event type and UID fields. These filters do NOT care about upper/lower cases and do NOT care about placement. You can think of these filters as a contains filter. If the text is anywhere in that field, I’m happy and will keep that row.

If you examine the s3.php file, you will see that I added code for storing the HTML body of the email along with the EML object.  After playing around with PHP email libraries in order to display the stored HTML portion of the EML object; I didn’t like the results and came to the conclusion that storing the HTML body into s3 for displaying would be a better approach. This has worked out nicely and in the reporting phase of the project, I plan to support a download process for the EML body from s3, in order to allow the auditor to display the email in their favorite email client.

Overall, the code is fairly straightforward except for the filtering code. Because I have two formats, and the collapsed format has a table within the table, finding the fields to filter on became a little messy. If you choose to make changes in that code, you will have to do some hunting to get to the correct fields.

So that’s a review of the UI. I have tried to keep it fairly simple and light since I don’t expect it to be used very often. The next phase, reporting may force me to make some changes, but that will be discussed in the next blog.

Happy Sending.

~ Jeff

Related Content

Building an Email Archiving System: The Challenges and of Course the Solution - Part 1

Our Senior Messaging Engineer describes the process he went through in order to store the email body onto S3 and all relevant log data in MySQL.

read more

Building an Email Archiving System: Storing the Email Body - Part 2

Our Senior Messaging Engineer describes the process he went through to store the email body onto S3 and ancillary data into a MySQL table.

read more

Building an Email Archiving System: Storing the Email Log Data - Part 3

In part 3 of his series on email archiving Senior Messaging Engineer, Jeff Goldstein, explains the process of storing log data associated with the email.

read more

Get started and start sending

Try SparkPost and see how easy it is to deliver your app’s email on time and to the inbox.

Try Now

Send this to a friend