Building an Email Archiving System: Reporting – Part 5

Jeff Goldstein
May. 3, 2019 by Jeff Goldstein


* This blog is addressing the process(s) in green

In what I expect to be my last installment of this project, I’m going to describe what I’ve done for reporting. Admittedly, if you look at the code which I have placed in the step4Reporting folder within this GitHub repository you will notice that I have made changes throughout the code base just on the principle of building a better product. Some of the changes are around aesthetics, some on usability, others are around security and then of course reporting.

Compared to the previous blogs in this series, this one will be rather short since most of the work on reporting was done in previous steps; thus only a few coding additions to support reporting were necessary. The first change I’m going to discuss was something that I should have done earlier and simply didn’t think of it until someone pointed it out (ok, I guess I never thought about it); I added code to store (then display) the headers of the archived email. In a perfect world, what we really would want are the headers of the rcpt_to email; but we can’t get those, so this is the second best thing. I decided to store the headers in its own string field in the same table that holds all of the other archive data that was captured in the first development step. Now that I have that information stored, I was able to add the headers into the details section when the email body is being displayed.

At first glance, I wasn’t really happy with how that looked. Depending on the sending and receiving platforms, there can be a lot of headers. An email can easily have 20-30 headers! So in my sample project, I decided to also add code that allows the user to display or hide the headers as they wish. This is definitely a personal issue and an option that many of you may think is going overboard and definitely not worth coding!

Now that I have the header information like I should of in the first place, I started to think about the actual reports. So how would an auditor want to see, print or export this data? I decided on four different ways:

  1. Simply print the page. That one was easy, not a thing to do for that one!
  2. Print the detailed email that was being reviewed.
    • Wide view – In the previous development step where I built the inbox display UI I decided to display the email details in an alert box, but I was forced to change that to a new browser window for this step. The changes were minor but I had to do that in order to have the browser print just the email details. When using an Alert box, the browser would attempt to print both the Alert window and the underlining browser page at the same time; that is not the result I was looking for, and thus changed it to a pop-out window. All-in-all, I think it also produced a better-looking product as well.
    • Collapsed view – This was a little trickier. The email body and details are displayed in a table cell next to the events in the UI. So printing the email details would mean printing out the UI page; not just the specific email. So I decided to add a button that the user would select when they wanted to print the details while leveraging the collapsed view. This action would produce a separate page for printing; just like in the wide view. In order to keep display/print consistency between the two views, collapsed and wide; anytime I create the email body and details output, I store the whole thing in a hidden field. Then when I want to display the details in a new window for printing, I simply grab that field and display it. Now when the auditor wants to print out the details, they simply use the browser print functionality for that new window.
  3. The last reporting option is to create a CSV file for all events still being displayed in the UI (not filtered events). Since there are two different formats for the UI, collapsed and wide, I decided the easiest way to do this was to call the database again and retrieve the data that matches the filter entries. The PHP application grabs the data from the database, creates a CSV file, then passes it back to the ajax call in order to have the file downloaded to the clients desktop. This does create a temporary file on the server so I have a process that removes the files every 24 hours.

So I think this concludes the “Archive for Audit Purposes” project. I described the problem and proposed the solution in the first blog. Then, I built the solution and documented it in the next four blog posts. While I think this is the end of this post series; after showing this code to a few people, I may take this code base and mold it into a more general use display and search tool for all historical SparkPost events. Many SparkPost customers build their own inbox displays similar to this one in order to support their customer service team and I think this project with a few tweaks can be a great start on that project.

Happy Sending.

~ Jeff

Related Content

Building an Email Archiving System: Storing the Email Body - Part 2

Our Senior Messaging Engineer describes the process he went through to store the email body onto S3 and ancillary data into a MySQL table.

read more

Building an Email Archiving System: Storing the Email Log Data - Part 3

In part 3 of his series on email archiving Senior Messaging Engineer, Jeff Goldstein, explains the process of storing log data associated with the email.

read more

Building an Email Archiving System: Searching and Displaying the Data - Part 4

In the fourth installment of his blog series on building an email archive, Senior Messaging Engineer, Jeff Goldstein explains phase one of the Archive UI.

read more

Get started and start sending

Try SparkPost and see how easy it is to deliver your app’s email on time and to the inbox.

Try Now

Send this to a friend