Email Testing Tools: 3 Handy Tricks
Signing up for email newsletters is fun. I usually spell my email address correctly. Sometimes, when I don’t, someone else with a clever typo in their email doesn’t understand why they’re getting the messages I signed up for. With SparkPost’s handy testing tools (and double opt-in), be protected from Mayhem like me.
We protect our users by detecting negative feedback from email receivers and automatically suppressing future messages from your account to those addresses. It can be difficult to test your integration since SparkPost only emits the events in question under specific difficult to reproduce circumstances. Let’s see how to use a couple of simple tools that recreate those test conditions for you to avoid future Mayhem by testing to make sure you’re applying those suppressions to your list as well.
Two types of negative feedback (AKA Mayhem) that can be tricky to test are 1) out-of-band (OOB) bounces, where a receiver initially accepts the message then later sends a bounce notification and 2) feedback loop (FBL) reports which typically mean a recipient clicked a “this is spam” button.
Clicking “this is spam” on your own messages is dangerous because that can damage your sending reputation and composing an OOB bounce is tricky to get right. But don’t worry, we’ve got you covered with some, you guessed it, handy tools!
Simulating Negative Feedback
The first step of testing both types of Mayhem is to send yourself a message through SparkPost. We’ve got a tool for that too! Once you’ve got the message in your Inbox, save the message, including all of the headers, and deleting any blank lines at the very top. You’ll be using this file shortly.
Both of these tests will add the receiving address to your suppression list! Afterwards, you’ll need to manually remove the address to continue to receive emails from SparkPost. I usually use our Postman collection for this. Here are some instructions showing how to get that up and running.
Now, unless you’re connecting from somewhere that blocks outbound connections on port 25 (behind a firewall, residential ISP), you’re all ready to test! The install instructions for both tools suggest some ways to get around blocked ports.
Fake an FBL
The fblgen tool lets you generate and send an FBL report. Here’s a dry run:
$ ./fblgen --file ./test.eml --verbose Got domain [sparkpostmail.com] from Return-Path Got MX [smtp.sparkpostmail.com.] for [sparkpostmail.com] Would send FBL from [firstname.lastname@example.org] to [email@example.com] via [smtp.sparkpostmail.com.:smtp]
Running the same command and adding the –send flag will attempt to deliver the message to the detected MX. This will trigger an FBL event of type spam_complaint, which will flow through to message events and any configured webhooks. This tool will also cause an increase (of 1) in the count_spam_complaint metric and the “spam complaints” value shown in the SparkPost summary report.
Instead of waiting around for an FBL or risking damage to your sending reputation by triggering one yourself, this tool gives you a way to predictably trigger an FBL event from your own systems.
Bogus OOB Bounces
The oobgen tool lets you generate and send an out-of-band bounce message. Here’s a dry run:
$ ./oobgen --file ./test.eml --verbose Got domain [sparkpostmail.com] from Return-Path Got MX [smtp.sparkpostmail.com.] for [sparkpostmail.com] Would send OOB from [firstname.lastname@example.org] to [email@example.com] via [smtp.sparkpostmail.com.:smtp]
Again, running the same command with the –send flag will attempt to deliver the message to the detected MX. This will trigger an OOB bounce event of type out_of_band, which will flow through to message events and webhooks. This tool will also cause an increase (of 1) in the count_outofband_bounce metric and the “out-of-band” bounce value shown in the SparkPost bounces report.
Once you’ve gotten a collection of events in JSON format, perhaps by using Postman and our message events endpoint, what’s the easiest way to do some analysis on that data? If you’ve made it this far, then you’re probably comfortable on the command line, and the answer is to use jq!
This handy tool can extract the parts you care about from a big blob of JSON, for example counting the number of events of each type that were received in the last two hours:
$ curl -H 'Authorization: YSBmYWtlIGFwaSBrZXksIG1hZGUgeW91IGxvb2sh' \ https://api.sparkpost.com/api/v1/message-events?from=$( \ date -uv -2H +'%Y-%m-%dT%H:%M') > ~/events.json $ jq '.results | ..type' ~/events.json | sort | uniq -c | sort -rn 355 "open" 249 "injection" 220 "delivery" 77 "delay" 59 "click" 36 "bounce" 4 "out_of_band"
With these handy email testing tools, testing those uncommon, hard-to-trigger events is now easy. Making sure these types of events are handled correctly in your integration with SparkPost will help you prevent future Mayhem from striking, by keeping your list clean.
Did you get a chance to try out any of the tools mentioned in this post? Let us know if and how they fit into your testing processes, and feel free to create an issue or submit a PR on Github if there are any other features you’d like to see.