How to Generate Realistic Test Traffic Patterns in your SparkPost Account

Steve Tuck
Jun. 15, 2018 by Steve Tuck

Recently I’ve had customers see our Sales Engineer-powered demos, and ask “can I have test traffic like that in my own account”? Usually, they want to give a walkthrough to their colleagues of the SparkPost reporting before they actually onboard real traffic. Sometimes it’s during the integration phase, when developers have implemented their webhooks receivers, sent some basic test emails, and are ready to test with realistic traffic conditions.

SparkPost has a nifty “sink” feature that accepts any mail you send it. That’s great for basic testing without harming your email reputation, but all you’ll see in the events stream is “accepted” and “delivered”. Real-world users and ISPs don’t behave like that. Messages sometimes bounce. Someone hits the “Mark As Spam” button and you get a “spam_complaint” event. Your real users also open the emails and click on the links.

Until now, testing these more complex behaviors has required either bespoke solutions or messages to be sent to real users. Debugging in production is always exciting, and not advisable!

We realized it would be helpful to have a “bouncy sink” you can use that behaves like real-world ISPs and users, together with a “traffic generator” to make it easy. These are unofficial tools, rather than officially supported SparkPost features, but we’ve tried to make them solid, easy to use, and you can change them.

These requirements give the following design goals:

  • Fast and easy to get started – no downloading or installing anything on to your computer.
  • Minimum “just get it done” configuration.
  • Responds like real destinations – including opens, clicks, bounces (in-band and out-of-band), and just a few spam complaints (FBLs) in the mix.
  • Make the generator code simple and easy to change, if you want to.
  • Generator and sink completely independent, so that you can generate test traffic another way (e.g. from a campaign management platform) and the sink will still do the right things.

** Note that all sent messages count towards your account billing **

Plugging It Together

Here’s what we need to plug in and make this work:

  • The traffic generator, set up to talk to your SparkPost account. This is the part you see and touch.
  • Your SparkPost account, which just does its thing – delivering messages, handling the opens, clicks, bounces, and FBLs as usual.
  • The bouncy sink, which does Interesting Things™ to traffic that comes its way. I’ll describe how that was built in a following article.

Here’s how it looks:

The traffic generator periodically injects messages to test addresses, using the SparkPost API.

The bouncy sink handles each incoming message based on a statistical model that mimics what actual ISPs / users might do. It runs a probability tree to decide what happens to your mail.

To simulate real email behavior, after a message is opened by the sink, there’s a probability it may be opened again. Independently of that choice, opened messages may be clicked once, twice, or not at all.

Crank up the generator

Follow the simple instructions in the sparkpost-traffic-gen Github repo, and you should be buzzing. Heroku’s free tier (validated with a credit card) is generous, and you can run several apps at the same time without charges.

You’ll see a screen where you enter a few settings.

Heroku then takes a minute or two to spin up your generator. Soon you should see “Your app was successfully deployed”.

As per the project README – don’t forget to configure the scheduler, set to run every 10 minutes.

Open the app (top right) to see a summary:

After 10 minutes or so, you should see a valid “Started Running” time and the “SparkPost accepted (msg)” count should increase. That’s it – you’re running! Advance to Go, collect your $200, perform a victory dance, and open beers!

The lights are on in your account

The generator varies the amount sent each period. After a while, you’ll see a traffic pattern something like this.

The Bounce Report will show a variety of bounce classifications (Block, Hard, Soft, Undetermined), and both In-Band and Out-of-Band bounce types.

You will also see Spam Complaint events, although the probability is set realistically low.

The Traffic Generator sends short bursts of traffic every 10 minutes, which you’ll see if you zoom in to show the last hour. After each delivery/acceptance burst, you might see Accepted rates go negative, because out-of-band bounces arrive a minute or two after.

JSON – your golden fleece

See that little note at the bottom of your app status page? Your app exposes its own “mini API” of JSON results, enabling you to monitor your generator externally. You can, for instance, fetch and display that with curl  and jq .

Your app page and the JSON endpoint are completely open for anyone to view. If you need your test to be secret, then you’d need to add authentication to the app, something I’ve left as an exercise for the reader. One nice thing about Heroku is that you get https:// “for free” without having to get your own certificates.

Down in the engine room

In case you want to go down to the engine room and see your generator humming along, Heroku provides a nice facility to look at the app logs via your browser.

You should see the output from Redis, the scheduled process, and the web app.

What about other platforms?

The traffic generator can run on any platform that supports Python and Redis, which means “just about anything”. Instructions for a basic Linux install are in the INSTALLING doc. I hope to add other platforms soon (AWS Lambda, serverless.com, Azure functions).

Now you’re powered up

When you want to stop traffic, the easiest way is to just stop the scheduled task. Click the scheduler clock icon on your Heroku app page, and select Remove. You can easily restart it again if needed.

Bounces will populate your suppression list. It’s good practice to purge those entries relating to the sink domains when you’ve finished. Here is a tool that you can use to clean up.

Finally …

The Bouncy Sink is running on a single node and is offered as a courtesy to our community. It should be good for a few million messages per hour in total. I keep an eye on it but it’s not formally supported. We want your messages to bounce, not break.

In the second article, I’ll go into some more features of the Bouncy Sink and describe how you can set up one of your own.

Don’t forget – ** all sent messages count towards your account billing **

If you have any comments, improvements, or other suggestions, please get in touch via Community Slack or direct on Github. Feel free to submit issues or PRs there, as per the CONTRIBUTING doc.

That’s it! Crank up your own generator, watch those messages fly, and happy sending!

-Steve

Related Content

Musings From A Sparky Decennial

After ten years at the company, SparkPost's Global Pre Sales Engineering Lead reflects on what it's like to be a "Sparky Decennial".

read more

Back to School Shopping: Tips for Successful Email Campaigns

Learn how to increase the effectiveness and responsiveness of back-to-school shopping email campaigns by segmenting your audience.

read more

Discovering Your Financial Services Competitive Superpower

Learn about financial services' competitive superpower: using email and the cloud to enable great customer experiences and outperform competition.

read more

Get started and start sending

Try SparkPost and see how easy it is to deliver your app’s email on time and to the inbox.

Try Free

Send this to a friend