SparkPost Labs: A/B Test API

Chris McFadden
Mar. 15, 2017 by Chris McFadden

Introducing SparkPost Labs

We love to experiment with new ideas at SparkPost! Our regular hackathons are one way our engineers bring to life new ideas. Our engineering team has cooked up some great user features and tools during hackathons or other engineer initiated projects. Some favorite features that began life as hackathon entries are webhooks, multi-factor authentication, the new real-time suppression api, CSS inlining, and even the use of AngularJS for our web UI.

But it isn’t easy to take a proof of concept that is hacked together in 24 hours and release it in the wild. As you might expect, we have high quality standards at SparkPost and expect our APIs to be fully vetted, conform to performance standards, and be versioned. Normally for a hackathon project to make its way to production as a fully supported feature, it will require additional work by Engineering to ensure the feature is fully baked. The downside of our super high standards is many of these very cool projects don’t get to the top of the priority list.

That’s why we created SparkPost Labs—a new program that helps our engineers more easily share new and experimental API and App functionality and other helpful tools with our developer community. This way we can find out sooner if there is interest in the feature, get valuable feedback from our community, and decide to fully-productize if it’s a hit. It’s a great way for our community to be actively involved in the direction of SparkPost!

We’re launching SparkPost Labs by introducing a new A/B Test API endpoint.

What You Need to Know

SparkPost Labs features are different from other new API features that we often expose to customers in beta form. When we beta a feature it is versioned, supported, and production-ready. While beta features may sometimes be initially limited in scope, we release them with the full intention that they will be supported and expanded over time. However, what we release under SparkPost Labs may be discontinued completely or re-envisioned before making it into the core supported product. It’s really up to you, our community, to help us determine what features are most valuable to take the extra mile. Caveat emptor!

Here are some helpful tips to keep in mind when you try out anything marked as “SparkPost Labs” in our API docs:

  • Labs API functionality is not versioned. As such, it is available under a distinct /api/labs/ path rather than /api/v1/ path.
  • Any API feature accessed via /api/labs/ is by definition experimental and subject to change. We may decide to change the behavior or discontinue it at any time. We also may decide to incorporate Labs functionality into the core supported product.
  • Normal support channels or SLAs are not available for SparkPost Labs. You can provide feedback via this Google form. We will route your feedback to our engineering and product teams.

A/B Test API

An A/B test is a fully managed API method of comparing templates to see which one performs better, and automatically choosing the better performing template. It is super simple to use: you provide two or more templates, a recipient list, a sample size, and a test duration to begin. The A/B Testing API endpoint provides the means to create new tests, and view completed results.

The API randomly chooses recipients from the recipient list equal to the sample size. The API distributes sample recipients among the templates and transmits emails.

 SparkPost labs image 1

When the test time has expired, the template with the best conversion rate wins.

 sparkpost labs image 2

All remaining recipients will be sent the winning template.

 sparkpost labs image 3

Example API Call

POST /api/labs/ab-testing

Response

The A/B API endpoint is our first SparkPost Labs project. If there is enough interest from the community in this kind of experimental feature we will do more – so please give us some feedback!

If you have any questions or comments about SparkPost Labs please don’t hesitate to connect with us @SparkPost – and we are always hiring.

Chris McFadden
VP Engineering

Related Content

How to Run a Hackathon: Tips and Tricks for Success

Our engineering teams just wrapped their annual winter hackathon. Hear from our VP of Engineering on how to run a hackathon and tips for your own future events.

read more

Community Driven Development at SparkPost

Over the past year, we’ve revised our product and engineering processes to embrace community driven development. Rich walks you through the new process.

read more

How SparkPost Built the Best Email API for Developers

How our strong engineering culture, technical excellence and API-first mentality have helped us build the best email API for developers.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and get started for free.

Get Started

Send this to a friend