Introducing SparkPost Labs

We love to experiment with new ideas at SparkPost! Our regular hackathons are one way our engineers bring to life new ideas. Our engineering team has cooked up some great user features and tools during hackathons or other engineer initiated projects. Some favorite features that began life as hackathon entries are webhooks, multi-factor authentication, the new real-time suppression api, CSS inlining, and even the use of AngularJS for our web UI.

But it isn’t easy to take a proof of concept that is hacked together in 24 hours and release it in the wild. As you might expect, we have high quality standards at SparkPost and expect our APIs to be fully vetted, conform to performance standards, and be versioned. Normally for a hackathon project to make its way to production as a fully supported feature, it will require additional work by Engineering to ensure the feature is fully baked. The downside of our super high standards is many of these very cool projects don’t get to the top of the priority list.

That’s why we created SparkPost Labs—a new program that helps our engineers more easily share new and experimental API and App functionality and other helpful tools with our developer community. This way we can find out sooner if there is interest in the feature, get valuable feedback from our community, and decide to fully-productize if it’s a hit. It’s a great way for our community to be actively involved in the direction of SparkPost!

We’re launching SparkPost Labs by introducing a new A/B Test API endpoint.

What You Need to Know

SparkPost Labs features are different from other new API features that we often expose to customers in beta form. When we beta a feature it is versioned, supported, and production-ready. While beta features may sometimes be initially limited in scope, we release them with the full intention that they will be supported and expanded over time. However, what we release under SparkPost Labs may be discontinued completely or re-envisioned before making it into the core supported product. It’s really up to you, our community, to help us determine what features are most valuable to take the extra mile. Caveat emptor!

Here are some helpful tips to keep in mind when you try out anything marked as “SparkPost Labs” in our API docs:

  • Labs API functionality is not versioned. As such, it is available under a distinct /api/labs/ path rather than /api/v1/ path.
  • Any API feature accessed via /api/labs/ is by definition experimental and subject to change. We may decide to change the behavior or discontinue it at any time. We also may decide to incorporate Labs functionality into the core supported product.
  • Normal support channels or SLAs are not available for SparkPost Labs. You can provide feedback via this Google form. We will route your feedback to our engineering and product teams.

A/B Test API

An A/B test is a fully managed API method of comparing templates to see which one performs better, and automatically choosing the better performing template. It is super simple to use: you provide two or more templates, a recipient list, a sample size, and a test duration to begin. The A/B Testing API endpoint provides the means to create new tests, and view completed results.

The API randomly chooses recipients from the recipient list equal to the sample size. The API distributes sample recipients among the templates and transmits emails.

 SparkPost labs image 1

When the test time has expired, the template with the best conversion rate wins.

 sparkpost labs image 2

All remaining recipients will be sent the winning template.

 sparkpost labs image 3

Example API Call

POST /api/labs/ab-testing

Response

The A/B API endpoint is our first SparkPost Labs project. If there is enough interest from the community in this kind of experimental feature we will do more – so please give us some feedback!

If you have any questions or comments about SparkPost Labs please don’t hesitate to connect with us @SparkPost – and we are always hiring.

Chris McFadden
VP Engineering

Community Driven Development

The Community Driven Development Approach

April 2015 was an exciting time to be an engineer at SparkPost. Our service had just launched. We were seeing the results of all the hard work and dedication from previous quarters spent building our functionality, infrastructure, and teams.

Fast-forward to late February 2016. SparkPost had been generally available for ten months. Our community was growing steadily. We continued to make refinements to our service and infrastructure. We didn’t know it at the time, but MailChimp had just announced some important changes to Mandrill that would change the way we interact with our community and how we improve and support our product.

In the days following the announcement, we’d go on to set up our community Slack and revise our product and engineering processes to embrace community driven development. We used the feedback and experiences to drive our decision making processes to execute on what really makes our community successful.

Channels for information

Several mechanisms for feedback from our community exist at SparkPost. I mentioned our community Slack. There, our community has access to SparkPost team members and, more importantly, each other. We learn from you not only through direct interaction but by observing, listening, and taking notes. Any member of our team can take feedback from community Slack and post it directly to a feedback channel in our internal Slack. There, our engineers, product managers, and executives have direct access to your comments.

community driven development slack screengrab

Support tickets and Github issues are another essential method for understanding ways we can make our community’s lives easier. Our support team does a great job of escalating issues internally to our engineering team as well as bringing up issues that are occurring repeatedly and brainstorming ways to address. Github issues allow us to understand how our client libraries are being used and where we can make improvements.

Attending meet-ups, conferences, and hackathons has been a major source of learning for us too. We get to spend quality time face-to-face with our community, learn about how you use SparkPost and what improvements might help you and your business be more successful. After each event we make a point to share learnings internally so that we can discuss ideas and issues that arise during our conversations with attendees.

What happens behind the scenes?

Collecting all this valuable input wouldn’t mean much if we didn’t act on it. In the past three quarters we’ve acted on a little over 50 items that were a direct result of the information we gathered from our community. These smaller items are what we consider quick wins – things like updating language in our UI, improving documentation, bug fixes, and minor feature improvements. Here are some examples:

  • Click to see event details in Message Events UI
  • Allow customers to clone Templates in UI
  • Create code samples for cc/bcc in the client libraries
  • Surface all 4xx API errors in the web UI
  • Add information about rate limiting and error responses to the docs
  • Fix click tracking on <area> elements

Additionally, the engineering, product, and support teams meet weekly to discuss customer impacting issues, solutions, and action items. Like many software companies, we also have a prioritized list of larger initiatives that we work our way through. Some examples of work completed from the past few quarters include:

Prioritization

Prioritization of new features and improvements is a constant work in progress. We do our best to compile all of the great feedback we receive and maintain a balance between addressing current issues and the planning and execution of new functionality and improvements. Some additional items we’ve heard our community request include:

  • Deliver webhooks by subaccount
  • View the contents of sent emails in the UI
  • View billing history in the UI
  • Add the ability to use a test API key
  • Reporting by geography in the UI
  • Support for stored templates with subaccounts

Lastly, the best way to stay up-to-date with what’s new as well as fixes we deploy is to join the #announcements channel in our community Slack or keep an eye on our change log article. We update both every Friday when we deploy changes that we want to make sure you know about.

Onward & Upward

With 2017 nearly upon us, we’re looking forward to implementing some of the improvements and features you’ve told us are important to you. Our community plays a major role in the development and evolution of SparkPost. We love to hear from you, so keep the feedback coming. And keep pushing us to deliver the best service we can.

—Rich Leland, Director of Growth Engineering

ps: Thoughts on community driven development? How do your teams prioritize and process feedback? Leave us a comment below.