Architectural Upgrades to Our Core Platform Making SparkPost Better

George Schlossnagle
Mar. 20, 2017 by George Schlossnagle

You might have heard that SparkPost is now the world’s fastest-growing email delivery service. (Our CEO, Phillip Merrick, recently wrote a blog about reaching that milestone.)

That’s why I’m writing. As our company’s co-founder, I wanted to say thank you for being such an important part of that growth. And as SparkPost’s CTO, I wanted to share some of the ways we’ve architected our platform to operate well at a very large scale—delivering billions of messages per month with minimal latency.

SparkPost’s cloud-native architecture is a big part of how we can support large senders so well. As we continue to grow, we are moving towards a centralized architecture that separates major components (things like message delivery, reporting, suppression services) into dedicated service tiers. In addition, we’re leveraging more aspects of the Amazon Web Services (AWS) stack to improve the performance of both visible features like our suppression list lookup and data reporting, as well as under-the-hood improvements.

Architectural upgrades like these are a little bit like an iceberg: some portion is visible, but much more lies beneath the surface. We’ve put in a lot of work and care to make these modifications without disrupting production systems. At that, we succeeded—if you’ve noticed any changes in your service, they’re likely to be positive ones.

This continuous improvement to our core platform has enabled some important enhancements. They include:

  • Faster Transmission API and SMTP injection performance. Message latency is a key performance benchmark, and we continue to strive to reduce it.
  • Increased elasticity of our core platform for even more responsive auto-scaling. And, across the board, separation of service tiers makes all components generally more reliable, performant, scalable, and secure. The value of that catch-all is hard to overstate.
  • Doubled webhook retention time from 4 hours to 8 hours. Webhooks sometimes are described as a “firehose” of data. A longer retention time is really helpful if your ability to consume webhooks is temporarily limited. We also reduced webhooks batch sizes from 10,000 to 100 to reduce the data-processing load on your servers.
  • Real-time suppression list updates. Previously, it could take up to an hour for additions or deletions to your suppression list to take effect. Now it is immediate. At the same time, suppression API rate limiting has been loosened up, and performance for lookups is significantly faster. These performance gains also have translated into more responsiveness in other parts of the architecture. (Note: if you are a SparkPost customer on our Enterprise add-on plan in the E.U., these suppression list enhancements will be rolling out to you soon.)

These enhancements all reflect something that drives our work every day: qualities like reliability and scalability are core features our service. In fact, they are very possibly the reason you selected us. So I’m very glad to be able to tell you about our ongoing platform work. That continuous improvement is a major benefit of a cloud service like SparkPost.

If you have any feedback or questions about our ongoing work building and improving the SparkPost platform, please share them with us. We genuinely would love to hear what you think.

-George Schlossnagle
CTO and Co-Founder, SparkPost

Any additional comments or questions on these architectural upgrades? Leave us a note below.

Share your Thoughts

Your email address will not be published.

Related Content

Get Up Out of That Chair! How to Stay Healthy at a Desk Job

A large percentage of office workers suffer from back pain. We have put together three simple steps on how to stay healthy while at a desk job.

read more

Calm Your Suppression List Anxieties with Python

Learn how to use Python to manage your suppression lists to avoid sending to stale, unsubscribed, bounced emails.

read more

Archiving Emails: A How-To Guide for Tracking Sent Mail

Many senders need the ability to archive sent emails to refer back to later. Here’s a sample app one of our engineers built to assist in archiving emails.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and get started for free.

Get Started

Send this to a friend