Global-EmailLet’s take a closer look at why Message Systems originally created the Adaptive Email Network, how it works, and why we’ve kept expanding its reach around the world. For email senders, the relationship with ISPs can seem adversarial. Their acceptance policies aren’t always clear, and the rules they impose can change dynamically. While some ISPs publish their rules to help guide senders into optimal sending practices, many others don’t publish rule sets at all. Some change their rules frequently, others seldom do. Given there are thousands of ISPs around the globe, it’s impossible for senders to a) keep track of rules worldwide, and b) continually tweak/update their sending infrastructure to reflect constant changes.

Turning Back the Clock: Adaptive Email Network Origin Story

In 2011, the Adaptive Email Network had collected rules data from all the world’s major ISPs that account for the vast majority of global email traffic. However, many senders using the Momentum platform send a significant portion of their messages to receivers in markets such as South America, Australia, China or remote parts of Eastern Europe where much traffic is handled by smaller ISPs. As these Message Systems customers expanded into new markets such as Brazil and India, and sent growing volumes of email to smaller ISPs, they weren’t seeing the consistently high deliverability rates they’d enjoyed in North America and Western Europe.

Our solution was to expand the reach of the Message Systems Adaptive Email Network to include rules for as many ISPs as possible. That effort started off in 2013 and progressed into 2014 to the point where AEN has now reached approximately 98% ISP domain coverage in North America, South America, Europe, China and Australia.

Q&A With David Rowley, Message Systems SVP and General Manager of SparkPost

david-rowleyToday is the day. For the past six months or so, the most important topic of conversation at Message Systems was the SparkPost project – the internal development effort to create a cloud-based version of our Momentum platform. Or more accurately, I should say the “Fuzzy Dragon” project, because that was the code name that the effort proceeded with through much of its existence. I recently sat down with David Rowley, the executive tapped to lead the new venture, to discuss how SparkPost came into being, why it’s big news and what it means for the email industry and the developer community. – JP


Q. Message Systems has always been a loud and proud enterprise software company selling on-premises solutions. Why SparkPost, and why now?
A. First, Message Systems still is a loud and proud enterprise software company, and our on-premises business remains core to everything we do. The simple answer to why SparkPost and why now is that the nature of the tech world has changed. Five or ten years ago, brands that needed to send significant amounts of email had two choices: 1) work with an ESP, or 2) implement an on-prem solution. And the choices with on-prem were limited. There were the big three open-source MTAs – Postfix, Sendmail and Exim – and then a handful of commercial offerings, of which Momentum became known as the platform of choice for ultra high-volume senders. In recent years, developer culture has fundamentally changed the way new technology products and companies get launched. The demand for cloud infrastructure as a service is strong and is just going to keep getting stronger – Gartner reports IaaS will be the fastest growing area of public cloud computing over the next few years. Our core email infrastructure platform has measurable advantages over all other MTA products on the market. Making it available in the cloud as an IaaS offering opens up vast new possibilities for developers and for the company.sparkpost270

Q. So SparkPost is designed as a developer-focused offering?

A. Exactly. SparkPost gives developers easy API access to the very same high performance email sending and delivery optimization platform as our installed base, but now in a public cloud, pay-per-use offering. Message Systems is all about infrastructure. Engineering is our core strength and we have no interest in becoming a full-on ESP with agency services and a creative department. Some of the longest tenured Message Systems customers are ESPs; we value those relationships very much and expect them to continue for a long time and in fact SparkPost is drawing a lot of interest from ESP startups who would like to use us as their delivery platform.

 

Q. How long has SparkPost been in planning and development?

A. Having a cloud offering is something we’ve been considering for as long as I’ve been with Message Systems. Our product people and engineers work with developers constantly – heck, they’re developers themselves – and we’ve kept a close eye on how the market for cloud transactional email has developed. More and more over the past couple of years we’d find ourselves talking with prospective customers, usually startups or mid-size tech companies, interested in leveraging the same technology as the peers they looked up to. But gradually it would turn to “oh, it’s only available as on-prem? Gee, if it was cloud-based this would be a no brainer.” More and more, the preference is for infrastructure in the cloud. The really high volume senders, and regulated industries like telecom and financial services, still have a strong preference for on-premises. But for everyone else in 2014, cloud is strongly preferred.

Q. It was a market-driven decision to launch SparkPost.

A. Absolutely, yes. As I said, George (Schlossnagle, president, co-founder and head of the company’s product management) and the team have been looking at cloud as a logical next step for some time. As the category continued to grow and evolve, consensus built within the organization that, hey, we really believe we can do this better than it’s being done currently. When our new CEO Phillip Merrick joined us back in April, one of the first decisions he made was to pull the trigger on a cloud offering. Development began in earnest shortly thereafter, and it’s been full speed ahead ever since. Today it all comes together.

Q. It’s a pretty crowded market out there. SparkPost is going to be competing directly with established cloud email offerings like SendGrid, Mandrill, Dyn and a few others. Where does SparkPost fit, and what makes you different?

A. No doubt there are a lot of choices out there and the SendGrids and the Mandrills have done a nice job of showing that email via API is an important category. What’s different is this: SparkPost is built on Momentum, the platform that powers 20 percent of the world’s legitimate email. It’s the choice of the industry’s biggest senders, like Twitter, Groupon, LinkedIn, Comcast, Facebook and Salesforce. Momentum is also in place as receiving infrastructure at some of the biggest ISPs in the world, including Rackspace, Cablevision and many, many others. With the volumes that Momentum carries, both outbound and inbound, we have better intelligence into what’s happening with email at any given point in time than all but a few. With our Adaptive Email Network we collect ISP bounce and disposition data all over the world. We’ve reached approximately 98% ISP domain coverage in North America, South America, Europe, China and Australia. That intelligence informs our Adaptive Delivery technology, which automatically shapes traffic to keep deliverability optimized at all times. These capabilities are not available anywhere else, neither from cloud IaaS providers or on-premises MTA offerings.

Q. So the sheer volumes that you handle and the data you can collect are differentiators?

A. Sure. Our head of field operations, Barry Abel, likes to say that we’re the rails over which the email train travels. We know what’s coming and going better than anyone else out there. But it’s important to keep in mind that at the end of the day, it comes down to the quality of the I in IaaS – how good is your infrastructure? That’s where we believe we’re head and shoulders above the field. Momentum was built from the ground up as a superior alternative to the open source message transfer agent (MTA) products – Postfix, Exim, Sendmail – that dominated the email landscape through the early years of the commercial Internet. And over the past ten years or so, Momentum has been conclusively proven to be superior to anything else out there. What kind of infrastructure are the incumbent players in the IaaS email space running on? They’re built on open source, or commodity commercial MTA technology that can’t measure up to what’s capable with Momentum. So their customers get poor inbox delivery, limited actionable visibility into email delivery statistics and sub-par service.

Q. Today’s announcement was a beta launch, and apparently SparkPost has some users already. What are the early reviews?

A. Yeah, Justin Newton from NetKi had some nice things to say about the excellent deliverability they’re seeing already from SparkPost, as well as the real-time data access and analytics, and we’ve already had customers using Momentum in the cloud since earlier this summer, sending millions of messages a day. We’re also thrilled to be integrating into the Heroku platform, enabling the excellent developers in that community to leverage SparkPost.

Q. It’s pretty remarkable that you were able to get what’s essentially a start-up company from conception to launch in about a six-month period.

A. I’ve had the privilege of working with some great teams throughout my career, but the team we’ve assembled here is absolutely the most talented, enthusiastic and committed group I’ve ever had the pleasure to work with. There is huge excitement in the team to be able to provide the technology that we’ve been so proud of for so many years and deliver it in the cloud into the hands of our developer peers in the industry – and we’re just getting started! We look forward to seeing all the innovations that our developer community will build on top of SparkPost!

shutterstock_130183808_600px

Let’s take a closer look at why Message Systems originally created the Adaptive Email Network, how it works, and why we’ve kept expanding its reach around the world. For email senders, the relationship with ISPs can seem adversarial. Their acceptance policies aren’t always clear, and the rules they impose can change dynamically. While some ISPs publish their rules to help guide senders into optimal sending practices, many others don’t publish rule sets at all. Some change their rules frequently, others seldom do. Given there are thousands of ISPs around the globe, it’s impossible for senders to a) keep track of rules worldwide, and b) continually tweak/update their sending infrastructure to reflect constant changes.

 Turning Back the Clock: Adaptive Email Network Origin Story

In 2011, the Adaptive Email Network had collected rules data from all the world’s major ISPs that account for the vast majority of global email traffic. However, many senders using the Momentum platform send a significant portion of their messages to receivers in markets such as South America, Australia, China or remote parts of Eastern Europe where much traffic is handled by smaller ISPs. As these Message Systems customers expanded into new markets such as Brazil and India, and sent growing volumes of email to smaller ISPs, they weren’t seeing the consistently high deliverability rates they’d enjoyed in North America and Western Europe.

Our solution was to expand the reach of the Message Systems Adaptive Email Network to include rules for as many ISPs as possible. That effort started off in 2013 and progressed into 2014 to the point where AEN has now reached approximately 98% ISP domain coverage in North America, South America, Europe, China and Australia.

How Does the Adaptive Email Network Work Its Magic?

The Adaptive Email Network and Message Systems Adaptive Delivery are unique offerings, and were designed specifically to automate the entire process of complying with ISP rules and remediating bounces and feedback loop responses (FBLs). The Adaptive Email Network collects bounce codes from ISPs worldwide. This data is compiled, processed and made available to Message Systems customers through daily live updates, which in turn automatically update sending configuration parameters within the Momentum platform. The AEN also collects data on unclassified bounce codes from the Message Systems customer base, and the deliverability team manually classifies these bounces and pushes global updates to all AEN members.

The actual process of expanding the AEN rules takes place largely through the analysis of hundreds and sometimes thousands of lines of bounce data. Unfortunately, as noted above, many ISPs don’t provide clear guidelines on whitelisting and sending rules. Some provide none at all, and others provide acceptance policy rules only in the local language. The Message Systems team logged many hours reaching out to ISP postmasters in these international markets to uncover exactly what their policies were. Once that picture became clear, the Message Systems engineering team scripted new capabilities into the Momentum platform in order for our customers to automatically comply with those ISP policies – just as they do with giant ISPs like Google, Microsoft and Yahoo.

… and thus the Adaptive Email Network was Reborn

As of this writing in late 2014, the AEN contains over 2,000 individual rules that collectively classify about 98% of bounces from ISPs around the globe. The system categorizes that data into 20 bounce categories in five different languages: Spanish, German, Portuguese, French and English. This classification process allows customers to make better decisions about future sends. Message Systems Adaptive Delivery technology uses the data collected through the AEN to automatically optimize sending practices for customers. It’s a proactive capability, automatically suspending or ramping up mailings based on real-time feedback. These automated traffic shaping capabilities ensure that customers maintain compliance with ISP policies such as the number of concurrent connections, volume of messages per hour, retry times, etc., which ensures consistently excellent deliverability.

Learn more about Adaptive Delivery and the Adaptive Email Network with following white paper!

Adaptive Delivery Whitepaper