What’s In Our Email Infrastructure Stack
Openness is extremely important to us at SparkPost. As engineers, we value transparency, and we bet you do too. We’re also deeply geeky about email infrastructure, so this post is a peek into our production software stack. On our tour, we’ll proceed “front to back,” as it were, following the path your API requests take from the public internet through our service. We’ll also touch briefly on the technologies that make up our email API and expose a few of the key technology decisions we made along the way. If you’d like a list of the tools we use in production, operationally and internally, our StackShare page has exactly that.
0: Network Edge
Our tour starts when a request arrives at our network from the public internet. Whether you’re delivering messages through our Transmission endpoint, checking engagement metrics or pulling fine-grained Message Events into your app, your request hits our load balancer first. It isn’t particularly complex; it maintains health checks on our internal hosts and distributes inbound traffic evenly across the SparkPost application.
If you’re paying attention, you might wonder how we handle your SMTP traffic. Well, SMTP comes in through the front door too, but it follows a different path than HTTP. We’ll explore SMTP in a moment. The next stop for HTTP requests is the application boundary: nginx.
As an aside: This section is labelled “0” not for any geeky array-indexing reason but because our network edge is really just outside of our application, so it’s sort of a “layer 0.”
1: The Application Boundary
The outermost layer in our application is based on the venerable nginx proxy. In actual fact, we use OpenResty, which packages nginx up with a load of very useful modules. For each inbound request, nginx first calls on our internal authentication service to identify and authenticate the caller. After tagging the request with authentication details, nginx then forwards it to the correct component for fulfillment.
As we scale and maintain each group of services in our infrastructure, nginx is also one of the most useful control points for fanning out requests to clustered services.
After this point, our request has 3 main paths: some go directly to our message management platform, others are handled by our application service layer, and we also use nginx to serve our web UI. We’ll look at all three components in turn.
2: Message Management
SparkPost’s Message Management layer is built around the Momentum MTA platform. Momentum has been a core element of many high-performance email stacks for over a decade and it really deserves an article of its own, so we’ll just cover the basics here. This is where Transmission requests turn into message streams. Momentum accepts Transmission requests and generates fully-formed messages. It manages queuing and IP pool assignment, then performs coordinated massively-parallel delivery across all customer traffic. Just from that list, it’s clear that Momentum is at the core of SparkPost’s email infrastructure.
Momentum is a modern standards-compliant and extensible message management platform. It’s written in C for performance and has both native SMP and clustering capabilities to let us scale up and out. We use both dimensions in the SparkPost stack. Those are all table stake features. Momentum’s other hugely valuable trait is in its flexibility. Embedded within Momentum is a Lua-based policy engine that lets us make fine-grained messaging decisions live as messages pass through the platform. Better yet, it lets us change the decisions we make with minimum friction and without sacrificing performance.
So it’s clear that SparkPost isn’t all API calls and UI. We also support SMTP delivery through our SMTP API. Interestingly, Momentum has its roots in bidirectional SMTP traffic management, so it both delivers outbound SMTP traffic and accepts inbound SMTP for our relay webhooks service.
SMTP’s path through our infrastructure is necessarily different than HTTP traffic. Of course, we accept SMTP through a load balancer at the boundary just like HTTP. However, Momentum’s policy engine lets us handle authentication, configuration, filtering, accounting, CSS inlining, engagement tracking (and the rest) in-process, in parallel and spread across our cluster of Momentum nodes.
2.3 Adaptive Email Network
We use Momentum for one other very important job: reputation protection by live traffic shaping. In essence, Momentum uses traffic shaping rules collected through our Adaptive Email Network to adhere to each receiving ISP’s acceptance policies. Our AEN rules also tie into system monitoring, alerting and automatic corrective capabilities – but that’s a whole other article.
For both SMTP and API-driven traffic, Momentum’s final task is to generate a stream of events that informs the rest of the application on everything it’s doing, from generation and delivery status to engagement tracking and accounting details. There’s more on this in the IPC and Storage sections below. First though, a very quick glance at the other parts of the SparkPost API.
Any request not serviced by our Message Management layer is handled by one of our other API services. Amongst other things, these Node.js based API endpoints provide our metrics, webhooks and account configuration capabilities, as well as fulfilling numerous internal services. Did you just have your daily limit bumped by our Support team? That was an internal service call. Switch from Pro to the SuperPro plan? Service call. Our services expose controls and config for our users and our staff. They’re small, fast, clustered and they all rely heavily on our IPC and Storage layers, coming up next!
4: Inter-process Communication
Email infrastructure is one part execution (y’know: delivery), one part analytics, and real-time email analytics needs lots of event data. As our Message Management layer generates, delivers and tracks user message streams, it pushes a detailed stream of tracking events through our internal event hose for the rest of the app to consume. Delivery phases, template composition results, bounces, FBLs, engagement events – all these (and the rest) are stuffed into our event hose for processing in our IPC layer.
The front-end of our IPC layer is based on RabbitMQ which distributes incoming events on durable queues to the rest of the application. On the consumption side, we use a suite of Node.js services that are responsible for extracting, transforming and loading events into final storage. These ETLs support the metrics, webhooks and message events services as well as various internal utilities, but they couldn’t exist without a resting place for all that data.
The SparkPost application uses two main forms of storage. We use Cassandra for operational data, account information, integration and configuration details. Our analytics capability is a little different though. Email analytics demands high volume, low-latency summaries of fine grained event data. To provide this type of live rollup query with user-controllable granularity, we spent a lot of time evaluating options and settled on the HP Vertica distributed analytics platform which is serving us (and you!) extremely well. Both of these are distributed services as you might expect but they’re quite different in their capabilities and platform requirements. Together though, they provide a solid data storage and retrieval substrate and in some ways they underpin all that we do at SparkPost.
Alright, here ends the tour. Modern email infrastructure is a broad and deep topic and we barely scratched the surface here, but hopefully you learned a little about our innards. To learn more about using the SparkPost service, check out our DevHub and if you’d like to chat or have an email-driven project in mind, come join us on Slack!
Successful email engagement is predicated on trust. Recipients need to trust that your emails are actually from you. That’s why SparkPost requires you undertake additional setup steps versus services that are less concerned with reputation.
A secure email service starts by preventing phishing—emails that fraudulently claim to be from someone else in an illicit attempt to gain benefit. It’s the email that claims to be from the reader’s bank, but downloads malware or steals a password when they follow the message links.
In order to reassure both human beings and protective computer systems that are looking for fraudulent email activity, SparkPost requires you to set up either SPF or DKIM at a minimum and potentially DMARC reporting as a means of reassuring these recipients and systems that messages are truly from you. SPF is like a guest list at a party—a list of servers that are authorized to send for your domain. DKIM uses encryption in the email headers that refers to the sender’s domain to legitimize the message for the recipient. It assures your recipients that even though your emails are coming via SparkPost, people and systems can really trust that your messages are from you. In part, DMARC is a reporting mechanism for DKIM failures, alerting domain owners when others try to use their domain without authorization.
Email security is also about protecting your recipients. Attackers are getting more sophisticated and they know that there are often ways to get information from people. They’ll leverage email to get bits of information that a person might not think are important, but in fact are stepping stones to getting more valuable information. That well-known data breach at Target started when someone found a bit of information in a company that did air conditioning services for stores; the attackers leveraged that third party’s access to hack into additional systems inside Target.
SparkPost requires you to use DKIM and SPF because they are proven best practices. When you’re sending just a few emails, things could go wrong, but the small quantity means it only qualifies as a hassle. When you’re sending email in very high volumes, there’s a lot of potential for damage. On the flipside, email security increases engagement and deliverability: messages set up with authentication allow recipients to trust that the links they click will not be fraudulent or malicious. That increases clicks (a measure of engagement), which in turn increases deliverability.
That’s why it’s important to secure your email, because any little piece of the puzzle can be a gateway to significant problems for you and your recipients. Our goal is to make you successful with email, and protecting your email is essential to securing your reputation as a sender.
Q&A With David Rowley, Message Systems SVP and General Manager of SparkPost
Today is the day. For the past six months or so, the most important topic of conversation at Message Systems was the SparkPost project – the internal development effort to create a cloud-based version of our Momentum platform. Or more accurately, I should say the “Fuzzy Dragon” project, because that was the code name that the effort proceeded with through much of its existence. I recently sat down with David Rowley, the executive tapped to lead the new venture, to discuss how SparkPost came into being, why it’s big news and what it means for the email industry and the developer community. – JP
Q. Message Systems has always been a loud and proud enterprise software company selling on-premises solutions. Why SparkPost, and why now?A. First, Message Systems still is a loud and proud enterprise software company, and our on-premises business remains core to everything we do. The simple answer to why SparkPost and why now is that the nature of the tech world has changed. Five or ten years ago, brands that needed to send significant amounts of email had two choices: 1) work with an ESP, or 2) implement an on-prem solution. And the choices with on-prem were limited. There were the big three open-source MTAs – Postfix, Sendmail and Exim – and then a handful of commercial offerings, of which Momentum became known as the platform of choice for ultra high-volume senders. In recent years, developer culture has fundamentally changed the way new technology products and companies get launched. The demand for cloud infrastructure as a service is strong and is just going to keep getting stronger – Gartner reports IaaS will be the fastest growing area of public cloud computing over the next few years. Our core email infrastructure platform has measurable advantages over all other MTA products on the market. Making it available in the cloud as an IaaS offering opens up vast new possibilities for developers and for the company.
Q. So SparkPost is designed as a developer-focused offering?
A. Exactly. SparkPost gives developers easy API access to the very same high performance email sending and delivery optimization platform as our installed base, but now in a public cloud, pay-per-use offering. Message Systems is all about infrastructure. Engineering is our core strength and we have no interest in becoming a full-on ESP with agency services and a creative department. Some of the longest tenured Message Systems customers are ESPs; we value those relationships very much and expect them to continue for a long time and in fact SparkPost is drawing a lot of interest from ESP startups who would like to use us as their delivery platform.
Q. How long has SparkPost been in planning and development?
A. Having a cloud offering is something we’ve been considering for as long as I’ve been with Message Systems. Our product people and engineers work with developers constantly – heck, they’re developers themselves – and we’ve kept a close eye on how the market for cloud transactional email has developed. More and more over the past couple of years we’d find ourselves talking with prospective customers, usually startups or mid-size tech companies, interested in leveraging the same technology as the peers they looked up to. But gradually it would turn to “oh, it’s only available as on-prem? Gee, if it was cloud-based this would be a no brainer.” More and more, the preference is for infrastructure in the cloud. The really high volume senders, and regulated industries like telecom and financial services, still have a strong preference for on-premises. But for everyone else in 2014, cloud is strongly preferred.
Q. It was a market-driven decision to launch SparkPost.
A. Absolutely, yes. As I said, George (Schlossnagle, president, co-founder and head of the company’s product management) and the team have been looking at cloud as a logical next step for some time. As the category continued to grow and evolve, consensus built within the organization that, hey, we really believe we can do this better than it’s being done currently. When our new CEO Phillip Merrick joined us back in April, one of the first decisions he made was to pull the trigger on a cloud offering. Development began in earnest shortly thereafter, and it’s been full speed ahead ever since. Today it all comes together.
Q. It’s a pretty crowded market out there. SparkPost is going to be competing directly with established cloud email offerings like SendGrid, Mandrill, Dyn and a few others. Where does SparkPost fit, and what makes you different?
A. No doubt there are a lot of choices out there and the SendGrids and the Mandrills have done a nice job of showing that email via API is an important category. What’s different is this: SparkPost is built on Momentum, the platform that powers 20 percent of the world’s legitimate email. It’s the choice of the industry’s biggest senders, like Twitter, Groupon, LinkedIn, Comcast, Facebook and Salesforce. Momentum is also in place as receiving infrastructure at some of the biggest ISPs in the world, including Rackspace, Cablevision and many, many others. With the volumes that Momentum carries, both outbound and inbound, we have better intelligence into what’s happening with email at any given point in time than all but a few. With our Adaptive Email Network we collect ISP bounce and disposition data all over the world. We’ve reached approximately 98% ISP domain coverage in North America, South America, Europe, China and Australia. That intelligence informs our Adaptive Delivery technology, which automatically shapes traffic to keep deliverability optimized at all times. These capabilities are not available anywhere else, neither from cloud IaaS providers or on-premises MTA offerings.
Q. So the sheer volumes that you handle and the data you can collect are differentiators?
A. Sure. Our head of field operations, Barry Abel, likes to say that we’re the rails over which the email train travels. We know what’s coming and going better than anyone else out there. But it’s important to keep in mind that at the end of the day, it comes down to the quality of the I in IaaS – how good is your infrastructure? That’s where we believe we’re head and shoulders above the field. Momentum was built from the ground up as a superior alternative to the open source message transfer agent (MTA) products – Postfix, Exim, Sendmail – that dominated the email landscape through the early years of the commercial Internet. And over the past ten years or so, Momentum has been conclusively proven to be superior to anything else out there. What kind of infrastructure are the incumbent players in the IaaS email space running on? They’re built on open source, or commodity commercial MTA technology that can’t measure up to what’s capable with Momentum. So their customers get poor inbox delivery, limited actionable visibility into email delivery statistics and sub-par service.
Q. Today’s announcement was a beta launch, and apparently SparkPost has some users already. What are the early reviews?
A. Yeah, Justin Newton from NetKi had some nice things to say about the excellent deliverability they’re seeing already from SparkPost, as well as the real-time data access and analytics, and we’ve already had customers using Momentum in the cloud since earlier this summer, sending millions of messages a day. We’re also thrilled to be integrating into the Heroku platform, enabling the excellent developers in that community to leverage SparkPost.
Q. It’s pretty remarkable that you were able to get what’s essentially a start-up company from conception to launch in about a six-month period.
A. I’ve had the privilege of working with some great teams throughout my career, but the team we’ve assembled here is absolutely the most talented, enthusiastic and committed group I’ve ever had the pleasure to work with. There is huge excitement in the team to be able to provide the technology that we’ve been so proud of for so many years and deliver it in the cloud into the hands of our developer peers in the industry – and we’re just getting started! We look forward to seeing all the innovations that our developer community will build on top of SparkPost!
Peek behind the IT curtain of nearly any large organization and you’re bound to find multiple email systems linked to multiple data sources, with little coordination companywide and no integrated messaging infrastructure in place. It stems from the common practice of partitioning message deployment systems by business division or function — with each department maintaining its own in-house and sometimes outsourced sending capabilities. As companies add mobile text, IM and social messaging capabilities to the customer communications mix, the problems posed by redundant messaging infrastructure are mushrooming. It’s not just that maintaining multiple systems and operations gets costly, what’s worse is that the disconnects between the communications layer and the data layer beneath it inevitably lead to less effective customer engagement.
Customer Communications on a Single Platform
Message Systems solutions are based on the concept of a single, unified system for managing all messaging across the organization. Our solutions enable even the largest businesses to consolidate their diverse messaging infrastructure, and help you to realize the full value of your customer data. It doesn’t matter if you have five data sources involved in your messaging programs or fifty, a Message Systems solution can serve as the central communication facility and control point for all your message-based customer communication. Because our messaging infrastructure platform includes a rules-based policy engine, scripting capabilities and modular framework with open APIs, you can tap any CRM instance or database in order to source customer insight for informed customer interactions.
Consolidate and Save
Many of the enterprises that rely on Message Systems solutions find that their investment in our technology pays for itself through lower operational expenses and higher ROI in a matter of months. A total economic study by Forrester Consulting found that a mid-sized company ($200 million in annual revenue) could expect a risk-adjusted ROI of 115%, along with a payback period of 10 months by consolidating messaging infrastructure on the Momentum platform. Total cost savings and revenue gain? $1,069,399 over three years:
Hardware avoidance savings: $58,192
Fewer servers are needed for the Momentum platform versus legacy on-premises solutions.
IT labor savings: $615,496
The Momentum platform automates many laborious and taxing manual tasks when it comes to traffic shaping and deliverability optimization, leading to far greater ease of management.
Incremental gross margin profit: $395,711
Many organizations on commodity MTAs suffer from poor deliverability. The Momentum platform increases deliverability, leading to increased revenue and profits.
Whether you’re replacing outdated MTA-based messaging infrastructure or bringing your sending operations in house through a phased deployment, consolidating on Message Systems technologies can increase ROI shortly after implementation. Message Systems delivers benefits like these because our solutions enable you to streamline your messaging operations in many different ways.
Read the full Momentum total economic impact report by Forrester Consulting – Learn how to drive revenue and enjoy cost savings with a more efficient email infrastructure.
When you’re a wildly successful business – especially an online business – wildly enormous messaging loads come with the territory. To handle the kind of digital messaging traffic that fast-growing businesses need to cope with (we’re talking millions per day and beyond) you need industrial-strength messaging infrastructure that can scale on demand. The problem is, most small businesses start off with the least expensive messaging infrastructure they can find, which usually means open source MTA technology.
Avoid the Open Source Trap
To meet increasing volumes with open source, you need to incrementally add hardware and staff, which gets expensive fast. Worse still, open source provides poor visibility into your messaging operations. There’s no way to capture data for triggered messaging, and no easy way to parse email disposition data to manage your sender reputation with ISPs. This lack of capabilities prevents you from using business rules and logic to create the kinds of email operations that drive engagement in today’s communications environment.
Messaging Infrastructure that Scales On Demand
Adopting an on-premises messaging solution like Momentum from Message Systems is not only a far more cost-effective approach, but also it gives you the power, scalability and flexibility to grow your business into a fully realized global brand. It’s no coincidence that top cloud, social media and e-business leaders like Facebook, Match.com, LinkedIn, PayPal, Groupon and so many other online innovators are Message Systems customers. Companies that adopt Message Systems right from the point of launch, or very early in their development, gain enormous advantages through superior scalability, deliverability and overall engagement rates. Here are some of them:
Meet Growing Volume Requirements
Carrier-grade performance means you can easily handle higher volumes of email, mobile and social messages – millions per hour on each server – as your business grows and demand increases.
Increase Your Agility
Scriptable policy engine and API toolkit make it easy to create fresh new message-based offerings, like triggered notifications, that drive traffic and increase engagement.
Make Scalability a Business Advantage
With the ability to scale and handle fast-growing message volumes, messaging becomes a business advantage you can use to outpace competitors and create barriers to entry for potential adversaries.
If you’re interested in learning more about why the ability to scale on demand is so important to your business, have a look at our free eBook, The High Cost of Free Messaging Software. Learn how open source MTAs can be hurting, rather than helping, your business.
The release of Momentum 4 back in June was something of a watershed for Message Systems. The Momentum platform had existed for 10 years as pure email delivery platform. Those of us responsible for telling the Message Systems story usually avoid the term message transfer agent (MTA) to describe Momentum, because it had long offered capabilities – including automated delivery optimization and data management features – not found in open source and commercial MTA productions. Still MTA functionality is a key part of what Momentum provides. Those deliverability features are still key to the platform, but Momentum 4 now provides capabilities for stages in the messaging process both pre-sending, and post-sending.
For Message Systems customers and email industry professionals eager to learn more about Momentum 4 and what it offers, product manager Amie Durr has helpfully put together a detailed white paper. Amie was the executive in charge of the Momentum 4 development effort, and is one of our most knowledgeable executive on the new product. The white paper walks through how Momentum 4 helps senders improve engagement by moving message generation capabilities from separate business intelligence, CRM or CMS platforms into the sending platform itself. Momentum 4’s native message generation capabilities provide for better performance and on-time delivery of email and mobile campaigns.
Additionally, a completely redesigned user interface in Momentum 4 provides real-time access to rich messaging and customer engagement data for deep analytics. These capabilities provide marketers and email delivery professionals with real-time access to critical messaging and engagement data. This reduces the expense and costly delays associated with delivery problems and performance issues that can result in blacklisting, hours poring through log files, and the need for direct outreach to ISPs for resolution. Users can easily discern the success of their campaigns, export data for analysis, and take immediate action instead of waiting for reports from providers that may not arrive until days later.
Remarking on the launch, longtime Message Systems customer Jack Hogan, co-founder and CTO at Lifescript, said:
With the release of Momentum 4.0, Message Systems is keeping its edge as the top-tier platform for enterprise-class senders. The analytical and data integration capabilities in the new platform will help email senders become more effective than ever before in creating compelling campaigns, reaching the inbox and engaging customers.
Learn more about Momentum 4 in our detailed white paper: A New Architecture For Today’s Complex Messaging Needs.
Automated deliverability optimization? Momentum’s Adaptive Delivery® can help you with that. Learn more in this white paper!