Using community feedback to build the best email API
Steve Jobs once said, “Simple can be harder than complex: You have to work hard to get your thinking clean to make it simple.” This is certainly true when it comes to building a simple and easy-to-use API, which for SparkPost is the primary user interface that developers interact with. So how did we create the best email API that developers love? Read on to learn some tips from the SparkPost Engineering team.
One of our first decisions was to adopt a RESTful architectural style. Representational state transfer (REST) based web APIs have been used since the early 2000s. In the last several years, it has become the de-facto standard of cloud APIs. RESTful APIs, including the SparkPost API, has a collection of resources (https://api.sparkpost.com/api/v1/sending-domains/mydomain.com for example). These resources, described using JSON, allow clients to interact with it over the network using the standard HTTP verbs (GET, POST, PUT, DELETE), and get standard HTTP response codes (200, 500, 400, etc ) back.
The RESTful architecture style improves the simplicity, portability, interoperability, and modifiability of an API. Using HTTP and JSON makes it easy for developers, regardless of which programming language they use – PHP, Ruby, and Java – to integrate with our API without knowing our underlying technology. Other styles that were more common in the past like SOAP and JSON-RPC are generally obsolete. Today, more developer-friendly RESTful APIs replace them.
Important as it is to decide on an architectural style, there are many choices to make when implementing the API’s interface. Oscar Wilde once said, “consistency is the last refuge of the unimaginative,” but if your choices are not a consistent set of conventions, then the API will be very hard to understand and use. At SparkPost, we have multiple teams working on different parts of our API. Within those teams, people work on different APIs at different times. Therefore, it’s very important that our API is consistent in the use of conventions. To achieve this, we established a governance group including a few engineers representing each team and a member of the Product Management team. This group is responsible for establishing, documenting, and enforcing our API conventions across all teams.
When we build a new API endpoint or enhance an existing one, we always document and peer review changes first before a single line of code is written. This approach helps enforce our conventions, keeps things consistent, and ensures the API is very user friendly. It also cuts down on the amount of rework, bugs, and breaking changes. We use an API documentation markdown language called API Blueprint and manage our docs in Github. We encourage our community to contribute and improve upon these open source docs. Initially, we published our docs to Apiary, a great tool for prototyping and publishing API docs. However, embedding Apiary into our website doesn’t work on mobile devices so we now use Jekyll to generate static docs. Eventually, we may switch over to Swagger (now Open API) as the markdown format since this new standard continues to gain traction.
Technical Excellence and Transparency
We have a strong engineering culture and are proud of what we build. It is not sufficient to just have an idea for a great API. We need to build APIs quickly, iterate on them rapidly, and verify that they’re reliable and stable. For this reason, we embrace Continuous Delivery methodologies so that we can make changes and deploy them quickly and reliably. It is important to developers that we provide a rock solid service.
We peer review all API code changes, run them through unit and functional tests, periodic performance tests, and automatically deploy and smoke test them. We roll back changes automatically without impacting our users if any failures are detected. We also have extensive monitoring using a variety of tools including Circonus and OpsGenie. Thanks to these tools, we are immediately alerted when there are availability issues that need attention. When production issues affecting customers arise, we can respond quickly since we have engineers on-call 24/7. We also use our Status Page to keep our developer community updated.
Listen and Respond to the Developer Community
When we first released our API, it was quite minimal, supporting the most important features of sending email via API and SMTP as well as some reporting with the Metrics API. We collected feedback through customer interviews, user testing, surveys, and developer conferences early on. Members of our developer community recently sent us a lot of feedback on our Slack channel, which we launched in response to the Mandrill announcement. Subaccount support and IP pool management and our developer friendly freemium pricing are examples of improvements to the API in response to developer feedback, and on a weekly basis we make many improvements based on suggestions from the community.
We have a small team of developer advocates that focus on listening to developers and improving the developer experience. This team helps maintain our API docs, client libraries, and community Slack channel. In addition, they attend and sponsor developer meet-ups like CharmCityJS, hackathons such as Bitcamp, and conferences including Fluent. This team also acts as internal advocates for the developer community and helps drive improvements across the service including the APIs and our support process. Beyond the small team, we have many who contribute on a part-time basis to the developer relations mission.
APIs Are For Integration
While RESTful APIs are easy to integrate against with any language, it is very convenient to have a client library that you can use to get up and running quickly. We are proud of the wide range of client libraries and integrations available for SparkPost. These include Node.js, Java, Go, Python, and others listed on our Developer Hub. We also have integrations for WordPress, Zapier, Heroku, and have several community contributed libraries for Ruby and C#.
Versioning and No Breaking Changes
We released v1 of our API, with the public beta of SparkPost, upon two deliberate decisions. The first is that versioning will be in the URI rather than as a header. This is easier than requiring an API or schema version to passed in as a header. Secondly, by explicitly putting the version into the URI, we committed to no breaking changes to the API. Breaking changes occur when the behavior of an API call changes, which breaks a customer’s integration. We take this seriously and prevent them through code reviews and automated tests. However, we can and do extend the API by adding new endpoints, parameters and inputs, and response values. Will we ever need a v2 of the API? Perhaps down the road when we wish to change the whole semantics for the API. However, we don’t foresee a need for that soon.
Don’t Forget About SMTP
We know that most developers still like the ease and portability of using SMTP to send their transactional or marketing email. Therefore, we added the ability to have much of the same functionality over SMTP as you have with the RESTful API. The X-MSYS-API header allows you to pass in campaign, meta data, and control data such as turning on engagement tracking or setting the IP Pool. The SMTP API also supports subaccounts.
I hope you learned a few things from how SparkPost built the best email API that developers love. Take what you learned and apply it to your own API development. Do you love building awesome APIs? Come join us, we are always hiring. If you have any questions or comments please mention us on Twitter or find us on our community Slack channel.
—Chris McFadden, VP Engineering