New API Feature: Message Events

Bob Evans
Sep. 18, 2015 by Bob Evans

New API Feature_Message EventsWe just released a new feature in our API: Message Events, which allows users to pull down recipient-level data. There has been a lot of demand for such an API, and I’m very proud to say implementing this was not as hard as you may think. Why? Because SparkPost is built using a microservice architecture and we are continually building additional microservices.  All of the event data is written to a message queue, and we create Node.js ETL processes that interface with a specific queue and implement specific business logic.  The advantage of this is we can reuse our event stream data for many different services: Metrics, Message Events, Event Webhooks, and Suppression Lists.

It was also very exciting to use a relatively new feature in Vertica called Flex Tables.  Think of it as MongoDB, but an enterprise quality version.  It did have to be tuned to support such high throughput, but since we had previous knowledge of Vertica for the Metrics API, it was straightforward and attainable in a few days. 

Also, we have built our first API that supports pagination. We are leveraging HATEOS (Hypertext As The Engine Of Application State) to build this feature. The intention is you can build applications that can chain REST calls. In the case of pagination, by default we return the most recent 1000 results in a given page. You can use the query parameter per_page to change this to a maximum of 10,000.  The data that is returned from the API will give you the results as well as total_count and links:

It allows you to programatically page through results by using the rel of first, previous, next, last. As you make subsequent requests the links returned in the response payload will dynamically change:

The primary use case is you can build a Web UI that can support dynamic paging, keeping the hard part of tracking number of pages/results out of your application and simply show/hide elements based on what is returned in the response payload.  At SparkPost we will leverage this in the future by building out our own Web UI.

One internal debate we had was the best query parameters to use: per_page/page and limit/offset.  Sure, limit/offset is the implementation behind the scenes but we wanted to provide a better user experience by translating limit/offset to more human readable/consumable names.

With the addition of Message Events to our API, we’ve added the power of searching for message event data based on recipients, campaigns, and more. Take a look at the documentation and as always, let us know what you think at developers@sparkpost.com

Learn more about microservices, Node.js,  ETL and HATEOAS here:

http://microservices.io/patterns/microservices.html

https://nodejs.org/en/

https://en.wikipedia.org/wiki/Extract,_transform,_load

https://en.wikipedia.org/wiki/HATEOAS

-Bob Evans, Manager, Application Engineering

Related Content

Preparing For Launch: The Ultimate Email Campaign Checklist

Our ultimate email campaign checklist takes the guesswork and anxiety out of launching large or small scale email campaigns.

read more

Meet Our New West Coast Developer Advocate

From Intern to Engineer to West Coast Developer Advocate -- Avi’s got big things ahead of him! Learn more about his journey and what he's most excited about.

read more

Community Spotlight: Rise And Shine With This Alexa Skill

Getting out of bed in the morning is easier with coffee and your new favorite Alexa skill: MyMorning.Online, our winners from the recent AngelHack.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and get started for free.

Get Started

Send this to a friend