New API Feature: Message Events

Bob Evans
Sep. 18, 2015 by Bob Evans

New API Feature_Message EventsWe just released a new feature in our API: Message Events, which allows users to pull down recipient-level data. There has been a lot of demand for such an API, and I’m very proud to say implementing this was not as hard as you may think. Why? Because SparkPost is built using a microservice architecture and we are continually building additional microservices.  All of the event data is written to a message queue, and we create Node.js ETL processes that interface with a specific queue and implement specific business logic.  The advantage of this is we can reuse our event stream data for many different services: Metrics, Message Events, Event Webhooks, and Suppression Lists.

It was also very exciting to use a relatively new feature in Vertica called Flex Tables.  Think of it as MongoDB, but an enterprise quality version.  It did have to be tuned to support such high throughput, but since we had previous knowledge of Vertica for the Metrics API, it was straightforward and attainable in a few days. 

Also, we have built our first API that supports pagination. We are leveraging HATEOS (Hypertext As The Engine Of Application State) to build this feature. The intention is you can build applications that can chain REST calls. In the case of pagination, by default we return the most recent 1000 results in a given page. You can use the query parameter per_page to change this to a maximum of 10,000.  The data that is returned from the API will give you the results as well as total_count and links:

It allows you to programatically page through results by using the rel of first, previous, next, last. As you make subsequent requests the links returned in the response payload will dynamically change:

The primary use case is you can build a Web UI that can support dynamic paging, keeping the hard part of tracking number of pages/results out of your application and simply show/hide elements based on what is returned in the response payload.  At SparkPost we will leverage this in the future by building out our own Web UI.

One internal debate we had was the best query parameters to use: per_page/page and limit/offset.  Sure, limit/offset is the implementation behind the scenes but we wanted to provide a better user experience by translating limit/offset to more human readable/consumable names.

With the addition of Message Events to our API, we’ve added the power of searching for message event data based on recipients, campaigns, and more. Take a look at the documentation and as always, let us know what you think at developers@sparkpost.com

Learn more about microservices, Node.js,  ETL and HATEOAS here:

http://microservices.io/patterns/microservices.html

https://nodejs.org/en/

https://en.wikipedia.org/wiki/Extract,_transform,_load

https://en.wikipedia.org/wiki/HATEOAS

-Bob Evans, Manager, Application Engineering

Related Content

Community Spotlight: How Topol Makes Creating Beautiful HTML Email Templates Easy

Building HTML email templates is hard. Learn about the app that Jan Tlapak and Sendmark created for anybody in need of building beautiful HTML emails.

read more

5 Easy Ways to Enhance Your Customer Segmentation Strategy

With hundreds of ways to segment email lists and personalize campaigns, these 5 methods of customer segmentation are an easy place to start.

read more

Learn To Love Your Unsubscribers

Unsubscribers are inevitable in email marketing. They’re often looked at negatively, but we prove how they do a lot of work for marketers by cleaning lists.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and send up to 100,000 emails per month for free.

Send 100K Emails/Month For Free

Send this to a friend