Consuming SparkPost Webhook Events with Loggly – Part 1 of 3

Benjamin Dean
Mar. 25, 2015 by Benjamin Dean

Continuing from my last blog post, How Developers Can Use Transactional Email for Customer Success, I thought it would be best to step through how to quickly setup a Loggly account to act as a SparkPost Webhook Event Consumer.

This will be released over the next three weeks for each of my blog posts:

  1. Consuming SparkPost webhook events with Loggly
  2. Configuring Loggly to deliver Node.js customer issues
  3. Translating data alerts into customer notifications in Node.js

Why would we want to setup a Loggly event consumer?

  • Time: We may need to get a system in place quickly to handle events until we can homeroll the code ourselves
  • Ability: As a SparkPost customer, your customers may need this benefit, but you may not have the technical ability. Loggly makes it pretty easy
  • Focus: It may not be your core focus, and you need something to handle events for you

What are webhooks?

According to Wikipedia, “A webhook (or WebHook), in web development, is a method of augmenting or altering the behavior of a web page, or web application, with custom callbacks.” So how does SparkPost implement Webhooks and how could we use them with our transactional email to improve customer success?

SparkPost Webhooks are a system that allows developers to easily configure the endpoints upon which they want to receive event notifications relating to their transactional and commercial email transmissions.

The following graphic illustrates a very common pattern seen while implementing transactional email using SparkPost Webhooks. [click to e]or inefficient time-management) on our customer’s expections of our applications. Let’s take a moment to review some general customer scenarios and what our customers might expect.

  • Inbox Deliverabilty: Customers expect transactional email to arrive in their Inbox vs their Spam folder.
  • Prompt Delivery: Customers expect transactional email to arrive in a timely manner. This is especially important for transactional emails such as:
    • Password recovery
    • Account abuse notifications
    • Billing change notifications
    • Legal change notifications
    • Purchase receipts
    • Physical delivery notifications
    • Any other type of operational or business transaction
  • Fault-Tolerant System: Our customers should not have to suffer a poor experience because a transactional email fails to deliver (for whatever reason).
  • Alternative Channels: When alternative channels are available, our exception handling is able to use them to ensure our customers feel successful; for instance, sending a text message if/when an email gets bounced or blocked.

Now that we have refocused on our customer’s experiences, let’s take a look at how we are able to leverage Loggly and the SparkPost Webhooks API to improve each of our customer’s experiences to become consistently positive!

What You Will Need

We’re going to focus only on the Loggly and SparkPost Webhooks API integration portion of this feature. In my next blog post I’ll address how to implement this into a Node.js application.

  • SparkPost Account: Signup for a new SparkPost account if you don’t have one.
  • SparkPost API Key: Make sure it ONLY has the data access permissions you minimally require
  • Loggly Account: They offer a great full-service freemium account which gracefully degrades (after 30 days) to a limited-service free account. Regardless, Loggly is great at what they do, so that’s why we’re using them here!

Technical considerations for SparkPost webhooks event delivery

Webhooks in SparkPost have a few technical considerations that we will need to think about when we define our Loggly endpoint:

  1. Event consumer URI is required to create a Webhook
  2. Only delivers events over HTTP (TCP/IP)
  3. The HTTP(S) request is a POST
  4. The POST Request Body is in JSON format
  5. Events are batch-delivered using one of two scenarios (whichever occurs first):
    1. When there have been 1000 recorded events
    2. Every sixty (60) seconds
  6. Currently does not support Web Sockets
  7. Does not support sending domain abstraction
  8. URI for consumer endpoint must be live (SparkPost will make a request to it prior to permitting you to save)
  9. There is a short-term queue mechanism in place when the event consumer endpoint is unreachable or returns anything other than a 200 to the HTTP POST Request with the event(s)

Create the Loggly endpoint to work with SparkPost webhooks

Let’s define our new Loggly event consumer for our webhooks. I say define because we are going to have to make some modifications later.

  1. Login to your Loggly account
  2. Choose “Source Setup” from the main menu: This is where we configure our webhook endpoint
  3. Choose “Server Side Apps” from the side-tabbed group: This is how Loggly organizes event sources, we could also use “Client Side Apps” to get to source type in our following step
  4. Choose “Direct from Application” => Endpoints => HTTP/S Bulk Endpoint: We have to use the Bulk Endpoint because SparkPost Webhook Events “can” be batched in groups of up to 1,000.
  5. Send a test event to Loggly: Follow their directions to send a test event using cURL or a REST client
  6. Verify that Loggly is receiving the test event: Use the “Verify now” button on the HTTP/S Bulk Endpoint configuration page (this may take up to 20 minutes)
  7. All Good?: Copy the URL in the “Step 2: Configure Your Endpoint” so we can create our webhook in SparkPost

Create the SparkPost webhook

Make sure you have copied the URL for the Loggly event consumer endpoint before proceeding with the following steps. Read the section above to learn how to do this.

  1. Login to your SparkPost account
  2. Select the “CONFIGS” menu item from the navigation on the left
  3. Create an API Key for use with SparkPost Webhooks View/Modify data access permissions configured. (make sure to copy the key, it is only displayed this one time)SparkPost-Webhook-API-Key
  4. Choose “WEBHOOKS” => “NEW WEBHOOK” (button)
  5. Complete the new webook form fields (see the screenshot below for an example)Loggly-Webhook-Configuration
  6. NOTE: You can choose the events you want to receive at your endpoint, but we are just choosing all the events. Below is a screenshot containing the full list of events you can filter to send to your event consumer.
  7. Test your new “Loggly Event Consumer” Webhook: Click the “Test” button to the right of the Webhook name in the UI to make sure the webhook is working (screenshot with the test button is below)Loggly-Webhook-Configured-List-View-Item
  8. Review the test payload being sent to Loggly:

     
  9. If the test checked out, then you should see a green message saying “The test was successful!” And you should see your webhook in the listings.

You have now configured SparkPost to send your email transmission events to Loggly. Here are our next steps:

  • You could go read how to use tags and notifications (aka: alerts) in Loggly (to get ahead of next post in the series)
  • In Part 2 of this mini-series I’ll show you how to configure tags and alerts in Loggly so we can communicate with our Node.js application based on certain thresholds of our event data.
  • Please comment below with any observations, ideas for improvements, or suggestions on topics that you’d like me to address.

Thanks for reading and happy coding!

Related Content

New Feature: Initial Renders Metric for Better Engagement Tracking

We added a new email open metric to SparkPost’s email analytics features. Learn how it works and how it delivers additional insight into user engagement.

read more

SparkPost Available in AWS Marketplace

Learn how the launch of SparkPost on AWS Marketplace makes it even easier for AWS customers to integrate SparkPost into their cloud applications.

read more

Creating the Next Generation of Interactive Emails with Rebel Mail

Meet our partner, RebelMail! Today they're demonstrating how to use their interactive email API to keep customers engaged with the inbox.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and get started for free.

Get Started

Send this to a friend