Tracking Recipient Preferences With The User Agent Header in PHP

Ewan Dennis
Mar. 24, 2017 by Ewan Dennis

Segmenting audiences is a basic practice in email marketing. From super consumers to influencers, iPhone users to desktop holdouts, learning about your recipient’s preferences is clearly important. In these days of deep personalization, it’s also just nice to know a little more about your customer base. Luckily, this is a pretty easy job with SparkPost message events.

In this post, I’ll review the content of the User-Agent header, then walk through the process of receiving tracking events from SparkPost’s webhooks facility, parsing your recipient’s User-Agent headers, and using the results to build a simple but extensible report for tracking Operating System preferences. I’ll be using PHP for the example code in this article, but most of the concepts are transferrable to other languages.

SparkPost Webhook Engagement Events

SparkPost webhooks offer a low-latency way for your apps to receive detailed tracking events for your email traffic. We’ve written previously about how to use them and how they’re built, so you can read some background material if you need to.

We’ll be focusing on just the click event here. Each time a recipient clicks on a tracked link in your email, SparkPost generates a click event that you can receive by webhook. You can grab a sample click event directly from the SparkPost API here. The most interesting field is naturally msys.track_event.user_agent which contains the full User-Agent header sent by your recipient’s email client when they clicked the link.

Grokking The User Agent

Ok, so we can almost pick out the important details from that little blob of text. For the dedicated, there’s a specification but it’s a tough read. Broadly speaking, we can extract details about the user’s browser, OS, and “device” from the user agent string.

For example, from my own user agent:

…you can tell I’m an Android user with a Huawei Nexus 6P device (and that it’s bang up-to-date 😉 ).

Caveat: user agent spoofing

Some of you might be concerned about the information our user agent shares with the services we use. As is your right, you can use a browser plugin (Chrome, Firefox) or built-in browser dev tools to change your user agent string to something less revealing. Some services on the web will alter your experience based on your user agent though so it’s important to know the impact these tools might have on you.

Harvesting User Agents From SparkPost Tracking Events

Alright, enough theory. Let’s build out a little webhook service to receive, process, and stash user agent details for each click tracked through our SparkPost account.

First we’ll have to accept and decode a JSON-encoded POST request (omitting the necessary error handling boilerplate for brevity):

Then we’ll unpack each nested event structure into something more manageable:

Let’s retain just the click events (when we register our webhook with SparkPost later, we can also ask it to send only click events):

There’s a lot of detail in a single event. It might be a good idea to pare things down to just the fields we care about:

Now we can enrich our events with new information. We’ll parse the user agent string and extract the OS:

Note: not all user agent strings will contain the detail we want (or even make sense at all), so we label all odd-shaped clicks with “OS: unknown”.

Alright, now we have an array of events like this, containing only interesting fields and with an extra “os” field to boot:

Generating Report-Ready Summary Data

At this point, we could just list each event and call our report done. We’ve come to expect some summarization in our reports, to simplify the task of understanding. We’re interested in OS trends in our email recipients, which suggests that we should aggregate our results: collect summaries indexed by OS. Here’s a simple implementation that groups array elements by 1 or more field values to help with that:

Now we can group our events by OS and finally count the events in each group:

Now we have summaries like this:

That matches surprisingly well with the Google Charts pie chart data format. All that remains is to collect those summary records in our storage service of choice and render a report.

We could stop there, citing “exercise for the reader,” but I always find that frustrating. Instead, here’s a batteries-included implementation which stores click event summaries in PostgreSQL and renders a simple report using Google Charts. Here’s what that final chart looks like:

email recipient OS preferences user agent graph-1

An Exercise For The Reader

I know, I said I wouldn’t do this. Bear with me: if you were paying attention to the implementation steps above, you might have noticed several re-usable elements. Specifically, I drew a few filtering and reporting parameters out for re-use:

  • event type filters
  • event field filters
  • event “enrichment” functionality
  • aggregation/grouping fields

With minimal effort, you could add, filter on, and group the campaign_id  field to see OS preference broken down by email campaign. You could use it as a basis for updating your own user database from bounce events with type=bounce , fields=rcpt_to , bounce_class  and so on.

I hope this short walkthrough gave some practical insight on using SparkPost webhooks. With a little experimentation, the project could be made to fit into plenty of use cases and I’d be more than happy to accept contributions on that theme. If you’d like to talk about your own event processing needs, SparkPost webhooks, or anything else, come find us on Slack!

– Ewan

Share your Thoughts

Your email address will not be published.

Related Content

SparkPost Client Libraries: Oh The Places You'll Go!

SparkPost's client libraries allow you to access our API in a native-to-them programming language. Easy peasy, lemon squeezy!

read more

How We Use User Agents To Identify API User Groups

Read how we use user agents to measure client library usage and identify popular tools and technologies within the SparkPost developer community.

read more

New Major Release of the SparkPost Node.js Client Library

We broke down our Node.js client library and built it back up on top of a stronger foundation. Aydrian walks you through the updates, we hope you like it.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and send up to 100,000 emails per month for free.

Send 100K Emails/Month For Free

Send this to friend