What A Year
I’m currently sitting at a college hackathon, figuring out how to set up a triggered email response to folks who need a reminder about their New Year’s Resolutions. A good portion of my day has been spent talking students through how to set up the proper SPF and DKIM records in their DNS settings. To say I’ve learned a few things over the last year would be putting it lightly!
As I come up on the end of my first year at SparkPost, it’s easy to count my blessings, which is appropriate given that it’s Thanksgiving week.
First and foremost, I’m grateful for you, our community. Our mission is to empower and energize our community to be successful with SparkPost. But, I think I speak for all of us when I say that you’ve energized us as well. You’ve stepped up in ways that we never expected, writing client libraries, submitting innumerable pull requests, offering feedback, and helping us build SparkPost into the best cloud email platform. You’ve helped each other out and found like minds in our Slack community. Best of all, you’ve given us invaluable feedback, helped us refine our user experience, and built some pretty awesome applications.
I’m also grateful for all of the interactions we’ve had in person! From San Francisco to New York and everywhere in between, we’ve talked to you, worked alongside you, watched you build amazing applications, and had a lot of fun getting to know you better.
Lastly, I’m grateful for all that I’ve learned because of and through you. Email is hard, and I’ve learned the ins and outs of SPF, DKIM, IP pools, deliverability, suppression lists, and more, and then done my best to make that new-found knowledge available to you as well. After all, the more you know, the more email-savvy you become, and that makes everyone’s lives better.
The Best is Yet to Come
As we prepare for 2017, there’s lots of good things to come! More tools and sample applications, better resources and ever-improving documentation. Also, even more conversations with you about how we can better serve your needs! Finally, speaking of conversations, if you’re planning to attend AWS re:Invent in Vegas next week, be sure to say hi! We’ll be the ones in bright orange t-shirts, and we’ve actually set some time aside on Wednesday to buy you a cup of coffee. Be sure to swing by!
Have feedback for us? Come chat in Slack — we’d love to hear what you’re working on and how we can help! Thank you for your efforts in making this community a great place to be!
We love our developer community and I personally love learning about you. How you use SparkPost, what tools you favor, where you are and what you need. In a recent post, we took a brief peek into the software stacks you collectively use. This time, we’ll go a little deeper into a one of the key community data sources that we follow: the User-Agent HTTP request header.
For those familiar with it, the User-Agent header might not seem like a very rich source of information. I often find it easy to go a little non-linear while analysing large data sets so it’s instructive to stick to just a single bite-sized chunk of data at a time: in this case, your user agent strings.
What’s In User Agents?
When your app hits the SparkPost API, it can optionally include a User-Agent HTTP header in its request. User-Agent includes some details on your HTTP client, your software stack and occasionally even your host operating system. Web admins commonly use it to learn about what browsers people use on their site. However, it’s also very useful with a developer-centric email API service like SparkPost because it shows what client libraries you all use with which versions, and some other interesting tidbits too. With that out of the way, let’s dig into some data.
Client Library Usage By Version
We like to follow the usage patterns of our “Big 3” client libraries: PHP, Python and Node.js. You can see how they stack up in my last post on the topic. We find it useful to break down client library usage of each one by version so our library maintainers can see the spread of versions you all use.
We’ll look at PHP first because it forms the largest single group with 44% of transmission requests in the last week.
Most requests come from php-sparkpost/1.x with only 13% coming from our 2.x series client which was released in June. The 2.0.0 release introduces a refactored client library with a new interface so it’s possible that mostly new PHP users will pick it up since 1.x still functions nicely and is presumably integrated with many customer stacks at this point. Still, those folks will also get a nicer experience. PHP was the first client library to bump to 2.0 – uptake is farthest along there.
Python is next on our leaderboard with 33% of transmission traffic. Python users are more spread across versions with about half using either last or last-but-one releases.
Last but by no means least, we have node-sparkpost where version 2 was released just a few weeks ago.
Node.js users seem to keep up quite well with client library releases. Even though node-sparkpost/2.x was released 2 weeks ago, we are already seeing traffic from it. Given that our Node.js client is 2 years old (happy birthday node-sparkpost!), the 2.x release was overdue so if you’re a Node coder, go check it out. With features like in-built promise support and a slew of cleanups and usability improvements, we predict some quick uptake here.
It’s Not Just The Software
Beyond identifying our own client libraries, there are other useful details in our trusty little User-Agent header. For example, we can identify subcommunities or at least users with common needs and once we do that, we can learn about their needs and how best to support them.
One such subcommunity we recently discovered is the Google App Engine user-base. After fielding questions from the occasional community member, we found that App Engine users live under quite specific constraints. For instance, python App Engine users need special attention due to the nature of the App Engine python environment. Also it’s interesting to see that App Engine adds its own details to its customers’ outbound HTTP requests. That means we can gauge popularity of the platform with our users. On the back of these discoveries, we’ll be paying close attention to our App Engine user-base from now on.
The Imperfection Of One-Dimensional Data
No data set is without its foibles and ours has some ‘unique’ features worth mentioning. We collect User-Agent information from calls to /api/v1/transmissions since we use it to send emails. Interestingly, a good half of requests to transmissions have no User-Agent header at all. That privacy is your right but it does reduce our sample size so we’re calling it out here.
Further, we’re comparing call volumes with a given User-Agent string. It’s tempting but not 100% correct to consider “call volume” as a proxy for popularity.
For instance, if we see 1 million calls marked with ‘User-Agent: captain-clown/1.0’, we might assume the captain-clown HTTP client to be quite popular when it might just be very inefficient. Maybe those 1 million calls could be condensed into 2 or 10. This case is exaggerated to show the issue. Call volume as popularity still has some value, especially across groups of User-Agents as we have done in this article.
I hope you’ve enjoyed our brief look into the SparkPost community. Our community is hugely diverse, using widely varied tools and technologies and with an exciting and ever changing spread of business models. We’re still discovering our community’s make-up and looking for ways to serve more people, better, quicker. We also strive to be as open and welcoming as possible. Want to learn more about our community, share your own experiences, or draw our attention to your own subcommunities? Come find us on Slack or fire us a tweet.
I have an old-school pocket dictionary at home (literally, old school… it’s the Primary School Dictionary of the English Language from 1874). As a child, I was fascinated with the “pictorial illustrations” on the back. Everything from flags and musical instruments to animals and forms of punishment. My favorite page, however, was always the “Vehicles for Land and Aërial Locomotion.”
It was always fascinating to me how differently everyone chose to travel. They might all be going to the same place, but how they got there was the true story in my mind. Was it a dog-cart, and how big were the dogs?!? Or a Calash, which always resembled Cinderella’s pumpkin carriage in my mind, or maybe even a balloon? My imagination could go wild, and it often did.
We want you to be as successful as possible in all of your email-sending endeavors, with whatever stack you happen to be using. So this is where our client libraries come into play. They’re designed to drop into your existing environment and allow you to access our API in a native-to-you programming language, meaning your lives are presto chango easier (almost) instantly! (You’re welcome 😉)
Since we know how important this service is to you, there’s a lot of process that goes into these libraries. We make sure that they’re tested and ready for you to use on a regular basis. I sat down with a few members of our team to find out about this process and the thought that goes into developing these libraries.
Why Client Libraries?
But all of this takes work, and we want that to be on us, not on you – easy peasy, remember? Early on in our client library days, we established a set of guidelines for testing, samples, code style, and an explanation of how you, our community, could contribute (wink wink, nudge nudge). These guidelines set us all up for success. For example, making sure that everything is 100% tested before it’s released. They also help make the process easier for you. Our requirement for consistent code style, based on community best practice, keeps our libraries readable and, thus more maintainable.
If It Ain’t Broke…Right?
When the libraries were first created, each author decided how they would interface with the SparkPost API. Did they want it to be lightweight and simply a wrapper? Or did they want to offer a variety of conveniences for the user, flattening out the parameters? It was the wild west, and we were grateful to any and all who stepped up to the siren call of client libraries.
But then the feedback started coming…
Rich Leland, Director of Growth Engineering, said, “We put our first cut of these libraries out just after having a product launch, and quickly ran into a good problem: we had so much community feedback that we couldn’t reply fast enough!”
We quickly learned that a thin wrapper allowed us to maintain the libraries more easily, as well as keep the libraries relatively lightweight for the community. The heavier libraries were bogging us down. They forced us to completely change each of the client libraries every time we made any changes or additions to the API.
Software Engineer Avi Goldman joined our team just after we noticed this exact problem occurring with our PHP library. He spent time identifying the good bits and then worked on refactoring the rest of it.
“There was already a really good framework for testing,” he said. “We settled on a simple request function and everything revolves around that one central point. Now it’s simply a thin wrapper with some syntactic sugar around the things that are difficult to do, like cc and bcc.”
Basically, there’s now a base resource that can be used to call out to the SparkPost API with a one-to-one mapping that’s consistent between payload parameters in PHP-land, and what is listed in our API documentation. In other words, we’re back to easy peasy lemon squeezy!
Our End Goal: Your Happiness
Ultimately, our goal is to make the lives of our community member’s easier – no matter where they’re coming from. With that goal in mind, we’re always working to improve our API as well as the client libraries. In addition, we’re building various tools and resources to help you along the SparkPost path.
So, what vehicles… err, languages, do you use to get to SparkPost? And how can we, as community advocates, make your journey easier? Come talk to us in Slack, or leave a comment below.