Twice a year, SparkPost hosts an employee hackathon. The idea is to give our hard-working technical staff a chance to step outside their usual areas of focus and to have free rein to explore their creative ideas that push our technology in all sorts of cool directions. These demonstrations, and the exercise itself, are fun and really inspiring to us all and an important part of our culture of innovation.
During the hackathon, our engineering and technical operations teams spend a day and a half playing with code and turning ideas into working proof of concepts. Though exploration is the main goal, many of the ideas end up being refined and incorporated into our products or to our internal toolsets.
As usual, the Fall hackathon teams impressed us with their orthogonal thinking and technical chops. The group produced a crop of innovative ideas that show the depth and breadth of our capabilities. Let’s take a look at some of the great hacks that came out of the event this time around.
Each hackathon event has a theme with several areas of focus. Our Fall event featured the following categories:
- Product: a new or improved product or feature, including end-user tools and add-ons
- Tools: an internal tool or infrastructure that can improve how we design, build, test, ship, or support our products
- Partner Integrations: a value added integration of our products and services with a third party
- Fun with Data: making more of the data we have in our cloud environments
- We Deliver: innovations in abuse detection, compliance, and deliverability
The rules are simple: use whatever tech you want, research as much or as little as you want, and don’t write any code ahead of time. The hackathon starts at 10 AM on day one and ends with presentations starting at 1 PM the following day. A panel of judges reviews the entries and picks a winning team for each category.
Best in Show
Our judges award best in show to the project that stands out to them the most. In this hackathon, it was a proof of concept for a new iteration of our Adaptive Delivery functionality, which auto-tunes outbound email delivery parameters and traffic shaping in real-time to avoid blocks, safeguard reputation and optimize delivery.
This functionality is currently a part of the core technology that powers SparkPost—the Momentum platform. The project used a combination of Vertica, Redis, and Node.js to power the rules system for Adaptive Delivery, allowing us to offload some of the work to more efficient services. The main benefits here are scalability, performance, and abstraction. Another potential benefit could be sharing Adaptive Delivery data to improve deliverability for all of our customers.
For our product category, two projects shared top honors: an in-application API explorer for SparkPost, and a hot backup for our PowerMTA product. The interactive discoverability made possible by in-application API explorer was so awesome that we’ve added it to the SparkPost UI! This interactive tool complements our traditional API documentation.
The PowerMTA project did a proof-of-concept system that enables one instance of PowerMTA to be the hot backup for another instance. Also addressed was a potential solution for recovery from failures when using hot backups. The team relied on their knowledge of database replication and SMTP to build the replication protocol and used TCP socket programming to implement the details.
The tools category produced a project allowing us to automate testing against our SparkPost Elite environments. Our tech ops team routinely spins up new SparkPost Elite customer environments with the requisite architecture and needs to test that everything is set up correctly and ready to send. To aid in automation of this process, one of the teams created autoscott, a command line utility that allows teams to run commands to perform actions like sending a single transmission, sending multiple transmissions, creating recipient lists, creating templates, and more.
Fun With Data
In the fun with data category, we had a team create a prototype for a weekly report card. This report would be delivered to our customers and contain a summary of their sending including number of emails sent, open rate, and click rate. It would also deliver helpful tips on how to improve your mailings based on trends in data. For example, if your open rate went down week-over-week, we’d let you know how to potentially correct this issue. Additionally the email contains a breakdown of volume by day for the past few weeks and some information about your sending habits as they relate to other senders on SparkPost.
Signing up for and signing in to SparkPost is simple. The winning entry in our partner integrations category wanted to take this a step further, so they built the ability to sign up and sign in via Github, Twitter, or Google. The project made use of the Node.js library passport and the adapters for Github, Twitter, and Google. As an added bonus, there is also a passport adapter for SAML, which we’re using to power our upcoming SSO functionality for our SparkPost Elite customers.
Deliverability is a big deal at SparkPost. We pride ourselves on keeping the bad actors out and the good actors sending reliably. In our we deliver category, the winning team worked on refining our ability to scan outgoing email for spam. One particular enhancement was a performance optimization to make scanning of large transmissions more efficient. This group worked using the fine CSDMC2010 spam corpus, which includes a training collection of labeled spam and ham emails, as well as an additional unlabeled test set. The group also worked to visualize the resulting data using SumoLogic, a commercial product which we use in our data analysis.
Another Successful Hackathon
Hosting employee hackathons has become a great tradition here at SparkPost. Our hackathons support our culture of innovation…and ultimately help our customers succeed. As you can see, our teams are constantly striving to improve our product and our tools. The Fall hackathon was no exception.
We’re always looking for bright people to join our team. If solving complex problems, using awesome tech, and participating in Hackathons sounds fun to you, check out our open positions.