Developer Innovation at the SparkPost Hackathon

Rich Leland
Dec. 18, 2015 by Rich Leland

Twice a year, SparkPost hosts an employee hackathon. The idea is to give our hard-working technical staff a chance to step outside their usual areas of focus and to have free rein to explore their creative ideas that push our technology in all sorts of cool directions. These demonstrations, and the exercise itself, are fun and really inspiring to us all and an important part of our culture of innovation.

During the hackathon, our engineering and technical operations teams spend a day and a half playing with code and turning ideas into working proof of concepts. Though exploration is the main goal, many of the ideas end up being refined and incorporated into our products or to our internal toolsets.

As usual, the Fall hackathon teams impressed us with their orthogonal thinking and technical chops. The group produced a crop of innovative ideas that show the depth and breadth of our capabilities. Let’s take a look at some of the great hacks that came out of the event this time around.

Guidelines

Each hackathon event has a theme with several areas of focus. Our Fall event featured the following categories:

  • Product: a new or improved product or feature, including end-user tools and add-ons
  • Tools: an internal tool or infrastructure that can improve how we design, build, test, ship, or support our products
  • Partner Integrations: a value added integration of our products and services with a third party
  • Fun with Data: making more of the data we have in our cloud environments
  • We Deliver: innovations in abuse detection, compliance, and deliverability

The rules are simple: use whatever tech you want, research as much or as little as you want, and don’t write any code ahead of time. The hackathon starts at 10 AM on day one and ends with presentations starting at 1 PM the following day. A panel of judges reviews the entries and picks a winning team for each category.

Results

Best in Show

Our judges award best in show to the project that stands out to them the most. In this hackathon, it was a proof of concept for a new iteration of our Adaptive Delivery functionality, which auto-tunes outbound email delivery parameters and traffic shaping in real-time to avoid blocks, safeguard reputation and optimize delivery.

This functionality is currently a part of the core technology that powers SparkPost—the Momentum platform. The project used a combination of Vertica, Redis, and Node.js to power the rules system for Adaptive Delivery, allowing us to offload some of the work to more efficient services. The main benefits here are scalability, performance, and abstraction. Another potential benefit could be sharing Adaptive Delivery data to improve deliverability for all of our customers.

winners-best-in-show

Product

For our product category, two projects shared top honors: an in-application API explorer for SparkPost, and a hot backup for our PowerMTA product. The interactive discoverability made possible by in-application API explorer was so awesome that we’ve added it to the SparkPost UI! This interactive tool complements our traditional API documentation.

explorer-2

 

The PowerMTA project did a proof-of-concept system that enables one instance of PowerMTA to be the hot backup for another instance. Also addressed was a potential solution for recovery from failures when using hot backups. The team relied on their knowledge of database replication and SMTP to build the replication protocol and used TCP socket programming to implement the details.

Tools

The tools category produced a project allowing us to automate testing against our SparkPost Elite environments. Our tech ops team routinely spins up new SparkPost Elite customer environments with the requisite architecture and needs to test that everything is set up correctly and ready to send. To aid in automation of this process, one of the teams created autoscott, a command line utility that allows teams to run commands to perform actions like sending a single transmission, sending multiple transmissions, creating recipient lists, creating templates, and more.

Fun With Data

In the fun with data category, we had a team create a prototype for a weekly report card. This report would be delivered to our customers and contain a summary of their sending including number of emails sent, open rate, and click rate. It would also deliver helpful tips on how to improve your mailings based on trends in data. For example, if your open rate went down week-over-week, we’d let you know how to potentially correct this issue. Additionally the email contains a breakdown of volume by day for the past few weeks and some information about your sending habits as they relate to other senders on SparkPost.

Partner Integrations

Signing up for and signing in to SparkPost is simple. The winning entry in our partner integrations category wanted to take this a step further, so they built the ability to sign up and sign in via Github, Twitter, or Google. The project made use of the Node.js library passport and the adapters for GithubTwitter, and Google. As an added bonus, there is also a passport adapter for SAML, which we’re using to power our upcoming SSO functionality for our SparkPost Elite customers.

We Deliver

Deliverability is a big deal at SparkPost. We pride ourselves on keeping the bad actors out and the good actors sending reliably. In our we deliver category, the winning team worked on refining our ability to scan outgoing email for spam. One particular enhancement was a performance optimization to make scanning of large transmissions more efficient. This group worked using the fine CSDMC2010 spam corpus, which includes a training collection of labeled spam and ham emails, as well as an additional unlabeled test set. The group also worked to visualize the resulting data using SumoLogic, a commercial product which we use in our data analysis.

winners-we-deliver

Another Successful Hackathon

Hosting employee hackathons has become a great tradition here at SparkPost. Our hackathons support our culture of innovation…and ultimately help our customers succeed. As you can see, our teams are constantly striving to improve our product and our tools. The Fall hackathon was no exception.

We’re always looking for bright people to join our team. If solving complex problems, using awesome tech, and participating in Hackathons sounds fun to you, check out our open positions.

Share your Thoughts

Your email address will not be published.

Related Content

Community Spotlight: How Topol Makes Creating Beautiful HTML Email Templates Easy

Building HTML email templates is hard. Learn about the app that Jan Tlapak and Sendmark created for anybody in need of building beautiful HTML emails.

read more

Running Your First Technical Workshop: The "I Have No Idea What I'm Doing" Edition

Running a technical workshop is difficult! Where do you start? How much time do you allot? What audience do you aim for? Cole talks through what he learned.

read more

Introducing Our SendGrid Template Migration Tool

Automatically migrate any SendGrid template over to SparkPost. This open source tool supports both UI- and API-driven email template migration workflows.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and send up to 100,000 emails per month for free.

Send 100K Emails/Month For Free

Send this to a friend