Like many things, it all started with an email. On May 30, AWS announced the general availability of its new graph database, Neptune, a fast, reliable, and fully-managed graph database service. Several members of the Engagement Optimization service team here at SparkPost took notice and immediately connected the dots – just as a graph database would connect nodes ;). We thought if only we could use those graph nodes to connect emails with recipients, we could really add some value to our customers’ experience. And now that we had AWS on board, we could do it fast and reliably for everyone without being experts in maintaining a graph database.
Our Hackathon Idea was Born
The idea was born and as the hackathon approached our idea was picking up interest and our team began to grow. On the day of the hackathon we had reached the maximum team size of 4. Team Neptune was comprised of Ian Scherer, Jim Braman, Greg Walls, and Tim Ecklund. Our team brought very unique skills spanning across almost all of the SparkPost services. Tim had formerly worked on the SparkPost Platform Team, which proved to be invaluable to integrating our project into the transmissions API. Greg came from our Analytics team, and helped us figure out how our project might leverage engagement events. Ian and Jim came from the Engagement Optimization team and were both very interested in how Neptune could find hidden relationships for our customers.
It was clear that we had both a team and an idea with a lot of potential. The challenge now was to build. But build what exactly? How would this thing work? We all knew generally what we were trying to accomplish, but also realized that beyond a few sentences, we had no idea how this thing would work. None of us had really used graph databases much. We soon realized that we had no idea how to even get data in or out of Neptune. But before we started just hacking and following AWS tutorials, we began a different kind of hacking, idea hacking. We all knew how much rework is involved in building projects with half-baked requirements and knew we had no extra time to spare if we were to get an end-to-end demo finished in one day. In spite of the time constraints, we resisted the urge to start banging on our keyboards and started whiteboarding. Before we knew it, we had spent half a day hacking our idea across an entire whiteboard wall of a conference room.
It was time for lunch and we didn’t have a single line of code. There was a lot of back and forth, erasing and rewriting, but by the time we headed downstairs to eat our delicious catered lunch, we knew that despite having no code to show for it, we had done it. We knew that all we had to do was write a very small amount of code, and we knew how to divide the work amongst ourselves and how every piece of the final puzzle fit together.
At lunch, teams discussed how their projects were progressing, and while we didn’t have any code written yet, we were fairly confident we’d be able to finish and still head home a bit early. We could explain in detail how we thought customers should be able to tag their templates with demographics, keywords, or really anything they wanted. And how if they did so, we could anonymize and tag recipients that engaged with those templates. Not only could we anonymously correlate templates with recipients using Neptune, but we could know what types of engagement that recipient had with that type of template. Neptune is a database built to store and find relationships between nodes. Our nodes are recipients, engagement events, and tags. The first relationship is between recipients and engagement types. The second is between engagement types and tags. These relationships could then unlock the hidden relationships between templates and recipients. Additionally, due to our ability to hash our customer’s data, we could make this GDPR compliant as well. We could build a deep understanding of recipients across customers without knowing their email addresses. All we needed to know was which template the anonymous recipient node engage with, based on tags!
In no time at all we had translated our idea into code and joined everyone for dinner, beer and video games. Over grilled cheese, we finished our lunch conversations with “All done, wrapping up the slides, and can’t wait for tomorrow!”. We had not only written a fully functional API, but had also updated the transmissions API to support it. Additionally, we provided the Analytics Team with future APIs they would need to update Neptune with engagement event types and tags!
This was our chance to sell our idea to our panel of judges and 17 other teams. We gave a 10 minute presentation, including a small slideshow and a fully functional, end-to-end demo showing a transmission sending tagged templates to recipients that had previously engaged with templates of a similar tag. We thought it was beautiful and so did the judges. Our CEO, Head of Product, and Dev Relations manager agreed that our hacked idea was in fact worthwhile. We ended up winning Hackathon X, hats and most importantly having our names forever engraved in the SparkPost Hackathon Best in Show trophy.
-Jim, Greg, Tim & Ian