sparkpost billboard you code we send aws reinvent 2016

AWS Reinvent 2016

AWS ReInvent is an annual conference organised by Amazon Web Services, Amazon’s cloud computing division. This year it was in Las Vegas, Nevada, USA, with 32,000 people attending from all over the world.

I attended, along with a number of my colleagues. The SparkPost service is hosted in the AWS cloud. We use a number of AWS services to build and operate the service. We are a couple of years into our cloud journey, and we have plenty left to learn.reinvent recap

What I Wanted to Get Out of the Conference

I wanted to know more of the nitty-gritty of the services we use – gotchas, recommendations, anti-patterns to avoid, etc. – by attending some of the 400+ breakout sessions. I also wanted to share problems and solutions with other folks.

Finally, I wanted to spend some quality time catching up with my colleagues. We have quite a geographically distributed team, spread across eight hours of time zones. So this was actually my first opportunity to meet some of them face-to-face!

 

Serverless Computing

I attended a number of sessions around serverless computing, where you don’t have dedicated infrastructure running all the time. On the surface, it seems like this could provide a cheaper way of providing some services, by automatically scaling the infrastructure up and down to match the load on the service.

AWS Lambda is serverless — it runs a single function in response to an input. Behind the scenes, the Lambda runs in a container, which can be reused across invocations. Resources created in the global scope of your Lambda code can be reused across invocations. For example, DynamoDB connection, to reduce latency.

AWS is increasingly using Lambda as an extensibility mechanism — like the cloud equivalent of “hooks” or “callbacks” in software, or webhooks. In the DynamoDB Deep Dive, Lambda stores procedures for DynamoDB, which is an interesting way to view it. Lambdas can be triggered when you make changes to a DynamoDB database, via Streams. This could be used to insert an alternative representation into another DynamoDB table, or push data into a Redis cache.

Amazon’s API Gateway allows you to build serverless REST APIs, with the gateway handling the HTTP traffic and a Lambda doing the work. API Gateway can also call AWS services like DynamoDB directly.

Databases and Analytics

Amazon Athena is a new service that allows you to query and analyze data in S3. You point it at your data in S3, define a schema, and then query using SQL. This is like bringing the database to the data, rather than the data to the database. We are considering using Athena as a replacement for the Message Events API backend.
Tracing Distributed Applications
AWS announced X-Ray a service that provides insight into where your distributed system spends its time, by tracing API requests. It can give you various visualizations of traffic, from a top-level map of your services down to a timeline view showing how much time an API call had to wait on databases and other services. This seems invaluable for gaining insight into applications.

From Monoliths to Microservices

A number of speakers had changed from a monolithic architecture to microservice architectures, or architectures with smaller more focussed components, in order to increase scalability and/or agility. This was to maintain competitiveness. Cost reduction, higher developer velocity and reliability increases seemed to follow naturally.

Closing Thoughts

The conference was very well organized conference, and everything went smoothly. Despite the number of people, it did not feel crowded. It was difficult to get into some of the sessions, but I was fortunate to get into nearly all of my chosen sessions. There were repeats for some of the more popular sessions, and it was possible to get into some of the full sessions by queueing on the day. The app (new this year) was also very helpful for navigating between sessions

Las Vegas was a fun setting with its numerous restaurants, bars and the impressive architecture of the Venetian Hotel.

It was great to spend time with my colleagues away from our “day jobs”, sharing what we’d learnt and thinking how we could use it to improve our service for our customers.

We were also fortunate to have some face-to-face meetings with Product and technical managers for some key AWS services that we use, where we discussed some of our experiences and challenges. They were open and keen to discuss to our feedback, and share some of the future direction of their services with us.

Lastly, I have barely scratched the surface of the conference. There were significant releases in Artificial Intelligence, Compute, Containers, Developer Tools, Hybrid Cloud and others. Please also see:

AWS re:invent 2016 | New Products & Services
Amazon Web Services YouTube channel