Tech Tips: Riak and Postgres and Lucene, oh my…

Tom Mairs
Sep. 1, 2012 by Tom Mairs

Four years ago, Message Systems had a single product offering in Momentum version 2. While this was a groundbreaking MTA, it was really built for the singular function of processing email very, very quickly. Momentum 2 stored a number of logs but had no local database and did not really need one.

When Momentum version 3 was introduced, it came with a new dashboard that used an embedded PostgreSQL database. Shortly after this release, the product offering expanded to include content creation and campaign tools that also made use of the PostgreSQL database. Intelligent adaptive message shaping was also added utilizing a Riak database. Most recently, introspection and tracing tools were added that make use of a Lucene database.

So why use all these databases? Each of these types of data stores are particularly good at certain functions so we use different databases to make the overall platform solution as efficient as possible. The PostgreSQL database is a powerful SQL relational database with the ability to store complex tables as well as procedures for advanced processing. This is used for the bulk of the data workload in the Message Systems platform.

Riak is a non-SQL database that is designed to work in a distributed environment. It is essentially a key-value pair data store, similar in concept to Hadoop, that is extremely fast and durable. It replicates natively across a cluster to provide fast access to data to any node in a cluster. We use this primarily for the storage and retrieval of message shaping settings for the Adaptive Delivery® component.

Lucene is an Apache project based in Java that delivers extremely high performance text search capabilities. In our case, we record message transaction logs to a Lucene data store so that we can search them at a later date. When necessary, a customer service representative can execute a search against this Lucene data store for any message using a wide variety of factors.

So you can see that each of these data store technologies are widely different and serve very different purposes. Message Systems includes each because they are the best at what they do and they work together to provide the most efficient platform overall.

More Information:

Message Systems Documentation (client login required):

Learn more about why the world’s best brands are using Momentum in the Total Economic White Paper by Forrrester!

Blog_Ads_TEI-Momentum_100413

Share your Thoughts

Your email address will not be published.

Related Content

Operating DNS on the AWS Network: Challenges and Lessons

Learn how our team worked with AWS to address a challenging DNS performance issue—and tips for troubleshooting with the AWS support team.

read more

Premium Service Offerings: What Our TAMs Can Do For You

Sending email isn't always smooth sailing. Our TAMs provide tailored premium support to customers navigating the tricky world of email deliverability.

read more

DKIM Validation: An Email Authentication Best Practice

An overview of DKIM validation including how to sign and validate work, interpreting DKIM signatures, what DKIM public keys look like and more.

read more

Start sending email in minutes!

The world’s most powerful email delivery solution is now yours in a developer-friendly, quick to set up cloud service. Open a SparkPost account today and send up to 100,000 emails per month for free.

Send 100K Emails/Month For Free

Send this to a friend