A little late to the game, I recently binge-watched season 7 of the Game of Thrones. One thing I noticed is that for all the thick walls, barred doors, and armed guards, most of the castles in the show were not taken through a direct attack on their outer defenses, but rather through infiltration. Invaders circumvented the castle’s defenses using backdoors and hidden tunnels. They even walked through the castle gate while impersonating a resident.

SaaS applications that send email—which is all SaaS applications—are similarly vulnerable. As an engineering leader, you may have followed security best practices: training, code reviews, third-party penetration testing, secured your infrastructure, and more. And yet all your efforts could come to naught if a bad actor steals a user’s credentials from that user.

A common approach bad guys take to stealing user credentials is to impersonate your email: a phishing attack. A malicious third party sends your users an email that looks like it came from your application. A link in the email takes users to a website resembling your own and requests their username and password to log in. The attackers now have access to your application, allowing them to walk right through your castle gate.

DocuSign phishing email courtesy of KnowBe4

An attack like this—impersonating your application—could result in customers losing trust in your service and present an existential threat to your business.

You may believe that you’re not liable for a customer’s internal security training and that there’s not much you can do to stop attacks like this. And you’re right in thinking you can’t easily stop these emails being sent. You can, however, strongly influence whether your users receive these emails.

Phishing is possible because, in the early days of email, the small group of organizations using it trusted one another. The Simple Mail Transfer Protocol (SMTP) lives up to the “simple” part of its name. A mail client can send a message by connecting to a remote Mail Transfer Agent (MTA), provide the To and From addresses of the message, and then the message body itself. The SMTP protocol does not specify a way to validate that the party sending the email has the right to send from any given address, allowing the sender to claim they are sending a message on behalf of anyone.

Fortunately, new standards are making it easier for companies to protect themselves and their users from phishing. These standards add the ability to authenticate the identity of a message sender. They allow mailbox providers such as Gmail and Microsoft, along with corporate email administrators and others, to filter messages that don’t pass validation and alert the company targeted by phishing to the impersonation attempt.

Apple phishing email courtesy of mailguard.

The first of these technologies is Sender Policy Framework, or SPF. SPF defines a way to validate that an email message was sent from an authorized mail server. SPF establishes a method for receiving mail servers to verify that incoming email was sent from a host IP authorized by the sender’s domain administrators. It piggybacks on the existing Domain Name System (DNS).

While SPF can define which servers are allowed to send messages on behalf of your domain, it doesn’t offer a mechanism to verify whether the message headers or body have been altered or forged. This is handled by DomainKeys Authenticated Mail, or DKIM. It works by using a private key to creating a unique digital signature for the email’s header and content and adding this to the existing headers of the message. The signature can then be validated against a public key in your DNS records.

Domain DNS verification section
SparkPost domain setup displaying DKIM configuration.

The third standard that serves to bring the first two together is Domain-based Message Authentication, Reporting, and Conformance, or DMARC. DMARC is another DNS-based technology that enables an organization to publish their authentication practices, advise receivers on how to treat messages that don’t validate against SPF and DKIM and to request that notifications be sent to the organization when non-authenticated messages are encountered.

With these combined standards, mailbox providers can identify messages that have been legitimately sent from your application and ensure that they are delivered to your user’s mailboxes. Those that aren’t may be sent to a spam folder or quarantined, with your team notified. The email purportedly sent from your application asking the user to change their password would likely not have been delivered to their inbox.

Your customer’s data may not be pilfered via a frontal assault on your application. Rather, using deception, a bad actor can fool your users into giving away the keys to your castle. There are, however, actions you can take to mitigate this risk and ensure that your customers remain safe.

—Steve Murray, SparkPost CISO

P.S. Want to learn more about preventing phishing and ensuring that your messages reach your user’s inbox? Our Email Deliverability Guide is a great resource. And if you’re implementing email authentication standards, check out our free tools for inspecting and validating your SPF and DKIM records.

 

When we speak of “Email Authentication”, we’re referring to a technique that provides to the recipient of a message some level of certainty that the message actually originated with the claimed source of the message. The idea behind such techniques is to achieve some level of defense against fraudulent email, such as phishing and spoofing, mail that could erode a recipient’s trust in receiving email. That said, the act of sending authenticated email does not assert that the email is good or wanted email; it only means that the mail is such that a reputation for the authenticated party can be reliably established and used in email acceptance and placement decisions.

There are two forms of email authentication in use today:

  • Sender Policy Framework (SPF)
  • Domain Keys Identifed Mail (DKIM)

In today’s post, I’m covering what DKIM is and how it works.

DKIM Overview

Unlike its authentication counterpart SPF, which provides a way for a domain to authorize a host to send mail on its behalf, DKIM provides a way for an entity (domain, organization, person, etc.) to take responsibility for a message, independent of the entity that actually sent the message. While in many cases the responsible entity and the sending entity will be the same, or at least closely related, with DKIM there’s no requirement that this be so.

My goals for you with this post are that you learn and understand the following concepts about DKIM:

  • DKIM is a “content-based” authentication, unlike the “path-based” SPF.
  • The responsible entity asserts its responsibility by “signing” the message with a pair of cryptographic hashes inserted in a message header.
  • DKIM validation is done by the receiving domain attempting to generate the same two hashes.
  • DKIM validation cannot be completed in many cases until the full message has been transmitted by the sending server.
  • Validation failures can be difficult to troubleshoot.

“Content-Based” Authentication

DKIM is referred to as “content-based” authentication, rather than “path-based”, because whether or not a message passes DKIM validation is based solely on whether or not the content has changed between the time it was signed and the time validation was attempted.

DKIM Signing and Validation

Organizations wishing to DKIM sign mail will first generate two cryptographic keys. One of the keys is kept private and available to the sending server for the signing of mail, and the other is to be made public in DNS for use by receiving domains in attempts to validate the signature. The methods for generating these keys and installing them are platform-dependent and beyond the scope of this post, although later I will describe the publishing in DNS of the public DKIM key.

The DKIM-Signature Header

To begin our understanding of DKIM, let’s first look at a DKIM-Signature header:

The DKIM-Signature header is a series of key-value pairs, some of more interest to the reader than others, but I’ll describe all of them here.

First, we’ll look at the ones that are mostly of passing interested to the reader:

  • v=1; – Specifies the DKIM version (1 is the only valid value)
  • a=rsa-sha256; – The algorithm used to construct the cryptographic hashes
  • c=relaxed/relaxed; – There are two sets of rules regarding removal of whitespace in the headers and body that can be applied when creating the hashes in a DKIM signature; these rules are called “canonicalization rules” (hence the key of c) and the rulesets are either “relaxed” or “strict”.
  • t=1454417737; – The timestamp of when the signature was created.

These three header parts contain the actual signature information:

  • bh=e+6RkdhJe69wcQKtRKw9rpDgkkPPbZ8Xwj/2Hi243Sc=; – This is the hash of the body of the message.
  • h=From:Reply-To:Subject:Date:Message-ID:To:MIME-Version:Content-Type; – This is a list of the headers that were used to create the signature data shown below.
  • b=KhK4OjejS4QEBr1RwL/naZKBNLoFnR/3lmDOWZC3av4c2aH5Yg/D4vqhh1CpcyfPvRm7cp5EvrnPEsOA7r3E15jarzNFNHXtwjxCFn4g8StsXFOio9vHkO7bmp6t2aLu8bPkX6cNHgULYS6TdqYd65y5xCDMEaQ9a3mnhF2TQss=; – This is the acutal DKIM signature data

These three parts are of most interest to the receiving server that will be validating the signature:

  • d=welcome.foo.com; – This identifies the domain that signed the message
  • s=notices; – The selector; domains can have multiple selectors that they use when signing messages.
  • i=@welcome.foo.com; – This is the identity on whose behalf the message was signed. Syntactically, this will look like an email address, and might even be one; the localpart of the email address can be empty, as in this example, and the domain part must be the same as, or a subdomain of, the domain in the d= part of the signature.

The reason these parts are of interest to the receiving server is that they provide the information necessary to help the receiver validate the signatures.

DKIM Validation

In addition to the requirement mentioned that the i= domain must be the same as or a sub-domain of the d= domain, the d= and s= bits are used by the validator to look up the signer’s public DKIM key in DNS. The key is a TXT record in DNS, and it’s always found at the location selector._domainkey.domain. So, in our example here, with s=notices and d=welcome.foo.com, the public DKIM key would be found in DNS at notices._domainkey.welcome.foo.com, and it might look something like this:

The validator uses that key (the p= bits) to produce its own set of hashes of the message; if those hashes match, then the message hasn’t been altered in transit, and so the message can contribute to, and perhaps benefit from, whatever reputation is in place for the signer of the message.

Validation Failure and Troubleshooting

I mentioned above that DKIM failures can be difficult to troubleshoot, and I’ll explain why that’s so here.

Some DKIM validation failures have obvious causes, such as the message not being signed, or the signing domain’s public key not being found in DNS or not being syntactically correct, or perhaps the message was obviously altered in transit. When those kinds of failures happen, it’s easy to figure out the problem and recommend a solution. The tough ones, though, and the ones which lead to the most frustrating support experience, are the cases where the message was signed, the public key exists in DNS, and the message wasn’t obviously altered, but the validator reports that the signature failed to validate.

The reason that these are tough to troubleshoot is because there’s no real way for either side to reproduce the conditions under which the message was signed and validated. The message has in its DKIM-Signature header the hashes that were generated by the signer at the time of signing, but the validator likely has no access to the signer’s infrastructure and so cannot try to reproduce the signature under the signer’s conditions. Similarly, the signer likely has no access to the validator’s infrastructure and so has no way to try to validate the message in the way that the validator did.

Failures like I’m describing here are rare occurrences, and DKIM validation failures by themselves usually don’t have an impact on delivery placement. It’s been my experience that such failures drive more support tickets than any other kind of DKIM issue.

-Todd

For further reading on the topic of email authentication, please see our blog for posts on SPF and DMARC

Big Rewards Blog Footer

Here at SparkPost we have a “single-page JavaScript app” that consists of about 50,000 lines of early 2015-era Angular 1.x code spread across more than 400 files. It’s not a small app. And as you’d expect with almost any language or framework over a period of 2 years, we’ve gotten pretty familiar with Angular’s good, bad and ugly sides. Angular 2, having been released as “Final” in September of last year, would seem like a pretty natural fit for us. But the title of this post has already given it away: we’re most likely not upgrading to Angular 2.

Why not Angular 2? Mostly because of a migration path that makes a strong case for considering almost anything else and maybe somewhat because of TypeScript, but if I’m being honest it’s mostly because it’s nice to try new things. We’re not an agency with a new greenfield project kicking off every few weeks or months where we can test out the latest pre-alpha releases of our favorite cleverly-named JavaScript libraries. Fifty thousand lines of code changes slowly. But that’s when the “tools app” showed up.

A rare greenfield project

Our team was asked to build a set of email tools that wouldn’t live inside of our existing app. These “hardcore email tools” help developers with deep cut email setup—the kind of stuff we already take care of for SparkPost customers —so we wanted them to have their own space out from behind our login. Suddenly, we had a place to explore something new [cue harp music].

We came up with some important criteria for what we’d use to build this new app:

  • It needed to be easy to learn
  • It needed to be fast to build
  • It needed to be something we could build in the open
  • It needed to not be Angular
  • It needed to probably just be React

After considering these criteria carefully and thoughtfully as a team, we came to a surprising decision to give React a try. At the time, I was the leading React expert on our team by way of having completed one Udemy course on the subject, so I began to throw something together.

Some things we accidentally did right

We already had a small part of the app designed and built. It’s hard to underestimate the value of a designed, styled, and approved working prototype of even just a small part of your app. The time that could have been spent arguing over button placement and wording was replaced with figuring out how to get a React app off the ground.

Speaking of which, we used Create React App. Think “html5boilerplate for React apps,” or maybe “Ember for React apps.” CRA gives you a working React starting point complete with all the dependencies (literally, it might download all of the dependencies) and with a working baseline Webpack configuration. Again, this let us focus on what we were actually building. And when you’re ready, CRA lets you “eject” and take control of the whole setup. It’s fantastic and you should use it.

You should also find a designer who can do both. And by both I mean design and understand React. I know this is a very unfair thing to suggest because it really seems to be incredibly hard to find, but we found one of these magical unicorns and they’ve been invaluable. (I even looked up “invaluable” just now to confirm that it does mean really freaking valuable.) If you can, make it a priority to hire this kind of person for your team. (And thanks for being awesome, Jon.)

We also made a decision early on to build the app using only setState / local state, i.e. no Flux, no Redux, etc. We eventually added Redux—another topic for another time—but starting with a simple React app made it much easier to onboard new developers who were getting up to speed with a lot of things at once. Not to mention, waiting on Flux also lets you decide if you really need it at all.

A few other things I’d recommend based on our first-timer experience:

  • Use Jest for your testing. Jest comes with Create React App and despite being 100% Mocha/Chai across all of our other projects, it was too hard for us to deny how great Jest is. Notably, the amazing Jest CLI and Snapshot testing have both been especially useful additions for us.
  • Use the dev tools. There are ones for React (Chrome, Firefox) and ones specifically for Redux if you use it. They’re free, they work great, and they’re incredibly useful.
  • Find a group of people you trust, ask them for advice, and do what they say. I’m fortunate to have friends in our local meetup group (CharmCityJs) and in the NYC JavaScript community (BoroJS), both with active Slack teams. Being able to ask “what do people use for x?” has been a huge help because really, you just need to pick something. Trusting someone else is as good a reason as any.

Fifty thousand lines of code changes slowly

So what about that 50,000-line Angular app? We won’t be migrating it to React, at least not directly, and it can’t really survive as an Angular 1.x app forever, either. But here’s an interesting thing I noticed as I was getting familiar with React: in some ways, it’s not that much different than Angular. Here’s an Angular 1.5+ component:

If you pretend the template string is some JSX and it gets returned from that controller’s render method, you basically have a React component (at least structurally). So instead of trying to haul 400 files’ worth of old-school, big-controller Angular code into a new framework, our plan is to focus on the patterns. Specifically, the patterns of “small, focused components” and “unidirectional data-flow”. I’ll talk more about that second part in a later post about our adventures with Redux, but refactoring our giant controllers into small Angular components has two big advantages:

  1. React isn’t forever. Any large-app rewrite/refactor is going to take a while, and if you haven’t noticed, the JavaScript ecosystem moves pretty quickly. By focusing on refactoring our existing app to use better patterns, we get it prepared to migrate to whatever happens to be the best solution at the time, when we’re finally in better shape to make that move.
  2. Iterative, incremental development is dangerous. One of my favorite images of how “agile development” should work is a drawing by Henrik Kniberg from a Spotify presentation, explaining how to be iterative in a productive way. You’ve probably seen it before:
react agile illustration
Source: http://blog.crisp.se/wp-content/uploads/2016/01/Making-sense-of-MVP-.jpg

If we spend 6 to 9 months or more trying to rewrite the app in React and don’t succeed, run out of time, or have the work shelved for other priorities, we end up with nothing useful at all. But with the refactor-first plan, the worst thing we end up with is a better, more maintainable Angular app. In other words, it’s an easy decision.

Angular, React, Kumbaya

No lie, we had a lot of fun building our new tools app in React/Redux. It’s a great library with a fantastic ecosystem and a lot of good patterns. But to be honest, our Angular app already works, and that’s fine. If you’re maintaining a big legacy app, remember:

  • Find small greenfield projects where you can build something with new tools.
  • Focus on patterns, and figure out how you can incorporate those patterns into your legacy app without having to rewrite the whole thing.

As I mentioned before, we built this in the open, so feel free to check out the code as well as the live app itself. If you’re coming from an Angular app, I’ve written up a bunch of notes about learning React that may be helpful for you, too. If this post was interesting to you for any reason at all, check back often as we continue to write more about our adventures with Angular, React, and front-end development. If you have any questions or if there’s anything else specific you’d like to hear about, let us know!

–Jason Rhodes

At SparkPost, we love email. The way you feel about those pictures of your cat you posted on Instagram? That’s how we feel about email. But we know not everyone shares our excitement about email RFCs. Which is why we work to share our knowledge of email standards and make sure that setting up and sending email is as easy as possible for as many people as possible. Except for you, Mr. Western Union scammer.

Email has grown a lot over the past 40+ years and along the way several standards have cropped up, meant to combat spam, phishing, and spoofing. Keeping up with these standards and implementing them correctly can be tough. On the flip side, setting them up the wrong way can really hurt your business. Set up incorrectly, your email won’t get delivered or, more dangerously for your customers, will let any malicious sender impersonate you online, even to the point of letting someone change your content while it’s in flight.

That’s why today we’re happy to announce the release of three new tools for building and verifying the state of your email. I’ll walk you through each of them quickly and how they can help you win the trust of ISPs and your users.

DKIM Validator

DKIM is a content authentication standard meant to let ISPs (Internet Service Providers, the ones who manage the inboxes you send to) verify that the email you sent is the one they received and that no content has changed since it left your system. DKIM requires you to set up a special DNS record with a public key and other metadata that receivers use to verify the content of the message they receive.

Our DKIM Validator lets you safely verify that your DKIM is set up correctly. It will give you a unique email address; send to that address from your domain (through the system you’re validating) and we will check your DNS entry and make sure the email we received complies with the DKIM policy you set up. We’ll even give you recommendations if we find something that could help you out.

Find something surprising in the results and want to share them? We got ya covered: all results are deep-linkable for easy sharing (click the “Share” button, and we’ll even put the URL in your clipboard).

Not sure about any of this DKIM stuff? Sign up for a SparkPost account and we’ll take care the hard parts for you.

SPF Builder

SPF is a standard used by ISPs to verify that an email addressed “from” your domain is coming from a server that you’ve specifically said is allowed to send for your domain. Setting up a correct SPF DNS record is one of the best ways to combat spoofing attacks and keep your email reputation high.

The thing is, SPF is deceptively complicated. There are only a handful of simple mechanisms in the SPF standard, but when they’re combined into your domain’s record, it soon becomes a complicated tree of hosts, any of which can send for your organization. It’s quite easy to be too broad and leave yourself open to spoofing attacks, or be too narrow and have your email appear to be illegitimate.

That’s why we made the SPF Builder – it will walk you through a series of simple questions to build out your SPF record. You don’t need to know any special syntax, we’ll translate everything for you and even copy it to your clipboard for easy pasting into your favorite DNS provider!

SPF Inspector

Whether your SPF record came from our builder or is artisanally hand-crafted, it’s good to check the validity of your record once it’s live. Our new SPF Inspector will not only verify your record’s syntax (and the syntax of any others you’ve included), it will also show you all IPs and hosts that are allowed to send on your behalf. If we find anything wrong, or slightly not great, we’ll give you a heads up along with some pointers on how to fix what’s not quite right.

All your results are shareable, just like the DKIM Validator. In addition, if you are signed in to your SparkPost account, we’ll keep a history of the domains you inspect over time. That becomes really useful as you start to manage and send from several domains.

Find SPF too confusing? Send through SparkPost and we’ll handle all the messy SPF details for you!

Go Check ‘em Out

We’re really proud of these SparkPost email tools and we think they will make your life easier and your email better. Do you have ideas for other tools that would help you send better email? Let us know on Twitter, Slack, or in the comments!

— Cole Furfaro-Strode

Here’s the full press release on the SparkPost email tools

When we speak of “Email Authentication”, we’re referring to a technique that is meant to provide to the recipient of a message some level of certainty that the message actually originated with the claimed source of the message. The idea is to achieve some level of defense against fraudulent email, such as phishing and spoofing, that could erode a recipient’s trust in receiving email. That said, the act of sending authenticated email does not, in and of itself, assert that the email is good or wanted email; it only means that the mail is such that a reputation for the authenticated party can be reliably established and used in email acceptance and placement decisions.

There are two primary forms of email authentication in use today—Sender Policy Framework (SPF) and DomainKeys Identifed Mail (DKIM). In this blog post, I’ll explain what SPF is, and how it works.

SPF Overview

Sender Policy Framework (SPF), to paraphrase RFC 7208, is a protocol that not only allows an organization to authorize hosts and networks to use its domain names when sending email, but also provides a way that a receiving host can check that authorization.

When you’re done reading this post, I hope you’ll have learned the following:

  • SPF is a “path-based” authentication system.
  • SPF policies are declared in DNS TXT records.
  • Validation is keyed to the Return-Path domain (what we here at SparkPost call the “bounce domain”) or the HELO domain.
  • Validation can be done early in the SMTP transaction, so it’s a good check to use to quickly reject unauthenticated mail.
  • It is prone to a relatively high percentage of false negatives.

“Path-Based” Authentication

SPF is deemed to be a path-based authentication system because it’s tied solely to the path the message took to get from its origin to its destination. When a sending server initiates an SMTP transaction with a receiving server, the receiving server will determine whether or not the sending server is authorized to send that message based on the domain’s SPF policy. It’s solely a decision based on where the message is coming from, with nothing whatsoever to do with the content of the message.

Declaring an SPF Policy

A domain’s SPF policy record is just a specifically formatted DNS TXT record, commonly containing one or more of the following “mechanisms”:

  • v=spf1 – Required first token to indicate that TXT record is SPF record (a domain can have multiple TXT records)
  • ipv4, ipv6 – Used to specify IP addresses and networks that are permitted to send mail for the domain
  • a – If the sending domain has a DNS “A” record that resolves to the sending IP, the IP is permitted
  • mx – If the sending IP is also one of the MX records for the sending domain, the IP is permitted
  • all – Required last token, always modified:
    • -all – Only what’s listed here can pass; reject failures.
    • ~all – Stuff that’s not here should softfail; accept it but note it.
    • ?all – Not sure if stuff that’s not here should pass.
    • +all – We think SPF is useless; pass everything.

Other less common mechanisms that are still in widespread use are:

  • include – Used when a domain not only has its own servers, but also uses someone else’s servers. Must be used judiciously. Each include is another DNS query, and SPF records can’t require more than ten DNS queries to resolve.
  • redirect – When domain A and domain B are owned by the same entity and use the same infrastructure. The owners typically want to maintain just one copy of the SPF record for both, and configure B to redirect queries to A.
  • exists – If a domain has lots of small, non-contiguous network blocks, it can use this mechanism to specify its SPF record, instead of listing them all. Useful to stay within size limit (512 bytes) for DNS response.

A note about the “ptr” Mechanism

A final mechanism, “ptr” existed in the original specification for SPF, but has been declared “do not use” in the current specification. The ptr mechanism was used to declare that if the sending IP address had a DNS PTR record that resolved to the domain name in question, then that IP address was authorized to send mail for the domain.

The current status of this mechanism is that it should not be used. However, sites doing SPF validation must accept it as valid.

Some Example SPF Records

Here are some example records, and their meanings. This example shows RFC 1918 addresses, but I recognize that such addresses will never show up in real SPF records.

  • MX record, one IP address, one IP network, softfail everything else:
    • “v=spf1 mx ipv4:10.0.0.1 ipv4:192.168.24.0/24 ~all”
  • A record, two IP networks, reject everything else:
    • “v=spf1 a ipv4:10.0.3.0/23 ipv4:192.168.55.0/26 -all”
  • We send no mail:
    • “v=spf1 -all”
  • We think SPF is stupid:
    • “v=spf1 +all”
  • The SPF record for the domain sparkpostmail.com (redirect mechanism used)
    • “v=spf1 redirect=_spf.sparkpostmail.com”
  • The SPF record for _spf.sparkpostmail.com (exists and ptr mechanisms used):
    • “v=spf1 exists:%{i}._spf.sparkpostmail.com ptr:sparkpostmail.com ptr:spmta.com ptr:flyingenvelope.com ~all”

At SparkPost, we’re currently using the ptr mechanism in our SPF record. We have found it necessary due to a lack of universal support for validating the exists mechanism. And to date, we’ve seen no SPF failures result due to our using the ptr mechanism.

How Does SPF Authentication Work?

Here is what a typical SMTP transaction looks like when SPF is involved:

  • The sending server (client) connects to receiving server
  • The receiving server notes the client’s connecting IP address
  • They exchange greetings, including client’s HELO hostname
  • Client issues SMTP MAIL FROM command

The receiving server can now look up SPF record for either MAIL FROM domain or HELO hostname to determine whether or not the connecting IP is authorized to send mail for the domain, and decide whether or not to accept the message.

MAIL FROM or HELO – Which to Check?

The SPF specification stipulates that receiving sites MUST check SPF for the MAIL FROM domain, and RECOMMEND checking it for the HELO hostname. In practice, checking only happens on the MAIL FROM domain, except perhaps for those times when the MAIL FROM address is the NULL sender (i.e., “<>”), in which case the HELO hostname is the only identity available.

SPF Is Not Foolproof

While it seems a relatively straightforward way to authenticate email, SPF is vulnerable to failure in the form of false negatives. These failures can occur more frequently than many are comfortable with.

Here are two common scenarios where perfectly legitimate mail can fail SPF validation and so appear to be fraudulent:

  • Automated forwarding, where a message sent to jsmith@alumni.university.edu, a mailbox set to automatically forward all mail to jsmith@isp.com
  • Mailing lists, where mail sent to talk-about-stuff@listserv.foo.com gets “exploded” and sent to each subscriber

In both of these cases, if the Return-Path is unchanged from its original, the mail may fail SPF checks (depending on the modifier used with the ‘all’ mechanism). This is because it arrives at its final destination from an intermediate server, not the original one, and that intermediate server is not likely to be in the domain’s SPF record. A technique called “Sender Rewriting Scheme” or “SRS” can mitigate this risk, and some larger sites have adopted it, but it’s not universal.

Wrap Up

So that’s SPF authentication, how it works, how to declare an SPF policy, and what the pitfalls are in using SPF. SPF was the first email authentication scheme to achieve widespread adoption, but it’s not the only one out there. SPF authentication is most effective when deployed in combination with other anti-fraud techniques.

-Todd Herr

For further reading on email authentication, please see these great resources about topics like SPF, DKIM, and DMARC.

10 Steps Great Deliverability Blog Footer

DMARC passport

Often mentioned in the same breath as the email authentication protocols SPF and DKIM, DMARC, or Domain-based Message Authentication, Reporting, and Conformance, is not in itself an authentication protocol. Instead, the purpose of DMARC is to allows you, the domain owner, to protect your email reputation by:

  • Announcing email authentication practices,
  • Requesting treatment for mail that fails authentication checks, and
  • Soliciting reports about mail claiming to be from its domain.

DMARC can be an effective tool for you to to use in your fight against fraudulent mail that targets their domain name (e.g., phishing and spoofing), and that can promote higher trust by your recipients for your mail. This higher trust should in turn lead to higher engagement with your mail, and mail that’s opened and generating clicks drives sales and higher ROI for your email campaigns.

In addition to protecting your domain, I predict that implementing DMARC now will be an excellent way to “future-proof” your domain. Here at SparkPost, we believe that as the industry moves to IPv6, it is almost certain to move from an IP-based reputation model to a domain-based reputation model. Domain-based reputation will require domain-based authentication, and DMARC, in concert with DKIM and SPF, will help domains in establishing a domain-based reputation long before one might be absolutely necessary.

In this post, I’ll tell you all you need to know about leveraging DMARC to protect your email reputation, and give you pointers on how to set it up for your domains.

Terms To Know

Before I get into setting up DMARC for your domain, I want to make sure we’re speaking the same language. Let’s start by defining some terms I’ll use throughout the rest of this document.

RFC5322.From Domain

The RFC5322.FromDomain is the domain part of the email address that’s usually seen by a recipient of your email when it’s being read. In the following example, the RFC5322.From domain is “joesbaitshop.com”

From: Joe’s Bait and Tackle <sales@joesbaitshop.com>

DKIM d= Domain

DKIM is an authentication protocol that allows a domain to take responsibility for a message in a way that can be validated by the receiver of the message; this is done through the use of cryptographic signatures inserted into the message’s headers as it’s leaving its origination point. These signatures are effectively snapshots of what the message looked like at that point in time, and the receiver can use these snapshots to see if the message has arrived unchanged at its destination. The process of producing and inserting these snapshots is called DKIM signing, and the domain that takes responsibility for the message by signing it inserts its name into the header in a key-value tag as “d=signingDomain”, and so it’s referred to as the DKIM d= domain.

Return-Path Domain

The Return-Path domain, sometimes called the RFC5321.From Domain or the Mail From domain, is the domain to which bounces are routed; it is also the domain on which SPF checks are done during the email transaction. This domain is usually not seen by the recipient unless the recipient is savvy enough to look at all the headers in a given message.

At this time, all mail sent through sparkpost.com will have sparkpostmail.com as its Return-Path domain, as in the following example:

Return-Path: <msprvs1=16880EmYZo7L3=bounces-2785@sparkpostmail1.com>

We are contemplating giving our customers the ability to personalize this domain in the future.

Organizational Domain

The term “Organizational Domain” refers to the domain that was submitted to a registrar to create the domain’s presence on the internet. For SparkPost, our organizational domains are sparkpost.com and sparkpostmail.com.

Domain Alignment

The last term to understand regarding DMARC is “Domain Alignment,” and it comes in two variants: “relaxed” and “strict.”

Relaxed Domain Alignment

Any two domains are said to have relaxed domain alignment when their Organizational Domains are the same. For example, a.mail.sparkpost.com and b.foo.sparkpost.com have relaxed domain alignment, because of their common Organizational Domain, sparkpost.com.

Strict Domain Alignment

Two domains are said to be in strict domain alignment if and only if they are identical. So, foo.sparkpost.com and foo.sparkpost.com are in strict alignment, as the two domains are identical. On the other hand, foo.sparkpost.com and bar.foo.sparkpost.com, are only in relaxed alignment.

DMARC Domain Alignment Requirements

In order for DMARC validation checks to pass, DMARC requires that there be domain alignment as follows:

  • For SPF, the RFC5322.From domain and the Return-Path domain must be in alignment
  • For DKIM, the RFC5322.From domain and the DKIM d= domain must be in alignment

The alignment can be relaxed or strict, based on the published policy of the sending domain.

How DMARC Works to Protect Your Email Reputation

When I speak of a mailbox provider or other domain “checking DMARC”, or “validating DMARC”, or “applying DMARC policy”, what I mean is that the domain receiving a message is performing the following steps:

  1. Figure out the message’s RFC5322.From domain
  2. Look up that domain’s DMARC policy in DNS
  3. Perform DKIM Signature validation
  4. Perform SPF Validation
  5. Check domain alignment
  6. Apply DMARC policy

In order for a message to pass DMARC validation, the message must pass only one of the two authentication and alignment checks. So, a message will pass DMARC validation if any of the following are true:

  • The message passes SPF checks and the RFC5322.From domain and Return-Path domain are in alignment, or
  • The message passes DKIM validation and the RFC5322.From domain and DKIM d= domain are in alignment, or
  • Both of the above are true

By default, mail sent through sparkpost.com will have “sparkpostmail*.com” as its Return-Path domain, so the only possible path to passing DMARC validation is through the DKIM check. We ensure that messages are signed twice, so there will be one DKIM-Signature with a d= domain matching the customer’s domain, and this domain should align with the RFC5322.From domain on the message.

It is possible for customers to get a custom Return-Path domain, as described by this support article. There are a number of reasons, in addition to DMARC, that a domain owner may want to pursue this option, but the choice is up to you.

Making DMARC Work for Your Domain

Now that I’ve explained the mechanics of DMARC, let’s talk about how to make DMARC work for you, which involves the following three steps:

  1. Making preparations to receive DMARC reports
  2. Deciding on what DMARC policy to use for your domain
  3. Publishing your DMARC record

I’ll cover each of these in detail below, but I’ll tell you straight out that step 1 above will consume about 95% of your preparation time.

Preparing to Receive DMARC Reports

Any domain that publishes a DMARC policy should first prepare to receive reports regarding their domain. These reports will be generated by any domain that does DMARC validation and sees mail claiming to be from your domain, and will be sent to you on at least a daily basis. The reports will come in two formats:

  • Aggregate reports, which are XML documents showing statistical data of how much mail was seen by the reporter from each source, what the authentication results were, and how the messages were treated by the reporter. Aggregate reports are designed to be machine-parsed, with their data stored somewhere to allow for overall traffic analysis, auditing of your domain’s message streams, and perhaps identification of trends in sources of unauthenticated, potentially fraudulent, email.
  • Forensic reports, which are individual copies of messages which failed authentication, each enclosed in a full email message using a format called AFRF. The forensic reports are supposed to contain full headers and message bodies, but many reporters strip or redact some information due to privacy concerns. Nonetheless, the forensic report can still be useful both for troubleshooting a domain’s own authentication issues and for identifying, from URIs in the message bodies, malicious domains and websites used to defraud the domain owner’s customers.

The preparation to receive these reports involves first creating two mailboxes in your domain to handle these reports, such as agg_reports@yourdomain.com and afrf_reports@yourdomain.com. Note that those mailbox names are completely arbitrary, and there are no requirements for the naming of the local part of the mailbox; you are free to choose any names you wish, but keep the two separate for easier processing.

Once the mailbox names are selected and created in your domain, the next thing to do here is to put tools in place to read these mailboxes and make use of the data, especially the aggregate data reports, which again are designed to be machine-parsed, rather than read by a human. The forensic reports, on the other hand, might be manageable simply by reading them yourself, but your ability to do so will depend both on your mail client’s understanding of how to display messages in the AFRF format and on the volume of reports you receive.

While it’s possible for you to write your own tools to process DMARC reports, until such time as we provide such services for sparkpost.com customers (something we’re considering, but not promising yet), we recommend that you make use of tools that are already available for the task.

Which DMARC Policy to Use 

The DMARC specification provides three choices for domain owners to use to specify their preferred treatment of mail that fails DMARC validation checks. They are:

  • none, meaning treat the mail the same as it would be treated independent of DMARC validation checks
  • quarantine, meaning accept the mail but place it somewhere other than the recipient’s Inbox (typically the spam folder)
  • reject, meaning reject the message outright

It is important to keep in mind that the domain owner can only request such treatment in its DMARC record; it’s up to the recipient of the message to decide whether or not to honor the requested policy. Some will do so, while others may be a bit more lenient in applying policy, such as only spam-foldering mail when the domain’s policy is reject.

We recommend to all of our customers that they launch with a policy of none, simply to be safe. While we are confident in our ability to properly authenticate your mail through DKIM signing, it’s still best to take some time to examine reports about your domain before getting more aggressive with your DMARC policy.

Publishing Your DMARC Policy

A domain’s DMARC policy is announced by publishing a DNS TXT record at a specific place in the DNS namespace, namely “_dmarc.domainname.tld” (note the leading underscore). A basic DMARC policy record for our example domain from earlier, joesbaitshop.com, might look something like this:

Breaking down this record, we have:

  • v=DMARC1 specifies the DMARC version (1 is the only choice right now)
  • p=none specifies the preferred treatment, or DMARC policy
  • rua=mailto:agg_reports@joesbait.com is the mailbox to which aggregate reports should be sent
  • ruf=mailto:afrf_reports@joesbait.com is the mailbox to which forensic reports should be sent
  • pct=100 is the percentage of mail to which the domain owner would like to have its policy applied. Domains just getting started with DMARC, especially those likely to generate a high volume of reports, may want to start with a much lower number here to see how their report-handling processes stand up to the load.

There are other configuration options available for a domain owner to use in its DMARC policy record as well, but the tips I’ve provided should be a good start.

Summary

There’s a lot to unpack in the information above! I hope you find the how-to for creating a DMARC policy record helpful. I also hope that my explanation of why DMARC matters helps make clear why you should start using this important tool for protecting your email reputation.

Of course, this isn’t a complete or authoritative document on the subject. If you want to dig in deeper or want more help, a great place to start is the official DMARC FAQ. And, it goes without saying that the crackerjack SparkPost support team is ready to help you with configuring your SparkPost account for DMARC as well.

Thanks for reading—and get started protecting your domains with DMARC today!

—Todd

SPF and DKIM

You might have heard about SPF and DKIM, and you might even know that they’re email authentication standards. But a lot of the information out there about SPF and DKIM is a little hard to understand. That’s why I’m going to lay it out in an easy-to-understand way and give you some tips for setting them up for your own domains.

In the earliest days of what we would recognize as modern email, there was little in the way of sender verification. Pretty much all spam, scams, and viruses that spread through email did so using falsified sender information – as they still do today. So how can you be sure that the sender is who they say they are?

When you go to www.google.com, you’re generally pretty confident that Google has control over what gets sent back to you. This is because the Domain Name System (DNS)—a distributed network of servers that act as a phonebook—connects the domain with a variety of records, including where to find the real google.com.

Email uses a later adaptation of this same system to verify senders. Specifically, we use what’s called a Sender Policy Framework (SPF) record. Much like your browser does when you type in google.com, your email client will look up the DNS entries for any domain you receive email from and make sure that the domains listed there agree that the server sending the email is legitimate.

Think of it like this: if you wrote your own name and someone else’s return address on an envelope and sent it to me, SPF is a way for me to call each of you up and make sure it’s legitimate. But then how do I know it’s actually you that wrote the contents?

That’s where DKIM comes in. DKIM is a combination of DomainKeys, originally designed at Yahoo, and Cisco’s Identified Mail.

DKIM uses public key encryption which, to criminally oversimplify, is basically a giant math problem that takes forever to solve unless you have two very specific hints. One hint I make publicly available, the other I send along with the message. If you feed both of those hints into the problem-solving machine and it gives you the answer, then you know I wrote the contents.

Implementing each of these layers of validation is pretty straight-forward and SparkPost makes it extra simple. In fact, SparkPost handles the SPF part automatically, so all email from your account is already SPF-compliant.

That leaves just DKIM for you to configure. Here’s how: When you log into your account, under Account > Sending Domains you can find the DNS entry you need to add. In your host dashboard, you should see a section for DNS entries. If you just set up this host or domain, this should look pretty familiar.

SPFDKIMSendwithus1

Some hosts will mention SPF records here, but you’re looking for a generic TXT record.

There are two parts to a DKIM record: the host name and its value. You will want to paste the DKIM record value you copied from the SparkPost dashboard earlier. Submit the form, and you’re set.

By the way, not all registrars allow you to add DNS records to arbitrary host names, which is necessary for DKIM setup. Most of the time, they can do it for you, but you might need to go through a support request for it. For specific instructions for different registrars, you can check out the DKIM support guide.

P.S. If you want to check to see if your email is SPF and DKIM enabled, you can use our Validator tool.

Alex Mohr

Alex Mohr runs marketing and analytics for sendwithus.com and works tirelessly to rid the world of bad coffee and bad emails.

 

 

10 Ways To Build Brand Trusty and Loyalty Through Transactional Email

Message Systems is pleased to announce today that we have qualified for the Online Trust Alliance’s 2014 Email Integrity Honor Roll. The audit evaluates a company’s adoption of email authentication practices and focuses in particular on email authentication that helps detect and block spoofed and forged email such as Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM) and Domain-based Message Authentication, Reporting & Conformance (DMARC) practices.

Companies are recognized as leaders in the space of email security and brand protection if they have:

  • Implemented SPF and DKIM at the corporate or top-level domain
  • Have a DMARC record published

As part of this audit, the Online Trust Alliance reviewed over 800 companies and brands, out of which only 12% passed, and Message Systems is honored to make the list!

From the positive effects we’re seeing across the industry, it’s clear that DMARC is succeeding as intended in deterring malicious email attacks – which is great news. Message Systems will continue to provide the technologies our customers need to quickly implement SPF, DKIM and DMARC to protect their brands and customers.

– Phillip Merrick CEO, Message Systems

Earlier this year, we also qualified for the Online Trust Alliance’s 2014 Online Trust Audit and Honor Roll – this latest honor further recognizes our commitment to educating the industry on email security for online brand protection through leading by example.

Here’s the Good News

The 2014 Email Integrity Audit had several key findings, both positive and negative, on the state of email authentication in the various industries. As always, let’s start with the good:

  • Adoption of SPF and DKIM is rising across all the industries. The Internet Retailer 100 had an 88% adoption rate, while the Internet Retailer 500 had the largest growth, rising from 56% to 74% adoption.
  • Adoption of DMARC is increasing slowly but steadily, and the top social sites had the highest score for DMARC adoption at 36%.

Email Domain Brand Protection

Among the list of companies that made the 2014 Email Integrity Honor Roll, many were our customers and partners, and we’d like to offer everyone a hearty congratulations!

Internet Retailer Top 500

American Greetings logo Groupon logo

Social Top 50

fb_icon_325x325 twittericon linkedin zyngalogo1

DMARC allowed us to dramatically reduce the number of forged emails sent to our users. DMARC was a direct benefit to our users by blocking these impersonations.

– Josh Aberant Postmaster, Twitter

OTA Members

Harland Clarke Digital logo Return Path logo

SPF and DKIM are vitally important for email senders to implement today, but they are merely table stakes in an escalating battle against email fraud. DMARC is a powerful solution empowering senders who are prone to brand infringement and malicious attacks.

– Robert Holmes General Manager, Fraud & Brand Protection Services, Return Path

… And Now for the Bad News

While adoption rates for SPF, DKIM and DMARC continued to grow, the news about the state of email authentication in the industry wasn’t all quite as rosy:

  • Of all the consumer domains sampled, only 8.3% have implemented SPF, DKIM and DMARC.
  • Brands are failing to authenticate at top level domains; SPF and DKIM adoption only grew at the level of sub domains, thus leading to limited brand and consumer protection.
  • While DMARC adoption is growing, it still remains low.
  • Top FDIC insured banks had the highest failure rate compared to all sectors due to a lack of email authentication – only 17% passed the audit.
  • The top 50 federal government sites consistently scored at the bottom of all email authentication metrics – only 4% passed the audit.

The 2014 Email Integrity Audit findings revealed that consumers are at a higher risk of receiving forged and spoofed email from major banks and federal government sites – a scary thought as these are institutions that generally command the trust of the public. The 2014 Email Integrity Audit report also specifically called out the security weaknesses in email infrastructure in enterprises and commodity message/mail transfer agents:

Unfortunately, in many enterprises the email infrastructure does not natively support outbound signing or inbound checking for SPF, DKIM or DMARC. Equally as concerning is the lack of support for inbound authentication from leading MTAs (Mail Transfer Agents), the hosting community and email technology providers.”

– 2014 Email Integrity Audit report

The report points to the inconsistency of email authentication in organizations due to email marketing being outsourced to disparate third party systems. We’d like to point out that our email infrastructure platform, Momentum, (available in on-premise and managed cloud versions) fully supports both inbound and outbound email authentication – one of the many reasons why the world’s largest senders choose us to send 20% of global legitimate email through Momentum.

Finally, we’d like to end with a note of caution from the audit report: a lack of email authentication exposes businesses to the risk of liability and class action suits in the event of a data breach. If you’re interested to find out whether your email service provider is authenticating your mail, test it with our email Validator – it checks for DKIM, SPF and DMARC email authentication.

Free Email Testing Tool - The Validator

Sign up for the Online Trust Alliance Email Security Training Academy today.

It’s our great pleasure to announce that we will be hosting the Online Trust Alliance’s Email Security Training Academy on July 9 at our San Francisco office. As a member of the OTA, Message Systems has made it our mission to continuously educate and promote the adoption of DMARC email standards among our clients and the industry as a whole. We were honored to be placed on OTA’s honor roll for two years in a row, and happy to support OTA’s goal of enhancing online trust in the industry and among organizations.

Covering the fundamentals of DMARC, SPF and DKIM, OTA’s full-day intensive training will include case studies from Microsoft, Facebook, Twitter and American Greetings, as well as best practices for implementing, managing and optimizing email and domain authentication. During training, technical details about deploying SPF, DKIM and DMARC will be discussed and attendees will also come to understand the importance and benefits of implementing and managing email authentication.

We believe that online businesses have a responsibility to protect users from phishing and other email abuse.  OTA’s training and guidance has been invaluable to our team and ultimately our customers, said Within 90 days of implementing we were able to block over 100,000 deceptive emails targeting our customers.

Sal Tripi Assistant Vice President of Digital Operations & Compliance, Publishers Clearing House.

Whether you’re an IT or security professional, an email administrator, email marketer or a brand owner, OTA’s email security training offers knowledge for everyone such as:

  • Enhancing brand reputation
  • Communicating with ISPs on what to do with malicious mail
  • Receiving email intelligence reports from ISPs and receiving networks
  • Reducing malicious mail
  • Optimizing email deliverability and clickthrough

If you’d like to learn more about why protecting customers and your brand from abuse is so important, have a look at the 2014 Data Protection & Breach Readiness Guide released by OTA earlier this year. According to the report, 2013 was the worst year for data breaches with 740 million records being exposed. Sadly, 89% of these incidents could have been avoided with simple security best practices such as DMARC email authentication.

Need another reason to register? Here’s one: The early bird discount ends this Friday, July 4, so be sure to sign up before then. Only 10 seats remain at the moment, so the sooner you act, the better!

Find out about DMARC, DKIM and SPF before you attend OTA’s workshop with the How DMARC Is Saving Email E-Book.

How DMARC Is Saving Email