Ah, the whitelist… A widely used, but misunderstood term. Many people believe that an email whitelist is a permanent, direct path to the inbox at an ISP, with no filtering applied. That may have been true at one point, long long ago, when mail systems were less sophisticated. Today, for the most part, whitelists are a little more flexible and have varied benefits. They are certainly worth understanding and considering, but they may not be the magic bullet you are hoping for.

What is a whitelist?

A whitelist is a list of IPs or domains associated with a good sending reputation. It is employed by ISP mail systems (generally after the gateway) to provide benefits associated with increased trust in the mail coming from the IPs or domain in question.

Why do whitelists exist?

Whitelists were initially developed back when systems receiving email (ISPs) were less sophisticated. As we know, the majority of the email out there is spam (I’ve read statistics between 85% and 95%), so anti-spam systems were overly aggressive in order to protect email users. Having a whitelist in place is a way to give your system a leg-up on correctly categorizing mail that people want.

What are the benefits of being on a whitelist?

There are various types of whitelists out there and the benefits vary depending on the 3rd party provider or ISP. While at one point, a whitelist might have placed your mail directly in the inbox with zero filtering, that is not very common today. Whitelist benefits might include some of the following:

  • Increased throughput limits (ability to send more mail faster)
  • Fewer levels of filtering
  • A bump up during the calculation of reputation

How do you get added to a whitelist?

Not just any sender can get themselves whitelisted. You have to meet the criteria of the whitelist provider, which generally align with email best practices:

  • Low complaint rates
  • Low/no spam trap hits
  • Low percentage of invalid addresses
  • Authenticated mail
  • Etc.

If you meet the criteria for being added to a whitelist, you must then apply for approval.

What are the available whitelists?

Here is a list of the more commonly known whitelists:

  1. Yahoo Bulk Sender (free)
  2. AOL Whitelist (free)
  3. Return Path Certification (paid, 3rd party)
  4. Certified Senders Alliance (paid, 3rd party)

Do you need to be on a whitelist?

As I mentioned above, to get on a whitelist, you have to meet (and maintain) certain standards and follow best practices. As a result, the argument is generally made that if you are a good sender and you are in tune with what your subscribers want, you shouldn’t need to be whitelisted.

At SparkPost, we consider it a standard practice to subscribe to the free, publicly available whitelists (AOL, Yahoo) as a precautionary measure. The decision of whether or not to pay for the 3rd party whitelisting services (Return Path and CSA) is a personal decision for our customers.

As I alluded to earlier, mail receiving systems have gotten more sophisticated over the years, and it’s easier for them to determine whether or not you are a legitimate sender based on the way users interact with your mail. Therefore, they largely no longer employ whitelists in a blanket, “direct to inbox” fashion. Some providers, like Gmail, don’t offer a whitelist at all as they are so confident in their system’s ability to correctly categorize the mail based on user behavior.

Things to note…

Even though you may have been accepted onto a whitelist, ISPs and 3rd party providers are constantly monitoring your traffic and you can be kicked off if you break the rules (e.g. You decide to mail to users who haven’t engaged in a very long time, resulting in tons of complaints, invalid address, and spam trap address hits.) You don’t always get notified when this happens, so it’s important to maintain good sending practices.

While the whitelists I’ve been talking about here apply to domain/IP level filtering at the ISP level, there is also personal whitelisting, which takes place at the user level, and can have a positive impact on your mail program… Stay tuned for a blog on personal whitelisting coming up.
Confused on how to be a good sender? SparkPost has Deliverability experts that can help.

Clea Moore
Deliverability Lead

9 Things ISPs Really Want Email Marketers to Know

When we speak of “Email Authentication”, we’re referring to a technique that is meant to provide to the recipient of a message some level of certainty that the message actually originated with the claimed source of the message. The idea is to achieve some level of defense against fraudulent email, such as phishing and spoofing, that could erode a recipient’s trust in receiving email. That said, the act of sending authenticated email does not, in and of itself, assert that the email is good or wanted email; it only means that the mail is such that a reputation for the authenticated party can be reliably established and used in email acceptance and placement decisions.

There are two primary forms of email authentication in use today—Sender Policy Framework (SPF) and DomainKeys Identifed Mail (DKIM). In this blog post, I’ll explain what SPF is, and how it works.

SPF Overview

Sender Policy Framework (SPF), to paraphrase RFC 7208, is a protocol that not only allows an organization to authorize hosts and networks to use its domain names when sending email, but also provides a way that a receiving host can check that authorization.

When you’re done reading this post, I hope you’ll have learned the following:

  • SPF is a “path-based” authentication system.
  • SPF policies are declared in DNS TXT records.
  • Validation is keyed to the Return-Path domain (what we here at SparkPost call the “bounce domain”) or the HELO domain.
  • Validation can be done early in the SMTP transaction, so it’s a good check to use to quickly reject unauthenticated mail.
  • It is prone to a relatively high percentage of false negatives.

“Path-Based” Authentication

SPF is deemed to be a path-based authentication system because it’s tied solely to the path the message took to get from its origin to its destination. When a sending server initiates an SMTP transaction with a receiving server, the receiving server will determine whether or not the sending server is authorized to send that message based on the domain’s SPF policy. It’s solely a decision based on where the message is coming from, with nothing whatsoever to do with the content of the message.

Declaring an SPF Policy

A domain’s SPF policy record is just a specifically formatted DNS TXT record, commonly containing one or more of the following “mechanisms”:

  • v=spf1 – Required first token to indicate that TXT record is SPF record (a domain can have multiple TXT records)
  • ipv4, ipv6 – Used to specify IP addresses and networks that are permitted to send mail for the domain
  • a – If the sending domain has a DNS “A” record that resolves to the sending IP, the IP is permitted
  • mx – If the sending IP is also one of the MX records for the sending domain, the IP is permitted
  • all – Required last token, always modified:
    • -all – Only what’s listed here can pass; reject failures.
    • ~all – Stuff that’s not here should softfail; accept it but note it.
    • ?all – Not sure if stuff that’s not here should pass.
    • +all – We think SPF is useless; pass everything.

Other less common mechanisms that are still in widespread use are:

  • include – Used when a domain not only has its own servers, but also uses someone else’s servers. Must be used judiciously. Each include is another DNS query, and SPF records can’t require more than ten DNS queries to resolve.
  • redirect – When domain A and domain B are owned by the same entity and use the same infrastructure. The owners typically want to maintain just one copy of the SPF record for both, and configure B to redirect queries to A.
  • exists – If a domain has lots of small, non-contiguous network blocks, it can use this mechanism to specify its SPF record, instead of listing them all. Useful to stay within size limit (512 bytes) for DNS response.

A note about the “ptr” Mechanism

A final mechanism, “ptr” existed in the original specification for SPF, but has been declared “do not use” in the current specification. The ptr mechanism was used to declare that if the sending IP address had a DNS PTR record that resolved to the domain name in question, then that IP address was authorized to send mail for the domain.

The current status of this mechanism is that it should not be used. However, sites doing SPF validation must accept it as valid.

Some Example SPF Records

Here are some example records, and their meanings. This example shows RFC 1918 addresses, but I recognize that such addresses will never show up in real SPF records.

  • MX record, one IP address, one IP network, softfail everything else:
    • “v=spf1 mx ipv4:10.0.0.1 ipv4:192.168.24.0/24 ~all”
  • A record, two IP networks, reject everything else:
    • “v=spf1 a ipv4:10.0.3.0/23 ipv4:192.168.55.0/26 -all”
  • We send no mail:
    • “v=spf1 -all”
  • We think SPF is stupid:
    • “v=spf1 +all”
  • The SPF record for the domain sparkpostmail.com (redirect mechanism used)
    • “v=spf1 redirect=_spf.sparkpostmail.com”
  • The SPF record for _spf.sparkpostmail.com (exists and ptr mechanisms used):
    • “v=spf1 exists:%{i}._spf.sparkpostmail.com ptr:sparkpostmail.com ptr:spmta.com ptr:flyingenvelope.com ~all”

At SparkPost, we’re currently using the ptr mechanism in our SPF record. We have found it necessary due to a lack of universal support for validating the exists mechanism. And to date, we’ve seen no SPF failures result due to our using the ptr mechanism.

How Does SPF Authentication Work?

Here is what a typical SMTP transaction looks like when SPF is involved:

  • The sending server (client) connects to receiving server
  • The receiving server notes the client’s connecting IP address
  • They exchange greetings, including client’s HELO hostname
  • Client issues SMTP MAIL FROM command

The receiving server can now look up SPF record for either MAIL FROM domain or HELO hostname to determine whether or not the connecting IP is authorized to send mail for the domain, and decide whether or not to accept the message.

MAIL FROM or HELO – Which to Check?

The SPF specification stipulates that receiving sites MUST check SPF for the MAIL FROM domain, and RECOMMEND checking it for the HELO hostname. In practice, checking only happens on the MAIL FROM domain, except perhaps for those times when the MAIL FROM address is the NULL sender (i.e., “<>”), in which case the HELO hostname is the only identity available.

SPF Is Not Foolproof

While it seems a relatively straightforward way to authenticate email, SPF is vulnerable to failure in the form of false negatives. These failures can occur more frequently than many are comfortable with.

Here are two common scenarios where perfectly legitimate mail can fail SPF validation and so appear to be fraudulent:

  • Automated forwarding, where a message sent to jsmith@alumni.university.edu, a mailbox set to automatically forward all mail to jsmith@isp.com
  • Mailing lists, where mail sent to talk-about-stuff@listserv.foo.com gets “exploded” and sent to each subscriber

In both of these cases, if the Return-Path is unchanged from its original, the mail may fail SPF checks (depending on the modifier used with the ‘all’ mechanism). This is because it arrives at its final destination from an intermediate server, not the original one, and that intermediate server is not likely to be in the domain’s SPF record. A technique called “Sender Rewriting Scheme” or “SRS” can mitigate this risk, and some larger sites have adopted it, but it’s not universal.

Wrap Up

So that’s SPF authentication, how it works, how to declare an SPF policy, and what the pitfalls are in using SPF. SPF was the first email authentication scheme to achieve widespread adoption, but it’s not the only one out there. SPF authentication is most effective when deployed in combination with other anti-fraud techniques.

-Todd Herr

For further reading on email authentication, please see these great resources about topics like SPF, DKIM, and DMARC.

10 Steps Great Deliverability Blog Footer

dkim three things you might not know

The big ISPs like Gmail, Yahoo! Mail, and Hotmail are pushing senders to authenticate their email through a carrot/stick approach: Do it well, and you’ll achieve higher deliverability rates. Authenticate poorly, and your email will be downgraded, degrading your overall deliverability. Other symptoms might include warnings or a lack of graphics and identifying logos.

Microsoft & Google are beginning to show consumers if an email has been authenticated. Note that Google will insert logos for authenticated email and “?” for non-authenticated email. Similarly, Microsoft will redact logos and graphics and add red indicators if an email lacks proper authentication. Authenticate properly, and the email will display a green shield and render all logos and graphics.

Valimail Post

Given this environment, we wanted to write a quick post on one aspect of email authentication that still breeds confusion: DKIM (short for DomainKeys Identified Mail). DKIM is an open, DNS-based email authentication standard that uses public-key encryption to authenticate email messages. There are several issues that a sender should consider when implementing DKIM.

The Right Way to Implement DKIM

No Key Sharing. Each customer, brand, or entity sending unique mail streams should have their own, dedicated DKIM key. When a sender doesn’t mix and match DKIM keys between mail streams, a compromised DKIM key can only impact a single brand or stream.

Regular Key Rotation. As recommended by the specification, DKIM keys should be changed (or “rotated”) on a regular basis, about 3–4 times per year. Rotation ensures that if a key is compromised for any reason (for example, by a hacker who obtains the private key), then the compromised key can only be used for a limited time. Once the old key is rotated out and replaced with a new key, the compromised key is useless.

Distributed, Encrypted Key Storage. Compromised DKIM private keys are extremely valuable—attackers can use them to impersonate senders and fly under the radar virtually undetected. Senders need to avoid storing private keys in plaintext, avoid maintaining a centralized database of keys, and follow best practices for PKI security.

The Reality of How DKIM is Implemented

Widespread Key Sharing. Because DKIM is relatively complex and proper key management is burdensome, senders often use the same key across their brands, entities and mail streams. The simplified configuration of a shared key provides relative ease in terms of implementation but it creates a unique vulnerability: a single compromised key gives a bad actor total and complete access.

Little to No Key Rotation. Because key rotation typically requires a sender to manually update one or more DNS records—or even worse, have their partners and sending entities manually update one or more DNS records—key rotation is extremely rare in practice. DKIM keys are typically set once and never changed. It’s not uncommon to see DKIM keys that are 5–10 years old in production!

Centralized, Plain Text Key Storage. Finally, even if a sender tries to do DKIM correctly—provide one key per sending entity or mail stream, and rotate keys on a regular basis—the path of least resistance is to store the keys for all their sending domains in a central database in plaintext, to simplify key management and distribution to the mail servers. Unfortunately this sort of architecture is a beacon to criminals, and makes it exceedingly easy to steal all of the sender’s keys during a breach.

Given that numerous major brands and ESPs have reportedly been breached over the last couple of years, this approach must be considered highly risky.

What’s the Answer?

Senders should use a DKIM system that supports frequent and automated key rotation, defines unique DKIM keys per mail stream, and stores the DKIM private keys in a secure way. Email authentication must be treated as an on-going policy and practice and not like a check box—too much depends on it like your brand’s integrity.

Whether or not you are interested in authentication, feel free to drop us a line and we’d be happy to discuss it further. Here’s to automated and secure authentication!

-Alex Garcia-Tobar

About the Author

alex garcia tobar valimail dkimAlexander García-Tobar is a serial entrepreneur and currently CEO & Co-Founder of ValiMail, providing Email Authentication as a Service™ . Previously, Alexander held various executive and analyst positions at leading research companies such as The Boston Consulting Group and Forrester Research and at Silicon Valley startups such as ValiCert, Sygate, and SyncTV.

 

 

 

Dev Survival Guide Blog Footer

The Validator - Corry

Earlier this year, we began building an email testing tool for marketers and other people who send emails out for their business. Before you assume this will help spammers get more spam gets into your inbox, I want to assure you that most of the things this tool helps with help make senders more wary of sending crap to you in the first place.

Our Goals

This tool is an attempt to introduce potential customers to Message Systems’s flagship software product Momentum. Momentum is built primarily for large companies, so it’s quite expensive. This makes it hard for them to give people a trial or demo of what Momentum can do except for in one-on-one meetings where our sales and their support staff can assist them.

So the Marketing and Engineering teams had an opportunity to fulfill a need while also creating a new way to attract traffic to the website. To recap, here are our goals in bullet form:

  • Give sales team a new tool to show prospects an example of some of the things our software can do
  • Increase website traffic
  • Give visitors another reason to keep coming back to website after the first time
  • Get some internet clout as experts in our field by offering a free resource
  • Brag a bit about the power of Momentum

How It Works

Here’s a quick video we made describing The Validator and what it does:

Basically, here’s what you do:

  1. We give you an email address.
  2. You send an email to that address. Ideally, a marketer would send a newsletter or a promotional email here to get a real business sense of this tool’s usefulness.
  3. You get a page full of results that will help you understand the strong and weak points of your email’s structure. Some of these will be based on the content of the message. Some of them will be the quality of the tool you’re using to send the message. All of it relates to how likely your message will get into your recipients’ inboxes instead of thrown into the spam folder.

The results page is probably the most important part of this tool. I designed it in a way that it allows users to get the quick answer for each test at a glance. All green means your message is awesome. Red marks mean there’s a problem that may cause your message to be flagged.

screen5

This information is useful for technical people, but most people (like me for instance) have no idea what these terms mean. Because of this, I decided to do some research of my own and explain each concept in terms that I understand as a layperson. You can find all of these explanations by clicking on the fields you’re not certain about.

screen6

If you want to dig even deeper into a concept, I included links to publications that explain at explicit detail what those terms mean and why you should care.

I feel that it’s a best practice in web UI design to allow the user to decide for themselves how far down the rabbit hole they want to go.

We’ve received great feedback both internally and externally about The Validator so far and it continues to be a resounding success among clients and prospects. Need to see if your email has the DMARC, SPF or DKIM seal of approval? Try it for yourself – Validate your email with The Validator.

Free Email Testing Tool - The Validator

The Basics of DMARC

You may have heard a lot about the DMARC standard in the past year, yet how does one go about ensuring that this standard for email authentication is met? Here’s a quick reference guide.

DNS entries that DMARC uses:

1 – The DMARC DNS text entry

The following is an example DMARC text entry for DNS :

v=DMARC1; p=none; rua=mailto:postmaster@mydomain.com; ruf=mailto:postmaster@mydomain.com; adkim=r; aspf=r; rf=afrf; sp=none

The above example was generated with the following utility: http://www.kitterman.com/dmarc/assistant.html

In order to get this in the real world use dig +short _dmarc.<domain> TXT

<snip>

[root@mymachine ~]# dig +short txt _dmarc.sovereignsociety.com

“v=DMARC1\; p=none\; rua=mailto:84tcdfj1@ag.dmarcian.com\;”

<snip>

2 – The SPF DNS text entry

The following is an example of a SPF DNS text entry:

mydomain.com. IN SPF “v=spf1 ip4:192.0.2.0/24 ip4:198.51.100.123 a -all”

You can find this for most domains by issuing a dig +short <domain> TXT and here is an example:

<snip>

[root@mymachine ~]# dig +short hotmail.com txt
“v=spf1 include:spf-a.hotmail.com include:spf-b.hotmail.com include:spf-c.hotmail.com include:spf-d.hotmail.com ~all”

[root@mymachine default]# dig +short spf-a.hotmail.com txt
“v=spf1 ip4:157.55.0.192/26 ip4:157.55.1.128/26 ip4:157.55.2.0/25 ip4:65.54.190.0/24 ip4:65.54.51.64/26 ip4:65.54.61.64/26 ip4:65.55.111.0/24 ip4:65.55.116.0/25 ip4:65.55.34.0/24 ip4:65.55.90.0/24 ip4:65.54.241.0/24 ip4:207.46.117.0/24 ~all” <snip>

More information on creating an SPF DNS text entry available here: http://www.openspf.org/SPF_Record_Syntax

For SPF validation you can use: http://www.kitterman.com/spf/validate.html

3 – The DKIM DNS text entry

The following is an example of an DKIM DNS text entry:

dkim1024._domainkey.mydomain.com. 86400 IN TXT “v=DKIM1; k=rsa; h=sha1; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCrZXNwzXOk0mRqPcgSUOnmVHro rg/BZHybpiBoDS/g6IaMjmVwaQf2E72x9yDBTgiUBtTCqydQRZJ3EbfYfvo+WAHq 2yz6HKR0XCwMDSE2S3brVe7mbV/GPEvnCuFPPEVjbfL4w0tEAd8Seb5h07uVQqy1 Q7jIOnF5fG9AQNd1UQIDAQAB”

You generally can find this by doing a dig +short _domainkey.<domain> TXT here is an example

<snip>

[root@mymachine ~]# dig +short google._domainkey.protodave.com TXT

“v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCGfiExKCF1qk/JMaESySByrwx2VjPYDZThQa8432pSTf9mj+AtFiY6wo9A4CMMDLfUBzbDhXFzw3s/qci/tTut+sqv+MSAHhCBJV72Kai64j6TjxUUnfW1RkEYvDhXL+9Wy9OODx2DBZeTpPd6N2Rm4ks3b5wvg73s7RCKjTA7XQIDAQAB”

<snip>

More details on validation of DKIM available here: http://dkimcore.org/tools/keycheck.html

Utility to create DKIM DNS entries: http://www.dnswatch.info/dkim/create-dns-record

The DMARC validation process.

In order for DMARC to begin passing a message, either the DKIM must pass or the SPF must pass, if neither passes then the action requested, in p (Domain policy) or sp (Subdomain policy) in the above DMARC DNS text entry will be adhered to. The options on the policies are none, quarantine or reject.

Once either DKIM or SPF have passed, and it can be both, DMARC will then take action based on the requested behavior of adkim or aspf.

For strict adherence:

  1. In all cases, the RFC5321:Mailfrom and the RFC5322:From must match exactly.
  2. If the adkim is set to strict then the d= entry must match exactly the RFC5322:From domain.
  3. If spf is set to strict then spf domain must exactly match the RFC5322:From domain.

For relaxed adherence:

  1. In all cases both RFC5321:Mailfrom and RF5322:From must share an organizational domain.
  2. For dkim relaxed the d= domain must share an organizational domain with the RFC5322:From domain.
  3. For spf relaxed the domain must share an organizational domain with the RFC5322:From domain.

We hope you find the information here of some use however, please note that this document is not meant to replace the information available on DMARC at these locations.

Enjoy!

Find out why businesses are turning to the DMARC email authentication standard when you get the free copy of the How DMARC Is Saving Email eBook today!

How DMARC Is Saving Email

Social media platforms rise and fall – through it all email remains a primary communication channel. Many “email killer apps” are rolling out features that integrate email notification in the engagement process.  In order to sign up to use such apps, the user is often required to have an email for authentication, identification and verification purposes.  In fact, most “email killer apps” still rely on email as a primary notification channel. Herein lies the irony… as well as the need for online brand reputation.

Sam Masiello Head of Application Security, Groupon highlighted the important role email and DMARC continue to play in online communications today in the Best Practices track during Interact 2013.

Sam Masiello Interact 2013

Email is a pre-requisite to signing up for any online accounts and the gateway to your online life — which is precisely why cybercriminals continue to target email to gain access your personal information. Many forms of email authentication exist, one of which is ADSP (Author Domain Signing Practices), an extension to DKIM.  None of these are perfect. ADSP for example, has key gaps with no reporting capability and visibility. It could only be used to set policy and was only adopted by one major provider as the standard – Google.

Enter DMARC

Unlike ADSP, DMARC enjoys the support of major ISPs and mailbox providers — one of the secrets to its success. DMARC celebrated its first birthday in January 2013, but was in development for two years before it was launched.

DMARC is an important part of the email ecosystem, and many global email providers and top senders of email have adopted DMARC. It protects 60% of global mail boxes or 1.9 billion mailboxes and 80% of consumer mail boxes. In Dec 2012, DMARC blocked 325 million messages.

Brands on the DMARC bandwagon include the following:

Brands with DMARC

Groupon, for example, sees phishing attacks every day on their brand, but DMARC is helping to defend against these attacks.  Cyber criminals are becoming increasingly sophisticated and it is getting harder to differentiate authentic emails. Sam illustrated this with an example:

Domain Spoofing

Think this is from Groupon? Think again. While it looks uncannily similar to Groupon’s daily deals, the address should set off warning bells.

Groupon currently has DMARC deployed in 9 countries. Blizzard is another brand that has successfully deployed DMARC to increase brand trust and reduce consumer trust erosion after large scale phishing attacks.

Why Do We Need DMARC?

Threats to email security emerged due to the impact of poor SMTP design. SMTP was developed to send a message from Point A to Point B with no thought to security.

Other email authentication tactics do exist, but DMARC’s advantage here is that it complements the existing email ecosystem. It sits on top of DKIM and SPF, allowing you to get better value out of email authentication.

  • SPF: Path-based authentication, authorized servers published via simple DNS record, low deployment cost
  • DKIM: Signature-based authentication, requires cryptographic operation by email gateways, public keys published via DNS, can survive auto-forwarding
  • DMARC: Leverages SPF and DKIM as authentication mechanisms, provides visibility, email senders set policy to declare how they want email receivers to process email that fails authentication

The Three D’s of DMARC

DMARC is for everyone, it’s easy to deploy and is not a deliverability tool.  DMARC is not able to stop all phishing attacks, however it solves the problem of direct domain spoofing. Authentication has been trending in the direction of giving senders and receivers control and visibility over messaging streams.

The business value of DMARC includes:

  • Increased user trust and loyalty in branded emails
  • Visibility into consistent application of best practices
  • Visibility into compliance assessment

The technical value of DMARC includes:

  • A way for ISPs and mailbox providers to confidently identify your mail
  • Support by major international ISPs and mailbox providers
  • Ability to set policy on how you want ISPs to handle illegitimate email
  • Feedback loop to understand where spoofed email is coming from and where your legitimate mail is failing authentication

Businesses do not have to authenticate all mail today to publish a DMARC record. DMARC protects your brand and it’s easy to get started. In the words of Sam,

DMARC deployment is a journey, not a destination.

To learn more about DMARC from Sam Masiello from Groupon, watch the Don’t Deprioritize DMARC webinar to discover the benefits of adopting DMARC email authentication!

Don't Deprioritize DMARC webinar

So you’re at dinner with Grandma enjoying some homemade apple pie when she pops the question. No, not when are you getting married, or when you are having kids, or when you are having grandkids, but what do you do at work? You panic. How do you explain something as complex as DKIM and email authentication to Grandma?

Never fear though. We at Message Systems have got your back! We have pre-empted this socially awkward situation with a great infographic on How To Explain DKIM To Your Grandmother. Heave a sigh of relief and read on!

DKIM Infographic Explain to your Grandmother

Now that you are an expert at DKIM, it’s time to learn about the email authentication standard, DMARC. Watch the Don’t Deprioritize DMARC webinar to find out why DMARC is so essential to businesses in preventing online fraud!

 

Weekly Email Marketing News Digest

We hone in on the debate surrounding confirmation opt-ins this week: When are they necessary and what percentage of marketers are deploying it in practice? Also on the agenda, tips for successful email personalization and most commonly followed email marketing best practices.

When COI Makes Absolutely No Sense

Andrew Cordek from Trendline Interactive challenges permission purists by questioning whether confirmed opt-in is necessary in certain situations. One such situation is where a user registers to access site information through the following process:

  1. Creating a username and password (with CAPTCHA)
  2. Filling out a long form with information
  3. Managing opt-ins for other email publications through a preference center (with cadence/frequency info)
  4. Confirming choices on another page
  5. Receipt of a welcome email which had all of the email choices listed

In such a scenario, Cordek argues that a closed loop email for subscribers to confirm their choices is unnecessary.

No, COI actually makes sense most of the time

And here’s the rebuttal by David Romerstein from LivingSocial. While Romerstein agrees that COI might not make sense in some scenarios, it certainly isn’t that one outlined by Cordek, for one reason: The detection of typos.

Confirmation emails help to eliminate typos, whether committed on purpose by a competitor trying to affect your reputation, or by accident. Minimizing typos helps in keeping one’s IP reputation clean.

Email Deliverability: Only 39% of marketers maintain an opt-in only subscriber list

In the MarketingSherpa 2013 Email Marketing Benchmark Report, the following question was put to marketers:

Q: Which of the following tactics is your organization using to improve email deliverability rates? Please select all that apply.

2013_MarketingSherpa_Email_Marketing_Benchmark_Survey_Methodology

The results showed that only 39% of marketers maintain an opt-in only subscriber list. Providing an easy unsubscribe process was the tactic that was most widely deployed by marketers at 62%. Of concern was the fact that only 21%of marketers used email authentication like DKIM or SPF to improve email deliverability.

The ultimate email marketing mistake

Here are three tips to make email personalization work for your business:

  • Use a valid reply email: A noreply@ address means you are missing out on opportunities to hear from your customer
  • Make sure you have the data: Skip personalization altogether if you don’t have many member names in your database
  • Avoid generic placeholders: Segment your list and send recipients without a listed name emails without a personalized greeting

73% of businesses carry out basic email segmentation: report

The new econsultancy email marketing census provides some interesting data for email marketers.

Top three email marketing practices are basic email segmentation at 73%, encouraging the sharing of content at 52% and regular list cleaning at 49%.

econsultancy_email_marketing_efforts

The most commonly used email trigger was automated response to website visit/sign-up emails at 35%.

econsultancy_automated_triggers

Abandoned basket automated email programmes provided the greatest return on investment according to 13% of companies surveyed.

econsultancy_automated_roi

Email systems have grown increasingly complex since they first appeared in the early 1970s. The growth of the Internet (as measured by users of internet-enabled services) is largely responsible for this complexity. Spam, phishing, and forgery attacks have plagued email users across the world in recent years.

The fundamental problem with modern email systems is that, although we ascribe identity to individual email addresses, that identity is generally not enforced. To this day, it is common to find poorly configured email servers and web email forms that allow malicious users to forge emails.

To understand how this is possible and why it is a problem, we need to understand how email works in the first place.

How does email work?

The internet protocol describing email (called SMTP: Simple Mail Transfer Protocol) functions very similarly to what we now know as “snail mail”. (It turns out that this similarity is very unfortunate). In snail mail, we have two major components: the envelope, and the letter.

The Envelope

The envelope contains addressing information: Who is this letter to? Who did the letter come from? The postal service uses the “To” address, printed on the front-center of the envelope, to get the letter to the right place. Sometimes, something is wrong with this address, and the “From” address (usually printed on the top-left or rear of the envelope) allows the postal service to return the letter to the sender.

If neither of these addresses make any sense, the letter will eventually be destroyed as it can neither be delivered or returned.

dkim-envelope

The Letter

A letter from a company to a legal office may not mention names of individuals on the envelope. However, the letter will usually have information embedded in it describing who the letter is individually to, and who individually is responsible for the contents of the letter.

dkim-letter-2

What Could Possibly Go Wrong?

The protocol for delivering letters via a postal service requires a large amount of trust. The example letter above is rather inflammatory, and we might have some doubts as to its validity. Consider the following questions:

  • Who is B. Baz?
  • Does B. Baz exist?
  • Did B. Baz write this letter?
  • Did Mr. Foo receive the mail, or did someone else?
  • Was the letter altered?

We could probably come up with many other questions related to this as well. Fundamentally, these questions boil down to trust. The postal service does not provide us any way to verify that the sender is who they claim to be. It does not provide us with any way to verify that the sender actually wrote the contents of the letter.

For years, people have sought solutions to these problems. We have invented opaque envelopes to prevent people from reading contents. We use better adhesives to prevent tampering (or at least make it obvious if the letter was tampered with).

With hundreds of years of experiencing mail forgery, delivery tampering, and the like, email must have considered these cases, right? Wrong. Email does not solve any of these problems. Why are they problems? Malicious individuals can exploit these issues of trust to send computer viruses or unwanted marketing email (spam). People exploit these issues of trust to send diseases like anthrax, and unwanted marketing mail (coupon books, credit offers, etc.)

Email and snail mail are not so different. But in email, we can do better.

Trusting An Identity Crisis

We have established two problems with email:

  • The identity of the sender and recipient of a mail are unknown.
  • The contents of the message are not known to be correct.

So, if we are the recipient of a letter (or of an email, for that matter), how do we know who sent it? How do we know what we are reading is what the sender actually wrote?

With regular mail, we rely heavily on context and signatures. Humans are generally pretty good at recognizing visual patterns (and discrepancies in those patterns). If we receive a letter from someone we know, we can make an educated guess as to whether they sent it based on factors like writing style, handwriting, signature, and address information.

Imagine receiving a letter from a friend. This letter is asking you for money. You know your friend is on vacation in the Galapagos Islands, and the signature looks funny. You are probably going to request additional information before you send money to your friend.

Email does not enjoy these same protections. There are no signatures — just addresses. And most people are not technologically inclined. So when King Mubotu emails from Nigeria telling you to send him $50 so he can send you $50,000,000, you do it. I mean, it is King Mubotu, right?! Right!!?

So many people are still unaware of this attack, that it is still successful. Again, the fundamental problem is that identity has not been established, and our normal contextual clues for establishing it are gone. So we fall back to trust. If you follow one rule on the Internet, it should be not to trust anyone unless you have any technical knowledge that would make you think otherwise.

Enter DKIM

DKIM (pronounced “DEE-kim”) allows organizations to claim responsibility for messages they send and guarantee the contents of the messages they send. Senders may also guarantee the identity of the individual responsible for sending the message (although this is not yet widely used).

How does the guarantee work?

In a word: math. DKIM relies on what is called “asymmetric cryptography” (also known as “public-key cryptography”). After a message is received, and before that message is delivered to its destination, DKIM uses a “private key” to create a signature for both the contents of the message, as well as some key headers of the message. This signature is attached to the message.

When the message is delivered to the destination, the destination server asks the sender for a public key to verify that the signature is correct. If the public key allows the destination server to decrypt the supplied signature to the same value it computes as the signature, it can assume the sender is indeed who they claim to be.

(Note that we can only assume that the sender is not lying about who originated the email. If the malicious party is the sender in the first place, we have to trust that they are not lying to us!)

As an analogy, consider a message sent between two corporate offices. The message includes a reference number and a phone number. The receiving office can call the sending office, provide the reference number, and read the message. The sending office can verify that this is in fact correct.

What if someone else answers the phone?

In the hypothetical scenario of a letter sent between two offices with a reference number and a phone number to verify, it is logical to assume that a malicious person could attack the security of the system. The attacker would have to:

  1. modify the message,
  2. hijack the phone line,
  3. impersonate the sending office, and
  4. lie about the validity of the message.

This is a difficult attack to pull off. But can we pull off the same attack against a DKIM-signed message? Not really.

When DKIM asks for the public key, it either receives the public key or it receives something that is not the public key. If it receives the wrong key, decryption will fail, and the message will appear invalid. If the message appears invalid, we can assume some aspect of the system has been tampered with, and we can try more strict verification of the message (such as an in-person discussion). This case does not break the guarantee made by the DKIM signature, because it can still be validated with the proper key.

If the attacker responds with the valid public key, all they have done is prove that the message was sent unmodified.

But people have hacked DKIM!

To mount an attack against DKIM, you have to own or reconstruct the private key. This has happened before in, for example, the attack against Google in late 2012. In this case, Google used an insecure key size for their private key and the attacker was able to reconstruct the key. The solution? Use a bigger key! Although this seems like a weak argument, the justification for why this works is outside the scope of this article. I do not wish to attempt to explain the complexity of integer factorization in simple terms. (At least not in this article.)

For the time being, this is a fine solution.

Invalidating Keys

If a key is compromised (either because someone obtains the key by “hacking” the sender or because they were able to reconstruct it by brute force), one can simply change the key. The problem with doing this is that once a key is known to have been compromised, it is impossible to guarantee with certainty that a message has not been tampered with. There are two reasons for this:

  1. Inherently, once the key is compromised, someone can forge messages that would have validated with the key. With some clever tricks, it might be possible for an attacker to even forge dates to make it look like the message was sent when the key was valid.
  2. The public key is generally no longer available. Once the private key has changed, the public key must change as well. If both the public and private keys are subsequently lost or forgotten, it is nearly impossible to reconstruct them.

When doing forensics work with DKIM, if the original key was compromised, it is usually not difficult to prove that messages were not forged. Most people lack the skill to forge keys and emails, and the ability to forge dates requires access to both the sending and receiving machines.

Conclusion

Hopefully this is a useful, lucid, and digestable explanation about the security weaknesses inherent to email (and snail mail, for that matter). If you have additional questions, feel free to contact me either via comments or via email.

If you choose email, I hope you are using DKIM!