AB Testing Email

Rules for AB Testing Email

When businesses A/B test, they’re testing two web pages at the same time to see which one performs better. Similarly, when we’re AB testing email, we’re testing two emails against one another to a subset of our subscriber base to see which performs better so that we can send the winning email to the remainder of our list.

AB testing email can be one of the most profitable and insightful things a company can do to optimize conversions. You want to make sure you:

Do try and do this

  • Pick your variables (decide what you’re testing)
  • Pick what you want to measure (#opens, #clicks, open rate %, deliverability %, etc.)
  • Determine which recipients to send to
  • Select a time frame to run your test
  • Pick a winner

But your success is dependent on you not making one of these common mistakes in AB testing email:

Don’t make these mistakes

  1. Testing the wrong thing
    If you have a low email open rate and want to AB test to increase your open rate, then changing the layout, headlines or imagery within the email is not going to help. However, changing the subject line and time of day you send your email, will. Make sure you are testing the right thing.
  2. Measuring multiple items
    Measure one thing per test. This means if you want to test subject lines, then only swap out the subject line but keep everything else the same — including the layout, number of images, button placements, etc. If you want to test your layout, then keep the subject line and content the same, but change the layout. The more changed elements you introduce, the harder it will be to measure.
  3. Unlimited calls to action with different goals
    Try to limit your calls to action to the same goal. If you have multiple calls to action in one email going toward the same goal (same landing page, sign-up, buy, etc.) that’s fine. But when you mix CTAs or try to introduce multiple CTAs, it clouds the results. For example, if you want them to make a purchase, read a blog post and subscribe to your newsletter, did they not click on the action you were hoping for of “buy” because you mixed CTAs? The point of A/B Testing your email is to make sure that you can accurately measure the results. When you mix variables, it makes the statistical analysis that much harder.
  4. Do not constantly alter the “From Name”
    Figure out what your relationship is to your subscriber. Are they more familiar with your company name, a thought leader in your company — such as the CEO, a mascot, or product name? People are more likely to open emails from names or brands they recognize. If you constantly change the ‘from name’, you may start to see a decline in your email open rate.
  5. Sample size is too small
    You don’t want to draw your conclusion on a samples size that is too small. You can A/B test on any sample size greater than 100. A good rule of thumb to test is usually 20-30% of your subscriber list so that your confidence level is high. (Click for more on sample size and confidence levels.)
  1. Testing different content pieces
    Email is not a good place to test how content performs because it could be that one piece of content performed better simply because it was written better or the headline of the article was more appealing to that particular audience member at that particular time.

sell more with email webinar banner

We’re continuing to list your favorite email marketing best practices eReads in 2013 this week! And now, for your Top 5, you’ve chosen…

5. The New Communications Standard white paper

Which is better, email marketing or mobile marketing? How about a third option, a new communications standard, that combines the best of both worlds? While a standard response rate for marketing emails might be 6%, a cross-channel campaign that includes mobile messaging sees a dramatic response increase of up to 45%!

4. The State of Mobile Technology Adoption: 2013 white paper

Mobile technology for both hardware and software is experiencing a frenetic pace of change. Originally valued at $500, this Forrester report by Principal Analyst, Julie Ask, is now free for a limited time only. Arm yourself with the necessary knowledge to tackle a mobile-first future and stay ahead of your competitors. Grab it, before it flies off our virtual book shelves!

3. Improving Delivery & Reducing Costs Through Automation white paper

For everything you ever wanted to know about Momentum’s Adaptive Delivery®, this white paper is a must-read. Written by email expert Len Schneyder, find out why Adaptive Delivery, an industry first, is continuing to make the waves with its automated monitoring of bounces, complaints, and adjustments of connection rates and throughput.

2. How To Send Zillions of Email A Day guide

Ah… the question on the mind of every email marketer with a large customer database. Beyond a doubt, email has been proven to generate the best ROI over other marketing tactics like social or search, but how do you make sure a large email send succeeds? Aside from using Adaptive Delivery (shameless plug), find out what you can do to get into ISPs’ good books and optimize your deliverability rates!

1. Email Marketing Best Practices 101 eBook

From email novices to seasoned veterans, everyone can appreciate the value of an eBook that highlights email marketing best practices. Come to think of it, I think it’s time for me to brush up on some of the guidelines now…

And that’s all folks! We hope you’ve enjoyed our Top 10 countdown series of both blog posts and eReads in 2013. Come Jan 2015, we’ll be featuring your Top 10 choices again, so we’re counting on you to keep reading until then!

Weekly Email Marketing News Digest

We’ve often emphasized email best practices in our blog, but what we haven’t talked much about is how to go about implementing email marketing best practices when you have limited resources in terms of time and manpower. If you are planning email marketing for small businesses here are some tips to get started.

Are Best Practices Holding You Back?

Loren McDonald advocates a “starting practices” approach where marketers can launch email marketing campaigns while adhering to best practices incrementally. The three approaches he describes in further detail in this articles are:

  1. Figure out where you want to end up
  2. Work backwards to a starting point
  3. Keep “best in class” principles as you move to each phase

The starting best practices approach is based on the principle of launching and improving programs gradually and not ditching quality all together.

Something for the Little Guys

AWeber predicted that 85% of SMBs will increase their use of email in 2013. Through the use of firmographics, or business and organization-related categories such as executive title, type of business, geography etc, in business email lists, small firms will be able to improve targeting and personalization of emails.

The New Email Marketing Rules

ClickZ interviews email marketing guru Chad White about his book, Email Marketing Rules, where he outlines 108 email marketing best practices in a way that is accessible to business with little email marketing experience. In this interview, Chad discusses how email works even when it is done badly, and the potential for driving greater ROI with best practices, as well as email frequency. And of course, do read part 2 of his interview, How To Ride High In Email Marketing, to find out why Twitter is important in email marketing.

It Takes More than a Phone Call to Deliver Your Email

Here’s a fiery piece from Ken Magill, where he dispels the widespread notion of a personal telephone hotline to inbox providers to resolve email deliverability issues. As the volume of emails sent places a financial burden on the inbox provider, these providers make it a point to filter all emails so that subscribers, who represent revenue to the inbox providers don’t leave.

Our very own experienced postmaster, Jill Resnick has spoken on the topic of how it is absolutely possible to have high email deliverability without a hotline to an inbox provider, in our Proven Tips For High Volume Sending webinar.

How Digital Marketing Has Changed Small Businesses [Infographic]

How Digital Marketing Has Changed Small Businesses

Want to learn more about email marketing? Check out our Email Marketing Best Practices 101 eBook!

Email Best Practices eBook

Email’s Image Issues

How many of you eagerly check your email every 5 minutes waiting for the promotional emails of brands to reach your inbox. Show of hands. Anyone? How often do you say, “WOW, fantastic email generic brand ABC! I can’t wait to see more from you!” The sad truth is, while there are brands that can stun and surprise us with awesome email marketing campaigns, it doesn’t really happen often enough.

Email has an image problem and it all has to do with brands sending us generic marketing emails with content that are not relevant to our interests. Experts in the email industry share their thoughts on what can make email better.

Making Email Delightful

Email is suffering from an image crisis, where consumers feel that they have lost control over a permission-based channel. Mike May offers some suggestions to brands on making the email experience a delight for the consumers.

  • Anticipation: Pre-empt customer requests for information and give it them before they ask for it.
  • Selflessness: Find a way to make the subscriber’s day a bit better and reward them for being a reader.
  • Surprise: Surprise your reader!
  • Personality: Ensure that your messages convey personality.

Hey, as Steve Jobs once said, “You can’t just ask customers what they want and then try to give that to them. By the time you get it built, they’ll want something new.”

The Future Of Email Marketing, Social Media, And Other Interviews

Our friend, Chris Penn, and VP of Marketing Technology at SHIFT Communications is interviewed by Salesforce.com Marketing Cloud, on email’s image issue, the role it plays in the marketing mix and its forseeable future. Chris has often wowed us with his depth of knowledge when it comes to marketing technology so be sure to watch the interviews!

Ten common email marketing gaps and errors

How well do you manage your email marketing? Independent email consultant, Tim Watson highlights 10 common mistakes. 5 & 9 in particular are definitely factors that lead to recipients hitting the spam button, so avoid these mistakes!

  • Mistake 1. Not setting long-term Email-specific goals.
  • Mistake 2. Not investing in list growth.
  • Mistake 3. Failing to track email signup sources.
  • Mistake 4. Not using all customer touchpoints for signup.
  • Mistake 5. Lack of a clear email value proposition.
  • Mistake 6. Not investing time to setup triggered messaging.
  • Mistake 7. Not Testing targeting.
  • Mistake 8. Frequency of send not optimised.
  • Mistake 9. Offer unclear up front.
  • Mistake 10. Lack of measurement beyond the click.

Email Like A pro: 20 Killer Lifecycle Email Campaigns from 2013

Chris Hexton, from Vero provides 20 examples of great lifecycle emails. Here are some examples:

The sense of urgency email

The Sense of Urgency Email - Email Marketing

The up sell email

Upselling Email - Email Marketing

The education email

Educational Email - Email Marketing

[Infographic] Email Marketing Best Practices

Want to make sure that your marketing emails are getting into your customer’s inbox? Check out our Email Marketing Best Practices 101 eBook!

Email Best Practices eBook

Weekly Email Marketing News Digest

We hone in on the debate surrounding confirmation opt-ins this week: When are they necessary and what percentage of marketers are deploying it in practice? Also on the agenda, tips for successful email personalization and most commonly followed email marketing best practices.

When COI Makes Absolutely No Sense

Andrew Cordek from Trendline Interactive challenges permission purists by questioning whether confirmed opt-in is necessary in certain situations. One such situation is where a user registers to access site information through the following process:

  1. Creating a username and password (with CAPTCHA)
  2. Filling out a long form with information
  3. Managing opt-ins for other email publications through a preference center (with cadence/frequency info)
  4. Confirming choices on another page
  5. Receipt of a welcome email which had all of the email choices listed

In such a scenario, Cordek argues that a closed loop email for subscribers to confirm their choices is unnecessary.

No, COI actually makes sense most of the time

And here’s the rebuttal by David Romerstein from LivingSocial. While Romerstein agrees that COI might not make sense in some scenarios, it certainly isn’t that one outlined by Cordek, for one reason: The detection of typos.

Confirmation emails help to eliminate typos, whether committed on purpose by a competitor trying to affect your reputation, or by accident. Minimizing typos helps in keeping one’s IP reputation clean.

Email Deliverability: Only 39% of marketers maintain an opt-in only subscriber list

In the MarketingSherpa 2013 Email Marketing Benchmark Report, the following question was put to marketers:

Q: Which of the following tactics is your organization using to improve email deliverability rates? Please select all that apply.


The results showed that only 39% of marketers maintain an opt-in only subscriber list. Providing an easy unsubscribe process was the tactic that was most widely deployed by marketers at 62%. Of concern was the fact that only 21%of marketers used email authentication like DKIM or SPF to improve email deliverability.

The ultimate email marketing mistake

Here are three tips to make email personalization work for your business:

  • Use a valid reply email: A [email protected] address means you are missing out on opportunities to hear from your customer
  • Make sure you have the data: Skip personalization altogether if you don’t have many member names in your database
  • Avoid generic placeholders: Segment your list and send recipients without a listed name emails without a personalized greeting

73% of businesses carry out basic email segmentation: report

The new econsultancy email marketing census provides some interesting data for email marketers.

Top three email marketing practices are basic email segmentation at 73%, encouraging the sharing of content at 52% and regular list cleaning at 49%.


The most commonly used email trigger was automated response to website visit/sign-up emails at 35%.


Abandoned basket automated email programmes provided the greatest return on investment according to 13% of companies surveyed.


Weekly Email Marketing News Digest

DDoS attacks continue to be the center of media spotlight.

We’re showcasing more of the controversy around Spamhaus in this week’s digest, together with a mish-mash of actionable email marketing best practices that boost conversion (eg. subject lines, autoresponders, social sharing). Pretty much something to pique everyone’s interest!

Global internet slows after ‘biggest attack in history’

Spamhaus has been under siege for well over a week and this issue was thrown into international spotlight after the BBC picked this news nugget up with the controversial headline that it was slowing the “global internet”.

While this might have caused initial panic among limited circles and networks, it was pretty much business as usual for most people. Gizmodo has a detailed account of the Internet Apocalypse – that wasn’t.

What is a fact is that this is the biggest DDoS attack in Internet history with attacks peaking at 300Gbps (gigabits per second) versus the usual 50Gbps against banks.

The record for the biggest DDoS attack was previously held in 2010 at 100 Gbps.

Study: Nearly Two-Thirds of Companies Do Not Use Autoresponders

A study on 500 companies from the Inc. 5,000 list has revealed that nearly 25% of companies do not offer an online contact form. Other startling statistics:

  • Only 37% of companies use autoresponders and send follow-up emails after an online form submission.
  • Of companies that do use an autoresponder, 78% send follow-up emails within one hour of a form submission

Several best practices recommended by the study:

  • Follow up form submissions with a timely personalized email
  • Having an unsubscribe option
  • Having a call-to-action

Six email deliverability lessons that you can learn from SEO

Search engines do not index emails, but there are still some lessons from SEO that email marketers can learn.

  1. Keep Coding Simple – A majority of clients are not able to load emails laden with scripts.
  2. Use Pre Headers – This provides a meta description of your email
  3. Find a good balance between Images and Text – Using too many images can lead to delivery issues
  4. Choose Keywords Wisely – Certain words are used to determine if the sender is a spammer hence monitor deliverability when using words like “free”
  5. Be Recognisable – Ensure email authentication processes are properly set up for a good sender reputation
  6. Be Relevant – The more relevant your content is to a recipient the less likely they are to “mark as spam”

Case Study: Creativity vs. clarity in email subject lines

I’ve often wondered about this myself and the AWeber team provides the answer. Emails with clear subject lines had:

  • 1,107% more comments
  • 315% more tweets
  • 331% more Facebook likes
  • 617% more traffic
  • 366% more email subscriptions

On average, each channel had 541% more response. For a detailed analysis on why detailed emails work better do read the full article.

Social Sharing Boosts Email CTR By 158% [REPORT + INFOGRAPHIC]

61% more users have social sharing buttons in their emails than last year. That’s good news for the industry as emails with social sharing buttons had 158% more clickthrough rate than those that did not. The average email CTR is 2.4% without social sharing and 6.2% with social sharing.


On this topic of social sharing, if you like what you are reading, why not sign up for our newsletter? And if you like our newsletter, we’d appreciate a social share!

Weekly Email Marketing News Digest

Devastating news for bloggers and journalists this week – Google is shutting down Google Reader. If you’re like me and your livelihood depends on perusing Google Reader’s news feed, you’re probably scrambling to find a replacement (aside from whining incessantly and sobbing uncontrollably on the floor). Here’s a useful read of alternative resources from LifeHacker, Google Reader Is Shutting Down; Here Are the Best Alternatives.

Moving on to email marketing news, we’ve collated a slew of articles with actionable tips for marketers to begin optimizing content and using tactics that could see an immediate improvement in ROI. Enjoy!

Case Study: 30 Percent Increase in Email List Growth

The above-the-fold camp may be taking a bit of a hit these days, but here’s an important thought: “Just because people can scroll doesn’t mean that they will”. Just by moving the email sign-up above the fold, a business saw a 30% increase in email list growth.

Email CTA Below-The-Fold


Email CTA Above-The-Fold


Why Nobody Is Talking About Your Email Content… and How to Change That

The problem is that it screams me, me and ME! Here are 7 reasons why email marketing campaigns fail:

1. They’re all about you.
2. They’re not helpful.
3. They’re not timely.
4. They’re not entertaining.
5. You don’t ask questions.
6. You don’t involve your readers.
7. You don’t tell your readers to share.

What will kill email?

Digital marketing guru, Christopher Penn argues that email will likely never die – unlike social networks with rapid birth-death cycles. IMAP, POP3 and SMTP are all open standard protocols so it’s easy for anyone to buy and set up a standards-compliant mail server. Not so for social media, where business models are based on exclusive ownership.

Most Emails Deployed in the Morning – But Best Results Seen in the Evening

I start off my mornings scrolling through a slew of marketing emails. If I see something interesting, I flag some for follow-up in the evening when I have some time to register, sign-up or purchase. Apparently, that’s possibly a behavior that is reflective of the average email consumer.

A study shows that a 40% majority of emails are sent between 8am to 11.59am, but the 16.1% unique open rate, 2.4% unique click rate and 0.13% transaction rate were the lowest among the tested time periods.

Between 8pm and 11.59pm, the 21.7% unique open rate, 4.2% unique click rate, 0.34% transaction rate, $0.48 revenue per email and $246 average order value were significantly higher than the other tested time periods. Days with the lowest email volumes such as Saturday and Sunday had the best response rate at a unique open rate of 17.8% and unique click rate of 2.9%.


3 ways to apply image psychology to your next email

Here’s a different way to approach email design – through the lens of neuroscience.

1. Use images of people in email campaigns that evoke emotions you want subscribers to have.


How it works: A part of the brain is wired to solely to process images of faces.

2. Use images of people creatively to support your call to action.


How it works: Our brains are naturally programmed to pay attention to people.

3. Use provocative images that support people’s survival instinct.


How it works: A part of the brain known as “the primitive brain” pays attention to food, danger or sex – things that ensure the survival of the species.

Weekly Email Marketing News Digest

In this week’s news, consumer dependence on email is seen through mobile behavior as determined by two separate studies. We also take a look at the missing link in big data strategy and  email marketing best practices that every marketer should take note of before launching a campaign. Oh, and DMARC continues to be in the spotlight! Now that the preview’s over let’s dive in!

Why (And How) Marketers Should Follow DMARC

The DMARC standard now protects almost two-thirds of the world’s 3.3 billion consumer mailboxes worldwide, and was responsible for blocking 325 million unauthenticated messages in November and December 2012 alone. Message Systems Chief Revenue Officer Ralph Lentz dives into the reasons why it’s going to be the industry standard going forward.

Big Data’s Big Omission

Customer identity management is crucial to big data strategy. Customer identity and relationship management is built upon the foundation from three pieces of data: email addresses, postal addresses, and phone numbers. While cookies are important, they are not the be-all and end-all in big data. The crux?

When it comes to big data strategy, email is a necessary component.

85% of Smartphone Users Would Rather Give up Water Than Mobile Apps

Give up mobile apps? Not a chance. 82% of respondents in a mobile app survey say there are critical apps they can’t go without — not even for one day. Those include email (57%), Facebook (41%) and alarm clock apps (31%).


7 Don’ts for Email Marketers

You can’t teach an old dog new tricks. But you can teach email marketers to optimize their campaigns according to these best practice guidelines:

  • Don’t forget the channel changers. Help your customers send information to another device, set reminders, or use email content while in your store. Cross-channel messaging anyone?
  • Don’t drop the data. Collect device usage data and develop a plan to associate opens, clicks, and conversions to specific mobile devices.
  • Don’t write a book: You can’t expect subscribers to consume massive amounts of text on their mobile devices.
  • Don’t overlook transactional messages: Order and shipping confirmations are sent during a peak engagement period of the customer life cycle. These are highly personal. Consumers are excited over their purchases and open these emails.
  • Don’t short the shortcuts: Test your messages to confirm that mobile subscribers can easily click through to track their orders, call customer service, or get directions to your nearest store.
  • Don’t put prices in pictures: Putting essential information such as pricing, deadlines, and calls-to-action inside images is a big mistake in the mobile age.
  • Don’t say the word ‘blast’ in regard to email marketing.

68% of people use their smartphone for email, 26% for shopping

Want to see how smartphone usage in the US and UK differs? The Nielsen Mobile Consumer Survey has the answer.

In US, the percentage of people who use their phones for:

  • Text – 86%
  • Mobile web – 82%
  • Email – 75%
  • Social networking – 63%

In UK, the percentage of people who use their phones for:

  • Text – 92%
  • Mobile web – 66%
  • Email – 75%
  • Social networking – 63%

As you can see the biggest gap lies in the use of mobile web, where common use is higher by 16% in the US. Text messaging in UK is higher than in the US by 6%.

nielsen-devices-we-use     nielsen-apps-we-use

What do you use your smartphone for? Let us know in the comments below! And if your business is embarking upon a mobile-centric drive to attract new customers, we’d encourage you to have a look at our white paper on mobile messaging – it brings up points of consideration to make your mobile marketing strategy a success.

The New Communications Standard

Just over a year ago a group of the Internet’s biggest companies announced the Domain-based Message Authentication, Reporting and Conformance (DMARC) working group, an industry-based approach to combatting spam, phishing and other forms of messaging abuse. And on the one-year anniversary last week, DMARC.org announced a rather remarkable accomplishment: the DMARC standard now protects almost two-thirds of the world’s 3.3 billion consumer mailboxes worldwide. The DMARC announcement also noted that the new standard was responsible for blocking 325 million unauthenticated messages in November and December 2012 alone.

Here at Message Systems, we’ve always made it a point to place email security and marketing best practices at the center of all that we do. It’s why we were one of the first email technology providers to throw our support behind DMARC, and why we continue working to help Message Systems customers get up to speed with authentication best practices in general and DMARC specifically. There are lots of great information resources for getting up to speed on implementing DMARC. Here’s a few from the Message Systems community:


Our partner, ReturnPath has created a 101 Guide on creating a DKIM record, Protecting Your Brand From Phishing: How to Create a DKIM Record. Here is a summary of the steps to be taken:

  1. Inventory all of your sending domains.
  2. Install and configure DKIM on your email server.
  3. Create a public and private key pair. 
  4. Publish your public key.
  5. Store your private key.
  6. Configure your email server
  7. Test


As part of that same series, ReturnPath also has a guide to SPF email authentication, Protecting Your Brand From Phishing: How to create your SPF record. Here are the 4 steps you have to take:

  1. Determine the domains that your email campaigns are sent from
  2. Gather the IP addresses that are used to send the emails
  3. Create your SPF record
  4. Publish your SPF to DNS

These steps are covered in a lot more detail in the entries above, so we’d recommend hopping over to check those entries out.


Additionally, Franck Martin, Postmaster at LinkedIn and a long-time friend of Message Systems, has developed a set of scripts for implementing DMARC on the Momentum platform. He’s made these scripts available online at GitHub. They provide an elegant solution for filtering incoming emails and rejecting those that fail DMARC. Franck has also developed other tools and scripts for managing and monitoring a Message Systems Momentum cluster.

Thanks Franck for these great resources!

If you’d like to find out more about DMARC, download the How DMARC Is Saving Email eBook!

How DMARC Is Saving Email