A few days ago, my colleague Dan looked at how email headers can tell you a lot about where a message came from, the path it took to an inbox, and even whether it actually was sent by who it claims to be from.

But the information contained in headers really is just a start. There’s more that can be measured about the journey of an email—and how well it’s working at the job you’re giving it.

Understanding Product Email Performance

Let’s look at a typical SaaS application email. What’s the basic reason your app is sending an email? It’s because your user has done something, and you need them to take an action to complete the task.

Account signup is a common example:

  1. A user signs up for an account
  2. You need the user to confirm her or his email address to complete the signup

Sounds pretty straightforward, right? Certainly, making sure that process works is essential to keeping your user happy (and not churning). Whether you’re a product manager or on the developer and devops side, making sure emails like this are working is an important part of measuring and managing your product’s overall performance.

Behind the scenes, though, the path that email takes is not as simple as it might seem. Most teams building SaaS products don’t have visibility into what happens once an email like this account confirmation message is sent.

With this article, I’m beginning a series that looks at the life of an account confirmation email, the stages it passes through on its way to the user’s inbox, and how data can be used to understand its performance at each stage.

Generating and Submitting a Product Email

The first step in our message’s journey is the generation and initial submission of the message. Depending on your application, this could happen in one of several ways:

  • Your app generates a fully-formed email and then submits it via Simple Mail Transfer Protocol (SMTP) to an external mail server for transmission.
  • Your app generates the content of a message and uses a simple mail sending function with a local SMTP server on your application host.
  • Or you can use an API provided by a dedicated email delivery service (in shorthand, an email API). This approach is the most scalable, as it allows your app to submit just the relevant data like name and address, while the specialized service handles message generation and delivery.

In any case, the email message is submitted to a Message Transfer Agent (MTA)—i.e., a mail server—that queues the message and takes responsibility for its delivery to the next hop in your message’s journey.

Why Latency Is a Key Metric for Product Emails

Measuring performance at this stage focuses on two metrics, injection count and latency.

The injection count is how many messages you submitted (injected) to the mail server for a given period of time. By itself this count won’t mean much, but it’s a good baseline against which to measure other data.

On the other hand, latency is a critical metric; when you send messages such as signup confirmations, your user’s experience depends on the timely delivery of the message. That’s especially important for something like an account confirmation email—the longer the message is delayed, the lower the likelihood the user will complete the registration and use your product.

If your message assembly and submission system has latency, you may want to review your code to see if it can be made more efficient, in addition, using an email API service can go a long ways towards improving performance, as it offloads the message generation from your own systems.

Typically, an app will only need to gather the recipient name, email, and other small amounts of personalization data and submit it to the API. The service then takes care of assembling and personalizing the messages and queuing them for delivery.

How to Measure Email Delivery Latency

Another component of email latency is how long it takes for your mail server (MTA) to deliver the message to its next stop, typically the inbound mail server operated by your user’s ISP. (Gmail or iCloud are typical examples.)

At this stage, some factors are out of your control, but latency here is a good indication of the overall quality of your email infrastructure. It’s a function of your infrastructure’s technical architecture and performance as well as its ability to manage delivery to ISPs. Both of these areas are where a high-performing email delivery service typically exhibits a major advantage over in-house email systems.

If your systems are slow at processing messages, or at managing queues, or are simply overworked, you’ll see latency of a minute or more. Efficient systems should be operating with a latency of less than ten seconds on average. There will always be some variation, as is common with all complex systems, but it’s the long-term trends that should concern you.

When you see latency numbers that are consistently high, you need to look at the underlying infrastructure: are there too many systems in the middle of your sending? Even your message generation process could be at fault: pickup directory systems can be relatively straightforward to build, but they are often based on scheduled pickups, which can introduce a long wait for a message submitted just after the last pass of the reader.

Latency after the first attempt is an indication of your reputation with the ISPs. Often an ISP will reject your message on the first attempt as a form of greylisting, refusing the first attempt at delivery to see if your MTA makes a second attempt (many spam bots will try once and move on). This causes the message to be queued and deferred temporarily for a subsequent attempt. The longer it takes to deliver a multi-attempt message, the more likely your sending reputation needs improvement.

How to Measure Latency by Hand

If you don’t have analytics in a dashboard for latency data, you’ll have to gather it yourself. Setting up scheduled jobs that send messages through your infrastructure to dedicated mailboxes that you maintain on the major ISPs, then retrieve the messages and check the timestamps of the first Received header and the Date header to determine the latency (note in the example below the time zones must be accounted for when calculating latency).

Email latency is a very important part of product email performance, but it’s just the start. Stay tuned for our next installment: understanding message delivery and what happens when a product email is delayed or never arrives.

—Mike

I don’t have to tell you that every product team wants users to be successful with their application. That notion of user success is a defining characteristic of nearly every quality we try to deliver: a compelling value proposition, highly functional workflow and features, and a delightful UX.

But that’s rarely enough. “Build it, and they will come” is never a reliable strategy—but for SaaS applications that face the constant risk of customer churn, a disengaged approach to user behavior is a big mistake.

So what’s the alternative? Last week, I shared several examples of effective email notifications from Pinterest and LinkedIn. Now, I’m going to dig further into how other SaaS businesses can make use of email in their products. They’re some of the most powerful tools product management teams have to drive conversion, retention, and growth.

Identify the actions that drive engagement and retention

But where to begin? Whether your SaaS product is B2B or B2C, there are some key moments in your user lifecycle that are essential to their (and your) success. These points might be specifically about user activation and conversion, or reflect potential roadblocks to renewal. But in each case, the more we provide users with the essential information they need to succeed, the more likely they are to take the actions that lead to more engagement and higher renewal rates.

Consider onboarding. As soon as your users sign up, you need to help them hit the ground running. That means making it easy to connect to your value proposition, discover the most important features, and take key configuration steps like confirming an email address or setting up two-factor authentication.

But onboarding is just one example. There are many additional times when alerting a user to take action in the app is essential to their engagement and success. Think about the user lifecycle in your own application. Perhaps a team member needs to assign a task to a colleague. Or a manager needs to approve a scheduled time-off request. Or a user can’t proceed until she acts upon a security notification or password reset.

Product Emails in the User Lifecycle

Situations like these can be make-or-break moments for your service. If the user takes the appropriate step, they’re more likely to integrate your app more deeply in their workflow. But if they miss the alert and work around the obstacle in some other way, they’re more likely to ignore your app… and eventually give up on it and churn.

Essential use cases for product emails 

Product email notifications aren’t generic. They should reflect the unique qualities of the apps and services that send them. However, there are several types of emails that nearly every SaaS app should be sending.

Activation emails: Account activation emails serve two purposes. First, they verify that the email address the user provided is valid and working. That’s essential for future messages. Second, they remind busy users that they chose to sign up for your product—believe it or not, that simple step sometimes makes the difference between an engaged user and a drive-by signup.

Trello Account Confirmation Product Email

Welcome and onboarding messages: Once a user has taken that first step of creating an account, the next logical piece of product email is to help your user actually hit the ground running. But don’t be tempted to provide a lengthy tutorial! Feature-driven onboarding messages are far less successful than emails that reconnect specific user activities to your core value proposition.

Eero Welcome Product Email

User invites and shares: Sharing content and inviting friends to try an app is a viral sort of product email often is associated with B2C social platforms. But the utility of this sort of message applies to almost every sort of SaaS app. Great examples include explicit invitations to colleagues to join a project team as well as implicit invitations in the form of workflow notices that require signups by users that haven’t yet joined a platform.

Asana Invitation Product Email

Activity notifications: For many SaaS products, activity and status notifications are the primary way offline users are drawn back into an app. Whether it’s a reminder to complete a task or a summary of missed highlights in a social media feed, these product emails help passive users reconnect to the problem your product addresses, even if they’re not logged in.

Lattice Activity Product Email

Reports and dashboards: For B2B SaaS products, reports on workflows are very tangible artifacts of the value of your service. Examples might include a detailed summary of a sales pipeline or a high-level overview of team members’ activities during the past week. Information like this is critical to your customers’ business. That means it’s also an essential email your product should deliver.

DocSend Report/Dashboard Product Email

Password resets and 2FA: Password resets are essential for any app. Email notifications also support related features like two-factor authentication. It’s hard to imagine more literal examples of product emails that either make or break user success—an undelivered product email of this sort is a showstopper.

Apple ID Password Reset Product Email

Security and account change notifications: Immediately providing updates to the user when their account information or login details have been changed is an important part of building trust and confidence in a SaaS product. These messages, as well as alerts about logins from new locations, also are crucial for helping users protect their accounts—and therefore your app!—from possible fraudulent use.

LinkedIn Account Change Product Email

How you implement product emails matters

I’d wager your app probably already relies on at least some of these types of product-generated emails, like account activations and password resets. How they’re implemented varies widely, though.

I’ve seen examples of basic messages hard-coded or otherwise treated in an ad-hoc fashion. But SaaS products that really handle their product emails right have generalized into a function to send notifications of any stripe. That gives product teams a lot more flexibility to drive user engagement and activity.

There’s another reason to think carefully about how to manage product-generated email: these messages only work if they’re delivered on-time and to your user’s inbox. Doing product emails the right way ensures that the complexity of handling email doesn’t become a problem that negatively affects your users.

Just think about the impact of three common examples of app-generated email that have gone missing:

  • If a sign-up confirmation is delayed or gets lost in the spam folder, you run the risk that your user won’t even complete the signup process, churning before you even have a chance to show them the value your application provides.
  • Delayed or lost password resets and two-factor authentication (2FA) messages can lock your users out of your application completely, resulting in show-stopping frustration and churn.
  • Even missing notices like status changes or receipts can lead to increased support costs and a gradual erosion of trust and engagement.

This kind of negative impact is why customers churn when messages are delayed or lost in spam folders.

Sending an email is simple, but getting it delivered on-time and to the inbox is not. Moreover, if product teams don’t have insight to what email is arriving (or opened, or clicked), they’re left with little visibility into the impact of these key drivers of user retention, conversion, and churn.

Mitigating that risk is an important part of what product teams (and their development colleagues) should think about when they implement app-generated email. Not sure where to begin? Our Email Deliverability Guide is a great resource.

—Amie

 

As a user of modern applications and services, you’ve almost certainly interacted with a variety of email notifications. These messages alert you when your post was shared on a social network, remind you to take a key step in activating your account for a productivity tool, or ask you to approve a scheduled bill payment from your bank.

Email notifications like these draw users back into apps and reinforce trust in services. They’re an important part of a great user experience and one of the most powerful tools product management teams have to drive conversion, retention, and growth.

As VP of Product at SparkPost, I’ve had the privilege of working with best-in-class companies such as Pinterest, Intercom, HubSpot, and LinkedIn. They use email notifications to build user engagement and drive business metrics like conversion and retention. These emails offer great lessons for teams building both B2C and B2B products.

Common use cases for email notifications

Although examples of email notifications are as varied as the apps and cloud services that send them, many uses cases apply to nearly any service.

Security and account changes: providing updates direct to the user when their account information and login details might be at risk is a trusted and strategic way of using email notifications.

Consider this email notification sent by LinkedIn when a new email address is added to an account.

It is direct, factual, and provides clear action steps when required. It employs both detailed information and cues like a security-specific return address to reinforce trust, an essential quality for services like this.

Information that prompts user action: well-targeted notifications to complete onboarding or to take other specific actions are key to increasing metrics such as activation and conversion rates.

Pinterest sends a series of emails to help new users get started using the product, including the following:

It’s effective in two ways. Most directly, it prompts a new user to take an action that’s key to becoming an engaged user of the service. But it also uses design and messaging elements that broadly reinforce qualities users value about Pinterest: a personal voice, striking photography, and what they “love to do.”

Here’s an email notification from LinkedIn to add a new connection to a user’s network of contacts.

This notification works because it prompts an action tied to the service’s core benefit, professional networking. Qualities like personalization and a direct subject line make it more likely to be opened.

Information and status updates: notices about activity that happened on a site while the user was away reminds him or her of a service’s value and can drive re-engagement—or even conversion for additional services.

Notifications like these often psychologically reward a user for their use of the product. This notice from Pinterest that a user’s content was shared is a good example. It provides multiple opportunities for engagement by highlighting the item that was shared as well as providing additional content for the user to explore.

LinkedIn uses a similar sort of notification to drive engagement when a user’s profile was viewed.

This example takes advantage of a user’s natural tendency to want to learn more as an opportunity for conversion on a premium service offering that shows more detail about who viewed her or his profile.

Designing an effective notifications strategy

These effective examples reveal several best practices that other apps and services can leverage to benefit from email notifications. If you haven’t begun thinking about email notifications as a core aspect of your app, now’s the time. Here are some questions a product team should ask to get started.

  • What specific user actions in your app increase conversions (or decrease churn)? These areas are where notifications prompting action will give product teams the most leverage.
  • What kinds of data increase the value your users perceive (or even bring them joy)? Notifications of this sort are a natural way to increase engagement and frequency of use, and drive conversion and upsell.
  • What information reinforces users’ trust and confidence in your service? Account and security alerts are essential notifications that every service needs.

Why email notifications overachieve

Email is accessible on every piece of technology that we own. From phones to computers, voice recognition devices to smart watches, email is there. Email has permanency, with many of us keeping important receipts, confirmation messages and notifications that we may want to refer back to in our inboxes. As a result, email conveys a level of legitimacy that is crucial when reinforcing confidence and trust.

The immediacy and relevance of notifications help these messages stand out from the rest of the inbox. At the same time, email’s performance, searchability, and permanence characteristics can make them more effective and reliable than a push notification, particularly when updating customers of important changes to things like account information.

Email notifications offer high-value functionality: delivering information users need to take action or to reinforce the value and trust they see in a service. As these examples from Pinterest and LinkedIn have shown, they’re also a key tool for drive conversion, retention, and growth.

—Amie

best practices customer onboarding

This is going to sound familiar to many product managers. And founders. And product marketers, developers, and….

You spotted a market opportunity, figured out what your customers need, built your product, did some marketing or other promotion. Or, to put it more bluntly, you spent time, you spent money, and you spent a huge dose of blood, sweat, and tears. And—amazingly—you find that people are downloading your app or signing up for an account.

Woot! What a good feeling. But then you realize they’re trying it once and not coming back. Or your conversion rates are uncomfortably low. Now that feeling’s a sinking one. What’s going on?

It might be all about first impressions. And I don’t mean just the bling. Getting started—effecting change to entrenched habits or processes—often is the hardest part.

The first few interactions a customer has with an app or a cloud service—the onboarding experience—arguably make up the most important chance you have to win a customer’s buy-in and engagement. Onboarding profoundly influences a customer’s views of a service, and it can make or break an entire customer relationship.

The discipline of user experience design (UX) in software is focused on usability, affective and emotional aspects of the product, making certain desired user activities intuitive, and so on. Designing the onboarding experience is a big part of that. In-app cues, incentives such as gamification, and triggered emails are all major drivers of the onboarding flow.

Most of these best practices apply to both B2C and B2B contexts. But what happens when requirements for getting started go beyond a user’s interactions with an app? For B2B services like SparkPost that power real-world business processes or that integrate with other business systems, onboarding encompasses broader considerations than getting through a series of steps in the app.

At SparkPost’s recent Insight user conference, our own Clea Moore was joined by a panel of customers who discussed their experiences with onboarding in the enterprise context. These professionals shared their (sometimes hard-won) advice about how to hit the ground running when moving business processes to a cloud-based platform. The discussion was wide-ranging and delivered a lot of insight into how to make migrating to the cloud a reality.

Here are some of the take-aways:

  • The cloud allows the businesses to focus on their strategic differentiators rather than managing commodity infrastructure. On this, the panelists were whole-heartedly in agreement. Seth Weisfeld of Pinterest observed that “cloud infrastructure is a strategic choice for Pinterest. It allows us to focus on our real value in content and experience.” Jonathan To of fashion retailer Tobi agreed, “To be able to focus on our customers and their engagement rather than infrastructure is just so great and so important to our business.”
  • Scalability and elasticity are big wins from an technical operations perspective. Travis Wetherbee noted, “I don’t have to worry about adding IPs, adding boxes, adding drive space to deal with peak volumes or to store and bounces.” Jonathan To added, “We’re not a huge company. We’re trying to stay lean. The cloud is a major win for time to market.”
  • Specialized expertise—for example, SparkPost’s deliverability services team—makes a huge difference among cloud providers. For mission-critical processes, cloud providers have got to back their technology with hands-on account management and real operational and onboarding expertise. Seth Weisfeld described, “It’s really huge to be able to trust our provider on issues like deliverability. We couldn’t always rely on that in the past.” Travis Wetherbee concurred, “Beyond the pure technology evaluation, services like deliverability expertise was a big criterion in our decision-making.”
  • Planning ahead makes all the difference for minimizing the risks of unexpected impacts or disruption. That includes technical legwork—Travis Wetherbee called out making sure DNS changes and suppression lists were managed systematically—as well managing the transition from a business perspective. Jonathan To added that “Thinking about the data you want to store means looking ahead and making conscious choices—even if you can’t use it today.” Seth Weisfeld described the importance of looking at the migration as a process, not something that can be done in one fell swoop. His advice was to begin with small, less critical mail streams and then gradually ramp up to the most strategic pieces as the system is proved out.

Learning from customers always has been the most rewarding part of my job. And, of course, getting information about how people actually use technology to solve problems in the real world is essential to every software or cloud product marketer. So, I was really thrilled to hear what the professionals on this panel had to say about their experiences moving their email infrastructure to the cloud.

By the way, in the coming weeks, I’ll be discussing the role of implementation and services teams in the onboarding experience. What do you think it takes for successful onboarding in the enterprise? Let me know—I’d love to hear about your own real-world experiences.

—Brent
@brentsleeper

 

Like this post? Check out some other Insight 2015 session recaps:

Zillions Guide Blog Footer

Every day when I pick up my kids from school, I ask them what they’ve learned that day. They proceed to tell me what they did—in class, after school, what they had for lunch, who they played with at recess. But getting them to articulate what they learned is a lot harder. So in the spirit of setting an example, I thought I’d report on what I learned this year as a product manager for SparkPost.

nightskytrails

First, let me back up: it’s been a year of remarkable change and growth for our company. We made the leap from our origins as an established, packaged software vendor to a software-as-a-service operation. We architected an entirely virtualized, cloud-based infrastructure. We built and launched our core SparkPost offering. We expanded upon that foundation to introduce the SparkPost Elite service with dedicated instances and service level agreements to suit the world’s most demanding senders. We built out a world-class operations, deliverability, and customer success team. And, we changed our brand from Message Systems to SparkPost to better reflect all of these changes.

But those are things that we did. What did we learn? Here are four lessons about doing business in the cloud that really hit home for me this year.

  • Offering a cloud service means more than engineering a technology stack. It requires a deep understanding of how customers actually integrate technology into their business processes. It also means publicly staking a claim with the right product/market fit and countering a new group of competitors. All in the open.
  • Another key lesson for us at SparkPost has been just how critically important it is to reduce friction throughout the customer lifecycle, from selling to onboarding to daily ease-of-use and support. In plain language: the cloud means we need, more than ever, to make it easy for customers do business with us. In our market of high-volume, high-value email, we want to make it drop-dead easy for legitimate senders, while freezing out spammers and phishers. Ultimately, dealing with the bad guys in the email world is where our rock-star compliance and deliverability teams give us a real competitive advantage. But as a product manager, I can assure you that it takes a lot attention to detail to get that balance just right.
  • The cloud changes everything, including the business model. If you’ve spent any time in the traditional software industry, you know how big, perpetual license deals are the name of the game. But there’s a reason why the business model for cloud businesses is called “software-as-a-service.” Services aren’t a one-and-done deal; instead, our accountants report recurring revenue as the primary metric for our shareholders. For customers, that’s good news: less up-front capital expenditures, more bite-sized spending, and a real incentive for the company—that’s us—to keep customers happy and earn that recurring revenue.
  • And this brings me to the thing I think about every day of the year. Of course I want to develop a product that has the most compelling features in the industry. Of course I want to see my product beat out competitors on the biggest deals. But the discipline that the recurring revenue model enforces on us means that customer retention (and that really means customer satisfaction) is simply crucial. To be frank, the same simplicity that makes the cloud so compelling also makes it pretty easy for a customer to switch to a new service provider. So, that means that I am always working to make SparkPost better-performing, easier to use, cost-effective, and a step ahead of my competitors in all the ways that matter to our customers, including email deliverability.

That last lesson is the most important thing any company needs to remember, and doing business in the cloud simply makes it all the more obvious. So, what I learned in 2015 (and will keep focused on for 2016) really is a reminder of what I and my colleagues have always believed: keeping our customers happy is the key to our success. It’s not the technology, and it’s not the marketing, or anything else except you. So a heart-felt thank you from all of us at SparkPost and from me personally. I’m looking forward to an awesome 2016.

—Irina, Cloud Queen 👸