Navigating Bounce Codes in the Wild West of Email
Giddy-up partner! We got varmints out yonder! Just kidding… but in all seriousness, bounce codes are a little like the wild west. They’re uncertain, fraught with danger, and should be approached with healthy amounts of respect and caution.
Before getting into the meat of bounce codes, let’s review the basics. There are two basic bounce that RFC 1893 defines (and in case you didn’t know, RFC’s are the standards that describe internet technologies and are maintained by the IETF, also check out RFC 5248):
- Transient bounces (also known as soft bounces) are defined as temporary failures or conditions that once they change will allow for the delivery of the message. There’s a numeric code that is attached, or used to define the temporary failure. We refer to this code as a 4.x.x (the x’s are variables).
- Permanent failures (also known as hard bounces) are just that, permanent failures that will prevent the message from being delivered to the intended recipient. These bounces are classified with a 5.x.x (again the x’s are variables).
- A sample bounce code might look something like this:
SMTP error from remote mail server after end of data:
host example.domain.com [2a00:1450:4013:c01::1b]:
550-5.7.1 [2a01:5b40:0:252::55 12] Our system has detected that this message is likely unsolicited mail. To reduce the amount of spam sent to DOMAIN, this message has been blocked. Please visit postmaster.example.domain.com
In addition to these numeric designations, extended descriptors shed more light on why messages aren’t being delivered. Email admins review these text strings, and their corresponding bounce codes to determine where problems occur – either on the sender’s side or the receiver’s. With this information in hand, an email admin makes the necessary changes on their end and can “be the town’s sheriff”. Ultimately affecting positive change, and altering the disposition of the message from bouncing to placement in the inbox.
This would be a perfect world.
However, we don’t live in that world. I wish we did, but we don’t.
Like the old west, not everyone uses words the same way. In the real world there’s a little problem with the seemingly innocuous binary construction of bounce codes and their corresponding description strings. While the RFC defines the two flavors of bounce code, there’s no blanket rule or set of strings that must be employed in the bounce to inform a sender on why delivery fails. A postmaster can, in all truth, write whatever he or she wants into the extended description making the job of the recipient infinitely more difficult. A sample bounce could then look something like this…
- SMTP error from remote mail server after end of data:
host example.domain.com [2a00:1450:4013:c01::1b]:
550-5.7.1 Spammer go die! No seriously, take a long walk off a short pier!
Imagine, every domain has their own set of unique bounce codes and extended descriptions that don’t conform to the simple binary construction defined by RFC. Well you don’t have to imagine it, that’s about the state of bounces all over the world… *deep deep sigh* it’ll be ok.
Hard vs. Soft
Unfortunately for you and me, the world isn’t a neat place where everything is black and white. There isn’t really a universal agreement on what a permanent vs. temporary failure code means. Although a 5.x.x should be ‘user doesn’t exist’ or ‘domain name doesn’t exist’ etc. that’s not always the case. Certain domains believe that if someone is blocked for spamming, or for including a URL (knowingly or otherwise) in the body of the email that’s on a URL blocklist, then that constitutes a permanent failure and will bounce it back with a 5.x.x. As a long time proponent and advocate of Best Common Practices around messaging, I can see how you might want to permanently fail a sender’s messages for spamming or looking like spam.
However, in reality, one man’s spam is another man’s ham. Spammy messages or poorly targeted campaigns are rectifiable situations. Maybe you didn’t think about the implications of sending to people who haven’t received your email in years, so they ran up the spam complaints and your message went to junk or was blocked. These are things that happen on occasion, prompting work to resuscitate your reputation. This may require enlisting the help of deliverability experts like the ones that work here at SparkPost, but nonetheless, it’s resolvable. Again, the problem is that every ISP and mailbox provider does it differently. There’s no true consensus, despite the RFC, on what goes into an extended description, what constitutes a hard bounce vs. a soft bounce or how to go about remediating.
Don’t believe me? Check out this list of postmaster pages and the varying degrees they provide:
- AOL Error Codes
- Microsoft (Hotmail/Outlook.com/Live.com) Postmaster Page
- Gmail SMTP Error Reference (which happens to live under google apps) and is different than the Postmaster Tools for bulk senders
- Comcast SMTP error codes
- Yahoo! SMTP Error codes
- Even the US Postal Service got into the SMTP error code game!
Confused yet? Don’t worry, we have your back! Although these are just a few of the numerous flavors of error code out there, the domains in the list above represent a lot of mail boxes. If you’re industrious and efficient, you solve for the lowest hanging fruit first. You can do this in a few steps. First, tackle the biggest domains. Incorporate those rules and learn them. Then, adjust sending volume, speed and connection rates. These are all in response to the kinds of errors that are being spit back from the domain’s mailbox provider’s MTAs. This helps to achieve righteously high inbox placement. Rest assured SparkPost does all of this.
Since 25% of the world’s legitimate and non-spam email traverses our software, we see a lot of email. As a matter of fact, we’ve been able to solve the bounce problem for more than the handful of domains I’ve listed above. SparkPost’s automated rule set, the Adaptive Email Network, employs over 3k rules that help us classify a vast majority of the bounces that currently exist in the wild.
Synchronous vs. A-Synchronous Bouncing
Here’s where your eyes might glaze over a bit. Bounces return in a couple of different ways. The most common is for the mail server to reject a message from a connecting IP and issue the bounce outright. This is called synchronous as it happens in synch with the conversation. A-synchronous bounces are what you and I see as users. You send an email to your uncle Joe but mis-type his address and receive an email back from Yahoo! saying the message cannot be delivered. The bounce email that you receive is sent back after the bounce, hence it doesn’t happen at the same time you try to deliver the email. The two different conditions change how and where you go looking for the bounces and your reaction to the situation.
But nothing is quite so simple. Have you gotten that feeling? A-synchronous bounces can happen in bulk. They aren’t the most common of bounces as it’s more efficient to send synchronous bounces vs. burning up your back end cycles trying to deliver email and then sending email back in response, but they do happen. Senders of bulk email need to deal with this situation in addition to processing bounces handed to them directly by the connecting mail server. Wild west, remember?!
What does this mean for you?
Simply put, when you know what a bounce code means you can take the appropriate actions to ensure your reputation doesn’t suffer and you’re able to reach the inbox. For example: a 421 from Yahoo! is sometimes called a gray listing and the appropriate action is to pause sending for 4 hours and retry. The only way you know this is to capture the bounce, read its contents and then apply automation rules to ALL outbound email. This puts you in compliance with the domain’s stipulated guidelines. Like I said, we got your back!
SparkPost’s vast rule set was built over 16 years of operation. Our deliverability team captures and evaluates the occasional bounce that we can’t classify, and then creates a rule to account for it in the future. Yep, we do that. Now do you believe me when I say we have your back?
Bounces are messy. Knowing what to do with them is complicated and requires the experience and ability to scale appropriate responses in real-time. That’s what we do for our clients, which makes them incredibly successful. More of our clients’ wanted email reaches their customers’ inboxes, that’s the SparkPost advantage. Come check it out, or let us know about your bounce woes. We are happy to help you out.
#1 There is no uniform standard for expressing bounce codes
As a sender’s subscriber base grows and the number of ISPs you’re sending to increases, the number of bounce types grows exponentially. In the real world, there is no uniform standard for expressing codes, and each ISP uses its own variation. Senders, therefore, must be able to track and understand codes from many, many ISPs. Further, as bounces return to the sending system, the operator must remediate those bounces, or take corresponding action to resolve the problem. For example, if an address is reported as invalid by the receiving domain, it is critical for the sender to immediately stop sending – or risk getting blocked to a greater extent by that ISP.
#2 Feedback Loop Processing Is A Complicated Programming Effort
Processing feedback loops (FBLs) is another time consuming task. FBLs are records of recipient responses to email — when you mark a message as spam, for instance — and most domains send a standard Abuse Reporting Format (ARF) report to the originator. But as with bounce codes, report formatting and content varies by reporting domain, making FBL processing a complicated programming effort. Low cost MTAs have little or no FBL functionality, at best providing a mechanism for routing FBL reports to a folder or dedicated mailbox.If the operator wants to automate the process for interpreting the FBL information and implementing remediation policies, that would require designing, building and maintaining an application to do so.
#3 Bounce and FBL processing represent a very expensive hidden cost of owning a legacy or an open source MTA
Some ISPs publish their rules to help guide senders into optimal sending practices. Others don’t publish rule sets at all. Some change their rules frequently, others seldom do. Given there are more than 12,000 ISPs worldwide (about 7,000 in the U.S. alone) and millions of IP domains, it’s impossible for senders to:
a) Keep track of rules worldwide, and
b) Continually tweak/update their sending infrastructure to reflect constant changes.
#4 Momentum includes dedicated bounce and FBL processing and remediation modules
SparkPost provides a simple mechanism for managing this critical part of your messaging infrastructure. We monitor unclassified bounce codes from the worldwide ISP community, and regularly upload new interpretations to our customers’ systems. The module automatically sorts bounces to approximately 20 classifications. Because the classification is available to the policy engine, a simple script can be written to manage remediation policies. The entire process is completely automated, or Momentum operators can choose to customize rules for their own environment.
Additionally, Momentum has an Adaptive Delivery® module, which uses rules and the data streamed through the FBL and bounce modules to optimize outgoing traffic, throttling it down or accelerating it based on ISP feedback.
With commodity infrastructure, users find that they know when messages leave their own campaign management or CMS application, but have no idea how long it takes their MTA to send the messages after that. In some cases, it could take several hours — and this is deeply frustrating for their clients. All of these issues are easily addressed by the SparkPost solution.