In the first part of this series, I covered a lot of ground on all the discrete bits you need to be aware of to create a successful email integration. Now it’s time to actually put those things into practice.
Iteration is something I believe in strongly. Trying to plan everything perfectly in advance usually results in never getting started at all. For me, it is most important just to start moving forward, and I always learn best by doing. So in that spirit, today’s post is all about stringing together the important pieces you need to get in place to send your first email using SparkPost.
Sending Domain Pitfalls
Picking a Good Root Domain
Now that we have an explosion of vanity top level domain (TLD) extensions available (.ninja, .rocks, etc), it is tempting to choose something that fits in well with your application, seems clever, or might be a bargain. Unfortunately, vanity domains are a favorite of spammers. Unless the sender behind a vanity TLD is sending large volumes of quality email, it is very difficult to have success with a vanity TLD. Even then, it takes a great deal of time and effort to establish the domain’s reputation. This is a place where I like to take the easy path to success and stick with the traditional top level domains: .com, .net, .org.
Use a Subdomain
I always thought it would be easier to just send all my email out from my primary domain, but I’ve learned that it is better to separate your mail into subdomains. Instead of sending from mydomain.com, it is better to send from email.mydomain.com. If you choose, you can further divide your sending domains by using separate subdomains, such as marketing.mydomain.com.
Segregating your email streams into appropriate subdomains helps separate the traffic into logical streams. This can help when smart email clients try to categorize your mail. Most importantly, spam filters generally treat each subdomain separately. That can be important for making sure an overzealous spam filter doesn’t filter out a password reset email.
Registering a Domain
Once you have picked a domain, registration seems like a fairly straightforward process. Just pick a registrar and get it done, right? Not exactly. Domain registrars are not all created equal. There are a few things that I have learned to consider when picking a registrar: domain privacy, web hosting, and DNS management.
I was surprised to learn that the practice of registrars obscuring domain ownership information (domain privacy) will hurt your email deliverability. I had always thought of domain privacy as a good thing to reduce spam and unwanted solicitations. After joining SparkPost, I learned that spammers are very likely to use domain privacy as a way to conceal that hundreds or even thousands of domains belong to the same entity.
A private domain registration may seem like it protects your privacy, but in reality what it is saying to the rest of the world is that there is no one that is taking responsibility for this domain. My best advice is to leave your WHOIS information public, and be very zealous about setting up email filters and reporting spam. Also, ensure that the phone number on the domain registration is on the do not call list.
Importance of a Website
It is important to be able to manage the DNS settings that are required for a successful email infrastructure. Some domain registrars restrict or complicate the management of DNS, so in my experience it is best to go one of the major domain registrars, such as Rackspace, GoDaddy, DreamHost, Cloudflare, or Namecheap.
It is important for users to know exactly how their personal information is managed. There are a myriad of laws and regulations that require that users be informed how their personal information will be managed. There are a few things I have learned that I use to help guide me through these perilous waters:
- Err on the side of caution when it comes to user data security and privacy.
- Encrypt the user data and never share it without the user’s express consent.
The Internet Runs on DNS
Set Up the SPF Record
The Sender Policy Framework (SPF) record is a TXT type DNS record. A valid SPF record traces from the return path domain to the server that sent the message. It tells the receiver of an email that the message was sent by a server actually authorized by the signing domain.I have learned a lot about SPF in my time at SparkPost, and most recently, I learned that SPF records are parsed in order. So it is not just a matter of getting the correct data in the record, but having it in the correct order.
Lucky for you, if you’re using SparkPost, the mail you send is already covered by sparkpostmail.com SPF record so there’s no need for you to publish your own. You can still publish an SPF record when using SparkPost, but it is not required. A properly formed SPF record using SparkPost would be: “v=spf1 include:<your sending domain> include:sparkpostmail.com ~all”. Once you put the record in place, you can test it using the Kitterman SPF Testing Tools.
Set Up the DKIM Record
The Domain Keys Identified Mail (DKIM) record is a TXT type DNS record. It contains a public key that is used to validate the digital signature that is attached to each email message. The digital signature generated by the sending mail server is attached to the message. The receiver can be certain that the content of the email was actually authorized by the signing domain.
One of the main challenges I run into with DKIM records is that the strings in DNS TXT records are limited to 255 characters, but DKIM keys can be much longer. The way to get around this is to separate the record entry into blocks less than 255 characters long. Each block needs to be surrounded by double quotes. When DNS reads this record, it will concatenate all of the strings together.
You can validate your DKIM record is working properly using the SparkPost DKIM Validator.
Resource: SparkPost DKIM Setup Guide
REST vs SMTP
Although Simple Mail Transfer Protocol (SMTP) is designed specifically for moving email around the internet. SMTP created a way for servers to communicate and reliably transfer email from sender to destination. The reliability of the protocol was very important in the early days of the internet when networking was not as reliable and often times not even 24/7.
However, SMTP is not always the best choice for integrating email into other applications. SMTP is very chatty as it validates that every part of the conversation is received and understood before moving to the next piece. When you are trying to achieve very high performance, SMTP can be a bit slower due to the communication overhead. On the other hand, SMTP is extremely reliable and can provide high confidence that a message is accepted by the server with little programming overhead. SMTP can also be useful in connecting with legacy systems that do not support communications through HTTP protocol.
API-based message generation implemented with Representational State Transfer (REST) is a more programmatic way of doing things. It relies on simple acknowledgement that the receiver has gotten the entire message. The receiver then responds in its own time with an appropriate status once it has processed the message. Think of it as a conversation where you start to say something and someone makes eye contact. You complete what you are saying, and the listener responds with a nod or a headshake. That is how REST works.
I prefer to use REST as a method for injecting email. It is much more efficient and puts the onus on my side for ensuring what I am doing is correct. It can be frustrating to debug when you make a mistake and you get a cryptic 400 response from the server. The frustration, in my opinion, is worth it in the long run given the gains in efficiency that can be achieved by reducing the communication overhead.
One of the other reasons I am a fan of REST over SMTP is that it is very easy to interact with a REST API from a terminal emulator. Utilities like curl, httpie, and Postman make it very easy to interact with REST APIs for testing and experimentation.
Resource: SparkPost In Postman
Resource: The Differences Between Using SMTP or API with SparkPost
The Home Stretch
Send The First Email
All the pieces are in place now. It is time to get that first email out of the gate and into the inbox. Fire up your chosen tools, grab an API key, and put together a test email. If that email gets into your inbox, you are all set! Take a victory lap.
What if it doesn’t get into the inbox? What now? Don’t panic. There are many reasons that an email might not end up in the inbox. The most critical is reputation. The first email from a new domain is very likely going to land in a spam folder. That seems somewhat unfair, but I have learned that the reasoning makes sense.
Break the Ice
A new sender has no reputation. The bread and butter of most spammers is to register a domain, set it up, and fire off a bunch of crappy email until they get blocked. Inbox providers got wise and started to be very skeptical of new senders.
The best way to remedy this situation is to start building your reputation. Even if you are not using a dedicated IP, following a warm-up plan can help you gain that reputation. In simplest terms, a warm-up plan means sending email to people that want it with a slowly increasing volume.
Even if you landed in the inbox, you need to start building that reputation. Reputation ages and slowly expires. That means keeping up a minimum volume of sending to help maintain that reputation.
The email is flowing. Reputation is steadily increasing. The minimum viable product (MVP) has been reached. What do we do for stretch goals?
Now it’s time to look at metrics and see how emails are performing. Are people opening? Are people clicking links? Is it being flagged as spam? In my next post, I will explore the basics of email metrics, how to access them using our APIs and webhooks, and some tips for acting on them.
— Nick Zimmerman
Senior Site Reliability Engineer