In part 1 of this blog series, I went over the challenge of the IP warm-up process. In this section, you’ll learn about the different situations that can arise when trying to implement the warm-up process.
Implementation Scenarios – The Easy
Let’s start with the easiest scenario. You have one application that sends one type of email and you are moving from one ESP to another. We will assume you have been using the old ESP for a while and those IP addresses are warmed up.
How is the application that is going to build and send the email know which ESP vendor to use? In the simplest of environments, you could have an input file which contains the number of emails that should be sent that day using this new sending domain/IP combination (SDIPC). Since you may have multiple email categories, each line needs to represent a different email stream. Then someone would update that file on a daily basis with the new count. Depending on the ESP, the information the email sending application needs may be different. For SparkPost, you need to know both the sending domain and what is called the IP_Pool name (Some ESP’s allow you to have your own IP addresses and bundle them together in order to meet sending demand for a specific sending stream, like Newsletters, Daily Job Updates, Passwords, etc. These bundles will be referenced as IP Pools throughout the blog. An IP Pool can consist of 1:1 or 1:many IP addresses). The file might be a simple CSV file like:
<stream>,<sending domain name>,<IP address/pool>,<daily number of emails>
The application will read the file and keep track of how many emails it sent within each stream. Once the application has run out of its daily pool, it will start using the older warmed up SDIPCs already available.
Coding for that simple approach isn’t very difficult for even a new developer, let alone an experienced one. But reality usually doesn’t take the simple path. What if you have more than one application that sends emails; maybe you have a full cluster of applications that send millions of emails an hour. Which one of those applications (services) uses that new SDIPC? What if that service goes down? What if one of the ESP’s are not responding? What if a Demogorgon is on a rampage through Hawkins, Indiana? How will you keep track of how many emails have been sent for each of those SDIPCs? What do you do if that SDIPC isn’t available?
Implementation Scenarios – Reality
Tackling this best practice isn’t as easy as it seems so I decided to start building a solution during a hackathon project in order to help customers and prospects through this process. It is important that this code supports large vendors that may have multiple high volume sending applications across a large number of SDIPCs. I also have to assume that the user is already using a previous ESP, so the system needs to support multiple ESPs at the same time. From a business continuity perspective, I think supporting multiple ESPs (or one ESP that can send from multiple unique locations) should be paramount to any email sending strategy. The rest of this blog post (and part 3 of this blog series) will describe the final product that blossomed out of that hackathon project.
In designing the SDIPC warm-up platform I used the following assumptions and philosophies:
- Support multiple ESP’s (or sending endpoints)
- Future proof if possible
- Support both the warm-up process and long-term sending through multiple endpoints
- Support for high volume (millions of emails/minute)
- Need support for when and SDIPC is unavailable
- Automate as much as possible
- Easy installation
- Support for SMTP and RESTful API (by staying away from the actual sending)
- UI to simplify
- Support the ability for an email application sending 1 to ‘n’ emails at a time
To support the above items, I ended up architecting a five-component system. Let’s go over the components:
- An adapter that can send email to a specific ESP
- One or many email generator(s)
- Something that keeps track of how many emails have been sent through a specific ESP, IP or pools of IPs (SDIPC)
- A controller process that updates how many emails can be sent through a specific SDIPC during a given time period
- Process to test if ESP is up
*Note* The controller process in my example was built as a library that will be ‘included’ into each Build/Send process. This allows the build/send processes to manage which SDIPC to use in parallel instead of waiting on a service.
In the final part of this blog series, I’ll go into more detail on this second more realistic implementation scenario.