A/B testing
| In the emails that we have sent out as tests and to the professor, the "From" section is a little messy. "A Matter of School," the name of the company, is in the "From" section, but so is my personal email. This could be confusing to recipients. As an appropriate A/B test, we could clean up the "From" section by inserting [email protected] for two of the four 18-22 recipients. And then we could send my email as the "From" and see what information comes back. This is a small sample size. In the real world, we would have much more data to work with. |  Wild, expressive health and beauty products For the 23-25 years olds, we could send one email with "[email protected]" in the "From" section and my email to the other two. Once again, it would be much more accurate to have a larger sample size. |
Expected rates
We are presented with two different set of expectations of statistical guidelines regarding delivery rate, open rate and click-through rate (CTR).
| Category Delivery rate Open rate Click-through rate | Oracle 99% 17.6% 2.17% | Stukent N/A 20% 3.5% |
The Oracle numbers are the midpoints of the range of the data given in each category. It should be noted that Oracle's test emails deliver at a 99.3% rate vs. about 99% of the population as a whole.
Since A Matter of School is sending emails to people who have previously purchased from them, it is reasonable to conclude that the open rate and CTR would be closer to the Stukent averages than the Bronto/Oracle numbers. Also, the delivery rate for A Matter of School should be closer to the test email at 99.3%.
Expected rates for A Matter of School email marketing campaign:
Since A Matter of School is sending emails to people who have previously purchased from them, it is reasonable to conclude that the open rate and CTR would be closer to the Stukent averages than the Bronto/Oracle numbers. Also, the delivery rate for A Matter of School should be closer to the test email at 99.3%.
Expected rates for A Matter of School email marketing campaign:
| Delivery rate Open rate CTR | 99.3% 20% 3.5% |
Suggestions
As the numbers roll in on the campaign, there will always be areas to improve.
Delivery rate - No matter where the numbers come in at - 97% or 99.9% - we should continually strive to maintain and improve our lists. We suggest that A Matter of School devote resources to this constant cleaning process by:
1. authenticating email addresses
2. improving the opt-in process
3. checking IP's and links
4. reviewing Delivery rate data in gender, geographical, time of day, and other categories to see if there are more bounces in specific categories.
Open rate - As with Delivery Rates, the data will tell us the story. We will look at the data and attempt to maintain and improve in four categories:
Segmentation - There is clear evidence that companies that segment ruthlessly experience better open rates. We should stress to the client to spend the money to comb the data for better ways to target and segment.
Timing - The data will tell us when it's best to send our emails in different segments and how often.
Subject lines - Although we did not do A/B testing of subject lines for this exercise, we could certainly do that going forward to improve open rates.
Content - One of the best ways to improve open rates is from repeat openers. If recipients know that you have in the best sent them content that is relevant to their lives, they will be more likely to open future emails.
Click-through rate - Content is king when it comes to CTRs. We must continually do A/B testing of different content to random subsets to ensure that we are achieving the maximum possible CTRs. The data will tell us a lot in this area.
1. authenticating email addresses
2. improving the opt-in process
3. checking IP's and links
4. reviewing Delivery rate data in gender, geographical, time of day, and other categories to see if there are more bounces in specific categories.
Open rate - As with Delivery Rates, the data will tell us the story. We will look at the data and attempt to maintain and improve in four categories:
Segmentation - There is clear evidence that companies that segment ruthlessly experience better open rates. We should stress to the client to spend the money to comb the data for better ways to target and segment.
Timing - The data will tell us when it's best to send our emails in different segments and how often.
Subject lines - Although we did not do A/B testing of subject lines for this exercise, we could certainly do that going forward to improve open rates.
Content - One of the best ways to improve open rates is from repeat openers. If recipients know that you have in the best sent them content that is relevant to their lives, they will be more likely to open future emails.
Click-through rate - Content is king when it comes to CTRs. We must continually do A/B testing of different content to random subsets to ensure that we are achieving the maximum possible CTRs. The data will tell us a lot in this area.
RSS Feed
