Measuring E-Mail Campaigns in the New Environment
Using the traditional metrics as a guide, you may consider testing new subject lines, improving your targeting or providing a more compelling offer to optimize your results. However, the best database management, the best creative and the best one-to-one targeting will not have a lick of impact on your results if your message simply never reaches your customer.
As the volume of spam skyrockets, ISPs, enterprises and individuals are deploying a plethora of filters that block, delete and redirect e-mails to junk/bulk folders. None of these methods is perfect, and an increasing percentage of opt-in e-mails is being mistaken as and treated as spam ("false positive").
Today, 20 percent of legitimate permission-based e-mails are not being delivered to the recipient's inbox. Deliverability, and specifically getting your e-mails into your customer's inbox, is the most important factor driving campaign results today.
Campaign-Related Delivery Metrics
Bounce rates, hard and soft. Your e-mail deployment vendor or your e-mail software typically provides "mailed," "bounced" and "delivered" statistics. Ensure you track the specific reason for the bounce by campaign and by ISP for "hard bounces" (delivery status notifications beginning with a "5" and indicating a permanent error such as an invalid e-mail address) vs. "soft bounce codes" (delivery status notifications beginning with a "4" and indicating a transient error such as the recipient's server is down).
The reason? The incidence of "false bounce reporting" is on the rise as ISPs send delivery error messages indicating a permanent failure when it is not. They do it to thwart spammers who use the delivery status notifications (or lack thereof) to confirm they have a valid e-mail address. For more information on bounce codes, visit: http://rfc.net/rfc1893.html.
True delivery or "% received." Standard e-mail campaign reporting provides a "delivered" statistic. However, this is not the true delivery rate as standard SMTP-based reporting subtracts only bounced e-mail to arrive at the delivery rate. However, this figure represents just what was delivered to the ISP. The ISP may now elect via its spam filters to delete some of this "delivered" mail without providing notice to the sender. On average, 10 percent of legitimate mail is being "deleted" by the ISP. There is only one way to track the "% Received": You will need to set up e-mail accounts at all the major ISPs and record what percentage of mail is received or hire a service to do this for you.
% bulkbox and % inbox. The true delivery quantity or percentage of mail actually received by your recipients (described in item No. 2) may be further broken down into "bulkbox" and "inbox." Bulkbox is the mail that has been redirected by the ISP to the recipient's junk, spam or bulk folder. Inbox is the mail that has been allowed to flow directly to recipients' regular inbox folder. These are important metrics, as many recipients do not bother to view e-mails in their bulkbox, and so many marketers consider bulkbox placement the equivalent of non-delivered mail.
Another consideration of bulkbox placement is the negative implications for your brand when your mail is placed in the same folder as spam. Thus, you want to minimize bulkbox placement and maximize inbox placement.
Measuring inbox vs. bulkbox placement cannot be done with the standard SMTP-based reporting provided by most e-mail deployment technology providers. Similar to tracking "true delivery or % received," you will need to hire a seeding service that specifically tracks these metrics for you.
Campaign-Related Marketing Metrics
Open, click and conversion rates. Open rates (the percentage of HTML-enabled recipients who open your e-mails), click rates (the percentage of recipients who click on one or more links within the e-mail) and conversion rates (the percentage of recipients who take the prescribed action -- e.g., register or place an order) form the basis of most marketing analysis. However, marketers historically have calculated these statistics using the "delivered" quantity provided by their standard SMTP-based reporting as the denominator.
This does not let you accurately compare different marketing efforts or performance by ISP, as it does not account for the amount of mail deleted by the ISP or redirected to the bulkbox. When calculating these statistics, you also will want to consider measurement and evaluation on three levels:
· Use "delivered" as the denominator.
· Use "true delivery" or "% received" as the denominator.
· Use "inbox" as the denominator.
Spam complaints, by campaign and ISP. Spam complaints are the number or percentage of individuals who complain that the e-mail you are sending them is spam. Typically, recipients will reply to the e-mail you send them. Most marketers today have what is called an "inbound mail handler." This is the system that processes inbound e-mails to your reply-to e-mail address.
More sophisticated systems automatically segregate these e-mails by content (e.g., unsubscribe requests, spam complaints, customer service requests) while less-automated systems direct the e-mails to a person or group in your company. Either way, it is important to record the number of individuals complaining that your e-mail is spam on a campaign and ideally on an ISP-within-campaign basis.
Over time, you may use this data to help determine the optimum frequency with which to e-mail your customers as well as determine where in the life cycle of communications your customers are most likely to voice their dissatisfaction.
Unsubscribe rate, by campaign and ISP. It is imperative to provide your customers multiple ways to unsubscribe in every e-mail. They should be able to:
· Click on a link to unsubscribe and/or reply to the e-mail.
· Contact you via traditional means (phone or mail).
Similar to spam complaints, the unsubscribe rate is an extremely important measure of how your e-mails are perceived by your customers. Wherever possible you should tie the unsubscribe rate directly to the campaign from which they originated as well as the ISP/domain.
Similar to spam complaints, over time the unsubscribe rate can be used to determine the ideal frequency with which to mail your customers and also provide directional input as to content (e.g., if the unsubscribe rate peaks on promotional e-mails versus newsletter e-mails, you could test combining the two). Remember, once a customer unsubscribes you may not e-mail them again until they re-subscribe, so managing this metric down is extremely important.
Non-Campaign-Related Monitoring Metrics
Along with metrics commonly tracked and related to specific campaigns, there are other measures you should track frequently. These metrics are important gauges of customer perception and your deliverability. The frequency with which you track and measure these variables should be driven by the frequency with which you mail.
At minimum, you should measure these variables monthly. Some outsourced e-mail deployment vendors may handle some of these tasks for you and share the metrics.
Additionally, independent delivery optimization services may be contracted to track these for you if you mail from in-house or your e-mail deployment vendor does not do this for you.
Monitor block lists to ensure your sending IP address is not listed. There are hundreds of block lists (also known as black lists and RBLs). They generally include listings of IP addresses and are used by ISPs and corporations to block e-mails originating from these IP addresses. Most of the block lists are unimportant as they are not widely used.
Some that are more important: MAPS, Spamcop, Spamhaus and SORBS. You may check whether the IP addresses you use to send your e-mails are on any block lists by visiting www.dnsstuff.com and entering your IP address in the spam database lookup. If your sending IP address is listed, you will want to explore the reasons you are listed and, if possible, work with the block list to be removed.
You may request assistance on tracking which block lists are listing your IP, gauging the importance of these block lists, and/or resolving listings from either your external e-mail deployment vendor or a delivery optimization company.
Monitor abuse message boards for postings about your company. There are a number of Google message boards on which the Internet community posts information about companies that they believe are abusing the Internet.
Some block list managers track postings about companies on these abuse boards and add their IP addresses to their block lists, so these postings may have an effect on your delivery. In addition, by monitoring and reading postings relating to your company, you may gain valuable insight into how some recipients view your e-mail. If there is no posting regarding your company on the abuse message boards, that is good.
To identify whether your company has been mentioned on any of the abuse message boards, click on the following link. When the Google search page appears, type in your company name in the search field; you may repeat the search a number of times with your domain, common abbreviations of your company name and product names:
Monitor the caliber of sites with links to your site. In general, you want other Web sites to have a link to your site. In addition to the marketing value, it also has a positive effect on certain search engine rankings. However, your brand is important, and if one of the sites that has a link to your Web site is not the type of company with which you would prefer to be associated, you will want to address this. This is particularly important if you have an affiliate program or pay any form of referral advertising fees as companies and individuals are incentivized to drive as many leads as possible to your site.
Additionally, with the increase in phishing companies masquerading as a company they are not for the purpose of defrauding customers, you may find that some of the sites linking to your site are actually pretending to be you. To monitor how many sites link to your site, you may use any of the popular search engines (Google, Yahoo, etc.), request this service from your e-mail deployment vendor or delivery optimization service. There also is a free tool that you may use at: