Have you ever wondered why so many rollouts are disappointing? Three of the authors’ recent DM News articles established that a part of the answer lies with statistical sampling theory. However, there is more to the story. Two other articles have explored how merge/purge can affect reported list performance.
We will continue to focus on the nonstatistical side this month, but will branch out beyond merge/purge.
Statistical testing theory is predicated on the assumption that all things are equal except for the test parameters. But in direct marketing nothing is ever equal. Some of the problem is caused by the poor decisions of circulation professionals, who often have not been trained in proper experimental design. We will focus on that in a future article.
A significant portion, however, is driven by fluctuating environmental influences, both competitive and noncompetitive. Direct marketers must understand and be mindful of these phenomena to correctly interpret rollout behavior.
Noncompetitive environmental influences. Consider the article “Mailing’s Eerie Timing Shatters Economist’s Response Goal,” which appeared on page 1 of the Jan. 28 DM News. “Can Islam and Democracy Mix?” That question and the photograph of a Muslim woman covered except for her eyes on a direct mail piece from The Economist that dropped Sept. 4 and began arriving in mailboxes Sept. 13 contributed to a response rate 30 percent greater than expected.
The Economist should be thankful for its fortuitous timing. However, it should be mindful that similar response boosts were experienced by certain direct marketers during the Gulf War and that these boosts did not hold up over time. Consider the veterans organization that built a prospecting model off a fundraising mailing that dropped during the Gulf War. As the organization learned when the model flopped during rollout, response patterns during this atypical time were not representative of future behavior.
Competitive environmental influences. What the competition does, and when it does it, can affect your promotions. The following case study illustrates this by focusing on the overlap that can exist between the customer universes of you and your competitors:
Several years ago, a statistics-based model was built to predict the future purchase volume of a direct marketing client’s customers. This model remains very effective in determining which individuals to promote, and how often.
Subsequent to the construction of the model, the client purchased its largest competitor. Currently, certain administrative functions have been consolidated. However, when it comes to customer and prospect contact strategies, the two companies continue to operate as if they are still stand-alone operations. We will refer to the original business as the “original division,” and to the purchased business as the “sister division.”
Recently, the overlap between divisions was quantified, with a mind to determining the level of cannibalization. The study focused on individuals who are customers of both divisions. The goal was to determine whether historical purchase information pertaining to the original division is predictive of purchase behavior from the sister division.
The predictive model for the original division – which by definition employs only historical information from the original division – was measured for its accuracy in estimating order activity from the sister division. Here’s how: Overlapping customers were scored by the original division model. Customers were sorted by score and broken into 10 groups called deciles. With this approach, decile 1 corresponds to the best performing 10 percent of the file, and so forth. Responses received by the sister division within a six-week time frame were appended to the file and tabulated for each of the 10 deciles.
It was remarkable just how effective the original division model was in identifying which customers were likely to make a sister division purchase. The results illustrate the interconnectivity that can exist between you and your competition.
Controlling for environmental influences. To the fullest extent possible, you as a direct marketer must anticipate and control for environmental influences when evaluating results. Without this perspective, you are evaluating list performance in a vacuum.
Clearly, it is tougher to do this for competitive than for noncompetitive environmental influences. However, it is not impossible. As long as you are receiving promotions from the competition, it is a straightforward matter to estimate the degree of overlap.
You can increase the probability of receiving the competition’s promotions to its house files by ordering multiple times from the opposition camp. A purchase or two likely guarantees multiple follow-up contacts. You can also achieve perspective on the competition’s prospecting by ordering from firms whose rental lists consistently performed well for you in the past. The list business is incestuous, and what works for you likely also has been effective for the competition.
