If you send a cross-sell offer to 1,000 customers and get 140 responses, would you consider it a successful campaign? What if 137 of the targets would have made the purchase without having received the offer? In this scenario, the solicitation provided an incremental lift of 0.3 percent, not 14 percent.
It’s common to look at total response and/or revenue from a campaign to evaluate its effectiveness. But this approach has two problems. First, it overstates the promotion’s value, as the aforementioned scenario illustrates. For example, catalog customers never have to receive additional catalogs to make future purchases. Similarly, credit card customers do not need to receive convenience checks to take cash advances.
Second, and more importantly, this approach focuses marketing efforts on the wrong population segments. Targeting customers who would take a desired action without solicitation means you fail to reach the people who might benefit from your offering, but need motivation to act.
To determine the true response to a marketing campaign, a best practice is to select a random sample from your target audience and not mail them. This is called the “no mail” control. The difference in lift between the two groups is the incremental value of the promotion. This methodology can be used regardless of marketing channel.
When the target group is too small to select a statistically significant control group, you can pull the control group from multiple solicitations over time and then combine them for later analysis.
Shortcomings of other approaches. Many marketers measure true response through various other tools. One involves including a key code with a specific offer. Though this can work for determining the effectiveness of a promotion for a specific offer, the promotion might easily provide positive and negative effects that cannot be tracked through key codes.
For example, a common measure of marketing success in packaged goods is coupon redemption. Have you ever received a discount coupon for a grocery item – say, a brand of ice cream – made the purchase and then remembered that you left the coupon on your kitchen counter? If so, the promotion motivated your purchase, but your implicit response would not be counted through coupon tracking.
Alternatively, suppose you sent a promotion that generated $10 in gross margin for a designer shirt that could be tied back only to the promotion. If the person making the purchase would have bought something else through another channel with a $15 gross margin instead, then the promotion hurt overall profitability.
Other marketers use flawed strategies to measure upsell and retention. Suppose you tried to upsell customers who called to buy a $45 wool sweater with a $269 cashmere sweater instead. You cannot assume that all of the cashmere sweaters sold this way is the incremental value of the upsell offer. Some prospects might decide not to buy either item, as the choice between two items could cause prospects to rethink a planned purchase decision.
The only way to determine the effectiveness of a retention effort is through implicit response. If you made a retention offer to a customer likely to defect and the person stayed, how could you know whether the person would have stayed anyway? You would need a random control group that did not get the offer and compare the subsequent retention rates to the targeted group.
A better approach. To target incremental response, you need to estimate the implicit response without a promotion. This is done by first selecting a random sample from a universe that qualifies for your offer. From this group, select a random “no mail” control similar to the best practice mentioned earlier for determining true response.
Once the test campaign is complete, build a model to predict who “responded” (exhibited the desired behavior) in the no-mail group and build a similar model for the mailed group (the same concept applies for telemarketing and e-mail).
When it’s time to roll out the campaign, score each customer with both models, subtract the no-mail-group score from the mailed-group score and rank order the difference. The result is the lift you theoretically will achieve from each customer.
In the following example, an offer for a free hotel room upgrade on a subsequent booking was sent to six customers. Customer A is the most likely to generate the highest profit (based on a high probability of booking a hotel room) if sent an offer. Yet, because he also is the most likely to book a room even if he did not receive a discount offer, the offer was least likely to change his behavior. The offer had the most effect on Customer D.
If you ignored implicit response, selecting the top three customers based on response to the offer (A, B and C) would result in an expected profitability of $268 ($223 from those receiving the offer and $45 from those who did not).
If instead you targeted based on the change in behavior (customers C, D and F), the expected profitability would be $305 ($119 from those getting the offer and $186 from those who did not).
Because marketing aims to change behavior, it’s important to consider what the customer or prospect would have done in the absence of any promo. By incorporating implicit response, you can maximize targeting and better track the effectiveness.
