One concern that must be addressed upfront when an organization begins using database marketing as a business channel is how the overall effort should be measured.
A time-tested approach is to define and implement an enterprise-wide control group strategy. This type of control group differs from – and does not interfere with – the A/B split testing typically used to test and measure one creative or communications approach vs. another. Rather, its purpose is to provide ongoing measurement of the efficacy of the database marketing effort globally.
Why would any organization need such a measure? Well, that supportive CEO, CFO or senior vice president of marketing might not be with the organization two or three years down the road, and you might be faced with providing proof of the value of your database marketing program to his or her skeptical replacement. To do that, you’ll need real numbers, not anecdotes. Believe me when I tell you it happens.
The basic idea is to isolate certain customers so that they do not receive any of the promotions or continuity communications that collectively make up the organization’s database marketing program. Comparing these customers with the ones who do get those promotions and communications will, over time, provide an incremental measurement of the overall value of the effort.
This is an important tool to prove the validity of investing in database marketing technologies and processes. Perhaps even more important, it provides trend measurement. As DBM programs mature, there should be an increasing divergence in customer value between the control group and the rest of the customer universe. Stated simply, it will let marketers answer the question, “What are we getting for all the money we’ve invested?”
The calculation for answering that question is straightforward.
Whatever measures are used, compare the average values in the control group with the same average values in the rest of the universe. Because the only variable is the presence of database marketing activity, the incremental difference between the two is the gain or loss that results directly from that activity.
A word of caution, however. Because the control group will be scaled for relatively gross measurements, never use it to measure more granular detail such as creative testing, market-to-market comparisons, etc. The presence of the enterprise-wide control group absolutely, positively does not free the marketers from the responsibility of thinking through testing scenarios and setting up test cells to measure them properly.
I suggest limiting control group measurement to the following dimensions of data:
· Average dollar value of the customer (revenues).
· Average number of relationships (cross-selling).
· Average length of time on the books (tenure).
· Average attrition (churn).
In all cases, the entire control group universe should be compared with the entire non-control group universe of customers, and never broken down into any finer “slices” of the data. This should be a monthly report, with measurements for the specific month the report was issued plus “rolling” 12-month averages. As the program matures, marketing can choose whether to share this report internally.
Select customers for inclusion in the control group randomly. This usually can be accomplished simply by flagging every “nth” customer in the file as a control group customer, then using the same nth method to add new customers to the control group with each update of the file. This constantly refreshes the control group to keep it homogeneous with the rest of the customer universe. However, ensure that whatever method is used, the sample is random. Nth-ing the file might not be random if the incoming data are ordered in some fashion.
For a file of 2 million to 3 million customers, I suggest setting the sampling routine so that it creates and maintains a control group of at least 50,000 customers, and not more than 100,000. Though this number probably will prove larger than required for statistical validity, it is best to err on the side of volume, at least at first. You’ll want the initial measures to be unassailable. With experience, the size of the control group might be reduced in the future.
And finally, it is crucial that the methodology to establish and maintain the enterprise-wide control group – and perhaps even knowledge of the existence of the control group itself – not be shared with the sales channel. If a salesperson knows who is in the control group and takes some special action to sell this previously untouched universe, the ability to deliver a true incremental measurement will be lost.
