Mailers often use the expression “let the model do the work.” However, many marketers overlook that a great deal of work needs to go into planning a model for the analytics to be effective.
Success in modeling generally results from three things: realistic goals, the right data to model and staying involved with the project. In this article we’ll look at techniques you can use to ensure that your models do what they are supposed to do: Correctly identify the best names to market to.
Step one involves preparing a needs statement that should capture all the information you’d like to leverage in the project and define your criteria for success. By using the needs statement to provide your modeling service provider with as much upfront direction and information as you can, you’ll help ensure that the results produced by the model match as closely as possible to your stated goals.
Models provide tiers of names called deciles, ranks or segments for you to select. You’ll want to test the top model ranks that provide the volume and performance you need. As with any list, list history and experience are the best guides to choosing which prospect files might qualify for your plan when modeled.
Modeling also provides a few options that straight list selection does not. First, you can use a model to help your mailing accomplish certain goals, such as delivering upfront response or generating a certain average order size or net conversion/approval rate. Second, an analysis of modeling results lets you forecast an estimate of performance.
Another early step in model development involves giving your modeler a selection of customer names from your house file. This sample should reflect the types of individuals you want your new model to identify. If all goes as planned, your new model will locate prospects that resemble the consumers in your sample.
For example, say you provide your modeler with a sample of your direct mail paid responders from the past 12 months. The model will use this sample of your “best” customers to find and rank prospect names that most resemble these consumers.
With hard offers, where price and terms are spelled out, a mailing’s performance often can be gauged by upfront response. Take advantage of this by including both paid and non-paid direct mail responders in the sample you provide. The model will have more names for analysis and validation, and top-tier prospects will have higher gross response.
Another objective of your mailing likely will involve increasing profitability or minimizing the cost of customer acquisition. Since models home in on characteristics associated with the behaviors and transactions provided in your sample, a strong responding model also could produce extremely low conversion rates on the back end.
More fine tuning is possible when two models are applied together to the same universe to qualify prospects who look like good responders and likely to spend a lot. Results of two-step models often are documented in a matrix or cross-tab fashion, featuring rows of response model tiers and columns of back-end model tiers. As a guide to help you select the best groups of names to mail, cells in the table contain performance measures and the prospect universe available for each combination of model ranks.
While lists pulled via straight selection qualify for a mailing based on profitability cutoffs, a modeled prospect list provides a means for selecting various tiers or ranks within a prospect universe – an advantage that lets you maximize response, conversion or both.
In this stage of the project, your modeler will supply you with a profile that illustrates the characteristics of your sample group of customers compared with a sample of all prospects in the database to be modeled. Be sure the profile confirms what you know about your customers’ demographics and lifestyles along with their transaction and channel preferences and patterns.
Remember that clear differences between prospects and existing customers is good and tells you that your modeling will have a strong chance for success.
The profile will lead to a discussion of market segments, as you may find you have a few groups of customers defined by, for example, gender, life stage or lifestyle. A brief discussion may be all that’s needed to nail down the prospect universe definition for the group you want to reach.
At this stage you are defining your target segment in broad terms to prequalify names within the prospecting database. In certain cases, more refined analyses can reveal distinct markets to be modeled separately.
Once the target market is defined, your modeler will begin the actual model build, with your sample of best customers in hand.
To help modelers sift the database variables that best fit your customer sample, analysts use generally accepted rules for modeling consumer behaviors such as response and conversion. A model provides weights for variables that, when combined, capture the patterns that identify contrasts between your customers and the “typical” consumer in the prospect universe.
Among the more important rules modelers use when choosing their final models is to limit combinations of very highly correlated variables. They also assess how quickly and smoothly the top model ranks identify parts of your customer sample and see whether the model can do the same when applied to a random subset of your sample that’s been reserved for validation scoring.
Remember to request documentation of the key features of the model for your files. You’ll want to review this after results come in. In particular, look for the list of the model variables and their relative importance or weight. The variables will include some of the distinctive characteristics you learned about your customer sample. A profile of the top ranks tells the complete story about what prospects the model will deliver to your campaign.
Performance reports, or gains charts, show the strength of a model and typically express this strength as an index. These reports may also summarize response and performance statistics for each model tier.
Though performance reports are important, there is no guarantee when it comes to forecasting what will happen when the top-tier names are selected for a mailing. However, by engaging in the steps above, you greatly improve the odds of having your model test succeed. Here are two tips to help you forecast realistic results:
· Assume you plan to mail an Nth of the prospect universe without modeling. Then develop an estimate of performance for this hypothetical mailing by drawing upon your list history. Finally, ratchet up this estimate by 60 percent of the gain reported in the model performance reports to create the forecast for your actual modeled selections.
· Use results from past models to help modify your projections, and, if possible, give the new model high priority in your merge/purge to help prove it is doing its job.