Technological advances in computer storage have increased the quantity and types of data acquired and stored by businesses. Demographic, census and transactional data have become a strategic asset to organizations in understanding customer behavior.
Analyzing this data, however, is a critical challenge to firms striving to stay ahead of the competition. Whether it’s acquiring customers or maintaining relationships, predictive modeling is the technique of choice to transform mountains of data into actionable predictions about customer behavior that can drive direct marketing programs and increase revenue opportunities.
Developed in the late 19th century, today’s standard predictive modeling approaches using linear and logistic regression have inherent limitations. As a result, business analysts struggle to keep up with the explosive growth of data. Many organizations are exploring new options to address these challenges. Only recently has computing power become cheap enough to make machine-learning technologies such as genetic algorithms an economically viable way to break this analytic bottleneck. Genetic algorithm technology helps analysts leverage more data attributes, build predictive models in less time than regression-based approaches and leave more time for tasks such as problem definition and results interpretation.
By using more advanced genetic algorithm technology, organizations can improve in customer targeting and developing DM programs to bolster their bottom line.
The conventional approach: regression analysis. The traditional data model development process is extremely time consuming. Delays in building models can harm program costs and response rates.
Depending on the nature of the business problem and the complexity of the data, model development can take weeks or even months to complete and may require more than 100 hours of hands-on analyst time. It is labor intensive, with the bulk of the effort (50 to 75 percent) spent on basic tasks such as preparing the data, selecting data attributes and evaluating the data. This takes away from the challenges of defining the business problem, interpreting the results or implementing the model.
Breeding better models. Combining the traditional statistical modeling approach with machine-driven genetic algorithms lets organizations rapidly build and deploy more accurate predictive models. The application of genetic algorithms to predictive modeling is based on Darwin’s principle of “survival of the fittest.”
An analogy may help explain how it works. Consider the common process used to produce a champion racehorse. Champions are usually the product of selective breeding and typically earn more money for their owners in stud fees than by winning races. Other horse breeders will pay significant money to have their mares produce a colt fathered by a champion. The logic is that the likelihood of producing a future winner is increased by breeding their mares with a known winner.
In genetic algorithm terms, the probability of winning a race is known as a fitness measure, and winning the Kentucky Derby is about the highest fitness measure a horse can have. If the mare is also a race winner, then they have bred two horses with high fitness measures, thereby increasing the chance that the colt will have a high fitness measure (probability of winning). However, a drawback in breeding horses is that from conception to birth takes about 340 days, so there is not a lot of output to review during a breeder’s career.
In a loose parallel to selective breeding, a genetic algorithm breeds an initial “generation” of random predictive models. Think of a model as a row containing a set of data attributes for a single customer, each in a different column. Each model is tested for fitness against user-defined criteria (i.e., likelihood of responding to an offer).
The process starts with a random selection of variables since at the outset the characteristics of a Derby winner versus a plow horse are unknown. The initial selection of models includes all available data. After an extensive “trial and error” process that may breed millions of models, over tens of thousands of generations, the genetic algorithm comes up with a Derby winner (king model) containing the characteristics (variables) necessary to predict the specific outcome.
Overall, better models (those with higher fitness measures) are more likely to be selected for breeding and will tend to survive while weaker models (plow pullers) will die out. By applying the basic genetic principles of cloning, mating and mutation, the genetic algorithms increase the diversity of models evaluated. Ultimately, the better “evolved” model “wins” (produces a Kentucky Derby winner) and represents the best solution to the business analytic problem.
Once the genetic algorithms are “done,” statisticians can apply their expertise (or magic) to produce a final model that is robust, explainable and satisfies business and regulatory policies.
Leverage more data attributes. Exploring more data attributes lets organizations produce better and more predictive scoring models. They can:
· Consistently generate predictive models with 10 percent or more improvements in key success factors such as response rate.
· Provide better insights into existing datasets by thoroughly exploring 100 percent of the available variables rather than an analyst-selected subset.
· Automate many of the activities in data mining, most of which are time consuming and mundane.
Using genetic algorithms technology, analysts have more time to develop insights and find innovative market opportunities because the software reduces manual data preparation steps and allows more time for problem definition and model interpretation.
