It sounds like a good idea: Survey customers to find out what they want and then let their answers shape your product development and marketing.
But in reality, it’s often a bust.
A case in point: According to an article in the Aug. 1 issue of BusinessWeek, the Gap began an intensive program of focus groups, surveys and other market research in 2002. But in the fiscal quarter ending April 30 of this year, sales fell 4 percent, and analysts expect them to drop another 2 percent for the quarter ending July 30. The reason: Eight former employees and two analysts say the Gap “has shifted too far toward research and away from the instinct and emotion favored by many successful clothing merchandisers.”
I see an insidious trend today, especially among traditional marketers turning toward direct marketing: They believe they can, through customer surveys, confirm that consumers will respond to the product, price, offer, headline and copy before producing and mailing the promotion.
They mistakenly think that if they get enough survey responses, they are guaranteed a winning promotion simply by stringing the consumer responses together into copy. However, experienced direct marketers know that a huge gap exists between what consumers say they will buy and what they do buy.
For this reason, the only accurate way to determine what will sell is through testing: selling a real product with a real offer to real consumers who vote their preferences not by answering a survey but with their purchases.
“In my 10+ years experience in DM publishing, I’ve found that customer surveys can be very misleading,” wrote a poster to my blog at www.bly.com/blog. “Recipients tend to give answers they think you want to hear. Case in point: a company spent over $100,000 developing a product that customer survey respondents said they would purchase. At the end of the day, they didn’t purchase. And the company lost a bundle. My advice: treat survey responses as minor gauges of interest. Tread lightly. And test, test, test!”
“A lot of the time customers really don’t know what they want,” wrote another. “The masses simply clamor about waiting for the next best thing. Not exactly the best group of folks to ask. This kind of thing permeates video game design discussions quite frequently. Everyone wants to predict what the next blockbuster title will be based on current trends, when in reality, you could spend millions of dollars on focus groups of current title enthusiasts and come up with an ultra-generic flop.”
“A few years ago an insurance company hired me to create a direct mail package. When the design and copy were completed to draft form, my client had a brilliant idea: ‘Let’s show this to a focus group of potential customers,'” wrote copywriter Steve Slaunwhite. “Nothing I said could dissuade him. Based on feedback from the group, my client made sweeping changes to the package, which included eliminating all references to the word FREE. ‘Too trite,’ they said. The package was mailed. And it flopped. What went wrong? Not the focus group, according to my client. ‘The focus group can’t be wrong,’ he asserts. ‘After all, they’re the customers!'”
One reason that customer surveys are particularly ineffective for evaluating promotional copy is that consumers like to think they are immune to direct marketing techniques. A focus group will loudly proclaim that your “buy one, get one free” offer won’t work with them because they don’t like gimmicks. Then, in an actual test, that offer will perform like gangbusters, beating the pants off the straightforward offer the focus group members all said they preferred.
So the next time someone at your company says, “Let’s do a customer survey,” keep these facts in mind:
• Survey results are not terribly accurate predictors of what products and offers will sell, the price consumers will pay or the headlines, copy or promotions they will respond to.
• Surveys, in particular focus groups, are useful for discovering the language that consumers use to talk about a particular product, need or problem.
• The Internet allows large numbers of consumers to be surveyed quickly and at low cost, with online tools such as www.surveymonkey.com.
