There is gold in them thar open-ended survey responses. As opposed to close-ended questions where a respondent is given a finite selection of answers (very satisfied, satisfied), open-ended questions have no specific answer and respondents are free to share their thoughts in their own words.
Direct marketers conducting or using surveys include open-ended questions to gather prospects’ or customers’ opinions on offers, to acquire names, for quality control or just to break the ice.
Fairytale Brownies understands the value of seeking customer opinion through surveys. It recently included a survey card in each package of its brownies. If customers answer a few questions – including an open-ended question, such as “How did the brownies taste?” – and return the postage-paid reply card within 30 days, they are entered into a monthly drawing for a dozen more Fairytale Brownies. In the first month alone, the company received more than 2,500 reply cards.
Company officials initially set up the survey for quality control purposes, but soon learned that the responses, particularly to the open-ended question, provided them with useful information such as new product ideas. The survey also serves as a vehicle for building customer relationships.
Respondents are asked whether they would like a representative from Fairytale Brownies to call them, and if they say “yes,” someone does. Co-founders Eileen Spitalny and David Kravetz also were able to glean some useful testimonials from the responses. One came from the quality assurance crew at Hershey’s Chocolates, which evaluates chocolate for a living. It gave Fairytale Brownies a five-star ranking.
Though Fairytale Brownies learned the value of open-ended survey responses, many surveyors choose to ignore open-ended questions because, unlike multiple-choice questions, open-ended answers are difficult to analyze. It’s time-consuming, tedious and expensive to code or categorize responses. In other words, making use of open-ended questions is a significant headache.
The standard method of analyzing open-ended survey responses is to assign codes (usually numbers) to the different re-sponses encountered. The person analyzing the open-ended responses develops a set of response categories that adequately represents the answers given. Then, the number of answers that fall into each category is determined.
If survey participants were asked “What features of a product most satisfy you?” their answers probably can be represented on a continuum of response options that runs from “very satisfied” to “not at all satisfied.” Coding is typically done after the survey is completed because the list of possible answers can only be generated after the survey has been conducted. There are, however, cases when the coding can start before the survey ends. This can be done manually or by using one of a number of software programs that can assist in accelerating the coding process. The coding is critical because it enables the responses to be statistically analyzed.
Even with technology, analyzing open-ended survey responses is still a labor-intensive process. The main reason it’s so difficult is that human speech, by its nature, is unstructured. Many words have several meanings, and there are countless ways to express a single idea. It is difficult to manually detect the semantic closeness of every possible pair of sentences as well as take into account all the possible spelling mistakes for every word.
New technologies can assist in categorizing text responses that combine manual techniques with advanced linguistic technologies to extract and classify key concepts from within survey text. They are based on the field of study known as Natural Language Processing, also known as computational linguistics. NLP technologies analyze text as a set of phrases and sentences whose grammatical structure provide a context for the meaning of the text response, avoiding the need to read every response.
This approach equates terms that are used in similar contexts. For example, NLP would equate terms such as “executive,” “manager” and even “mgr,” if they are used in similar contexts, whereas a non-linguistics-based solution may not equate manager with executive if it weren’t specifically stated. NLP technologies also can interpret the tone of text and distinguish between a positive and negative response.
A customer comment about “new phone” may be distinct from a related concept, “new phone ASAP,” which implies some negativity and urgency. The understanding of text cuts through the ambiguity of text, making linguistics-based text analysis the most accurate approach.
The ability to personalize offers and better understand customers is a key factor in successful direct marketing. And, the best way to find out what is in their heads is by asking them. n
