Direct marketers are forever saying, “Test, test, test.” Yet in actuality, many DMers do little or no testing at all.
Sure, big-volume consumer direct marketers test all the time. But many small and midsize companies say they lack the budget, time or universe size to make testing worthwhile.
Even among big DMers, testing often is limited to simple A/B split tests – headline A vs. headline B – or a price test among $99, $199 and $299. And that’s in direct mail. In space ads, A/B split testing is increasingly rare, as the majority of publications neither offer nor encourage it.
But thanks to the Internet, testing is undergoing a revival using a technique called “Taguchi testing.” If you’re already using Taguchi testing, you may get a few useful ideas from this article. If you are not, then what I am about to tell you could be the most important development in your Internet marketing this decade.
I am not a Taguchi expert, so I won’t go into the technical or statistical details, which I don’t really understand anyway. Instead, let’s discuss Taguchi testing on a high level. Taguchi testing is a system where, with a landing page or other online direct response promotion, you test not one but many variables – economically and in a relatively short time.
David Bullock, president of Results Squared, a consultancy offering Taguchi testing, said that his program typically involves testing these promotion elements: three pre-heads, six headlines, three subheads, three salutations (e.g., “Dear Home Builder” vs. “Dear Lumber Buyer” vs. “Dear Wood Trader”), three lead paragraphs, three visuals (e.g., a product photo vs. a photo of the inventor vs. a photo of a happy customer), three guarantees and three calls to action.
But you can test other elements such as bonuses, prices, even different lists of bullets in the copy. Other factors Mr. Bullock often tests are traffic source (e.g., organic search traffic vs. Google Adwords vs. e-mail) and what he calls “predisposition to purchase.” Predisposition to purchase measures how convinced the prospect is of the offer’s value before he even clicks onto the landing page. For instance, a visitor responding to an e-mail sent by a joint venture partner to his list of loyal readers has a greater predisposition to believe the message than a visitor who finds the page from a keyword search.
With specialized Taguchi testing software, each unique visitor to the site sees the landing page with a different combination of the elements being tested. Results are measured, tabulated and analyzed. Reports are generated to show which headline pulled best, which lead paragraph pulled best, which visual pulled best and so on.
The advantage is that you test multiple versions of many variables in landing page performance, not just two versions of one variable, as is usually the case with traditional A/B split tests. Therefore, conversion rises incrementally for each variable: e.g., a 20 percent lift in orders for the best headline, a 17 percent increase in conversion for the best lead and so on. By incorporating the winning versions of all variables tested in the final landing page, Taguchi testing can double, triple, even quadruple the conversion rate of your landing pages.
To do Taguchi testing, you need to write complete copy for your landing page along with the elements listed above: the six headlines, three subheads and so on. You can find Taguchi testing vendors listed under “Taguchi Testing” on the Vendors page of my site at www.bly.com.
In the “good old days,” it took weeks or even a couple months to get a valid reading on a direct mail test. Even then, it was usually just an A/B split of two different packages, prices or headlines. But because Taguchi testing is done online, you get the results much faster. Depending on the amount of traffic being driven to the URL and the conversion rates, Mr. Bullock said, a test can be completed and verified in three to five weeks.
The amount of traffic required also depends on conversion rate, because statistical validity of testing is based not on “number of pieces mailed” or amount of traffic online, but on number of responses or (for a landing page) number of sales made. Typically, you want at least 1,000 unique visits per test cell. Depending on the number of test cells, you need 12,000 to 20,000 visits for a full test.
