In 1921, a young copywriter named John Caples sat down and wrote “They Laughed When I Sat Down at the Piano” for the U.S. School of Music. The headline was a huge success. And Caples knew precisely how successful it was because the advertisement he wrote contained a coupon – marketing’s first scientific measurement tool.
Caples was an early champion of testing and his book, “Tested Advertising Methods,” added an element of science to selling. The book is still in print and continues to draw raves from Amazon reviewers.
The power to measure campaign results has exploded in the past few years. And as the Internet continues to level the playing fields of marketing, testing e-mail campaigns is a bigger deal than ever.
“If you can’t measure it, you can’t manage it,” management guru Peter Drucker said. We’d only add beware of spending time and money measuring parameters which won’t be actionable.
So what should e-mail marketers test? Headlines, subject lines, copy, mailing lists, offers, timing and any other variable they believe they can use.
Before e-mail testing was available, it was often impractical to test so many variables because by the time tests were put in the field and responses analyzed, conditions in the market had changed.
Now, e-mail can be sent and results analyzed almost overnight. It’s fast and 50 times cheaper than the United States Postal Service. And, e-mail testing can produce good information that can be used offline, in sales, marketing and advertising.
OFAT versus factorial design
The classic A/B split, or OFAT (one factor at a time), still makes sense. But factorial design, in which two or more variables – price and subject line, for example – can be measured in the same test, produces more information and is ideally suited to e-mail testing.
Factorial design tests multiple factors simultaneously and beats OFAT in several ways. You learn more, faster. You save time and money because you won’t need to send as much mail to test more than one factor. And factorial design provides richer information – not just the effect of each factor, but the interaction between the factors.
If there’s a problem with factorial design it’s that you need someone on hand who understands the math and can interpret the results, a talent which can be outsourced if your statistics are rusty.
But in the end, accountability for marketing spend is here to stay and e-mail testing needs to become a regular line item in your marketing budget. The dividends in revenue and customer insight simply can’t be ignored.
