How to use ad creative as part of a multivariate test

Share this content:
Ian Lurie
Ian Lurie

Multivariate testing has made it into the mainstream: tools like Google Website Optimizer and Widemile's Page Opti­mizer make sophisticated tests relatively simple.

But it causes many search marketers to focus on landing pages and to mistakenly ignore their ads. Many other search marketers treat the ads as a separate test. That's a mistake, too.

For the best results, include the ads and the landing pages in the same multivariate test (MVT, for short) matrix. Why bother? Creating an MVT matrix is already a pain in the rear. After their first month or so working with me, most clients flinch when I mention one. So why add yet another variable?

A matrix is just a grid showing each variable in your test. So, if you have two possible headlines and two possible product pho­tographs, you'd create a two-by-two matrix. If this is over your head, don't feel bad, but do a bit of reading on multivariate landing page testing before you go any further in this article.

Let's say I've got four pos­sible landing page “recipes” — all part of a single multivariate test. I've also got two ads. If I ignore the fact that I've got two ads, then I have four possible recipes. I run my test. Visitors see each landing page recipe 25% of the time. One recipe outperforms the others, and I choose that. I'd be wrong, though, because I didn't include the ads in the recipes. If those ads played a part in the result, then I may choose the wrong landing page as my winning recipe.

Factoring in the ads gives me eight recipes, not four. I may have missed ad/lander combinations that generate a very high conversion rate; achieve very high per-sale value; or appear to achieve a high conversion rate, but only do so for one ad, while performing horribly for the other.

A pay-per-click ad does more than make someone click. It sets expectations, establishes tone and qualifies your audience. Ad text, the promise you make in it, and the circumstances under which someone clicks that advertisement will affect what they do when they see your landing page. Ignoring that leads to skewed results and a lot of head shaking later on.

What now? You need to plan your tests around landing page configurations and ad creative. If your test will use only a small trickle of visitors, you'll probably need to limit your ad cre­ative. That will reduce the number of variables, so that you can achieve a statistically significant sample with less traffic.

If your test has a nice flow of traffic, test various ads and put them in your test matrix. You'll learn more, generate better results and leave fewer customers waiting at your doorstep.

Ian Lurie is president of Portent Interactive. Reach him at


Next Article in Marketing Strategy

Sign up to our newsletters

Company of the Week

Since 1985, Melissa has helped thousands of companies clean, correct and complete contact data to better target and communicate with their customers. We offer a full spectrum of data quality solutions, including global address, phone, email, and name validation, identify verification - available for batch or real-time processes, in the Cloud or on-premise. Our service bureau provides dedupe, email/phone append and geographic/demographic append services for better targeting and insight. For direct mailers, Melissa offers easy-to-use address management/postal software, list hygiene services and 100s of specialty mailing lists - all with competitive pricing and excellent customer service.

Find out more here »

Career Center

Check out hundreds of exciting professional opportunities available on DMN's Career Center.  
Explore careers in digital marketing, sales, eCommerce, marketing communications, IT, data strategies, and much more. And don't forget to update your resume so employers can contact you privately about job opportunities.

>>Click Here

Relive the 2017 Marketing Hall of Femme

Click the image above