Fast-track, Foolproof In-Market Testing
Michael Goerz, account director, Hacker Group
Knowing your ROI is a powerful thing. ROI and all its marketing and sales pipeline bedfellows—cost-per-lead, cost-per-sale, etc.—allow direct marketers to optimize budgets, predict leads and sales, and sustain them. Everyone knows you're a winner when you hit your numbers.
But there are circumstances when performance history is not available to build your success roadmap. For example, when:
- You're running a startup that needs results, has limited budget and can't afford to lose even when testing
- Everything that was working isn't working anymore, so you need a plan B
- You're launching a new marketing channel or product
- You need to up your old model and start from scratch with fresh ideas
Regardless of the size of your marketing budget, fear not: You can prove what works best (and how well), across and within vehicles. You can learn what works in real-life marketing situations using multiple marketing media. You don't need to guess what will work; instead, you let the market dictate your strategy. And you do it so nimbly that you can read results and launch in days or weeks rather than months. This approach has worked for well-established brands like AT&T, helped Netflix scale membership, and has been applied to newer companies like Box and One Medical Group. Here are just a few of the things you might discover the answers to:
- How much should you be spending on vehicles like search, direct mail and display?
- What will give you a stronger cost-per-lead: direct mail or mobile?
- Should your online media placements be direct-to-site or ad networks?
- Should you be offering up a sweepstakes or only engaging people with related opportunities (for example, free trials)?
Here's how to start.
1. Brainstorm everything you want to test and the questions you want answered.
2. For the sake of more versus less testing, throw out statistical validity. If you test everything you can think of—or at least everything within reason and within your budget—you'll get good, directional learning. Testing more versus less allows you to spread your risk and avoid spending too much pursuing combinations of variables that won't work out. While individual test cells are not significant, you can likely achieve statistically significant results when you look at results in a larger, aggregated number.
3. Calculate a budget range for testing. Determine what you need to spend at rollout (e.g., if you need 5,000 sales at $150/each, you'll need $750,000 for rollout), which subtracted from your overall budget will determine what you have for testing. Then, working with your test budget, prioritize ideas, vehicles and all your other questions. Focus on where you can get your most bang for your buck (and how quickly).
Tip: One cost-saving measure—and a great way to mitigate risk—is to test in digital media first, to learn about targeting and offers quickly and less expensively, and then move into more scalable channels, such as DRTV and direct mail. Those strong trends in your results tell you what to do next.
4. Make sure you determine how to track test variables. Follow this simple rule of thumb: If you can't track it, don't test it.
5. Stay the course, check your biases at the door, and make sure you pay attention to the details.
6. Once you find a winner, validate your directional results. In phase 2, test using quantities that are likely to be statistically valid based on your initial results.
That's it. Now that you have a control (or champion), you can keep program performance high using a standard test methodology to test something every time you run a marketing campaign. If possible, test in 10% to 20% of your universe. That way you can learn each time you go out to your audience, while still maintaining results through using your control strategy with the remaining 80% to 90%. Then you only need to conduct this more radical type of testing strategy again if you need to.
Learning what to do next doesn't have to be complicated. To reduce your risk of failure and ensure that your next marketing dollars are spent in the most efficient way possible, begin with a disciplined test-and-learn approach and take it to the next level by testing multiple ideas simultaneously. With this approach, you can usually find a winning combination of vehicles and variables in the first test, and almost always by the second test event. These will be the keys to determining where to spend your rollout budget in a more informed way so you can maximize ROI and have a greater chance at hitting your goals.
Michael Goerz is an account director at Hacker Group