A/B Testing Helps RunKeeper Keep Pace with its Customers

Share this content:
A/B Testing Helps RunKeeper Keep Pace with its Customers
A/B Testing Helps RunKeeper Keep Pace with its Customers

Mobile is an essential part of a healthy marketing regimen.

“We're seeing many organizations where mobile represents 50 to 60% of the traffic,” says Bernd Leger, VP of marketing for Localytics, a marketing analytics platform for mobile and Web apps. “If that's the case, you have to have a different mind-set of ‘how do we engage users on mobile, and what's that experience like?'”

Jon Gilman, product manager of fitness technology company RunKeeper, admits that apps are the “bread and butter” of the brand (carbs aside). More than 25 million organically acquired users have downloaded RunKeeper's flagship iPhone and Android apps to track their workouts, share their successes, and integrate their activity data into other health-related apps.

Traditionally, RunKeeper focused on the cardio activity market by offering tracking and integration capabilities for biking, running, and walking workouts. But after listening to its users, the company discovered that many people were participating in other forms of exercise, such as downhill skiing and meditation. So RunKeeper decided to redesign its app to enable those engaging in other fitness activities to track their progress.

“What we wanted to do was start expanding the base of RunKeeper beyond just running, walking, and biking and move to activity types like yoga [or] strength training," Gilman says. "Though we knew those activities were an important piece of our users' fitness lifecycle, they weren't [being] captured."

But RunKeeper didn't want to jump the gun and overhaul its app without knowing how that might affect metrics. So the company decided to focus on its starting screen first. RunKeeper then teamed up with mobile A/B testing organization Vessel and Localytics this past July and conducted a one-month A/B test to see if people would be more likely to log fitness activities other than running if given the option.

To conduct the test, RunKeeper randomly assigned Android users to one of two groups: A and B. Group A users saw the app's old starting screen, which featured a form listing users' activity type, attributes associated with that activity (such as whether they shared their workout), and a start button. RunKeeper automatically set the activity type to running. If group A users wanted to select a different activity, they would have to change it manually. For group B users, however, RunKeeper didn't automatically select an activity. Unlike group A, group B saw a new screen featuring eight different activities to choose from.

RunKeeper saw a 235% increase in non-running activities logs when it showed consumers the group B version. Furthermore, RunKeeper didn't experience a decline in GPS-tracked walk, run, or bike activities. Since conducting the A/B test, RunKeeper has improved entry flow by eliminating extra starting page steps, and increased the number of people who complete an activity after manually logging it.

While A/B testing can seem like an intimidating exercise, Gilman says it can lead to big benefits.

“It's OK to try testing big things,” Gilman says. “The benefit of having an A/B testing and analytics framework is that you can make wholesale changes to an experience that you think are risky or may not work. But, being able to measure and quantify the exact impact of those changes against your old experience gives product managers [and] designers a lot of flexibility to try new things and think outside the box.”

DMNotes is DMN's around-the-clock blog. Yes, a blog in 2016.

Bookmark this section and follow our RSS Feed here


Next Article in Mobile Marketing

Sign up to our newsletters

Company of the Week

Since 1985, Melissa has helped thousands of companies clean, correct and complete contact data to better target and communicate with their customers. We offer a full spectrum of data quality solutions, including global address, phone, email, and name validation, identify verification - available for batch or real-time processes, in the Cloud or on-premise. Our service bureau provides dedupe, email/phone append and geographic/demographic append services for better targeting and insight. For direct mailers, Melissa offers easy-to-use address management/postal software, list hygiene services and 100s of specialty mailing lists - all with competitive pricing and excellent customer service.

Find out more here »

Career Center

Check out hundreds of exciting professional opportunities available on DMN's Career Center.  
Explore careers in digital marketing, sales, eCommerce, marketing communications, IT, data strategies, and much more. And don't forget to update your resume so employers can contact you privately about job opportunities.

>>Click Here

Relive the 2017 Marketing Hall of Femme

Click the image above