On September 22, the Wall Street Journal “broke” a story in which they reported that Facebook “vastly overestimated average viewing time for video ads on its platform for two years.”
Facebook’s “average view time” was, according to Facebook’s critics, “artificially inflated…inaccurate data” because it doesn’t count views of videos that are less than three seconds long. In Facebook parlance, a video is only counted as “viewed” if someone watches for three seconds or longer. This is to eliminate any “views” triggered by automatic scrolling or mistakes.
The “scandal” led to the invariable follow-ups from marketing and tech publications, including DMN, and an apology and restatement from Facebook.
About a month ago, we found an error in the way we calculate one of the video metrics on our dashboard – average duration of video viewed. The metric should have reflected the total time spent watching a video divided by the total number of people who played the video. But it didn’t – it reflected the total time spent watching a video divided by only the number of “views” of a video (that is, when the video was watched for three or more seconds). And so the miscalculation overstated this metric. While this is only one of the many metrics marketers look at, we take any mistake seriously.
The WSJ’s story—and Facebook’s above mea culpa—treated the “mistake” as if it were a crisis. In reality, this treatment is puzzling; not only had Facebook clearly disclosed what it was doing in 2014, but it was also doing the right thing for marketers.
Now, Facebook has rectified the “mistake” for the worse.
Facebook announced its analytics methodology when it rolled out its video-ad program in 2014, nearly 29 months before the WSJ’s supposedly breaking story (bolded emphasis ours):
Introducing Video Metrics…
Today, as a Page owner, you can only see how many people started watching your video. When the new metrics roll out over the coming weeks, you will also see information like video views, unique video views, the average duration of the video view and audience retention. These new metrics are designed to help you learn what’s resonating with people and determine how to more effectively create and promote your videos on Facebook.
See video views and the number of people that watched your video
We’ll show both the total number of video views and the number of people who watched your video. A “video view” is defined as a view of three seconds or more and will appear for all videos, including those that come to life as people scroll through News Feed. We’ve also renamed the “video plays” metric “clicks to play video.” These register after a person has clicked to play a video and it has started.
The problem, as critics put it, is a problem of omission. While Facebook very clearly states that it’s only counting views as any video-play event that lasts longer than three seconds, it does not go out of its way to explicitly beat readers over the head with the fact that this definition of a “video view” applies equally to the calculation of average duration of video views.
If you knew what views were, so Facebook assumed, you likely knew that calculations of the average duration of a video view would follow that same definition.
Meanwhile, the WSJ and other journalists-cum-critics have implied that ad buyers may have made bad decisions because they had no idea that Facebook’s definition of a video view applied to Facebook’s metrics based upon, um, video views.
Leaving aside the question of whether Facebook’s disclosure on a blog was enough, the decision of not counting those views in average duration, for which they apologized, is just smart analytics.
In fact, Facebook faced criticism over a year ago because—according to last year’s critics—three seconds was too short of a cutoff point.
“Before [the 30-second mark], I’m willing to bet that 80% of viewership is simply the [e]ffect of Facebook’s auto-playing of my video as people scroll through their feeds,” wrote vlogger Hank Green in a July 29, 2015, Medium post provocatively titled “Is Facebook Lying about Video Views?” “Facebook … is explicit [in clarifying that] they count it as a view after three seconds of a silent, auto-played video.”
“While there is no broad industry standard for view measurement, three seconds is one common choice, and gives us a consistent metric for all video on Facebook,” rebutted Facebook product manager Mark Pakes five days later. “If you have stayed on a video for at least three seconds, it signals to us that you are not simply scrolling through feed and you’ve shown intent to watch that video.”
Facebook’s research supports Pakes’ claims. A 2015 Facebook-commissioned Nielsen study showed that people who saw even a single, sub-second moment of a video ad (i.e., before the ad was “viewed”) were more likely to recall both the ad and the brand – and even more likely to purchase. Nonetheless, Facebook and Nielsen both further found that these three factors increased substantially for views lasting longer than three seconds—as much as tripling to quintupling from momentary sub-second views to three-second views.
Facebook further concluded that there’s more to measuring campaign value than view counting – a discovery that some marketers have already observed.
To be fair, Facebook’s putative reporting failure represents a missed opportunity. Better practice would have been for the social network to release two sets of metrics – one based upon its (well advertised) “video view” definition, and another based upon all actual views of video. From there, Facebook could have explained the difference – and why its way of measuring video-view data is thereby better than “standard” methods used by its competitors (suggesting that all those other providers may be clueless at best or misleading at worst).
Good marketers, however, understand that if Facebook had used the WSJ’s preferred methodology that that might be “artificial inflation” of metrics. The marketers and marketing platforms that count ultra-short views and factor them into their metrics may be misleading others.
Now that Facebook will start counting these in average duration, marketers will have just a different set of skewed numbers.
Nobody should be beholden to reporting sub-optimal statistics just because everyone else does— or be punished for “misleading” by not reporting data in the same misleading way as everyone else. Industry custom is one thing, but this scandal is more artificial than any of Facebook’s metrics.
