Drudging Up the NumbersOnline tabloid reporter Matt Drudge recently posted an article accusing Web site audience measurement company Media Metrix of fudging clients' numbers.
And while the publisher of the Drudge Report (www.drudgereport.com) used incomparable numbers to support his claim, the incident sheds light on the murky world of counting eyeballs on an advertising medium that, while often touted as the most measurable of all time, is anything but.
The Drudge piece began with the incredulous headline: "MSNBC Website Has 10X the Viewers as Cable Channel?!"
It was prompted by a press release from MSNBC.com claiming that, according to Media Metrix, MSNBC.com's traffic for July ranked No. 1, ahead of CNN.com, and that it received a record-breaking 2.4 million unique visitors on July 21 - the day the bodies of John F. Kennedy Jr., his wife, Carolyn Bessette Kennedy, and her sister, Lauren Bessette, were recovered.
"[The figure was] so fantastic it represents an audience of more than 10 times that of the average viewership on MSNBC TV!" the Drudge Report said. During prime-time blocks on MSNBC TV in July, audience levels averaged slightly more than 220,000 viewers, according to Nielsen.
"Is it because the cable channel sucks, or because the Web site is so fresh? Or could it be because something called Media Metrix is passing off phony numbers to market researched-drenched executives who need to tell their bosses and shareholders that it is all working out?" the Drudge Report asked.
As it turns out, the answer is more complicated than that.
"The numbers just aren't comparable. Period," said Stacie Leone, director of marketing communications at Media Metrix, adding that Drudge did not contact the company about the story.
Indeed, as MSNBC.com executives point out, Nielsen's television ratings are an audience estimate at a specific time, while Web site ratings are an estimate of the number of people who logged onto the site during an entire day.
"We have 100,000 to 200,000 people on our site during any given hour. Over the whole day [on July 21], that came out to 2.4 million people. If cable bothered to measure how many heads - not households, heads - watch in a 24-hour period, they would come up with a much larger number," said Steve White, director of research, MSNBC.com, Redmond, WA.
Bad comparisons aside, Drudge points out that there's no way to know how many people visit a site without its server logs.
To arrive at its figures, Media Metrix monitors the Web surfing behavior of a panel of 50,000 Internet users and extrapolates.
"Media Metrix has no access to my logs," said Drudge. "They don't know who's visiting, and they're passing themselves off as if they do. It's a complete fraud based on who's subscribing."
However, server logs have shortcomings, too. Among other things, they include the activity of robots and spiders, which click on things but aren't human. Also, since cookies track computers, not individuals, people logging on from work and home are counted twice.
Meanwhile, Drudge points out that there are whole chunks of humanity that Media Metrix misses.
"There's no way of knowing if I have, say, 3 million readers in Israel," said Drudge.
But neither Media Metrix, nor its competitor, Nielsen/NetRatings, claim to measure visits from Israel. The reason: Not enough people who buy advertising care how many Israeli Internet surfers there are, and in order to measure for them, a panel must be set up for it beforehand.
"It's a matter of economics," said Manish Bhatia, vice president, interactive services, Nielsen/NetRatings, New York. He conceded that missing foreign visitors is becoming a larger issue, though, and as a result, Nielsen/NetRatings plans to begin accounting for users in Europe starting with Britain and Ireland sometime next year. Media Metrix also claims to have foreign initiatives underway.
Then there's the question of people logging on from businesses. It's much harder to get people to agree to be panelists from work. As a result, Nielsen/NetRatings misses them completely. Media Metrix said it began adding business users to its panel more than two years ago and that it currently monitors more than 7,000 at-work users.
Executives from Nielsen/NetRatings, however, question the quality of Media Metrix's business panel. Workplace users who accept monitoring software apparently tend to come from small and home offices and, therefore, are probably not representative of the overall workplace population.
Also, the Internet's astronomical number of Web sites - compared to television's relatively few channels - means Media Metrix and Nielsen/NetRatings are best for tracking the Web's top sites. The less popular the site, the more unstable the numbers get, according to Nielsen/NetRating's Bhatia.
In any case, Media Metrix and Nielsen/NetRatings offer two pieces of information difficult to get any other way: Web site demographics, and how a property stacks up against its competitors.
While a site owner can gather demographic data by requiring registration, panel data is a way to get it without driving people away. And without panel data, the only way to gauge a competitor is by looking at their visitor logs.
"I put [panels information] in the 'nice-to-know' category," said an online ad agency representative who did not want to be identified. "They let me tell clients whether or not a property is performing as anticipated in terms of volume."
Also, during campaign budget allocation, advertising buyers need to be able to speak the same language across different media.
"Every advertising campaign has a target reach and a target frequency," said Nielsen/NetRatings' Bhatia. "At the end of the day, advertisers want to know how many people they reached, and how many times they reached them."
And if a campaign can hit its Internet reach and frequency goals on one site or network, all the better. "But if you have to reach those same people on 50 different sites, it becomes too expensive to manage," said Bhatia.