After dumping Google for its own algorithmic search technology two weeks ago, Yahoo said today it would offer to let advertisers pay to have their Web pages crawled for inclusion in its search index.
Through the paid inclusion program, Site Match, advertisers can submit their Web pages to Yahoo by an XML feed to be crawled by Yahoo's algorithmic search engine. Results are based on relevancy.
Yahoo built the program from the paid inclusion technology it acquired through its Inktomi and Overture acquisitions, combining Inktomi's technology with offerings from AltaVista and FAST.
Overture will offer programs for large and small advertisers. Large advertisers submitting more than 1,000 URLs for inclusion pay a fixed cost-per-click fee up to $1; smaller advertisers pay a CPC fee in addition to an annual fee. For most categories, Overture said, the CPC is 15 cents, with some popular categories as high as 30 cents.
Site Match listings will appear in search results on Yahoo's page, in addition to its search partners, including Microsoft's MSN.
By turning to paid inclusion, Yahoo further differentiates itself from Google, which has said it thinks such programs hurt the legitimacy of search results. Chris Bolte, vice president of strategic alliances at Overture, said Site Match would benefit searchers and advertisers.
“Our first and foremost priority is the user,” he said. “We're providing a value-added service, and we want to share in some of that value.”
Bolte said algorithmic search spiders simply cannot keep up with the Web's exploding growth. For sites that change content frequently, paid inclusion lets them ensure that search engines display up-to-date information instead of waiting for a search spider to crawl their sites. Yahoo would crawl a Site Match advertiser's sites every 48 hours, instead of a several-week period for others.
Fredrick Marckini, chief executive of search marketing firm iProspect, said Yahoo's program would further differentiate its search from Google's, giving it a better opportunity to return comparable search results to those Google culls from its index of over 4 million Web pages. Google does offer advertisers paid inclusion.
“They know that search is their business and the quality of the search results produces the traffic,” he said. “They learned that from Google.”
Nate Elliott, a Jupiter Research analyst, said search engines needed to be careful that paid inclusion did not give the appearance of tilting results in favor of those big enough to pay.
“Paid inclusion is better for the advertiser than the consumer,” he said.
Jupiter Research estimates the paid inclusion market could grow from $110 million this year to $500 million in 2006.
Smaller advertisers will need to pay more to be part of the Site Match program. Beyond the CPC fee, which Overture bases on bidding data from its paid search program, small advertisers will pay $49 for the first URL, $29 for the next nine and $10 for those up to 1,000.
Bolte said Site Match would, to some degree, rein in what he called the “cat-and-mouse game” between search engines and search optimization firms. Yahoo, Google and other search engines long have sought to keep their indexes free from outside interference, jealously guarding their algorithms from those hoping to game the results. The popularity of search, however, has led to more and more search spam — irrelevant Web pages included in algorithmic results by taking advantage of a search algorithm.
Unlike paid search listings, paid inclusion results will not be labeled differently from non-paid results, though Yahoo will include notice that some results were included in the index through Site Match. Bolte said about 99 percent of results would be non-paid.
“What users care about is relevancy,” he said.
Google implicitly criticized Yahoo's move, reiterating its opposition to paid inclusion. “The company has no plans at all of using paid inclusion,” said David Krane, a spokesman for Google. “Our view is a trusted relationship between a site and its user is a critical aspect of its success.”
Elliott said that though relevancy determined Yahoo's search rankings, it would be better if Yahoo disclosed to searchers which results were submitted through paid inclusion.
“There can be an appearance of impropriety,” he said.
For that reason, he doubts Google will be swayed to follow suit.
“Google has always been very adamant about this as a principle,” he said. “I can't see them changing.”
Under the banner of the Content Acquisition Program, Yahoo will let non-commercial sites feed their Web pages through for free, signing up National Public Radio, the Library of Congress and The New York Public Library, among others.
