Google Helps Web Sites Get Crawled -- for Free

Share this article:
Known for its regular introduction of new products as much as for its regularly increasing stock price, Google is continually evolving. Recently the Internet giant unveiled another worthwhile product, Google Sitemaps, aimed at what it calls an attempt to expand its coverage of the World Wide Web and increase the freshness of its indexed Web pages.


The free service lets Webmasters place an XML-formatted file on their servers so Googlebot can better crawl all Web pages on a site. The new offering also lets Webmasters make updates to the Sitemap when changes to the Web site have been made so that the Google spiders will know to crawl the new content as well. At its best, this would mean that no Web page is left behind and possibly lead to increased search relevancy -- a benefit for searchers and the search industry, in general -- and better search placement for participating Web sites.


Google could be using this feature to play catch-up after a recent "study" of 10,000 rated searches, conducted by RustyBrick to test which engine (Google, Yahoo Search, MSN Search or Ask Jeeves) returned the most relevant results. So far, Yahoo is beating Google by a small margin. But when it comes to the biggest search engine in the world losing out to its main competitor, even a small margin can become a big problem.


Google Sitemaps became part of the discussion at last week's WebmasterWorld Search and Marketing Conference in New Orleans. One thousand attendees -- including myself -- were present to witness a not-so-friendly exchange of words between Google (Matt Cutts) and Yahoo (Tim Mayer) representatives over -- what else? -- search engine rankings. One good point made by Google engineer Cutts was that "Google Sitemaps is like paid inclusion for free." Yahoo, the only search engine to offer paid inclusion, allows a Web site to pay for spiders to conduct regular visits to the site, better ensuring that all, or at least the most important, pages on a site will be scanned and have a better chance to appear higher in the search engine results.


Yahoo has received some negative feedback for offering this service, with many claiming that paid inclusion basically lets Web sites pay for placement in regular search results, therefore skewing the whole purpose of a search engine -- to provide the most relevant, unbiased search results.


And though some could argue that the new Google Sitemaps potentially may skew search results because it is being offered free of charge to anyone, someone else could argue equally well that the new service will actually help ensure better search relevancy because it lets any Web site have its pages regularly updated, even if just for the purpose of "keeping up with the Joneses."


Share this article:
You must be a registered member of Direct Marketing News to post a comment.
close

Next Article in Digital Marketing

Follow us on Twitter @dmnews

Latest Jobs:

Featured Listings

More in Digital Marketing

News Corp. Chief Brands Google an 'Unaccountable Bureaucracy'

News Corp. Chief Brands Google an 'Unaccountable Bureaucracy'

Robert Thomson warns the EU that an antitrust deal with Google will lead to a decrease in competitive options for marketers and an increase in piracy.

Video's Going Programmatic, New Study Contends

Video's Going Programmatic, New Study Contends

Some 60% of brands now buy online video programmatically, according to a study from AOL's Adap.TV.

Dollar Growth Rate of Video to Peak This Year

Dollar Growth Rate of Video to Peak This ...

It will increase by 56% to $6 billion, then taper off due to growth in inexpensive mobile placements, says a new study.