Google Helps Web Sites Get Crawled -- for Free

Share this content:
Known for its regular introduction of new products as much as for its regularly increasing stock price, Google is continually evolving. Recently the Internet giant unveiled another worthwhile product, Google Sitemaps, aimed at what it calls an attempt to expand its coverage of the World Wide Web and increase the freshness of its indexed Web pages.

The free service lets Webmasters place an XML-formatted file on their servers so Googlebot can better crawl all Web pages on a site. The new offering also lets Webmasters make updates to the Sitemap when changes to the Web site have been made so that the Google spiders will know to crawl the new content as well. At its best, this would mean that no Web page is left behind and possibly lead to increased search relevancy -- a benefit for searchers and the search industry, in general -- and better search placement for participating Web sites.

Google could be using this feature to play catch-up after a recent "study" of 10,000 rated searches, conducted by RustyBrick to test which engine (Google, Yahoo Search, MSN Search or Ask Jeeves) returned the most relevant results. So far, Yahoo is beating Google by a small margin. But when it comes to the biggest search engine in the world losing out to its main competitor, even a small margin can become a big problem.

Google Sitemaps became part of the discussion at last week's WebmasterWorld Search and Marketing Conference in New Orleans. One thousand attendees -- including myself -- were present to witness a not-so-friendly exchange of words between Google (Matt Cutts) and Yahoo (Tim Mayer) representatives over -- what else? -- search engine rankings. One good point made by Google engineer Cutts was that "Google Sitemaps is like paid inclusion for free." Yahoo, the only search engine to offer paid inclusion, allows a Web site to pay for spiders to conduct regular visits to the site, better ensuring that all, or at least the most important, pages on a site will be scanned and have a better chance to appear higher in the search engine results.

Yahoo has received some negative feedback for offering this service, with many claiming that paid inclusion basically lets Web sites pay for placement in regular search results, therefore skewing the whole purpose of a search engine -- to provide the most relevant, unbiased search results.

And though some could argue that the new Google Sitemaps potentially may skew search results because it is being offered free of charge to anyone, someone else could argue equally well that the new service will actually help ensure better search relevancy because it lets any Web site have its pages regularly updated, even if just for the purpose of "keeping up with the Joneses."


Next Article in Digital Marketing

Sign up to our newsletters

Company of the Week

USAData helps businesses find new customers and grow their current customers through a combination of data and digital marketing services, and easy-to-use SaaS technology products. We enrich customer data so businesses can more effectively target and communicate with customers, and connect them with their best look-alike prospects through digital and traditional channels. We make it easy through simple, self-serve applications and APIs, as well as through full-service programs managed by our Data and Digital experts. 

Find out more here »

Career Center

Check out hundreds of exciting professional opportunities available on DMN's Career Center.  
Explore careers in digital marketing, sales, eCommerce, marketing communications, IT, data strategies, and much more. And don't forget to update your resume so employers can contact you privately about job opportunities.

>>Click Here

Relive the 2017 Marketing Hall of Femme

Click the image above