Google Helps Web Sites Get Crawled -- for Free

Share this content:
Known for its regular introduction of new products as much as for its regularly increasing stock price, Google is continually evolving. Recently the Internet giant unveiled another worthwhile product, Google Sitemaps, aimed at what it calls an attempt to expand its coverage of the World Wide Web and increase the freshness of its indexed Web pages.

The free service lets Webmasters place an XML-formatted file on their servers so Googlebot can better crawl all Web pages on a site. The new offering also lets Webmasters make updates to the Sitemap when changes to the Web site have been made so that the Google spiders will know to crawl the new content as well. At its best, this would mean that no Web page is left behind and possibly lead to increased search relevancy -- a benefit for searchers and the search industry, in general -- and better search placement for participating Web sites.

Google could be using this feature to play catch-up after a recent "study" of 10,000 rated searches, conducted by RustyBrick to test which engine (Google, Yahoo Search, MSN Search or Ask Jeeves) returned the most relevant results. So far, Yahoo is beating Google by a small margin. But when it comes to the biggest search engine in the world losing out to its main competitor, even a small margin can become a big problem.

Google Sitemaps became part of the discussion at last week's WebmasterWorld Search and Marketing Conference in New Orleans. One thousand attendees -- including myself -- were present to witness a not-so-friendly exchange of words between Google (Matt Cutts) and Yahoo (Tim Mayer) representatives over -- what else? -- search engine rankings. One good point made by Google engineer Cutts was that "Google Sitemaps is like paid inclusion for free." Yahoo, the only search engine to offer paid inclusion, allows a Web site to pay for spiders to conduct regular visits to the site, better ensuring that all, or at least the most important, pages on a site will be scanned and have a better chance to appear higher in the search engine results.

Yahoo has received some negative feedback for offering this service, with many claiming that paid inclusion basically lets Web sites pay for placement in regular search results, therefore skewing the whole purpose of a search engine -- to provide the most relevant, unbiased search results.

And though some could argue that the new Google Sitemaps potentially may skew search results because it is being offered free of charge to anyone, someone else could argue equally well that the new service will actually help ensure better search relevancy because it lets any Web site have its pages regularly updated, even if just for the purpose of "keeping up with the Joneses."


Next Article in Digital Marketing

Sign up to our newsletters

Company of the Week

We recently were named B2B Magazine's Direct Marketing Agency of the Year, and with good reason: We make real, measureable, positive change happen for our clients. A full-service agency founded in 1974, Bader Rutter expertly helps you get the right message to the right audience at the right time through the right channels. As we engage our clients' audiences along their journey, direct marketing (email, direct mail, phone, SMS) and behavioral marketing (SEM, retargeting, contextual) channels deliver information relevant to the needs of each stage. We are experts at implementing and leveraging marketing technologies such as CRM and marketing automation in order to synchronize sales and marketing communications. Our team of architects and activators plan, execute, measure and adjust in real time to ensure the strategy is working as needed and change things if it's not.

Find out more here »

Career Center

Check out hundreds of exciting professional opportunities available on DMN's Career Center.  
Explore careers in digital marketing, sales, eCommerce, marketing communications, IT, data strategies, and much more. And don't forget to update your resume so employers can contact you privately about job opportunities.

>>Click Here

Relive the 2017 Marketing Hall of Femme

Click the image above