What are Googlebot and Mediabot?

Search engines use software programs (so-called bots or spiders) to index web pages. These bots follow all links they can find on the Internet to index web pages for their search engines.

The name of Google's bot is Googlebot. If you see an entry with the name Googlebot in your web site statistics then Googlebot has visited your web site.

In addition to Googlebot, Google also operates another bot with the name Mediabot. Mediabot is the bot that is used by Google to index web sites that display AdSense ads. Google indexes these pages to find out which ads Google should display on these pages.

What's the problem with Mediabot?

Until recently, Google has denied that displaying AdSense ads on your web pages has any effect on your Google rankings. However, some webmasters noticed that web pages that have only been visited by Mediabot and not by Googlebot showed up in Google's search results.

Google's Matt Cutts confirmed at the Boston PubCon 2006 that the MediaBot pages can also be used for Google's main index:

"Pages with AdSense will not be indexed more frequently. It's literally just a crawl cache, so if e.g. our news crawl fetched a page and then Googlebot wanted the same page, we'd retrieve the page from the crawl cache. But there's no boost at all in rankings if you're in AdSense or Google News. You don't get any more pages crawled either."

How can this get you in trouble?

If you serve different pages to Googlebot and Mediabot then Google might ban your web site because you use cloaking on your web site. Cloaking means that you're showing different pages to users than to search engines.

As Mediabot analyzes the web pages you serve for your visitors it's easy for Google to find out if you present extra pages for Googlebot. If you do something special for Googlebot then Google knows that you're cloaking.

What does this mean to you?

You shouldn't use cloaking or any other unethical search engine optimization method for your web site. Search engines will detect it sooner or later and they will ban your web site from their indexes.

Don't try to cheat search engines. If you want lasting results, you should concentrate on ethical search engine optimization. Ethical search engine optimization leads to a symbiotic relationship between webmasters, search engines and web surfers so that everyone wins.


Copyright by Axandra.com
Web site promotion software