Why have search engines problems with dynamically created pages?

Many search engines don't like dynamically created pages because they might get in an infinite loop if they follow some dynamically created pages.

Google has an official statement about this in its webmaster guidelines:

"If you decide to use dynamic pages (i.e., the URL contains a "?" character), be aware that not every search engine spider crawls dynamic pages as well as static pages. It helps to keep the parameters short and the number of them few."

What did Google change?

Until November, Google had the following statement in its webmaster guidelines:

"Don't use "&id=" as a parameter in your URLs, as we don't include these pages in our index."

This entry has been removed from Google's guidelines. Here's the official statement from Google's blog:

"(Google now indexes) URLs that contain that parameter. So if your site uses a dynamic structure that generates it, don't worry about rewriting it -- we'll accept it just fine as is.

Keep in mind, however, that dynamic URLs with a large number of parameters may be problematic for search engine crawlers in general, so rewriting dynamic URLs into user-friendly versions is always a good practice when that option is available to you.

If you can, keeping the number of URL parameters to one or two may make it more likely that search engines will crawl your dynamic urls."

What does this mean to your web site?

If you have a dynamically created web site that uses only one or two parameters to create web pages then it's likely that Google can index your web pages without problems.

If you use more parameters then you should rewrite your URLs. Rewriting your URLs makes sure that most search engines will be able to index your site.

If you cannot rewrite your URLs then it helps if at least one stable link points to a dynamically created page. If a web page can be found through a direct link then it is more likely that search engines index that page.

If you want to find out how Google's search engine spider sees your web pages, take a look at IBP's search engine spider simulator. The spider simulator is available in IBP's free demo version and it allows you to check your web pages with any search engine spider.


Copyright Axandra.com
Web site promotion software