Virtually any dynamic site can be indexed, given certain conditions. If a page can be constructed when it is fed the URL and it can output HTML that the spider can see, then it will more than likely get indexed. If however, the site is so dynamic that it literally creates pages on the fly, then it' likely will not get crawled. For example, if a site relies on a person's geo-location (by setting a cookie) before it can output the page, then it won't likely be crawled, because crawlers don't accept cookies.

If the page is javascript driven, it is unlikely to be crawled because the crawlers won't execute the java.

If, however, the pages do exist merely because of a URL string, that is a spider can request a URL and get a page, then it should get indexed.

So, as you can see, merely getting indexed can be a hurdle for some sites.

But indexing isn't the only thing. In order to drive traffic, a site needs to rank highly, and as it stands now, a static site will almost always out rank a dynamic site.

When I say static versus dynamic I mean a static site like www.somesite.com/page.asp and by dynamic I mean a site like www.somesite.com/page.asp?variable1&variable2.

The primary reason is that a search engine considers a static page more difficult to change than a dynamic page. Therefore it is "worth" more because if you are taking the time to change the page, then it is important to you, therefore it is more important to the engine. With a dynamic page, it is considered that you can easily change one element on the site, and the changes will easily propagate through the site. If you aren't willing to take the time to manually change static pages, then they aren't considered as worthy.

Of course I am generalizing and likely over simplifying the situation, but in essence that's what it is.

So, how do you make a search engine think that your dynamic site is more worthy? By implementing a URL rewriter.

Now before you run out and implement this tool, consider the following implications.

First, you need to know what web server is hosting your site. There are different URL rewriters for the many different hosts. Also, different rewriters perform differently. Some are more flexible, while others are fairly rigid. You will need to find one that best suits your needs, based on your current site, and hosting. Another consideration is in fact your hosting. Some webmasters and hosting providers will not implement changes to their servers which could affect other clients. Therefore they may not implement a URL rewriter just for you. So before you go any further, make sure you can actually implement the change.

Once you are sure you can, a good place to start is back on the web. See who your competitors are and get a feel for their site structure. Are they likely using rewriters? If so look at the URLs ranking. This is a good indicator of how you want your site to appear.

Are they HTML or ASP or PHP? Sometimes you can rewrite even the file extension (i.e. from PHP to HTM) Any competitive advantage you can come up with by looking at the sites currently ranking can help you determine what rewriter will work best for you.

Once you are pretty sure what you need to do, you will need to find the rewriter that suits your needs. As I said above, some are fairly flexible while others are fairly rigid.

I will give you an example. Let's say my site is:

www.somesite.com/page.asp?state=NY&city=NY&Country=US&Lang=EN-US.

Some URL rewriters are limited to replacing characters. In other words, you could replace all the "?" and "+" and "&" with "/" so the URL would look like:

www.somesite.com/page.asp/state/NY/city/NY/Country/US/LangEN-US.

Not bad - you have made it more indexable. The problem here is that your pages reside 8 folders deep. The general rule of thumb is that your pages should reside in the top 4 folders. It gets back to that whole worth and importance thing. Engines consider that if you are burying the content that deep, then it's not worth much to you, therefore it's not worth much to them.

So in this situation we need a rewriter which is more flexible. Lets say we have one that can replace all the "=" and "&" with "~" and the "?" with "/" We now have:

www.somesite.com/page.asp/state~NY~city~NY~Country~US~Lang~EN-US.

This is a little better, but still a little convoluted. While you've removed the excess folders, the URL string now makes no sense.

I hope you can see where I am going here - you need to find a URL rewriter which can be customized to suit your needs. You need to remove and/or replace as many variables as you can, making the URL more user and search friendly. Ideally, this URL should be rewritten as follows:

www.somesite.com/US~English~NewYork~NewYork/page.asp.

As you can see the URL is a little more user friendly - the user can see what the general area of the page is, and it is also more search friendly. Further, the pages appear in the top subfolder, one step down from the root making them "worth" more to the search engines. And also appear static, making them worth more.

I guess what I am trying to say here is, you need to really research what is needed - what are your requirements - as well as the options available. With an Apache server, for example, there is a built in URL rewriter, therefore there is no initial outlay of cash. But it does take some understanding in order to fully implement it properly.

There are also other free ones out there for a variety of platforms including Microsoft IIS.

Your success in implementing such a rewriter depends on your comfort with it and your understanding of what the outcome should be.

Rob Sullivan
Production Manager
Enquiro.com

Copyright 2004 - Searchengineposition Inc.