Obviously we told them this was a no-no and that the search engines were smart enough to realize that this is duplicate, even if the URLs are different.

But there are other places where duplicate content could exist which could result in a penalization or banning from a search engine.

Aside from the obvious, such as situations mentioned above, or using hidden text, there are other places you may not have considered, but which the search engines may consider to be duplicate.

Remember that the engines rely on software to make most of their decisions. Therefore while you may not consider a page to be duplicated across the site there is a good chance that the algorithms used will.

For example, on many sites you will see a 'print this article/page' link. And that link is usually just an HTML page formatted to be printer friendly.

But did you know that this page could be considered duplicate content? It's true. That's because the content is the same as content found on the non-printer friendly page.

Now, the way to fix this is simple - you could either use a JavaScript pop-up to embed the link so that it isn't crawlable by search engine spiders (but remember that they are getting smarter, so this tactic will only probably work for a short time) or, the preferred way would be to create a separate subfolder for printer friendly versions and use your robots.txt to exclude this folder.

There's another source of potentially duplicate content which is common and could get a site penalized or banned. In fact, I just fixed such a problem on this site.

You see, with any large database driven site, sometimes coding mistakes happen (as was the case here) and you inadvertently create a duplicate content issue.

In our case, there are multiple paths to get to our articles. And generally, they reside in the articles subfolder. However upon performing routine maintenance on the site I came across a few pages where the pages were incorrectly referred to in the 'Articles' subfolder. And while both versions of the URLs worked, we were in effect generating duplicate content because articles would appear in both subfolders.

That means there would have been two URLs for any given article - hence the duplicate content issue.

This was a simple fix for me to do and I had it done within a few minutes, but who knows if it has been causing a problem with the engines? It is quite likely that this site has been penalized for these 'duplicate' articles.

These are just two examples of the types of content which can affect your search engine positioning. I'm sure there are more, but they give you an idea of what you need to watch for if your site is going to be successful in the search engines.

A great way to combat duplicate URL's is to use a free program like Xenu. You can run this program on a regular basis, looking for duplicate or similar URL's (as well as a host of other things like broken links) and manually verify that they are indeed unique.


Rob Sullivan
Head Organic Search Strategist
Enquiro Full Service Search Engine Marketing

Tags: Google MSN SEO Yahoo

Copyright 2003 - 2006 - Searchengineposition Inc.