What is a sitemap?

In 2005, Google launched the Sitemap Protocol. This protocol allows a webmaster to let search engines know about pages on a site that are available for spidering/crawling - i.e indexing.

A Sitemap is usually a single file in XML format. XML stands for Extensible Markup Language (XML). Before your eyes start rolling back in your head, you don't need to learn XML, or even HTML for that matter to create an XML sitemap; but we'll get to that shortly.

In addition to letting a search engine know what pages are on a site, additional information can be added also; including when the page was updated, its importance and how often it changes.

Basically, a sitemap is a directory for search engines, allowing the engine to gather a stack of information about your site in one convenient location.

Sitemap myths

Having a sitemap will not necessarily bolster your search engine rankings, nor will not having one prevent you from ranking. If your site is constructed properly, search engine spiders should not have any problem crawling/indexing your site.

Sitemap advantages

Some search engine companies such as Google have plenty of resources for sending out robots to crawl the web. On some of my sites, Googlebot is active to a degree basically 24/7/365. Other engines, and particularly Ask.com at the moment, have less active spiders; so you need to make the most of them while they are on your site.

Why am I mentioning Ask.com? Well, it's rapidly gaining popularity and is now the number 4 engine in the USA, rapidly closing in on MSN Search.

The less time a robot spends looking for pages, the more it can retrieve. A sitemap tells the robot "grab this, this and this, don't forget about this.. oh, and this too, but this is more important.. eat up!"

By adding a sitemap, you'll:

- speed up page discovery.
- (hopefully) increase the number pages indexed.
- assist the search engine spiders in prioritizing page retrieval

Sitemaps are particularly useful for large sites where there may be many categories and internal links to some pages buried deep that search engines may overlook.

Creating an XML Sitemap

As mentioned, creating a sitemap isn't difficult and you really don't need a lot of tech-savvy to do so given some of the great tools around now. One I've been trying out is XML Sitemaps. They offer 2 versions; one is a free XML sitemap generator where you can create a sitemap in just 4 steps:

- Enter your domain and some optional info
- Click start, wait a minute or two while it churns through your site
- Grab the sitemap file and save it as sitemap.xml in the root folder of your site
- Add the following line to your robots.txt file

Sitemap: http://www.example.com/sitemap.xml
(where "example.com" is your domain name)

If you're not sure on what a robots.txt file is, read this article.

The free sitemap generator will index up to 500 pages per site. XML-Sitemaps also offer a very economical premium service that will generate sitemaps for unlimited numbers of pages, plus has some extra features including automation of generation and pinging, plus special reporting tools.

Submitting your sitemap

If you include the above in your robots.txt file; the next time that the Google, Yahoo, MSN Search or Ask.com search spiders pay a visit, they should pick up on it. If you wish to manually submit a site map (aka as "pinging"):

Google

http://www.google.com/ping?sitemap=http://www.example.com/sitemap.xml
(remember to replace "example.com" with your own domain name):

Yahoo

Go to:

https://siteexplorer.search.yahoo.com/submit
.. and enter the URL of your SiteMap in the "Submit Site Feed" box

MSN Search

As far as I know, there's still no direct sitemap submission tool for MSN, but you can submit your sitemap to them via MoreOver:

http://api.moreover.com/ping?u=http://www.example.com/sitemap.xml

Ask.com

http://submissions.ask.com/ping?sitemap=http%3A//www.example.com/sitemap.xml

So, there you have it; sitemaps without much of the technobabble - I hope this guide has helped de-mystify these beasties for you!


Michael Bloch
Taming the Beast
Visit http://www.tamingthebeast.net for free Internet marketing and web development articles, tutorials and tools! Subscribe to our popular ecommerce/web design ezine!