Site Maps: Let Search Engines Find Your Pages

With 40 million websites in existence, and more than 3 billion web pages indexed by Google at the time of this writing (July 2003), it's no wonder that more and more people are relying on search engines to find their way through the unruly world that the web has become.

Nowadays, it is crucial to get your pages indexed by the most important search engines. To maximize traffic to your site, you must make sure that all your internal pages are indexed, not just your main page (homepage).

Fortunately, you don't need to submit each of your pages manually. The most efficient way is to create a Site Map (a list of links to all the pages in your site) and link to it directly from your homepage.

How Will A Site Map Help Me?

Search engines find pages by "crawling" the web. They go through the code of all the pages in their database (also called index), following links to other pages and adding them to the database (in fact, more pages are added this way than by manual submission).

However, search engines have trouble following links from pages buried too deep within the directory structure of a site. A Site Map solves this problem by giving the engines access to the links to all your pages once they follow the Site Map link in your homepage. For more effectiveness, place your Site Map in your root directory (where your index page is).

Site Maps: Not Just for Search Engines

While some web users will find their way through your site by following navigation links or by using the search box, others will turn to your Site Map. If you design your Site Map carefully, it will not only be useful to the search engines, but to your human visitors as well.

Here are some pointers:

• The Site Map should act pretty much like the table of contents of a book.

• The Site Map must clearly show all the sections of your site, and the information contained in each of those sections.

• Every item in your Site Map must be hyperlinked to its URL.

• If it's not too long or cumbersome, use each page's TITLE as the link text, since this tends to increase the relevance of your site. Otherwise, use the word or the short phrase that best describes the content of the page.

• Make sure that you place the link to your Site Map at a visible location in your homepage (users shouldn't need a map to find your Site Map!).

• Don't get creative: simply call the link "Site Map".

• Make your Site Map a simple text link. If you use javascript the search engines will ignore it.

How can I check if my pages have been indexed?

Once you have created and uploaded your Site Map and placed a link to it in your homepage, submit both your homepage and your Site Map page to the search engines. You will then have to wait until the search engines do a web crawl. In the case of Google, the largest search engine, this happens approximately once a month.

To check if a page on your site has been picked-up and indexed by Google's, go to www.google.com and use the "allinurl" command in the search box:

allinurl:yourdomain.com/yourpage.html

Where "yourdomain.com/yourpage.html" is the URL of the page you want to check.

To get a list of all the pages in your domain that have been indexed by Google, you'll have to use the "site" command, followed by your domain name plus a word (or group of words) that you know appear in all your pages (for example, a copyright statement or some footer text):

site:yourdomain.com commonword

If after typing this command you get a list of all your pages (or at least a significant number of pages that weren't in the index before), this will be a strong indication that your Site Map has been successful.

-----

You can freely reprint this article. Just include the following resource box at the end:

About The Author

Mario Sanchez publishes The Internet Digest ( http://www.theinternetdigest.net ), an online collection of web design and Internet marketing articles and resources. You can freely reprint his weekly articles in your website, ezine, newsletter or ebook.

---

Search Engine Spiders Lost Without Guidance - Post This Sign!

The robots.txt file is an exclusion standard required by all... Read More

Google Sitemaps Just Got Better

Having a Google Sitemap just got better! Not only does... Read More

Increase Your Page Rank Through SEO

Search Engine Optimization (SEO) must be considered a process and... Read More

Utilizing Popular Directories as Free Link Sources

If you're a webmaster, you've probably spent almost as much... Read More

5 Qualities to Look for in a Search Engine Optimization Program for Your Website

These days with millions of websites out there, you need... Read More

10 Ways to Tackle Keyword Research and Selection

You need to be extremely careful with keyword research so... Read More

Keywords, Choose Them Wisely

By now you have likely heard that keywords and keyword... Read More

Internet 2005: The Ten Billions Pages Search Engines

To see each and every page during 10 seconds would... Read More

Internet Directory Submission Tips

Internet Directories and their ImportanceThere are two very pertinent reason... Read More

Torpedo and Sink the Ship SS Search Engine Ranking

I was recently contacted by one of my best clients... Read More

A Classified Way To Drive Business To Your Web Site

There are more than 105 million of them in the... Read More

Boost Your Website Rankings with Expert Content

The chase for a high web ranking is constantly on... Read More

Has Google Lost the Plot?

With the help of MSN, recently I've been reminiscing about... Read More

Article Marketing: Fox in the Competitor Hen House or Chicken Little?

I recently was asked by an author to remove a... Read More

Drive More Traffic to Your Website With Your Web Page Title!

One of the most overlooked, but important components, on your... Read More

The Top 3 Mistakes That Can Ruin Your Websites Search Engine Rankings- and How to Fix Them!

Getting your website up and running is hard enough. After... Read More

The Write to Link Popularity

Incorporating writing as a means of boosting your link popularity... Read More

Submitting Your Site To The Open Web Directory: Some Dos And Don?ts

One of the most important steps in any site's publicity... Read More

SEO Expert Guide - Paid Site Promotion (Marketing) (part 7/10)

In parts 1 - 6 you learnt how to develop... Read More

You Cannot Hide From the Public Record Search Engines

As a search engine optimization specialist, I often run across... Read More

5 Things to Keep an Eye on in the SEO World in 2005...

After the latest PR update at Google and MSN's beta... Read More

SEO Help: Dont Try to Fool the Search Engines

Writing articles is all the rage these days on the... Read More

Tops In Toolbars?

Most internet marketers are aware of, and probably use, the... Read More

The Mystery of the Magical Keyword Density Formula

Keyword density. When it comes to SEO copywriting, this has... Read More

Is Google Having a Tough Time with Their Website Limit?

If you are one to pay attention to what happens... Read More

Search Engine Optimization - A Must

As you surf the web take a look around at... Read More

Search Engine Saturation Tool ? A Must Have SEO Tool

Search Engines have become the soul of the Internet. They... Read More

Google Bring Deskbar Search To Windows Desktop. Now Any Website Can Take Advantage Of This

Google's premier of desktop search proves that the desktop is... Read More

Put the Full Power of Google to Work with 11 Google Power Search Tips

Google has many ways to help you find want you... Read More

Five Ways To Win The Favor Of Search Engines

You've got a cool new website with all the works:... Read More

PageRank and How It Gets Assigned

We know that each and every website page is assigned... Read More

Commonly SEO terms and Jargon used by Search Engine Gurus

What does all this SEO jargon mean.Have you ever been... Read More

3 Deadly Search Engine Marketing Sins

My inbox this week provided glaring examples of three all... Read More