Do Search Engines Like Your Web Site?

Between 75% and 98.8% of visitors to Web sites come from searches made at search engines. If you're going to get high levels of traffic - and hence the levels of ROI you're looking for - it's very important that the search engines can access all the information on your Web site.

Do the search engines know about all of your pages?

You can find out which pages on your site the search engines know about by using a special search. If you search for 'site:' and your Web site address, the search engine will tell you all of the pages on your Web site it knows about.

For example, search for: site:webpositioningcentre.co.uk in Google. Yahoo or MSN Search, and it will tell you how many pages they know about.

If the search engines haven't found some of the pages on your Web site, it is probably because they are having trouble spidering them. ('Spidering' is when the search engine uses an automated robot to read your Web pages.)

Spiders work by starting off on a page which has been linked to by another Web site, or that has been submitted to the search engine. They then read and follow any links they find on the page, gradually working their way through your whole Web site.

At least, that's the theory.

The problem is, it's easy to confuse the spiders - especially as they are designed to be wary of following certain kinds of link.

Links which confuse spiders

If your links are within a large chunk of JavaScript code, the spider may not be able to find them, and will not be able to follow the links to your other pages.

This can happen if you have 'rollovers' as your navigation - for instance, pictures that change colour or appearance when you hover your mouse pointer over them. The JavaScript code that makes this happen can be convoluted enough for the spiders to ignore it rather than try to find links inside.

If you think your rollovers are blocking your site from being spidered, you will need to talk to your Web designers about changing the code in to a 'clean link' - a standard HTML link, with no extra code around it - that is much easier for the spiders to follow.

Links like these will look something like this:

Home Page

Page addresses to avoid

Spiders will also ignore pages if they don't like the URL (the address needed to find the page).

For example, a Web site that has URLs containing several variables can cause spiders to ignore the page content. You can spot pages like these as they have a ? in them, and &, for instance:

http://webpositioningcentre.co.uk/index.php?page=12&cat=23&jib=c

This URL has three variables, the parts with the = in them, between the ? and &s. We find that if a page has one variable, or even two, the top search engines will spider them without any problems. But if a URL has more than that, often the search engines will not spider them.

Spiders particularly avoid URLs that look like they have 'session IDs' in them. They look something like this:

http://webpositioningcentre.co.uk/index.php?page=12&id=29c8d7r2398jk27897a8

The set of numbers and letters do not make much sense to humans, but some Web sites use them to keep track of who you are, as you click through their Web site.

Spiders will generally avoid URLs with Session IDs in them, so if your Web site has them, you need to talk to the people who developed the site about re-writing it so they do not use these IDs, or at least that you can get around the Web site without them.

Clean links = happy spiders

If you use clean, easy to follow links without several variables in them, your Web site should be spidered without problem. There are, of course, many other facets to successful Search Engine Optimization, but if the search engines can't spider your content, your site will fall at the first hurdle.

Paul Silver and David Rosam are Head of Technical SEO and Head of SEO Copywriting at Web Positioning Centre (http://webpositioningcentre.co.uk). Paul has been involved with the Web commercially since 1996 and David has been writing marketing copy for 20 years, and writing for the Web for a decade.

---

SEO Hints and Tips and Free SEO Tools

Do you realize that if you manage your website, SEO... Read More

Opinion ? Search Engine Success

This article is actually the summary to a book soon... Read More

2 Powerful Ways To Capitalize on Your Search Engine Traffic

Many marketers know that search engine marketing is among one... Read More

Beginners - Getting Listed In Search Engines

Once you have your web site complete and ready to... Read More

Do Not Drop Your Web Site Off the Search Engine Cliff

If you've been feeling like Tom Cruise climbing up the... Read More

Keyword Selection- The Dark Horse of Search Engines Optimization

Below are what I call the "10 Commandments" for Keywords.To... Read More

Submitting Your Site To The Open Web Directory: Some Dos And Don?ts

One of the most important steps in any site's publicity... Read More

Linking for Fun and Profit

Well actually, linking isn't fun at all. In fact, it's... Read More

Increase Web Site Sales with a SEO Proposal - Part 2

Part I of this article discussed some of the points... Read More

20 Ideas for Creating Traffic Rich, Search Engine Friendly Pages

Sometimes questions will arise around the subject of gateway information... Read More

Search Engine Tips & Techniques

As you are building your site or getting your site... Read More

Using Meta Tags Wisely to Attract Search Engines and Visitors

There are certain tricks and techniques in SEO industry, which... Read More

Search Engine Rankings for Beginners

Search engine optimization is best left in the mystical land... Read More

3 Deadly Search Engine Marketing Sins

My inbox this week provided glaring examples of three all... Read More

How to Get Tons of Top 10 Rankings Guaranteed

Okay I admit it. I was kidding. I can't guarantee... Read More

Playing By Googles Rules

As the undisputable leader in search engines, Google places a... Read More

Offpage Optimization: Does Article Marketing Cut the Mustard?

For those who haven't heard: article marketing is the new... Read More

Have You Been Dropped By Yahoo?

Yahoo has fairly recently updated its guidelines. You can read... Read More

When Google PR Tool Bar is Down!

Where is the need to worry? Who needs a PR... Read More

Search Engine Optimization Tip: Dont Buy Anything Before You Learn More!

Thinking about purchasing that premium SEO software that will hyper-optimize... Read More

SEO - Are You Making The Search Engines Mad?

If you've been involved in SEO (search engine optimization) for... Read More

10 Quick Ways To Kick-Start Your Profit Pulling Keywords

First, you must realize that targeting the right keywords or... Read More

Easy Steps to Get Onto Google Top Search Pages

To get on Google's top pages can be accomplished by... Read More

Search Engine Optimization - A Must

As you surf the web take a look around at... Read More

SEO for CEOs ? Search Engine Optimization Unmasked for CEOs

If you're like most other CEOs, the term "search engine... Read More

Google Local Search And The Impact On Natural Optimization

With the advent of Google Local, a service that helps... Read More

The Search Engine Optimization Secret that Everybody Ignores

Search engine optimization is a very critical task in the... Read More

Link Swapping - How to Win the Website Marketing Game in 3 Easy Steps

If you are a webmaster, chances are one of your... Read More

Linking for Traffic not Positioning!

With more and more experts and search engine enthusiastsclaiming the... Read More

The Google Sitemap For Idiots

I don't mind admitting that every time some new fangled... Read More

An Ethical Alternative to Doorway Pages

Definition: A doorway page is content created specifically for the... Read More

How To Syndicate Your Content

Note: In order to preserve the proper format for the... Read More

Organic SEO: Patience For Long Term Ranking Results

When does long term SEO show ranking results? It takes... Read More