Do Search Engines Like Your Web Site?

Between 75% and 98.8% of visitors to Web sites come from searches made at search engines. If you're going to get high levels of traffic - and hence the levels of ROI you're looking for - it's very important that the search engines can access all the information on your Web site.

Do the search engines know about all of your pages?

You can find out which pages on your site the search engines know about by using a special search. If you search for 'site:' and your Web site address, the search engine will tell you all of the pages on your Web site it knows about.

For example, search for: site:webpositioningcentre.co.uk in Google. Yahoo or MSN Search, and it will tell you how many pages they know about.

If the search engines haven't found some of the pages on your Web site, it is probably because they are having trouble spidering them. ('Spidering' is when the search engine uses an automated robot to read your Web pages.)

Spiders work by starting off on a page which has been linked to by another Web site, or that has been submitted to the search engine. They then read and follow any links they find on the page, gradually working their way through your whole Web site.

At least, that's the theory.

The problem is, it's easy to confuse the spiders - especially as they are designed to be wary of following certain kinds of link.

Links which confuse spiders

If your links are within a large chunk of JavaScript code, the spider may not be able to find them, and will not be able to follow the links to your other pages.

This can happen if you have 'rollovers' as your navigation - for instance, pictures that change colour or appearance when you hover your mouse pointer over them. The JavaScript code that makes this happen can be convoluted enough for the spiders to ignore it rather than try to find links inside.

If you think your rollovers are blocking your site from being spidered, you will need to talk to your Web designers about changing the code in to a 'clean link' - a standard HTML link, with no extra code around it - that is much easier for the spiders to follow.

Links like these will look something like this:

Home Page

Page addresses to avoid

Spiders will also ignore pages if they don't like the URL (the address needed to find the page).

For example, a Web site that has URLs containing several variables can cause spiders to ignore the page content. You can spot pages like these as they have a ? in them, and &, for instance:

http://webpositioningcentre.co.uk/index.php?page=12&cat=23&jib=c

This URL has three variables, the parts with the = in them, between the ? and &s. We find that if a page has one variable, or even two, the top search engines will spider them without any problems. But if a URL has more than that, often the search engines will not spider them.

Spiders particularly avoid URLs that look like they have 'session IDs' in them. They look something like this:

http://webpositioningcentre.co.uk/index.php?page=12&id=29c8d7r2398jk27897a8

The set of numbers and letters do not make much sense to humans, but some Web sites use them to keep track of who you are, as you click through their Web site.

Spiders will generally avoid URLs with Session IDs in them, so if your Web site has them, you need to talk to the people who developed the site about re-writing it so they do not use these IDs, or at least that you can get around the Web site without them.

Clean links = happy spiders

If you use clean, easy to follow links without several variables in them, your Web site should be spidered without problem. There are, of course, many other facets to successful Search Engine Optimization, but if the search engines can't spider your content, your site will fall at the first hurdle.

Paul Silver and David Rosam are Head of Technical SEO and Head of SEO Copywriting at Web Positioning Centre (http://webpositioningcentre.co.uk). Paul has been involved with the Web commercially since 1996 and David has been writing marketing copy for 20 years, and writing for the Web for a decade.

---

Search Engine Optimization Techniques

Search engine optimization is the process of increasing the amount... Read More

Search Engines Are Going to Love You for This

What's the most potent free traffic source on the web?***... Read More

Increase Page Rank with Search Engine Optimization

Utilizing effective search engine optimization techniques will improve the page... Read More

Search Engine Optimization Strategies Guaranteed to Skyrocket your Rankings

The point of optimizing your website is so that you... Read More

Going To Market: Keywords and Backlinks, Part 1

As I waded into the 'Make Money Online' waters on... Read More

SEO Expert Guide - Free Site Promotion (PR) (part 6/10)

In parts 1 - 5 you learnt how to develop... Read More

Pay-per-click ? the Ultimate Tool to Boost Affiliate Sales

The old ways are not always the best ways.The traditional... Read More

SEO Quickstart - 3 Things You can Do to Improve Search Engine Rankings Right Now

Increasing your search engine rankings should certainly be one of... Read More

Expert Help From Google Answers

Web users turn to search engines for answers to their... Read More

SEO, the Simplified Version

Lets get things straight. SEO is a very competitive market.... Read More

8 Essential SEO techniques

1) Title Tag - The title tag is the most... Read More

Easy SEO in 6 Simple Steps

If you want to increase traffic to your website and... Read More

What is PageRank?

Chances are you have been on the Internet and have... Read More

What Constitutes a Complete and Effective SEO Campaign?

Unfortunately, not many Search Engine Optimization companies know what this... Read More

Ten Steps To A Well Optimized Website - Step 5: Internal Linking

Welcome to part five in this search engine positioning series.... Read More

Search Engine Optimization: Site Structure and Popularity

In the Global Internet era the industry presence is undoubtedly... Read More

Understanding Googles Algorithm

What is the Google Algorithm?An algorithm is a mathematical equation... Read More

Website Copy-writing for Higher Sales and Higher Search Engine Ranking

Why, you ask? Mainly, because search engines want to provide... Read More

Effective Search Engine Use

The Internet is a wonderful place full of resources that... Read More

Search Engine Optimisation - Getting Targeted Traffic

Getting SEO right is an art and it's based on... Read More

Possibly The Biggest Misconception About Ranking Well In The Search Engines

Onpage search engine optimization are things that you can change... Read More

Search Engine Optimization :: The Basics and Why Websites Need It

We all know that the most targeted traffic we can... Read More

Get Listed in Google Without Submitting Your Site

With Google delivering so much traffic, it is only normal... Read More

The Search Engine Optimization Game Is Getting Tricky

For more than 4 years now I have been modifying... Read More

7 Steps to Highly Relevant, Search Engine Friendly, and People Useful Web Site Content

The task is to build a Content Rich Search Friendly... Read More

Your Top Search Rankings at Yahoo & MSN Search are Worthless

It seems I now do this rant every single year... Read More

Increase Your Page Rank Through SEO

Search Engine Optimization (SEO) must be considered a process and... Read More

Search Engine Wars - Quality Searches Vs Quantity!

It is no secret that Google and Yahoo are on... Read More

Got Spiders?

Many internet marketers blow mountains of start-up cash on their... Read More

The Google Sitemap For Idiots

I don't mind admitting that every time some new fangled... Read More

Onpage Optimization: Essential for Effective Offpage Optimization

Onpage optimization is the process by which various elements on... Read More

Website Optimization, Good Overall Optimization is Key

Good overall optimization, the right keyword phrases and quality content... Read More

Stuck on Keywords

Keywords have turned into big business on the internet. If... Read More