More on the Robots Meta Tag

by | Mar 7, 2007 | Search

I have seen many articles and the like discussing proper use of the Robots Meta tag. It almost seems common knowledge of the purpose of this tag, but every now and then you find some confusion over its purpose and how it works. Recently Vanessa Fox over at the...

Search Engine Ranking Problems? Validate Your Robots.txt File

by | Aug 2, 2006 | Tools

This is a short posting but it has the potential to have a wonderous impact on your website. Back in June Google implemented a Robots.txt validator that allows you to check for errors. The robots.txt file can easily ruin your chances for ranking if it is not carefully...

The Ask Search Engine Robot – Is It Depressed?

by | Aug 2, 2006 | Search

Have you noticed how infrequently and deeply Ask appears to be indexing websites? I reviewed a number of my client’s websites and discovered that of the big 4 robots (Google, Yahoo, MSN and ASK) Ask is dramatically falling behind. Where the other search engines...

Arachnophilia, the Joy of Playing with Spiders

by | Jul 6, 2005 | Tips

Spiders make great geek pets, at least virtual ones do. Here at StepForth, we keep a couple spiders on our system to test sites, pages and documents in the hopes of learning more about the behaviours of common search engine spiders such as GoogleBot, Yahoo’s...