Gravatar
Wednesday, June 3rd, 2009

Block Your Competitors from Using IIS SEO Toolkit

 

I was just reading more about the IIS SEO Toolkit when I came across the following interesting tutorial on how to block competitors from running the IIS SEO Toolkit on your servers. If you have an asp-based website then this is a mandatory update in my opinion unless you want your competitors to have easy access to your site’s information. That said, a lot of the information that the IIS SEO Toolkit provides is likely available through other tools but at least if you block the Toolkit you will make it that much harder on the competition.

Here is the tutorial:

All HTTP requests made by IIS Site Analysis Web Crawler have an HTTP header “user-agent” set to:

“iisbot/1.0 (+http://www.iis.net/iisbot.html)”IIS Site Analysis Web Crawler is fully compliant with the robots exclusion protocol. This means that you can use robots.txt file to prevent IIS Site Analysis Web Crawler from crawling your web site. For example you may want to do it to prevent other people from running IIS Site Analysis against your web sites.

In order to prevent IIS Site Analysis Crawler from crawling a web site, add the following lines at the end of the robots.txt file located at the site’s root directory:

User-Agent: iisbot
Disallow: /

NOTE: Thanks to SMX Advanced for giving a great show, I wouldn’t have had access to so much breaking info without this great show. Kudos!


2 Responses to “Block Your Competitors from Using IIS SEO Toolkit”

  1. mike gallagher

    So will this robots.txt still block iisbot, even though the first line says allow everyone?

    User-Agent: *
    Allow: /

    User-Agent: iisbot
    Disallow: /

    Thanks for the podcasts.
    Mike

  2. Ross Dunn

    Hi Mike, I generally don’t waste my time with the user-agent allow notation simply because that is a given. As a result, I would take that out and just leave the disallow in there. Cheers, Ross Dunn

Leave a Reply

Google Adwords Certified Partner Member of SEO Consultants Directory EMarketing Association