A couple of weeks ago, Leslie Rohde from WindRose Software was minding his own business when a user of their signature tool, Optilink, emailed him to ask about the 404-page generated when querying Google.
Optilink is used to trace incoming links to the myriad of sites they originate from. While reporting the URLs of incoming links, Optilink also gathers other information about those sites such as the PageRank score Google displays and the anchor text used in those links. In order to gather that information, Optilink makes multiple queries of the search tools it visits. Some users use Optilink a lot, thus using a lot of resources at Google and other search engines, a practice discouraged and often banned by search firms.
Optilink made use of the search query “link:www.domainname.com”. Recently, Google has limited the number of daily queries that an individual IP can make and added a “ransom note” text-box triggered by large numbers of link searches as a barrier against automated systems. This posed a problem that sent Rohde to work at a local cafe where the WiFi was cheap and easy and the distractions less severe.
In an email sent to all Optilink users, Rohde announced the release of Optilink3.4.0, a revision that works around the Google link:domain ban.
“The trap imposed by Google is limited to the “link:” query that is used in the “Linking Page Popularity” option. Ironically, this is probably one of the least useful options in OptiLink thanks to Google’s filtering of linking page information.
Three years ago when OptiLink was first introduced, there was comparatively little filtering of linking page data so the option had real benefit as a research feature. That is no longer true today and the only reasonably reliable linking information comes from MSN and Yahoo.
On balance, the limited benefit derived from the Linking Page Popularity information provided by Google is simply not worth the risk and effort, so GOOGLE IS NO LONGER AN AVAILABLE SELECTION IN THE LINKING PAGE POPULARITY CONTROL.
If Google’s linking information were someday to _become_ useful, there are indeed ways to get it, but the techniques I have so far discovered — which do beat the current filter — are pretty involved and include some undesirable side effects.
Fortunately, the initial query that OptiLink uses to fetch links is undisturbed by Google’s new trap, probably because it occurs just once per execution of OptiLink. The browser interface that OptiLink presents to Google would make elimination of _that_ feature very difficult without eliminating the “link:” query all-together.
The PageRank query is likewise undisturbed by the new trap, so in total, we have the two most valuable features retained and the single most questionable feature penalized. 🙂 “
Rohde understands the situation Google is in when it comes to drains on their resources but at the same time, he noted in yesterday’s phone interview that search users and advertisers require tools to engineer better, more effective campaigns.
“Google put in place a means to stop people from automatically downloading search results. They recognize everyone wants a piece of them. They want human eyeballs only.”
He went on to say that he thinks Google targeted tools that gather link information such as his, rather than the dozens of automated position-reporting tools, because links matter so much more in terms of search engine placement than the actual content of the site. It’s the links that get the site to that position and knowing the position is not going to change it. Knowing your links will.