Friday, August 22nd, 2014

Fix Crawl Errors to Increase Revenue


When StepForth performs a website audit we look at many different aspects of a websites health ranging from onsite factors such as suitable content and site architecture, offsite factors like social media and inbound links, and various performance metrics like bounce rate and crawl errors.

The primary goal of an audit is to uncover any underlying issues that may prevent a site from achieving maximum visibility. A website does not need to be performing poorly in order to benefit from an audit, even strong sites are often missing some pieces that could improve performance once corrected.

Sometimes interesting findings occur accidently when there is an overlap of unique problems occurring on a website. When completing an audit recently a combination of high bounce rates and soft 404’s, along with proper ecommerce tracking, provided clues that a site may be seeing a leak in revenue that could be relatively easy to patch.

First, I’ll get a little jargon out of the way. A “Bounce” is when a visitor comes to a website and leaves without viewing more than a single page. The “Bounce Rate” is the percentage of overall visitors that “Bounce”.  Typically a bounce rate of less than 50% does not raise concerns.

Soft 404’s are not much different than a typical “Page Not Found” error that all of us have encountered. The big difference is that in the background the server sends a status of anything other than “404”. To the end user there is not much difference as they are often still served the correct error page, but to search engines this incorrect response code can create problems.

Not found errors are never good, and can often lead to reduced sales. The recent audit we completed had unique set of errors that allowed us to understand more closely what the financial loss may have been. For the purposes of this article I have rounded all figures for easy reading and calculations and I won’t be sharing any identifiable client information.

When digging around in Google Analytics we noticed that there were hundreds of pages on the site with extremely high bounce rates of 80% or higher. Many of these high-bounce pages also had traffic exceeding 500 visitors a month. When we went to check out the live page to see why the bounce rate may be so high, we noticed that 404 errors were being displayed while the original URL was still loaded.

Some further digging within Google Webmaster Tools (GWT) and then with the use of an HTTP Header Checker, we found that there were several hundred Soft 404’s. The not found errors, instead of returning a 404 status were returning a 200 OK. Certainly not uncommon especially with many content management systems. The end result allowed all the error URL’s to still be available within Google Analytics.

Normally the recommendation at this point is fairly straight forward. Correct the links that are creating the errors and 301 redirect these dead pages to suitable replacements. A recommendation to correct the 404 error handling was also recommended. For pages that are not showing errors but have a high bounce rate, work to improve content, call to action, etc, to try and engage site visitors and reduce the bounce. Nothing really unexpected here.

While this recommendation will clean up the issue, I wanted to try and dig a little deeper and see if a monetary value could be applicable here. Because this site had their ecommerce settings running properly within Google Analytics, perhaps we could make a rough estimate on lost revenue as a result of these errors. After all, the traffic data was available.

We could see that the site had an overall conversion rate of about 2% and that the average transaction value was around $150. The site wide average bounce rate was also noted at around 40%. Taking one of the high bounce errors (80% bounce rate and 500 visitors in the last 30 days) would could use all these numbers to estimate the lost revenue that may have been retained if the old page was properly redirected to either its new home or a similar replacement.

It could be possible to really nail down some accurate figures and data, but because this calculation was for a rough estimate and not an in-depth analysis, I simply made some educated guesses and assumptions. For instance, what was the page about originally? By looking at the URL it was a very safe bet that it had been a specific product page. Next I have to assume that this page would have performed comparably to the other pages on the site, and that a replacement product (from a 301 redirect) would be appealing to a site visitor. In a perfect world, the figures below are likely close.

Because the page had an 80% bounce rate, we know that 20% of the visitors went on to do other things. Subtracting this 20% from the original 500 visitors leaves us with 400 bounced visitors.

The sites average bounce rate was 40%. As a result we assume that 40% of these 400 visitors would also leave even if the page was in good standing. This leaves us with 240 potential customers.

The overall conversion rate on the site was 2%. With 2% of the 240 visitors likely to make a purchase we could reasonably expect 4.8 sales (240 * .02). We’ll round that up to 5.

Given that the average transaction is about $150, all in all the site may have lost $750 in revenue in the preceding 30 day window as a direct result of that single 404 error – and this was for just a single error. If it were not for checking the bounce rates in Google Analytics and crawl errors in Webmaster Tools the entire issue may have gone unnoticed.

Now it is important to note that these figures and final result are all very rough calculations, but it does help to illustrate the value of eliminating crawl errors. Be sure to keep an eye on these metrics. Often a simple quick fix can result in improved padding on your bank account!

One Response to “Fix Crawl Errors to Increase Revenue”

  1. Chris Finnegan

    Hi Scott,
    detecting and fixing 404 errors is so often overlooked, and such an easy fix. Thanks for highlighting it!


Leave a Reply