Over the past two years, both the business and practice of search engine marketing have become much more complex. The same is true of the websites search engine marketers work with. The evolving design techniques and technologies that have made the website of today far more versatile than those of previous years, have also made them harder to work with for third party service vendors such as SEOs. Mix in rapidly growing levels of keyword competition, new styles of search engine and the continued strength of the Pay per Click advertising option and a sector that once resembled a paint-by-numbers art project is now more of an abstract exercise in impressionism.

The changes that have happened over the span of the search marketing industry are enormous but have happened in four distinct waves, the latest of which is peaking right now. These changes have forced a shift in the way strong SEO and SEM firms approach clients and campaigns, most wanting to perform a short round of consulting services before diving into a major campaign. With the increased complexity of the environment in which search engine marketers work, choosing the right keywords and placing them strategically on a document just doesn’t cut it any longer. As the search environment is effected by a greater number of factors, helping clients weave their way through layers of information and analysis is increasingly the first stage of a strong search engine optimization and marketing campaign.

Today we are interested in dozens of different factors than we were in previous years. This increased complexity is often lost on potential clients, some of whom seem to see the initial consultation period as an unnecessary waste of time or money. For a good SEO, the information gleaned during these initial market studies forms an indispensable instruction manual for the conduct of the campaign.

StepForth is interested in finding out a lot about clients and their competition before embarking on a major SEO project. During pre-campaign planning, we weigh several factors against each other to determine the best marketing tactics and techniques for each client. Search engine marketers need to know about a website’s architectural structure, the products or topics featured on it, the relationships a website (and its documents) share with other websites and documents, who the competition is, and what that competition is doing. With so many factors to figure out, initial studies can take over ten hours per campaign, a significant amount of time considering the expense associated with a competent technician’s rates. When completed however, the information revealed can save thousands of dollars in time and effort.

The need to examine the competitive environment before competing for Top10 listings is a huge step away from the devil-may-care early days of SEO. Even up to 2003, search engine optimization could often be performed after a quick keyword target analysis. The maturing of the search sector has prompted a more mature attitude in search engine marketing as a short history of the search sector shows.

In the beginning optimization was easy. In 1997, SEO was about keyword stuffed meta tags, keyword enriched content and a title based on multiple incidents of target keywords and phrases. While the search results pages might take a month or more to update, a good SEO treatment generally achieved Top20 placements without much trouble. To keep things interesting, there were a larger number of popular search engines nine years ago than there are today. Names like AltaVista, Lycos, Excite, Magellan, Infoseek, WebCrawler and Northern Light were once considered relative equals in the realm of search.

Ironically, it was the wide variety of search engines back then that led to the development of a lot of spammy pages and sites. Since there were a number of unique algorithms in play, search engine optimizers made a practice of creating several unique pages or unique sites for each of the major search engines. Known as doorway pages or leader sites, this practice quickly gummed up search engines with spammy pages and duplicate content, leading to the development of more sophisticated methods of sorting content and determining relevancy between documents. Although the days of using keyword stuffed meta tags are long over, it is continually surprising to hear how many people still think the technique achieves results.

Just after the turn of the century a seismic shift on par with the great San Francisco earthquake of 1906 hit the Silicone Valley and spread to the rest of the wired world. An estimated 8-Trillion dollars just sort of vanished in the span of a few short months following the March 2000 dot-com stock meltdown. Many of the aforementioned search firms were victims in the dot-bomb fallout. Just before the tech-stocks tanked, a smaller, less cluttered search engine started being adopted by search engine users looking for a less corporate alternative.

Google was uniquely different from any search engine before it. While keyword recognition played a critical role in determining which documents matched which search queries, Google based its rankings primarily on the number of links pointing to a document. By early 2001, Google was unquestionably the dominant search engine. Towards the end of 2001, Google was responsible for over 75% of all search results, including those shown at Yahoo, Netscape, AOL, MSN, and many other branded search-names. For search engine marketers, the universe had shifted.

The dominance of Google allowed search engine optimizers to begin adopting basic standards of practice loosely based on the Google guidelines and started to eliminate the dodgy technique of building doorway pages. All things being equal the rise of Google spelled the fall of SEO doorway spam but it led to the creation of an industry based on another dodgy tecnique, link-spam.

Over the next two to three years, Google’s rankings determined the rankings of other search engines to one degree or another. Search engine optimizers became obsessed with achieving strong placements at Google, an obsession both shared with and driven by client expectations. Google was the end-all-be-all in the search world, holding a prominence where a Top10 placement could make or break an online business.

During this time, another type of search based advertising began to make an impact on the overall search marketplace. Originally introduced by Overture, the pay-per-click model introduced an instant but potentially expensive way of putting a keyword based advertisement in front of the eyes of search engine users. Overture has since been purchased by Yahoo in a move that started to move Yahoo away from Google and back to a primary position among the top ranked search engines. While Overture built a highly profitable business model out of pay-per-click, Google innovated on the concept and built their monstrously profitable income generator known as AdWords.

About eighteen months ago, Yahoo started generating its own results, independent of Google. They were quickly followed by MSN which began producing its own results in 2004. Running its way into the pack, Ask Jeeves introduced a number of dramatic improvements to its search engine in 2003/2004 and now ranks with Google, Yahoo and MSN as “the Big4” of search engines.

For search engine optimizers the adoption of search as a commercial advertising medium by businesses of all sizes has become a double edged sword of sorts. While there has never been more business for good SEOs with strong reputations, the amount of work needed to be done to provide a strong campaign has never been greater. Search engine marketing reaches more eyeballs than ever before but at the same time, there are more sites seeking those eyeballs than ever before.

Advertisers should start to expect a greater emphasis on research and recommendation before expecting SEOs to start digging into their documents and sites. While the expense might be slightly higher in the short term the results of a well informed optimization effort saves more money over the long-run.