As searchers become more sophisticated, the search engines are forced to evolve to keep up with the changing times.
During a session at WebmasterWorld’s Publishers Conference VI, Tim Mayer from Yahoo! Search explained how Yahoo! has improved its abstracts, the presentation of its results pages, and how it has developed a new algorithm designed specifically to focus on longer query searches using more terms.
Yahoo! now offers a larger index and additional changes have been made to its page cache. Yahoo’s daily crawler discovers and crawls millions of new documents each day. The company is also experimenting with submission quality and the new Yahoo! Slurp bot will crawl links from the URLs that are submitted.
Additionally, Yahoo! Search RSS will be arriving on April 15th.
New slurp commands: Slurp now recognizes the NOARCHIVE meta tag command and crawl-delay, although Yahoo! admits it is still struggling with the way Slurp handles redirects.
Yahoo’s Crusade Against Spam. Yahoo! takes the fight against spam very seriously, using both automated and human filters for fighting spam. Yahoo! Search will be combining intelligences with Yahoo! Mail to offer better anti-spam technology.
Have you been banned? If your site has been banned from Yahoo! for spamming, Slurp will visit your robot.txt but will not go deeper into your site. Paid inclusion members might notice their URLs appearing lower on keyword query pages.
What can you do? If your site has been banned and you’ve cleaned up your act you can submit your page to the paid inclusion program for review in April or send an email to email@example.com and include what method got you in trouble and how you’ve corrected yourself.