Fastest Search Engines

There are differing views on which search engines are the most fast and the most effective. By reconciling several sources declaring to list the "major search engines," we will mention and recommend submitting your URL to AltaVista, Excite, HotBot, Infoseek, Lycos, WebCrawler and Yahoo. Let us discuss these engines and the services they provide.

AltaVista (http://www.altavista.digital.com/) first went online in December of 1995. Run by Digital, AltaVista quickly gained widespread name recognition. In June 1996, AltaVista formed an agreement with Yahoo, effectively becoming the "preferred" search engine.

AltaVista is a "search engine," the distinction of which we made in the beginning. Digital has named its spider 'Scooter.' When one submits one's URL to Scooter, it reads the document at that location, accepting the "description" and "keywords" META tags, and of course indexing the title and URL. Scooter then follows all links from the originally submitted URL. This means that no additional submissions made from that same web site would benefit the site's publisher.

Excite (http://www.excite.com/) was launched in late 1995, growing quickly to prominence, and absorbing two of its competitors— Magellan in July 1996 and WebCrawler in November of that same year. It has, however, allowed those two engines to operate separately.

Excite is a full-text spider, thus it is correctly called a "search engine." However, the editors of the site have also created a directory system, thus the Excite web site truly offers a "hybrid" information locator. (Remember: A search executed with a hybridized search engine does not search the edited directory, but only the crawled database.) The Excite spider recognizes and recommends using the two types of META tags we have discussed, but the editors of the site also recommend writing specific, focused copy, to ensure proper indexing.

HotBot (http://www.hotbot.com/) arrived on the Internet in May 1996—Wired Magazine's addition to the search engine community. Until recently, HotBot had been solely a "search engine" machine. Recently, however, the site's editors have included a directory of sites they have reviewed. HotBot has therefore evolved into a hybrid system—a form most search engines seem to evolve into. HotBot's crawler and indexer, created by Inktomi, are full-text indexers. The indexer recognizes both the "description" and "keywords" META tags, and reads entire documents for categorization. One is allowed to submit up to fifty URLs per domain, from which all links are "crawled." A domain is the name we all use to browse to information. It is part of the URL e.g. pepsi.com, csuchico.edu, lightside.net. I would recommend submitting only the URLs of the main sections of one's web site. I have found that HotBot returns the most hits for each query made to its database.

Infoseek (http://www.infoseek.com/) has been around since early 1995. It is a well-known, well-reviewed, and well-connected. In its first stages, the "Infoseek Guide" index cataloged only one to two million URLs. The new fifty million URL service was introduced in Fall of 1996.

Like HotBot, only fifty URLs from a single domain are allowed to be submitted to the Infoseek indexer. This can be done by e-mail, or a hypertext form. Both "summary" and "description" META tags are supported and relied upon for categorization.

Lycos (http://www.lycos.com/) has been around since May 1994, and is one of the oldest major search engines on the Internet. Its roots are as a project at Carnegie Mellon University. Lycos is a hybrid search engine, listing sites in two main ways. There are the search engine listings (the crawled database) and the associated directory, called "Web Guides." The first version of a directory listing that Lycos added to its search engine service in February 1996, was named "A2Z." Since then, the directory has evolved into a subject-area oriented guide—News, Shopping, etc. Lycos also runs a reviewing service called "Top 5%." This is a list of reviewer picks, and are deemed the best sites on the Internet.

Lycos' spider does not use META tags, only utilizing document titles to index pages. For this reason, we need to pay special attention to our document titles, if we want our pages to show up in a Lycos search.

WebCrawler (http://www.webcrawler.com/) opened to the public in April 1994. Like Lycos, WebCrawler also started as a research project, but, for the University of Washington. In March 1995, America Online purchased WebCrawler and made it the commercial service's preferred search engine until November 1996 when the little spider was acquired by Excite. WebCrawler has been allowed to run independently of Excite and has evolved into a hybrid engine. It offers a directory listing entitled "WebCrawler Select."

WebCrawler, like many of its competitors, utilizes META tags when they are made available by web page authors. It is also a full-text spider, and recommends writing specific, focused copy for one's web site.

We have left Yahoo (http://www.yahoo.com/) for last because it is the only non-hybrid on the Internet. Yahoo is a directory, as opposed to a search engine, which makes it a great place to start looking for information. Every URL listed in Yahoo's database was verified by a human editor.

Yahoo has been in existence since late 1994. It is well-known, well-used and well-respected, listing over 500,000 web sites—the largest directory on the Internet. Because it is a directory based on user submissions, it may not have some sites in its catalog that a crawler would. Consequently, users may wish to consult a search engine if not receiving desired results. Yahoo makes this easy, and will pipe your query to any of the major search engines with one click. Recently, Yahoo has partnered with AltaVista, making the Digital-owned search engine Yahoo's preferred crawled database.