Why so many search engines?

Let's consider Yahoo! (a subject tree) and AltaVista (open text search). A couple of graduate school dropouts from Stanford created Yahoo! whereas AltaVista was created by Digital Equipment, so it has corporate roots. The two search engines have a unique relationship since Yahoo! will pass along an open text search to AltaVista for processing. So why use anything else? Well, like all products in the service sector, each search engine has unique features.

Some catalog a greater percentage of the Web, some are faster, some have more advanced search features, some are geared toward a specialized area of information, and some will translate your words into a concept and then search for the concept. There are even meta search engines that simultaneously feed your query to multiple search engines for processing! The best approach is to learn how to use one or two search engines well and then generalize your knowledge to other search engines as needed.

Subject Tree Index

A subject tree, as we have pointed out, is an index to the World Wide Web that is organized by categories and sub-categories. The most comprehensive subject tree index on the Web is Yahoo!. Try it yourself. To start Yahoo! you need first to launch your Web browser. This will probably be either Microsoft's Internet Explorer or Netscape Navigator, which looks like this:

Figure 2.4 - Netscape Navigator/Communicator is one of the most widely used Web browsers available today.

To get to Yahoo!'s site you type: www.yahoo.com in the Address or Location line of your browser and then press the return key. If your search requires specific Canadian content, then try www.yahoo.ca which is the Canadian Yahoo! site. Some of the top-level Yahoo! categories appear below:

Figure 2.5 - Yahoo is the most comprehensive subject tree index on the Web.

To explore the business category more in depth, click on Business and Economy. This brings up a list of all of the business and economics sub-categories, which looks like Figure 2.6.

Yahoo! boldfaces sub-categories and indicates the number of entries in parentheses. For example, we know that the Marketing sub-category contains 460 entries, some of which are relatively new. To explore the marketing category more deeply, click on Marketing and Advertising and you will see what is shown in Figure 2.7.

At this level we see that there are further sub-categories such as Demographics, Employment, and Organizations. Below these sub-categories we also see entries that are direct links to marketing-related sites on the Internet. For example, by scrolling down and clicking on Marketing Resource Center we are lead directly to a wealth of information along with hyperlinks to other marketing-related sites (see Figure 2.8).

Figure 2.6 - By digging down through sub-categories, a subject tree allows you to search for specific information on any given topic.

Note that, for the first time, the address has changed so that we are no longer on Yahoo!'s site. We have followed a link off of Yahoo!'s index to another site somewhere else on the Internet.

Your Turn! Using the Yahoo! subject tree index, locate the site for Advertising Age: The Advertising Century. Begin at the top level and drill down until a direct link is found. How many levels did you have to drill down to reach the site? What other links at the final level look like they might be worth exploring?

Open Text Search

An open text search is a keyword search through the entire Web. All pages matching the keywords are returned by the search engine and then ranked in order of relevance. Generally, pages containing the keywords the greatest number of times are given a higher relevance rating. Since thousands of documents can contain the same keywords it is important for you to be as specific as possible in your search.

Figure 2.7 - Below the sub-categories, Yahoo! lists direct links to other web sites that might be of interest to you.

Figure 2.8 - The Marketing Resource Center page is one step on an infinite path of hyperlink connections.

Yahoo! incorporates open text searching in two ways. First, it allows you to search the entire Yahoo! site when you enter a term in the search box and then press the search button. For example, if you return to www.yahoo.com and enter "Nike" in the search box, you will get the results shown in Figure 2.9. This tells us that Yahoo! lists Nike under different categories. Additionally, it has categorized links to dozens of different sites that contain information about the company. Second, Yahoo! allows us to transfer the search to AltaVista if none of the links provided are useful, just by clicking on the AltaVista item. The results can be seen in Figure 2.10.

Note that AltaVista has found 204,325 Web pages that mention Nike. The first 10 matches are displayed on the page. The first one shown is supposed to be the most relevant. But is it? Can you begin to see why we like you to start with Yahoo!?

Your Turn! Using the Yahoo! open text search feature, conduct the three searches in the following table, which progressively narrow down a subject. Record the number of matches given with each search string.

What conclusions can you draw from this experiment?

Sometimes the keywords appear on the same page but with hundreds of words separating them. This results in a list of irrelevant sites. Open text searches also incorporate ways to demand that the words not only have to appear on the same page but also must be right next to one another. The easiest way to accomplish this is to put the words that must appear together in quotes. The best way to understand this point is to try it out.

Try an open text search for northeast economic forecast.

How relevant are your search results?

Now try an open text search for northeast "economic forecast."

How relevant is the information returned?

Propose an explanation for the difference in results.

Knowing which words to put in quotes is not always obvious since there are times when the keywords never appear right next to each other. Try an open text search for "northeast economic forecast."

Did you get any results?

We hope you're starting to feel comfortable travelling the Net. Now we're going to move on to advanced search techniques.


or continue ot the next section - Advanced Search