Research Resource -- The Tao of Search (part 2)

Research Resource -- The Tao of Search (part 2)
by
Sandra Chevalier-Batik

This is the tenth in a series of articles exploring on-line sources and search strategies for writers. This month, part 2 of the writer's guide to understanding search engines is featured.

The Tao of Search

Today's writer has several choices when initiating an on-line search. Search engines fall into one of several categories: Meta Search Engines, General Purpose Search Engines, and Search Agents/Bots.

Meta Search Engines are very efficient and effective as they explore numerous search-engine sites simultaneously, then combine the results in a hyper-linked report for your review. My favorite Meta search engines are:

Ask Jeeves, http://www.ask.com, This is a good 'first shot' gateway site, as all requests are framed using natural language. The site boasts a high 'hit rate' thanks to a database of more than 7 million questions, and links.

Dog Pile, http://www.Dogpile.com, is the tool I use when I'm in a hurry. It searches the major sites, including Yahoo, AltaVista, Lycos, About.com, Infoseek, Exite, Magellan and several lesser-known search engines. Dog Pile technology translates queries into the closest similar syntax for each of the target search sites. This feature accounts for Dog Pile's accuracy.

Mamma, http://www.mamma.com, "The Mother of All Search Engines" is considered a 'Smart Meta Search Engine'. When the user enters a query, Mamma's powerful proprietary technology simultaneously queries 10 of the major Search Engines and properly formats the words and syntax for each source being probed. Mamma then creates a virtual database, organizes the results into a uniform format and presents them by relevance and source. In this manner, Mamma.com usually provides the end user with a highly relevant and comprehensive set of search results.

Yahoo! - http://www.Yahoo.com, is the web's most popular web directory. Its directory format and Metasearch technology makes subject search easy. It also offers specialized searches for timely news, stock quotes, classifieds, etc. worldwide. I find it the most useful tool for tracking people and places resources.

AltaVista, http://www.AltaVista.com is one of the web's most comprehensive search engines. It features a full range of search functionality - Boolean, proximity, plus truncation. Personally, I sometimes think of this site as 'to much of a good thing'. Carefully formatted search parameters are necessary for selectively refined results.

General Purpose Search Engines offer something unique that is not generally found in the other major search engines.

Web My Way, http://www.webmyway.com is a general directory comprised of the best sites for the most common search requests. It is an easy way to find a general listing of the best sites per specific fields. This limits irrelevant search results. For the more specific searches, they offer links to all the top search engines and directories.

Google, http://www.google.com uses sophisticated text-matching techniques to find sites that are relevant to your search. Google also prefers pages in which your query terms are near each other. To enter a query into Google, just type in a few descriptive words and hit the 'enter' key to generate a list of relevant results. Google searches are not case sensitive. All letters, regardless of how you type them, will be understood as lower case. I have always found Google to be particularity helpful for Technology-related searches.

Search Agents and Bots are software tools for digging through data. You give a bot directions and it brings back answers. A bot is the perfect way to perform complex methodical searches. The term bot has become interchangeable with search agent, to indicate that the software can be sent out on a mission, usually to find information and report back. Strictly speaking, an agent is a bot that goes out on a mission. Because data mining often requires a series of searches, bots can save labor as they persist in a search, refining it as they go along. Agents and bots allow you to search multiple sources simultaneously.

The Bot Spot, http://bots.internet.com, offers you a comprehensive list and links to dozens of Bots and agents. It also offers a frequently asked questions section to help you make the most of your bot search

Search Engine Tips and Tricks

Some general strategies to help you find the on-line information you need:

· Match your search tool to the information you are seeking

· Decide which search features would be most helpful-Boolean logic? Proximity? Truncation? Natural language?

· Start your search with a specific request and then widen your parameters as necessary

· Use a number of search engines and strategies. Don't be afraid of the 'Shot in the Dark' school of information retrieval.

Where To Go When You Can't Find What You Need

http://www.informationoutpost.com The Information Outpost staff will search the Internet for you for free! Tell them what you are looking for and they'll find the best, related sites and send you an email to let you know when they have finished searching for you. Their goal is to respond within 24 hours. Please provide as much background data as possible-they don't always understand your jargon and abbreviations.

There is a great research list through Painted Rock. Go to their site at http://www.paintedrock.com/memvis/about/listsrv.htm#anchor145624 and subscribe to the prock-research list. It's a list. You post research questions and people either answer them or refer you to good sources. Sometimes you don't get an answer, but usually someone comes up with something.

One of the best ways to connect to 'peer information sources' is http://www.liszt.com - Liszt is a directory of Internet discussion groups: mailing lists, newsgroups, and chat channels.

Search Engines Can't Find Everything

Some information on the World Wide Web is not accessible, even to the most aggressive search engine technology. The following is a list of the type of information that is usually search engine proof.

· Content of Adobe PDF and formatted files
· The content in sites requiring a log in
· CGI output such as data requested by a form
· Intranets; pages not linked from anywhere else
· Commercial resources with domain limitations
· Sites that use a robots.txt file to keep files and/or directories off limits

Retro Search

Not all the information you're seeking will be on the Web. Information shows up on the web only if someone had a personal, educational or business reasons for putting it there. Often your best resource is people. Find a pertinent newsgroup or mailing list and ask you question. There are often a lot of experts on esoteric subjects frequenting specific mailing lists and newsgroups and often people who know everything about the most esoteric subject and are eager to help. Or of course you could always call or contact the writer's best friend…the librarian. Often just talking with a knowledgeable researcher will send you off in a new direction. There is always more than one to get to that one nugget of information you need.

If you'd like to add some excellent books about the ins and outs of on-line research to your personal bookshelf, the following are excellent resources.

Great Scouts! : CyberGuides for Subject Searching on the Web by Nora Paul, Margot Williams, Paula Hane (Editor); Cyberage Books; ISBN: 0910965277;
$19.96

Find It Online: The Complete Guide to Online Research, by Alan M. Schlein, James R., Jr Flowers (Editor), Shirley Kwan Kisaichi; Facts on Demand Pr; ISBN: 1889150207; $17.95

Super Searchers Do Business: The Online Secrets of Top Business Researchers, Mary Ellen Bates. CyberAge Books. $19.96.

The Extreme Searcher's Guide to Web Search Engines: A Handbook for the Serious Searcher, by Randolph Hock, Paula Berinstein; Information Today Inc; ISBN: 0910965269; $19.96

Researching Online for Dummies® by Reva Basch; Dummies Technology Press IDG Books Worldwide; ISBN: 0764503820; $24.99

All of these books are available for purchase at Amazon.com. Be sure to connect to Amozon.com through the HODRW web site.

Save yourself some typing. These, as well as other sources not listed here are located on my web site:
http://www.pleiadespublishing.com/Searchengines.html

Sandra Chevalier-Batik is senior researcher and technical writer for Pleiades Publishing Services. She develops content and information design for web sites and corporate communications.