Path: senator-bedfellow.mit.edu!dreaderd!not-for-mail
Message-ID: <internet/info-research-faq/[email protected]>
Supersedes: <internet/info-research-faq/[email protected]>
Expires: 11 Dec 2000 09:41:41 GMT
References: <internet/info-research-faq/[email protected]>
X-Last-Updated: 2000/08/27
Organization: none
From: [email protected] (David Novak)
Newsgroups: alt.internet.research,sci.research,alt.answers,sci.answers,news.answers
Subject: Information Research FAQ v.4.3 (Part 9/9)
Followup-To: poster
Approved: [email protected]
Summary: Information Research FAQ: Resources, Tools & Training
Originator: [email protected]
Date: 28 Oct 2000 09:43:22 GMT
Lines: 1866
NNTP-Posting-Host: penguin-lust.mit.edu
X-Trace: dreaderd 972726202 5725 18.181.0.29
Xref: senator-bedfellow.mit.edu sci.research:20692 alt.answers:52014 sci.answers:12300 news.answers:194687

Archive-name: internet/info-research-faq/part9
Posting-Frequency: monthly
Last-modified: variable:last_modified
URL: http://spireproject.com
Copyright: (c) 2000 David Novak
Maintainer: David Novak <[email protected]>

                 Information Research FAQ     (Part 9/9)
                             Also known as:
                 Searching, Information and the Internet
         By David Novak of the Spire Project (SpireProject.com)

   Welcome. This FAQ addresses the methods, resources and skills used in
   information research. Particular attention is paid to the role of the
   Internet as both a reservoir and gateway to information resources.
   Midway through 2000 we began to add a narrative to this work to make the
   read more interesting, the search skills more obvious.

   This FAQ/ebook is an element of The Spire Project, the primary free
   reference for information research and an important resource for search
   assistance. Do visit the website. It is free and compliments this FAQ
   with greater depth, forms, links and tools. This document resides at
   http://spireproject.com/faq.txt and http://spireproject.co.uk/faq.txt

   Enjoy,
   David Novak - [email protected]
   The Spire Project : SpireProject.com and SpireProject.co.uk


                             Search Tactics.
                                Section 5

   If searching be science, art and experience, the science of searching is
   the easiest of the three. There are just a few search elements to
   remember and search techniques to apply.

   Firstly, there are the simple search tactics of Boolean, proximity,
   truncation, field searching, target searching and further enhancements.

   You must also become familiar with the basic classification schemes: the
   Dewey decimal system (for books) The WIPO and US Patent Classification
   Systems (for patents), the Standard Industrial Classification (SIC)
   Codes (for industry) and a number of additional classification systems
   founded on the same principles. It helps to be familiar with the
   organization of large directories like Kompass and the Gale Directory of
   Databases, with Subject Listing, Alphabetical Listing, Geographical
   Listing, and then a separate numerical arrangement of specific data.
   Working with these directories can be very confusing at first. Certainly
   experience makes this easier. Understanding the arrangement in a general
   sense allows you to apply the same tactics in other similar situations.

   Lets start with the technique associated with searching a text database.

   Straight Word Searching:
   All search situations allow you to ask for the presence of words in a
   block of text. Obviously it helps if you ask for the right word or words
   - the ones present If you ask for the right words, they you will quickly
   locate the information you desire. For best results, you obviously want
   to chose a word or words which accurately describes what you are looking
   for.
   search the desired text several times with different terms, and you
   consider the possibility of different spellings for the same words. I
   use this frequently to locate information in web pages, in large
   documents like online directories or the archives of past discussion on
   forums.

   Text Fragments:
   The simplest refinement to straight searching involves searching for
   parts of a word - if you are interested in surfing, search for surf
   better yet, search for " surf" with the space in front of the word.

   Truncation:
   Some search engines don't allow searches for text fragments, and you
   must explain your intention by adding a truncation mark (usually * or ?)
   to the ends of words. For most professional researchable alga? will
   include both algae and algal. I was once badly lost because of the
   spelling difference between aging and ageing. There are a number of
   improvements on this concept to. Sometimes there are special symbols for
   a non-space character car?a, sometimes there is automatic awareness of
   multiple spellings (colour & color). Sometimes there is even automatic
   awareness of synonyms. Often you are initially unaware important
   information is indexed under slightly different spelling, so truncation
   is strongly suggested for most searching.

   Thesaurus:
   An improvement on truncation is the opportunity to look directly at a
   list of words, either keywords, or descriptors. This allows you to see
   the range of spellings before you search. This is also ideal for
   searches of company names or proper places so you can select only the
   words you are interested in. In a simple way, some library catalogues
   present subject searches in this way: a list of subject categories
   arranged alphabetically.

   Boolean operators:
   Changing tack, searching for multiple words calls for "and, or, not"
   concepts. I want this word and that word, but not another word. It is
   simple enough. Many of the search engines allow for this with the -sign,
   and commercial databases often add brackets. Use of the not symbol is
   frowned upon in textbooks (too easy to dismiss information you are
   interested in it is said), but the 'and & or' is absolutely necessary
   for complex questions like I want [(spaghetti or noodle) and pasta] or
   (Italian and cuisine). With most Internet search engines, but not all
   commercial searches, you will find 'and' is assumed.

   Proximity operators:
   The next dramatic improvement fixes the position of words relative to
   one another. In this category we have adjacent (often written as adj,
   next, or "inserted in quotes"), near (by how many words), or in the same
   sentence. Often it is wise to stretch the distance a little (within
   two), but where available, proximity is best way to remove the dross
   without affecting the value of information. "Patent near Research" is
   much more precise than "Patent and Research".

   Fields:
   By separating information into different fields, we can selectively
   search different portions of the information. I want the title to show
   the words "Patent" and the abstract to include the words "Patent
   Research". Field searching is a common way to refine a search, but be
   aware searching titles is very likely to remove some desired
   information, where as searching descriptors and not abstracts may
   dramatically improve the content.

   Date Fields:
   Are you really interested in information more than 15 years old? Library
   catalogues frequently have many aging books, and date limiting is very
   wise.

   Further Enhancements:
   Ranking and the ability to search multiple databases are some of the
   further enhancements which select databases permit. There are also
   advances that do not have a grand impact - like natural language.
   Natural interpretation allows the searcher to phrase a question with
   common sentence structure. The computer then interprets what you want.
   In theory natural language is liberating but in practice the strengths
   of Boolean, proximity and field searching far exceed the benefits of
   natural language searching. Lastly, there are special techniques like
   target searching available on a few systems that bear discussing.
   Sorting allows you to shape the presentation of the information. When
   applied to financial information, this is particularly valuable. Alerts
   allow you to automatically repeat a previous search and have the
   information sent to you. Multiple database searching allows you to
   search a collection of databases concurrently. Ranking positions certain
   information at the top. These techniques can be valuable in certain
   circumstances.

   These technical options improve the blunt system of simply asking for a
   word. You will find most search functions allow for some of these
   options and all commercial quality databases provide for numerous
   functions. The good news is an experienced searcher can accomplish
   wonders - collecting articles of 70%+ interest regularly on expensive
   database. The bad news is most of the best of search technology is not
   implemented on all the databases you will search and only occasionally
   on databases free on the Internet.
   ___________________________________________________

   Classification

   There are several search techniques associated with library catalogues.
   Beyond the simple author/title/subject search, we should also consider
   searching by dewey number, and searching first for any title - then
   selecting the subject fields.

   Dewey Searching
   The Dewey decimal system is similar in many ways to the patent
   classification system. Each step is divided into 10 - getting more and
   more specific. See this CAL State Dewey list
   (http://www.calstatela.edu/library/guides/Dclass.htm) to get an idea of
   its structure. This number here refers to a book called Australian
   government assistance to local government projects:

   The Dewey system is arranged by Discipline, not subject groupings. Each
   digit to the right becomes progressively more detailed. The system works
   well in organizing books - and libraries expand it to suit their needs -
   but it is different from a subject catalogue. Because it is arranged by
   discipline, subject fields may be split.

   In searching, we want to duplicate the walk to the shelves and browsing
   other publications that share similar numbers. We do this electronically
   by searching/browsing books that share most of a number. Drop a digit -
   expand the field of interest.

   The Dewey system is a bit congested in certain areas, giving rise to
   very long numbers. For this and historical reasons, several national
   libraries do not use the Dewey system. The Library of Congress, for
   example, has its own classification scheme (Outlined here
   http://lcweb.loc.gov/catdir/cpso/lcco/lcco.html ).

   Subject Searching
   We can do better than searching the subject index of a library
   catalogue. Try instead to search for a book which interests you - which
   you can usually find easily with a simple title search - and then
   selecting the subjects that book are indexed under.

   Many of the library catalogues are making this particularly easy by
   incorporating links into the catalogue results. A quick look at the
   Library of Congress, for example, will show how all the subject fields
   are linked to further searching.

   We can show this in action by looking at the book Earth Time [1] by
   David Suzuki, at my State Library. As you can see down the bottom, it is
   indexed under Social Ecology [2] and Human Ecology [3].

   This kind of 'locate then expand' is an effective search technique used
   in a number of situations. In commercial databases, we may search for a
   company then expand to make sure we catch any different company
   spellings. We may also wish to search for a book, then search for books
   by the same publisher.

   [1]
   http://henrietta.liswa.wa.gov.au/search/asuzuki+david/1,2,46,B/frameset&asuzuki+david+t+1936&11,,45
   [2]
   http://henrietta.liswa.wa.gov.au/search/dsocial+ecology/-5,-1,0,B/browse
   [3]
   http://henrietta.liswa.wa.gov.au/search/dhuman+ecology/-5,-1,0,B/browse
   _______________________________________________

   Patent Classification

   All patents are given a special number.
   Unfortunately, each country has a distinct numbering scheme: US patents
   are assigned a consecutive patent number (currently 6 million+).
   Australian patents have an alphanumerical which includes the year.
   Canadian patents are numbered.

   Above these numbering systems, we have the International Patent
   Classification (IPC), by the World Intellectual Property Organization
   (WIPO[20]). Most every country uses the IPC to classify patents, save
   the US. US Patent Classification is similar in many ways.

   International Patent Classification
   Thanks to the World Intellectual Property Organization (WIPO) [1], the
   International Patent Classification (IPC) works as a universal
   classification for patents. Started in 1975 and periodically updated, we
   currently use IPC 6th Edition (1994). Work on IPC 7th Edition is well
   advanced.

   Section, Class & Group. The International Patent Classification looks
   like this: A 02 J 1/00
   At the heart of the IPC is the unique coding of every invention by its
   specific form or function. The system is highly specific and logical,
   and includes numerous cross-references to other codes of similar form or
   function. Think of this as the Dewey Decimal System for patents.

   The first letter is the section - one of eight broad categories labeled
   A through G. 'A' represents Human Necessities. 'B' covers Transport.

   Each section is divided into Classes. Each class includes two numbers.
   In addition, each class is divided into subclasses, the letters which
   follow the first number.

   Each subclass is then divided into groups and subgroups. The number
   before the slash is the group, the number after the slash is the
   subgroup. Subgroups only have two digits, with further numbers
   considered as resting behind a decimal point: 3/46 then 3/464, then
   3/47.

   Thus A 47 J 27/09 includes the safety device on your rice cooker and B
   63 G 11/00 covers your various aircraft carriers.

   The IPC system is fully described in these published directories:
   The Official Catchword Index by World Intellectual Property
   Organization.
   International Patent Classification: Guide, Survey of Classes & Summary
   of Main Groups
   International Patent Classification: Section G - Physics
   International Patent Classification: Guide

   Thanks to the World Intellectual Property Organization (WIPO), these
   full documents are online [2]. We now have direct access to the
   International Patent Classification (6th Edition): Official Catchword
   Index [3], Guide to the IPC[4], and the complete Class and Section books
   [5].

   Note: The International Patent Classification includes plenty of
   internal references - indicating this group is similar to another group;
   motorized boats take precedence over boat function. These internal
   references are important to effectively searching databases. There is
   more to the IPC, and we strongly recommend you read the Introductory
   Manual to the International Patent Classification (IPC)[6] found on the
   WIPO website.


   US Patent Classification

   US Patents are classified with 400+ main classes and thousands of
   subclasses. Sound similar to the International Patent Classification? It
   is. US patents are numbered sequentially.

   This means you can find US patents:
   - by full text searching through the USPTO database CASSIS (found at US
   patent libraries),
   - by bibliographic & abstract text searching online through the USPTO or
   IBM Patent Library,
   - by US Patent number by US Patent Classification class & subclass - to
   list similar patents by an effective combination search
   - by the searching recent notices in the Official Gazette... available
   online.

   The USPTO allows you to search or browse the US Manual of Classification
   [4] online. The Internet Patent Search System [7] lets you to browse US
   Patent titles by class/subclass.

   A little more information can be found with the Patent Guide to using
   CASSIS [8], at the University of Michigan.

   Patent Search Strategies
   Here are the avenues open to you:
   1_ Full text search and retrieval through a commercial database.
   2_ Free bibliographic & abstract searching online followed by selective
   patent perusal/ordering.
   3_ Paging manually through the relevant official gazette (the US gazette
   is searchable [9]).
   4_ Retrieval of the titles & abstracts within appropriate class/subclass
   then selective review and patent perusal/ordering.

   This last avenue is particularly resourceful and swift. Start by
   reaching for The Official Catchword Index [3], a book by World
   Intellectual Property Organization (WIPO). This will tell you the
   possible class/subclasses that will interest you. You could word-search
   a patent database and note all the class/subclasses found. Lastly, you
   can always reach for the three separate printed guides that lead you
   from section to subclass.

   The result should be a collection of class/subclasses that may interest
   you.

   With this information, you can now browse all the patents in the
   class/subclass. This process will help you locate all the patents that
   may interest you since patent classification is more reliable than free
   text search. (Note, both British and American spelling appears in patent
   databases.) This also allows you to quickly review the patents in other
   countries.

   If you are undertaking a novelty search - is a patent sufficiently
   unique from other existing patents - then you must review more than one
   country. There can be a significant delay before patent applications
   reach other countries without affecting the protection. Case in point:
   Australia only accounts for 7% of the world's patents.

   Further Search Strategy
   Patent search strategy is further discussed in the Introductory Manual
   to the International Patent Classification (IPC)[6] found on the WIPO
   website. You may also wish to reach "Searching for Patents" [10] from
   the University of Michigan, and "Patents" [11] by Simon Fraser
   University Libraries.

   [1] http://www.wipo.org
   [2] http://www.wipo.org/eng/clssfctn/ipc/intro.htm
   [3] http://www.wipo.int/eng/clssfctn/ipc/ipc6en/nfcatch/index.htm
   [4] http://www.wipo.int/eng/clssfctn/ipc/ipc6en/guide/ent00001.htm
   [5] http://www.wipo.org/eng/clssfctn/ipc/ipc6en/index.htm
   [6] http://www.wipo.org/eng/general/ipc/manual
   [7] http://metalab.unc.edu/patents/intropat.html
   [8] http://www.ummu.umich.edu/library/PTO/newCASSIS.html
   [9] http://www.uspto.gov/web/offices/com/sol/og/
   [10] http://www.ummu.umich.edu/library/PTO/newpatsearch.html
   [11] http://www.lib.sfu.ca/kiosk/nelles/patents.htm
   ___________________________________________________

   Trademarks

   Trademark law is designed to protect consumers from confusion. The law
   can work to protect business investment in brands & slogans, but only if
   the business behaves in particular ways which protect consumers from
   confusion: actively using the trademark, working to restrict the
   trademark from becoming generic, routinely searching for unauthorized
   use.

   For a very clear description of trademark use, and the responsibilities
   of trademark owners, read the short webpages A Guide to Proper Trademark
   Use[30], and How are Marks Protected[31] both by Gregory Guillot.

   Trademark Law has implications for searching: Just because a potentially
   conflicting trademark has been found does not mean it should concern
   you. It may be simple to show or argue that trademark ownership has
   lapsed and become abandoned unintentionally.

   A Guide to Proper Trademark Use[1] by Gregory H. GuillotA common law
   search involves searching records other than the federal register and
   pending application records. It may involve checking phone directories,
   yellow pages, industrial directories, state trademark registers, among
   others, in an effort to determine if a particular mark is used by others
   when they have not filed for a federal trademark registration.

   The system may appear particularly legalistic, and it is. Recent
   Australian Trade Marks Office Decisions[32] information ultimately
   supplied by IP Australia, displays this vividly. However, much trademark
   activity is self-evident. In Australia, A$350 and a minimum of seven and
   a half months will usually earn you a registered trademark. Should you
   chose a trademark and find another has used it, you will most likely
   receive a 'cease & desist' letter and forfeit the value you may have
   invested in the trademark.

   This leads us to the importance of commercial trademark databases,
   watching services and other commercial services. Searching both prevents
   investment in an unusable trademark and inadvertent infringement by
   others - a responsibility of trademark owners.

   Trademark Classification
   A concise list of the 42 classes of the International Trademark
   Classification codes courtesy of Master-McNeil Inc[33]. WIPO is in
   charge of the full class description, currently The 7th edition of the
   Nice Classification[34], but this is rather lengthy. IP Australia has a
   simple search feature of classification terminology[35].

   Trademarks are assigned to a particular class of product or service. A
   slogan or mark, for example, could be registered for use in movies but
   not computer products. The situation has changes recently but let us
   explain the difference down the page a bit.

   Originally, all goods and services were broken down into 42 classes.
   These classes are international divisions organized by WIPO (World
   Intellectual Property Organization), so are the same from country to
   country. Registered trademark documents will explain at length the types
   of products & services covered by a particular trademark.

   There is some bleeding between categories, and trademark examiners are
   unlikely to grant requests for nearly identical trademarks in similar
   categories, but class plays a role in granting trademarks.

   Recently it became necessary to list specifically the products or
   services to be covered, and the 42 classes have been expanded to a
   collection of specific sub-classes, which is reminiscent of patent
   classification, but far less useful.

   Class is important as trademarks are class-specific. You can search by
   class in certain registered trademark databases, but this is not
   particularly a good search technique: you are far too likely to miss a
   comparable trademark.

   Trademark Picture Descriptors
   Search Image Descriptors[36], by IP Australia, here abbreviated, needs
   basic words - simple like bird or butterfly.


   One difficulty with trademark searches is that all the tools apply best
   to words which appear in trademarks. What of the picture? The solution
   appears to be image descriptors. I am uncertain of the international
   nature of image descriptors, but at least in Australia, there is a
   standard set of image descriptors. IP Australia allows you to search for
   other trademarks with a particular picture element - irrespective of the
   words involved. But to do this, you must first select the appropriate
   image descriptor.

   Conclusion
   Trademarks are just one element of intellectual property rights;
   patents, copyright, industrial design rights, circuit layout rights and
   plant breeders rights. As certain registered trademark databases are
   free online, some trademark research can be accomplished quite simply by
   the novice.

   Why search?
   1_ To find existing trademarks similar to one you plan to register.
   2_ To find existing trademarks similar to one you plan to use as a
   trademark.
   3_ To see if a trademark is similar to a business name you consider
   using.
   4_ To search for possible infringing trademarks.

   This is further explained in this help file [37] by IP Australia.

   Further Assistance
   Misc.int-property has a lively usenet discussion on Intellectual
   Property. Access the newsgroup directly: misc.int-property [40] or
   search the past discussion through Deja.com's usenet archive).

   For a lively discussion of how trademark law affects Internet domain
   names, consider the trademarks-l mailing list at Washburn University
   (read the Scout Report description [41]).

   [30] http://www.ggmark.com/guide.html
   [31] http://www.ggmark.com/protect.html
   [32] http://www.austlii.edu.au/au/cases/cth/ATMO/recent-cases.html
   [33] http://www.naming.com/icclasses.html
   [34] http://www.wipo.int/eng/clssfctn/nice/about/index.htm
   [35] http://xeno.ipaustralia.gov.au/tmgoods.htm
   [36] http://xeno.ipaustralia.gov.au/device.htm
   [37]
   http://pericles.ipaustralia.gov.au/atmoss/falcon/help/help.html#WHY_SEARCH
   [40] news:misc.int-property
   [41] http://scout7.cs.wisc.edu/pages/00000138.html
   ___________________________________________________

   Industry Classification

   Lastly, we have not yet researched the categorization of industries
   using standard SIC or NAICS codes. In simple terms though, all
   industries are given a specific code. Sub-industry is given a more
   specific code. More and more specific codes refer to the production of
   more and more specific items. Of course, some companies will be involved
   in a collection of industries.

   Two competing standards, the SIC and NAICS, have different codes but the
   same coding system. Each code system can be mapped on the other, so will
   cause you no undue concern. Trade statistics, digital business
   directories, and national statistical bureau industry data will all use
   the industry codes.





                          Information Quality.
                                Section 6

   Information has value. It also has other qualities that will assist you
   to judge information you may consider buying.

   Accuracy: the factual nature of the information presented. If the
   statistics purport to show a particular trend - how large is the margin
   of error? How large is the sample size? How likely are there to have
   been factual errors in their development? The measurement of statistical
   error is now a refined science in some fields. A statistical result can
   be inaccurate when the sample size is too small, if the margin of error
   is too large, the sample collection procedure incorrect, or a number of
   other situations.

   Reliability: the support for trusting the solutions, both from
   additional resources and from being able to duplicate the conclusions.
   This includes the reputation of the researchers. No matter how
   inaccurate and biased you may believe certain facts to be, successful
   independent support of a suggested fact does improve its value.

   Bias: conscious or subconscious influences that affect information. Bias
   can occur in collection, preparation and presentation of information.
   Most information you find will be tainted. Secondary information is
   deeply affected. Statistics are not necessarily less biased.

   We counter bias in several ways. Firstly, we try to be aware of bias.
   Where is bias likely? Which direction would the bias affect the
   information? Secondly, we try to collect information with different
   bias. This is why research based solely on government research, no
   matter how accurate and reliable, is less valuable. Often information
   from different countries can counter bias. Thirdly, we need to accept
   bias is likely to exist. This is why primary sources are often more
   valuable than secondary sources. This is why tertiary sources, like
   experts, can rarely stand alone.

   Age: The date information was created or compiled will feature
   prominently in the value of information. Dates given sometimes mean the
   date information was created, or the date information was compiled. How
   old is a book compiled in 1995, which took the author 10 years to
   finish? I find statistics often forecast information, prominently
   displaying recent compilation dates but still use old census data or the
   like to draw their conclusions. Information on the Internet typically
   has no date, and can be severely challenged because of this.

   Purpose: purpose merits further discussion. When you are uncertain about
   potential bias, you can look for reasons to distrust the information
   instead. Suspicion is not equivalent to bias, but it can be thought
   provoking. Privately, I have heard repeated rumours important national
   statistics have been fudged in different countries. A government
   research report investigating the price of books in Australia would have
   a political purpose, a purpose that provides the climate for some
   potentially significant bias. A tell-all book by industry experts often
   includes a tremendous quality of insider experience difficult to find
   elsewhere. While there may be a purpose of self-aggrandizement, the
   purpose is less a climate for significant bias. Medical research has
   perhaps the greatest climate for significant bias, and this suggests the
   greatest standard of proof and external, reliable support.

   Accuracy, reliability, bias, age and purpose are very important in
   research. This is what leads us to an appraisal of value. For years, the
   tobacco industry funded 'independent' research finding smoking minimally
   harmful to health. It is now likely there may have been errors brought
   on by accuracy, and bias. Certainly, purpose was in doubt. As new
   studies show smoking is harmful, we can also say the original research
   lacked reliability. In some topics, like the Internet, research is
   perpetually suspect because it also ages so quickly.

   I have seen further discussions that add 'Coverage' and 'Authority' to
   this checklist. Both have bearing on the value of the information
   contained. By coverage, we mean how much detail is invested in covering
   a specific topic. Sparse or shallow coverage is closely tied to missing
   critical aspects of information. News stories frequently have limited
   coverage.

   Once you are acclimatized to these elements, you begin to see potential
   for error in a whole range of information. Real-estate association
   figures, expert opinions, Toothpaste advertisements and National GDP
   figures all occasionally display some degree of warping and
   manipulation, clouding the truth. The solution is awareness, comparison
   and careful analysis. As a personal aside, this is part of the reason
   for my personal dislike for market research: it is often taken far more
   seriously than warranted and mean far less than suggested.

                         Searching as Industry.
                                Section 7

   Of interest to you now, the Internet offers you a very good look at the
   information industry. Most organizations involved in the information
   industry publish exhaustive product descriptions on the net. Most
   commercial products are delivered electronically.

   Professional Search Resources

   As a profession, researchers have diverse skills and needs. Constantly
   working with information, in a competitive market, professional
   information seekers are often starved for high quality information about
   new research techniques, skills and sources. This can be found through
   discussion groups like Buslib-l, websites on library science like
   LisNews.com, associations like the Association of Independent
   Information Professional (AIIP) and the Society of Competitive
   Intelligence
   Professionals (SCIP), events and conferences as listed in the journal
   Online & CDROM Review.

   As a more introductory resources, start with the a selection of books
   and webpages like:
   - The Intelligence Cycle[1], courtesy of the CIA library - a single-page
   summary of the research process.

   - The Information Broker's Handbook by Sue Rugge and Alfred
   Glossbrenner, McGraw-Hill. Third Edition (1997) - a must-read for those
   interested in the business side of information research.

   - Secrets of the Super Searchers by Reva Basch. Unfortunately a 1993
   book, but unique as a look into the field of information brokers.
   Published by Eight Bit Books. (Dewey 025.524 BAS)

   - Online is a good bi-monthly magazine for information brokers. (Dewey
   025.04).

   There are a number of interesting periodicals, most owned and marketed
   by Information Today Inc. BUBL lists a number more [2]. Others are
   electronic publications, like LIBRES [3]: Library and Information
   Science Research Electronic Journal, a biannual scholarly journal and
   Information Research [4].

   The commercial databases of interest are LISA (Library and Information
   Science Abstracts), ALISA (Australian LISA), Information Science and
   Library Literature.

   The links for these resources and more are on the Spire Project at
   http://spireproject.com/links.htm#3

   [1] http://www.odci.gov/cia/publications/facttell/intcycle.htm
   [2] http://bubl.ac.uk/journals/lis
   [3] http://aztec.lib.utk.edu/libres
   [4] http://www.shef.ac.uk/~is/publications/infres/ircont.html
   ___________________________________________________

   The Professional Search

   Professional research demands a more effective, timely use of resources
   at hand. It is challenging, and it is an occupation.

   Unlike research undertaken for your own needs, professional researchers
   often know little about the topic they are asked to investigate. We may
   not know the phrases which accurately describe a specific concept, we
   sometimes don't recognize gold if its labeled copper, but we have to do
   everything fast - lest the cost escalate above the expectation of the
   client.

   Client. Yes, professional research starts with the client.

   Professional research involves far less book and library work, and far
   more interviewing, database access and online article purchasing. When
   money is involved, time becomes very precious. The first luxury lost:
   the luxury to get to know the topic in leisurely detail.

   Instead, professional research starts with a careful description of
   exactly what information is desired (and why). You must quickly build a
   good plan about who you will ask and where you will look. This is, after
   all, your primary skill others have great difficulty in duplicating -
   traversing the information sphere swiftly and skillfully.

   Many researchers today can search databases. Most researchers are
   familiar with library work. Personal research has the added benefit of
   being part of the learning process. So why reach for a professional?

   The first unique skill we must refine is our knowledge of the research
   tools. Computer databases may be easily accessible, but are not easy to
   search. Interviewing is conceptually simple, but is not simple in
   practice. Each aspect of research can and must be refined.

   The second unique skill: interpretation. Working with information
   frequently allows us to better judge the reliability and bias of the
   information we retrieve.

   Most information you find will be tainted. Secondary expertise almost
   always present information in a biased way. You will counter this bias
   both by being aware of the bias and by interviewing someone with a
   different view. An inventor proclaims a devise in near completion - do
   we believe? Obviously it requires further study. This is often lost on
   amateur researchers - by collecting information from a variety of
   different resources, with a range of bias, we can create a superior
   assessment of the value of each item of information. Research based
   solely on government research, no matter how well done, is
   unprofessional.

   The third unique skill is speed. We must be able to provide research as
   a service, as a business, quickly. This goes beyond research to the
   banal work of copyright and legal protection, selecting effective
   research tools, finding fast expertise to supplement your own.

   The skills of professional research are like the artist. They take a
   lifetime to learn. The work is just business.
   ___________________________________________________

   The Database Industry

   The commercial information sphere existed in the 1970's and earlier. It
   is far more developed, far better organized, far better funded, almost
   always far more valuable and expensive than every other research
   resource.

   For the most part, commercial information is arranged reasonably
   uniformly in large databases of full-text or bibliographic information.
   Some databases are small, single source documents, while others are vast
   unfocused collections of, for example, all the news from the last 15
   years.

   Most directories and journals can be made into a database, but
   single-source databases do not enjoy much financial success. The market
   is too limited and the cost of promotion too high (except in a local
   market with newspapers). To overcome this difficulty, single sources are
   grouped together into larger collections of databases on a particular
   topic. These large database groups have become primary tools in
   commercial research.

   Developing these databases requires considerable expertise and expense.
   Sometimes data requires abstracting, interpreting, and as with some
   Lexis-Nexis and WestLaw databases, even expert legal interpretation.
   Sometimes firms develop a portfolio of databases. Sometimes firms build
   just one.

   The marketing and consumer billing of such databases is then provided by
   a relatively small collection of large database retailers. A list can be
   found in our "Commercial Databases" article. As an indication of the
   size of this market, Knight-Ridder sold Dialog & Datastar for a figure
   approaching half a billion dollars.

   This industry consisting of a wide collection of players, each improving
   and developing the information from individual periodicals, journals,
   news items... All very confusing for the end user.

   This is elegantly illustrated by the database descriptions for
   Lexis-Nexis databases (their preferred term is libraries). See
   http://www.lexis-nexis.com/lncc/sources/ as an example of specific
   databases. In particular, see their library on patents.

   Many single-sources appear in different commercial databases. Further,
   different databases sometimes include different information from the
   same single-source. One database may include just abstracts, another may
   include fulltext, chemical indexing and more.

   As a result, most researchers are unfamiliar with what exactly is being
   searched.

   This state of affairs is not unproductive. Searching a 'Database about
   Patents', is uncomplicated. You receive information on Patents. It is
   simple, informative and incomplete. Of course, researchers are busy
   people. Time is critical. Results matter. This system also gives rise to
   great customer loyalty to database retailers. Comparative information is
   dropped in favour of simplicity. (There is too much complexity for
   researchers anyway.) Unfortunately, I am hard pressed to compare prices
   let alone describe the differences between information products.

   Prices actually model many a developed industry, remarkably similar to
   the telephone or banking industry. As one friend commented, "bullshit
   baffles the brains". The prices are complex on purpose. It becomes very
   unrewarding to compare prices, and any conclusions are only valid in
   specific circumstances - and will not hold in others. This trend,
   familiar to us as a multitude of banking changes and telephone pricing
   schedules, reinforces our need to stop price hunting and trust our
   favoured information retailers.

   This is not to say we should not compare prices - but for the most part,
   you will find comparing prices a most unrewarding experience. It really
   requires you to search and retrieve the same information on different
   systems - and this does not even begin to touch different databases, or
   database groupings, or variables that change over time like download
   speeds.

   Optimistically, there are actually very few important databases in each
   field. It may be simple to browse each of the databases in your field
   and compare directly. You may never need to know more than a few
   databases intimately.

   Realistically, you will yearn for a simpler solution.

   The commercial information industry has distributed information this way
   for several decades. It is both sophisticated and quite difficult. You
   will need to become experienced with inverted indexes, search techniques
   (Boolean, truncation, proximity, field limits ...) and properly phrasing
   the question in a way that will be answered by a database search. I have
   always found the value of a database search directly proportional to the
   length of the search query.

   If you are incompletely skilled at database research, you will take
   longer, pay more and locate far more information (or unwisely discard
   more) than desired.

   This is very different from searching Altavista and Webcrawler.

   Doing your own research offers an opportunity to more closely influence
   the research process. Sometimes only you understand the topic and
   sometimes you can more quickly discard unimportant details. Certainly it
   is becoming simpler to undertake some work yourself.

   Many of the commercial databases are also available in a CD format.
   Substantial subscription costs limit their availability to large
   research institutions and libraries, but exceptions exist. I believe
   world books in print costs AU$5000+. Provided you can find casual
   access, it will cost you far less. Keep an eye on the age, though.
   Sometimes (and only sometimes) online information is more recent.

   The decision between undertaking research on your own or seeking
   external help is really a decision based on your research expertise,
   your budget, your access to information, your time, and the importance
   of finding all the information available. It also depends on your access
   to some decent research assistance. I will soon be able to help with
   this.

   What I do know is a newcomer to the commercial information sphere will
   seriously underestimate the difficulty involved in searching, and
   underestimate both the cost of research and the cost of research
   assistance. Keep in mind this same system serves the needs of large
   commercial conglomerates, professional legal research, and well financed
   government studies. The commercial information sphere contains far more
   valuable information than you need. Sometimes the Internet is just an
   interesting sneeze in comparison.

   �  Article: The State of Databases Today:2000 by Martha E Williams,
   tracts the development of this industry with survey results. Found in
   the forward of the Gale Directory of Databases.
   ___________________________________________________

   The Information Service Industry

   Private Detectives, Professional Database Researchers, Library
   Researchers, Legal Researchers, Commercial Database Producers,
   Commercial Database Retailers, Magazines, News Organizations, Libraries,
   this is a big industry. Information Research is just a process linking
   together people seeking information with people who provide it.

   It seems in vogue to reconsider all businesses as being in the
   information business. My accountant and your stockbroker both provide
   information services. While I agree these two professions are intensive
   users of information, I purchase their interpretation of information. It
   is not a subtle difference but nonetheless it serves to cloud the true
   size of the industry just involved in selling you access to information.

   From university days, I was aware of the large commercial database
   retail giants (Dialog, Dun&Bradstreet) and the database producers. I
   also met with some of the firms distributing largely to the library
   market (like SilverPlatter). Little further information about these
   businesses leaks beyond the research industry.

   Some of the businesses are aimed primarily towards the library
   community. Database subscriptions are unlikely to interest an
   individual. Few are appropriate to businesses. Let us briefly scan the
   products and services intended for a consumer.

   Commercial Database Retailers - These organizations devote their effort
   at bringing commercial database information to individuals. Dialog,
   Datastar, Infomart, Lexis-Nexis and others will assist you to access
   information only available through commercial databases. (See our
   article, "Commercial Databases".)

   Current News and Current Awareness  - If you want to know of new
   articles and news important to you as it is reported, then there are a
   selection of services available: news by email, news by newsgroup, news
   by periodic automated database search, and other novel approaches. Costs
   for this service have fallen dramatically: effective solutions start at
   about US$10/month and are not strictly dependent on range & quality of
   information. (See our article, "Newswires & News Databases".)

   Information Brokers  - There is a whole industry of specialized
   researchers who will try to locate and compile research to your
   specifications. The backbone of this industry is payment for access to
   commercial databases, but different information brokers will gladly
   enter into any effort required to locate information. Information
   brokers, business librarians, legal researchers and others all use the
   tools described in this website, as a service for their clientele. (See
   our article, "Research as a Discipline".)

   Patent Assistance  - Patent searching is one of the more difficult
   branches of serious research. Some of the resources are free on the
   Internet, and commercial patent databases are readily available through
   the database retailers. If there is serious money at stake, you must
   consider legal assistance. Certainly use lawyers for patent applications
   (beyond the scope of the Spire Project). But patents can also be a
   research tool. Patent research can provide you with what is often the
   first appearance of costly commercial research. This is both a source of
   cutting edge solutions and competitive intelligence.

   Media Monitoring  - Certain firms solely focus on monitoring TV, radio &
   newspapers. These firms typically run teams who page through newspapers
   looking for matching articles, then post or fax to the client. New
   technologies are also advancing into this field.

   Document Delivery  - Most local bookstores will gladly help you locate a
   book from their directories but if you want a book from abroad, or an
   article from a journal or magazine, you will need the assistance of
   another set of information workers. A distinct but similar approach
   assists with the distribution of journal articles. Many of the document
   delivery firms are closely tied to information organizations. Little
   information is available about these organizations.
   ___________________________________________________

Trends in the Information Sphere

   For the past few years, individual database owners/maintainers have been
   flirting with the idea of making paid access available through the
   Internet, rather than the existing system of allowing database retailing
   firms to promote and market their databases. I have heard rumours most
   database producers earn up to 30% of retail price when delivered through
   database retailers - 70% being retained by the database retailer.

   The Internet is not a commercially viable alternative...yet, but some
   databases have emerged with alternative funding despite this (Library of
   Congress, ERIC, Medline). Others are creeping in around the edges by
   offering subscribers access at a much reduced flat annual fee (Computer
   Select at one time). I expect most database producers are waiting for a
   meaningful way to charge. Digital money holds the key but despite the
   hype, practical use appears to be a medium to long-term reality.

   A second trend is Internet publishing itself. Gradually, the information
   is getting easier to locate. (Don't laugh please - its undignified.) We
   are also getting better at using the Internet as a tool to disseminate
   information. We have the very visible, if perhaps short-lived, search
   engines but also other efforts like archives of FAQs, archives of
   guidebooks, applying the Dewey decimal system to the Internet,
   specialist directories, subject guides, specialist search engines. This
   will be a lively field for several years to come. As it gets easier to
   locate the good information, perhaps the lines between commercial
   quality and Internet quality will begin to merge in places.

   The third trend is the very promising prospect of paying for information
   by the page through the Internet - viewing the results in a web page
   immediately. There are some technical hurdles yet, but certain elements
   are already appearing in ventures like DialogWeb. This step may prove
   profitable for ATM vendors and owners of Internet cafes, pubs and
   kiosks. It will also herald a dramatic drop in the cost of information.
   ___________________________________________________

   Are We Developing an Informative Internet?

   Several serious glitches have delayed the further improvement of the
   Internet as an effective information resource. Oh, sure it is the
   world's largest library and thousands of new webpages are published
   every hour. But this trite statement disguises how slow the informative
   value of the Internet is developing.

   Vision:
   The Internet holds so very much promise. Marketing mantras tell us so,
   but few of us grasp this technology will completely rewrite the rules of
   community, government and the exchange of intellectually valuable
   information.

   One of the hurdles is vision. We are not yet delivering the information
   pertaining to community, government and the exchange of intellectually
   valuable (improved) information. We are only proceeding quickly with
   market information and computer-related information. We are still toying
   with further ways the Internet can transform other areas of our life.

   We should have achieved more by now.

   Organization:
   The net is still very disorganized. A number of developments promise to
   eventually make the Internet less confusing and better organized. To
   date, we have several cumbersome techniques, a large collection of
   search tools and a great deal of potentially interesting links.

   Publishing:
   As mentioned, thinking about who is publishing assists us with our
   search. Applying this to where information is emerging - and we learn
   much of the best information is not reaching the Internet. Certainly,
   the commercially generated information is not reaching the Internet
   (covered below). The large research studies paid for by public funds and
   slowly aging on the shelves of government and non-government
   organizations are also not coming online. Government, institutional and
   commercial organizations primarily publish brochure-ware - as befitting
   the presentation of market information. (Even offering to publish such
   documents freely does not appreciably affect this trend as the
   restrictions are not financial, but mindset. See our past work.)

   We should recognize few of the more valuable documents emerge online.

   Further Reading: Socially Responsible Publishing on the Internet ('97)
   (http://cn.net.au/cn/past/docs/publish.html)
   A Census of Regionally Important Documents on the Web ('96)
   (http://cn.net.au/cn/past/docs/webscan4.html)

   Discussion:
   The Internet excites me with the promise of a real community rebirth
   arising from this technology. For the first time in history we should be
   able to discuss in an informed manner any number of issues from crime to
   taxation. Tied into this are issues of government transparency,
   international assistance, anti-corporate market reform and community
   involvement. Unfortunately, my experience with mailing lists and more
   recently with a newsgroup confirm the difficulties in developing
   discussion. Discussion groups function as notice board. Unfortunately,
   the difficulty in developing participation, and in moderation, are just
   a little too cumbersome to be successful. For many discussion groups,
   the chaff overwhelms the wheat, and the information content is far from
   considerable.

   The financial rewards are also minimal for establishing and maintaining
   discussion groups. Dramatic improvement to the informative value of the
   Internet is unlikely to emerge here.

   Further Reading: How to build a discussion on the Internet
   (http://cn.net.au/cn/past/docs/forums.html)

   Rewards:
   We have alluded to the importance of editorial and organization on the
   Internet. There are several severe limitations to this - first and
   foremost the difficulty in gathering financial rewards for meaningful
   work improving and organizing information.

   I am being circumspect here. There is money available - just not where
   it is needed. The most important resources in professional research are
   the contents of the commercial information sphere. This sphere existed
   decades before the Internet, is far better funded, and is far larger. To
   compare commercial and Internet information is almost heresy. A bridge
   between these two, Internet and commercial, emerges slowly.

   Digital money should grease the exchange of information by dropping the
   cost of exchange considerably. Today, credit cards provide this service.
   This works, at times, but digital money would allow for small amounts of
   money to change hands. This appears to be a critical threshold for
   bringing much of the commercial information to the net.

   About 5 years ago I was introduced to the Thesius Model - an economic
   model to pay the intellectual investment in publishing and organizing
   interactive multimedia. Years earlier there was Xanadu. While I have
   serious reservations about both, they do illustrate the intellectual
   foundations for effective use of a tool for exchanging small amounts of
   money. It opens the doors to direct delivery of copyright work - which
   in turn opens an effective economic model for publishing improved
   information on the Internet.

   Without digital money, proprietary information can only be exchanged
   digitally by gift (that is free - the initial driving force of the
   Internet information sphere, or by credit-card purchase of access to
   passwords to external networks - the current method of accessing
   database retailers.

   This has the unfortunate effect of limiting the interest both of
   Internet users in the commercial information sphere and the commercial
   information retailers in the Internet. Oh, there is movement in both
   directions, but not at the scale experienced in other industries.

   Further Reading:  The UWA Theseus Project
   (http://www.arts.uwa.edu.au/TheseusWWW/)
   The Xanadu project (http://www.xanadu.com or concise summary -
   http://www.sfc.keio.ac.jp/~ted/XU/XuPageKeio.html)
   ___________________________________________________

   A Look at Information Congestion

   Finding information on the Internet is a skill. Finding information on
   the commercial information sphere is also a skill. There is a great
   degree of overlap. The awareness of the general public as measured by
   use of commercial resources is very limited. This is further seen from
   the simple use of search engines & the abundance of simple web search.

   To hammer this point in, let's take a momentary look at search engines.
   Most searches end in 1000's of results: here are the first 10. Do you
   really think the first 10 or 20 or 100 sites listed are particularly
   better than the next? No - you have a random selection of resources. A
   selection generated by computer based on the most simple of criterion.
   (We should also mention how some search engines sell placement in search
   results).

   Remarkably, the search engine is the much-vaulted entryway to the world
   of information!?! Clearly search engines will not dramatically improve
   the informative value of the net - not by themselves.

   Multiplication of Information
   One complication of poor information organization is an inflation of
   information overlapping nuggets. Information on the Internet is so
   difficult to locate we have almost a continual need for more publishing.
   Information must exist in numerous locations to reach an intended
   audience. Promotion of the simplest nature - recognition for the best
   for a given topic - becomes exceedingly difficult. Only when 20 sites
   publish or report a given fact does it become accessible.

   Curiously, this is the state of affairs in the wider community.
   Promotion is an expensive speciality. Numerous copies, distributors and
   references are required to generate any kind of significant awareness.
   Why should the Internet be different?

   Actually, why should the Internet be the same? Definitive like the US
   Census Bureau have no need to duplicate this information; to have
   alternative presentation sites. Yet such sites appear the exception.
   Consider a search for the best resources for patent research, we are
   greeted with 954 websites (Altavista search for "patent research"
   Jan-19-2000). Presumably, most of these sites discuss patent research -
   Right? There is no technical or theoretical need for such confusion. I
   wonder if such duplication may be more of an affliction than natural
   tendency.

   Justification
   It is relatively difficult to earn money from publishing improved
   information, or organizing information already on the Internet. Given
   the intense interest in this technology, a collection of models have
   emerged. A brief tour of these models will highlight the financial
   limitations to improving the Internet as an informative resource.

- - - Working for fame (but not payment)
   This model works well in open source software programming, and some of
   this ethic certainly extends to publishing information.
   Simple altruism/complete lack of justification
   School students and Internet novices in particular may not need to
   justify anything. Unfortunately, such work is usually neither consistent
   nor persistent.
   - - - Commercial promotion
   Promotional funds can be used to publish information. Most promotion is
   short-sighted, limited to presenting market information (like product
   information), but in time government and associations will fund
   publishing in-house information for purely promotional reasons.
   - - - Invested commercial businesses
   There are certain commercial opportunities to earn money through banner
   advertising and sponsorship.

   Direct payment for improved information (perhaps with digital money),
   direct payment to authors (Theseus model, royalty systems), and direct
   state sponsorship need not be necessary to fundamentally improve the
   Internet as an information resource. Academic peer-reviewed journals do
   not pay for articles. Commercial periodicals are supported by
   advertising, and the token subscription costs of magazines usually just
   covers distribution costs. Fame motivates many efforts, not just online,
   and we do not feel the need to habitually justify everything we do.

   In no small way, as more people become adept at publishing quickly,
   important information will move on the net faster. Similarly,
   information will also gradually become better organized. Economic models
   will not improve the informative value of the Internet like direct
   payment. Most current limitations have economic solutions.
   Unfortunately, my reasoned opinion is no economic system will arrive in
   time to make a difference.

   Conclusion
   We know something of how information gets published, and how many
   important documents do not reach the Internet. We have described how
   information is organized on the Internet and how limited editorial
   vetting and organization have given rise to certain traits which give
   rise to the traits like superficial indexing, information duplication,
   and a need for research skills.

   Financial rewards and financial tools are unlikely to solve these
   difficulties. We can only hope for a gradual growing out of our current
   difficulties. We will have more of the same for several years to come.
   It is simply the nature of the Internet (as currently constructed).

   For you, a greater understanding of the Internet will assist you to
   judge the worth, likely source and likely venues of the information you
   seek. The same is true in the larger world... database, book & article.
   Each has different traits and qualities, reinforced over time. Your
   understanding of these traits and qualities in part defines your skill
   as a researcher.

   As to the future of the Internet, on the positive side, there are
   certain qualities to Internet communication that make it uniquely
   valuable. Internet communication is inexpensive, relatively rapid, and
   increasingly accessible. On the negative side, the Internet is badly
   vetted, potentially very time consuming, and up against very well
   entrenched systems that have been running for either decades or
   millenniums (considering databases or books). Elements like a promised
   but functionally absent digital money, and the lack of a meaningful way
   to recoup the costs of vetting online information, make matters worse.
   Despite this, despite ALL the teething and fundamental difficulties, the
   Internet is sufficiently superior to ensure considerable continued
   effort to improve the informative value of the net.
   ___________________________________________________

   The Multiplication of Information Effect.

   Just as the Internet premits a multitude of voices and perspectives, so
   it permits - and promotes - a multitude of the same information.Yes, for
   a several reasons we shall explore first, the Internet multiplies the
   amount of information there is on a topic. This insight can be used to
   improve searching for information, as I will show at the end of this
   article.

   The Internet is a system of communication. Like all other systems
   (books, articles) the Internet systems affect the way we communicate in
   different ways. The absolute number of books depends on what is thought
   can be commercially viable. We could say books permit, and promote a
   limited number of books on the same topic.

   The Internet does the opposite.

   The sheer ease of publishing information on the net is one factor in
   information overkill. The net is an easy place to publish information,
   requiring only individual effort. There is no budgetary concerns, nor
   does attracting an audience initially enter into the publishing process,
   as they would with articles or books.

   The ageless state of the Internet also rapidly builds information. Old
   information is not removed from the web automatically as in mailing
   lists. Old books go out of print and past magazine articles are shelved,
   indexed and categorized so we must intentially include them in our
   search. The web is not built this way, and information well past its
   natural expiry date remains.

   A dramatic change is also occuring as our society becomes digital. In
   the pre-Internet economy experts and specialists in every field are
   distributed to meet needs. In the networked world, expertise is not only
   shared more rapidly, but is required in less places - whether we speak
   geographically or intellectually. Said another way, in cyberspace,
   competition for expertise is most fearce. To be an expert, you need to
   be more expert than others within reach - and since gradually more and
   more experts are within reach - digitally - we form a glut of experts.

   Oh, this is not a doomsday message - merely a middle ground on the way
   to increased specialization and focus. Historically we can easily see
   Newton was a Scientist but Einstein was a nuclear theorist. Today we
   have quantum theorists. The future is full of very long job titles.

   A by-product of this movement is a current glut of experts - perhaps a
   permanent glut of experts. With more people connected and satisfied with
   distant communication, a vet who writes about immunizing your dog
   becomes one of many you can reach for, in several countries. Previously
   we may have been limited to those in your state - but no longer! Now we
   can pick up immunization recommendations from any number of experts
   previously separated by distance or with minimal overlapping media
   outlets.

   We can see this clearly on the web. I wrote an article on country
   profiles and yes, as expected, the UK, US, Canada & Australia all write
   and publish traveller advice notices on the web. Are they different?
   Occassionally. Is this a case of multiplication of information? Yes. We
   have reached beyond the applauded Internet trait of permitting a
   multitude of communication and reached a state where similar information
   is interpretted by different organizations, and distributed
   electronically.

   This is not unique to the Internet. News stories also contain
   considerable overlap from one newspaper to another. A search for dog
   immunization on one of the large news databases will result in numerous
   articles all presenting essentially similar information. Business
   periodicals also have considerable overlap, and while each may attempt
   to differentiate their articles from others, there are severe limits -
   and besides, most likely articles do not have an overlapping clientelle.

   But on the Internet, there is overlapping readers. An article written
   for the web is an article written for everyone. Anyone can read it.
   Thanks to the popularity of search engines, it can be available to
   anyone. At least in theory.

   This leads us to Internet promotion. Information on the web is sometimes
   so difficult to locate we have an almost continual need for more
   publishing. Real traffic is difficult to promote normally, so websites
   devoted primarily to delivering information have a real difficulty
   reaching their audience. This translates either to the need for
   expensive commercial promotion, which often can not be justified, or
   into reaching only those who search carefully for your information. The
   latter means multiplication of the same information.

   In writing this article, I see the effects mentioned will lead to
   changes in the future. As I write "attracting an audience initially
   enter into the publishing process", I think to myself this will
   obviously change. Attracting an audience will emerge in timeas the
   primary step in publishing. There are many places to take this
   discussion, but my job is a researcher, or rather an Internet-focused
   search theorist. (Long job titles will be in vogue). Let us focus on how
   these changes effect this Internet as an information resource.

   1) Any effort to organize the Internet is diluted because of these
   efforts.
   2) Any effort by the researcher to find different perspectives will be
   confounded by the number of people with the same perspective publishing
   in the same medium.
   3) Certain fields are more heavily hit than others. Internet advice on
   what search engines to use are ubiquitous. Java Programming hints are
   numerous. More specialized topics (like Internet-focused search theory)
   are less affected.
   4) Viral marketing - a catchword for sure, hopes to achieve promotion by
   seeding many sites with information. Perhaps an innovative way around
   accepting the multiplication of sites delivering the same or similar
   information.

   In phrasing the question you wish to answer, before the search,
   experienced researchers will focus on what information is likely to be
   available in numerous overlapping versions. These questions can be
   answered with the search tools which cover information in a more random
   manner: Search Engines do this very well. Tightly focused questions,
   less likely to be distributed so completely, should be approached with
   different tools: mailing lists and nexus points, long complex search
   queries and index points.

   In conclusion, the Internet will become far more cluttered than we had
   expected. I had previously predicted that search engines would grow to
   meet the needs, but this is not to be. Search engines will continue to
   serve up answers available from multiple places in the world. There is
   market enough in this, and minimal need to tackle anything more.
   ___________________________________________________

Squeezing the Info-Broker

   I was reading an interesting article by Anthea Statigos in ONLINE [1]
   that stirred me to thinking about the future of Information Brokerage.
   The article in question outlined the shift of information brokers into
   the marketing department, towards new roles in negotiating information
   access licenses, helping people understand and select appropriate
   resources - and oddly, in overseeing the intranet development process so
   as to deliver the information people need.

   The article premise is rather accurate - as far as it goes. But I wonder
   if the true message behind this shift is the decline and death of
   information brokering as a profession? If information brokers (also
   known as information professionals) are moving to new roles, are they
   vacating the old roles, the traditional roles in the research process?

   In my library, I reach for the Information Broker's Handbook [2] for a
   relevant quote:

   "The heart and soul of the information broker's job is information
   retrieval. But many individuals offer information organization services
   as well."

   So, Information Retrieval, and Information Organization.

   Anyone who has seen the simple information retrieval options
   incorporated in recent information packages can be in no mind that the
   information retailing industry is certainly minimizing the need to reach
   for an intermediary. Technology is certainly closing the gap - but this
   development has always been in the cards.

   A central difficulty for information brokers is a simple maxi: provide
   better results than clients doing the search themselves. Often working
   in unfamiliar territory, a researcher may find it very difficult to
   excel. There are two dilemmas here. Firstly, while we may pride
   ourselves in accomplishing unique requests, we have expensive costs
   associated with one-off searches. There is little likelihood someone
   else will ask a similar question. There are simply no possible economies
   of scale.

   Secondly, our search difficulty is not shared by the client. The client
   has difficulty with the technology - certainly. The client does not have
   difficulty with recognizing the wheat from the chaff, the gold embedded
   in the articles and at a basic level, the search words you will need to
   get to the right stuff.

   There is a very good reason why university students are pushed to learn
   basic and sophisticated search technologies.

   There is another take on this story.

   Creating Value in the Network Economy [3] includes a chapter by Philip
   Evans and Thomas Wurster.

   "emerging open standards and the explosion in the number of people and
   organizations connected by networks are freeing information from the
   channels that have been required to exchange it, making those channels
   unnecessary or uneconomical."

   "Newspapers and banking are not special cases. The value chains of
   scores of other industries will become ripe for unbundling. The logic is
   most compelling - and therefore likely to strike soonest - in
   information businesses ... All it will take to deconstruct a business is
   a competitor that focuses on the vulnerable sliver of information in its
   value chain."

   And in the back of my mind comes the thoughts that maybe the information
   retrieval function we have been providing is just one such information
   business. This business, attempting to be the pinnacle of the research
   process, is ripe for unbundling. Not only can our function be
   incorporated directly into the advertising and technology of the
   information resources we use, but our skill can also be coded into
   simpler and simpler guides and resources like my work on The Spire
   Project.

   Perhaps as an industry we never managed to secure our captive market.

   Initially, this will affect that mainstay of information brokerage:
   commercial database retrieval. And like the newspapers that will begin
   lose the profit center of classified advertising (ripe for unbundling
   and delivered electronically,) additional pressure will be applied to
   the business of providing information research services.

   Eventually, we retreat to other areas as information professionals:
   Information Organization, Research Education and Training.

   Somewhere in amidst this story lies a new role for researchers. The need
   for research certainly exists and is forecast to grow dramatically as
   the information age develops. What is lost, sadly, is an understanding
   of the ease at which this work will be done. This is certainly destined
   to move away from being an industry for professionals working at $50/hr
   to $150/hr + costs! Others can provide this work, easier than now.
   People we will most likely call researchers - and not information
   brokers.

   This is more than a push towards specialization. There is another way to
   see this transformation. The information broker was a retail point for
   wholesalers who are now firmly selling directly to the consumer. There
   is much less of a need for an intermediary between database retailers
   and information consumers - and there is a firm trend in this direction.

   Information brokers defined their role in the information industry as
   masters of the difficult technology of research, capable of finding most
   anything. Come to us when you are lost and we will find the answers -
   for a price. We know the technology, the meta-resources, the tricks used
   to find information. We routinely retrieve a higher quality of
   information, far faster, than you can yourself. The standard model: a
   library run service offering primarily database search & retrieval for
   their patrons.

   This business model is coming to an end.

   Yes, perhaps the information broker is dead. Soon to be replaced with
   low-wage researchers and research assistants, and high-end information
   executives and research trainers. Like it or not, most of us will
   incorporate a little more research into our current work, and reach for
   a little more intelligible research resources. Everything else will be
   accomplished by true specialists.

   [1] Online (a periodical with some coverage of library & information
   research. July/August 1999 p71-73, by Anthea Statigos of Outsell Inc.
   [2] The Information Brokers Handbook  p.21, by Sue Rugge and Alfred
   Glossbrenner. Windcrest/McGraw-Hill. 1992.
   [3]Creating Value in the Network Economy  Edited by Don Tapscott.
   Chapter 2: Strategy and the New Economics of Information by Philip Evans
   & Thomas Wurster. p.18 & 25. A Harvard Business Review Book.

   Getting the Best from the Internet.
   Section 8

   A search for information on the Internet is not essentially different
   from the standard information search process. You still need to start by
   outlining carefully just what you are hoping to locate. You also need to
   be aware of the peculiarities of the Internet as a researchable resource
   (or rather a collection of resources). If you expect instant delivery of
   exactly what you require, free, then you need a reality check (and I am
   sure you will get one real soon). Sadly, the printed media tends to
   overlook this.

   As with all resources, the more familiar you are with a given resource,
   the more efficiently you will work. Get to know the Internet for a time
   first. Understand how it works. Then re-adjust your expectations and
   file it as just another collection of resources, perhaps preferable in
   certain circumstances.

   A Structured Approach to Searching
   Much of this book has been devoted to describing what we could call a
   structural approach to finding information. We build a question, select
   a format and then search in an essentially static manner. There are only
   a few resources of interest for each format.

   On the Internet, we again do the same. If you want to search online
   periodicals (a specific format for information with specific qualities
   that might be appropriate) there are just a few sites to review. The
   search is simple and straightforward.  Search then read then reassess if
   it helped answer your question.

   The structured approach has been a simpler way to introduce a far more
   important application. Searchers know where answers are already -
   without ever having read the answer before - without having studied the
   topic. This is, after all, one of the few reasons to even consider
   paying for professional search assistance.

   How does a searcher know where answers lie?

   By building up a clear understanding of what information is out there,
   where it resides, and how to get to it, a searcher learns to anticipate
   the location of answers. Anticipation is everything.
   ___________________________________________________

   Know Where to Look

   Lets look at information itself. Information passes from producer, to
   organizer, to consumer. It travels many paths in this journey.
   Superficially, we can observe Internet communication travels via email,
   newsgroups, and webpages (and others). Let's call these tools.

   Looking deeper, we observe information emerges from just a few
   generalized sources: knowledgeable individuals, informed government
   employees, grant funded educational projects, commercial organizations
   and a few others. Each source produces a particular type of information,
   distributes (publishes & promotes) in particular channels, and hopes to
   pay for (or justify) their effort in a particular way.

   Efficient Internet research is infused with an understanding of who
   publishes, where and why.

   Before information reaches the consumer, it passes through a vetting
   which organizes and filters both the quality and the presentation style
   of the information. Let us call these systems. The FAQ is a pivotal
   piece of a system that may start with a post to a mailing list or
   newsgroup, involves the vetting of the faq maintainer, then proceeds to
   an faq archive then to the end consumer. The webpage is published by
   someone who has justified their time and expense, is indexed by a search
   engine or definitive-topic-website or webring or what have you, and then
   is found and read by the end consumer. The Internet has many such
   systems.

   Each system again defines many of the traits of the resulting
   information. Faqs are semi-authoritative, collaborative pieces, often
   dense and factual. Private mailing lists are sometimes more informative,
   discussive, as well as serving as a notice board. Newsgroups involve far
   less natural vetting and quality control, but excel in distributing
   popular volume resources like graphics. Search engines don't vett, but
   can be searched.

   Each system reinforces the uniqueness it brings to the whole Internet.
   When I blindly declare "Information Clumps" at the start of this faq, I
   am really describing a trend whereby certain information accumulates in
   a particular location, others out of self-interest add to the pile, and
   further information reinforces both the logic and uniqueness of that
   pile of information.

   It is just a short jump from this to understanding how faq archives grow
   but maintain a good quality, how the grand Internet search engines began
   to lose value about 15 months ago then recently began regaining a
   position of strength, and how ftp archives still exist for many computer
   topics.

   The internal logic to the organization of information is based on simple
   principles. It defines the environment within which we strive to improve
   the Internet as an effective information resource. We take this
   understanding and build sophisticated expectations about what kind of
   information rests at which format.

   Further Reading: Searching the Web: Strategy
   (http://spireproject.com/webpage.htm#5)
   ___________________________________________________

   Multiple Windows

   Make your browser work for you. All browsers allow you to open multiple
   windows panes. Open a few and send them off in different directions
   fetching information. You do not have to wait for each page to return to
   you before you read. With a little practice, you can juggle four window
   panes, collecting information from different tools, following different
   trains of thoughts, reading your way through four websites as they are
   downloaded.

   The technique is a little like reading four books at once. It certainly
   keeps your mind nimble. Worked successfully, multiple windows will
   double the speed of searching and free you from the speed of your
   Internet connection.

   Three technical tips are involved. Firstly, a second window pane is
   opened by selecting File : New : New Window.  Secondly, in Microsoft
   Explorer, depressing your shift key as you click a link will open the
   distant file in a new window. In Netscape, depress the control button as
   you click a link.  Thirdly, if you are running windows, the Alt + Tab
   button jumps between window panes.

   Taken together you can read down a page, find something interesting,
   shift+click a link, continue reading the original page, then flip over
   to reading the second page in a new window.

   Keep in mind, juggling windows is difficult and requires practice. If
   you do this in public, be prepared to lose novice surfers who are not
   ready to use more than one window.
   ___________________________________________________

   Launch Pages

   Bookmarks are a fine tool for beginners to build. It is not, however,
   the best organization of tools for a searcher. One of the roles of the
   Spire Project has been the construction of a far more effective tool,
   based on having the more common search tools and supporting information
   close together, on your own computer.

   Beyond being a plug for you to look at our free shareware
   SpireProject.zip (http://spireproject.com/spire_latest_version.zip) and
   single-page shortcut "The Spir" (http://spireproject.com/spir.htm),
   there is a serious issue here.

   If you are familiar with the use of search engines - and you have fast
   access to the search box for the search engines - you no longer need the
   URLs for specific resources. With a name, you can always quickly locate
   a page. Besides, URLs change. Far better to just keep a list of
   resources by name.

   At the start of this FAQ, we mentioned a searcher knows where to find
   information.
   "Knowing of specific resources is helpful. Knowing the tools to help you
   find resources, the meta-resources, is vital."
   Fast access to information resources is valuable. Fast access to the
   tools to find information is critical. Build your launch pages with
   these tools in mind.


   Searching is Art.
   Section 9

   Pharaoh: I am being attacked and backstabbed. I must kill these mutinous
   people.
   Shawn: Good Idea. So who is involved?
   Pharaoh: I don't know. I must find this out.
   Shawn: Find out what?
   Pharaoh: Who my enemies are, of course.
   Shawn: Enemies?
   Pharaoh: People who want me dead.
   Shawn: But not those who want a better ruler,
   Pharaoh: No not them.
   Shawn: What about the ones that want a better ruler, and would not mind
   you dead.
   Pharaoh: That sounds like everyone
   Shawn: And those that want you dead, but would not do anything about it.
   Pharaoh: Well, so long as they don't help anyone else.
   Shawn: Then you just want the ones who will try to kill you.
   Pharaoh: Yes,
   Shawn: Good. We know who we need to find. We need to determine those who
   will try to kill you.

                       <>    <>    <>    <>    <>

   Napoleon was an expert tactician, except at Waterloo. The recreation of
   past battles is not a favorite pastime of mine but it is an exciting
   topic all the same. The battle terrain was set. The troops have known
   abilities and limitations. The movement and direction of the army units
   is your responsibility. Do you have the strategy involved?

   Early in his career in an important fight against the Prussians,
   Napoleon employed a dramatic tactic where he initially held an important
   hill in the center of the battlefield, then surrendered the hill to the
   Prussians. The Prussians, confident at this stage, marched the majority
   of their army around the hill to right, between the hill and a lake, to
   push the fight on to Napoleon. Napoleon, however, retook the hill with a
   costly attack up the hill by some of his best units. Success left him in
   control of the high ground, much of the Prussian army below, moving
   between the hill and the lake. Unable to dislodge Napoleon from the hill
   a second time, and unable to withdraw the army from their exposed
   position, Napoleon pushed on to defeat the Prussians most decisively.

   The armies were almost evenly matched prior to this conflict and success
   seemed unlikely. An average general would have fought in a bland way,
   retreated perhaps, and fought to a stalemate. Napoleon inflicted a
   decisive defeat. Such generalship goes beyond technical skill to
   encompass a vision, a strategy, an art.

                       <>    <>    <>    <>    <>

   If I have not been careful, I will have presented searching as shopping
   in a supermarket. The goods are in a large store but there is a decent
   enough structure to find it. Third aisle for baby food. Go there and
   look around.

   Of course, we have discussed two further types of search improvements.

   There is the skills around properly asking questions. You want a
   question which accurately describes what you are looking for but you
   also want the question to be framed in a way which the resources can
   answer.

   There is also the awareness of where information SHOULD be. If you know
   what kinds of information exist and you ruminate long enough on the
   likely motivations of publishing, we can make some fairly detailed
   judgements on the whereabouts of the answers you are looking for.

   There is further skill in dealing with the technical difficulty of
   information overload. You have limited time and limited resources.
   Finding information is often a hit or miss affair, so there is an art to
   selecting the right words to search, the right Boolean prefixes to
   attach to search terms, the right search tactics to employ to get the
   most out of each situation.

   For much of this, you need only experience. If you know in advance a
   skilled searcher can handle the task of sifting reams of data for useful
   information, then you can focus on how its done, practice, and learn.
   The search technology itself is simple.

   The trouble lies in retrieving from databases with far too much
   information for simple word selection. It also flares when you are
   dealing with databases charging up from $2 a minute and an additional
   cost per item retrieved. You decide very quickly to get good at
   searching once you receive a bill for $200 of irrelevant information.

   The simplest solution to this difficulty is to practice. You will find
   all research libraries provide access to slightly older articles through
   CD-rom databases. Search these to hone your skills.

   I saw a small book on search techniques from an early course in my state
   library - but it is very basic. Most librarians build experience in
   using search systems either internally, or through a series of courses
   given by travelling database officers like the periodic training by
   Dialog-Insearch. These are expensive, but include some free time
   searching the expensive databases (no, they don't let you take
   information back with you).

   Now, there must be something else I can share with you on this topic.
   First, learn something about how the databases are built in the first
   place. It helps if you know what an inverted text database looks like.

   Second, something personal about technique... I always find the uglier
   the search query, the better the result. Honestly. A search combining
   numerous elements improves your chances of getting it right.

   Third, I always try to change my search techniques to match the medium.
   I am likely to be more careful of broad searches of expensive database,
   where as free databases often lead me to gather 50 articles, then
   weeding them out by hand. (most CD-ROMs allow you to select only the
   ones you want). Always bring a 3.5'' floppy with you when visiting a
   library on the of-chance you want to download and look at results
   another time.

   Fourth, I almost always find the initial challenge is in locating those
   specific terms that appear in 80% of the documents that interest you.
   When searching the Internet for information about government use of the
   web, the specific terms required were government and publishing (not
   even government publish was close) All other search terms gave far to
   much garbage. Yes, of course, being an expert in a particular field is
   an edge in already knowing these special terms.

   There are two escape hatches here. If you can find one or two articles
   that interest you, often you can browse these articles for those special
   words. Sometimes even, the descriptors of an interesting article will
   give you a specific subject heading. I've heard this technique called
   the "Pearl Development Technique" but I just think of it as a good idea.
   The second escape hatch is the use of free databases to prepare you for
   going online. If you have ready access to a CD-rom database, search this
   first - get the right search words on the free databases, then go
   online.

   Oh, of course, there is also the issue of just asking someone involved
   for the proper words. I like to ask my clients if they know what words
   are likely to be used. It's not a mark of an amateur to be asked, by the
   way.

   A couple of side issues

   1) Keep an eye on the type of document you are searching. If you want
   full text - don't go looking in bibliography databases. More to the
   point, don't start word searching databases with really big files
   without using the proximity indicators and descriptive fields. I hated
   paying for that 20-page document which included all the words I was
   interested in - but on different pages.

   2) Also, keep an eye on the quality of the documents you are retrieving.
   I know a search of newspapers sounds impressive, but they are rarely
   capable of explaining anything in depth and are notorious at being
   advertorials. I try to keep newsprint for locating experts - not for
   information. I have also been trapped by obscure magazines with
   appealing articles, only to learn the magazine is one of a large number
   of very basic business mags which likes to use fillers, or just doesn't
   like to pay for good journalism. A single article of 5 pages from
   Scientific American blows 20 small fillers out of the water. In fact the
   length of an article is a hint of depth.

   Oh, if you are looking for some really good books on this issue, try the
   manuals Dialog sends you to start, look for text databases in you
   library, then proceed to one of the search books recommended at the end
   of our 'research as a discipline' article.

   Basic Techniques to research change slowly, though the technology is
   improving and specific information resources are in rapid flux. It makes
   for interesting times.

   So many resources. So many techniques. Its strange to have written down
   so very much that is dull and tiring yet get it right. You simply must
   muddle through all those links to get a decent result.

   Yet the end result is to portray searching as an intensely dull
   experience. We have very few choices. The information exists in certain
   clearly marked places. We merely need collect it.

   If we are not careful we will present you the idea that searching is
   more like shopping in a supermarket. The goods are in a large store but
   there is a decent enough structure to find it. Third aisle for baby
   food. Go there and look around.

   Actually, this is the general approach to searching. There is no art, no
   talent, just skill and knowledge of the technology. Want a webpage on
   dogs - go to Yahoo and type in dogs. Want a telephone number - take out
   the white pages and remember the alphabet. Want a book and you are near
   the library, walk in and ask a librarian. Alternatively, walk in and
   type a few words in the library book database.

   But there is more - so very much more. And all of this makes for
   exceptional searching.

   Lets look at an example. We want information on how to improve the
   schooling of your exceptionally gifted child. A simple request. What do
   we do?


   The art is a kind of magic, of choosing just the right words at the
   right times, and in phrasing your request for information in a way that
   tightly describes your interest without removing information that should
   interest you. The art of searching relies heavily on an understanding of
   what is possible within a given system. Much of this, you guessed it,
   involves creative visualizing.



   Last Word.

   Shawn stood before the entrance to the tomb. It was not quite complete.
   The glyphs were complete for only the first thirty feet of the
   passageway, and workers were still preparing the burial chamber. The
   thick dusty air made it hard to breath, but at times it was better than
   staying outside where the temperature continued to climb.

   Shawn admired the art on the wall. Meaning within meaning. The divine
   representations stood offering the pharaoh recognition. In exchange the
   pharaoh offered his just reign. The scene worked well. Such work was one
   of the few ways the pharaoh could communicate with the gods.

   Yet there were other layers to the picture. The gods were depicted as
   pleased with the work of the pharaoh. Their recognition was a reward for
   the years of work the pharaoh ruled Egypt..

   There, further in the picture, was reference to the accomplishments of
   the pharaoh. Much of the writing was dictated by tradition, and the
   individual scribes were all instructed in the tale, so meaning was
   particularly important in what was different from other tombs. It was
   the small differences that made this work unique, that elevated the work
   from that suitable for any important person to that fit for a king.
   References to the pharaoh's conquests in Nubia. The special position of
   Horus, the falcon god that helped Egypt through invasion attempts from
   the desert oases of Libya.

   Then there was the technology. Sparkling stars on blue covered the
   ceiling. This was a new development, unseen before in crypt or building.
   It had a pleasant effect, expanding the space within the tomb, making it
   look larger than it really was.

   And then there was the artistry to the carving. These were fine scribes,
   skilled in carving. He would report the work satisfied him well.

                       <>    <>    <>    <>    <>

   Searching is an attitude. It is a way of looking at the world, and at
   information, which is different from distinct. Predictably, it has
   little tolerance for spin, puffery or questionable interpretation of
   statistics. It is a critical attitude and applies to all types of
   searching from industrial R&D to industrial espionage. Information means
   just a little of what it could mean. Without the luxury of knowing
   everything, we must recognize and consider what we can know for sure and
   can only suspect from available information.

   Searching can be a very negative attitude - and this is our last lesson.
   Search with a critical mind, but also with acceptance of how at some
   point in time you must be able to say enough. Enough searching, it is
   time to make a decision. This line is not defeat, not an acceptance of
   poor work. It is merely acceptance that all decisions are made on
   incomplete information. Make yours when you are ready.
   ___________________________________________________

   Acknowledgements

   I would like to thank my wife Fiona, whom I love and cherish dearly.

   The Spire Project is the culmination of several years bridging
   information research and Internet development. The information research
   industry is on the verge of a radical transformation set to add meaning
   to the oft-used saying "Information Revolution". The development of the
   Internet is currently delayed by many factors but to grow further we
   need to radically improve the middle ground of content-rich
   resource-linked webpages. I feel this is the most beautiful form
   information can take in this emerging information landscape. It is also
   a most effortful area to work in.

   Lastly, thanks to the many readers who assist in building and refining
   this information. Your help is appreciated.

   David Novak - [email protected]
   The Spire Project : SpireProject.com and SpireProject.co.uk
   ___________________________________________________
   Copyright (c) 1998-2000 by David Novak, all rights reserved. This FAQ
   may be posted to any USENET newsgroup, on-line service, website, or BBS
   as long as it is posted unaltered in its entirety including this
   copyright statement. This FAQ may not be included in commercial
   collections or compilations without express permission from the author.
   Please post permission requests to [email protected]

You are viewing proxied material from ftp.icm.edu.pl. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.