---------- Forwarded message ----------
Date: Wed, 18 Aug 1999 16:06:49 -0500
From: Charles Hickman <[log in to unmask]>
To: [log in to unmask]
Cc: [log in to unmask]
Subject: indexing digital geographid data (was: NTIS and the rebellion)

>>> Posted to MAPS-L for
>>> Doug Nebert <[log in to unmask]> 08/18 2:40 PM >>>

> Date: Mon, 16 Aug 1999 13:57:43 -0600
> From: Bill Thoen <[log in to unmask]>
> To: Maps and Air Photo Systems Forum <[log in to unmask]>
> Subject: Re: NTIS and the Rebellion...
>
> Actually, I suspect that continuing development of search engines
> will do more to revolutionize how libraries index information
> than the other way around.
>

I agree completely with Bill that unstructured, document-like
information has a great promise for discovery through emerging
search engines. The challenges for indexing digital geographic data
however have not yet been met by existing search engines without
some help. We are helping to overturn some of these problems
with commercial vendors, but the following obstacles remain:

Formalization of spatial and temporal tags. Sure, place names
can be found in documents and metadata, but unless they are done
with some intelligence or known classification system, the search
and matching of results is very problematic yielding many possible
matches that one must wade through. Fortunately FGDC metadata
includes dates and bounding coordinates that can be easily
parsed from the XML representation of the metadata into unambiguous
locations. Good metadata collection and search tools also hide
the management of these numeric coordinates behind place name pick
lists or map-based selectors that derive coordinates on the back
end.

Management of large collections. Many of the richest and deepest
collections of geospatial data products store their metadata in
relational databases. With millions of entries it is not very
smart to export and synchronize the entire collection with an
XML or HTML form to be visited by search engines. In these cases
the product series metadata might get exposed for search but the
details that get the user to accessing a specific product are
still hidden from view. I see great potential in combining search
engines within every website that are threaded into any local
data bases to provide rapid local discovery. I see indications
that Microsoft and Oracle are heading this way, making the
possibility of a targeted distributed search a real possibility.
Site "centroids" or descriptive records end up getting exposed
to crawlers and search engines. Searches then are passed only
to site indexes that match a certain profile where current public
content is searched. We are still a year or two away from this
vision that can only work reliably with some search standards
in place.  Perhaps the OpenGIS Consortium's Catalog Services
specification, passed last week can help out here.

Links between data and metadata.  I agree that the traditional
external catalog model is outdated. In the digital world the
data and the metadata should become one. Metadata are simply
the properties returned from querying a data object. In the
not too distant future when one places a spatial data object
into the public database, it properties become the things
against which searches are posed. Software uses them; nice
user presentations of the metadata can be made from them. But
again we are still far from having commercial search
engines recognize existing spatial data structures let alone
spatial data with embedded metadata.

In the next year we will be revisiting the potential of the
centralized and distributed elements of Clearinghouse to better
achieve the vision of spatial data discovery and access. And
we will be doing so with the leverage of emerging standards
and commercial search technology. I hope to make the use of
the Clearinghouse potentially a transparent one that your GIS
software could just use programmatically to help get your job
done. Other browseable interfaces will lead GIS-naive users
from finding a map, past needing to interpret complex metadata,
and will present them the picture that they are seeking in
response to a problem. New solutions are looming on the horizon
with open, interoperable web mapping interfaces from the OpenGIS
Consortium with support from all the major (and some minor)
GIS vendors. Big changes are afoot!

Doug Nebert
FGDC