GEON Developments for Searching, Accessing, Integrating , and Visualizing Distributed Data

Fall AGU Meeting, December, 2005

GEON Developments for Searching, Accessing,

Integrating, and Visualizing Distributed

Data

Charles Meertens UNAVCO

Dogan Seber,Chaitan Baru

San Diego Supercomputing Center

Mike WrightDLESE Program Center

Image source credits:Mantle Tomography - Shapiro and Ritzwoller; Megnin and RomanowiczGeodynamics Model - McNamara and ZhongGlobal Strain Rate Map and Plate motions -Kreemer and Holt


GEON Overview

GEON Best Practices

GEON Portal:SearchData RetrievalResource RegistrationIntegration

UNAVCO/GEON Distributed Data Serving example

Visualization with the IDV

Outline


GEON “GEOsciences Network” ActivitiesNSF Large Scale Information Technology Research (ITR)

• Develop a distributed, services-based system that enables geoscientists to publish, share, integrate, analyze, and visualize their data, ontologies, tools, workflows, applications, and models

• Conduct integrated scientific studies on targets of opportunities in the test beds, in concert with geosciences community

Website: www.geongrid.org


• Members• Arizona State University• Bryn Mawr College• Penn State University• Rice University• San Diego State University• San Diego Supercomputer

Center / University of California, San Diego

• University of Arizona• University of Idaho• University of Missouri,

Columbia• University of Texas at El Paso• University of Utah• Virginia Tech• UNAVCO, Inc.• Digital Library for Earth System

Education (DLESE)

• Partners

• California Institute for Telecommunications and Information Technology (Cal-(IT)2)

• Chronos• CUAHSI• ESRI• Geological Survey of Canada• Georeference Online• IBM• Kansas Geological Survey• Lawrence Livermore National

Laboratory• U.S. Geological Survey (USGS)• HP• Other Affiliates• Southern California Earthquake Center

(SCEC), EarthScope, IRIS, NASA

NSF ITR Supported Partners

GEON PIs and Partner Institutions


GEON Cyberinfrastructure (CI) Principles

GEON is based on a service-oriented architecture (SOA) with support for “intelligent” search, semantic data integration, visualization of 4D scientific datasets, and access to high performance computing platforms for data analysis and model execution -- via the GEON Portal.

While focused on Earth Sciences, GEON cyberinfrastructure is generic and broadly applicable to a variety of other sciences and other application domains.


GEON Cyberinfrastructure Principles Cont.

• CI: Support the “day to day” conduct of science (e-science), in addition to “hero” computations

• The “two-tier” approach– Use best practices, including use of commercial tools and open

standards, where applicable…– …while developing advanced technology, and doing CS research

• An equal partnership – IT works in close conjunction with science to help create data sharing

frameworks, best practices, and useful and usable capabilities and tools• Create shared “science infrastructure”

– Integrated online databases, with advanced search and query engines– Online models, robust tools and applications

• Leverage from other intersecting projects– Much commonality in the technologies, regardless of science

disciplines, e.g. BIRN, SEEK, and many others


Physical model

Model results

HPCC

Data

Modeling Environment

Core Grid ServicesAuthentication, monitoring, scheduling, catalog, data transfer,

Replication, collection management, databases

Portal (login, myGEON)

Physical GridRedHat Linux, ROCKS, Internet, I2, OptIPuter (planned)

Registration Services

Data Integration Services

Indexing Services

Workflow Services

Visualization& Mapping Services

Registration GEONsearchGEONworkbench

Community ModelingEnvironment

GEON Overall Architecture


The GEONgridCommon Systems Development

PoP + Compute

PoP + Data Partner services

USGS

GeologicalSurvey ofCanada

KGS

Partner Projects

Chronos

CUAHSI

SCEC

PoP node

ESRI

UNAVCO


Search for ResourcesCurrent Types:ASCIICUAHSI DataGMT RasterGeoTiffRelational DatabaseShapefileToolWMS ServiceWeb Service

Search Constraints:Metadata Relation, Type, Subject, KeywordSpatial CoverageTemporal CoverageOntology/Concept Relation

Working in the GEON Portal:GEON Search

Example of unconstrained search response


Working in the GEON Portal:GEON Search with constraints

Sample search for:Resource Type - “Relational Database”Keyword - “GPS”

Search for ResourcesCurrent Types:ASCIICUAHSI DataGMT RasterGeoTiffRelational DatabaseShapefileToolWMS ServiceWeb Service

Search Constraints:Metadata Relation, Type, Subject, KeywordSpatial CoverageTemporal CoverageOntology/Concept Relation


Working in the GEON Portal:GEON Ontology-based Query Search - a

stepping stone to intelligent searchSearch can be constrained using selected ontologies and concepts. New ontologies are being developed by the science team through workshops and mechanisms exist at the GEON portal to register ongologies and resources to ontologies.

Example: User selects an ontology (in this case the NASA “Earth Realm” with IceCore concept


The Future: Scientific Data Integrationor: … from Queries to Questions can be resolved through

cyberinfrastructureTo be able to “GEONSearch”: What is the distribution and U/Pb zircon ages of high chrome spinel

kimberlites in the Appalachian Geologic Province? How does it relate to the thickness of the sub-cratonal lithospheric mantle?

Geologic map Lithosphere thickness Geochronology Bulk chemistry Mineral chemistry

Information Integration Geologic knowledge

Raw Data

Database Mediation

Ontologies

A.K.Sinha, Virginia Tech, 2005

Quantitative Numerical Integration


GEON Resource Access

• Access Resources• View Metadata• Download files• Query Databases

Example: GEON Portal Query generation tool and sample response from the GPS Seamless Archive Centers (GSAC) database located remotely on UNAVCO/GEON PoP Node


GEON Resource Registration

• Register Resources• Submit metadata about

resource (follows ADN Schema)

• Register to Ontology.

- Discovery of data resources (e.g., gravity, geologic maps, etc) requires registration through use of high level index terms (a type of ontology)


GEON Portal Resource Integration

• Gather Search-Selected Resources into your workbench

• Make integrated maps (shapefiles, WMS)

• Integrate Database• Integrate Workflows• “Integration Cart”

Example: Integration of two external WMS services into new GEON WMS Web Browser Map Viewer (OGC-supplied)


GEON 4D Data Access and Visualization

An ongoing GEON effort is to address 4D (xyzt) representation of earth science datasets and models in a grid computing environment. Current desires and approaches include:

• Capable volume-time Integrative Visualization tools: Enhancing the

Unidata IDV Java Application for earth science data• 4D (and multi-parameter) Data Model: Adopting netCDF (used by IDV

and soon by ESRI). Extensive Common Data Model effort at Unidata• Data Discovery: GeonSearch at the GEON Portal• Data delivery: html, OPeNDAP, OGC (WMS), Interoperable• Automated Data/metadata registration: currently exploring OAI/ADN,

DLESE Data Collection System and webservice, THREDDS (Unidata)

Basically give me the specific types of data I want, for only the specific time and volume I specify, and in a way that I can find it quickly and easily use it with any application I desire!


UNAVCO/GEON PoP Data Services and Visualization

Example Below: Geodynamic and Tomographic models on OPeNDAP Server. Visualization with IDV.


srb

ThreddsCatalog

http

ftpgridftpOGC*

*OGC: WMS/WCS/WFS

OPeNDAPOPeNDAPDataServers

-netCDF-Freeform

UNAVCO/GEONPoP Data Server1D/2D/3D/4D

-Tomography-GPS data/vectors-Earthquakes-Focal Mechanisms-Strain rate-Topography-Image maps-Geodynamics-Faults-Paleogeography

Plus IDV visualization“Bundles” (.xidv files)

Thredds-Catalog elements

Digital LibraryMetadata elements(via OAI or DLESEWebservice)

WMS Catalog *OGC=WMS/WFS/WCS

UNAVCO/GEON PoP Server Details:Data, Models, Catalogs, Metadata

Data Access

Dataset Catalogs and Metadata Access


UNAVCO/GEON Data Node

OPeNDAP ServerExample: Seismic Tomography in NetCDF format

WMS ServerExample: GPS Seamless Archive Postgres/PostGIS Database

THREDDS/OPeNDAPCatalog

WMS Catalog

HTML File

Other distributed data servers

Integrative Data Viewer

UNAVCO/ GEON Data Server and Visualization Access

Local File

Still under development: registration of netCDF files and OPeNDAP servers into the GEON

Portal; embedded GEONSearch within the IDV


Summary

• Using a best practices approach, GEON has been developing data sharing frameworks, a registry for distributed databases, concept-based search mechanisms, advanced visualization software, and grid-computing resources for earth science and education applications.

• GEON Cyberinfrastructure will enable new interdisciplinary research in the geosciences, while extending the access to data and complex modeling tools from the hands of a few researchers to a much broader set of scientific and educational users.

• We invite you to try out the tools on the GEON Portal and to Submit or Use Resources of any type. Come and see GEON demos at the GEON Booth!

Website: www.geongrid.org

GEON Developments for Searching, Accessing, Integrating , and Visualizing Distributed Data

Documents

Transcript of GEON Developments for Searching, Accessing, Integrating , and Visualizing Distributed Data