Post on 27-Mar-2015
1 CEOS WGISS-21
April 10, 2023
NOAA’s Global Earth Observation
Integrated Data Environment (GEO-IDE)
CEOS WGISS-21Budapest, Hungary 2006
David Clark NOAA for
Glenn Rutledge, GEO-IDE Co-Chair
2 CEOS WGISS-21
April 10, 2023
NOAA’s GEO-IDE
Scope – NOAA-wide architecture development to integrate legacy systems and guide development of future NOAA environmental data management systems in support of US-GEO and GEO
Vision – NOAA’s GEO-IDE is envisioned as a “system of systems” – a framework that provides effective and efficient integration of NOAA’s many quasi-independent systems
Foundation – built upon agreed standards, principles and guidelines
Approach – evolution of existing systems into a service-oriented architecture
Result – a single system of systems (user perspective) to access the data sets needed to address significant societal questions
3 CEOS WGISS-21
April 10, 2023
*"Things should be made as simple as possible, but no simpler.“
Albert Einstein
Service-Oriented Architecture*
4 CEOS WGISS-21
April 10, 2023
Why is GEO-IDE Needed?Important societal issues require data from many observing systems
Atmospheric Observations
Land Surface Observation
Ocean Observations
Space Observations
Data Systems Coordinated, efficient,integrated, interoperable
Discipline Specific View Whole System View
Current systems are program specific, focused, individually efficient.But incompatible, not integrated, isolated from one another and from wider environmental community
4
5 CEOS WGISS-21
April 10, 2023
Today’s Challenges Incompatible syntax (formats) and semantics (terminology) among
science disciplines within NOAA. Thousands exist. Several examples:
• Naming standards – Surface Air Temperature• Meteorology (WMO) named “Temperature/dry bulb temperature• Meteorology (air pollution) named “Boundary layer temperature”• Oceanography named “Air Temperature”
• Location standards (latitude, longitude, elevation)• Lat/Lon can be degrees/minutes/seconds or degrees to tenths and hundredths • Latitude E/W, 0-180 positive and negative, or 0-360 running east or west• Z used to designate elevation in both atmosphere and ocean but positive is up in the
atmosphere and down in the ocean• Formats (>50 formats used within NOAA; translators and standards needed)
• GRIB, NetCDF, HDF and others used for gridded data• BUFR, NetCDF, and many others used for observations
Potential for no answer or the “wrong” answer to important societal questions due to separate NOAA data management systems
6 CEOS WGISS-21
April 10, 2023
Goals
Through GEO-IDE NOAA will Identify and address integration gaps in data management systems Create interoperability across existing data management systems Develop and adopt data standards for formats and terminology Integrate measurements, data, and products Examine the need for future data management requirements
and will achieve Cost avoidances in NOAA business through improved efficiency
and reduced duplication Reduced risks for US IEOS and GEOSS
7 CEOS WGISS-21
April 10, 2023
GEO-IDE Bridging the gaps between stove-pipe systems
Weather Climate Oceanography Biology Geophysics
Standard procedures, protocols, metadata,
formats, terminology.Web Services
• Integration of data across disciplines• Improved data stewardship
• Increased efficiency• Leverage industry and community initiatives
• Integration of data across disciplines• Improved data stewardship
• Increased efficiency• Leverage industry and community initiatives
HydrologyGeospatialFramework
8 CEOS WGISS-21
April 10, 2023
GEO-IDE - an essential component ofenvironmental information management for NOAA
Integrated observing, data processing and information management systems
Connected by NOAA’s Integrated Data Environment
Contributes to U.S. Global Earth Observation System (USGEO) andInternational Global Earth Observing System of Systems (GEOSS).
9 CEOS WGISS-21
April 10, 20239
Why Now?
Critical to USGEO– six near term opportunities
Uncoordinated development leads to inefficiencies, incompatibilities, and duplication of effort.
Integration of data among systems is needed to answer questions that address diverse societal benefits
Increased efficiency is needed to handle the expected exponential increase in data volumes that will occur over the next decade
Societal Benefits
• Improve weather forecasting
• Reduce Loss of Life and Property from Disasters
• Protect and Monitor our Ocean Resources
• Understand, Assess, Predict, Mitigate and Adapt to Climate Variability and Change
• Support Sustainable Agriculture and Combat Land Degradation
• Understand the Effect of Environmental Factors on Human Health & Well Being
• Develop the Capacity to Make Ecological Forecasts
• Protect and Monitor Water Resources
• Monitor and Manage Energy Resources
10 CEOS WGISS-21
April 10, 2023
Vision
“System of systems” – a framework to effectively and efficiently integrate NOAA’s many systems
Minimize impact on legacy systems
Utilize standards: – Adopt, adapt and only
reluctantly create– Emphasis on flexibility
Work towards a service-oriented architecture
11 CEOS WGISS-21
April 10, 2023
Approach
Standards– Adopt, adapt and only as a last resort, create– Open, inclusive process for adoption– Inclusive not exclusive use of standards
Service Oriented ArchitectureReference: Federal CIO Council, Jan ’06 "Services and Components Based Architectures: A Strategic Guide for Implementing Distributed and Reusable Components and Services in the Federal Government"
12 CEOS WGISS-21
April 10, 2023
Standards
Standard names and terminology Metadata standards
– FGDC and ISO 19115 w/ remote sensing extensions, Format standards
– XML Schemas, Spatial Databases (SQL), data formats (WMO, NetCDF, HDF, etc.)
Open Geospatial Consortium (OGC) standards:– Features, Coverage (data), Geographic Markup
LanguageWeb Services Standards (World Wide Web
Consortium)
13 CEOS WGISS-21
April 10, 2023
Service-Oriented Architecture
Under an SOA, capabilities are built one at a time to create “Web Services”
The fabric of the SOA is built upon standards for:– discovery (e.g. CF, FGDC, OGC, ISO)– transport (e.g. HTTP, FTP, OPeNDAP, Grid)– use (e.g. netCDF, HTML, OGC, etc.)
Can be tightly coupled (SOAP) or loosely coupled (REST)
14 CEOS WGISS-21
April 10, 2023
SOA Concept
15 CEOS WGISS-21
April 10, 2023
Key Development Strategies
Maintain and minimize impact on legacy systems
Evolutionary development through pilot projects
Coordinate activities through “Structural Data Types”– Grids, time-series, moving-sensor multi-dimensional,
profiles, trajectories, geospatial framework, point data and metadata
16 CEOS WGISS-21
April 10, 2023
Project Management
Undersecretary for Atmosphere and Oceans
DMITData Management Integration Team
NOSCNOAA Observing System
Council (NOAA CIO Member)
DMCNOAA Data Management
Committee (CIO Council Co-Chair)
NOAA Goal Theme:
Weather & Water
NOAA Goal Theme:
Ecosystems
NOAA Goal Theme:
Commerce & Transportation
NOAA Goal Theme:
Climate
17 CEOS WGISS-21
April 10, 2023
Future Direction - Priorities FY07 Work with scientists/data system managers to
assess requirements and systems– Develop enterprise architecture and GEO IDE
Implementation Plan– Implement standards process– Active out-reach activities- inclusive (CEOS WGISS)
FY08/09 Incrementally execute work packages– Develop data standards and interoperability
mechanisms, e.g., translators and directory services– Direct, test and evaluate changes being made to
data management systems
FY10/11 Re-evaluate architecture related to new data systems (across NOAA & with national /international partners)
18 CEOS WGISS-21
April 10, 2023
Conclusions
NOAA faces daunting challenges of vastly increasing data volumes and an increasing need for interdisciplinary use of data
NOAA is committed to enhancing access and ensuring data and products of enduring value are preserved for future generations
NOAA has initiated several activities to actively respond to these challenges. GEO-IDE is a key component