UNEP: TRAINING WORKSHOP ON ECOSYSTEM APPROACHES TO COASTAL AND OCEAN MANAGEMENT
Ocean Information Technology New approaches to ocean data management
description
Transcript of Ocean Information Technology New approaches to ocean data management
COD, Brussells, 25-27 Nov
Ocean Information TechnologyOcean Information TechnologyNew approaches to ocean data New approaches to ocean data
managementmanagement
Neville Smith
BMRC, Australia
COD, Brussells, 25-27 Nov
The The Motivation … The Motivation … The IssuesIssues
We have only 5-10% of the needed telemetry capacity.
Data Assembly - no broad agreement on how we ensure coherent, integrated data sets.
Lack of agreement on standards, formats and practices.
Poor modes of data transport, both routinely for operational requirements and for science.
Poor or inappropriate uptake of modern information technology.
No proper assessment of the future need. …
COD, Brussells, 25-27 Nov
Some FoundationsSome Foundations
Some good examples of D&IM at work IODE, JCOMM OBIS (CoML), Argo DM, ODINAfrica, US IOOS Data plans …
but poorly integrated
• “not invented here” syndrome many realizations of the same function/form
Too much in-house technology, too little out-sourced functionality• Too little fusion / cross-sector adoption
No / little engagement with scientific community• Innovation not within
– …
COD, Brussells, 25-27 Nov
Rationale (The Why)Rationale (The Why)
• Approach IT as you would a Science Program
Effective telecommunications Common standards, practices and protocols
(metadata management) Data and product service matched to the
participants and users Innovative data inquiry, access and delivery Intra-operability and interoperability …
COD, Brussells, 25-27 Nov
The Components … the The Components … the “How”“How”
1. Telemetry
2. Metadata Management
3. Data assembly, data set integrity, quality control
4. Data circulation and transport
5. Archives and archaeology
6. Applications and user interfaces
7. Capacity enhancement, training
8. Governance, oversight, metrics
COD, Brussells, 25-27 Nov
1. Telemetry and 1. Telemetry and CommunicationsCommunications
… getting the data “in” (instructions out?)
a) Availability and capabilityBandwidth, 2-way comms, connectivity, …
b) Requirements for climate, open ocean
c) Requirements for coastal regions
d) Polar / remote regionsBandwidth
… enabling
COD, Brussells, 25-27 Nov
2. Metadata Management2. Metadata Management
IODE, JCOMM (CBS) and others have made considerable progress
Data
Syntactic Metadata
Semantic Metadata• Data about data …
• Its environment
• Its structure
• Its meaning
• Keywords
COD, Brussells, 25-27 Nov
2. Metadata …2. Metadata … Developing an agreed vocabulary
– Perhaps agreeing on a Markup Language: MarineXML or … (e.g., IODE/ICES Project)
– An essential element for intra-operability, interoperability and exchange
– Non-physical / unconventional: a challenge– Complying with mandated requirements
Complete, consistent, searchable metadata– Keywords, geospatial references– Adopt standardized, shared vocabularies– Developing catalogues– How to do highly effective searches in distributed
systems? How to automate, encourage, …
COD, Brussells, 25-27 Nov
3. Data assembly, quality 3. Data assembly, quality control, …control, …
Performance level monitoring
More formality, accepted procedures, …– From Level 0 (raw) through Level 3 (analysed)– TOGA, WOCE, JGOFS experience
COD, Brussells, 25-27 Nov
3. Datum and Data Set 3. Datum and Data Set IntegrityIntegrity
Develop methodology for uniquely identifying original data and variants
• Argo is being used as one test-bed• Original data given and always retains “tag”• Variations referenced against original
Accreditation– Eg, at level 2: a) no QC; b) auto; c) scientific– Institute A, A+, A++ system, esp. for c)
• Recognize value adding • Recognize scientific involvement• Agreed peer-review system• …
?
COD, Brussells, 25-27 Nov
4. Data Circulation and 4. Data Circulation and ServiceService
TimelinessTimeliness, efficiency Architecture for data/product serving and exchange
• Data packets and sets arrive with standard metadata and “tag”• Systems like IDD are used to “push” data to routine users• The sources may be real-time or high-quality delayed-mode• Each ocean data distribution centre has instruction sets for
distribution; event driven Also request-reply (pull) servers
– US Data Communicationinitiative …
COD, Brussells, 25-27 Nov
US IOOS ModelUS IOOS Model
Circuits, data conveyers
COD, Brussells, 25-27 Nov
Together Together New New FunctionalityFunctionality
The XML standard (or an equivalent) should ensure all data are properly described / characterized
Some of the needed metacode can be generated automatically
The “tags” allow identification and removal of duplicates
There is immediate identification of non-conforming data insertion
No need to be a major centre to participate (provider or user)
Some traditional data archive (backup) could be automated
COD, Brussells, 25-27 Nov
5. Archives, Centres5. Archives, Centres
Reference other sessions Centres might specialise in a type of service
– Routine versus ad hoc / itinerant– Sophisticated versus non-specialist– Strong link to User Interface theme– Catalogues and Portals: Modern Library
Irreplaceable observations, data products of lasting value and associated metadata preserved in perpetuity– Accessible using standards, keywords– Integrated into real-time and delayed mode data transport
mechanism
Data archaeology
COD, Brussells, 25-27 Nov
6. The User Interface6. The User Interface
Must make data and information:– Available, usable, accessible, …
Scope– Assess “external” user requirements
– “Wizards” to simplify interface
– Assess “internal” user requirements– Robust interfaces; guaranteed responses
Several existing innovative client interfaces– Live Access Server and derivatives– GIS Systems; Web Map Server
COD, Brussells, 25-27 Nov
Menu for UKFOAM &POLCOMSdata
http://www.nerc-essc.ac.uk/las Daily 1/9o FOAM N Atlantic data
SST for 7 Jun 2002
E.g., E.g., GODAE GODAE LAS LAS Product Product Server: UKServer: UK
COD, Brussells, 25-27 Nov
7. Capacity, “education” … 7. Capacity, “education” …
Some of the greatest challenges are in the area of community outreach and organizational behavior rather than in technology! [US IOOS DCS SC]
Challenges– Breaking down traditional barriers– Capacity enhancement: even, “fair”– Adoption and acceptance– Uneven access to technology, people
Reference other presentations
COD, Brussells, 25-27 Nov
ConclusionConclusion
There is broad consensus on the need to act We have an outline of the strategy: why, how Enormous potential to impact science and
applications; not discipline bound Architects and engineers are already busy Critical that we have broad participation Starting now– http://ioc.unesco.org/oit
COD, Brussells, 25-27 Nov
Thank youThank you
MOI?