The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 1 New Internet Developments:...

The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 1

New Internet Developments: New Internet Developments: Marine XMLMarine XML

Peter Pissierssens & Greg Reed (IOC/IODE)


SummarySummary

• Background

• Why MarineXML?

• IODE and MarineXML

• Benefits for marine community

• Current initiatives


BackgroundBackground• Emergence of XML as a data transfer

protocol provides the global oceanographic community with a mechanism to support the exchange of marine data.

• A marine XML structure will encapsulate marine data and provide an efficient means to store, transfer and display marine data.

• XML: describes content, not style


Why marineXML?Why marineXML?

• Exchange of data

• Extensibility

• Plain Text

• Data Identification

• Stylability

• Hierarchical


IODE and marineXMLIODE and marineXML• 16th Session of the IOC Committee on

International Oceanographic Data and Information Exchange (IODE) in 2000 proposed to develop an XML interoperability framework based on a ‘Marine Mark-Up Language’ (MML) specification


Benefits for Ocean CommunityBenefits for Ocean Community• MML interoperability framework will

have considerable benefits to the oceanographic community, supporting the objectives of data interoperability

– Data providers can operate without changing their existing data structures and systems

– Developers can build common programs and instruments to support multiple formats

– Users can access heterogeneous datasets across the internet or local networks


Current InitiativesCurrent Initiatives• Current international initiatives

include:– The ICES/IOC SG-XML– XML ‘Brick’ concept– The EU Marine XML Project– The MEDI metadata authoring tool– GE-TADE activities– Establishment of a MarineXML community

portal site – OIT Pilot Project


SG-XMLSG-XML


SG-XMLSG-XML• Study Group on the Development of

Marine Data Exchange Systems Using XML (SG-XML)

• Jointly established by ICES-IOC


SG-XMLSG-XML TOR TORThe Terms of Reference for the Group:a) develop a framework and methodology for the use of XML in marine

data exchange in close consultation with IOC and the Marine XML Consortium;

b) develop a workplan that within 4 years will lead to published protocols for XML use in the marine community;

c) explore how to best define XML tags and structures so that many ocean data types can be represented using a common set of tags and structures;

d) test and refine these common tags and structures using designated case studies i.e.:

a) Point (physical/chemical) data (profile, underway, water sample); b) Metadata (cruise information, building from the ROSCOP/Cruise

Summary Report); c) Marine Biology data (integrated tows, e.g., zooplankton-

phytoplankton tows, demonstrate the use of taxonomy).


SG-XMLSG-XML actions actions• First meeting April 2002• Action plan developed to include:

– Parameter Dictionary. Create, evaluate and discuss intersessional work on SGXML parameter dictionary including the population of the dictionary for distribution via a defined XML structure.

– Point Data Investigation. Evaluate the generalised brick approach with application to various point data types. Evaluate the point data structure from the perspective of the IODE data centres. Provide biological and taxonomic input to the bricks.

– Metadata Investigation. Evaluate linkages to other metadata standards and the implications of a generalised metadata model to existing models.


XML BricksXML Bricks


XML BricksXML Bricks• Concept developed by R. Keeley of

MEDS• The brick structure represents a basic

building block for packaging data and metadata

• Employs a small number of generally defined structures that can be assembled in different ways to reflect the structures of a variety of data collected over a broad spectrum of disciplines.


XML BricksXML Bricks• Bricks can describe

– data units– measured (or computed) variables– space and time location of the

measurement– data quality flags– supporting information such as sampling

techniques, species taxonomy, etc.


XML Bricks – current listXML Bricks – current listBrick Definition

analysis_method Information about the analysis employed

archive_information Information attached by the recipient of the data

availability A marker to control access to the data

calibration Information about instrument calibration

comment General comments on the data

data_dictionary Used to identify the dictionary in use in the data file

data_point Contains a single data value and supporting information

history Processing history of the data

instrument Information about the instrument used to make the measurements.

location The x, y, z, or t values for a given set of observations.

provenance The originator of the data

quality A marker providing the assessment of data quality

quality_testing Information about how the data quality assessment was made


XML BricksXML Bricks: Example: Example• Example: the Provenance Brick• Holds information about the originator of the

data. • Where data is collected from a single source,

would take the form: provenance

> originator > project > country > date_received > platform_name > agency > originator_identifier > data_grouping > description

> platform_type


EU MarineXML ProjectEU MarineXML Project


Project OverviewProject Overview• To demonstrate that XML technology

can be used to develop a framework that improves the interoperability of data for the marine community and specifically in support of marine observing systems

• To develop a prototype of an XML-based Marine Mark-up Language (MML)


Partnerships!Partnerships!• MarineXML is to be developed in

partnership with international agencies, government departments and organisations responsible for data standards to ensure that the research meets the needs of key stakeholders with interests in global ocean observing systems.


Consortium ParticipantsConsortium Participants1. HRW HR Wallingford UK (Coordinator)2. UKMIC UK Marine Information Council UK 3. 7CS SevenCs DE4. NERSC Nansen Environmental and Remote Sensing

Centre NO 5. CLRC Central Laboratory of the Research Council UK 6. RIKZ Rijkswaterstaat NL 7. VLIZ Flemish Marine Institute BE8. SCO Social-change On-line AU 9. SMHI Swedish Metrological and Hydrological Institute

(EuroGOOS) SE 10. IOC/IODE International Oceanographic Data and

Information Exchange Committee INT


Other InitiativesOther Initiatives


MEDIMEDI

• Marine Environmental Data Inventory• A directory system for marine related

datasets and data inventories within the framework of the IOC’s International Oceanographic Data and Information Exchange (IODE) programme.


MEDI Authoring ToolMEDI Authoring Tool• The MEDI authoring tool has been

developed to encourage data collectors and scientists to produce metadata descriptions for their datasets.

• MEDI authoring tool is browser-enabled and operates in a client-server configuration. Clients can access MEDI on a local network or over the internet


MEDI formatMEDI format• MEDI uses the Directory Interchange

Format (DIF) developed by NASA's Global Change Master Directory

• A DIF is a collection of fields which detail specific information about the data which are necessary for users to decide whether a particular dataset would be useful for their needs

• Metadata are stored as DIF-XML files• XML is used to transfer DIF records


MEDI flowMEDI flow• Operates as a service

under Apache Tomcat 4.0.4 using HTML

• Uses standard HTTP protocol, hence can be accessed via internet or intranet

• Metadata records are stored as DIF-XML files

• Spatial functionality is delivered using Scalable Vector Graphics (SVG)

• Data can be tranferred using XML


GE-TADEGE-TADE• IODE Group of Experts on Technical

Aspects of Data Exchange• Mission of its work plan:

“The development of an End-to-End Marine Data Management Framework”

• Present situation:– large number of data formats in existence– many different data types– many different systems – result is a high level of duplication of effort.


GE-TADE and MarineXMLGE-TADE and MarineXML• Objective is the development of marine

XML as a mechanism to facilitate format and platform independent information, metadata and data exchange.

• A limited pilot project to demonstrate an activity within the E2EDM framework is currently in progress at the Russian NODC.


Marine XML PortalMarine XML Portal• Established to

provide a discussion forum for MarineXML activities.

• Hosted by IOC

http://www.marinexml.net


Marine XML PortalMarine XML Portal• News• About marineXML• EU MarineXML

Project• ICES-IOC SGXML• Events• Documents• XML Links• Forum• FAQ

• Uses IODE BeeBox:– You can register– Content informer– Multiple writers


OIT-PPOIT-PP

• Ocean Information Technology Pilot Project

• JCOMM / GOOS / IODE


OIT: Why?OIT: Why?• We have only 5-10% of the needed telemetry

capacity• Data Assembly - no broad agreement on how we

ensure coherent, integrated data sets• Lack of agreement on standards, formats and

practices• Poor modes of data transport, both routinely for

operational requirements and for science• Poor or inappropriate uptake of modern information

technology• No proper assessment of the future need.


OIT VisionOIT Vision

• ““Create an efficient and effective data Create an efficient and effective data and information management system and information management system for the ocean and marine environment, for the ocean and marine environment, based on leading-edge [ocean] based on leading-edge [ocean] information technology, and serving the information technology, and serving the oceanographic community and beyond.”oceanographic community and beyond.”

• Project Leader: Neville Smith


ReferencesReferences

MarineXMLMEDI

OITIODE

http://www.marineXML.nethttp://ioc.unesco.org/MEDIhttp://ioc.unesco.org/oithttp://www.iode.org

The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 1 New Internet Developments:...

Documents

Transcript of The Colour of Ocean Data, Brussels, Belgium, 25-27 November 2002 Slide 1 New Internet Developments:...