Information System Report-progresses in CSR, EDIOS, EDMED, EDMERP, EDMO, CDI
E. Balopoulos (HCMR), F. Nast (BSH), L. Rickards (BODC), D. Schaap (Maris)
presented by S. Iona (HCMR)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Outlines
Objectives
Structure
Maintenance
Population tools and updating mechanisms
Current status and system upgrade
Summary
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Objectives
The SeaDataNet Information System is an integrated catalogue service, aiming at facilitating marine data searching, location and retrieval.
The Objectives are:
Maintain and expand the national metadata-bases. Standardize the information using common vocabularies
and reference tables. Interconnect the national inventories in a common Pan-
European directory.
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Structure
It is composed of several thematic inventories of different levels:
European Directory of Marine Environmental Datasets (EDMED)
Cruise Summary Reports (CSR) European Directory of the initial Ocean-observing
Systems (EDIOS) European Directory of Marine Environmental Research
Projects (EDMERP) European Directory of Marine Organizations (EDMO) Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Structure
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Structure
High Level Directories EDMED, CSR, EDIOSfor describing collected observation datasets (by ships,
by laboratories, by continuous observing system)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Structure
Common Reference Tables EDMERP, EDMOhold research projects and organizations metadata
common to higher directories
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Structure
The Common Data Index (CDI)provides access to the data, information and products by data type and/or any other field
distributed by the TA platforms
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Maintenance
During the first development phase of the Project (V0) the inventories are organized in centralized databases at the coordinating centres (BSH, BODC, MARIS) based on the scheme of the past SEASEARCH Project:
The TA data centres populate the metadata-bases by compiling, formatting, standardizing their national data and transmitting them to the coordinating centres.
The metadata are accessed through the common metadata-bases that are published as part of the SeaDataNet discovery service: http://www.seadatanet.org/services/catalogues
At V0 the catalogue page links to the SEASEARCH sites.
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Population tools and updating mechanisms
Through on-line (like CMS forms) and off-line tools that produce XML ISO 19115 compliant exchanges and developed by the Technical Task Team in the joint research activities JRA1,JRA2.
Sea-Search siteLocal interface orJava tool MikadoJava tool MikadoCDI
EDMERP siteAccess DatabaseEDMERP [email protected]
EDMO siteOn lineEDMO
EDIOS siteunder developmentEDIOS
[email protected] ManualLocal interface or MikadoJava tool Mikado
Access DatabaseEDMED
[email protected] site
Local interface orJava tool Mikado
Java tool Mikado(XML converter)On line CSR
Contact and AssistanceDocumentationautomatic XML entriesAlternate
manual entryManual entriesInventory
Need further help? Please contact HCMR/HNODC
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Population tools and updating mechanisms
NEMO a new reformatting tool to convert ascii files to Medatlas and ODV format (two of the SeaDataNet data exchange formats).
NEMO interacts with Mikado to produce ISO 19115 XML metadata descriptions.
Tools for education and capacity building developed in the frame of the SeaDataNet European project by M.Fichaut et al., IMDIS-2008 Conference.
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Current status and system upgrade
European Directory of Marine Environmental Datasets (EDMED)
Cruise Summary Reports (CSR)
European Directory of the initial Ocean-observing Systems (EDIOS)
European Directory of Marine Environmental Research Projects (EDMERP)
European Directory of Marine Organizations (EDMO)
Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
European Directory of Marine Environmental Data(EDMED)
Currently, EDMED describes:
Over 3500 datasets 700 data holding centres
In 2007/2008, BODC received updates from 13 partners, and added 190 new datasets to the database.
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Updated EDMEDs since 07/03/2007
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Total No. of Datasets and Data Holding Centres
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMED improvements
BODC are currently developing the software tools that will be used to ingest an EDMED XML record from a partner, containing the metadata from one dataset, into the Master EDMED Oracle schema, where it will be available for retrieval and viewing. Work is also being carried out for the future ingestion of the existing EDMED records.
Software Tools
to carry out the quality control checks on the EDMED record
to ingest the record into the Master EDMED Oracle schema
to allow a partner to retrieve an EDMED record through a web interface
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
XML validator tool/Quality Assurance
BODC, along with RNODC, have been working to:
Provide an XML validator tool accessible to all SeaDataNet partners via a web browser.
Enable SeaDataNet partners to validate EDMED, CDI, CSR, EDIOS and EDMERP XML documents before they are delivered to the directory object manager for processing.
Enable the object managers to re-validate the XML files before loading the records into the directory database.
Provide a controlled approach to the publishing of XML schemas, and any subsequent versions, which are used to validate the XML documents.
Ensure previous versions of the XML schema files are maintained.
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Current status and system upgrade
European Directory of Marine Environmental Datasets (EDMED)
Cruise Summary Reports (CSR)
European Directory of the initial Ocean-observing Systems (EDIOS)
European Directory of Marine Environmental Research Projects (EDMERP)
European Directory of Marine Organizations (EDMO)
Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Cruise Summary Reports Directory (CSR)
Number of entries (in an Oracle DB):39 648
New entries since Trieste, 19/03/2007:1263
Characteristics:From 1873-2008, 2105 ships from 48 countries
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Continuing operation and populationContinuing operation and population
Heraklion/Triest we got 1071 CSRs
Online 33 % XML 34 % ICES 25 % Others 8 %
Since Triest we got 1263 CSRs
Online 52 % XML 44 % ICES 4 % Others 0 %
Cruise Summary Reports Directory (CSR)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Cruise Summary Reports Directory (CSR)
2007: 19 countries online submissions
19/36 = 52.78 %
Now: 29 countries online submissions
29/36 = 80,56 %
Entries since last annual meeting Entries since last annual meeting (23.03.2007 (23.03.2007 -- 31.03.2008)31.03.2008)
a big upgrade through France through XML submission, but also by Italy, Tunesia,Turkey, Ukraine
Country Online Totalupdates new updates new
Albania 1 1Algeria 7 7Belgium 1 22 23Bulgaria 39 39Croatia 21 21Cyprus 2 2Denmark 3 2 5Estonia 8 8Faroe Islands 18 18Finland 4 4France 20 247 1 268Georgia 0Germany 125 1 126Greece 29 29Iceland 1 1Ireland 54 54Israel 2 2Italy 12 12Latvia 10 10Lebanon 7 7Lithuania 3 3Malta 3 3Morocco 1 1Netherlands 3 2 5Norway 59 59Poland 9 9Portugal 3 3Romania 12 12Russia 28 28Slovenia 13 13Spain 46 46Sweden 94 94Tunisia 91 91Turkey 38 38Ukraine 14 146 160United Kingdom 20 20
Total 591 20 555 0 48 1214
in percent 49% 4%
XML ICES
CSR Entries since last annual meeting (23.03.2007 - 31.03.2008)
47%
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
NEW V1 XML FORMAT AND SCHEMA Laboratory, scientist, platform, project now defined by
common vocabularies or by SDN inventories, codes must be used now by CSR V1.2
ISSUE Not all existing CSRs could be harmonised to the
common vocabsACTION-SUGGESTION Partners get export to update those CSRs either online
or by Mikado (V1.2)
Cruise Summary Reportsts Directory (CSR)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Current status and system upgrade
European Directory of Marine Environmental Datasets (EDMED)
Cruise Summary Reports (CSR)
European Directory of the initial Ocean-observing Systems (EDIOS)
European Directory of Marine Environmental Research Projects (EDMERP)
European Directory of Marine Organizations (EDMO)
Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDIOS Progress (1)
No new input has been requested over the year Awaiting new technological developments (XML
schema and new version of Mikado) BODC has produced an improved Oracle
(database) schema Supported by common vocabularies, EDMO and
EDMERP A major issue is population of revised schema with
legacy content and quality assurance against original MIF stock
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDIOS Progress (2)
Within the UK BODC has a national project, UK Directory of the Marine
Observing System (UKDMOS) Adopted the EDIOS schema with some additions Preparing a new UK EDIOS collection
In cooperation with MARIS BODC has defined and prepared an exchange format of
6 csv files Exchange format can work both ways between BODC
database and interface database at Maris Provides basic model for tuning the XML schema later
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDIOS Progress (3)
New User Interface for both UKDMOS and for SeaDataNet - EDIOS
For UKDMOS see:http://www.ukdmos.org/welcome.asp
For SeaDataNet EDIOS:http://seadatanet.maris2.nl/v_edios/search.asp
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDIOS way forward
Two update routes:(1) Exchange format => XML schema => MIKADO version
=> XML output to BODC for validation => export to MARIS for loading in Interface
(2) MARIS builds CMS on the new database model Request partners to rebuild their EDIOS entries Bring in some data from the older EDIOS as basis plus
view on old EDIOS details for comparison when filling in all the Vocabularies
Exchange format in CSV (later as XML) to BODC for loading and validating
Return to MARIS for loading in Public Interface
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Current status and system upgrade
European Directory of Marine Environmental Datasets (EDMED)
Cruise Summary Reports (CSR)
European Directory of the initial Ocean-observing Systems (EDIOS)
European Directory of Marine Environmental Research Projects (EDMERP)
European Directory of Marine Organizations (EDMO)
Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Total No. of EDMERPs
Total No. of EDMERPs
Albania 0 2Algeria 3 4Belgium 209 128Bulgaria 32 35Croatia 9 7Cyprus 20 9Denmark 40 41Estonia 11 9Finland 16 14France 93 90Georgia 86 102Germany 18 23Greece 114 117Iceland 16 13Ireland 179 160Israel 12 4Italy (OGS +ENEA) 52 48Latvia 31 29Lebanon 5 12Lithuania 28 21Malta 31 30Morocco 5 4Netherlands 71 67Norway 3 4Poland 42 35Portugal 26 26Romania 18 16Russia (SIO + RIHMI) 84 103Slovenia 0 4Spain 91 79Sweden 30 15Tunesia 0 3Turkey 24 28Ukraine 62 79United Kingdom 46 236Other countriesTOTAL 1507 1600
Country
After 1st year SeaDataNet After 2nd year SeaDataNet
EDMERPEDMERP
Number of EDMERP entries is quite stable
Partners have been maintaining existing entries and adding new entries
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMERD developments
Recently the EDMERP system has been upgraded to Version 1 :
new V1 data format and XML schema, it is fully supported by the Common Vocabularies, double entries have been taken out, because projects are entered
and maintained by the country of the coordinator, who can add via EDMO entries all related partners,
Maintenance is now done by online Content Management System (CMS)and no longer by MS-Access software: capability of creation of sub-accounts for institutes in the NODCs
country, while the NODC safeguards the quality by having the chief editor role before publishing,
New Web service for retrieving EDMERP entries in XML.
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Recommendations
Partners are asked to complete the existing EDMERP records with keywords from the Common Vocabularies and partners from EDMO to support the content upgrading from Version 0 to Version 1
Partners are asked to keep EDMERP up-to-date, also with entries from other institutes in their country, because EDMERP is being used in other SeaDataNet Directories
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
New EDMERP V1 user interface
Browse list
Additional details
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMERP Content Management System (CMS)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Current status and system upgrade
European Directory of Marine Environmental Datasets (EDMED)
Cruise Summary Reports (CSR)
European Directory of the initial Ocean-observing Systems (EDIOS)
European Directory of Marine Environmental Research Projects (EDMERP)
European Directory of Marine Organizations (EDMO)
Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMOEDMO
Number of EDMO entries is quite stable
Partners have been maintaining existing entries and adding profile texts, logos and geographic positions
As part of producing CDI records a lot of new organisations have been added; also existing entries have been altered due to re-organisations, double entries etc.
No. of EDMO entries
No. of EDMO entries
Albania - PUT 0 1Algeria 1 1Belgium 63 69Bulgaria 14 12Croatia 5 5Cyprus 3 3Denmark 36 31Estonia 5 5Finland 9 10France 197 198Georgia 13 14Germany 58 58Greece 59 57Iceland 3 3Ireland 104 46Israel 3 2Italy (OGS +ENEA) 66 60Latvia 9 6Lebanon 2 2Lithuania 7 6Malta 7 7Morocco 3 3Netherlands 11 22Norway 7 23Poland 39 40Portugal 43 45Romania 9 8Russia (SIO + RIHMI) 94 88Slovenia 1 1Spain 47 68Sweden 29 32Tunesia 1 1Turkey 25 26Ukraine 12 10United Kingdom 136 153Other countries 17TOTAL 1122 1134
Country After 1st year SeaDataNet After 2nd year SeaDataNet
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMO recommendation
Recommendation: Try to keep the info up-to-date because EDMO is
being used in all other SeaDataNet Directories (EDMED, EDMERP, CDI, CSR and EDIOS)
Still a number of entries have no logo and no geographic position => please add
Do not keep entries inactive (red button in Content Management System)
In case of new entries, please inform MARIS for making the initial entry
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMO developments
Upgraded user interface
New option to keep track of no longer existing organisations with a link to present organisations
Google maps in the Content Management System (CMS) for improved placement of geographic locations
Web service for retrieving EDMO entries in XML
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Upgraded EDMO user interface
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
EDMO CMS
EDMO CMS geo-locator via Google maps
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Current status and system upgrade
European Directory of Marine Environmental Datasets (EDMED) Cruise Summary Reports (CSR)European Directory of the initial Ocean-observing Systems (EDIOS)European Directory of Marine Environmental Research Projects (EDMERP)European Directory of Marine Organizations (EDMO) Common Data Index (CDI)
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDICDI
All partners have been busy and the training workshop has been very useful for transferring expertise and instructing partners in use of CDI format and tools
A number of partners still have no entries, but are busy with producing test entries or production entries on short term. These include:o Albania PUT : has problems ; needs extra instructionso Belgium MUMM: waits for the CDI V1 versiono Estonia MSI: test files almost OKo Iceland MRI: some misunderstanding of CDI dataset definitiono Malta IOI-MOC: test files supplied, but some corrections required o Morocco INRH: test files supplied, but some corrections required o Poland IMGW: first test files supplied; almost ready for production o Portugal IHPT: test files supplied, but some corrections required
No. of CDI entries No. of CDI entries Albania - PUT 0 0Algeria ISMAL 0 17Belgium MUMM 21 21Belgium VLIZ 0 912Bulgaria IO-BAS 0 606Croatia IORS 0 7111Cyprus CYODC 0 1056Denmark NERI 13676 17042Estonia MSI 0 0Finland FIMR 0 1667France IFREMER 71314 74762France - CLS 0 8Georgia TSU 0 99Germany BSH 1594 29295Greece NCMR 10588 11588Iceland MRI 0 0Ireland MI 0 4727Israel IOLR 0 4817Italy - OGS 37926 37926Italy ENEA 0 5395Latvia IAE-UL 0 104Lebanon NCRS 0 7Lithuania CMR 0 810Malta IOI-MOC 0 0Morocco INRH 0 0Netherlands - NODC 5270 41588Norway IMR 13546 14099Poland IMGW 0 0Portugal IHPT 0 0Portugal - IST 0 396Romania NIMRD 0 2461Russia RIHMI 2668 12587Slovenia MBS-NIB 0 2709Spain IEO 0 15547Sweden SMHI 0 2600Tunesia INSTM 0 1497Turkey METU IMS 0 2477Ukraine MHI 0 7476United Kingdom BODC 40122 40122
TOTAL 195131 341499
Country After 1st year SeaDataNet After 2nd year SeaDataNet
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Recommendations
Try to keep the info up-to-date, especially the URLs for data access
Missing partners to undertake extra efforts
All partners to prepare for transgression from CDI V0 to CDI V1
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDI developments
Small updates to the CDI V0 user interface
Focus on V1 developments
Synthesis of the technical activity State of development of the on-line system V1by D.Schaap
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDI V0 User Interface
CDI V0 Search results
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDI V0 result
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDI V0 Overview map of point measurements
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDI V0 Overview map of point measurements around Greece
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
CDI V0 Overview map of point measurements in Black Sea region
Second Annual SeaDataNet Meeting, 3-4 April, 2008, Athens, Greece
Conclusions
Contents of the SeaDataNet Information System:
2006/2007 EDMED: 3.000 CSR: 38.525 EDMERP: 1.507 EDMO: 1.122 CDI: 195.131
2007/2008 EDMED: 3.500 CSR: 39.648 EDMERP: 1.600 EDMO: 1.134 CDI: 341.499
A very important increase in the CDI contents but full coverage of their national data
Top Related