Data and Data Requirements
description
Transcript of Data and Data Requirements
![Page 1: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/1.jpg)
Project number: 283465
Data and Data Requirements
Wouter LosUniversity of Amsterdam
![Page 2: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/2.jpg)
Project number: 283465
Environmental Science
oceanic and atmospheric
processes
long-term development of
the climate system
Biological processes
biodiversity
development of the cryosphere and lithosphere
Earth as a single complex and coupled system
23/10/12 W. Los - ENVRI @ EUDAT 2
![Page 3: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/3.jpg)
Project number: 283465
ESFRI Environmental Research Infrastructures
• Tropospheric research aircraft
COPAL
• Upgrade of incoherent SCATter facility
EISCAT-3D
• Multidisciplinary seafloor observatory
EMSO
• Plate observing system
EPOS
• Global ocean observing infrastructure
EURO-ARGO
• Aircraft for global observing system
IAGOS
• Integrated carbon observation system
ICOS
• Biodiversity and ecosystem research infra
LIFEWATCH
• Svalbard arctic Earth observing system
SIOS
23/10/2012 W. Losi - ENVRI @ EUDAT 3
![Page 4: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/4.jpg)
Project number: 283465
23/10//12 -ENVRI @ EUDAT 4
![Page 5: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/5.jpg)
Project number: 283465
![Page 6: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/6.jpg)
29/03/12 Pasquale Pagano - ENVRI @ EGI CF 2012 6
Radar interference dataGas (CO2 etc) fluxes
∂ (concentration)
Areal andsatelliteobservation
Species data, distributions, abundance, biomass, etc.
Observations, sensor data,collection data, DNA, etcMarine
sensors
Currents, salinity,deposition, etc
Platetectonics
Seismic data,satellite data,sensors, etc
![Page 7: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/7.jpg)
Project number: 283465
Goal
Enable multidisciplinary scientists to access and study data from multiple domains for “system level” research
by providing solutions and guidelines for the RIs common needs
Multiple data producersMultiple data consumers
723/10/12 W. Los- ENVRI @ EUDAT
![Page 8: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/8.jpg)
Project number: 283465
8
Approach
discover data which are heterogeneous in format, content, and metadata description
harmonise, integrate and analyse data across domains and RIs Pr
omot
e Ac
cess
ibili
tyPreserve Specificity
23/10/12 W. Los - ENVRI @ EUDAT
PROVIDE SOFTWARE TOOLS TO
![Page 9: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/9.jpg)
Project number: 283465
Data infrastructure requirements
Integrated data discovery across various catalogues(Near) real-time data handlingFederation over infrastructures/servicesPersistent identifiers mechanismMetadata definition and assignmentAttribution / crediting author/ownershipQuality control of dataProvenance and preservationArchiving vs. regeneration of data and/or results (processed data)Single sign-on, delegated authorisationRunning complex modelsData staging or moving computation to data
23/10/12 W. Los- ENVRI @ EUDAT 9
![Page 10: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/10.jpg)
Project number: 283465
First steps - priority areas
Integrated data discovery across various centres / catalogues
(near) Real-time data handling
Federation over existing (national or international) infrastructures / services
23/10/12 W. Los - ENVRI @ EUDAT 10
![Page 11: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/11.jpg)
Project number: 283465
1: Integrated data discovery
Integrated data discovery across various centres / catalogues
The challenge of being able to easily discover data which are heterogeneous (in format, content, and metadata description) and which are stored at different placesENVRI partners ESA, CNR, UvA and CSC are tackling thisDoes EUDAT see a role to contribute?
![Page 12: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/12.jpg)
Project number: 283465
Geospatial Data Services
Geospatial Repositories
Data Discovery
Data Access Data Process
OGCOpenSearch
Linked Open DataCatalogue Services
OGCWCS
THREDDS
OGCWPS
WPS 52N
P1 P2 P..
WPS Hadoop
Hadoop Cluster
HDFS
Data Pub. /Vis.OGC
WMS, WFS
GeoServer
gCub
e Da
ta
stag
ing
by courtesy of P. Pagano
![Page 13: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/13.jpg)
Project number: 283465
2: (Near) Real Time Data Handling
(near) Real-time data handlingThe challenge(s) of being able to handle real-time data Challenges include:
collecting, storing and cataloguing data as it arrives in real-time from sensorsprocessing data into derived data products in real-time analysing data in real-time
It was suggested that EUDAT might take this up
![Page 14: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/14.jpg)
Project number: 283465
3: Federation
Federation over existing (national and international infrastructures / services
The challenge of bringing together existing infrastructure components / services as contributions to the construction of an Research InfrastructureThe challenge of bringing about interoperability (syntactic and semantic) between separately owned and operated facilities that each contribute to the Research InfrastructureIs EUDAT also interested?
![Page 15: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/15.jpg)
Project number: 283465
Data infrastructure requirements
Integrated data discovery across various catalogues(Near) real-time data handlingFederation over infrastructures/servicesPersistent identifiers mechanismMetadata definition and assignmentAttribution / crediting author/ownershipQuality control of dataProvenance and preservationArchiving vs. regeneration of data and/or results (processed data)Single sign-on, delegated authorisationRunning complex modelsData staging or moving computation to data
23/10/12 W. Los- ENVRI @ EUDAT 15
![Page 16: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/16.jpg)
Project number: 283465
Need a Common Data Infrastructure
Managing the growing amount of dataImproving interoperability between infrastructures and across disciplines Promoting collaboration and clarifying roles and responsibilities
23/10/12 W. Los- ENVRI @ EUDAT 16
EUDAT contributions to the ENVRI consortium is welcome!
![Page 17: Data and Data Requirements](https://reader035.fdocuments.us/reader035/viewer/2022062310/5681641d550346895dd5d9e1/html5/thumbnails/17.jpg)
Project number: 283465
Thank you
http://envri.eu/
23/10/2012 W. Los – ENVRI @ EUDAT 17