1Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Challenges in the provision of large data sets to multiple user communities
14-16 July 2014, Boulder CO
MONITORING WEATHER AND CLIMATE FROM SPACE
2Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Mission Statement
We are the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT).
EUMETSAT is an intergovernmental organisation and was founded in 1986. Our purpose is to supply weather and climate-related satellite data, images and products
• 24 hours a day, • 365 days a year
to the National Meteorological Services of our Member and Cooperating States in Europe, and other users worldwide.
www.eumetsat.int
3Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Headquarters (Darmstadt, Germany)
4Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Data Services - High Level Overview
Users
Discover, search, register, order and subscribe
Subscribe to services
Discover/search products
Register to messages
Distributed search, remote order
Order/ subscription products
Subscribe to services
GEONETCast Catalogue
EUMETCast/ GEONETCast
Unified Meteorological
Archive(U-MARF)
EUMETSAT Partner Agencies/
GMES, WMO
User Notifcations(UNS)
EUMETCast / GEONETCast
Other Dissemination
Services
Registration
Discovery (Product Navigator)
Order Followup
Access to applications & data
EUMETSAT EO Portal
5Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Data Services - Data Flows
EUMETSAT GROUND INFRASTRUCTURE
6Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Single Access Point
One stop shop
Data discovery
User registration (>10.000)
Subscription for data services: Near-real time data
Online Data Access
Long Term Archive
Additional Image Gallery (WMS/ WCS)
User Notifications
Operational Real-Time Services Status Indicator
EUMETSAT Earth Observation Portal
7Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Dissemination Near Real Time data delivery to users in 3 continents via Satellite
Users (2/2014)
4
3
2-10
11-50
51-200
201-500
1 User
500+
0 User
3,280 users worldwide,4168 stations5 new users per month
EUMETCast AmericasSES 6, C-BAND
EUMETCast AfricaEUTELSAT 5 WEST A, C-BAND
EUMETCast EuropeEUTELSAT 9 A, KU-BAND
8Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Dissemination:Integrating terrestrial dissemination
9Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
MULTICAST ACCROSS GÉANT and NREN INFRASTRUCTURE
GEANT/NRENs
EUMETCast Platform (MMDS)
EUMETCast User Stations
Reference Stations
Multicast ...Multicast
Multicast
Exploring the multicast usage on GÉANT infrastructure in Europe and their world wide connectivity;
2011 Proof of concept on terrestrial multicast on GÉANT test bed
2012 3 month feasibility study on terrestrial multicast via GÉANT
2013 Multicast test to KMA Korea
2014 Multicast test to NOAA (Boulder)
2014 Start of 2 year prototype emulating operational scenarios of multicast across Europe and our overseas partners using GÉANT, NREN, I2 and Teint infrastructure.
10Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Terrestrial real-time monitoring
- Multiple Multicast channels- SSM mode- 20-30Mbit/s- FEC ¾- Closed loop monitoring based on successful file receipt- Packet/channel based ack/nack retransmission possible but not yet enabled
Acknowledgements: N-WAVE&NOAA Alex Hsia, David Hartzell
11Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETCast Dissemination:Low cost, off the shelf Receiving Station
DVB PCI Card
EKU
AntennaLaptop/ PC
SoftwareDVB USB
DVB Router
Typical total cost of a station < 2000 Euros
12Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
EUMETSAT Long Term Archive
Archived data starting 1981 All EUMETSAT Satellite data
including reprocessed data > 1 Petabytes in the archive
for MTP, MSG, Metop Retrieval/Ingestion: 10 to 1 Long-Term Data Preservation
in practice since 1995; Disaster resilient - 3rd Copy Access via EUMETSAT EO
Portal
13Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
What we are working on…
14Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook – Enhance Future abilities in terms of
• Scalability• Flexibility,• Responsiveness to emerging needs,• Resource efficiency• While aiming for:
• Better sharing of resources,• Avoidance of data islands,• Reducing costs
• Fit the organisational structure
EUM/TSS/VWG/14/762641, 7 July 201415
Many aspects to consider => Challenge
Viewing Angle
16Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Product delivery and management
• Generation of user tailored products• Efficient for write once, read many times• Suitable products for user needs and
dissemination infrastructure (bandwidth)• Avoid/ minimise processing where possible• Efficient for slicing
• Usage of Format evolutions• Secure the raw data (golden copy)• Initially smaller products which increase over
time• Reprocessing to re-build formats
17Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Bringing data to Users
• Adaptive and flexible services to deliver data to users/ communities (e.g. EUMETCast)
• Community driven formats optimised for dissemination mean
• Serving large user groups from different communities
• Managing data policies in accordance with data provider requirements via a data policy and legal framework support by the related technical mechanisms
18Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Bringing Users to data
Standardisation activities in relation to Discovery, Access and Retrieval (DAR) of data
Strong focus on interoperability and ease of user experience
One stop shop experience for discovering services and data
Facilitating the Interoperability of portals by exploiting standards (OGC, INSPIRE)
Enhance Long Term Archive, Online Data Access
Outreach activities Exploiting capabilities to execute user algorithms on data
Post-Processing, Hosted Processing
19Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Outlook - Information to Users
• Delivering Information to users rather than data
• Processing where the data is located
• Service Interfaces instead Facility interfaces
• Application of (private) cloud principles
20Improving Data Mobility & Management for International Climate Science, [email protected] & [email protected]
Thank you for your attention
Top Related