Evolutions in the Copernicus Coordinated Data Access System...

INSPIRE Conference 2014 Aalborg, 19 June 2014

Evolutions in the Copernicus Coordinated Data Access System

for EO Satellite Data

J. Martin, R. Knowelden, G. VingioneGround Segment and Mission Operations Department,

Earth Observation Programmes Directorate,

European Space Agency


Overview

1. The Copernicus Programme and the CSCDA: from pre‐operations to operations, data access, data policy and growing demands, service charter and user communities

2. The Copernicus Ground Segment and the Coordinated Data Access System (CDS). CDS data flows, functionality and tools. CDS current status and ongoing evolutions.

3. Next generation Earth Observation (ngEO) functionality and integration with CDS

4. Bulk dissemination via Data Hub Service (DHuS) Software, functionality and access examples.

2


Space Component

In‐Situ Component

Services Component

The Copernicus Programme and the CSCDA

Pre‐Operations (since 2008 until Sentinels): more than 10 different ground segment and 20 EO space mission and has served a growing user community with a rapidly increasing library of EO products.

Operations: with the Sentinel satellites, outstanding increases gradually expected in number and volume of data products as well as in number of users.

3

Copenicus Space Component Data Access (CSCDA) evolutions shall support a smooth and seamless transition to the routine operations


Copernicus Space Data Access – A major undertaking

A major European Earth Observation Ground Segment design challenge:

Systematic processing and availability of the complete acquired data corresponding to a sustained generation rate (24h/7d) of a continuous stream of

500Mb/s of user products (Sentinel‐1,‐2,‐3 A series)

Distributed interconnected network of centres (improved redundancy)

Local centre access capacity built on top of a powerful and scalable network

4

With First Results!

Northern Antartic Peninsula – Coastal Islands(Sentinel-1, 13 April 2014)

“Sentinel‐1 A+B will be the backbone of

future ice charting”

Leif Pedersen, MyOcean

Rotterdam Harbour, Netherlands(Sentinel-1, 15 April 2014)Rotterdam Harbour, Netherlands(Sentinel-1, 15 April 2014)

“Sentinel‐1 provides a new dimension for maritime monitoring "

Dr. Olaf Trieschmann, EMSA


Copernicus Data Policy – Free, Full and Open

Users shall have free, full and open access to Copernicus dedicated Sentinel data and Copernicus service information

The CSC Ground Segment architecture implements this policy, and includes an evolutionary approach to further enhance the data exploitation by the broad user community

The CSC Ground Segment features dedicated data access infrastructure solutions, tailored to the needs of the various user typologies

7


CSC Ground Segment Data AccessTailored infrastructures adapted to different User Typologies

MSs Collaborative

Access

Copernicus ServicesAccess

CSC GS

Data Access

Scientific / OtherAccess

International Agreements

Access

8

ESA funded R&D projects

Scientific/ Other users

Copernicus Services

MSs CollGS Userse.g. public, scientific,

commercialInternational

users

Operated by MSs

National Mirrors National Mirrors (Inc local station)National Mirrors National Mirrors (Inc local station)

National Mirrors

International MirrorsInternational

Mirrors (Inc local station)

Operated by int. partners

Int’l Mirrors Int’l Mirrors


Common User Service Charter

9

The CSC Ground Segment provides a set of main services to all users, independently from the access infrastructure

User REGISTRATION Management of user account creation with dedicated username and password

User ACCESS MANAGEMENT Management of user requests for category assignment (upon agreement of Terms and Conditions)

DISCOVERY Service Dataset and product search (including visibility of planned acquisitions)

VIEW Service Visualization of browse image

DOWNLOAD Services Interactive Download (via web browser or using a specific download manager) Systematic background download based on defined filters (e.g. area, time window, product type, …)

BULK DISSEMINATION Service Dissemination of large volume of products on physical media, by land mail or via mirroring on dedicated data hub.

EMERGENCY Service Opening of an emergency “dossier” upon user call and management of data collection (tasking, selection from archive)

OFFLINE MAPPING Service Coordination of data collection for ad‐hoc requests

HELP DESK Service Support and communications to users. Management of inquiries, complains, and suggestions. Advertisement of datasets. Escalation of issues to dedicated mission level specialists and help desks.

RSS and HOSTED PROCESSING Service

Research Support Service and Hosted Processing Service operations


Data Flows from CCMs to Copernicus Users


• The CDS is currently in routine operations in version 2.1

• Most of the CCMs are already integrated, including recent upgrades to finalize machine‐to‐machine interfaces or include new missions (SPOT‐6/7, PROBA‐V). Completion of currently tasked CCMs expected Q2 2014.

• Major datasets and ad‐hoc orders routinely delivered to users

CDS Current Status


Major Current CDS Tools

Datawarehouse


• Overall evolutions of the CDS to version 3 and beyond have recently commenced (Q1 2014). These are foreseen to last 2 years and will prepare the system for:

• Sentinels integration and increase in data volumes thus entailed,

• Next Generatation Earth Observation (ngEO) integration ngEO (see later)

• Data Hub Service (DHuS) Software (see later)

• The aforementioned Services Charter

• All evolutions will ensure continuity with the current baseline.

• A clear objective is to design the CDS system taking into account the INSPIRE Directive, ensuring compliancy with Discovery and View Services and aiming to maximize the accessibility of the geospatial data and services it provides.

CDS On‐going Evolutions


• A core evolution of the CDS will be the integration of ngEO into version 3, providing:

• Core data access services offered through a uniform web‐based interface, supported on major browsers

• Comprehensive (INSPIRE‐compliant) DISCOVERY, VIEW and DOWNLOAD services provided

• Friendly to Web and OGC standards for Earth Observation

• Detailed product image as map layers (in selected cases)

• Sharable shopcarts, dataset search, browse and product URIs

• Systematic user‐defined downloads using performant download manager

next generation EO (ngEO) Evolutions


ngEO Service Highlights

• DOWNLOAD Service– Download Manager for download of individual products selected via DISCOVERY service– Catalogue search results provide a download address for each product– Download Manager is an open source multi‐platform application provided by ESA

• VIEW Service– Viewing of browse images of products found in the catalogue as a map overlay, allowing

panning and zooming– Support up to full resolution image for selected products/datasets– Compatible with third party map clients supporting the standards Web Map Service

(WMS) protocol from the Open Geospatial Consortium

• DISCOVERY Service– Search the catalogue for available or planned products, using user selected filters– Subscription to catalogue search with RSS notification of new products– Sharing of catalogue searches as URIs– Interferometric search support


Bulk Dissemination: Data Hub Software (DHuS)

• BULK DISSEMINATION Service– Online Access to Rolling Archive of Sentinel Data Products– Simple Open and Free Registration– Search, Preview, Inspection and Download– Intuitive Web and Scripting Interface for bulk download via http open data protocol


DHuS: Search

bash$ curl –u user –p password https://senthub.esa.int/odata/product?$search=“sentinel AND germany”


DHuS: Online Product Inspection

bash$ curl –u user –p password https://senthub.esa.int/odata/product(219)/Content?$path=annotation/*/product/qualityInformation/qualityDataList/qualityData/downlinkQuality/*


DHuS: Individual and Bulk Download

#!/bin/bash

# Disclaimer: API will change – backwards compatibility to be maintained fom Data Hub version 1

echo starting download …

curl –u user –p password –-metalink “https://senthub.esa.int/odata/product?$search=“sentinel AND germany”&$format=metalink”

echo done!


Further Information

• CSCDA Homepage: http://copernicusdata.esa.int/

• Sentinel Technical Web: http://sentinel.esa.int/

• Sentinel Data Hub: https://senthub.esa.int/Free and Open (feel free to register)

Sample Data Until Sentinel‐1 In Orbit Commissioning

Review (9/2014 TBC)

Full data access and detailed API to be

published accordingly

Salar de Uyuni, Bolivia(Sentinel-1, 20 April 2014)Salar de Uyuni, Bolivia(Sentinel-1, 20 April 2014)

Evolutions in the Copernicus Coordinated Data Access System...

Documents

Transcript of Evolutions in the Copernicus Coordinated Data Access System...