1. ESA Unclassified - For Official Use Big Data from Space Research, Technology and Innovation 2. ESA UNCLASSIFIED For Official Use We are surrounded by BIG DATA Textual information (Google crawlers) Geo-location (Google Earth, Google Maps) Sharing multimedia Social Networks Credits: Psdblast.com 3. ESA Unclassified - For Official Use Big data from Space 4. ESA Unclassified - For Official Use Big data from Space: VOLUME Sheer value of sensed data: archived data is currently reaching the exabytes scale. 5. ESA Unclassified - For Official Use Big data from Space: VELOCITY New data is acquired almost on a continuous basis and with an increasing rate 6. ESA Unclassified - For Official Use Big data from Space: VARIETY Data is delivered by sensors acting over various frequencies of the electromagnetic spectrum in passive and active modes 7. ESA Unclassified - For Official Use Big data from Space: VERACITY Sensed data is always associated with uncertainty and accuracy measurements 8. ESA Unclassified - For Official Use Big data from Space: VALUE Our capacity to extract information and meaning from data 9. ESA Unclassified - For Official Use Big data from Space: focus on USERS Various space-related domains are concerned: Earth Observation, Space Science, Solar System Objects, Space Situational Awareness, Secure Societies, etc. 10. ESA Unclassified - For Official Use Overview Data Lifecycle Infrastructures and Services for Big Data Big Data from Space Scenarios Objectives and Activities 11. ESA UNCLASSIFIED For Official Use Big Data Lifecycle Acquire Organize Information Provision Analyse 12. ESA UNCLASSIFIED For Official Use Big Data Lifecycle: Supporting Resources SUPPORTING RESOURCES AND INFRASTRUCTURES Architectures Data selection & pre-processing Data & Software Frameworks Processing Algorithms Provenance & Security 13. ESA Unclassified - For Official Use Big Data from Space: Technology Themes Examples Data Lifecycle Processing and Analysis Visualization and Visual Analytics Multi-temporal Analysis Onboard Computing, Compression and Transmission Quality, Provenance and Trust Infrastructures and Services for Big Data Storage and computing platforms Interoperability and Standards Heterogeneous Data Sources (incl. auxiliary and collateral data) Linked Data and Semantics Data Openness, Privacy and Security Software Defined Networking User Management and Access to Resources 14. ESA Unclassified - For Official Use Scenario 1: Space Science Limited downlink capabilities Data Volumes and Variety Transmission on huge distances Use of landers Use of relay satellites 15. ESA Unclassified - For Official Use Scenario 1: Space Science Ground Segment Big Data Stream Big Data Analysis Processing & Triage Scalable Archiving Distributed Archives Data Analysis Science Team Research Community Data Models & Simulations Cal/Val Activity 16. ESA Unclassified - For Official Use Scenario 2: Earth Observation Satellite constellations Nano Satellites, RPAs, etc. Satellite instruments Relay satellites 17. ESA Unclassified - For Official Use Scenario 2: Earth Observation Ground Segment Big Data Stream Big Data Analysis Cal/Val Activity Validation Research Community Data Models simulations Applications Satellite constellations Nano Satellites, Drones, etc. Satellite instruments Services 18. ESA Unclassified - For Official Use Scenario 3: Space and Security Satellite Data Aerial, In-Situ and Collateral Data Decision Makers First Responders Operational Information Support EU external security policies Border SecurityResilience against Disasters Protection of Critical Infrastructures 19. ESA UNCLASSIFIED For Official Use Scenario 4 : Big Data Exploitation Current: bring data to users Data Ground Segment User Segment Mission Operator User Processing resources Data Request Discussion New results Long feedback loop Data Download Communities 20. ESA UNCLASSIFIED For Official Use Scenario 4 : Big Data Exploitation Future: bring users to data Ground Segment User Segment Mission Operator User Discussion Processing resources New results Data Request & Processing Data Shorter feedback loop Communities 21. ESA Unclassified - For Official Use Big Data from Space: Objectives 1. Identification of priorities and implementation of a plan for research, technology development and innovation 2. Widening competences and expertise of universities, research institutes, labs, SMEs and industrial actors in the field 3. Foster networking of experts and users towards better access and sharing of data, tools and resources 4. Leverage innovation, spin-in / spin-off of technologies and business development arising from research and industry progress 5. Increase and promote the value stemming from the huge quantity of data made available nowadays (and in the future) 6. Contribute to the EO innovation for Europe, as one of the main pillars for the Ground Segment evolution strategy 22. ESA UNCLASSIFIED For Official Use The Big Data from Space initiative shall: 1. Foster and coordinate the European Research & Development activities to address forthcoming Big Data challenges 2. Define and implement technology development programmes supporting cross- cutting enabling technologies in the Big Data field 3. Establish and promote a close and permanent dialogue with key industry and SMEs actors 4. Contribute to the exploration and implementation of partnership models for the actual exploitation of future Big Data flows 5. Contribute to the EO innovation for Europe, as one of the main pillars for the Ground Segment evolution strategy 6. Support academic activities to build a new figure: the data scientist Recommendations 23. ESA UNCLASSIFIED For Official Use BiDS Activities in EO / EOP-G 1. Research and Service Support (RSS) Since 2007, RSS - as part of ESA EO Ground Segment Department - provides tools and services to support the user communities in EO data exploitation (http://wiki.services.eoportal.org/tiki-custom_home.php): G-POD (Grid Processing on Demand): operational since 2007 - operational processing environment - access to available EO products - integration of specific applications/processors - coupled with high-performance and sizeable computing resources Cloud Toolbox - customized Virtual Machines (VMs) deployed on a cloud infrastructure - fast access to the ESA RSS Data Farm archive. - access through user devices (pc, laptop, tablet) from anywhere 24. ESA UNCLASSIFIED For Official Use BiDS Activities in EO / EOP-G 2. Exploitation Platforms (EP) Virtual workspaces providing user communities with access to: a) large volumes of data (EO, auxiliary, in-situ) b) algorithm development and integration environment c) processing software and services (e.g. toolboxes, visualization tools) d) computing resources (e.g. hybrid cloud/grid) e) collaboration tools (e.g. forums, wiki, KnowledgeBase, social networks) f) general operation capabilities (UM, AA, accounting) Thematic Exploitation Platforms: under development (Geohazards [pre-operational qualification on-going], Urban, Forestry, Polar, Hydrology, Coastal) Mission Exploitation Platforms (Proba-V, etc.) 25. ESA UNCLASSIFIED For Official Use BiDS Activities in EO / EOP-G 2. Generic GS functions in support of Big data management Generic functions under development and pre-ops: a) Federated User Management (SSO in operations, pilot completed, generic component under development) b) Data Hub Software (DHuS): open source component used in the frame of Copernicus/Sentinels for data access management including data hub relays (sentinels.copernicus.eu) 26. ESA UNCLASSIFIED For Official Use BiDS Activities in EO / EOP-G 4. Technology Projects (TRP/GSTP) Identify requirements and develop relevant technologies in support to the EO Ground Segment: a) Harmonization and Standards (HMA, SMARTHMA) b) EO Ontology (OTEG, RARE, SMAAD, OBEOS, ProdTrees) c) EO Image Information Mining (KIM, KEO, ASIM, EOLib, PFA, OSIRIDE, DAMATS) d) Exploitation Platforms demonstrators (EP4SM, EO-RCP) e) E-Collaboration platforms (E-CEO) 27. ESA Unclassified - For Official Use Big Data from Space Big Data from Space Conference - BiDS16 15-17 March 2016, Tenerife, Spain! 28. ESA UNCLASSIFIED For Official Use BiDS16 Organising Committee J. Amador Monteverde ESA, C. Arviset ESA, S. Albani SatCen, S. Baillarin CNES, P. Baumann Jacobs University, L. Bruzzone University of Trento, M. Datcu DLR, M. Iapaolo ESA, J. Knapen IAC, P.G. Marchetti ESA, P. Soille JRC, J.L. Valero SatCen 29. ESA Unclassified - For Official Use Big Data from Space