Post on 06-Jan-2016
description
2. WP9 – Earth Observation Applications
ESA DataGrid Review Frascati, 10 June 2002
1. Welcome and introduction (15m)
2. WP9 – Earth Observation Applications (50m)
3. ESA and WP9: infrastr, appl, TB 0/1 (50m)
4. Demonstration of EO applications (45m)
5. ESA and WP9: scale up, effort, recovery plan (50m)
6. Side ESA actions related to GRID (30m)
7. Reviewer separate meeting (60m)
8. Conclusions (30m)
ESA DataGrid Review – 10 June 2002 – n° 2
Summary item 2
WP9 – Earth Observation Applications (50m)
DataGrid EO requirement (LF, 15m)
DataGrid WP9 tasks, WP9 Plan (JL, 15m)
ESA DataGrid Review – 10 June 2002 – n° 3
Earth Observation Community GRID interactive scenario
Common access to EO missions cataloguesAcquisition plan, order, delivery
Parametric data fusion and models integration
Collaborative publishing of results
On demand high level products generation
ESA DataGrid Review – 10 June 2002 – n° 4
EO and Network Computing data models:EO is more than “parallel task execution”!!
Distributed Computing Integration of data from various instruments
and missions
High-Throughput Computing Interferometry …
On-Demand Computing Generation of EO user products…
Data-Intensive Computing Archive data re-processing, climate modeling…
Collaborative Computing Scientists application interactions, Instrument
cal/val …
Ian Foster and Carl Kesselman, editors, “The Grid: Blueprint for a New Computing Infrastructure,” Morgan Kaufmann, 1999
ESA DataGrid Review – 10 June 2002 – n° 5
The Grid from a Services View
:
:E.g.,
Resource-specific implementations of basic servicesE.g., Transport protocols, name servers, differentiated services, CPU schedulers, public keyinfrastructure, site accounting, directory service, OS bypass
Resource-independent and application-independent services authentication, authorization, resource location, resource allocation, events, accounting, remote data access, information, policy, fault detection
DistributedComputing
Toolkit
Grid Fabric(Resources)
Grid Services(Middleware)
ApplicationToolkits
Data-Intensive
ApplicationsToolkit
CollaborativeApplications
Toolkit
RemoteVisualizationApplications
Toolkit
ProblemSolving
ApplicationsToolkit
RemoteInstrumentation
ApplicationsToolkit
Applications Chemistry
Biology
Cosmology
High Energy Physics
EO and Environment
...
Sci
ence
& A
pp
licat
ion
Sys
tem
ESA DataGrid Review – 10 June 2002 – n° 6
Number crunching: interferometry, subsidence,DEM generation
Pomona (Cal): subsidence velocity fields40 ERS1/2 images (92-99), Ambiguity: 28 mm
Digital Elevation Model
GRID requirements:• large data files (10+ GB) • stages with intensive
processing • science driven value adding
ESA DataGrid Review – 10 June 2002 – n° 7
• Provide a single access point to space systems to emergency & rescue organisations in case of disasters
• Participating Space Agencies: CNES, CSA, ESA, ISRO, NOAA, …
• Missions: RADARSAT; ERS, (Envisat); SPOT; IRS; NOAA, …
Charter for Disaster Management
ESA DataGrid Review – 10 June 2002 – n° 8
ESRIN
MATERA (I)
NEUSTREL.ITZ (D)
KIRUNA (S)- ESRANGE
MASPALOMAS (E)
TROMSO (N)
MATERA (I)
METADATABROWSEWEB
SEAWIFS
SPOT IRS-P3
LANDSAT 7TERRA/MODIS
STANDARDPRODUCTION CHAINS
MULTIMISSIONDATABASES FOR REMOTE ACCESSAND USER SERVICES
USERS
HISTORICALARCHIVES
USERS
PRODUCTS
AVHRR
ESA EO facilities real-time infrastructure
ESA DataGrid Review – 10 June 2002 – n° 9
No Projects1-25 Projects26-5051-100100+ Projects
Countries
P.I. geographic distribution
AOs: Stimulating scientific research world-
wide
AOs: Stimulating scientific research world-
wide
AO-1 (1986)
AO-2 (1994)
AO-3 (1998)
AO- ENVISAT (2000)0
100
200
300
400
500
600
700
Nr
of
pro
jec
ts
3500+ science Users of ESA data
3500+ science Users of ESA data
120 New Cat-1 Projects in 2001120 New Cat-1 Projects in 2001
700 Envisat AOs to start in 2002700 Envisat AOs to start in 2002
Envisat science community
ESA DataGrid Review – 10 June 2002 – n° 10
Why GRID in EO? (1)
EO Community: Progressive refinement of data from many data sources to produce higher quality products
Product generation chain involving distributed organisations and users
Collaborative: distributed users and data – large international cooperation
Discovery: large numbers of products & resources
Interoperabiltiy of catalogue and metadata already in operation
Web based data services
ESA DataGrid Review – 10 June 2002 – n° 11
Why GRID in EO? (2)
Massive, non-stop data volumes New instruments, sensors & product types
Distributed archives
Historical dataset reprocessing
Complex numerical processing algorithms
Near real-time turnover
ESA DataGrid Review – 10 June 2002 – n° 12
Needed GRID technologies
Resource-independent and application-independent services (middleware)
authentication, authorization, resource location, resource allocation, remote data access,
accounting, security, quality of services, fault detection, real time services, …
Specialized protocols, procedures, data standards, operational environments, interfaces to EO legacy systems…
EO dedicated portal and user access…
ESA DataGrid Review – 10 June 2002 – n° 13
OZONE: a case of Global Environmental Monitoring
GOME analysis detected ozone thinning over Europe 31 Jan 2002
GRID requirements:• Multi instrument data fusion• Distributed data sources,
science and institutional users
• Complex data processing (1d data = 20 dprocessing
• Near real time deliv.
ESA DataGrid Review – 10 June 2002 – n° 14
Application of DataGrid in EO
Focus on One EO application (Ozone) but explore:
Collaborative environment, parallel data processing, … Interface to legacy and COTS systems Develop generic components Re-use components to add new applications Integrate compatible technologies Integrate other data types (new envisat instruments)
Testing in “controlled” GRID environment (ESRIN-ENEA) and in “wide-European” environment
Feedback to DataGrid developers and Architecture Group