Astronomical GRID Applications at ESAC [email protected] Science Archives and Computer...
-
Upload
charleen-wilcox -
Category
Documents
-
view
220 -
download
0
Transcript of Astronomical GRID Applications at ESAC [email protected] Science Archives and Computer...
Astronomical GRID Applicationsat ESAC
Science Archives and Computer Engineering Unit
Science Operations Department
ESA/ESAC – Madrid, Spain
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 2ESA Science Operations
Department
European Space Astronomy Centre
ESAC default location for: Science operations Centres (SOC),
long history with astronomical missions
expanding with solar system missions
Science archives, Astronomy Planetary Solar Systems
ESA Virtual Observatory (VObs) activities, ESAC is the European VObs
node for space-based astronomy
http://www.esa.int/SPECIALS/ESAC/Located near Madrid, Spain
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 3ESA Science Operations
Department
ESA’s sentinels in the Solar SystemFrom the Sun to the planets
SOCsArchivesat ESAC
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 4ESA Science Operations
Department
The wonders of the UniverseAstronomy and fundamental physics at ESA
SOCsArchivesAt ESAC
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 5ESA Science Operations
Department
ESAC GRID Objectives
ESAC to be a Data Centre and a Data Processing Centre Natural link between Science Archives, GRID and VObs
ESAC Scientists to use the GRID for their daily science tasks Must be easy to use (“my homedir on the GRID”, “gridlogin”) User support to port their software to the GRID New massive computing facilities to substitute user workstation
GRID in support to the projects Project Standard Data Processing On-the-fly reprocessing from the Archives Instruments calibration monitoring and trend analysis HPC in the GRID
Collaboration with other GRID partners / organizations EGEE, Planck, UCM, IFCA, IVOA
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 6ESA Science Operations
Department
GRIDs at ESAC
ESACGRID On ESAC DMZ Part of EGEE Collaboration with other
EGEE sites Project Data Processing Close link to the ESAC
Archives
SCIGRID Intranet Globus based Scientist daily use
DMZ
INTRANET
FW
Computing Element (CE)
LCG CE(GT-preWS)
PBS
User Workstation
GW client commandsNFS
… SE, LFC, MON, VOMS ...
Worker Nodes(Blades PE1955)
SCIGRID PRIVATE LAN
node node...
SGE
GRIDWAY
GT4
SCIGRID
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 7ESA Science Operations
Department
ESACGRID Hardware Details
In production since the beginning of 2007
40 Blade Servers for around 280 cpus and 860GB RAM
Hardware features: DELL PowerEdge 1855, 1955 and M60. 2 Dual/Quad core Intel CPU 3.20GHz and 1.86GHz
OS: Scientific Linux CERN 4 32 bits or 64 bits
Based on EGEE middleware gLite 3.1 Part of EGEE
Planck and eurovo-dca Virtual Organizations
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 8ESA Science Operations
Department
Herschel Pipeline Data Processing
Herschel Science Centreat ESAC
MissionOperationsCentreESOC
Raw DataDB
Herschel PipelineRun on the GRID
Herschel Science Archive
Raw data
Calibrationdata
Scientific Products
Search and Download Herschel data
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 9ESA Science Operations
Department
Traditional XMM-NewtonScience Analysis Workflow
Vizualize data
Run SAS tasks andStore results
On Local computer
Download and Install SAS,Calibration DB,
3rd party SW, libraries, …
Search and download datafrom XMM-Newton Archive
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 10ESA Science Operations
Department
XMM-Newton RISA – Remote Interface for Science Analysis
Vizualize Results withVObs tools (VOSpec, Aladin)
VObs Access data
VObs DisplayRequest
Save Results onVOSpace
VObs Search data
XMM-Newton Archive
Define SAS Tasks
Search data with RISA Web client
Run SAS tasks on ESAC GRID
VObs Download data
Send SAS TasksTo the GRID
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 11ESA Science Operations
Department
XMM-Newton Mosaic Construction
Send Processingto the GRID
VObs Downloaddata
XMM-Newton Archive
Final Mosaics
Basic processing to get the pipeline
products (PPS) run in parallel for every
observation
4 maps x 5 bands
x 3 instrum.
Total mosaic (group all previously
generated mosaics)
3 finalmosaic
Build mosaics for every observation per band
3 combinedimagesper obs10 observations in 5 energy bands:
10 obs x 3 instrument x 5 bands x 4 maps (cts, exp, var, bkg) 600 maps before mosaicking+60 x 3 (cts, exp, var) = 180 summed maps per obs
10 obs = 780 maps
Construct all images and spectra (per band and per instrument)
18 mapsper obs
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 12ESA Science Operations
Department
XMM-Newton Mosaic Construction
Send Processingto the GRID
VO Downloaddata
XMM-Newton Archive
Final Mosaic
Basic processing to get the pipeline
products (PPS) run in parallel for every
observation
4 maps x 5 bands
x 3 instrum.
Total mosaic (group all previously
generated mosaics)
1 finalmosaic
Build mosaics for every observation per band
1 mosaicper obs
10 observations in 5 energy bands:10 obs x 3 instrument x 5 bands x 4 maps (cts, exp, var, bkg) 600 maps before mosaicking+60 x 3 (cts, exp, var) = 180 summed maps per obs
10 obs = 780 maps
Construct all images and spectra (per band and per instrument)
18 mapsper obs
Christophe ARVISETEGEE 09, ESAC GRID activities, 24/09/2009, page 13ESA Science Operations
Department
Acknowledgments
ESAC Computer Support Group ESAC Science Archives and VObs Team ESAC Herschel Data Processing Team ESAC XMM-Newton SAS Team Guillaume Belanger for the XMM-Newton Mosaicing work