THE ATLAS COMPUTING MODEL - Gaziantep Üniversitesibeddall/GridWorkShop2008/downloads/... ·...
20
Transcript of THE ATLAS COMPUTING MODEL - Gaziantep Üniversitesibeddall/GridWorkShop2008/downloads/... ·...
Outline
● LHC Data● Needs for LHC experiments● Data types● Brief summary ATLAS Facilities and their roles● The ATLAS Tier and Cloud model● The Tier centers● Growth of resources● CPU, Disk, Mass Storage
Resources
https://atlas.gantep.edu.tr/GridWorkShop2008/downloads/supplements/
LHC data
LHC will produce 40 million collisions per second per experiment
After filtering, ~100 collisions per second will be of interest
A Megabyte of data digitised for each collision = recording rate of 0.1 Gigabytes/sec
1010 collisions recorded each year = 10 Petabytes/year of data
1 Exabyte (1EB)
= 1000 PB
1 Petabyte (1PB)
= 1000TB
1 Terabyte (1TB)
= 1000GB
1 Gigabyte (1GB)
= 1000MB
Needs for LHC experiments
Needs for LHC experiments
• each event is independent of the others• computing power ~100.000 PC (P4 3 GHz)• storage capacity for LHC experiments
• 20 petabytes/y on magnetic tape• 1 petabyte/y on disc for the analysis
• possibility to access data from institutes• production of Monte Carlo data necessary to the understanding of results of the analysis (30 mn/event)
Data replication and distribution
In order to provide a reasonable level of data access for analysis, it is necessary to replicate the ESD, AOD and TAGs to Tier-1s and Tier-2s.
RAW: Original data at Tier-0 Complete replica distributed among all Tier-1 Data is steamed by trigger type (inclusive streams)
ESD: ESDs produced by primary reconstruction reside at Tier-0 and are exported to
2 Tier-1s (ESD stream = RAW stream) Subsequent versions of ESDs, produced at Tier-1s (each one processing its own RAW), are
stored locally and replicated to another Tier-1, to have globally 2 copies on diskAOD:
Completely replicated at each Tier-1 Partially replicated to Tier-2s (~1/3 – 1/4 in each Tier-2) so as to have at least a complete set in
the Tier-2s associated to each Tier-1 (AOD stream <= ESD stream) Cloud decides distribution; Tier-2 indicates which datasets are most interesting for their
reference community; the rest are distributed according to capacityTAG:
Access to subsets of events in files and limited selection abilities TAG replicated to all Tier-1s (Oracle and ROOT files) Partial replicas of the TAG will be distributed to Tier-2 as ROOT files
Each Tier-2 will have at least all ROOT files of the TAGs matching the AODsSamples of events of all types can be stored anywhere, compatibly with available disk capacity, for particular analysis studies or for software (algorithm) development.
Event Builder
Event Filter
Tier3
10 GB/s
320 MB/s
~ 100 MB/s 1010
~20 MB/s
~PB/s
Tier2 3-5/Tier13-5/Tier1
Tier0
Tier1
ATLAS Event Data Model
All Tiers1TAG:event-level metadata
Tier-3s (e.g. your laptop)Sometimes called N-Tuples
10DPD:Derived Physics Data
Tier-0, Tier-1/2 - at least one complete copy per cloud, Tier-3 – subsetQuantities in particular relevant for physics analysisMain input for analysis, distributed to many sites
100AOD:Analysis Object Data
Tier-0 and Tier-1sOutput of the reconstructionOften large; difficult to analyze the full set
500ESD:Event Summary Data
Tier-0 and Tier-1sHave to be recorded on permanent storage
1600RDO: Raw Data Objects
AccessibilitySize (kB)
Data Type
1.6 MB1.6 MB
TargetTarget
500 kB500 kB
CurrentlyCurrently
750/900 kB750/900 kB
TargetTarget
100 kB100 kB
CurrentlyCurrently
250/290 kB250/290 kB
10% of AOD10% of AOD
Processing
● Tier-0:– Copy RAW data to CERN Castor Mass Storage System tape for archival– Copy RAW data to Tier-1s for storage and subsequent reprocessing– Run first-pass calibration/alignment (within 24 hrs)– Run first-pass reconstruction (within 48 hrs)– Distribute reconstruction output (ESDs, AODs & TAGS) to Tier-1s
● Tier-1s:– Store and take care of a fraction of RAW data (forever)– Run “slow” calibration/alignment procedures– Rerun reconstruction with better calib/align and/or algorithms– Distribute reconstruction output to Tier-2s– Keep current versions of ESDs and AODs on disk for analysis– Run large-scale event selection and analysis jobs
● Tier-2s:– Run simulation (and calibration/alignment when/where appropriate)– Keep current versions of AODs and samples of other data types on disk for analysis– Run analysis jobs
● Tier-3s:– Provide access to Grid resources and local storage for end-user data– Contribute CPU cycles for simulation and analysis if/when possible
ATLAS Analysis Model
● Basic principle: Smaller data can be read faster– Skimming - Keep interesting events– Thinning - Keep interesting objects in events– Slimming - Keep interesting info in objects– Reduction - Build higher-level data
● Derived Physics Data– Share the schema with objects in the AOD/ESD– Can be analyzed interactively
9
The LCG service model
RAL
CC-IN2P3
BNL ...FNAL
Tier-1
Tier-2
LHC: 140 sites connected on LCG gridTier-0 (CERN):• acquisition of raw data and 1st reconstruction• long term mass storage• distribution of data to the Tier-1s
GRIF
LPC
Rome
NIKHEF
Cambridge
USC
Taipei
...
FZK
LAPP
CPPM
Tier-0
Tier-3
Tier1 (regional and national centres):• receive a part of raw data (storage)• reprocessing (alignements, calibrations)• provide summary data to the Tier2s • national and regional grid support
• 11 Tier1s up to now
Tier3 (in labs/on desk):• data analysis
Tier2 (close to the labs):• data analysis – batch and interactive • data simulation• receive summary data from Tier1• send simulated data to the Tier1
• 40 Tier2s up to now
10 G
bits
/s1
Gbit
s/s
ATLAS Computing System (R. Jones)
Tier2 Centre ~200kSI2k
Event Builder
Event Filter~159kSI2k
T0 ~5MSI2k
UK Regional Centre (RAL)
US Regional Centre French Regional Centre
Italian Regional Centre
SheffieldManchesterLiverpoolLancaster
Workstations
10 GB/sec
450 Mb/sec
100 - 1000 MB/s
●Some data for calibration and monitoring to institutes
●Calibrations flow back
Physics data cache
~Pb/sec
~ 300MB/s/T1 /expt
Tier2 Centre ~200kSI2k
Tier2 Centre ~200kSI2k
≥ 622Mb/s
Tier 0Tier 0
Tier 1Tier 1
DesktopDesktop
Northern Tier ~200kSI2k
Tier 2Tier 2 ~200 Tb/year/T2
~7.7MSI2k/T1 ~2 Pb/year/T1
~9 Pb/year/T1 No simulation
≥ 622Mb/s
Tier 1 centres
TaiwanASGC
SpainPIC
DK/FI/NO/SENordic Data Grid Facility
NetherlandsNIKHEF/SARA
GermanyFZK-GridKA
68106Total
USAFNAL
USABNL
United KingdomRAL
ItalyCNAF
FranceCC-IN2P3
CanadaTRIUMF
LHCbCMSATLASALICE
Experiments served with priorityCountryInstitution
Sou
rce:
CR
RB
– W
LCG
Mem
oran
dum
of U
nde
rsta
ndin
g –
200
6/06
/01
Computing resources requirements
53 3518Mass Storage
[PB]
5719317Disk[PB]
142 M61 M56 M25 MCPU
[SpecInt2000]
TotalAll Tier-2sAll Tier-1sCERN (T0 + AF)
• More than 53.000 today’s dual-core AMD Opteron•Around 355 TFLOPS
ATLAS Requirements for 2008
Resources for Analysis (2008)
90%67%60%Analysis
10%-20%Reprocessing
-33%20%Simulation
CAFTier-2sTier-1sCPU share
25%39%10%DPD
20%25%25%AOD
30%35%55%ESD
25%1%10%RAW
CAFTier-2sTier-1sDISK share
ATLAS Requirements start 2008, 2010
201020082010200820102008
41.1
28.7
1.0
11.4
65.4
22.1
40
2.8
0.5
112.2
51.5
50
4.6
6.1
10.518.941.4Total
7.717.5Sum of Tier-2s
7.71018.1Sum of Tier-1s
0.41.02.1CERN Analysis Facility
2.40.153.7Tier-0
Tape (PB)Disk (PB)CPU (MSi2k)
ATLAS T0 Resources
ATLAS T1 Resources
ATLAS T2 Resources
THANKS!
