Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG :...
Transcript of Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG :...
![Page 1: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/1.jpg)
Louis Poncet IT/GD
LCG
Computing resources for handling LHC data and using them.
![Page 2: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/2.jpg)
Louis Poncet IT/GD
Summary
• LCG– The detectors– Data production– LHC computing– What is a Grid ?– LHC scientists usage– LHC Computing Grid– From events to computers– LCG Service hierarchy– Current activity on LCG
• EGEE– EGEE in numbers today– Collaborating Projects– Achievements– Sustainability– Links
2
![Page 3: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/3.jpg)
Louis Poncet IT/GD
ALICE
3
![Page 4: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/4.jpg)
Louis Poncet IT/GD
ATLAS
4
![Page 5: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/5.jpg)
Louis Poncet IT/GD
LHCb
5
![Page 6: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/6.jpg)
Louis Poncet IT/GD
CMS
6
![Page 7: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/7.jpg)
Louis Poncet IT/GD
Data production
• With 150 million sensors delivering data out of 40 million event per second
• LHC is :– The hugest scientific project in the wold– The coolest area of the known part of the universe– The biggest data producer and computing
resource in the world for one project• The computing resource requirement is
proportional to this machine
7
![Page 8: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/8.jpg)
Louis Poncet IT/GD
LHC computing
• The accelerator will be completed in 2008 and run for 10-15 years
• The target of LCG is to give to physicist computing resources able to use the data coming out from the LHC detectors
• Estimation of 15 Petabytes of usable data per year• Average of 0.1 – 1 GB/sec• We need to process the data around 150,000 cores with
at least 2 Gig of RAM per core• Requires many cooperating computer centers, as CERN
can only provide ~20% of the capacity• GRID technology is matching our requirements
8
![Page 9: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/9.jpg)
Louis Poncet IT/GD
What is a GRID ?
• The creation of a "virtual supercomputer" by using a network of geographically dispersed computers. Volunteer computing, which generally focuses on scientific, mathematical, and academic problems, is the most common application of this technology (Wikipedia definition)
9
![Page 10: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/10.jpg)
Louis Poncet IT/GD
LHC scientists usage
10
70!
538!
27!
4603!637!
55!
22! 87!
10!
Over 6000 LHC Scientists world wide
Europe: 267 Institutes, 4603 UsersOther: 208 Institutes, 1632 Users
![Page 11: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/11.jpg)
Louis Poncet IT/GD
LHC computing Grid
• What is LCG :– The architecture choose for processing LHC data– A deployment team of 300 persons– An application support for the 4 experiments– A team of data storage experts
• Maintenance :– Monitoring all sites every 4 hours– Information about sites status renew every 7 minutes– Continental support for computing centers– Tracking problems system
11
![Page 12: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/12.jpg)
Louis Poncet IT/GD
From events to Computers
12
![Page 13: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/13.jpg)
Louis Poncet IT/GD
From events to Computers
12
![Page 14: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/14.jpg)
Louis Poncet IT/GD
From events to Computers
12
![Page 15: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/15.jpg)
Louis Poncet IT/GD
From events to Computers
12
![Page 16: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/16.jpg)
Louis Poncet IT/GD
LCG Service hierarchy
13
Tier-0: the accelerator centreData acquisition & initial processingLong-term data curationDistribution of data Tier-1 centres
Tier-1: “online” to the data acquisition process > high availability
Managed Mass Storage - grid-enabled data service
Data-heavy analysisNational, regional support
Tier-2: ~140 centres in ~35 countries
•! Simulation
•! End-user analysis – batch and interactive
![Page 17: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/17.jpg)
Louis Poncet IT/GD
Current activity on LCG
14
![Page 18: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/18.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
EGEE project
15
•The EGEE project brings together experts from more than 50 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure which is available to scientists 24 hours-a-day.
•The project provides researchers in academia and business with access to a production level Grid infrastructure, independent of their geographic location. The EGEE project also focuses on attracting a wide range of new users to the Grid.
•The project's main focus is:
• To expand and optimize Europe's largest production Grid infrastructure, namely EGEE, by continuous operation of the infrastructure, support for more user communities, and addition of further computational and data resources.
• To prepare the migration of the existing production European Grid from a project based model to a sustainable federated infrastructure based on National Grid Initiatives for multi-disciplinary use.
![Page 19: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/19.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
EGEE project
15
•The EGEE project brings together experts from more than 50 countries with the common aim of building on recent advances in Grid technology and developing a service Grid infrastructure which is available to scientists 24 hours-a-day.
•The project provides researchers in academia and business with access to a production level Grid infrastructure, independent of their geographic location. The EGEE project also focuses on attracting a wide range of new users to the Grid.
•The project's main focus is:
• To expand and optimize Europe's largest production Grid infrastructure, namely EGEE, by continuous operation of the infrastructure, support for more user communities, and addition of further computational and data resources.
• To prepare the migration of the existing production European Grid from a project based model to a sustainable federated infrastructure based on National Grid Initiatives for multi-disciplinary use.
ArcheologyAstronomyAstrophysicsCivil ProtectionComp. ChemistryEarth SciencesFinanceFusionGeophysicsHigh Energy PhysicsLife SciencesMultimediaMaterial Sciences…
>250 sites48 countries>50,000 CPUs>20 PetaBytes>10,000 users>150 VOs>150,000 jobs/day
![Page 20: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/20.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
EGEE in numbers today• Infrastructure
– Number of sites connected to the EGEE infrastructure: 259– Number of countries connected to the EGEE infrastructure: 52– Number of CPUs available to users 24/7: ~ 72,000– Storage capacity available: ~ 20 PB disk + tape MSS
• Users– Number of VO using the EGEE infrastructure: > 200– Number of registered users: >7500– Number of people benefiting from the existence of the EGEE
infrastructure: ~14000– Number of jobs : >150k jobs/day – Application domains using EGEE is more than 15
16
![Page 21: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/21.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
Collaborating Projects
17
Applications
improved services for academia, industry and the public
Support Actions
key complementary functions
Infrastructures
geographical or thematic coverage
![Page 22: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/22.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
Achievements
18
• Results :– Data transfer worldwide record
- “First 500 Terabytes Transmitted via LHCGlobal Grid” (Slashdot Apr 25 2005).
– Around 30K cpus on the grid active from 150 to 270 active site, 15 petabytes.
– Around 30K jobs running per day.– BIOMED has used 420 Years of CPU time Worth of Data within 4
Months (Slashdot Sep 17 2007).• Interoperability :
– Interoperation with OSG is perfectly working.– Interoperability with NAREGI ready.
• Quality of cooperation increase every day.
![Page 23: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/23.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
Sustainability
19
• Need to prepare for permanent Grid infrastructure in Europe and the world• Ensure a high quality of service for all user communities• Independent of short project funding cycles• Infrastructure managed in collaboration
with National Grid Initiatives (NGIs)• European Grid Initiative (EGI)• Future of projects like OSG, NorduGrid, ... ?
![Page 24: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/24.jpg)
Andreas Unterkircher CERN GD
Enabling Grids for E-sciencE
INFSO-RI-508833
More information
20
www.cern.ch/lcg www.eu-egee.org www.opensciencegrid.org
www.eu-egi.org/
www.gridcafe.org
![Page 25: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/25.jpg)
Louis Poncet IT/GD
Links
• Monitoring tools :– http://gridportal.hep.ph.ic.ac.uk/rtm/
• CERN LCG website– http://lcg.web.cern.ch/LCG/
• GRID Cafe– http://gridcafe.web.cern.ch/gridcafe/
• NOVA science physics program– http://www.pbs.org/wgbh/nova/sciencenow/3410/02.html– http://www.pbs.org/wgbh/nova/elegant/
21
![Page 26: Computing resources for handling LHC data and using them. · LHC computing Grid • What is LCG : – The architecture choose for processing LHC data – A deployment team of 300](https://reader035.fdocuments.us/reader035/viewer/2022071219/605598fcba6f1b348b4b5ce5/html5/thumbnails/26.jpg)
Louis Poncet IT/GD
Questions
22