Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI -...
Transcript of Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI -...
![Page 1: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/1.jpg)
INFSO-RI-508833
Enabling Grids for E-sciencE
www.eu-egee.org
Preparing the LHC Computing Grid for MPI ApplicationsRichard de Jong & Matthijs Koot
![Page 2: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/2.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 2
Enabling Grids for E-sciencE
INFSO-RI-508833
Outline
• Context– CERN– Grid– LCG– YAIM
• Problems (and solutions) – YAIM
• Context (ct’d)– Parallel programming
• Problems (and solutions) (ct’d)– Single-site MPI– Cross-site MPI
• Conclusion
![Page 3: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/3.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 3
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: CERN & The LHC Project
• A new particle accelerator is being built at CERN
• 10 PB/year• Save (backup) &
Analyze all data• By physicists around
the world• O(8K) nodes at
CERN• But that’s not
enough…• Solution: Tier 0, 1, 2
![Page 4: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/4.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 4
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: Data flow
![Page 5: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/5.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 5
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: Grid
• A grid is a system that (Foster):– coordinates resources that are not subject to centralized control– using standard, open, general-purpose protocols and interfaces– to deliver nontrivial qualities of service.
• Keyword: Middleware
![Page 6: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/6.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 6
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: LHC Computing Grid• LCG middleware:
– LCG– gLite
• Both based on existing technology:– Globus Toolkit– VDT– GridFTP – Condor
• Components in the LHC architecture:– User Interface (UI) – To submit a job, retrieve output– Resource Broker (RB) – To find a suitable CE– Compute Element (CE) – To schedule the job – LRMS– Worker Node (WN) – To execute the job– Many more…
![Page 7: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/7.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 7
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: YAIM
• YAIM = Yet Another Installation Mechanism• A tool for Grid middleware deployment
– Scope– Structure– Evaluation
![Page 8: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/8.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 8
Enabling Grids for E-sciencE
INFSO-RI-508833
YAIM: Scope
• EDG - WP4:– "provide means to install OS and applications over the network
according to policies, to bring a machine in a desired state"• Configuration by LCFG(ng), but inflexible and error-
prone• So: QUATTOR
– Scope on OS and applications– But: not (so much) on Grid middleware
![Page 9: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/9.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 9
Enabling Grids for E-sciencE
INFSO-RI-508833
YAIM: Scope
• CERN IT-GD: "should not be so hard" YAIM– Configuration template– For small and/or simple sites
• But for larger sites?
![Page 10: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/10.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 10
Enabling Grids for E-sciencE
INFSO-RI-508833
YAIM: Structure
• Configuration files– site-info.def - Site wide configuration– node-info.def - What to configure for which role
• Scripts– install_node - Install packages– configure_node - Configure services
• Functions– One function covers an atomic piece of configuration – One file per function, one function per file
• Utilities– Helper routines
![Page 11: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/11.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 11
Enabling Grids for E-sciencE
INFSO-RI-508833
YAIM: Evaluation
• Pro:– Straightforward– Modular– Simple– Easy to use
• Con:– Not atomic– Unstructured output– Reconfiguring– Unconfiguring– Users learn to ignore the errors
• Conclusion: fine for small and/or simple sites– Meanwhile: QWG to integrate YAIM functionality for large sites
![Page 12: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/12.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 12
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: Parallel Programming- Traditional:
- Use a single processor to solve multiple problems = SISD
- Parallel programming:- Using multiple processors to
solve a single problem- Divide the work:
- Functional decomposition- Each processor a different role =
MISD- Domain decomposition
- Each processor different data = SIMD
- Both- Each processor it’s own program
and data = MIMD
![Page 13: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/13.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 13
Enabling Grids for E-sciencE
INFSO-RI-508833
- SISD
- SIMD
Context: Parallel Programming
![Page 14: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/14.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 14
Enabling Grids for E-sciencE
INFSO-RI-508833
- MISD
- MIMD
Context: Parallel Programming
![Page 15: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/15.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 15
Enabling Grids for E-sciencE
INFSO-RI-508833
Context: MPI
• MPI = Message Passing Interface• The MPI library is a widely known and used standard
for parallel programming• Multiple implementations available
– MPICH– LAM/MPI– OpenMPI– MS MPI– etc
• Traffic profile depends on parallel algorithm
![Page 16: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/16.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 16
Enabling Grids for E-sciencE
INFSO-RI-508833
• Next in line: [email protected]
![Page 17: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/17.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 17
Enabling Grids for E-sciencE
INFSO-RI-508833
Outline
• Context– CERN– Grid– LCG– YAIM
• Problems (and solutions) – YAIM
• Context (ct’d)– Parallel programming
• Problems (and solutions) (ct’d)– Single-site MPI– Cross-site MPI
• Conclusion
![Page 18: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/18.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 18
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Single-Site MPI
• MPI within a single grid-connected cluster
• Integration between MPI and LRMS
![Page 19: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/19.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 19
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
![Page 20: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/20.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 20
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
![Page 21: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/21.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 21
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• Job decomposition– Functional ~, domain ~– Is it possible to automate?
![Page 22: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/22.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 22
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• Co-allocation– Resource scheduling is tough!– Learn from Koala in DAS-2?– LRMS interoperability?
![Page 23: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/23.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 23
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
![Page 24: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/24.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 24
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• MPI interoperability– MPI is portable, not interoperable– Sites should provide common MPI
![Page 25: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/25.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 25
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• Networking– Latency(-tolerance?)– Protocols (public IP addresses?)– Security (dynamic firewall with VO-EAP?)
![Page 26: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/26.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 26
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• Networking (2)– Collective ops?
![Page 27: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/27.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 27
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• Integration is needed between MPI and grid middleware (AuthN/Z, monitoring)
• MPICH-G2 demonstrates integration between MPICH and Globus
• Might (still?) be usable on LCG
![Page 28: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/28.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 28
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
![Page 29: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/29.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29
Enabling Grids for E-sciencE
INFSO-RI-508833
Problems: Cross-Site MPI
• It has been demonstrated– Legion (2001)– TeraGrid (2003)– K*Grid in Korea (2005)
• …but not yet over a high-latency, heterogeneous, Internet-based grid
![Page 30: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/30.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 30
Enabling Grids for E-sciencE
INFSO-RI-508833
• Single-Site MPI is probably OK
• Cross-Site MPI is tough, but possible– Good software engineering required
• YAIM needs improvement
Conclusion(s)
![Page 31: Preparing the LHC Computing Grid for MPI Applications · Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 29 Enabling Grids for E-sciencE INFSO-RI-508833](https://reader033.fdocuments.us/reader033/viewer/2022043010/5f9eb3c71834d732af207ab5/html5/thumbnails/31.jpg)
Preparing the LHC Computing Grid for MPI - Richard de Jong & Matthijs Koot 31
Enabling Grids for E-sciencE
INFSO-RI-508833
Questions
• Questions?