Best Practices Supporting Science at CSCS
Transcript of Best Practices Supporting Science at CSCS
![Page 1: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/1.jpg)
Best Practices Supporting Science at CSCS Michele De Lorenzi (CSCS)
![Page 2: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/2.jpg)
Case 1 – Simplify the usage of HPC resources GPU Accelerated In-Situ Visualization
§ Peter Messmer (NVIDIA Co-Design Lab for Hybrid Multicore Computing at ETH Zurich)
![Page 3: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/3.jpg)
Best Practices Supporting Science at CSCS 3
HPC Workflow
Workstation
Viz Cluster Supercomputer
File System
Setup
Dump, Checkpointing Visualization,
Analysis
Analysis, Visualization
![Page 4: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/4.jpg)
Best Practices Supporting Science at CSCS 4
Traditional Workflow: Challenges
Workstation
Viz Cluster Supercomputer
File System
Setup
Dump, Checkpointing Visualization,
Analysis
Analysis, Visualization
Lack of interactivity prevents “intuition”
I/O becomes main simulation bottleneck
Viz resources need to scale with simulation
High-end viz neglected due to workflow complexity
![Page 5: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/5.jpg)
Eliminating the IO Bottleneck: In-Situ ViZ
Workstation
Viz Cluster Supercomputer
Setup Analysis, Visualization
Best Practices Supporting Science at CSCS 5
![Page 6: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/6.jpg)
Eliminating the IO Bottleneck: In-Situ ViZ
Workstation
Viz Cluster Supercomputer
Setup Analysis, Visualization
Best Practices Supporting Science at CSCS 6
![Page 7: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/7.jpg)
Different Visualization Scenarios
§ 1) Legacy workflow § Separate compute & Viz system § Communication via filesystem
§ 2) In-situ partitioned or integrated § Different nodes for viz & compute § Communication via High Performance Network
§ 3) In-situ co-processing
§ Compute and visualization on same node
HPC System
Compute
GPU
Viz System
Viz GPU
Filesystem
HPC System
Compute
GPU
Viz GPU Network
HPC System
Compute+Viz nodes
Best Practices Supporting Science at CSCS 7
![Page 8: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/8.jpg)
Typical Scientific Visualization system
§ Visualization is more than Rendering
§ Simulation: Data as needed in numerical algorithm
§ Filtering: Conversion of simulation data into data ready for rendering § Typical operations: binning, down/up-sampling, iso-surface extraction, interpolation, coordinate transformation, sub-
selection, .. § Sometimes embedded in simulation
§ Rendering: Conversion of shapes to pixels (Fragment processing)
§ Compositing: Combination of independently generated pixels into final frame
Visualization
Filtering Rendering Compositing Simulation
Best Practices Supporting Science at CSCS 8
![Page 9: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/9.jpg)
Visualization Dataflow
CPU
GPU
Simulation
Filtering Rendering
Best Practices Supporting Science at CSCS 9
![Page 10: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/10.jpg)
Fully GPU accelerated Visualization
CPU
GPU
Simulation Rendering
Filtering
Best Practices Supporting Science at CSCS 10
![Page 11: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/11.jpg)
FuLLy GPU accelerated Visualization
CPU
GPU
Simulation Rendering
Filtering Speedup
Best Practices Supporting Science at CSCS 11
![Page 12: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/12.jpg)
Bonsai With In-Situ Viz On Daint
Best Practices Supporting Science at CSCS 12
![Page 13: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/13.jpg)
Acknowledgement
§ Bonsai development: Evghenii Gaburov (SURFsara), Jeroen Bédorf (CWI)
§ OpenGL on Daint: Gilles Fourestey (CSCS), Nina Suvanphim (Cray)
§ OpenGL on Titan: Don Maxwell (ORNL)
Thomas Schulthess (CSCS) and Jack Wells (ORNL) for providing access to Daint and Titan for these experiments
Best Practices Supporting Science at CSCS 13
![Page 14: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/14.jpg)
Case 2: Software as a service Supporting the MARVEL community
14
![Page 15: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/15.jpg)
MATERIALS’ REVOLUTION: COMPUTATIONAL DESIGN AND DISCOVERY OF NOVEL MATERIALS
EPFL-ETHZ-UNIBAS-UNIFR-UNIGE-USI-UZH-IBM-CSCS-EMPA-PSI MARVEL
Best Practices Supporting Science at CSCS 15
![Page 16: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/16.jpg)
Access Pattern
Piz Dora
External Login Access (ELA) CSCS
Best Practices Supporting Science at CSCS 16
EPFL
AiiDA Server
/store
Repository access
MARVEL Researcher
Research Community
Traditional access (batch jobs)
Access through AiiDa
Scientific Community access
![Page 17: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/17.jpg)
Case 3: Supporting scientific communities The Platform for Advanced Scientific Computing (PASC)
17
![Page 18: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/18.jpg)
Swiss Platform for Advanced Scientific Computing
§ Promote joint effort to address key scientific issues in different domain sciences exploiting the potential of the next generation of HPC architectures
§ Interdisciplinary collaboration between § domain scientists,
§ computational scientists,
§ software developers,
§ computing centres
§ hardware developers.
§ Builds on the principle of co-design involving all these actors
18 Best Practices Supporting Science at CSCS
![Page 19: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/19.jpg)
Best Practices Supporting Science at CSCS 19
PASC Organization
![Page 20: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/20.jpg)
PASC instruments
Organized in four pillars:
§ Application Support Network (ASN)
§ Co-design projects
§ Institutional computing system pillar supporting the procurement of institutional computing systems
§ The training pillar supporting the Swiss Graduate Program Foundations in Mathematics and Informatics for Computer Simulations in Science and Engineering (FOMICS).
20 Best Practices Supporting Science at CSCS
![Page 21: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/21.jpg)
Application Support Network (ASN)
§ ASN are intended to coordinate and support the effort in different domain science areas to pursue the objectives of the project
§ The following Domain Science Networks are active today: § Climate - Climate and Atmospheric Modelling
§ Earth - Solid Earth Dynamics
§ Life Sciences - Life Sciences Across Scales
§ Materials - Materials Simulations
§ Physics - Plasma Physics, Astrophysics and Multiscale Fluid Dynamics
21 Best Practices Supporting Science at CSCS
![Page 22: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/22.jpg)
Co-design projects
§ Interdisciplinary projects involving several (or all) actors § address the following priorities for optimal exploitation of new and
emerging supercomputing platforms: § Refactoring of key application codes and re-engineering of their algorithms.
§ Development of numerical libraries and their integration into application codes.
§ Incorporations of innovative sub-systems (e.g. I/O, data streams, etc.) into existing simulation codes
§ Development of programming environments or components, performance tuning and analysis/monitoring tools.
22 Best Practices Supporting Science at CSCS
![Page 23: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/23.jpg)
Supported projects
Physics and Astrophysics: § Particles and fields; PI: Laurent Villard (EPFL)
§ DIAPHANE Radiative Transport in astrophysical simulations; PI: Lucio Mayer (University of Zurich)
Material Science
§ ANSWERS: nano-device simulations; PI: Mathieu Luisier (ETH Zurich)
§ Electronic Structure Calculations: electronic structure calculations; PI: Jürg Hutter (University of Zurich)
§ ENVIRON A Library for Electronic-structure Simulations; PI: Stefan Goedecker (University of Basel)
23 Best Practices Supporting Science at CSCS
![Page 24: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/24.jpg)
Supported projects (continued)
Life Science § Angiogenesis in Health and Disease: In-vivo and in-silico; PI:
Petros Koumoutsakos (ETH Zurich)
§ Coupled Cardiac Simulations: HPC Framework for Coupled Cardiac Simulations; PI: Rolf Krause (University of Lugano), Alfio Quarteroni (EPFL)
§ Genomic Data Processing; PI: Ioannis Xenarios (University of Lausanne)
§ HPC-ABGEM: agent-based general ecosystems models; PI: Christoph Zollikofer (University of Zurich)
24 Best Practices Supporting Science at CSCS
![Page 25: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/25.jpg)
Supported projects (continued)
Geo-physics and Climate
§ GeoPC: hybrid parallel smoothers for multigrid preconditioners; PI: Paul Tackley (ETH Zurich)
§ GeoScale: A framework for multi-scale seismic modelling and inversion; PI: Andreas Fichtner (ETH Zurich)
§ Grid Tools: Towards a library for hardware oblivious implementation of stencil based codes; PI: Oliver Fuhrer (Meteosuisse)
Computer Science
§ Heterogeneous Compiler Platform for Advanced Scientific Codes; PI: Torsten Hoefler (ETH Zurich)
§ Multiscale applications: Optimal deployment of multiscale applications on a HPC infrastructure; PI: Bastien Chopard (UNIGE)
25 Best Practices Supporting Science at CSCS
![Page 26: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/26.jpg)
PASC conference
§ PASC organizes yearly a Platform of Advanced Scientific Computing Conference.
§ The goal is to bring together research groups within diverse scientific domains to foster interdisciplinary collaborations and to strengthen (HPC) knowledge exchange.
§ PASC15 – June 1-2, 2015 hosted by ETH Zurich: http://www.pasc15.org/
Best Practices Supporting Science at CSCS 26
![Page 27: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/27.jpg)
Best Practices Supporting Science at CSCS 27
![Page 28: Best Practices Supporting Science at CSCS](https://reader033.fdocuments.us/reader033/viewer/2022050100/626c5937bc4f4a3e9522185c/html5/thumbnails/28.jpg)
Thank you for your attention.