Exascale Oriented Solutions and Their Application to the ... · PDF fileLinear Fresnel...
Transcript of Exascale Oriented Solutions and Their Application to the ... · PDF fileLinear Fresnel...
Exascale Oriented Solutions and Their Application to the
Exploitation of Energy Sources
Rafael Mayo-GarcaCIEMAT Assoc Prof and Harvard-IACS Fellow
Rensselaer Polytechnic InstituteMarch, 28th 2017
Will this be useful for me?
Why HPC and energy production
A hint on some exascale trends
Some examples
Index
2
Interesting?
3
Energy is one of the major societal challenges that we must afford
New computational methodologies are suitable to be applied to other fields
Potential collaborations could be an outcome
In the worst case, this will not take too long, I promise
Energy is not a joke
4
Energy & HPC
5
CIEMAT (http://www.ciemat.es) is the main Spanish public organism devoted to research on energy and its related technologies The Spanish DOE
Really???? DOE budget: $32.5 Billion CIEMAT budget: $140 Million
Energy & HPC
6
CIEMAT was created in 1951 and was pioneering in supercomputing activities in Spain (1959) It is logic to infer that it focuses on merging/comparing
experimental and simulated results
IACS is skilled in Stochastic Optimization Methods, Monte Carlo Methods for Inference and Data Analysis Working on HPC & HPDA convergence
Multidisciplinary aspects are a must
Simulation (Computer Science) is probably the most transversal and complementary field nowadays
Energy & HPC
7
Energy & HPC
8
NUMBERTotal scientific-technical facilities 60Number of laboratories 161Other facilities (training, protection, medicalservice, ..) 23
Pilot pelleting plant (300-500 kg/h)
PET/CT of small animalsfor biomedicalapplications
Ionizing RadiationMetrology Laboratory
(LMRI)
Wind Testing LaboratoryHeliac Flexible TJ-II
Solar Platform of Almera (PSA)
Dosimetric measurementsin mannequin
Pilot membrane gasseparation plantSafety SystemAnalysis Laboratory
(Peca Facility)
9
Approaching a Disruptive Exascale Computer Architecture
Previous steps
10
In order to fully exploit a future exascale architecture, it is needed to study the mapping and optimization of the codes (and integrated kernels) of interest in terms of New infrastructure
New designed architectures GPUs, Phis, embedded processors
New developments in the underlying software infrastructure
The goal is to optimize the performance but keeping a high degree of portability The ratio flops/watt obtained should be analyzed as well
Co-design
Some trends and current actions
11
Porting to architectures based on symmetric multicore processors with NUMA memory Optimizing the performance
Current main target architectures are Intel and AMD New platforms based on ARM processors are being analyzed
A key point will be load balancing and data placement New scheduling algorithms able to improve locality
Some trends and current actions
12
The management of the MPI level parallelism must be guaranteed for achieving a high scalability of the applications on HPC clusters with millions of cores
Creation of tools for migration of running parallel tasks inside clusters
Hierarchical MPI structures to manage coupled multiphysics problems
Parallel I/O optimization Design of efficient check-pointing strategies Fault tolerance strategies at system and application level
Creation of tools for migration
Some trends and current actions
13
Performance analysis ought to bear in mind all the parallel levels Paraver, Triva, Ocelotl, TAU, etc
Roof-line analyses inside a computational node to understand the bottlenecks of the architectures
At the cluster level, network traffic, I/O traffic, and load balancing to pursue application scalability
Performance prediction tools can be used to analyze the potential benefits of architecture or algorithm modifications DIMEMAS, BOAST, etc.
14
Simulators for Exascale Computations
Main lines of research
15
Numerical schemes for Partial Differential Equations (PDE) Scalable implementations of high order schemes for wave
propagation models Scalable sparse linear solvers
Generic (i.e. algebraic) parallel solvers for large sparse linear systems of equations
Adaptivity Mesh and (local) time-step adaptive algorithms in order to
optimize the use of computational resources. Data management
Numerical schemes
16
High order finite element methods and (standard and mimetic) finite difference schemes should be considered for both time- and frequency-domain
A high level of parallelism is kept if suited to a mixed coarse grain/fine grain (MIMD/SIMD) parallelization targeting many-core heterogeneous systems Time domain: Multiscale Hybrid-Mixed (MHM) methods
combined with Discontinuous Galerkin (DG) or Stabilized Continuous Galerkin (SCG)
Frequency-domain: Hybridized DG formulations for reducing the number of globally coupled degrees of freedom
Scalable Sparse Linear Solvers
17
Both direct and hybrid direct/iterative solvers will be needed
The implementation of algebraic domain decomposition is a must See MaPHyS for example Parallel sparse direct solvers such as PaStiX2 for each sub-
problem can be added too Krylov subspace methods can also be included on top
Adaptivity
18
Mainly applied to PDEs (numerical schemes) via algorithms
They involve adapting the grids in space and time to minimize errors in the simulation The numerical simulation of PDEs can be performed with
arbitrary unstructured discretizations on serial and parallel platforms
Adaptive time stepping controlling strategies
Big Data
19
BD / HPDA and HPC convergence is fully required Vast field to work on
Data Science with stochastic methods can provide excellent approximations
Levering the core calculi with proactive techniques Pre-processing of data to be performed on site Spatial-temporal time series predictive data from numerical
simulations Management of the uncertainty on numerical simulation
data, integrated with a probabilistic database system Scientific workflow management system focused on
managing scientific data flow with provenance data support
20
Some CS Results
Accelerators
21
The ALYA multi-physics code presents speed-ups (w.r.t. CPU code) 36x GPU, 16x OmpSs, 2x Phi with 2 new
BOAST-based kernels
Elastic-acoustic wave propagation kernels 11x Phi, 3x GPU, both with Motifs classes for
characterization
Accelerators
22
Wind model applied to Olan mountain for weather forecast 40x GPU
Performance analysis of memory hierarchy usage of NVIDIA Kepler 2x using either shared or texture memory
Occupancy is twice better with texture memory
Multi-Core performance and energy efficiency
23
Mapping threads and pages according to memory locality (Intense Pages Mapping) Improvements of 39% in performance and 12% in energy eff.
Elastic-acoustic wave propagation kernels ported to Intels many-core architectures Haswell surpassing KNC on acoustic propagation (6.8x
performance and 2.4x energy efficiency Elastic propagation (Sandy Bridge): 6.5x performance
Multi-Core performance and energy efficiency
24
Affinity clause for task directives OpenMP task directive and scheduler extended New OpenMP function for affinity and HW topology queries Performance and scalability improves between 2x and 4x
Paralellism & I/O Management
25
Hybrid MPI/OpenMP performance by increasing overlaping time Use a dedicated thread to improve asynchronous
communications in MPI (not communication at the wait call): 9.7% in computational time
Using low-power (ARM) architectures as file system servers with the Hou10ni app 1 regular server by 2 ARMs doubles bandwidth and decreases
energy consumption by 85% without performance loss Coordinate the access of I/O nodes to data servers for
reducing contention (TWINS scheduler proposal) Read performance of shared files 28% better over other
alternatives and by up to 50% over not forwarding I/O requests
Resilience
26
Checkpointing can be performed at both Application (user-defined) level (FTI) System level (DMTCP)
Slurm workload manager is being extended with third parties plugins to these APIs Resilience is increased New scheduling algorithms can be designed Proactive and reactive action to failures and load balancing
Streams of data efficiently and in real time with data replication (Kafka-Apache)
27
Some Mathematical approaches
Scalable High Order Numerical Schemes
28
Time-domain elastodynamics (Finite Difference Methods) Applied to a 3D elastic wave propagation in the presence of
strong topography FD method based on a staggered grid of Lebedev type or a
fully staggered grid Uses a grid deformation strategy to make a Cartesian grid
conform to a topographic surface Uses a mimetic approach to
accurately deal with the free-surfacecondition
High order FD stencils Better ratio precision/computational
cost than DG & other FDs approaches
Scalable High Order Numerical Schemes
29
Frequency-domain elastodynamics (Hybridizable Discontinuous Galerkin) Unstructured tetrahedral mesh, high or