Claude TADONKIMines ParisTech – LAL / CNRS / INP2P3

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance ComputingHigh Performance Computing Challenges and Trends Challenges and Trends

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

The need of competitive HPC systems

Key Factors

Large-scale scientific & technical computing (numerical and non numerical) Large-scale data mining and statistics in experimental physicsImage and signal processingVideo and 3D animations

Molecular biology and structural genomicSorting and pattern matchingMeteorology, Atmospheric studies, Medical researchScientific and technical simulationHigh-standard industrial activities, services, and research investigations

Gaming

And more …

Massively parrallel computers & systemsDedicated architectures Specialized processors Processor frequency High level integrationMemory space, latency, and bandwidth High speed interconnection Advances in parallel algorithm synthesis and programming language featuresPowerful compilers

Increasing Need of HPC (range of applications and computing power)

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

Parallel computing easily justifies nowadays (processor frequency evolution)

HPC interest covers a larger spectra of applications, hence a wider audiencePerformance expectations in some specific areas are beyond the capacity of standard computers

• Frequency has been multiplied by 10 since 1993

• The number of transistors in Intel proc has been multiplied by … 100 000 since 1971

• Core voltage has been reduced by 10 (1,2V , the min is 0,7V)

However, this evolution is closed to its asymptotic threshold !!!

Alternatives and trendsMulti-core processorsTLP-based parallel machinesGPU (assume a skillful use!)

Hardcoded (embedded) solutionsReconfigurable HPCGrid/Meta Computing

Smallpox Research Grid (Oxford University/IBM/United Devices) Oustanding achievment in large-scale molecular analysis.

Observations

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

http://www.fujitsu.com/global/news/pr/archives/month/2011/20110620-02.html

The K Computer RIKEN / Fujitsu ( JAPAN )

Number one the 37th TOP500

8.162 petaflops (Linpack) 93%

68,544 energy-efficient CPUs

672 computer racks

Complete deployment in 2012

10 petaflops expected (2012)

800 computer racks (2012)

Worldwide shared use

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

https://asc.llnl.gov/computing_resources/sequoia/

The Sequoia System LLNL / IBM ( USA )

BlueGene technology

500 teraFLOPS

1.6 million of cores

96 computer racks

98,304 compute nodes

1.6 petabytes of memory

10 times faster than today’sMost powerful system

Sequoia in 1 hour=

6.7 billion people calculating 24h/24h during 320 years

Sequoia in 1 hour=

6.7 billion people calculating 24h/24h during 320 years

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

About sustained performance

Before we calculate, we need data

Time-to-CPU could be long % flops

Memory space decreases with level

Shared use of memory slowdown

Synchronization and data exchange

Control flow (if, while, for, case, …)

Even an optimal algorithm will run at a fraction of the peak performance !!!

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

Important considerations

Energy consumption and dissipation

Integration ( more powerful unit/system in a smaller chip/surface )

Total cost of the system and maintenance issues

Programmability (consider the case of the IBM CELL)

Accessibility (remote and shared use among several entities)

Software and tools (system, programming, monitoring, profiling, …)

Lifetime and evolution of the system (extensibility, devices change/upgrade, …)

Computing nodes interconnection (topology and speed)

vital on embedded systems

significant heat and cause of failure

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

Fundamental aspects

Design of efficient parallel algorithms (modelling & scheduling)

Complexity & Performance analysis

Algorithmic and programming paradigms

Code generation and transformations (automatic parallelization)

Compilation techniques

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI

Programming models

Distributed memory parallel programming (MPI)

Shared memory parallel programming (OpenMP, Pthreads)

Accelerator-based programming (GPU, FPGA, CELL, …)

Vector programming (SSE, VMX, SPU intrinsics, …)

Hybrid programming (MPI+OpenMP/Pthread, CPU+GPU, PPU+SPU, …)

University of Oujda (Morocco) – October 7, 2011 University of Oujda (Morocco) – October 7, 2011

High Performance Computing: Challenges and Trends Claude TADONKIHigh Performance Computing: Challenges and Trends Claude TADONKI