The HPC Challenge Benchmark

10
The HPC Challenge Benchmark The HPC Challenge Benchmark Jack Dongarra Innovative Computing Laboratory University of Tennessee and Computer Science and Mathematics Division Oak Ridge National Laboratory http://icl.cs.utk.edu/hpcc/

Transcript of The HPC Challenge Benchmark

Page 1: The HPC Challenge Benchmark

1

The HPC Challenge BenchmarkThe HPC Challenge Benchmark

Jack Dongarra

Innovative Computing LaboratoryUniversity of Tennessee

and Computer Science and Mathematics Division

Oak Ridge National Laboratory

http://icl.cs.utk.edu/hpcc/

Page 2: The HPC Challenge Benchmark

2

Phases I Phases I -- IIIIII

02 05 06 07 08 09 1003 04

ProductsMetrics,Benchmarks

AcademiaResearchPlatforms

EarlySoftware

Tools

EarlyPilot

Platforms

Phase IIR&D

3 companies~$50M each

Phase IIIFull Scale Development

commercially ready in the 2007 to 2010 timeframe.$100M ?

Metrics and Benchmarks

System DesignReview

Industry

Application Analysis

PerformanceAssessment

HPCSCapability or

Products

Fiscal Year

Concept Reviews PDR

Research Prototypes

& Pilot Systems

Phase III Readiness Review

TechnologyAssessments

Requirementsand Metrics

Phase IIReadiness Reviews

Phase IIndustry

Concept Study5 companies

$10M each

Reviews

Industry Procurements

Critical Program Milestones

DDR

Productivity TeamIndustry:

Mission partners:

Productivity team (Lincoln Lab lead)

MIT LincolnLab

LCS OhioState

PI: Koester PIs: Vetter, Lusk, Post, BaileyPIs: Gilbert, Edelman,Ahalt, Mitchell

PI: Kepner PI: Lucas PI: BasiliPIs: Benson,Snavely

PI: Dongarra

Page 3: The HPC Challenge Benchmark

3

Motivation for Additional BenchmarksMotivation for Additional Benchmarks

♦ From Linpack Benchmark and Top500: “no single number can reflect overall performance”

♦ Without HPL Linpack only peak will be reported

♦ Clearly need something more than Linpack

♦ HPC Challenge Benchmark

Goals HPC Challenge BenchmarkGoals HPC Challenge Benchmark

♦Stress CPU, memory system, interconnect

♦Allow for optimizations Record effort needed for tuning

♦Provide verification of results♦Archive results♦Requires: MPI and BLAS

Page 4: The HPC Challenge Benchmark

4

HPC Challenge Benchmark HPC Challenge Benchmark Initial Release 11/03Initial Release 11/03

Consists of basically 5 benchmarks; Think of it as a framework or harness for adding benchmarks of interest.

1. HPL (LINPACK) ― MPI on whole system (Ax = b)

2. STREAM ― single CPU *STREAM ― embarrassingly

parallel whole system

3. PTRANS (A A + BT) ― MPI on whole system

4. RandomAccess ― single CPU *RandomAccess ― embarrassingly parallelRandomAccess ― MPI on whole system

5. BW and Latency – MPI

Coming soon: FFT and Matrix Multiply proci prock

Random integerread; update; & write

ApplicationsApplications

Temporal locality

Spat

ial l

ocal

ity

ComputationalFluid

Dynamics

Digital Signal Processing

TravelingSales

Person

Radar Cross Section

high

low

low

Memory Access Patterns

Page 5: The HPC Challenge Benchmark

5

Temporal locality

Spat

ial l

ocal

ity

Digital Signal Processing

STREAM / PTRANSHPL Linpack

FFT (coming soon) RandomAccess

Radar Cross Section

high

low

low

Memory Access Patterns

ApplicationsApplicationsSignaturesSignatures

ComputationalFluid

Dynamics

TravelingSales

Person

low

How Will The Benchmarking Work?How Will The Benchmarking Work?♦ Single program to download and run

Simple input file similar to HPL input♦ Base Run and Optimization Run

Base run must be madeUser supplies MPI and the BLAS

Optimized run allowed to replace certain routines

User specifies what was done♦ Results upload via website♦ html table and Excel spreadsheet

generated with performance resultsIntentionally we are not providing a single figure of merit (no over all ranking)

♦ Goal: no more than 2 X the time to execute HPL.

Page 6: The HPC Challenge Benchmark

6

http://icl.cs.utk.edu/hpcc/http://icl.cs.utk.edu/hpcc/

Coming soon FFT and Matrix Multiply

Go toGo to……

♦http://icl.cs.utk.edu/hpcc/

Page 7: The HPC Challenge Benchmark

7

Example of OutputExample of Output http://icl.cs.utk.edu/hpcc/

Page 8: The HPC Challenge Benchmark

8

Page 9: The HPC Challenge Benchmark

9

http://http://icl.cs.utk.edu/hpccicl.cs.utk.edu/hpcc//

Expanded Set of BenchmarksExpanded Set of Benchmarks

♦Constructing a framework for benchmarks

♦Developing machine signatures♦ Plans are to expand the benchmark collection

♦Currently working onDGEMM and *DGEMMFFT (1d Complex)

Page 10: The HPC Challenge Benchmark

10

Future DirectionsFuture Directions

♦Port to new systems♦Provide more implementations

Languages (Fortran, UPC, Co-Array)Environments Paradigms

♦Other basic operationsSparse matrixI/O

CollaboratorsCollaborators♦Piotr Łuszczek, U of Tennessee♦David Bailey, NERSC/LBL♦Jeremy Kepner, MIT Lincoln Lab♦David Koester, MITRE♦Bob Lucas, ISI/USC♦John McCalpin, IBM, Austin♦Rolf Rabenseifner, HLRS Stuttgart

http://icl.cs.utk.edu/hpcc/


HPC, HIPAA, and FISMA: Meeting the Regulatory Challenge ...

HPC, HIPAA, and FISMA: Meeting the Regulatory Challenge ...

STAR-CCM+ Performance Benchmark and Profiling › pdf › STAR-CCM_Analysis_Intel_E5_2… · – Participating vendors: CD-adapco, Intel, Dell, Mellanox – Compute resource - HPC

STAR-CCM+ Performance Benchmark and Profiling › pdf › STAR-CCM_Analysis_Intel_E5_2… · – Participating vendors: CD-adapco, Intel, Dell, Mellanox – Compute resource - HPC

HPC Benchmarking, Rudi Eigenmann, 2006 SPEC Benchmark Workshop this is important for machine procurements and for understanding where HPC technology is.

HPC Benchmarking, Rudi Eigenmann, 2006 SPEC Benchmark Workshop this is important for machine procurements and for understanding where HPC technology is.

HPC Challenge Benchmark Suite 2006 SPEC Workshop January 23, 2006 Austin, TX Jack Dongarra Piotr Łuszczek

HPC Challenge Benchmark Suite 2006 SPEC Workshop January 23, 2006 Austin, TX Jack Dongarra Piotr Łuszczek

A platform for delivering superior HPC cloud services ...€¦ · Technical white paper Page 3 . Executive summary For enterprises, managing on-premises . HPC. environments is a challenge.

A platform for delivering superior HPC cloud services ...€¦ · Technical white paper Page 3 . Executive summary For enterprises, managing on-premises . HPC. environments is a challenge.

GPU Performance Benchmark - HPC Advisory Council€¦ · 9 NAMD Performance – GPU utilization • Dataset – STMV (virus) benchmark (1,066,628 atoms, periodic, PME) • Using GPUs

GPU Performance Benchmark - HPC Advisory Council€¦ · 9 NAMD Performance – GPU utilization • Dataset – STMV (virus) benchmark (1,066,628 atoms, periodic, PME) • Using GPUs

The HPC Challenge Benchmark: A Candidate for Replacing LINPACK in

The HPC Challenge Benchmark: A Candidate for Replacing LINPACK in

Windows HPC: Launching to expand grid usage · •HPC Profile Working Group (Started 2006) ... HPLinpack 1.0a -- High-Performance Linpack benchmark -- January 20, 2004 Written by

Windows HPC: Launching to expand grid usage · •HPC Profile Working Group (Started 2006) ... HPLinpack 1.0a -- High-Performance Linpack benchmark -- January 20, 2004 Written by

Analyzing Large Sets of Bioinformatic Data with HPC Challenge · Analyzing Large Sets of Bioinformatic Data with HPC ... data requests from the many different researchers working

Analyzing Large Sets of Bioinformatic Data with HPC Challenge · Analyzing Large Sets of Bioinformatic Data with HPC ... data requests from the many different researchers working

The HPEC Challenge Benchmark Suitearchive.ll.mit.edu/HPEC/agendas/proc05/Day_1/Presentations/1550_Haney.pdfMIT Lincoln Laboratory Haney - 4 HPEC 10/20/2005 HPEC Challenge Benchmark

The HPEC Challenge Benchmark Suitearchive.ll.mit.edu/HPEC/agendas/proc05/Day_1/Presentations/1550_Haney.pdfMIT Lincoln Laboratory Haney - 4 HPEC 10/20/2005 HPEC Challenge Benchmark

A Survey of HPC Usage in Europe and the PRACE Benchmark Suite · A Survey of HPC Usage in Europe and the PRACE Benchmark Suite Dr Mark Bull Dr Jon Hill ... NAMD, CPMD, VASP, QCD,

A Survey of HPC Usage in Europe and the PRACE Benchmark Suite · A Survey of HPC Usage in Europe and the PRACE Benchmark Suite Dr Mark Bull Dr Jon Hill ... NAMD, CPMD, VASP, QCD,

Fujitsu and the HPC PyramidROI is often not clear: in-house HPC versus Cloud HPC Learning curve may take time and money Mental challenge: loosing control: hardware, software, data

Fujitsu and the HPC PyramidROI is often not clear: in-house HPC versus Cloud HPC Learning curve may take time and money Mental challenge: loosing control: hardware, software, data

2007/11/2 First French-Japanese PAAP Workshop 1 The FFTE Library and the HPC Challenge (HPCC) Benchmark Suite Daisuke Takahashi Center for Computational.

2007/11/2 First French-Japanese PAAP Workshop 1 The FFTE Library and the HPC Challenge (HPCC) Benchmark Suite Daisuke Takahashi Center for Computational.

ANSYS CFX Performance Benchmark and Profiling - HPC Advisory

ANSYS CFX Performance Benchmark and Profiling - HPC Advisory

STREAM and RA HPC Challenge Benchmarks - Chapel

STREAM and RA HPC Challenge Benchmarks - Chapel

Fujitsu and the HPC Pyramid - Unical · ROI is often not clear: in-house HPC versus Cloud HPC Learning curve may take time and money Mental challenge: loosing control: hardware, software,

Fujitsu and the HPC Pyramid - Unical · ROI is often not clear: in-house HPC versus Cloud HPC Learning curve may take time and money Mental challenge: loosing control: hardware, software,

1 PBB: A Parallel Bioinformatics Benchmark Suite for Shared Memory Multiprocessors CHEN Wenguang HPC Inst., CS Dept., Tsinghua University.

1 PBB: A Parallel Bioinformatics Benchmark Suite for Shared Memory Multiprocessors CHEN Wenguang HPC Inst., CS Dept., Tsinghua University.

VASP Performance Benchmark and Profiling · VASP Performance Benchmark and Profiling ... • System to be redirected to explore HPC in the Cloud applications • Workload profiling

VASP Performance Benchmark and Profiling · VASP Performance Benchmark and Profiling ... • System to be redirected to explore HPC in the Cloud applications • Workload profiling

through Linpack benchmark” - HPC Advisory · PDF file“Understanding Bulldozer architecture through Linpack benchmark ... Round robin scheduling 6clks ... “Understanding Bulldozer

through Linpack benchmark” - HPC Advisory · PDF file“Understanding Bulldozer architecture through Linpack benchmark ... Round robin scheduling 6clks ... “Understanding Bulldozer

Energy Profiling And Analysis Of The HPC Challenge Benchmarks

Energy Profiling And Analysis Of The HPC Challenge Benchmarks