HIGH PERFORMANCE

COMPUTING

MPI and C-Language Seminars 2010

Seminar Plan Week 1 – Introduction, Data Types, Control Flow, Pointers Week 2 – Arrays, Structures, Enums, I/O, Memory Week 3 – Compiler Options and Debugging Week 4 – MPI in C and Using the HPSG Cluster

Week 5 – “How to Build a Performance Model”

Week 6-9 – Coursework Troubleshooting(Seminar tutors available in their office)

Performance Models

Performance Models Aim to predict the runtime of an application.

Allow you to predict beyond grid size and unknown hardware.

Gauge the scalability of the algorithm / code.

Help analyse the parallel efficiency of code.

See where bottle necks are. Both hardware and software.

Factors of a Model Computation – Active Processing:

The time spent doing actual work.More processors => less work per processor.Overall computation time should fall with increase in processors.

Communication – Message Passing:Communication between processors.Overall I/O time will increase with processor count.More network contention means slower communication.

Getting the Balance (1 / 2)

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 630

200

400

600

800

1000

1200 Communication Vs Computation

Comm Time

Comp Time

Total Time

Tim

e (S

)

Number Of Processors

Getting the Balance (2 / 2)

Fixed Costs:The work done by all processors.More processors will not reduce this time.

Variable Costs:Portion of work which varies with processor count.Generally based on problem size decomposition.

Timers Lots of different timers:

CPU Time – Actual time spent of CPU. Wall Time – Total time since program start.

Different timers have different overheads.

Try and avoid timing timers.

Recommend C timer – Need to call with 2 double pointers.double cpuStart, wallStart;Timers(&wallStart, &cpuStart);

Building a Model

Where is the Expense? Need to establish what the expensive operations are:

Functions which are called frequently.Functions which take a long time.Work out a percentage break down of total time.

Is it communicational or computational expense?

Is it a fixed cost or a variable cost?

Computational Model (1 / 2)

How will the number of processors effect the amount of work done by each processor.

Will they all the same amount of work? Even decomposition.

Are loops dependent on the problem size?

Need to look at:How long operations take.How many times they are performed.

Computational Model (2 / 2)

A basic model:Time how long each different operation takes.Calculate how many times those operations are used.Add them all up.

Inaccuracy:When using timers consider their overhead.Always more accurate to time the repetition of operations and

divide through.

Note: Communication will show in wall time.

Communication Model (1 / 2)

Many different types of communication: Send and receives. Collective operations. Barriers.

Need to build a model of the network: Can use existing programs: PingPong / SKaMPI Or write your own.

How much data is being sent?

Communication Model (2 / 2)

Communication times are based on packet size. There is an initial cost of a send – Handshake. Then a variable cost – Payload.

Where is the data being sent? Are the source and destination on the same node?

0 512 102415362048256030723584409646085120563261446656716876808192870492160

20406080

100120140160180

Send Time For Message Size

Ethernet

Tim

e (

uS

ec)

Message Size (Bytes)

Bringing it all Together What you need:

Computation benchmark application.Communication benchmark application.Spreadsheet model.

Run benchmarks on cluster and plug data into model.

Make predictions for different processor configurations.