1

Recap

2

No. of Processors

C.P

.I

1 2 . . . .

Computational Power Improvement

Multiprocessor

Uniprocessor

3

Summing m NumbersExample: summing m numbers

On a sequential computer we have,sum = a[0];for (i=1;i<m;i++) {

sum = sum + a[i];}

Would expect the running time be be roughly proportional to m.

4

Summing m Numbers in Parallel

• What if we have N processors, with each calculating the m/N numbers assigned to it?

• We must add these partial sums together to get the total sum.

5

Summing Multiprocessors

P1

1

P1

3

P1

2

P2

1

P2

3

P2

2

P3

1

P3

3

P3

2j

i

Suppose we have N processors.

The algorithm is as follows:

1. Each processor finds the local sum of its m/N numbers

2. Each processor passes its local sum to another processor in a coordinated way

3. The global sum is finally in processor P11.

6

Shared Memory Multiprocessors

P/C

Cache

NIC

MB

P/C

Cache

NIC

MB

Bus/Custom-Designed Network

Shared Memory

7

MPP (Massively Parallel Processing)Distributed Memory Multiprocessors

P/C

LM

NIC

MB

P/C

LM

NIC

MB

Custom-Designed Network

MB : Memory Bus NIC : Network Interface Circuitry

8

Cluster

Commodity Network (Ethernet, ATM, Myrinet)

MB MB

P/C

M

NIC

P/C

M

Bridge Bridge

LD LD

NIC

IOB IOB

LD : Local Disk IOB : I/O Bus

9

Grid

P/CP/C

SM SMNICLD

Hub/LAN

Internet

IOC

P/CP/C

SM SMNICLD

Hub/LAN IOC

10

TOP500 architectures