19 Parallel Models

8/17/2019 19 Parallel Models

1/24

1

Parallel Programming Aaron Bloomfield

CS 415Fall 2005


2/24

2

Why Parallel Programming?

• Predict weather

• Predict !read of SA"S

• Predict !ath of h#rricane

• Predict oil lic$ !ro!agation• %odel growth of &io'!lan$ton(fiherie

• Str#ct#ral im#lation

• Predict !ath of foret fire• %odel formation of gala)ie

• Sim#late n#clear e)!loion


3/24

*

Code that can &e !aralleli+ed

do i= 1 to max,

a[i] = b[i] + c[i] * d[i]end do


4/24

4

Parallel Com!#ter

• Programming mode ty!e

, Shared memory , %eage !aing


5/24

5

-itrited %emory Architect#re

• .ach Proceor ha direct acce only to it local memory

• Proceor are connected /ia high'!eed interconnect

• -ata tr#ct#re m#t &e ditrited

•-ata e)change i done /ia e)!licit !roceor'to'!roceorcomm#nication end(recei/e meage

• Programming %odel

, Widely #ed tandard %P

, ther P3% .)!re P4 Chameleon PA"%ACS

P0Communication

Interconnect

...

Memory MemoryMemory

P0 P1 Pn


6/24

6

%eage Paing nterface

%P !ro/ide• Point'to'!oint comm#nication• Collecti/e o!eration

, Barrier ynchroni+ation

, gather(catter o!eration , Broadcat red#ction• -ifferent comm#nication mode

, Synchrono#(aynchrono# , Bloc$ing(non'&loc$ing , B#ffered(#nffered

• Predefined and deri/ed dataty!e• 3irt#al to!ologie• Parallel ( 7%P 28• C(C99 and Fortran &inding

• http://www.mpi-forum.org


7/24

:

Shared %emory Architect#re

• Proceor ha/e direct acce to glo&al memory and (thro#gh or fat witching networ$

• Cache Coherency Protocol g#arantee conitencyof memory and ( accee

• .ach !roceor alo ha it own memory 7cache8

• -ata tr#ct#re are hared in glo&al addre !ace

• Conc#rrent acce to hared memory m#t &e coordinated

• Programming %odel , %#ltithreading 7;hread


8/24

=

!en%P

• !en%P !orta&le hared memory !arallelim

• >igher'le/el AP for writing !orta&le m#ltithreaded

a!!lication

•Pro/ide a et of com!iler directi/e and li&rary ro#tinefor !arallel a!!lication !rogrammer

• AP &inding for Fortran C and C99

http://www.OpenMP.org


9/24


10/24

10

A!!roache

• Parallel Algorithm

• Parallel


11/24

11

Parallel


12/24

12

Fortran for !arallelim

• Fortran 90 ' Array lang#age ;ri!letnotation for array ection !eration andintrinic f#nction !oi&le on array

ection

• High Performance Fortran (HPF) '

Similar to Fortran 0 t incl#de datalayo#t !ecification to hel! the com!ilergenerate efficient code


13/24

1*

%ore !arallel lang#age

• P< ' array'&aed lang#age at W

Com!ile into C code 7highly !orta&le8

• CD ' C e)tended for !arallelim


14/24

14

&Eect'riented

• Conc#rrent Smalltal$

• ;hread in a/a Ada thread li&rarie for

#e in C(C99

, ;hi #e a li&rary of !arallel ro#tine


15/24

15

F#nctional

• G.S


16/24

16

Paralleli+ing Com!iler

A#tomatically tranform a e#ential !rogram into

a !arallel !rogram

1 dentify loo! whoe iteration can &ee)ec#ted in !arallel

2 ften done in tage

I Which loo! can &e r#n in !arallel?

I >ow ho#ld we ditrite the wor$(data?


17/24

1:

-ata -e!endence

Flow dependence ' "AW "ead'After'Write AJtr#eJ de!endence "ead a /al#e after it ha&een written into a /aria&le

Anti-dependence ' WA" Write'After'"eadWrite a new /al#e into a /aria&le after the old/al#e ha &een read

Output dependence ' WAW Write'After'WriteWrite a new /al#e into a /aria&le and then lateron write another /al#e into the ame /aria&le


18/24

1=

.)am!le

1 A K 0L

2 B K AL

* C K A 9 -4 A K 5L


19/24

1

-e!endencie

A !aralleli+ing com!iler m#t identify loo! that donot ha/e de!endence B.;W..G ;."A;GSof the loo!

.)am!le

do I = 1, 1000

A(I) = B(I) + C(I) D(I) = A(I)

end do


20/24

20

.)am!le

Fork one thread or each !roce""or

#ach thread exec$te" the %oo!&

do I = m'%o, m'hi

A(I) = B(I) + C(I)

D(I) = A(I)

end do

ait or a%% thread" to ini"h beore

!roceedin


21/24

21

Another .)am!le

do I = 1, 1000

A(I) = B(I) + C(I)

D(I) = A(I+1)end do


22/24

22

Met Another .)am!le

do I = 1, 1000

A( (I) ) = B(I) + C(I)

D(I) = A( (I) )end do


23/24

2*

Parallel Com!iler

• ;wo concern

• Paralleli+ing code

, Com!iler will mo/e code aro#nd to #nco/er

!arallel o!eration

• -ata locality

, f a !arallel o!eration ha to get data from

another !roceor@ memory that@ &ad


24/24

24

-itrited com!#ting

• ;a$e a &ig ta$ that ha nat#ral !arallelim

• S!lit it #! to may different com!#ter acro a

networ$

• .)am!le S.;N>ome !rime n#m&er

earche Ooogle Com!#te etc

• -itrited com!#ting i a form of !arallel

com!#ting

19 Parallel Models

Documents

Transcript of 19 Parallel Models