Post on 19-Dec-2015
1
Friday, September 29, 2006
If all you have is a hammer, then everything looks like
a nail.
- Anonymous
2
Domain Decomposition1. Divide data in approx. equal parts
2. Partition the computation
Functional Decomposition1. Divide the computation into disjoint tasks
2. Determine data requirements of these tasks
3
Domain Decomposition1. Divide data in approx. equal parts
2. Partition the computation
Functional Decomposition1. Divide the computation into disjoint tasks
2. Determine data requirements of these tasks If data is disjoint then partition is complete If there is significant overlap consider
domain decomposition.
4
Task dependency graph?
5
Typically maximum degree of concurrency is less than the total number of tasks.
6
Typically maximum degree of concurrency is less than the total number of tasks.
Degree of concurrency depends on shape of task dependency graph.
7
Degree of concurrency depends on shape of task dependency graph.
8
Mapping
Maximize use of concurrency.Task dependencies and interactions are
important in selection of good mapping.Minimize completion time by making
sure that the processes on critical path execute as soon as they are ready.
9
Mapping
Maximize concurrency and minimize interaction among processors Place tasks that are able to execute
independently on different processors to increase concurrency.
Place tasks that communicate frequently on same processor to increase locality.
10
Mapping
Cannot use more than 4 processors. Why?
11
Mapping
Prevent inter-task interaction from becoming inter-process interaction
12
Agglomeration
13
Data partitioning: Block distribution
Higher dimensional distributions may help reduce the amount of shared data that needs to be accessed
14
Data partitioning: Block distribution
Higher dimensional distributions may help reduce the amount of shared data that needs to be accessed.
n2/p +n2 vs.
2n2/√p
15
Sum N numbers
N numbers are distributed among N tasks
Centralized algorithm
16
Sum N numbers
N numbers are distributed among N tasks
Distributing computation
17
Recursive Decomposition
Divide and conquerSet of independent sub-problems
18
Recursive Decomposition
Sum N numbers.
How many steps required?
19
20
21
Hybrid decomposition
Possible to combine different techniquesFinding minimum of a large set of
numbers by purely recursive decomposition is not efficient.
22
Hybrid decomposition
23
Exploratory Decomposition
24
Unfinished tasks can be terminated once solution is found
25
26
Read: Speculative Decomposition
27
Communications
Most parallel problems need to communicate data between different tasks.
28
Embarrassingly Parallel Applications
No communication between tasks.One end of spectrum of parallelization
Examples?
29
Factors involving communication
Machine cycles and resources that could be used for computation are instead used to package and transmit data.
Sending many small messages can cause latency to dominate communication overheads.
Package small messages into a larger message results in increased bandwidth.
30
Factors involving communicationSynchronous communications.
Asynchronous communications.
Interleaving computation with communication.
31
npoints = 10000
circle_count = 0
do j = 1,npoints
generate 2 random numbers between 0 and 1
xcoordinate = random1 ; ycoordinate = random2 ;
if (xcoordinate, ycoordinate) inside circle then circle_count = circle_count + 1
end do
PI = 4.0*circle_count/npoints
32