Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6....
Transcript of Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6....
![Page 1: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/1.jpg)
1 © 2011 The MathWorks, Inc.
Parallel Computing with MATLAB
Tim Mathieu
Sr. Account Manager
Gerardo Hernandez
Application Engineer
Abhishek Gupta
Application Engineer
![Page 2: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/2.jpg)
2
Some Questions to Consider
Do you want to speed up your algorithms?
Do you have datasets too big to fit on your computer?
If so…
Do you have a multi-core or multiprocessor desktop
machine?
Do you have access to a computer cluster?
![Page 3: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/3.jpg)
3
Solving Big Technical Problems
Large data set
Challenges
Long running
Computationally
intensive
Wait
Larger Memory Pool
(e.g. More Machines)
Solutions
Larger Compute Pool
(e.g. More Processors)
Reduce size
of problem
You could…
![Page 4: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/4.jpg)
4
Utilizing Additional Processing Power
Built-in multithreading
– Core MATLAB
– Introduced in R2007a
– Utility for specific matrix operations
– Automatically enabled since R2008a
Parallel computing tools
– Parallel Computing Toolbox
– MATLAB Distributed Computing Server
– Broad utility controlled by the MATLAB user
![Page 5: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/5.jpg)
5
Worker Worker
Worker
Worker
Worker Worker
Worker
Worker TOOLBOXES
BLOCKSETS
Parallel Computing with MATLAB
![Page 6: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/6.jpg)
6
User’s Desktop
Parallel Computing Toolbox
Compute Cluster
MATLAB Distributed Computing Server
MATLAB Workers
Parallel Computing with MATLAB
![Page 7: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/7.jpg)
7
Programming Parallel Applications
Level of control
Minimal
Some
Extensive
Required effort
None
Straightforward
Involved
![Page 8: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/8.jpg)
8
Programming Parallel Applications
Level of control
Minimal
Some
Extensive
Parallel Options
Low-Level
Programming Constructs: (e.g. Jobs/Tasks, MPI-based)
High-Level
Programming Constructs: (e.g. parfor, batch, distributed)
Support built into
Toolboxes
![Page 9: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/9.jpg)
9
Example: Optimizing Tower Placement
Determine location of cell towers
Maximize coverage
Minimize overlap
![Page 10: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/10.jpg)
10
Summary of Example
Enabled built-in support for
Parallel Computing Toolbox
in Optimization Toolbox
Used a pool of MATLAB workers
Optimized in parallel using fmincon
![Page 11: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/11.jpg)
11
Parallel Support in Optimization Toolbox
Functions:
– fmincon
Finds a constrained minimum of a function of several variables
– fminimax
Finds a minimax solution of a function of several variables
– fgoalattain
Solves the multiobjective goal attainment optimization problem
Functions can take finite differences in parallel
in order to speed the estimation of gradients
![Page 12: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/12.jpg)
12
Tools with Built-in Support
Optimization Toolbox
Global Optimization Toolbox
Statistics Toolbox
SystemTest
Simulink Design Optimization
Bioinformatics Toolbox
Model-Based Calibration Toolbox
… http://www.mathworks.com/products/parallel-computing/builtin-parallel-support.html
Worker
Worker
Worker
Worker Worker
Worker
Worker TOOLBOXES
BLOCKSETS
Directly leverage functions in Parallel Computing Toolbox
![Page 13: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/13.jpg)
13
Programming Parallel Applications
Level of control
Minimal
Some
Extensive
Parallel Options
Low-Level
Programming Constructs: (e.g. Jobs/Tasks, MPI-based)
High-Level
Programming Constructs: (e.g. parfor, batch, distributed)
Support built into
Toolboxes
![Page 14: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/14.jpg)
14
Running Independent Tasks or Iterations
Ideal problem for parallel computing
No dependencies or communications between tasks
Examples include parameter sweeps and Monte Carlo
simulations
Time Time
![Page 15: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/15.jpg)
15
Example: Parameter Sweep of ODEs
Solve a 2nd order ODE
Simulate with different
values for b and k
Record peak value for each run
Plot results
0,...2,1,...2,1
5
xkxbxm
0 5 10 15 20 25-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Time (s)
Dis
pla
cem
ent
(x)
m = 5, b = 2, k = 2
m = 5, b = 5, k = 5
0
2
4
6 12
34
5
0.5
1
1.5
2
2.5
Stiffness (k)Damping (b)
Pe
ak D
isp
lace
me
nt (x
)
![Page 16: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/16.jpg)
16
Summary of Example
Mixed task-parallel and serial
code in the same function
Ran loops on a pool of
MATLAB resources
Used Code Analyzer to help in converting existing for-loop
into parfor-loop
0 5 10 15 20 25-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Time (s)
Dis
pla
cem
ent
(x)
m = 5, b = 2, k = 2
m = 5, b = 5, k = 5
0
2
4
6 12
34
5
0.5
1
1.5
2
2.5
Stiffness (k)Damping (b)
Pe
ak D
isp
lace
me
nt (x
)
![Page 17: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/17.jpg)
17
The Mechanics of parfor Loops
Pool of MATLAB Workers
a = zeros(10, 1)
parfor i = 1:10
a(i) = i;
end
a a(i) = i;
a(i) = i;
a(i) = i;
a(i) = i;
Worker
Worker
Worker Worker
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
![Page 18: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/18.jpg)
18
Converting for to parfor
Requirements for parfor loops
– Task independent
– Order independent
Constraints on the loop body
– Cannot “introduce” variables (e.g. eval, load, global, etc.)
– Cannot contain break or return statements
– Cannot contain another parfor loop
![Page 19: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/19.jpg)
19
Advice for Converting for to parfor
Use Code Analyzer to diagnose parfor issues
If your for loop cannot be converted to a parfor, consider wrapping a subset of the body to a function
Read the section in the documentation on classification of variables
http://blogs.mathworks.com/loren/2009/10/02/using-parfor-loops-getting-up-and-running/
![Page 20: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/20.jpg)
24
Core 1
Core 3 Core 4
Core 2
Cache
Performance Gain with More Hardware
Using More Cores (CPUs) Using GPUs
Device Memory
![Page 21: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/21.jpg)
25
What is a Graphics Processing Unit (GPU)
Originally for graphics acceleration, now also
used for scientific calculations
Massively parallel array of integer and
floating point processors
– Typically hundreds of processors per card
– GPU cores complement CPU cores
Dedicated high-speed memory
* Parallel Computing Toolbox requires NVIDIA GPUs with Compute Capability 1.3 or greater, including
NVIDIA Tesla 10-series and 20-series products. See http://www.nvidia.com/object/cuda_gpus.html
for a complete listing
![Page 22: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/22.jpg)
26
Summary of Options for Targeting GPUs
1) Use GPU array interface with MATLAB
built-in functions
2) Execute custom functions on elements of
the GPU array
3) Invoke your CUDA kernels directly from
MATLAB
Ea
se
of
Us
e
Gre
ate
r Co
ntro
l
![Page 23: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/23.jpg)
27
Performance: A\b with Double Precision
![Page 24: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/24.jpg)
29
Parallel Computing enables you to …
Larger Compute Pool Larger Memory Pool
11 26 41
12 27 42
13 28 43
14 29 44
15 30 45
16 31 46
17 32 47
17 33 48
19 34 49
20 35 50
21 36 51
22 37 52
Speed up Computations Work with Large Data
![Page 25: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/25.jpg)
30
Limited Process Memory
32-bit platforms
– Windows 2000 and XP (by default): 2 GB
– Linux/UNIX/MAC system configurable: 3-4 GB
– Windows XP with /3gb boot.ini switch: 3 GB
64-bit platforms
– Linux/UNIX/MAC: 8 TB
– Windows XP Professional x64: 8TB
![Page 26: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/26.jpg)
31
TOOLBOXES
BLOCKSETS
Distributed Array
Lives on the Cluster
Remotely Manipulate Array
from Desktop
11 26 41
12 27 42
13 28 43
14 29 44
15 30 45
16 31 46
17 32 47
17 33 48
19 34 49
20 35 50
21 36 51
22 37 52
Client-side Distributed Arrays
![Page 27: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/27.jpg)
32
Enhanced MATLAB Functions That Operate
on Distributed Arrays
![Page 28: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/28.jpg)
33
spmd blocks
spmd
% single program across workers
end
Mix parallel and serial code in the same function
Run on a pool of MATLAB resources
Single Program runs simultaneously across workers
– Distributed arrays, message-passing
Multiple Data spread across multiple workers
– Data stays on workers
![Page 29: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/29.jpg)
36
Programming Parallel Applications
Level of control
Minimal
Some
Extensive
Parallel Options
Low-Level
Programming Constructs: (e.g. Jobs/Tasks, MPI-based)
High-Level
Programming Constructs: (e.g. parfor, batch, distributed)
Support built into
Toolboxes
![Page 30: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/30.jpg)
37
MPI-Based Functions in
Parallel Computing Toolbox™
Use when a high degree of control over parallel algorithm is required
High-level abstractions of MPI functions
– labSendReceive, labBroadcast, and others
– Send, receive, and broadcast any data type in MATLAB
Automatic bookkeeping
– Setup: communication, ranks, etc.
– Error detection: deadlocks and miscommunications
Pluggable
– Use any MPI implementation that is binary-compatible with MPICH2
![Page 31: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/31.jpg)
38
Example: Image De-Noising
(Large Image Processing)
Use median filtering to reduce “salt & pepper” noise.
Noisy Image Filtered Image
http://hirise.lpl.arizona.edu/
From - NASA/JPL/University of Arizona
![Page 32: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/32.jpg)
39
Noisy Image – too large for a desktop
![Page 33: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/33.jpg)
40
Distribute Data
![Page 34: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/34.jpg)
41
Distribute Data
![Page 35: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/35.jpg)
43
Pass Overlap Data
![Page 36: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/36.jpg)
44
Pass Overlap Data
![Page 37: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/37.jpg)
45
Pass Overlap Data
![Page 38: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/38.jpg)
47
Apply Median Filter
![Page 39: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/39.jpg)
48
Combine as Distributed Data
![Page 40: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/40.jpg)
49
Combine as Distributed Data
![Page 41: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/41.jpg)
54
Scheduling Applications
![Page 42: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/42.jpg)
55
Interactive to Scheduling
Interactive
– Great for prototyping
– Immediate access to MATLAB workers
Scheduling
– Offloads work to other MATLAB workers (local or on a cluster)
– Access to more computing resources for improved performance
– Frees up local MATLAB session
![Page 43: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/43.jpg)
56
Scheduling Work
TOOLBOXES
BLOCKSETS
Scheduler
Work
Result
Worker
Worker
Worker
Worker
![Page 44: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/44.jpg)
58
Example: Schedule Processing
Offload parameter sweep
to local workers
Get peak value results when
processing is complete
Plot results in local MATLAB
0 5 10 15 20 25-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Time (s)
Dis
pla
cem
ent
(x)
m = 5, b = 2, k = 2
m = 5, b = 5, k = 5
0
2
4
6 12
34
5
0.5
1
1.5
2
2.5
Stiffness (k)Damping (b)
Pe
ak D
isp
lace
me
nt (x
)
![Page 45: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/45.jpg)
59
Summary of Example
Used batch for off-loading work
Used matlabpool option to
off-load and run in parallel
Used load to retrieve
worker’s workspace
0 5 10 15 20 25-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Time (s)
Dis
pla
cem
ent
(x)
m = 5, b = 2, k = 2
m = 5, b = 5, k = 5
0
2
4
6 12
34
5
0.5
1
1.5
2
2.5
Stiffness (k)Damping (b)
Pe
ak D
isp
lace
me
nt (x
)
![Page 46: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/46.jpg)
62
Scheduling Workflows
parfor
– Multiple independent iterations
– Easy to combine serial and parallel code
– Workflow
Interactive using matlabpool
Scheduled using batch
jobs/tasks
– Series of independent tasks; not necessarily iterations
– Workflow Always scheduled
![Page 47: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/47.jpg)
63
Scheduling Jobs and Tasks
TOOLBOXES
BLOCKSETS
Scheduler
Job
Results
Worker
Worker
Worker
Worker
Task
Result
Task
Task
Task
Result
Result
Result
![Page 48: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/48.jpg)
64
Example: Scheduling Independent
Simulations
Offload three independent
approaches to solving our
previous ODE example
Retrieve simulated displacement
as a function of time for
each simulation
Plot comparison of results
in local MATLAB 0 5 10 15 20 25
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Time (s)
Dis
pla
cem
ent
(x)
m = 5, b = 2, k = 2
m = 5, b = 5, k = 5
![Page 49: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/49.jpg)
65
Summary of Example
Used findResource to find scheduler
Used createJob and createTask
to set up the problem
Used submit to off-load and
run in parallel
Used getAllOutputArguments
to retrieve all task outputs 0 5 10 15 20 25
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
Time (s)
Dis
pla
cem
ent
(x)
m = 5, b = 2, k = 2
m = 5, b = 5, k = 5
![Page 50: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/50.jpg)
66
Factors to Consider for Scheduling
There is always an overhead to distribution
– Combine small repetitive function calls
Share code and data with workers efficiently
– Set job properties (FileDependencies, PathDependencies)
Minimize I/O
– Enable Workspace option for batch
Capture command window output
– Enable CaptureDiary option for batch
![Page 51: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/51.jpg)
67
MATLAB and Parallel Computing Tools
Industry Libraries
Message Passing Interface (MPI)
Parallel Computing with MATLAB
Built in parallel functionality within specific toolboxes (also requires Parallel Computing Toolbox)
High level parallel functions
Low level parallel functions
Built on industry standard libraries
matlabpool batch parfor
jobs, tasks
ScaLAPACK
Op
tim
iza
tio
n
To
olb
ox
Glo
bal O
pti
miz
ati
on
To
olb
ox
Sys
tem
Te
st
Sim
uli
nk
De
sig
n
Op
tim
iza
tio
n
Bio
info
rma
tic
s
To
olb
ox
Mo
de
l-B
ase
d
Ca
lib
rati
on
To
olb
ox
An
d m
ore
…
![Page 52: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/52.jpg)
68
Parallel Computing on the Desktop
Desktop Computer
Parallel Computing Toolbox
Rapidly develop parallel
applications on local computer
Take full advantage of desktop
power by using CPUs and
GPUs
Separate computer cluster
not required
![Page 53: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/53.jpg)
69
Scale Up to Clusters, Grids and Clouds
Computer Cluster
MATLAB Distributed Computing Server
Scheduler
Desktop Computer
Parallel Computing Toolbox
![Page 54: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/54.jpg)
70
Licensing: MATLAB® Distributed Computing
Server™
One key required per worker:
– Packs of 8, 16, 32, 64, 128, etc.
– Worker is a MATLAB® session,
not a processor
All-product install
– No code generation or deployment
products
Worker
Scheduler Worker
Worker Parallel
Computing
Toolbox
MATLAB
Simulink
Blocksets
Toolboxes
Task
Result
Job
Result
MATLAB Distributed
Computing Server
![Page 55: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/55.jpg)
72
Open API for others
Support for Schedulers
Direct Support
TORQUE
![Page 56: Parallel Computing with MATLABberezinlab.wustl.edu/Portals/0/Documents/Uploads/CHPC/... · 2014. 6. 22. · Parallel Computing with MATLAB . 7 Programming Parallel Applications Level](https://reader035.fdocuments.us/reader035/viewer/2022071302/60a585ddb9021c2b17094402/html5/thumbnails/56.jpg)
73
MathWorks Contact Information
For pricing, licensing, trials and general questions:
Tim Mathieu
Sr. Account Manager
Education Sales Department
Email: [email protected]
Phone: 508.647.7016
Customer Service: [email protected]
508.647.7000
Technical Support: [email protected]
508.647.7000