Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1...
Transcript of Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1...
![Page 1: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/1.jpg)
MPI Special Challenge 3
Matrices multiplication using MPI
Cloud Computing | Computer Science and Engineering | WS 2017/18
14.12.2017, 10:00 – 13:15
![Page 2: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/2.jpg)
Rodion
Slepnev
Theory and realization of
algorithm
2
Project Team
Minh
Nguyen
Test cases
Karen
Gharslyan
Introduction
and
Coordination
![Page 3: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/3.jpg)
3
Same program for Master and Worker
![Page 4: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/4.jpg)
Row on column
4
Matrix Multiplication (1-3)
![Page 5: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/5.jpg)
5
Matrix Multiplication (2-3)
Rows on column
![Page 6: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/6.jpg)
Rows on matrix
6
Matrix Multiplication (3-3)
![Page 7: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/7.jpg)
7
How tasks are assigned to Workers
• Ideas to Parallel Matrix Multiplication:
• 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B
• A x B with A (a x c) and B (c x b) will have in total a*b tasks
• Simple approach:
• Divide tasks based on number of rows in matrix A
• Rows are divided equally to each workers in cluster
![Page 8: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/8.jpg)
8
How tasks are assigned to Workers
• Implementation
![Page 9: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/9.jpg)
9
Send and Receive MPI functions
![Page 10: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/10.jpg)
10
Test Case
• Matrix A x Matrix B Size matrix A: NRA x 10 Size matrix B: 10 x 10
• Test case varies 3 variables:
• NRA: 10,100,1000,10000,20000,50000,70000,100000
• Number of Nodes: 2,5,7,10,20,30,50,70,100
• MPI Variant: mpich, openmpi
• Execution time and memory consumption of each test case is captured
![Page 11: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/11.jpg)
11
Execution time (log scaled)
![Page 12: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/12.jpg)
12
Execution time (log scaled)
![Page 13: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/13.jpg)
13
Execution time: Small problem size
![Page 14: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/14.jpg)
14
Execution time: Medium problem size
![Page 15: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/15.jpg)
15
Memory consumption
![Page 16: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/16.jpg)
16
Memory consumption
![Page 17: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/17.jpg)
17
Takeaway
• Same code, different MPI library produces different result
•OpenMPI is faster than mpich in general
•mpich is unexpectedly slow for nodes range: 5~10
-> Results not useful
•Optimal number of nodes for case 20000 x 10, 10 x 10
-> 20 nodes (OpenMPI)
![Page 18: Matrices multiplication using MPI · 2020-02-11 · Ideas to Parallel Matrix Multiplication: • 1 single task: Multiplication of 1 row in matrix A to 1 column in matrix B • A x](https://reader033.fdocuments.us/reader033/viewer/2022041900/5e6036cdfc99d2521842dcf2/html5/thumbnails/18.jpg)
18
Thank you for listening !