Patterns for Parallel Programming Markos Chandras Salil Ponde Wenjiang Xu COMP60611: Fundamentals of...
Transcript of Patterns for Parallel Programming Markos Chandras Salil Ponde Wenjiang Xu COMP60611: Fundamentals of...
Patterns for Parallel Programming
Markos Chandras
Salil Ponde
Wenjiang Xu
COMP60611: Fundamentals of Parallel and Distributed Systems (2010)School of Computer Science
University of Manchester
Distributed data moving mechanisms
Reduced synchronization overhead
Fine-grain parallelism
Difficulties in construction and managment of memories for data matching
Load balancing problems in large systems
Input/output data define the task interface
Hidden implementation details
No relationship among tasks is explicitly defined
The scheduler manages the tasks via the interface
Optimized scheduling using data packing,, data reuse
Fault tolerated, task migration
References
The 'data-driven' model
1) Dan I. Moldovan (1993) - ”'Parallel Processing – From Applications to Systems” (page 399)2) V. D. Tran, L. Hluchý, G. T. Nguyen - “Parallel Programming with Data Driven Model” ( Parallel and Distributed Processing, 2000. Proceedings. 8th Euromicro Workshop, Rhodos, Greece, 2000 )
Dealing effectively with large arrays
Array elements should be close to the Processing Element (PE) which processes it
Ways of distribution
(a) One Dimensional
(b) Two Dimesional
(c) Block Cyclic
Problem is defined in terms of global array indices
Program is written in terms of local array indices
Choosing the way of distribution
References
The Distributed Array Pattern
1) Timothy G. Mattson; Beverly A. Sanders; Berna L. Massingill (2004) - ”'Patterns for Parallel Programming”
Semi-independent tasks interact in an irregular fashion.
(a) interaction is determined by the flow of data
(b) the data implies the ordering constrains between the task
(c) direct graph with cycles
Solution: events & tasks
(a) Task: receive event-->process event -->(possibly) send events
(b) Event Flow: need a form of asychronous communication of events
message-passing: from task that generate the event to task that process it
(c) Avoiding deadlock (1) more informational (2) timeouts
(d) One task per PE or Multiple task (need good load balance)
(e) Communication if events should be efficient
References
The Event-Based Coordination Pattern
1) Timothy G. Mattson; Beverly A. Sanders; Berna L. Massingill (2004) - ”'Patterns for Parallel Programming”
List of ReferencesDan I. Moldovan (1993). Parallel Processing – From Applications to Systems. Morgan Kaufmann Publishers Inc. San Francisco, CA, USA
V. D. Tran, L. Hluchý, G. T. Nguyen Parallel Programming with Data Driven Model. Parallel and Distributed Processing, 2000. Proceedings. 8th Euromicro Workshop, Rhodos, Greece, 2000
Timothy G. Mattson, Beverly A. Sanders, Berna L. Massingill (2004). Patterns for Parallel Programming. Addison-Wesley