Scheduling in Cloud
-
Upload
quinn-valenzuela -
Category
Documents
-
view
31 -
download
2
description
Transcript of Scheduling in Cloud
![Page 1: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/1.jpg)
Scheduling in Cloud
Presented by: Abdullah Al MahmudCourse: Cloud Computing(Fall 2012)
![Page 2: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/2.jpg)
Papers
Quincy: Fair Scheduling for Distributed Computing Clusters
Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, Andrew Goldberg @ MSR Silicon Valley
Optimized Resource Allocation & Task Scheduling Challenges in Cloud Computing Environments
Dominique A. Heger, DHTechnologies (DHT)
![Page 3: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/3.jpg)
Quincy: Fair Scheduling for Distributed Computing Clusters
Michael Isard, Vijayan Prabhakaran, Jon Currey, Udi Wieder, Kunal Talwar, and Andrew Goldberg
Modified version of www.sigops.org/sosp/sosp09/slides/quincy/QuincyTestPage.html
![Page 4: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/4.jpg)
Problem Setting
• Homogenous Cluster• Fine grain resource sharing (multiplex all
computers in the cluster between all jobs)• Independent tasks(less costly to kill a task and
restart the task)
![Page 5: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/5.jpg)
Goal of Quincy
• Fair Sharing and Data Locality• N computers, J concurrent jobs-Each job gets at least N/J computers-Place tasks near data to avoid network
bottlenecks-Joint optimization of fairness and data locality
![Page 6: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/6.jpg)
Cluster Architecture
![Page 7: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/7.jpg)
Baseline: Queue Based Scheduler
![Page 8: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/8.jpg)
Baseline: Queue Based Scheduler
• Greedy: Running the first available job in the queue
• Simple Greedy Fairness: Starving a job that submits large number of workers
• Fairness with preemption: Killing workers from a job that already have submitted large number of workers.
![Page 9: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/9.jpg)
Flow Based Scheduler: Quincy
• Construct a graph based on scheduling constraint and cluster architecture
• Finding a matching in the graph is equivalent to finding a feasible schedule.
• Can assign a cost to any matching• Fairness constraints: number of tasks that are
scheduled• Goal: Minimize matching cost while obeying
fairness constraints
![Page 10: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/10.jpg)
Graph Construction• Start with a directed graph representation of the cluster
architecture
![Page 11: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/11.jpg)
Graph Construction (2)
![Page 12: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/12.jpg)
Graph Construction (3)
![Page 13: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/13.jpg)
A Feasible Matching
![Page 14: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/14.jpg)
Final Graph
![Page 15: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/15.jpg)
Result: Makespan when network is bottleneck(s)
![Page 16: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/16.jpg)
Result: Data Transfer (TB)
![Page 17: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/17.jpg)
Conclusion
• New computational model for data intensive computing
• Elegant mapping of scheduling to min-cost flow/matching problem
![Page 18: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/18.jpg)
Optimized Resource Allocation & Task Scheduling Challenges in Cloud
Computing EnvironmentsDominique A. Heger
![Page 19: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/19.jpg)
Resource Allocation in the Cloud
• Each task's resource demand can be described via a multi-dimensional vector such as that the task i requires x processing cores, y GB of memory, and z GB of storage.
• Classical Bin Packing instance(Three Dimensional) which is a well known NP Complete problem
![Page 20: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/20.jpg)
ANN Based Task Scheduling
![Page 21: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/21.jpg)
Conclusion
• This paper discusses some theoretical aspects of Task Scheduling and Resource Allocation
![Page 22: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/22.jpg)
Question?
![Page 23: Scheduling in Cloud](https://reader035.fdocuments.us/reader035/viewer/2022070400/56813532550346895d9c9832/html5/thumbnails/23.jpg)
Thank You