CherryPick: Adaptively Unearthing the Best Cloud Configurations...
Transcript of CherryPick: Adaptively Unearthing the Best Cloud Configurations...
![Page 1: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/1.jpg)
O. Alipourfard et al.
CherryPick: Adaptively Unearthing the Best Cloud Configurations for Big Data Analytics
Presented by Dmitry Kazhdan
![Page 2: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/2.jpg)
Overview
•Background
•Prior work
•CherryPick
•Evaluation
•Criticism
•Recent work
•Conclusions
•Questions
![Page 3: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/3.jpg)
Background
![Page 4: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/4.jpg)
Background
Opportunities:
• Cloud computing
• Big data analytics
• Cost savings
![Page 5: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/5.jpg)
Background
Challenges:
• Complex performance model
• Cost model tradeoffs
• Heterogeneous applications
• Limited number of samples (from a large configuration space)
![Page 6: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/6.jpg)
Prior Work
•Ernest
•Coordinate descent
•Exhaustive search
•Random search
![Page 7: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/7.jpg)
CherryPick
![Page 8: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/8.jpg)
CherryPick
• Uses Bayesian Optimisation to build performance models
• Finds optimal/near-optimal configurations in only a few test runs
• Uses the acquisition function to draw samples
![Page 9: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/9.jpg)
CherryPick
Initial:
Modified:
![Page 10: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/10.jpg)
CherryPick Workflow
![Page 11: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/11.jpg)
CherryPick Implementation
•Search Controller
•Cloud Monitor
•Bayesian Optimisation Engine
•Cloud Controller
![Page 12: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/12.jpg)
Evaluation
![Page 13: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/13.jpg)
Evaluation
• Applications: TPC-DS, TPC-H, TeraSort, SparkReg, SparkKm
• 66 cloud configurations
• Objective: reduce cost of execution under runtime constraint
• Compared with:
• Exhaustive search
• Coordinate Descent
• Random Search (with a budget)
• Ernest
![Page 14: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/14.jpg)
Evaluation
• Metric 1: the expense to run a job with the selected configuration
• Metric 2: the expense to run all sampled configurations
• 20 independent runs
• 10th, 50th and 90th percentiles computed
![Page 15: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/15.jpg)
Evaluation
![Page 16: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/16.jpg)
Evaluation
![Page 17: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/17.jpg)
Evaluation
• Investigated parameter tuning
• Investigated performance behaviour
![Page 18: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/18.jpg)
Evaluation
• Handling workload variation
![Page 19: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/19.jpg)
Criticism
![Page 20: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/20.jpg)
Criticism/Discussion
“With 4x cost, random search can find similar
configurations to CherryPick on the median”
![Page 21: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/21.jpg)
Criticism/Discussion
![Page 22: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/22.jpg)
Criticism/Discussion
• 3/4 comparison tasks are easy to beat (nothing to compare with)
• Not using available information efficiently
![Page 23: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/23.jpg)
Recent Work
![Page 24: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/24.jpg)
Recent Work
•PARIS
•Scout
•Arrow
•Micky
![Page 25: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/25.jpg)
Conclusions
![Page 26: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/26.jpg)
Conclusions
• Introduced CherryPick
• Compared to existing systems
• Presented evaluation results
• Criticism
![Page 27: CherryPick: Adaptively Unearthing the Best Cloud Configurations …ey204/teaching/ACS/R244_2018... · 2018-11-21 · CherryPick: Adaptively Unearthing the Best Cloud Configurations](https://reader036.fdocuments.us/reader036/viewer/2022081611/5f03834a7e708231d4096de4/html5/thumbnails/27.jpg)
Questions?