Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw,...
Transcript of Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw,...
![Page 1: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/1.jpg)
Dynamic Query Re-planning using QOOP
Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akellaw, Shuchi Chawlaw
1
![Page 2: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/2.jpg)
What is QOOP?
• QOOP is adistributed data analytics systemthat performs well underresource volatilities
• Core Ideas –
• Re-architect the data analytics system stack
• Enable Dynamic Query Re-planning
• Simplify Scheduler
2
![Page 3: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/3.jpg)
Agenda
• Overview• Distributed Data Analytics Systems
• Resource Volatilities
• Overcoming Inefficiency #1• Static Query Planner
• QOOP’s Dynamic QEP Switching
• Overcoming Inefficiency #2• Complex and Opaque Scheduler
• QOOP’s Scheduler Choice
• Implementation
• Evaluation
3
![Page 4: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/4.jpg)
Overview – Distributed Data AnalyticsJob = SQL Query
Resource Cluster
Query Planner
Execution Engine
Scheduler
Evaluate
Query Execution Plans
(QEP)
Resource Requests Resource Allocations
4
![Page 5: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/5.jpg)
Overview – Distributed Data AnalyticsJob = SQL Query
Resource Cluster
Query Planner
Execution Engine
Scheduler
Resource Requests Resource Allocations
5
Choose optimal QEP
![Page 6: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/6.jpg)
Overview – Resource Volatilities
Job = SQL Query
Resource Share
more or less fixed
6
Resource Volatilities
significantly changes over time
![Page 7: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/7.jpg)
Overview – Resource Volatility; Spot MarketSingle Job
Spot MarketResource Cluster
7
bid $
bid $ > market $
bid $ < market $
![Page 8: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/8.jpg)
12
Overview – Resource Volatility; Spot Market
8
• Fixed budget cost-saving bidding strategy in AWS Spot Market
• 20% resource volatile event – 20% change in #machines over time
• 50 such events in a 5-hour span
10
![Page 9: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/9.jpg)
Overview – Resource Volatility; Small ClusterJob1 Job2
Small Shared
Resource Cluster
9
Job1’s
Resource Share
Resource
Contention
Only
Job1
Job2
enters
Job2
exits
![Page 10: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/10.jpg)
Overview – Resource Volatility; Small Cluster
10
• TPC-DS online workload + Carbyne (OSDI’16) scheduler managing 600 cores
• 38% queries experience at least one 20% resource volatility event
![Page 11: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/11.jpg)
Motivating QOOPJob = SQL Query
Resource Cluster
Query Planner
Execution Engine
Scheduler
Resource Volatilities
How well do
Distributed Data Analytics Systems
perform under Resource Volatilities?
11
![Page 12: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/12.jpg)
Motivating QOOPJob = SQL Query
Query Planner
Execution Engine
Scheduler
12
Static
Optimal QEP
is fixed
Complex, Opaque
No
Resource Volatility
Feedback
![Page 13: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/13.jpg)
Simple Scheduler
Design
Motivating QOOPJob = SQL Query
Query Planner
Execution Engine
Scheduler
13
Dynamic
Re-architect
the stack
Optimal QEP
changes greedilyResource Volatility
Feedback
![Page 14: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/14.jpg)
Agenda
• Overview• Distributed Data Analytics Systems
• Resource Volatilities
• Overcoming Inefficiency #1• Static Query Planner
• QOOP’s Dynamic QEP Switching
• Overcoming Inefficiency #2• Complex and Opaque Scheduler
• QOOP’s Scheduler Choice
• Implementation
• Evaluation
14
![Page 15: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/15.jpg)
Static Query Planner; Example
15
A
B
C
D
AB
CD
ABCD
A join B join C join D
A
B
C
D
AC
BD
A
B
C
D
A
B
C
D
AB
ABC
AB
CD
Three alternate Query Execution Plans (QEP’s)
each with different join order
![Page 16: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/16.jpg)
Static Query Planner; Terminology
16
What is a QEP?
Directed Acyclic Graph (DAG)
What is a Task?
TaskResource
Time
Task
Task
Task
Task
Task
Task
Task
Vertex: TaskEdge: Dependency
Work
![Page 17: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/17.jpg)
Static Query Planner; Example
17
Choose an “optimal” QEP
Optimal – reduce query running time
A join B join C join D
![Page 18: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/18.jpg)
•Clarinet (OSDI ‘16) Query Planner
• Estimates network IO, memory, and compute resources just before job execution
• Estimates running time of each QEP by simulating execution
•Chooses QEP with minimum estimated running time
Static Query Planner; Clarinet
19
![Page 19: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/19.jpg)
•Given ‘r’ amount of resources at time t = 0
•Clarinet calculates running time of each QEP
Static Query Planner; Clarinet
20
Time
Resource
rA
B
C
D
AB
CD
ABCD
A
B
C
D
AB
CD
ABCD
A
B
C
D
AB
CD
ABCD
6
Time=
6
![Page 20: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/20.jpg)
•Given ‘r’ amount of resources at time t = 0
•Clarinet calculates running time of each QEP
Static Query Planner; Clarinet
21
Time
Resource
rA
B
C
D
AB
ABC
AB
CD
A
B
C
D
AB
ABC
AB
CD
A
B
C
D
AB ABCAB
CD
Time=
7.5
7.56
Time=
6
![Page 21: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/21.jpg)
•Given ‘r’ amount of resources at time t = 0
•Clarinet calculates running time of each QEP
Static Query Planner; Clarinet
22
Time
Resource
r Time=
7.5
Time=
6
7.56
A
B
C
D
AC
BD
A
B
C
D
A
B
C
D
AC
BD
A
B
C
D
A
B
C
D
AC
BD
A
B
C
D5
Time=
5
![Page 22: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/22.jpg)
•Given ‘r’ amount of resources at time t = 0
•Clarinet calculates running time of each QEP
•Clarinet chooses Blue Plan
•However this choice is static and does not change during job’s lifetime
Static Query Planner
23
Time=
7.5
Time=
6
Time=
5
![Page 23: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/23.jpg)
•What if the amount of resources changes from rto r’ at time t = 3?
Static Query Planner; Bad Outcomes
24
Time
Resource
r
A
B
C
D
AC
BD
A
B
C
D
A
B
C
D
AC
BD
A
B
C
D
A
B
C
D
AC
BD
A
B
C
D
Time= 5Time= ∞
r’
3
Clarinet
Starvation
Sub-optimal time
Unbounded work
![Page 24: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/24.jpg)
•What if at t = 3 we switch to the Green plan
•Overcome starvation
Motivating QOOP’s Dynamic QEP switching
25
Time
Resource
r
A
B
C
D
12
r’
3
CDA
B
C
DAB ABCD
Query Execution Plan switching
can be beneficial
Time= 12
![Page 25: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/25.jpg)
• Static QEP – under adversarial resource volatilities can lead to bad outcomes• Sub-Optimal behavior• Starvation• Unbounded work
•To overcome – QOOP proposes dynamic QEP switching –• Backtracking• Checkpointing • Greedy behavior
QOOP – Dynamic QEP switching
26
![Page 26: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/26.jpg)
• Switch from the Blue QEP to the Green QEP
• Backtracking – sacrifice current work and redo work in prior stages
Dynamic QEP switching; Backtracking
27
backtrack
sacrifice
partial work
A
B
C
D
A
B
C
D
repeat work from
prior stages
![Page 27: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/27.jpg)
• Switch from the Blue QEP to the Green QEP
• Backtracking – sacrifice current work and redo work in prior stages
Dynamic QEP switching; Backtracking
28
backtrack
sacrifice
partial work
A
B
C
D
A
B
C
D
repeat work from
prior stages
Only re-plan future work?
![Page 28: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/28.jpg)
• Switch from the Blue QEP to the Green QEP
• Backtracking – sacrifice current work and redo work in prior stages
Dynamic QEP switching; Backtracking
29
backtrack
sacrifice
partial work
A
B
C
D
A
B
C
D
repeat work from
prior stages
Only re-plan future work?Backtracking essential to avoid
bad outcomes
![Page 29: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/29.jpg)
• Switch from the Blue QEP to the Green QEP
• Backtracking – sacrifice current work and redo work in prior stages
Dynamic QEP switching; Backtracking
30
backtrack
sacrifice
partial work
A
B
C
D
A
B
C
D
repeat work from
prior stages
Backtracking essential to avoid
bad outcomes
What if we keep repeating work
in an unbounded manner?
![Page 30: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/30.jpg)
•Checkpoint and resume from checkpoints to bound work
• Switch to Green QEP resumes from checkpoint
Dynamic QEP switching; Checkpointing
32
A
B
C
D
A
B
C
D
bound repeated
work and resume
from checkpoint
![Page 31: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/31.jpg)
•Checkpoint and resume from checkpoints to bound work
• Switch to Green QEP resumes from checkpoint
Dynamic QEP switching; Checkpointing
33
A
B
C
D
A
B
C
D
Which QEP to switch to?
bound repeated
work and resume
from checkpoint
![Page 32: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/32.jpg)
• Switch to QEP (red) with least running time in current resources
Dynamic QEP switching; Greedy
35
Time
r
A
B
C
D
10
r’
3
CDAB ABCD
A
B
C
D
AB
CD
ABCD
A
B
C
D
AB
ABC A
B
C
D
AB ABCAB
CD
9
![Page 33: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/33.jpg)
• Switch to QEP (red) with least running time in current resources
Dynamic QEP switching; Greedy
36
Time
r
A
B
C
D
10
r’
3
CDAB ABCD
A
B
C
D
AB
CD
ABCD
A
B
C
D
AB
ABC A
B
C
D
AB ABCAB
CD
9
Intuition – Without knowledge of future
resource volatilities,
greedily maximize current progress
![Page 34: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/34.jpg)
• Switch to QEP (red) with least running time in current resources
Dynamic QEP switching; Greedy
37
Time
r
A
B
C
D
10
r’
3
CDAB ABCD
A
B
C
D
AB
CD
ABCD
A
B
C
D
AB
ABC A
B
C
D
AB ABCAB
CD
9
Intuition – Without knowledge of future
resource volatilities,
greedily maximize current progress
Theorem: Greedy QEP switching has
competitive ratio 2
![Page 35: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/35.jpg)
Agenda
• Overview• Distributed Data Analytics Systems
• Resource Volatilities
• Overcoming Inefficiency #1• Static Query Planner
• QOOP’s Dynamic QEP Switching
• Overcoming Inefficiency #2• Complex and Opaque Scheduler
• QOOP’s Scheduler Choice
• Implementation
• Evaluation
38
![Page 36: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/36.jpg)
Complex and Opaque Schedulers
39
Job
Query
Planner
Execution
Engine
SchedulerFairness
Packing
Job Completion Time
Job
Query
Planner
Execution
Engine
• Increasing complexity of schedulers
•Manage multiple objectives –fairness, packing, job completion time
•QEP-dependent heuristics• Task Size – better fit (Tetris)
• Dependencies – critical path (Carbyne)
![Page 37: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/37.jpg)
Complex and Opaque Schedulers
40
Job
Query
Planner
Execution
Engine
SchedulerFairness
Packing
Job Completion Time
Job
Query
Planner
Execution
Engine
• Increasing complexity of schedulers
•Manage multiple objectives –fairness, packing, job completion time
•QEP-dependent heuristics• Task Size – better fit (Tetris)
• Dependencies – critical path (Carbyne)
Opaque – Hard to model job behavior
if an alternate QEP is picked
![Page 38: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/38.jpg)
Complex and Opaque Schedulers
41
Job
Query
Planner
Execution
Engine
SchedulerFairness
Packing
Job Completion Time
Job
Query
Planner
Execution
Engine
• Increasing complexity of schedulers
•Manage multiple objectives –fairness, packing, job completion time
•QEP-dependent heuristics• Task Size – better fit (Tetris)
• Dependencies – critical path (Carbyne)
Opaque – Hard to model job behavior
if an alternate QEP is picked
Obstructs Dynamic QEP switching – requires ability
to estimate alternate QEP’s performance
![Page 39: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/39.jpg)
QOOP’s Scheduler Choice
49
Job
Query
Planner
Execution
Engine
Scheduler
•We go back to a simpleQEP independent scheduler –simple max-min fair scheduler
• Each job gets a fair resource share guarantee
• Enables feedbackabout resource volatilities
• Supports dynamic QEP switching
Resource Share
=
Total Resources
/ # Active
Queries
![Page 40: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/40.jpg)
Resource Volatility
feedback
= change in resource
share
Scheduler
Dynamic
Simple Scheduler
Design
QOOP Overall DesignJob = SQL Query
Query Planner
Execution Engine
50
Re-architect
the stack
Greedy
dynamic QEP switching
Fairness
Packing
Job Completion Time
![Page 41: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/41.jpg)
Agenda
• Overview• Distributed Data Analytics Systems
• Resource Volatilities
• Overcoming Inefficiency #1• Static Query Planner
• QOOP’s Dynamic QEP Switching
• Overcoming Inefficiency #2• Complex and Opaque Scheduler
• QOOP’s Scheduler Choice
• Implementation
• Evaluation
57
![Page 42: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/42.jpg)
QOOP ImplementationJob = SQL Query
Resource Cluster
Query Planner
Execution Engine
Scheduler
New Resource
Share
Hive – Cache multiple
QEP’s and send to Tez
Tez – estimate runtime
of QEP’s and greedy switch
YARN – simple max-
min fair with feedback
58
![Page 43: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/43.jpg)
QOOP Evaluation
• Testbed –• 20 bare-metal servers
• Micro-benchmark Workload –• Single Query under different
spot market resource volatility regimes
• Macro-benchmark Workload –• 200 queries randomly drawn from TPC-DS
• Online arrival of queries following Poisson process
59
Regime Volatility%
Low < 10%
Medium 10% - 20%
High > 20%
![Page 44: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/44.jpg)
QOOP Evaluation – Micro-benchmark
• Factor of Improvement = Running Time with Clarinet / Running Time with QOOP
• Gains increase with increasing resource volatility
• ~10% jobs > 4x gains
• ~35% queries see no improvements –• low complexity queries
• low duration queries
61
![Page 45: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/45.jpg)
QOOP Evaluation – Micro-benchmark
• Increasing complexity i.e. number of joins => higher gains
• More alternative QEP’s => higher likelihood to find a better QEP switch
62
![Page 46: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/46.jpg)
QOOP Evaluation – Micro-benchmark
• Backtracking is beneficial
65
![Page 47: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/47.jpg)
QOOP Evaluation – Micro-benchmark
• Backtracking is beneficial
• 5.7% of all QEP switches involve backtracking• pre-dominantly due to high
resource volatility
• at-most 2 stages deep
66
![Page 48: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/48.jpg)
QOOP Evaluation – Macro-benchmark
• Job Performance
• Carbyne (OSDI’16) + Clarinet (OSDI’16) – two complex solutions put together
• Closest to ideal baseline SJF –even with a simple max-min fair scheduler
69
0
100
200
300
400
500
600
Avg JCT (seconds)
SJF Carbyne+Clarinet QOOP
![Page 49: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/49.jpg)
QOOP Evaluation – Macro-benchmark
70
0
1000
2000
3000
4000
5000
6000
7000
8000
Makespan (seconds)
Tetris Carbyne+Clarinet QOOP
• Cluster Efficiency
• Carbyne (OSDI’16) + Clarinet (OSDI’16) – two complex solutions put together
• Closest to ideal baseline Tetris – even with a simple max-min fair scheduler
![Page 50: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/50.jpg)
QOOP Evaluation – Macro-benchmark
71
0
1000
2000
3000
4000
5000
6000
7000
8000
Makespan (seconds)
Tetris Carbyne+Clarinet QOOP
• Cluster Efficiency
• Carbyne (OSDI’16) + Clarinet (OSDI’16) – two complex solutions put together
• Closest to ideal baseline Tetris – even with a simple max-min fair scheduler
Each job’s greedy behavior is beneficial
![Page 51: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/51.jpg)
QOOP Summary
• Resource volatilities exist in practice
•QOOP is suited for distributed data analytics under resource volatilities• Simple scheduler choice + feedback
• Dynamic QEP switching at the Query Planner
72
Thank you!
Poster #40
Questions?
![Page 52: Dynamic Query Re-planning using QOOP · Dynamic Query Re-planning using QOOP Kshiteej Mahajanw, Mosharaf Chowdhurym, Aditya Akella w, Shuchi Chawla 1](https://reader030.fdocuments.us/reader030/viewer/2022041113/5f1fffed2267f62dce74d17b/html5/thumbnails/52.jpg)
Backup Slide – Prevalence of Small Clusters
#Machine % Users
1 - 99 75%
100-1000 21%
1000+ 4%
73
Reference: Mesosphere Survey, 2016.