Oboe: Auto-tuning Video ABR Algorithms to Network...
Transcript of Oboe: Auto-tuning Video ABR Algorithms to Network...
![Page 1: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/1.jpg)
Oboe: Auto-tuning Video ABR Algorithms to Network Conditions
Zahaib Akhtar★, Yun Seong Nam★, Ramesh Govindan, Sanjay Rao, Jessica Chen, Ethan Katz-Bassett, Bruno Ribeiro, Jibin Zhan, Hui Zhang
★: Co-primary authors
1
![Page 2: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/2.jpg)
Internet Video Streaming Today
● Internet video is delivered over:○ Heterogeneous networks: WiFi, wired, 3G/4G LTE○ Highly varying or challenging network conditions
2
![Page 3: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/3.jpg)
Internet Video Streaming Today
● Internet video is delivered over:○ Heterogeneous networks: WiFi, wired, 3G/4G LTE○ Highly varying or challenging network conditions
● Quality of experience (QoE) issues are common place
Low quality Rebuffering
3
![Page 4: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/4.jpg)
Internet Video Streaming Today
● Internet video is delivered over:○ Heterogeneous networks: WiFi, wired, 3G/4G LTE○ Highly varying or challenging network conditions
● Quality of experience (QoE) issues are common place
Low quality Rebuffering
Low QoE adversely impacts user engagement and revenue 4
![Page 5: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/5.jpg)
Background: Adaptive Bitrate Streaming
5
Video
Bitra
tes
Time
A video clip is encoded with multiple qualities (bitrates)
![Page 6: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/6.jpg)
Background: Adaptive Bitrate Streaming
6
Video
Bitra
tes
Time
Bitra
tes
Time
Each bitrate is split into chunks
![Page 7: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/7.jpg)
Background: Adaptive Bitrate Streaming
Video Client
Video Server
7
Request nth chunk at bitrate rBi
trate
s
Time
Net
wor
k C
ondi
tions
Time
![Page 8: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/8.jpg)
Background: Adaptive Bitrate Streaming
Video Client
Video Server
8
Request nth chunk at bitrate rBi
trate
s
Time
Net
wor
k C
ondi
tions
Time
![Page 9: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/9.jpg)
Background: Adaptive Bitrate Streaming
Video Client
Video Server
9
Request nth chunk at bitrate rBi
trate
s
Time
Net
wor
k C
ondi
tions
Time
![Page 10: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/10.jpg)
Background: Adaptive Bitrate Streaming
Video Client
Video Server
10
Request nth chunk at bitrate r
Adaptive Bitrate Algorithms(ABR)
Bitra
tes
Time
Net
wor
k C
ondi
tions
Time
![Page 11: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/11.jpg)
Background: Adaptive Bitrate Streaming
11
ABR algorithms
Too aggressive
Too conservative
Low quality
Rebuffering
![Page 12: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/12.jpg)
Background: ABR algorithms
12
Performance of Designed Adaptation based ABRs critically depends on configurable parameters
ABR algorithms
Designed Adaptations(e.g., MPC[2], BOLA[3], HYB[4], BB[5])
Data-Learned Adaptations(e.g., Pensieve[1])
[1] Hongzi Mao, et al., SIGCOMM, 2017.[2] Xiaoqi Yin, et al., SIGCOMM, 2015.[3] Kevin Spiteri, et al., INFOCOM, 2016.[4] An ABR algorithm that’s widely used in industry.[5] Te-Yuan Huang, et al., SIGCOMM, 2014.
![Page 13: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/13.jpg)
Parameters are sensitive to network conditions
13
![Page 14: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/14.jpg)
Parameters are sensitive to network conditions
14
Network Condition A
Network Condition B
Network Condition A
![Page 15: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/15.jpg)
Parameters are sensitive to network conditions
15
Network Condition A
Network Condition B
Network Condition A
Widely deployed ABR algorithm with parameter !
![Page 16: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/16.jpg)
Parameters are sensitive to network conditions
16
![Page 17: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/17.jpg)
Parameters are sensitive to network conditions
17
Parameters of ABRs must be set in a manner sensitive to network conditions
![Page 18: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/18.jpg)
The problem with ABR algorithms
18
ABR algorithms Parameter
MPC Discount factor d
BOLA Parameter !
HYB Safety margin "
BB Reservoir r
ABR algorithms usefixed parameter value
or simple heuristic
![Page 19: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/19.jpg)
The problem with ABR algorithms
19
ABR algorithms Parameter
MPC Discount factor d
BOLA Parameter !
HYB Safety margin "
BB Reservoir r
Do not perform well across all network conditions
ABR algorithms usefixed parameter value
or simple heuristic
![Page 20: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/20.jpg)
Goal of our work
20
Design a system to make ABR algorithms work better over a wide range of network conditions
![Page 21: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/21.jpg)
Key Challenges
How to model network conditions?
21
How to find the best parameter for a given condition?
How to adapt to changes in network conditions?
![Page 22: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/22.jpg)
Contributions
How to model network conditions?
22
How to find the best parameter for a given condition?
How to adapt to changes in network conditions?
● Leverage stationarity of network connections
● Pre-compute offline
● Detect change points online and adjust parameters
![Page 23: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/23.jpg)
Contributions
How to model network conditions?
23
How to find the best parameter for a given condition?
How to adapt to changes in network condition change?
● Leveraging stationarity of network connections
● Pre-computing in offline
● Online change point detection and adjusting ABR algorithms in online
Our system, Oboe improves state-of-art ABRs (MPC, BOLA and HYB) upto 38%
and outperforms Pensieve by 24%
![Page 24: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/24.jpg)
Key Challenges
How to model network conditions?
24
How to find the best parameter for a given condition?
How to adapt to changes in network conditions?
● Oboe Offline Stage
● Oboe Online Stage
![Page 25: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/25.jpg)
Modeling network conditions
TCP connection throughput
can be modeled as a
piecewise stationary[6-10]
sequence of network states
25
stationary segment
stationary segment
[6] Hari Balakrishnan, et. al. SIGMETRICS, 1997[7] James Jobin, et. al. INFOCOM, 2004[8] Dong Lu, et. al. ICDCS, 2005[9] Guillaume Urvoy-Keller. PAM, 2005.[10] Yin Zhang, et al. IM, 2001
![Page 26: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/26.jpg)
Modeling network conditions
26
<!1, "1>
<!2, "2>Network state s = <!s, "s>
where !s is the mean and "s is the
standard deviation of throughput
![Page 27: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/27.jpg)
Modeling network conditions
27
<!1, "1>
<!2, "2>Network state s = <!s, "s>
where !s is the mean and "s is the
standard deviation of throughput
Key idea : Use the best parameter for each network state
![Page 28: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/28.jpg)
Key Challenges
How to model network conditions?
28
How to find the best parameter for each network state?
How to adapt to changes in network state?
● Oboe Offline Stage
● Oboe Online Stage
![Page 29: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/29.jpg)
Finding the best parameter : Oboe Offline Step 1
29
Generate synthetic stationary traces for each network state
<!1, "1>
<!2, "2>
... ...
![Page 30: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/30.jpg)
Finding the best parameter : Oboe Offline Step 2
30
<!1, "1>
<!2, "2>
... ...
Virtual Player ABR with param. #
Explore parameter space for each state and get QoE vectors
![Page 31: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/31.jpg)
Finding the best parameter : Oboe Offline Step 2
31
<!1, "1>
<!2, "2>
... ... ...
Virtual PlayerABR with param. #
Explore parameter space for each state and get QoE vectors
Param #=0.1 #=0.2 #=0.3 ...
QoE ...
Param #=0.1 #=0.2 #=0.3 ...
QoE ...
![Page 32: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/32.jpg)
Finding the best parameter : Oboe Offline Step 2
32
<!1, "1>
<!2, "2>
... ...
Param #=0.1 #=0.2 #=0.3 ...
QoE <3.2, 0%> <3.8, 0%> <4.0, 2%> ...
...
Virtual Player ABR with param. #
Param #=0.1 #=0.2 #=0.3 ...
QoE <1.7, 0%> <2.0, 2%> <3.2, 5%> ...
Explore parameter space for each state and get QoE vectors
![Page 33: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/33.jpg)
Param !=0.1 !=0.2 !=0.3 ...
QoE <3.2, 0%> <3.8, 0%> <4.0, 2%> ...
Param !=0.1 !=0.2 !=0.3 ...
QoE <1.7, 0%> <2.0, 2%> <3.2, 5%> ...
Finding the best parameter : Oboe Offline Step 3
33
<"1, #1>
<"2, #2>
... ... ...
Virtual PlayerABR with param. !
Find the best parameter by vector dominance for each state
Best
Best
Network State
Best Param.
<"1, #1> !=0.2
<"2, #2> !=0.1
... ...
Mapping
![Page 34: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/34.jpg)
Oboe Offline Stage: Design Questions
● Use real or synthetic traces?
● How to quantize network state space?
● How to reduce the cost of parameter space exploration?
● How to decouple ABR algorithms from Virtual Player?
● How to take publisher preferences into account?
34
![Page 35: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/35.jpg)
Key Challenges
How to model network conditions?
35
How to find the best parameter for each network state?
How to adapt to changes in network state?
● Oboe Offline Stage
● Oboe Online Stage
![Page 36: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/36.jpg)
Adapting to network state changes
36
<!2, "2><!1, "1>
<!2, "2>
Online change point detection[11] algorithm identifies
network throughput distribution changes in real time[11] Ryan Prescott Adams and David JC MacKay. Bayesian Online Changepoint Detection. In arXiv:0710.3742v1, 2007
![Page 37: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/37.jpg)
Adapting to network state changes : Online Step 1
37
<!2, "2><!1, "1>
<!2, "2>
Online change point detection[11] algorithm identifies network throughput distribution changes in real time[11] Ryan Prescott Adams and David JC MacKay. Bayesian Online Changepoint Detection. In arXiv:0710.3742v1, 2007
Change detected
![Page 38: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/38.jpg)
Adapting to network state changes : Online Step 2
38
<!2, "2><!1, "1>
<!2, "2>
Find the best parameter from the mapping for a new state
Network State
Best Param.
<!1, "1> #=0.2
<!2, "2> #=0.1
... ...
Mapping
Change detected
![Page 39: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/39.jpg)
Adapting to network state changes : Online Step 3
39
<!2, "2><!1, "1>
<!2, "2>
Reconfigure ABR algorithm parameter
ABR algorithm parameter
#=0.1
Network State
Best Param.
<!1, "1> #=0.2
<!2, "2> #=0.1
... ...
Mapping
#=0.2Change detected
![Page 40: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/40.jpg)
Oboe Online Design Questions
● Why not a simple moving average?
● How many throughput samples to detect changes?
● Are computational overheads acceptable in real time?
40
![Page 41: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/41.jpg)
Evaluation methodology
41
Comparison
QoE Metrics
● Existing ABRs vs. Existing ABRs + Oboe
● Average Bitrate● Rebuffering Ratio● Bitrate change magnitude● QoE-lin (Linear combination of three metrics)
![Page 42: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/42.jpg)
Evaluation methodology
42
TestBed Setup
Dataset
● 600 throughput traces from real users
● Real users used a desktop or a mobile
● Upto 6 Mbps
Chrome Browser Video Client Apache Video Server
Video
- 3 min long
- Encoded with 6 bitrates
Video player
- Dash.js
- Chrome DevTool API
![Page 43: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/43.jpg)
Various Evaluations
43
Method Detail
ABRs performance MPC, BOLA, BB, HYB and Pensieve
Public datasets HSDPA[11] and FCC[12]
Various settings Live setting and different videos
Alternative predictors Ideal predictors on MPC
Publisher specification Different rebuffering tolerance
Pilot Deployment Partial deployment on AWS
[11] Haakon Riiser, et. al. MMSys, 2013
[12] Federal Communications Commission. Raw Data 2016
![Page 44: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/44.jpg)
RobustMPC vs MPC+Oboe
44
Solves an optimization problem to choose bitrates using predicted throughputs and player buffer occupancy
Discount factor d● Reduces a predicted throughput by d to compensate
prediction errors● Simple heuristic based on previous prediction errors
![Page 45: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/45.jpg)
RobustMPC vs MPC+Oboe
● Improves QoE-lin for 71% of sessions
● For 19% of the sessions, more than 20% benefit
Over
45
![Page 46: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/46.jpg)
RobustMPC vs MPC+Oboe
● Improves QoE-lin for 71% of sessions
● For 19% of the sessions, more than 20% benefit
Over
46
71%
![Page 47: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/47.jpg)
RobustMPC vs MPC+Oboe
● Improves QoE-lin for 71% of sessions
● For 19% of the sessions, more than 20% benefit
Over
47
19%
![Page 48: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/48.jpg)
Where does benefit come from over RobustMPC
Similar average bitrates
Reduces the # of sessions with rebuffering from 33% to 5%
Improves the median per chunk change magnitude by 38%48
![Page 49: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/49.jpg)
Where does benefit come from over RobustMPC
Similar average bitrates
Reduces the # of sessions with rebuffering from 33% to 5%
Improves the median per chunk change magnitude by 38%49
5%
33%
38%Similar
![Page 50: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/50.jpg)
MPC+Oboe vs Pensieve
Over
50
● Improves QoE-lin for 80% of sessions
● 24% better in average QoE-lin
80%
![Page 51: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/51.jpg)
Where does benefit come from over Pensieve?
● Hypothesis : Pensieve performs better when it is trained with a constrained throughput range
51
Model Trained Tested Result
Pen-Specialized 0 - 3 Mbps0 - 3 Mbps
Better
Original Pensieve 0 - 6 Mbps
● Pensieve unable to specialize to network conditions● Oboe specializes parameters for every network state
![Page 52: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/52.jpg)
Summary
Oboe is a system to make ABR algorithms work better in a wide range of network conditions
● by auto-tuning parameters to current network state
Oboe is general● Can be applied to many existing ABRs● Improves existing ABRs upto 38% in QoE metrics● Outperforms Pensieve by 24% in average QoE
![Page 53: Oboe: Auto-tuning Video ABR Algorithms to Network Conditionsconferences.sigcomm.org/sigcomm/2018/files/slides/paper... · 2018-08-30 · Oboe is a system to make ABR algorithms work](https://reader033.fdocuments.us/reader033/viewer/2022041611/5e3791dcb97494617a1c07a8/html5/thumbnails/53.jpg)
Live demo tomorrow in the demo session!https://github.com/USC-NSL/Oboe
Thanks!
53