Post on 12-Sep-2021
Ph.D. Oral Defense
Distributed Rate Allocation for Video Streaming over Wireless Networks
Tuesday, Tuesday, June 10, 2008June 10, 2008
Information Systems LaboratoryInformation Systems Laboratory
Stanford UniversityStanford University
Xiaoqing ZhuXiaoqing Zhu
X. Zhu: Distributed Rate Allocation for Video over Wireless 2
Wireless Home Video Networking
54 Mbps
6 Mbps
12 Mbps
12 Mbps
HDTV
Mediagateway
SDTV
Microwave
Cordless phone
X. Zhu: Distributed Rate Allocation for Video over Wireless 3
What’s Wrong with TCP?
Link Speed: 54 Mbps
Throughput : 20 Mbps
Shared : 1.2 Mbps(~ 6% Channel Time)
Link Speed: 2 Mbps
Throughput : 1.4 Mbps
Shared : 1.2 Mbps(~ 85% Channel Time)
Video Source @ 3 Mbps
File Transfer Size: 3.7MB
X. Zhu: Distributed Rate Allocation for Video over Wireless 4
Outline
• Review– Network and media heterogeneity
– Related work
• Media-aware distributed rate allocation– System model and optimization
– Practical protocol design
– Performance evaluation
• Extension to wireless video multicast– Rate adaptation via scalable video coding (SVC)
– Comparison with TFRC-based heuristics
X. Zhu: Distributed Rate Allocation for Video over Wireless 5
Network Heterogeneity
Channel Time
…
X. Zhu: Distributed Rate Allocation for Video over Wireless 6
TCP Throughput over Wireless
10 20 30 40 500
5
10
15
20
Nominal Speed of Second Link (Mbps)
Thr
ough
put (
Mbp
s)
54Mbps
Stream 2
Stream 1
6 - 54 Mbps
Simulation in NS2, for 802.11a network
Stream 1, alone
Stream 2, alone
Stream 1, shared
Stream 2, shared
[Heusse et al. 2003]
12
3 4
X. Zhu: Distributed Rate Allocation for Video over Wireless 7
Flow Starvation with TCP
1
2
3
5
8
4 6
97
Stream 1
Stream 2
Stream 3
[Xu et al. 2003]
TCP Throughput
0
1
2
3
4
5
6
7
8
Stream 1 Stream 2 Stream 3R
ate
(M
bp
s)
X. Zhu: Distributed Rate Allocation for Video over Wireless 8
Media Heterogeneity
FountainHD
CitySD
BusCIF
0 5 10 15 2020
25
30
35
40
45
Rate (Mbps)
PS
NR
(dB
)
• Different video has different utility of rate, depending on resolution, codec choice, and video content
• How to best trade off their quality?
X. Zhu: Distributed Rate Allocation for Video over Wireless 9
Subjective Evaluation
Control Panel
HD/SD Display
Subject
• Evaluation of quality combinations of HD/SD image pairs
• Mean-opinion-scores (MOS) from 28 viewers, 4 data sets
X. Zhu: Distributed Rate Allocation for Video over Wireless 10
MOS of Allocation Results
2
C1: 20 Mbps1
3
4
C2: 1 - 20 Mbps o: subjectivex: weighted MSE+ : MSE∗: TFRC
IncreasingC2
HD stream
SD stream
X. Zhu: Distributed Rate Allocation for Video over Wireless 11
MOS of Allocation Results
2
C1: 20 Mbps1
3
4
C2: 1 - 20 Mbps
HD stream
SD stream
Media-aware allocation improves viewing experience
X. Zhu: Distributed Rate Allocation for Video over Wireless 12
• Rate allocation for wired network– TCP congestion control
– TCP-Friendly Rate Control (TFRC)
– Pricing-based mathematical framework
• Video streaming over wireless networks – Flow control via multiple TFRC connections
– Media-aware centralized channel time allocation
[Kelly 1997-98]
[V. Jacobson 1988][Floyd, Fall 1999]
Related Work
[Kalman, van Beek, Girod 2005]
[Chiang 2007]
[Chen, Zakhor, 2004-06]
X. Zhu: Distributed Rate Allocation for Video over Wireless 13
Outline
• Review– Network and media heterogeneity
– Related work
• Media-aware distributed rate allocation– System model and optimization
– Practical protocol design
– Performance evaluation
• Extension to wireless video multicast– Rate adaptation via scalable video coding (SVC)
– Comparison with TFRC-based heuristics
X. Zhu: Distributed Rate Allocation for Video over Wireless 14
Wireless Network Model
• Link utilization:
• Utilization over interference set:
Stream s
Ll
Interference set
X. Zhu: Distributed Rate Allocation for Video over Wireless 15
Video Distortion Model
[Stuhlmüller et al. 2000]
• Dependent on video content and encoder settings
• Parameters updated for every GOP (~0.5 s)
Scene cuts
Rate
Rate R
Dis
tort
ion
D
X. Zhu: Distributed Rate Allocation for Video over Wireless 16
Optimization Objective
weight of stream
importance
upper limit
distortion of stream
total utilization
• Motivated by results from subjective viewing test
• Convex objective function with linear constraints
X. Zhu: Distributed Rate Allocation for Video over Wireless 17
Distributed Solution
Accumulated congestion price:
Residual utilization
Rate R
• Congestion price update:
• Video rate update: Dis
tort
ion
D
X. Zhu: Distributed Rate Allocation for Video over Wireless 18
Protocol Design
Video packet Acknowledgment Link state message
Advertise video rate
Exchange link state with neighbors:
Report accumulatedcongestion price
Update congestion price:
X. Zhu: Distributed Rate Allocation for Video over Wireless 19
Overhead vs. Convergence Time
2
1
5
6
43
Crew
54Mbps
54Mbps
54Mbps
X. Zhu: Distributed Rate Allocation for Video over Wireless 20
Adaptation to Stream Arrival/Departure
12
3 4
0 5 10 15 20 25 30 35 400
20
40Fountain vs. Crew
Cap
acit
y (M
bps)
0 5 10 15 20 25 30 35 400123
Pri
ce
Λ
0 5 10 15 20 25 30 35 400
102030
All
ocat
ed
Rat
e (M
bps)
0 5 10 15 20 25 30 35 4030354045
Time (s)
PS
NR
(d
B)
Fountain Crew
Convergence time: 1 - 2 seconds
Demo
X. Zhu: Distributed Rate Allocation for Video over Wireless 21
Fountain Crew
Start: 37.8 dB
End: 35.0 dB
Start: 28.3 dB
End: 38.8 dB
X. Zhu: Distributed Rate Allocation for Video over Wireless 22
Adaptation to Video Scene Cut
12
3 4
0 5 10 15 20 25 30 35 400
20
40Fountain vs. Fountain/Crew
Cap
acit
y (M
bps)
0 5 10 15 20 25 30 35 400
5
Pri
ce
Λ
0 5 10 15 20 25 30 35 400
102030
All
ocat
ed
Rat
e (M
bps)
0 5 10 15 20 25 30 35 4025303540
PS
NR
(d
B)
Time (s)
Fountain Fountain/Crew
X. Zhu: Distributed Rate Allocation for Video over Wireless 23
Adaptation to Link Speed Drop
12
3 4
0 5 10 15 20 25 30 35 400
20
40Crew vs. Crew
Cap
acit
y (M
bps)
0 5 10 15 20 25 30 35 400
50
Pric
e Λ
0 5 10 15 20 25 30 35 400
102030
All
ocat
ed
Rat
e (M
bps)
0 5 10 15 20 25 30 35 4030354045
PS
NR
(d
B)
Time (s)
Crew, 54Mb Crew, 6Mb
X. Zhu: Distributed Rate Allocation for Video over Wireless 24
Varying Link Speed: Dijana vs. Raven
Demo
54Mbps
6 - 54 Mbps
12
3 4
Stream 1, media-aware
Stream 2, media-aware
Stream 1, TFRC
Stream 2, TFRC
Improvement of average video quality: 0.7-1.8 dB
X. Zhu: Distributed Rate Allocation for Video over Wireless 25TFRC Allocation
X. Zhu: Distributed Rate Allocation for Video over Wireless 26Media-Aware Allocation
X. Zhu: Distributed Rate Allocation for Video over Wireless 27
TFRCPSNR: 33.3 dB
Media-AwarePSNR: 36.8 dB
X. Zhu: Distributed Rate Allocation for Video over Wireless 28
Varying Link Speed: Multi-Hop Network
1
2
3
4
5
6
10 20 30 40 50
30
31
32
33
34
Link Speed (Mbps)
PS
NR
(dB
)
Fountain vs. Cyclists
media-awareTFRC
10 20 30 40 5029
30
31
32
33
34
35
Link Speed (Mbps)
PS
NR
(dB
)
Dijana vs. Raven
media-awareTFRC
10 20 30 40 5034
35
36
37
38
Link Speed (Mbps)
PS
NR
(dB
)
Crew vs. Crew
media-awareTFRC
10 20 30 40 5033
34
35
36
Link Speed (Mbps)
PS
NR
(dB
)
Raven vs. Fountain
media-awareTFRC
1.3 - 2.0 dB 1.9 - 2.5 dB
0.7 - 1.0 dB
0.8 - 0.9 dB
6 - 54 Mbps
X. Zhu: Distributed Rate Allocation for Video over Wireless 29
Fairness Among Streams
TCP TFRC Media-Aware0
1
2
3
4
5
6
7Fountain
Rat
e (M
bps)
Stream 1Stream 2Stream 3
1
2
3
5
8
4 6
97
Stream 1
Stream 2
Stream 3
X. Zhu: Distributed Rate Allocation for Video over Wireless 30
Fairness Among Streams
TCP TFRC Media-Aware0
1
2
3
4
5Fountain
Rat
e (M
bps)
Stream 1Stream 2Stream 3Stream 4
1
Stream 1Stream 2
Stream 3Stream 4
2 3 4 5
X. Zhu: Distributed Rate Allocation for Video over Wireless 31
Protocol Scalability
2 3 4 5 6 7 80
1
2
3
4
5
6
7
Grid Size
Ove
rhea
d (%
)
video ACK LSM update
X. Zhu: Distributed Rate Allocation for Video over Wireless 32
Outline
• Review– Network and media heterogeneity
– Related work
• Media-aware distributed rate allocation– System model and optimization
– Practical protocol design
– Performance evaluation
• Extension to wireless video multicast– Rate adaptation via scalable video coding (SVC)
– Comparison with TFRC-based heuristics
X. Zhu: Distributed Rate Allocation for Video over Wireless 33
Video Multicast over Wireless
Need rate adaptation at each peer
X. Zhu: Distributed Rate Allocation for Video over Wireless 34
LlInterference set
Fl, Cl
Optimization Framework• Objective:
3
2
5
6
8
7
9
10
1
4
11
Stream s
Rsl
• Constraints: – Limit utilization within
each interference set
– Rate of child limited by rate of parent
• Distributed solution:
X. Zhu: Distributed Rate Allocation for Video over Wireless 35
Scalable Video Coding in H.264
• Each frame contains one BL packet and one EL packet
• Rate adaptation by dropping EL packets
Frames
T0 T0 T0T1 T1 T2T2T2T2
Base Layer (BL)
EnhancementLayer (EL)
X. Zhu: Distributed Rate Allocation for Video over Wireless 36
0.6 0.8 1 1.2 1.4 1.6 1.828
30
32
34
36
PSN
R (
dB)
4CIF Sequences @ 60fps
Rate (Mbps)
CityCrewSoccerHarbor
Rate Adaptation with H.264/SVC
X. Zhu: Distributed Rate Allocation for Video over Wireless 37
Allocation over Single Tree: Traces
Convergence time: ~ 2 seconds
1
2
3
4
5
6 8
24 Mbps
54 Mbps
X. Zhu: Distributed Rate Allocation for Video over Wireless 38
Allocation over Single Tree: PSNR per Peer
1
2
3
4
5
6 7
10 20 30 40 5028
30
32
2
10 20 30 40 5028
30
32
3
10 20 30 40 5028
30
32
6
10 20 30 40 5028
30
32
4
10 20 30 40 5028
30
32
5
10 20 30 40 5028
30
32
7
Link Speed (Mbps)Link Speed (Mbps)
PS
NR
(dB
)P
SN
R (
dB)
PS
NR
(dB
)
media-aware TFRC
Demo
54 Mbps
6 - 54 Mbps
X. Zhu: Distributed Rate Allocation for Video over Wireless 39
TFRC: 29.3 dB
Video Quality of Peer 2
Media-aware: 31.5 dB
X. Zhu: Distributed Rate Allocation for Video over Wireless 40
Allocation over Two Trees
10
1
2
3 5
6 8
4
9
7
11
0.3 - 0.6 dB
0.2 - 0.6 dB
0.3 - 0.5 dB
6 - 54 Mbps
X. Zhu: Distributed Rate Allocation for Video over Wireless 41
Summary of Contributions
• Media- and network-aware optimization framework– Incorporates different video utility functions
– Accommodates heterogeneous wireless link speeds
– Captures impact of traffic contention among neighboring links
• Practical distributed allocation protocol– Cross-layer design: fast convergence via rate advertising
– Approximates subjectively preferred allocation results
– Outperforms media-unaware TFRC
• Extension to video multicast over wireless– Graceful quality adaptation at intermediate nodes via SVC
– Higher average video quality than TFRC-based heuristics
X. Zhu: Distributed Rate Allocation for Video over Wireless 42
Main Publications
• Overview and tutorial articles – X. Zhu and B. Girod, “Video Streaming Over Wireless Networks”, Invited Tutorial, Proc EUSIPCO’07
– E. Setton, T. Yoo, X. Zhu, A. Goldsmith and B. Girod, “Cross-layer Design of Ad Hoc Networks for Real-Time Video Streaming”, IEEE Wireless Communications Magazine, August 2005, Invited Paper
• Media-aware rate allocation– X. Zhu and B. Girod, "Subjective Evaluation of Multi-User Rate Allocation for Streaming
Heterogeneous Video Contents over Wireless Networks”, Proc. ICIP ‘08, to appear
– X. Zhu, T Schierl, T. Wiegand, and B. Girod, "Video Multicast over Wireless Mesh Networks with Scalable Video Coding (SVC)", Proc. VCIP’08
– X. Zhu, P. Agrawal, J. P. Singh, T. Alpcan, and B. Girod, "Rate Allocation for Multi-User Video Streaming over Heterogenous Access Networks", ACM Multimedia 2007, Best Student Paper Award
– X. Zhu, P. van Beek and B. Girod, "Distributed Channel Time Allocation and Rate Adaptation for Multi-User Video Streaming over Wireless Home Networks“, Proc. ICIP’07
– X. Zhu and B. Girod, "Distributed Rate Allocation for Video Streaming over Wireless Networks with Heterogeneous Link Speeds", Proc. ISMW’07
– X. Zhu, J. P. Singh, and B. Girod, "Joint Routing and Rate Allocation for Multiple Video Streams inAd Hoc Wireless Networks", Proc. PV’06
– X. Zhu and B. Girod, "Media-Aware Multi-User Rate Allocation over Wireless Mesh Networks", Proc. OpComm’06
– X. Zhu and B. Girod, "Distributed Rate Allocation for Multi-Stream Video Transmission over Ad Hoc Networks", Proc. ICIP’05
– X. Zhu, S. Han and B. Girod, "Congestion-Aware Rate Allocation for Multipath Video Streaming over Ad Hoc Wireless Networks", Proc. ICIP’04
X. Zhu: Distributed Rate Allocation for Video over Wireless 43
Main Publications (cont’d)
• Cross-layer design for video over wireless
– S. Adlakha, X. Zhu, B. Girod and A. Goldsmith, "Joint Capacity, Flow and Rate Allocation for MultiuserVideo Streaming over Wireless Ad-Hoc Networks", Proc. ICC’07
– X. Zhu and B. Girod, "Analysis of Multi-User Congestion Control for Video Streaming over Wireless Networks", Proc. ICME’06
– X. Zhu, E. Setton and B. Girod, Congestion-Distortion Optimized Video Transmission over Ad Hoc Networks, Journal of Signal Processing: Image Communications, September 2005
– E. Setton, X. Zhu and B. Girod, "Congestion-Optimized Scheduling of Video over Wireless Ad Hoc Networks", Proc. ISCAS’05
– X. Zhu, S. Rane and B. Girod, "Systematic Lossy Error Protection (SLEP) for Video Transmission over Wireless Ad Hoc Networks", Proc. VCIP’05
– X. Zhu and B. Girod, "A Distributed Algorithm for Congestion-Minimized Multi-Path Routing over Ad Hoc Networks",Proc. ICME’05
– T. Yoo, E. Setton, X. Zhu, A. Goldsmith and B. Girod, "Cross-Layer Design for Video Streaming over Wireless Ad Hoc Networks", Proc. MMSP’04
– E. Setton, X. Zhu and B. Girod, "Congestion-Optimized Multipath Streaming of Video over Ad Hoc Wireless Networks", Proc. ICME’04
– E. Setton, X. Zhu and B. Girod, "Minimizing Distortion for Multipath Video Streaming over Ad Hoc Networks", Proc. ICIP’04
X. Zhu: Distributed Rate Allocation for Video over Wireless 44
Other Contributions
• Application to surveillance camera networks
– P. Baccichet, X. Zhu, and B. Girod, "Network-Aware H.264/AVC Region-of-Interest Coding for a Multi-Camera Wireless Surveillance Network", Proc. PCS’06
– X. Zhu, E. Setton and B. Girod, "Content-Adaptive Coding and Delay-Aware Rate Control for A Multi-Camera Wireless Surveillance Network", Proc. MMSP’05
– X. Zhu, E. Setton and B. Girod, "Rate Allocation For Multi-Camera Surveillance over an Ad Hoc Wireless Network", Proc. Picture Coding Symposium, Proc. PCS’04
• Light field compression
– C.-L. Chang, X. Zhu, P. Ramanathan, and B. Girod, "Light Field Compression Using Disparity-Compensated Lifting and Shape Adaptation", IEEE Trans. Image Processing, April 2006
– X. Zhu, A. Aaron and B. Girod, "Distributed Compression for Large Camera Arrays", Proc SSP’03
– C.-L. Chang, X. Zhu, P. Ramanathan and B. Girod, "Inter-View Wavelet Compression of Light Fields With Disparity-Compensated Lifting", Proc. VCIP’03
– C.-L. Chang, X. Zhu, P. Ramanathan and B. Girod, "Shape-Adaptation for Light Field Compression", Proc. ICIP’03
– B. Girod, C.-L. Chang, P. Ramanathan and X. Zhu, "Light Field Compression Using Disparity-Compensated Lifting", Proc. ICASSP’03
X. Zhu: Distributed Rate Allocation for Video over Wireless 45
Acknowledgments
• Prof. Bernd Girod• Prof. Fouad Tobagi• Dr. Peter van Beek• Prof. Nick Mckeown• IVMS members and alumni• Kelly Yilmaz• Funding: SGF, NSF, Max Plank Center, Sharp
Labs, Huawei/HiSilicon …• Friends• Mom & Dad• Keji
Markus, Sila, Prashant, Yi, Sangeun, Rui, Mark, Chuo-Ling, Eric, Anne, Shantanu, David R, Houda, Pier, Jatinder, Hun, Aditya, David V, Thomas, Yap-Chung, Keiichi, Vijay, Gab, David C, Sam, Zhi, Mina, …
Sachin, Taesang, Wei-Tao, Neda, Carri, Kelin, Jing, Jia, Yifan, Yijie, Chen, Grace, Tao, Yanyan, Ling, Shaohua, …
Thanks to All!