Wireless Home Video Networking

Ph.D. Oral Defense

Distributed Rate Allocation for Video Streaming over Wireless Networks

Tuesday, Tuesday, June 10, 2008June 10, 2008

Information Systems LaboratoryInformation Systems Laboratory

Stanford UniversityStanford University

Xiaoqing ZhuXiaoqing Zhu

X. Zhu: Distributed Rate Allocation for Video over Wireless 2

Wireless Home Video Networking

54 Mbps

6 Mbps

12 Mbps

12 Mbps

HDTV

Mediagateway

SDTV

Microwave

Cordless phone


What’s Wrong with TCP?

Link Speed: 54 Mbps

Throughput : 20 Mbps

Shared : 1.2 Mbps(~ 6% Channel Time)

Link Speed: 2 Mbps

Throughput : 1.4 Mbps

Shared : 1.2 Mbps(~ 85% Channel Time)

Video Source @ 3 Mbps

File Transfer Size: 3.7MB


Outline

• Review– Network and media heterogeneity

– Related work

• Media-aware distributed rate allocation– System model and optimization

– Practical protocol design

– Performance evaluation

• Extension to wireless video multicast– Rate adaptation via scalable video coding (SVC)

– Comparison with TFRC-based heuristics


Network Heterogeneity

Channel Time

…


TCP Throughput over Wireless

10 20 30 40 500

5

10

15

20

Nominal Speed of Second Link (Mbps)

Thr

ough

put (

Mbp

s)

54Mbps

Stream 2

Stream 1

6 - 54 Mbps

Simulation in NS2, for 802.11a network

Stream 1, alone

Stream 2, alone

Stream 1, shared

Stream 2, shared

[Heusse et al. 2003]

12

3 4


Flow Starvation with TCP

1

2

3

5

8

4 6

97

Stream 1

Stream 2

Stream 3

[Xu et al. 2003]

TCP Throughput

0

1

2

3

4

5

6

7

8

Stream 1 Stream 2 Stream 3R

ate

(M

bp

s)


Media Heterogeneity

FountainHD

CitySD

BusCIF

0 5 10 15 2020

25

30

35

40

45

Rate (Mbps)

PS

NR

(dB

)

• Different video has different utility of rate, depending on resolution, codec choice, and video content

• How to best trade off their quality?


Subjective Evaluation

Control Panel

HD/SD Display

Subject

• Evaluation of quality combinations of HD/SD image pairs

• Mean-opinion-scores (MOS) from 28 viewers, 4 data sets


MOS of Allocation Results

2

C1: 20 Mbps1

3

4

C2: 1 - 20 Mbps o: subjectivex: weighted MSE+ : MSE∗: TFRC

IncreasingC2

HD stream

SD stream


MOS of Allocation Results

2

C1: 20 Mbps1

3

4

C2: 1 - 20 Mbps

HD stream

SD stream

Media-aware allocation improves viewing experience


• Rate allocation for wired network– TCP congestion control

– TCP-Friendly Rate Control (TFRC)

– Pricing-based mathematical framework

• Video streaming over wireless networks – Flow control via multiple TFRC connections

– Media-aware centralized channel time allocation

[Kelly 1997-98]

[V. Jacobson 1988][Floyd, Fall 1999]

Related Work

[Kalman, van Beek, Girod 2005]

[Chiang 2007]

[Chen, Zakhor, 2004-06]


Outline


– Related work







Wireless Network Model

• Link utilization:

• Utilization over interference set:

Stream s

Ll

Interference set


Video Distortion Model

[Stuhlmüller et al. 2000]

• Dependent on video content and encoder settings

• Parameters updated for every GOP (~0.5 s)

Scene cuts

Rate

Rate R

Dis

tort

ion

D


Optimization Objective

weight of stream

importance

upper limit

distortion of stream

total utilization

• Motivated by results from subjective viewing test

• Convex objective function with linear constraints


Distributed Solution

Accumulated congestion price:

Residual utilization

Rate R

• Congestion price update:

• Video rate update: Dis

tort

ion

D


Protocol Design

Video packet Acknowledgment Link state message

Advertise video rate

Exchange link state with neighbors:

Report accumulatedcongestion price

Update congestion price:


Overhead vs. Convergence Time

2

1

5

6

43

Crew

54Mbps

54Mbps

54Mbps


Adaptation to Stream Arrival/Departure

12

3 4

0 5 10 15 20 25 30 35 400

20

40Fountain vs. Crew

Cap

acit

y (M

bps)

0 5 10 15 20 25 30 35 400123

Pri

ce

Λ

0 5 10 15 20 25 30 35 400

102030

All

ocat

ed

Rat

e (M

bps)

0 5 10 15 20 25 30 35 4030354045

Time (s)

PS

NR

(d

B)

Fountain Crew

Convergence time: 1 - 2 seconds

Demo


Fountain Crew

Start: 37.8 dB

End: 35.0 dB

Start: 28.3 dB

End: 38.8 dB


Adaptation to Video Scene Cut

12

3 4

0 5 10 15 20 25 30 35 400

20

40Fountain vs. Fountain/Crew

Cap

acit

y (M

bps)

0 5 10 15 20 25 30 35 400

5

Pri

ce

Λ

0 5 10 15 20 25 30 35 400

102030

All

ocat

ed

Rat

e (M

bps)

0 5 10 15 20 25 30 35 4025303540

PS

NR

(d

B)

Time (s)

Fountain Fountain/Crew


Adaptation to Link Speed Drop

12

3 4

0 5 10 15 20 25 30 35 400

20

40Crew vs. Crew

Cap

acit

y (M

bps)

0 5 10 15 20 25 30 35 400

50

Pric

e Λ

0 5 10 15 20 25 30 35 400

102030

All

ocat

ed

Rat

e (M

bps)

0 5 10 15 20 25 30 35 4030354045

PS

NR

(d

B)

Time (s)

Crew, 54Mb Crew, 6Mb


Varying Link Speed: Dijana vs. Raven

Demo

54Mbps

6 - 54 Mbps

12

3 4

Stream 1, media-aware

Stream 2, media-aware

Stream 1, TFRC

Stream 2, TFRC

Improvement of average video quality: 0.7-1.8 dB

X. Zhu: Distributed Rate Allocation for Video over Wireless 25TFRC Allocation

X. Zhu: Distributed Rate Allocation for Video over Wireless 26Media-Aware Allocation


TFRCPSNR: 33.3 dB

Media-AwarePSNR: 36.8 dB


Varying Link Speed: Multi-Hop Network

1

2

3

4

5

6

10 20 30 40 50

30

31

32

33

34

Link Speed (Mbps)

PS

NR

(dB

)

Fountain vs. Cyclists

media-awareTFRC

10 20 30 40 5029

30

31

32

33

34

35

Link Speed (Mbps)

PS

NR

(dB

)

Dijana vs. Raven

media-awareTFRC

10 20 30 40 5034

35

36

37

38

Link Speed (Mbps)

PS

NR

(dB

)

Crew vs. Crew

media-awareTFRC

10 20 30 40 5033

34

35

36

Link Speed (Mbps)

PS

NR

(dB

)

Raven vs. Fountain

media-awareTFRC

1.3 - 2.0 dB 1.9 - 2.5 dB

0.7 - 1.0 dB

0.8 - 0.9 dB

6 - 54 Mbps


Fairness Among Streams

TCP TFRC Media-Aware0

1

2

3

4

5

6

7Fountain

Rat

e (M

bps)

Stream 1Stream 2Stream 3

1

2

3

5

8

4 6

97

Stream 1

Stream 2

Stream 3


Fairness Among Streams

TCP TFRC Media-Aware0

1

2

3

4

5Fountain

Rat

e (M

bps)

Stream 1Stream 2Stream 3Stream 4

1

Stream 1Stream 2

Stream 3Stream 4

2 3 4 5


Protocol Scalability

2 3 4 5 6 7 80

1

2

3

4

5

6

7

Grid Size

Ove

rhea

d (%

)

video ACK LSM update


Outline


– Related work







Video Multicast over Wireless

Need rate adaptation at each peer


LlInterference set

Fl, Cl

Optimization Framework• Objective:

3

2

5

6

8

7

9

10

1

4

11

Stream s

Rsl

• Constraints: – Limit utilization within

each interference set

– Rate of child limited by rate of parent

• Distributed solution:


Scalable Video Coding in H.264

• Each frame contains one BL packet and one EL packet

• Rate adaptation by dropping EL packets

Frames

T0 T0 T0T1 T1 T2T2T2T2

Base Layer (BL)

EnhancementLayer (EL)


0.6 0.8 1 1.2 1.4 1.6 1.828

30

32

34

36

PSN

R (

dB)

4CIF Sequences @ 60fps

Rate (Mbps)

CityCrewSoccerHarbor

Rate Adaptation with H.264/SVC


Allocation over Single Tree: Traces

Convergence time: ~ 2 seconds

1

2

3

4

5

6 8

24 Mbps

54 Mbps


Allocation over Single Tree: PSNR per Peer

1

2

3

4

5

6 7

10 20 30 40 5028

30

32

2

10 20 30 40 5028

30

32

3

10 20 30 40 5028

30

32

6

10 20 30 40 5028

30

32

4

10 20 30 40 5028

30

32

5

10 20 30 40 5028

30

32

7

Link Speed (Mbps)Link Speed (Mbps)

PS

NR

(dB

)P

SN

R (

dB)

PS

NR

(dB

)

media-aware TFRC

Demo

54 Mbps

6 - 54 Mbps


TFRC: 29.3 dB

Video Quality of Peer 2

Media-aware: 31.5 dB


Allocation over Two Trees

10

1

2

3 5

6 8

4

9

7

11

0.3 - 0.6 dB

0.2 - 0.6 dB

0.3 - 0.5 dB

6 - 54 Mbps


Summary of Contributions

• Media- and network-aware optimization framework– Incorporates different video utility functions

– Accommodates heterogeneous wireless link speeds

– Captures impact of traffic contention among neighboring links

• Practical distributed allocation protocol– Cross-layer design: fast convergence via rate advertising

– Approximates subjectively preferred allocation results

– Outperforms media-unaware TFRC

• Extension to video multicast over wireless– Graceful quality adaptation at intermediate nodes via SVC

– Higher average video quality than TFRC-based heuristics


Main Publications

• Overview and tutorial articles – X. Zhu and B. Girod, “Video Streaming Over Wireless Networks”, Invited Tutorial, Proc EUSIPCO’07

– E. Setton, T. Yoo, X. Zhu, A. Goldsmith and B. Girod, “Cross-layer Design of Ad Hoc Networks for Real-Time Video Streaming”, IEEE Wireless Communications Magazine, August 2005, Invited Paper

• Media-aware rate allocation– X. Zhu and B. Girod, "Subjective Evaluation of Multi-User Rate Allocation for Streaming

Heterogeneous Video Contents over Wireless Networks”, Proc. ICIP ‘08, to appear

– X. Zhu, T Schierl, T. Wiegand, and B. Girod, "Video Multicast over Wireless Mesh Networks with Scalable Video Coding (SVC)", Proc. VCIP’08

– X. Zhu, P. Agrawal, J. P. Singh, T. Alpcan, and B. Girod, "Rate Allocation for Multi-User Video Streaming over Heterogenous Access Networks", ACM Multimedia 2007, Best Student Paper Award

– X. Zhu, P. van Beek and B. Girod, "Distributed Channel Time Allocation and Rate Adaptation for Multi-User Video Streaming over Wireless Home Networks“, Proc. ICIP’07

– X. Zhu and B. Girod, "Distributed Rate Allocation for Video Streaming over Wireless Networks with Heterogeneous Link Speeds", Proc. ISMW’07

– X. Zhu, J. P. Singh, and B. Girod, "Joint Routing and Rate Allocation for Multiple Video Streams inAd Hoc Wireless Networks", Proc. PV’06

– X. Zhu and B. Girod, "Media-Aware Multi-User Rate Allocation over Wireless Mesh Networks", Proc. OpComm’06

– X. Zhu and B. Girod, "Distributed Rate Allocation for Multi-Stream Video Transmission over Ad Hoc Networks", Proc. ICIP’05

– X. Zhu, S. Han and B. Girod, "Congestion-Aware Rate Allocation for Multipath Video Streaming over Ad Hoc Wireless Networks", Proc. ICIP’04


Main Publications (cont’d)

• Cross-layer design for video over wireless

– S. Adlakha, X. Zhu, B. Girod and A. Goldsmith, "Joint Capacity, Flow and Rate Allocation for MultiuserVideo Streaming over Wireless Ad-Hoc Networks", Proc. ICC’07

– X. Zhu and B. Girod, "Analysis of Multi-User Congestion Control for Video Streaming over Wireless Networks", Proc. ICME’06

– X. Zhu, E. Setton and B. Girod, Congestion-Distortion Optimized Video Transmission over Ad Hoc Networks, Journal of Signal Processing: Image Communications, September 2005

– E. Setton, X. Zhu and B. Girod, "Congestion-Optimized Scheduling of Video over Wireless Ad Hoc Networks", Proc. ISCAS’05

– X. Zhu, S. Rane and B. Girod, "Systematic Lossy Error Protection (SLEP) for Video Transmission over Wireless Ad Hoc Networks", Proc. VCIP’05

– X. Zhu and B. Girod, "A Distributed Algorithm for Congestion-Minimized Multi-Path Routing over Ad Hoc Networks",Proc. ICME’05

– T. Yoo, E. Setton, X. Zhu, A. Goldsmith and B. Girod, "Cross-Layer Design for Video Streaming over Wireless Ad Hoc Networks", Proc. MMSP’04

– E. Setton, X. Zhu and B. Girod, "Congestion-Optimized Multipath Streaming of Video over Ad Hoc Wireless Networks", Proc. ICME’04

– E. Setton, X. Zhu and B. Girod, "Minimizing Distortion for Multipath Video Streaming over Ad Hoc Networks", Proc. ICIP’04


Other Contributions

• Application to surveillance camera networks

– P. Baccichet, X. Zhu, and B. Girod, "Network-Aware H.264/AVC Region-of-Interest Coding for a Multi-Camera Wireless Surveillance Network", Proc. PCS’06

– X. Zhu, E. Setton and B. Girod, "Content-Adaptive Coding and Delay-Aware Rate Control for A Multi-Camera Wireless Surveillance Network", Proc. MMSP’05

– X. Zhu, E. Setton and B. Girod, "Rate Allocation For Multi-Camera Surveillance over an Ad Hoc Wireless Network", Proc. Picture Coding Symposium, Proc. PCS’04

• Light field compression

– C.-L. Chang, X. Zhu, P. Ramanathan, and B. Girod, "Light Field Compression Using Disparity-Compensated Lifting and Shape Adaptation", IEEE Trans. Image Processing, April 2006

– X. Zhu, A. Aaron and B. Girod, "Distributed Compression for Large Camera Arrays", Proc SSP’03

– C.-L. Chang, X. Zhu, P. Ramanathan and B. Girod, "Inter-View Wavelet Compression of Light Fields With Disparity-Compensated Lifting", Proc. VCIP’03

– C.-L. Chang, X. Zhu, P. Ramanathan and B. Girod, "Shape-Adaptation for Light Field Compression", Proc. ICIP’03

– B. Girod, C.-L. Chang, P. Ramanathan and X. Zhu, "Light Field Compression Using Disparity-Compensated Lifting", Proc. ICASSP’03


Acknowledgments

• Prof. Bernd Girod• Prof. Fouad Tobagi• Dr. Peter van Beek• Prof. Nick Mckeown• IVMS members and alumni• Kelly Yilmaz• Funding: SGF, NSF, Max Plank Center, Sharp

Labs, Huawei/HiSilicon …• Friends• Mom & Dad• Keji

Markus, Sila, Prashant, Yi, Sangeun, Rui, Mark, Chuo-Ling, Eric, Anne, Shantanu, David R, Houda, Pier, Jatinder, Hun, Aditya, David V, Thomas, Yap-Chung, Keiichi, Vijay, Gab, David C, Sam, Zhi, Mina, …

Sachin, Taesang, Wei-Tao, Neda, Carri, Kelin, Jing, Jia, Yifan, Yijie, Chen, Grace, Tao, Yanyan, Ling, Shaohua, …

Thanks to All!

Wireless Home Video Networking

Documents

Transcript of Wireless Home Video Networking