LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang...
-
Upload
elvin-walton -
Category
Documents
-
view
218 -
download
2
Transcript of LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang...
![Page 1: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/1.jpg)
1
LBVC: Towards Low-bandwidth Video Chats on Smartphones
Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng
College of William and Mary
![Page 2: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/2.jpg)
2
Video Chats on Smartphones
![Page 3: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/3.jpg)
3
Video Chat Apps - Big Data Consumer E.G., Skype
500MB = 1.85 h
![Page 4: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/4.jpg)
4
Our Objective:
Reduce Data Usage of Video Chats on Smartphones
![Page 5: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/5.jpg)
5
Outline Measurements
LBVC Design
LBVC Implementation
LBVC Evaluation
Related Work
Conclusion
![Page 6: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/6.jpg)
6
Reducing Video Frame Rate (1)
Bandwidth vs. Frame Rate • Video Quality vs. Frame Rate
Typical frame rates adopted by video
chat apps
TVM Score – a recent objective metric for mobile video quality [mobicom’12]
(H.264 as codec) (H.264 as codec)
Good News Constraints
![Page 7: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/7.jpg)
7
Reducing Video Frame Rate (2)
Power Consumption vs. Frame Rate (H.264 as codec)
Power reduction is a minor benefit.
![Page 8: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/8.jpg)
8
Research Question
How to reduce frame rate of mobile video chats without introducing obvious video quality degradation?
![Page 9: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/9.jpg)
9
Frame Interpolation to Rescue Video Quality The sender lowers bandwidth usage
through reducing frame rate The receiver interpolates “missing” frames
to rescue video quality
However, large scene changesbetween two input framesresult in strong artifacts in the interpolated frames
![Page 10: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/10.jpg)
10
Context-aware Frame Rate Adaption At small frame rates, severe device
vibrations are the main cause of large scene changes between two consecutive frames
To keep scene changes small between consecutive frames, We use inertial sensors to determine whether a
device vibration is severe or not Video chats are only recorded with a small
frame rate when a device vibration is not severe
![Page 11: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/11.jpg)
11
LBVC Architecture - SenderLBVC – Low-bandwidth Video Chats
New Component
![Page 12: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/12.jpg)
12
LBVC Architecture - Receiver
Delay
Intermediate Frames
New Component
![Page 13: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/13.jpg)
13
LBVC Implementation Implement LBVC as an extension to Linphone
Easy to add new components
a. Modify the MediaStreamer2 library Add filters into the library to implement the cross dissolve
algorithm Use Android API in NDK to collect sensor readings
b. Implement the GUI with Java SDK JNI to pass user input frame rate to the C library
![Page 14: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/14.jpg)
14
Evaluation – Setup Use Shark for Root to capture network
traffic and WireShark to analyze network traffic
Use Monsoon Power Monitor to measure power consumption
Implement a video chat recorder to record video chats and organize a user study to analyze video quality
![Page 15: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/15.jpg)
15
Evaluation – Bandwidth Usage 10 subject pairs, each of them performs
video chats while standing or sitting LBVC achieves bandwidth saving by up to
43%
39% 43% 35% 13%22%
Smaller thresholds at lower frame ratesresult in larger bandwidth usage variance.
![Page 16: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/16.jpg)
16
Evaluation – Power Consumption 10 subject pairs, each of them performs
video chat while standing or sitting Power saving is not a major benefit of
LBVC.
![Page 17: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/17.jpg)
17
Perc
en
tag
e o
f Tim
e a
t U
ser
Sele
cte
d F
ram
e
Rate
(%
)
Consider Different Mobility Cases
Sitting or standing
Sitting in a vehicle
Walking
Highest chance to save bandwidth
Lowest chance to save bandwidth
Car moves fast, but user’s smartphone is relatively stable;increased chances to save bandwidth
User Selected Frame Rate (fps)
![Page 18: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/18.jpg)
18
Evaluation – Objective Video Quality Record video chats and calculate TVM
scores with MATLAB.
LBVC is able to objectively maintain video quality.
![Page 19: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/19.jpg)
19
Evaluation – Subjective Video Quality 21 subject pairs; each pair rates video chats at
different frame rates with LBVC disabled and enabled
LBVC maintains subjective video quality ( >= 4 fps)
At low frame rates, minor interpolation artifacts and frequent frame rate switches still affect subjective video quality
acceptable level
![Page 20: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/20.jpg)
20
Related Work Video streaming bandwidth reduction
Video compression methods, such as H.264 and VP8 Advanced compression methods, such as compress
sensing [Meng, Infocom 2012] and context-aware image compression [Bao et al., Hotmobile 2011]
LBVC reduces bandwidth of streaming video chats through frame rate adaption instead of video compression
Image interpolation Has been widely used in computer graphics, such as
creating animations from sets of images [Kemelmacher-Shlizerman et al., SIGGRAPH 2011]
LBVC exploits image interpolation techniques to rescue the video quality of mobile video chats with decreased frame rate
![Page 21: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/21.jpg)
21
Conclusion LBVC
adapts the video frame rate with respect to the smartphone vibrations at the sender side
interpolates the missing frames at the receiver side
saves bandwidth by up to 35% under typical video chat scenarios as well as maintains acceptable video quality
considering the bandwidth saving, video quality and introduced delay, we recommend 4 fps (35% bandwidth saving).
![Page 22: LBVC: Towards Low-bandwidth Video Chats on Smartphones Xin Qi, Qing Yang, David T. Nguyen, Gang Zhou, Ge Peng College of William and Mary 1.](https://reader035.fdocuments.us/reader035/viewer/2022062515/56649d025503460f949d4dbf/html5/thumbnails/22.jpg)
22
Questions?
Thank you!