Joint Buffering and Rate Control for Video Streaming over ... · Joint Buffering and Rate Control...
Transcript of Joint Buffering and Rate Control for Video Streaming over ... · Joint Buffering and Rate Control...
Joint Buffering and Rate Control for Video Streaming
over Heterogeneous Wireless Networks
by
Lei Hua
A thesis submitted in conformity with the requirementsfor the degree of Master of Applied Science
Graduate Department of Electrical and Computer EngineeringUniversity of Toronto
Copyright c© 2010 by Lei Hua
Abstract
Joint Buffering and Rate Control for Video Streaming over Heterogeneous Wireless
Networks
Lei Hua
Master of Applied Science
Graduate Department of Electrical and Computer Engineering
University of Toronto
2010
The integration of heterogeneous access networks is becoming a possible feature of 4G
wireless networks. It is challenging to deliver the multimedia services over such integrated
networks because of the discrepancy in the bandwidth of different networks. This thesis
presents an adaptive approach that combines source rate adaptation and buffering to
achieve high quality VBR video streaming with less quality variation over an integrated
two-tier network. Statistical information of the residence time in each network or local-
ization information are utilized to anticipate the handoff occurrence. The performance of
this approach is analyzed under the CBR case using a Markov reward model. Simulation
under the CBR and VBR cases is conducted for different types of network models. The
results are compared with a dynamic programming algorithm as well as other naive or
intuitive algorithms, and proved to be promising.
ii
Acknowledgements
I would like to express my sincerest gratitude to my supervisor, Professor Ben Liang,
for this exciting opportunity to work under his supervision at this prestigious institu-
tion. During the whole process he provided me with invaluable guidance, inspiration and
support, without which I couldn’t have completed this work.
I am thankful to the members of my thesis committee, Prof. Elvino S. Sousa, Prof.
Raviraj Adve, and Prof. Jason H. Anderson for the time spent in reviewing my thesis,
and for their helpful feedback and comments on improving its content.
I thank all my current and former colleagues in my research group for their useful
inputs and suggestions on the research work itself and also the presentation of the work.
Special thanks to all of my friends at University of Toronto, Colin Jiang, Eric Yuan,
Junqi Yu, Lilin Zhang, Weiwei Li, Yuan Feng, Yunfeng Lin and others, whose company,
care and encouragement made the two years of Master’s studies much more enjoyable.
Last but never the least, I dedicate this thesis to my family, who are always there for
me in my life.
iii
Contents
1 Introduction 1
1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 Video Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Heterogeneous Wireless Networks . . . . . . . . . . . . . . . . . . 2
1.1.3 Buffering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.4 Rate Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.5 Contribution of the Thesis . . . . . . . . . . . . . . . . . . . . . . 4
1.2 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2 Literature Review 7
2.1 Video Rate Adaptation Techniques . . . . . . . . . . . . . . . . . . . . . 7
2.1.1 Transcoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.2 Joint Source/Channel Coding . . . . . . . . . . . . . . . . . . . . 8
2.1.3 Scalable Video Coding . . . . . . . . . . . . . . . . . . . . . . . . 9
2.1.4 Content-Aware Coding Techniques . . . . . . . . . . . . . . . . . 10
2.2 Rate Control in Heterogeneous Wireless Networks . . . . . . . . . . . . . 11
2.3 Buffering in Heterogeneous Wireless Networks . . . . . . . . . . . . . . . 12
3 Problem Statement 14
3.1 Application Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.2 Models and Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
iv
3.2.1 Rate Adaptation and Playback . . . . . . . . . . . . . . . . . . . 16
3.2.2 Residence Time and Rate Estimation . . . . . . . . . . . . . . . . 18
3.2.3 Feedback Control Mechanism . . . . . . . . . . . . . . . . . . . . 19
3.3 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
4 Generic Network Model 22
4.1 Control Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
4.1.1 Adaptive Control Algorithm . . . . . . . . . . . . . . . . . . . . . 22
4.1.2 Simple Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1.3 Mean Residual Life Based Algorithm . . . . . . . . . . . . . . . . 26
4.1.4 Simple Shaping Algorithm . . . . . . . . . . . . . . . . . . . . . . 27
4.2 Analytical Framework and Analytical Results . . . . . . . . . . . . . . . 29
4.2.1 Analytical Results for Generic Model . . . . . . . . . . . . . . . . 30
5 Markov Chain Network Model 36
5.1 Markov Decision Process Model . . . . . . . . . . . . . . . . . . . . . . . 36
5.2 Dynamic Programming Algorithm . . . . . . . . . . . . . . . . . . . . . . 38
5.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
5.4 More Realistic 3-Zone Network Model . . . . . . . . . . . . . . . . . . . . 43
5.5 PH-Fitting of Residence Times . . . . . . . . . . . . . . . . . . . . . . . 45
5.6 Estimation in Adaptive Control Algorithm . . . . . . . . . . . . . . . . . 47
5.6.1 Utilizing Statistical Information . . . . . . . . . . . . . . . . . . . 47
5.6.2 Utilizing Localization Information . . . . . . . . . . . . . . . . . . 48
5.6.3 Simulation Results for 3-Zone Model . . . . . . . . . . . . . . . . 49
5.7 Simulating with VBR Network and VBR Video Stream . . . . . . . . . . 52
6 Conclusion 57
Bibliography 59
v
List of Tables
3.1 Notations in system model . . . . . . . . . . . . . . . . . . . . . . . . . . 20
4.1 Analysis parameters - 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
4.2 Analysis parameters - 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
5.1 Simulation parameters for 2-zone Markov model . . . . . . . . . . . . . . 40
5.2 Simulation parameters for 3-zone model . . . . . . . . . . . . . . . . . . . 49
5.3 Simulation parameters for VBR network and VBR video . . . . . . . . . 52
vi
List of Figures
3.1 Integrated two-tier network . . . . . . . . . . . . . . . . . . . . . . . . . 15
3.2 Relationship between the transmission sequence in time and the playback
sequence in time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
4.1 Illustration of proportional feedback controller . . . . . . . . . . . . . . . 24
4.2 Distributions of residence times . . . . . . . . . . . . . . . . . . . . . . . 32
4.3 Analysis vs simulation results: generic model, Gamma distribution - 1 . . 34
4.4 Analysis vs simulation results: generic model, Gamma distribution - 2 . . 35
5.1 DP: variation and utilization vs. α . . . . . . . . . . . . . . . . . . . . . 41
5.2 Adaptive algorithm: variation and utilization vs. β . . . . . . . . . . . . 41
5.3 Comparison between algorithms: utilization vs. variation . . . . . . . . . 42
5.4 Integrated two-tier network with 2-zone T2N . . . . . . . . . . . . . . . . 44
5.5 An example of the generated user’s moving trace . . . . . . . . . . . . . . 44
5.6 CDF’s of residence times in different zones . . . . . . . . . . . . . . . . . 46
5.7 PH-fitted Markov chain network model . . . . . . . . . . . . . . . . . . . 47
5.8 DP on 3-zone model: variation and utilization vs. α . . . . . . . . . . . . 50
5.9 Adaptive algorithm on 3-zone model: variation and utilization vs. β . . . 51
5.10 Comparison DP and AA: utilization vs. variation . . . . . . . . . . . . . 51
5.11 VBR simulation: adaptive algorithm with statistical information . . . . . 53
5.12 VBR simulation: adaptive algorithm with localization information . . . . 53
vii
5.13 VBR simulation: simple adaptive algorithm . . . . . . . . . . . . . . . . 54
5.14 Simulating VBR case - variation vs. β . . . . . . . . . . . . . . . . . . . 55
5.15 Simulating VBR case - utilization vs. β . . . . . . . . . . . . . . . . . . . 55
5.16 Simulating VBR case - utilization vs. variation . . . . . . . . . . . . . . . 56
viii
Chapter 1
Introduction
1.1 Overview
1.1.1 Video Streaming
Online video has become a mainstream medium and the single most influential factor
driving the need for increased mobile network capacity [8]. It would take 28 years to
watch the video uploaded to YouTube in the week of April 29th, 2010 [20]; HD(high defi-
nition) movies and television programs are widely available online with the help of CDNs
(Content Distribution Networks) and P2P (Peer-to-Peer) networks; video conferencing
and video phones are not the exclusive rights of large companies any more, but can be
enjoyed by individuals and families. It is then important and interesting to research on
improving video streaming techniques.
There are two types of video streaming applications: live streaming, which captures
real-time events and provides the video to users, and on-demand streaming, which offers
stored video contents. Application scenarios of live streaming include video conference,
video phone and live event broadcasting, which have stringent delay requirement. In this
thesis we consider the transmission of pre-encoded video, which is used for delivering all
kinds of published video contents and user generated contents online and is expected to
1
Chapter 1. Introduction 2
account for sixty-six percent of the world’s mobile data traffic by 2014 [21].
In comparison to other traffic flows such as Web browsing and E-mail, video streaming
has its unique characteristics and therefore may impose certain requirements on the
network. Video streaming traffic is inelastic. Unlike web browsing or file downloading,
where data can be transmitted at any rate, video streaming requires certain amount of
data to be delivered and decoded before the playback deadline. Hence it is sensitive to
variations in both bandwidth and transmission delay. Video streaming applications are
loss-tolerant. Robust coding techniques allow video to be decoded with certain loss of
data. However, this does not mean any level of loss can be tolerated. In high error-
rate networks, it is challenging to develop loss-prevention techniques for robust video
transmission.
1.1.2 Heterogeneous Wireless Networks
With the rapid growth of mobile communication technology, various wireless networking
technologies have evolved and become widely deployed all over the world, allowing people
to access the Internet with all kinds of mobile computing devices, at all times and all
places. The popular access technologies include IEEE802.11 wireless local area networks
(WLAN), WiMAX, GPRS, UMTS, and CDMA2000, etc. These technologies are hetero-
geneous in certain attributes, such as coverage area, protocol, signaling mechanism, data
rate, error rate, etc. However, it is common for the personal mobile devices (laptops,
smart phones, PDAs, digital media players) to support more than one wireless access
technologies simultaneously.
With the coexistence of heterogeneous wireless networks and the devices supporting
multiple access technologies, the integration of heterogeneous wireless networks is be-
coming a trend and is part of the 4G network design [30]. This feature allows user to
seamlessly switch among different wireless network interfaces and enjoy greatly enlarged
coverage and more reliable wireless access on a single device.
Chapter 1. Introduction 3
However, there are many challenges in deploying such an inter-technology roaming
environment. Active research topics on heterogeneous wireless networks involve admis-
sion control, hand-off mechanism, mobility management, traffic flow assignment, etc.
The heterogeneity of wireless access technologies also imposes great challenges on video
streaming applications running on a mobile device in the integrated network.
In heterogeneous wireless networks, handoffs inside one technology and between tech-
nologies, can cause extra delays, which exaggerates the challenge on the delay require-
ments of video streaming applications. A more substantial problem in the heterogeneous
wireless networks is that, different access networks offer different ranges of bandwidth,
which greatly exacerbates the variations in streamed video quality if we simply match
the video source rate to the available transmission rate. Hence, in this thesis we mainly
focus on reducing the variation in streamed video quality while maintain high average
quality.
1.1.3 Buffering
Two types of video streaming techniques are commonly applied in both wired and wireless
networks to combat the varying network bandwidth and delays: buffering and video rate
adaptation.
Buffering sustains the video playback when available bit rate (ABR) is low, by
prefetching and storing a certain amount of data ahead of (playback) time. With a
finite buffer size, two types of event will happen and may cause detrimental effects to the
streaming process: buffer underflow and buffer overflow. Underflow may happen when
the playback rate (data consumption rate) is higher than the transmission rate, which
leads to playback jitters (stops). Overflow may happen when the transmission rate is
higher than the playback rate, while at the same time the buffer size is small. Buffer
overflow may lead to loss of data and then playback jitters.
Another factor to consider in buffering is the initial buffering delay, i.e. the waiting
Chapter 1. Introduction 4
time between starting the buffering and starting the playback. There is a trade-off
between the initial buffering delay and the buffer size when we aim to provide satisfiable
video streaming service[19].
In this thesis, we consider the longer-term variations in the transmission rate in
heterogeneous wireless networks, hence we assume an infinite buffer size. Also we set the
initial buffering delay to be minimal. We are primarily interested in how much data to
buffer for the future in every time slot and at what quality should we buffer it.
1.1.4 Rate Adaptation
Rate adaptation techniques match the video source rate to the network transmission
rate, when the transmission rate is low, at the cost of lowering the perceived quality of
decoded video. Various video rate adaptation techniques have been proposed over time,
such as transcoding, joint source/channel coding, multiple file/rate switching, scalable
video coding, and content-aware coding techniques [2]. We present some of them in
Chapter 2.
While theoretically rate adaptation can ensure continuous playback as long as the
ABR is higher than the minimum required rate of the specific adaptation technique, it
introduces fluctuations in the perceived quality of the video, which can be annoying to
users. This problem is exaggerated in heterogeneous wireless networks since the variation
of ABR there is much higher than in homogeneous wireless networks.
1.1.5 Contribution of the Thesis
In this thesis, we consider the problem of streaming pre-encoded video on a moving
mobile terminal (MT) in heterogeneous wireless networks. The video source is stored in
a remote server and transmitted through the backbone network to the local access points
(AP) or base stations (BS) and then to the MT through different wireless networks. The
bottleneck of the connections always lies in the last hop (i.e. the wireless hop.) There is
Chapter 1. Introduction 5
a receiving buffer on the device, which is used for storing prefetched video contents.
We focus on coping with the variable ABR in heterogeneous wireless networks. The
effects of other network characteristics, such as varying end-to-end delay and high error
rate, are assumed to be resolved using any available technique. Our objectives are con-
tinuous playback, high image quality, and low variation in the perceived image quality
(or constant-quality playback).
To achieve these objectives, we propose to combine buffering and rate adaptation
techniques with prediction of certain attributes of the network. Our scheme predicts
the residence times at each individual network, then dynamically allocates the ABR to
each unit of video sequences being transmitted, hence controlling the buffer and rate
adaptation at the same time, under the constraint of fully utilized network resources.
In order to determine the optimal way to allocate the ABR, we propose an adaptive
video rate control scheme using a linear feedback control technique on a generic network
model for a two-zone network. In designing the scheme, we divide the whole streaming
process within the heterogeneous wireless networks into cycles and try to achieve local
optimality within each cycle.
To show the applicability of our scheme to any arbitrary distribution, the performance
of our proposed scheme in a simplified constant-bit-rate (CBR) network scenario with
CBR video source is evaluated in an analytical framework based on Markov chains, where
the state space is dimensioned by the normalized quality levels and the buffered lengths
of video at the end of each cycle. We associate with each state a cost being the quality
variations within the cycle and calculate the average cost per cycle. Other naive and
intuitive algorithms are also studied within the analytical framework in order to show
the advantage of our adaptive scheme.
Then, for the special case of exponential network residence times, we formulate the
streaming process into a finite-horizon controlled Markov Decision Process (MDP), and
solve the optimization problem using a dynamic programming based optimal control
Chapter 1. Introduction 6
algorithm. Although this method is assumed to provide the theoretical optimality, it
involves a large amount of computation, cannot deal with increasing dimensionality, and
is not applicable to more generalized residence time distributions. On the other hand,
the aforementioned adaptive algorithm is much simpler than the dynamic programming
algorithm in terms of the amount of computation involved, works with more generalized
distributions, and requires less knowledge of the network. Through simulations we show
that this scheme provides near optimal performance.
Furthermore, we develop a more realistic network model by modeling the movement
of the MT and fitting the actual residence times using Phase-Type distributions. We
also increase the number of network zones to three. We show through simulation that
our scheme also gives near-optimal performance under the new model. Furthermore, we
simulated our scheme with variable-bit-rate (VBR) networks and VBR video sources, and
explored the effects of utilizing different estimations of residence times, i.e. the statistical
information extracted from history mobility traces, and the geographical information
provided by localization service. Our algorithm proves to provide significantly improved
performance with VBR networks and VBR video source than the naive algorithms.
1.2 Thesis Outline
The thesis is organized as follows. The next chapter reviews work in the related areas
of video rate control and buffering techniques in both homogeneous and heterogeneous
wireless networks. The system setup and the problem statement are presented in Chapter
3. Chapter 4 is focused on a generic network model and the design of our adaptive
control algorithm. We also present the analytical performance evaluation framework. In
Chapter 5, we introduce the Markov Chain network model and the dynamic programming
algorithm. We also extend the system models to a 3-zone model with PH-fitted residence
times and simulate the VBR case in Chapter 5. Finally, Chapter 6 concludes the thesis.
Chapter 2
Literature Review
This chapter briefly reviews the existing research progress on video streaming technologies
on both homogeneous and heterogeneous wireless networks and the current challenges,
which motivated our research work. We first discuss some research works on video rate
adaptation in general variable-bit-rate (VBR) networks. Then we present some related
research on buffering and rate control techniques in video streaming over heterogeneous
wireless networks.
2.1 Video Rate Adaptation Techniques
Various video rate control or adaptation techniques have been proposed to combat short-
term variations in homogeneous VBR wireless channels when performing video streaming.
Specifically for streaming pre-encoded video, research focus has been put on transcod-
ing [22, 3, 11], joint source/channel coding [12, 14, 5], scalable video coding [9], and
other techniques such as content-aware or motion-aware coding [18, 7, 25, 27]. While in
commercial systems, the multiple file/rate switching techniques are widely implemented
[2].
These rate adaptation proposals work well in homogeneous wireless network where
average ABR doesn’t vary over time. However, they could not provide satisfactory per-
7
Chapter 2. Literature Review 8
formance in terms of quality variation in heterogeneous wireless networks, since purely
adapting the source bit rate to the channel bit rate will lead to a large variation of video
quality over different sub-networks.
2.1.1 Transcoding
Transcoding is a technique to adapt the video source rate through recompression. [22,
3, 11] are three examples of research studies on video rate adaptation with transcoding.
These techniques dynamically choose the quantizer used in encoding each frame or block,
and try to minimize the total distortion while matching the video source rate with the
network rate. Their focus is mainly on analyzing the specific encoding technique and
extracting the rate-distortion models. The heavy computation involved in transcoding is
its main disadvantage.
2.1.2 Joint Source/Channel Coding
The authors of [12] show that the perceptual source distortion decreases exponentially
with the increasing MPEG-2 source rate, and the perceptual distortion due to data loss
is directly proportional to the number of lost macro blocks. Hence they propose to
use Joint Source/Channel Coding (JSCC) technique, specifically adding FEC (forward
error correction) bits, to protect the data from loss. The optimal channel coding FEC
parameters can be selected according to the aforementioned relationships and the total
rate of received video stream can be controlled to minimize the total distortion.
Similarly, [14] and [5] both consider the Joint Source/Channel Coding problem with
FEC channel coding and focus on how to choose the channel coding parameters. In [14],
the authors translate the Quality of Service (QoS) requirements of the video streaming
applications into a threshold of occupancy of playback buffer. By adapting the JSCC
parameters their scheme tries to maintain a certain level of buffer occupancy to sustain
continuous playback.
Chapter 2. Literature Review 9
In [5], a probabilistic QoS requirement, i.e. the buffer starvation probability has
been proposed. The authors use cycle-based rate control with cycles being successively
alternating between good(non-fading) and bad(fading) period, while guaranteeing an
upper bound on the probability of starvation at the playback buffer. The cycle-based
idea inspired us to divide the streaming process in heterogeneous networks into cycles, but
our “cycle” have a completely different definition from theirs in that our cycle contains
intervals of the MT residing in different sub-networks.
While the Joint Source /Channel Coding technique can adapt the source rate within
certain range, it usually involves cross-layer design with information flows across PHY
/MAC /Network layers, which might be applicable in homogeneous networks but could
become extremely complex in terms of implementation and computation in heterogeneous
networks.
2.1.3 Scalable Video Coding
The layered or scalable video coding techniques are said to be suitable for adapting
to longer-term bandwidth fluctuations [2]. There has been a substantial body of re-
search works developing efficient scalable compression techniques. A scalable extension
of H.264/AVC [31], Scalable Video Coding (SVC) [24] has been standardized, which pro-
vides scalability of temporal, spatial, quality resolution, or a combination of scalability on
these three dimensions, of a decoded video signal through adaptation of the bit stream.
Fine Granularity Scalability (FGS) coding [17] and Fine Granularity Scalability Tempo-
ral (FGST) coding [29] have also been adopted as amendments to the MPEG-4 standard.
Multiple Description Coding (MDC) [13] is another type of scalable video coding where
each description (substream) of the video stream is of equal weight and independent of
each other in contrast to the Base Layer/Enhancement Layer structure of SVC.
The authors of [10] developed a heuristic rate control algorithm for 2-layer FGS coded
video over TCP-friendly “connection”, which can achieve the same level of smoothness
Chapter 2. Literature Review 10
over both TCP and TCP-friendly protocols. Their algorithm works with CBR coded
video and the loss model is simple. The authors of [16] proposed a stochastic dynamic
programming algorithm for VBR scalable coded video with a more realistic loss model.
The authors of [9] studied the problem of minimizing the average distortion of FGS
video under a limited transmission rate. The authors provided a framework which jointly
considers the effects of packet scheduling at the sender and the error concealment at the
receiver.
The authors of [33] explicitly considered the effect of fading in wireless channel and
develops cross-layer rate adaptation algorithm for layered video in fading channels. The
complexity in cross-layer design makes it difficult to implement even in homogeneous
wireless network.
The authors of [23] introduced a novel streaming strategy to improve the probability
of successfully stream a scalable coded video sequence by adaptively selecting the number
of layers according to mobility information in Ad-Hoc wireless networks. This is relevant
to our work in that our proposed algorithm can also utilize the mobility and location
information to predict the MT’s movement and channel status, as described in Chapter
5.
In our control scheme, we can use either the transcoding or scalable coding techniques
to perform rate adaptation. However, we assume a generic rate-quality relationship in
our model and that one cannot change the quality of a video sequence that is already
transmitted.
2.1.4 Content-Aware Coding Techniques
There exists many other video rate adaptation algorithms which try to achieve different
QoS objectives. Content-aware encoding/playout has been an interesting and contro-
versial topic for video rate adaptation, as there exists no generally accepted standard
for perceived quality of motion pictures when we consider the presentation of the actual
Chapter 2. Literature Review 11
content, instead of quantifiable metrics such as resolution, frame rate and PSNR (Peak
Signal-to-Noise Ratio). Representative works of content-aware rate adaptation/playout
control include [18, 7, 25, 27], etc. These works try to analyze the amount of motion
or interested objects in each frame, and allocate the available bit rates unfairly among
different objects / frames to achieve best perceived quality when the network rate is not
high enough to present the full pictures.
2.2 Rate Control in Heterogeneous Wireless Net-
works
Video streaming in heterogeneous wireless networks has been a relatively new topic. Most
of the available works address the issues in architecture design and hand-off handling.
The authors of [26] analyzed the effects of handoffs on rate control and proposed a cross-
layer solution to anticipate the handoff occurrence and to adjust the data rate. They
use transport-layer dummy packets to probe the channel in their solution, while in this
thesis we propose to utilize the statistical information of the residence times and ABR
in each sub-network.
Some researchers consider video streaming in a multiple stream environment with
heterogeneous access technologies and focus on fairness or priority among all users/flows.
In [38] the authors study the rate allocation problem in streaming over wireless networks
with heterogeneous link speeds. The focus of their work is on how to allocate the rate
between multiple video streaming sessions on heterogeneous links to maximize the aver-
age quality among all users, while the quality enjoyed by a single user is not explicitly
considered.
The authors of [1] addressed the problem of flow rate control for different types of
traffic flows and heterogeneous wireless links, and employs an H-infinity optimal rate
controller to achieve efficient utilization of all channels while taking the requirements of
Chapter 2. Literature Review 12
different flow types in to account. Both of them assume that all the access networks in
the integrated network are available all the time, while our assumption is that the user
is moving and the trajectory is not always covered by both sub-networks.
The authors of [35, 36, 37] considered the video streaming problem in an integrated
3G/WLAN network from a monetary cost point of view. Their system setting of the
heterogeneous networks is the most similar one to ours, yet the objective and control
actions are completely different. Different streaming strategies are proposed to decide
how much data to be streamed (i.e. the transmission rate) in each individual network as
well as when to hand off to the other network, so that the monetary cost of streaming the
data is minimized. While in our system model we also consider the cost effects of each
network, we assume that it is always good to fully utilize the ABR and that the MT will
switch to the higher-rate, lower-cost network whenever it is available, as this strategy is
simple and already implemented in commercial systems such as the iPhone.
It is worth mentioning that, some of the techniques mentioned above, such as [33] and
[5] have similar objectives as ours, i.e. minimizing the variations in adapted video rate
caused by variations in network transmission rate. However, in homogeneous wireless
networks, the variation caused by fading and other short-term effects are quite different
from the variation caused by handoff between different access technologies in both time
scale and magnitude. Hence these techniques cannot be directly applied to our problem.
Furthermore, because the time scale of variation in our problem is longer, we may have
more ways to predict the variation, such as utilizing geographical information.
2.3 Buffering in Heterogeneous Wireless Networks
For streaming pre-encoded video, buffering is another technique to overcome the mis-
match between video source rate and channel bit rate. By caching enough data in the
client buffer ahead of time, continuous high-rate playback can be sustained when the
Chapter 2. Literature Review 13
channel throughput is low. Buffering schemes for streaming VBR video over heteroge-
neous wireless networks are studied in [15]. These schemes include fixed/jointly optimized
schemes based on buffering delay, buffered playout data, and playout time. Analysis on
both the jitter frequency and the buffering delay are conducted for these schemes.
However, without rate adaptation buffer underflow happens frequently when the av-
erage channel throughput is lower than the average video source rate, leading to playback
jitters. Hence we propose to combine rate adaptation with buffering for long-term varia-
tions to smooth out the streaming process in heterogeneous networks. We choose only the
playout time based buffering scheme, as when we introduce rate adaptation the buffering
delay become meaningless, and the buffered playout data become highly variable with
the changing video rate.
To the best of our knowledge, this thesis is the first work to propose the combination
of rate adaptation and buffering to address the problem of smooth video playback in
heterogeneous networks. Nevertheless, our work is inspired by and based on the related
works listed here in that the scheme shall utilize a generic rate adaptation techniques
mentioned above to perform the control actions.
Chapter 3
Problem Statement
In this chapter, we explain in details the problem we study in this thesis. We first present
the application scenario, then introduce our way modeling of the system based on this
scenario with some practical assumptions. A mathematical formulation of our problem
is then presented based on these common assumptions.
3.1 Application Scenario
We study video streaming over the heterogeneous wireless networks with the overlapping
of two networks (or two zones), as shown in Figure 3.1. The Tier-1 Network (T1N)
is assumed to provide universal coverage with low bit rate, while Tier-2 Network (T2N)
covers limited areas around the Access Points (APs), with high bit rate. In reality, T1N is
usually more costly than T2N. A proper example of T1N can be the 3G cellular network,
while T2N can be Wireless LAN. A user prefers to access the Internet through T2N due
to its high bandwidth and low cost, thus whenever he/she enters a T2N covered area,
the mobile device switches to T2N for transmission. While our scheme is independent
of lower layer (e.g. PHY/MAC layer) implementations and can actually handle the
simultaneous transmission over both sub-networks, we maintain the assumption of using
only one sub-network at the same time since it is more practical to do so in reality.
14
Chapter 3. Problem Statement 15
T1N
BS
AP
T2N
T2N
Figure 3.1: Integrated two-tier network
Two types of handoffs take place in this network: intra-technology handoff or Hori-
zontal Handoff (HHO) in which the mobile terminal (MT) switches between two Access
Points (AP) or Base Stations (BS) using the same access technology, e.g. from one
WLAN AP to another, and inter-technology handoff or Vertical Handoff (VHO), which
occurs when the MT roams between different access technologies, e.g. switching from 3G
to WLAN when entering a WLAN covered area. VHO affects different system perfor-
mance metrics, such as the signaling load, resource utilization and user perceived QoS. In
particular, the available bit rate (ABR) in our model may vary by one order of magnitude
after any VHO. Both types of handoffs may cause extra delays in the transmission, but
we do not consider the extra delays here and assume that there is a seamless handoff
handling scheme which can eliminate the delays caused by both types of handoffs (which
may be achieved at the cost of ABR). In practice, a handoff handling scheme such as the
one presented in [6] can be employed to satisfy this assumption.
A video streaming session is running on the device while the MT traverses through
this integrated two-tier network. The streaming server lies outside this wireless network,
but the bottleneck of the connection is the last hop - the wireless link between the
MT and the BS/AP. The MT keeps sending control messages to request the server to
adjust the video source rate and transmission rate. The server then makes adjustments
accordingly and transmits the data to the MT. The MT has a buffer, which stores the
Chapter 3. Problem Statement 16
received data before they are used for playback. Our goal is then to develop a control
scheme to determine how to choose the video source rate and how much to buffer ahead
of time given some statistical and observed channel information. The streaming session is
assumed to be very long and we analyze the performance of our scheme on a time-average
basis.
3.2 Models and Assumptions
3.2.1 Rate Adaptation and Playback
We model the variation in the wireless channel, the error control scheme, and the handoff
handling effects all into the random ABR of the network, denoted by R(i), which is always
positive. In reality, ABR is the amount of error-free video data received by the MT at
each time slot.
We divide the whole streaming session, which consists of several “in-T1N” and “in-
T2N” intervals, into cycles, and denote each cycle by its sequence number in the whole
streaming session, j, where j = 1 represents the first cycle in this session. Each cycle
starts at a T1N-to-T2N VHO and contains one “in-T2N” interval followed by one “in-
T1N” interval, the lengths of which are denoted as T2 and T1 respectively. (Note that, here
we assume the streaming session always starts in T2N, i.e. the high rate network. This is
reasonable since there is not much we can optimize before the first VHO if it starts within
T1N, and we are considering the average performance in the whole streaming process,
so edge effect at the beginning can be ignored.) We further model the video streaming
process as time-discrete with time slots of equal length, and denote each time slot by its
sequence number in the current cycle, i, where i = 1 represents the first time slot in
the current cycle. The control actions are decided and performed at the beginning of
each time slot.
We assume that we have original video streams with very high quality. The average
Chapter 3. Problem Statement 17
rate of video can be higher than the highest network rate, and we can adjust the source
encoding rate to any level at any granularity up to the original rate. This assumption of
rate adaptation at any granularity can be accommodated by quantization in practice. By
making this assumption, we eliminate the possibility of playback stop, since transmission
rate R(i) is assumed to be always positive.
The original source rate of the whole video sequence as well as the rate-quality rela-
tionship of the video are transmitted to the MT before the streaming starts, thus the MT
can utilize this information to adjust the rate of “future” parts of the video which are
being streamed. Here we do not define the specific technique used to adapt the source
rate. The transcoding or SVC technique mentioned in Chapter 2 may be employed in
reality to perform rate adaptation.
It is worth noting that, although control actions are performed at the beginning of
each time slot, it is not necessarily effective for only one time slot in playback time.
This is because control decisions are made at every time slot of transmission time, in
which period more than one time slot of video data in the playback sequence may be
transmitted. We illustrate the relationship between transmission time and playback time
in Figure 3.2.
We use a “quality level” defined as q(k) = f(r(k), r0(k)) to control the source rate
adaptation at time slot k, where f(x, y) is monotonously increasing with x, r0(k) is the
original video rate at time slot k, and r(k) is the adapted video rate at time slot k. Hence
for each time slot, the perceived image quality increases with q(k). While there are several
methods to characterize the quality of the picture, e.g. PSNR, distortion, etc, we do not
choose a specific metric, but assume a generic form of the rate-quality relationship. For
simplicity, we assume the original video stream is constant-quality encoded (whether it
is CBR or VBR), thus the perceived quality only depends on the adjusted bit rate and
the original bit rate, and q(k) = f(r(k), r0(k)) = f(r(k)).
Based on this model, we use the metric V = E(∑
0<i<Tg(|q(i+1)−q(i)|)
T) to evaluate the
Chapter 3. Problem Statement 18
Transmission Time
Playback Time
Transmission Rate
Playback Rate
Original Rate
…1
2
2
3
4
4
5
…
1 2 3 4 5m
m
+
1
m
+
1
m+
2
m+
2
2T
m
2 1T T+
2
1
T
T
+
2
1
T
T
+
m+
2
2
1
T
T
+
m
+
1
…
…
Figure 3.2: Relationship between the transmission sequence in time and the playback
sequence in time
performance of our algorithm, where T can be a fixed length of time or the length of one
cycle, and g(·) is a monotonously increasing function. This metric reflects the average
amount of total quality variation of the rate-adapted video stream over time.
3.2.2 Residence Time and Rate Estimation
We assume that statistical information such as mean and variance of the residence time
within each sub-network and also the average transmission rates R2 and R1 throughout
the two intervals can be obtained form the AP. This information can be extracted from
previous data collected by the AP. The MT requests the information at the beginning
of each cycle (i = 0). Another possibility is to utilize the geographical information and
MT’s moving speed/direction to estimate the residence time in each network, which will
Chapter 3. Problem Statement 19
be briefly discussed in Chapter 5.
3.2.3 Feedback Control Mechanism
We assume that the ABR of current time slot, R(i), as well as the current connected
network can be estimated with good accuracy and fed back by the MT to the server at
the beginning of this time slot. The server then transmits data at the estimated rate.
This mechanism guarantees that our algorithm runs under the following constraint.
Constraint: the ABR of the network is 100% utilized.
However, this doesn’t mean that the transmitted data are 100% utilized. We introduce
another metric called “data utilization”, calculated as the total data used for playback
up to T divided by the total data transmitted up to T . The difference between these
two values is the amount of data left in the buffer at time slot T . From the design of our
algorithms we will see it is not possible to achieve 100% data utilization, but we still try
to maintain the data utilization above an acceptable level within our observation horizon.
The notations for the system models in this thesis are summarized in Table 3.1.
3.3 Problem Formulation
We can translate the aforementioned objectives, assumptions and constraints into the
following optimization problem:
Chapter 3. Problem Statement 20
Notation Description
i Index of time slots (in transmission time).
R(i) ABR in ith slot of current cycle.
k Index of time slots (in playback time).
r0(k) Video source rate (to be played) in kth slot.
r(k) Adapted video rate (to be played) in kth slot.
T1, T1, T1
Residence time in T1N (random variable), its mean and
the estimation value of it used in the rate control scheme.
T2, T2, T2
Residence time in T2N (random variable), its mean and
the estimation value of it used in the rate control scheme.
R1, R2 Average ABR in T1N and T2N.
R1, R2 ABR in T1N and T2N in CBR network.
q(k) Quality of adapted video (to be played) in kth time slot of current cycle.
j Index of cycles.
T1(j) ,T2(j) Residence time in T1N and T2N in jth cycle.
q0(j) Quality of adapted video in the last time slot of the (j − 1)th cycle.
Table 3.1: Notations in system model
Chapter 3. Problem Statement 21
min V = E(
∑1<k<T
g(|q(k + 1) − q(k)|)
T)
where q(k) = f(r(k), r0(k))
s.t. 0 < r(k) < r0(k) (3.1)
∑
1<k<t
r(k) ≤∑
1<i<t
R(i), 1 < t < T
ǫ = 1 −
∑1<k<T
r(k)
∑1<i<T
R(i)≤ ǫ0
Here we minimize the expected variation while keeping the data utilization above a
certain level. The output of our algorithm is then the q(k)’s, or equivalently the r(k)’s,
as they are 1-to-1 matched given the r0(k)’s, which are assumed to be known (and fixed
in the CBR case).
Given the randomness of the R(i), this optimization problem is not directly solvable.
In the generic model in Chapter 4, T = T1 + T2 in the above formulation. In the
Markov Chain based channel model in Chapter 5 , we assume T1 and T2 are exponentially
distributed, and T is set to be a fixed length of time instead of the length of each cycle.
Chapter 4
Generic Network Model
Three algorithms are developed for the generic network model with one simple algorithm
as comparison. All of them can work with any type of residence time distributions.
4.1 Control Algorithms
4.1.1 Adaptive Control Algorithm
We first propose an adaptive control algorithm which can use statistical information
or localization information to estimate the residence times, and adaptively adjusts the
quality level based on the estimation at each time slot. The estimation horizon is within
the current cycle. Let Te(i) denote the estimation of residence-time-to-go at time slot
i in the current cycle. If the MT is in T2N at time slot i, Te(i) includes the remain-
ing residence time in T2N and the estimated length of the following “in-T1N” interval:
Te(i) = [T2(i) T1(i)]. Otherwise, T2(i) = 0 and the estimation gives us the remaining
residence time in T1N, i.e. Te(i) = [0 T1(i)].
In one approach, we can use the average residence times, T2 and T1, as the estimations
of residence times. At every time slot i in cycle j, the estimation Te(i) is calculated as
follows:
22
Chapter 4. Generic Network Model 23
If the MT is in T2N at time slot i,
T2(i) = max{T2 − i, 1} (4.1)
T1(i) = T1 (4.2)
If the MT is in T1N at time slot i,
T1(i) = max{T1 − (i − T2(j)), 1} (4.3)
T2(i) = 0 (4.4)
where T2(j) is the actual residence time the MT spent in T2N in the current cycle.
We can also use other forms of estimations in our algorithms. In Chapter 5, we
describe a case where our algorithm can work with estimations generated by localization
service.
Our adaptive control algorithm is as follows: At the beginning of each cycle (time
slot 0), we have the original playback curve (i.e. the cumulated video source rate curve)
starting from the current time slot, denoted as P (t). We may already have some data
buffered from the previous cycle, denoted as TB(0) seconds or B(0) bits in the buffer.
Also, we want to buffer some data for the next cycle, i.e. at the end of this cycle, we
should have TBh seconds of video in the buffer. TBh is set to be 0 in the CBR cases but
to a very small value in the later VBR cases to accommodate short-term variations in
VBR networks.
To simplify the calculation here, we choose a simple form of “quality level”, q(i) =
f(r(i), r0(i)) = r(i)/r0(i), i.e. quality level is proportional to the adjusted video rate
given a fixed original rate. But for any other f(x, y) monotonously increasing with x,
similar calculations can be applied here for all the algorithms mentioned below.
Then we calculate the initial quality level q(0), which is, the average quality level we
can sustain throughout the current cycle under the assumption of perfect estimation:
q(0) =R2T2(0) + R1T1(0)
P (T2(0) + T1(0) + TBh − TB(0))(4.5)
Chapter 4. Generic Network Model 24
β q-
Estimation
Error Signal Control Signal
Feedback Signal
Desired State
System
Figure 4.1: Illustration of proportional feedback controller
At each subsequent time slot i, the MT keeps estimating Te(i). Also, we have the amount
of buffered data at this time is B(i) bits. Then if we keep using the quality level decided
at the previous time slot, q(i− 1), the estimated amount of buffered data (in bits) at the
end of this cycle would be:
Be(i) = B(i) + R2T2(i) + R1T1(i) − q(i − 1)[P (i + T2(i) + T1(i)) − P (i)] (4.6)
The estimated length of buffered video is TBe(i), which is the solution of the following
equation:
q(i − 1)[P (i + T2(i) + T1(i) + TBe(i)) − P (i + T2(i) + T1(i))] = Be(i) (4.7)
Then, we decide q(i) as follows:
q(i) − q(i − 1) = β · (TBe(i) − TBh) (4.8)
Here, we actually construct a linear negative-feedback controller with a proportional gain
β (See Figure 4.1 for an illustration). The input of our system is q(i − 1), the output
is TBe(i), and our setpoint is TBh. The controller attempts to minimize the “error”
between the given setpoint and the output by adjusting the control inputs according to
the proportional negative feedback law. If the q(i) we get from the above controller is
higher than 1, which means that the new rate will be higher than the original rate (which
is impossible under our system assumptions), we set q(i) = 1. Also, if the new rate cannot
be sustained by the current network transmission rate, i.e. there is not enough buffer
(TB(i) < 1) and the transmission rate is low, we set q(i) to be the highest level that can
Chapter 4. Generic Network Model 25
be supported for the current time slot:
q(i) =R1
P (1 − TB(i))
Hence,
q(i) =
max{1, q(i − 1) + β · (TBe(i) − TBh)}, if TB(i) ≥ 1
min{ R1
P (1−TB(i)), q(i − 1) + β · (TBe(i) − TBh)}, if TB(i) < 1
(4.9)
Other forms of feedback controllers, such as PID (Proportional-Integral-Derivative)
controller [32], can also be applied here, but since our system is non-linear and stochastic
in general, it is difficult to provide theoretical guidance on choosing the parameters.
Hence we keep the controller simple with only one adjustable parameter, β. Even for the
only parameter, we cannot provide a method to choose the proper β which guarantees
optimality and system stability (which is the case for most PID controllers). Generally,
there might be some empirical rules for choosing parameters in PID control systems,
but the effectiveness of these rules depends highly on the system structure. However,
we discuss the choice of the parameter for the specific set of system parameters in the
analysis of our simulation results in Chapter 5 and provide some intuitions on tuning the
parameters.
During each time slot, the server sets the quality levels of video sequences to be
transmitted according to the adaptive algorithm and transmits them to the MT.
4.1.2 Simple Algorithm
To compare our algorithm with other bandwidth smoothing techniques without long-term
prediction or estimation for the VHOs, we design another simple adaptive algorithm. In
this algorithm, we don not use any information about the network distribution but decide
q(i) only based on the current buffered length, i.e. at every time slot i we decide q(i) as
Chapter 4. Generic Network Model 26
follows:
q(i) =
max{1, q(i − 1) + β · (TB(i) − TBh)}, if TB(i) ≥ 1
min{ R1
P (1−TB(i)), q(i − 1) + β · (TB(i) − TBh)}, if TB(i) < 1
(4.10)
Where TB(i) is the buffered length of video in seconds at time slot i, which is the solution
for the following equation:
q(i − 1)[P (i + TB(i)) − P (i)] = B(i) (4.11)
We are employing the same negative feedback control technique here, except that our
system does not include the estimation part any more. Instead, the algorithm uses the
observation of the buffer level as the input.
4.1.3 Mean Residual Life Based Algorithm
Assume we know the exact distribution of the residence times. One intuitive algorithm
we can come up with is to calculate the expected remaining residence time (or Mean
Residual Life, MRL) at every time slot given the elapsed residence time the MT spent in
current sub-network, then allocate the expected network resources evenly according to
the MRL.
To be more specific, assume we have the Probability Density Functions (PDFs) for
T1, T2 to be f1(t), f2(t) respectively. At each time slot i in T2N, the algorithm calculates
the MRL in T2N as follows:
ˆMRL2(i) = E(T2 − i|T2 > i) (4.12)
= (−1
∫ ∞i f2(t)dt
∫ ∞
itf2(t)dt) − i
MRL2(i) = max{ ˆMRL2(i), 1} (4.13)
Then, it calculates the quality level to be used according to the MRL (also assuming the
proportional rate-quality relationship here):
q(i) = max{1,R2MRL2(i) + R1T1
P (MRL2(i) + T1 − TB(i))} (4.14)
Chapter 4. Generic Network Model 27
where TB(i) is the length of video the MT has buffered at time slot i.
If at time slot i the MT is in T1N, the calculation for the MRL is similar, but the
q(i) is calculated as follows:
q(i) =
max{1, R1MRL1(i)P (MRL1(i)−TB(i))
}, if TB(i) ≥ 1
min{ R1
P (1−TB(i)), R1MRL1(i)
P (MRL1(i)−TB(i))}, if TB(i) < 1
(4.15)
For a relatively stable network distribution, the MRLs given different elapsed residence
times can be pre-calculated and stored within a table, hence this algorithm does not
involve large amount of calculation. However, since the transfer function from the resi-
dence times to the time-averaged variation is not linear, the MRL is not necessarily the
optimal estimation for the remaining residence time if we want to achieve the minimum
expected variation.
Furthermore, the performance of the MRL based algorithm depends heavily on the
distributions of residence times. If the residence times are exponentially distributed,
then the MRL is always the mean of the exponential distribution regardless of how long
the MT has stayed in the current sub-network. Thus the MRL based algorithm will
keep increasing the quality level from the beginning of the cycle, until it reaches the
end of T2N, and keep decreasing the quality level from the beginning of the following
T1N interval until the end of the cycle. Such curve introduces quality variations in itself
instead of eliminating them. With certain other light-tail distributions, the MRL based
algorithm gives better performance.
4.1.4 Simple Shaping Algorithm
Another intuitive solution is to explicitly consider the form of our optimization objective:
the time averaged variation. The algorithm then designs a “best shaped” quality level
curve according to the specific form of this metric. Again we consider the quadratic form
of variation we have used all the time: E(∑
0<i<T(|q(i+1)−q(i)|)2
T).
Chapter 4. Generic Network Model 28
At every time slot, we have an estimation of how much data to receive from current
time slot to the end of current cycle. We also have the record of the quality level set by
the algorithm in the previous time slot. Now assume the estimation is accurate, the best
way to allocate the deliverable data to each time slot of video is to perform a quadratic
programming over the remaining time of the cycle within the constraint of using up all
the deliverable data at the end of the cycle, and the objective of minimum variation
(which is in a quadratic form). We use the expected values of residence times, T1 and T2
as the estimations.
However, if the estimation is inaccurate, which is almost always the case, the output
of quadratic programming is never the optimal result. The problem is not substantial
when the estimation of T2 is inaccurate, as at the end of the T2N period, we always
end up with plenty of buffer to compensate for the low-rate in T1N , which can be used
to guarantee a smooth degrading curve afterwards if the MT enters T1N earlier than
expected. The only case when there is a problem is when T1 > T1. Because when this is
the case, after the (T2 + T1)th time slot, there is nothing in the buffer, and the quality
level of time slot (T2 + T1 + 1) will be forced to set to R1/r0, which will usually cause
a huge difference between q(T2 + T1) and q(T2 + T1 + 1). In order to avoid this, we add
a further constraint to the quadratic programming, that at time slot (T2 + T1 + 1), the
quality level is set to be R1/r0.
So the quadratic programming part in this algorithm becomes:
min∑
T2+2≤i≤T2+T1
(q(i + 1) − q(i))2 + (q(T2) − q(T2 + 1))2 + (R1/r − q(T2 + T1))2
s.t.∑
T2+1≤i≤T2+T1
q(i) = TB1 · R1/r + T1 · R1/r (4.16)
R1/r ≤ q(i) ≤ 1, T2 + 1 ≤ i ≤ T2 + T1
When the MT is in the following T1N network, the algorithm keeps using the “best
shaped” quality curve determined by the quadratic programming until it reaches the end
Chapter 4. Generic Network Model 29
of this cycle. If T1 > T1, the quality levels for each time slot between [T1, T1] will be set
to R1/r.
4.2 Analytical Framework and Analytical Results
In order to evaluate the performance of these algorithms, we further introduce an ana-
lytical framework based on Markov chains, which can work with any kind of residence
time distribution.
Following the generic network model, we model the streaming process as a discrete-
time Markov chain, with states {TB, q0}, where TB is the playback length of buffered video
(in seconds) at the beginning of each cycle, and q0 is the quality level set at the last time
slot in the previous cycle. We assume T2(j) and T1(j) are i.i.d. random variables (note
that this is an assumption in the analytical framework, we do not make this assumption
when we design the rate control scheme), where j is the index of cycles. Then the values
of {TB(j), q0(j)} depend only on the previous state, {TB(j − 1), q0(j − 1)}. To make the
state space finite, we adopt a uniform quantization scheme on the range of TB and q0
values.
We can see that, since we are always trying to achieve an expected amount of buffered
data of 0 at the end of each cycle, (i.e. high data utilization) the playback length of
buffered video should never exceed the estimated value of T1, T1 (since in T2N period,
the algorithm buffers for no longer than T1(R2 − R1)/R2 to achieve a zero buffer at the
end of the cycle, while in the T1N period, the algorithm uses up the previously buffered
data). Hence this Markov chain is a finite-state Markov chain, with state TB being
{0, 1, 2, ...T1(R2 − R1)/R2}, and q0 being {ql, ql + qs, ql + 2qs, ..., qh}. Where ql is the
lowest quality level allowed by the algorithm, and qh is the highest quality level allowed
by the algorithm.
Between any two states, there is a transition to each other, making this Markov chain a
Chapter 4. Generic Network Model 30
fully connected graph. To analyze the overall performance of our algorithm, we associate
a cost Je(TB, q0) = E(
∑1<i<T2+T1
g(|q(i+1)−q(i)|)+g(|q0−q(1)|)
T2+T1) on each state, the smaller the
expected total cost is, the better our algorithm performs. Note that, the difference
between this cost and the cost in the later introduced dynamic programming algorithm
is that the later covers a fixed length of time, while the former covers the (random) length
of one cycle. Then we calculate the transition probabilities between states according to
the distribution of residence times and the algorithm being evaluated:
P (TB(j + 1) = TBm, q0(j + 1) = q0n|TB(j) = TBl, q0(j) = q0k)
= P (TBm, q0n|TBl, q0k)
=∑
T2,T1
P (TBm, q0n|TBl, q0k, T2(j) = T2, T1(j) = T1) · P (T2(j) = T2) · P (T1(j) = T1)
(4.17)
Now we can compose the transition matrix PA for our defined Markov chain. Solving
this Markov chain, we can obtain the steady state distribution, π, which satisfies π = πPA.
Then we can calculate the expected cost per cycle:
E(Je) =∑
TB ,q0
πTB ,q0· Je(TB, q0) (4.18)
Note that, in this analytical framework we do not consider the data utilization metric
(while in later studies of the Markov chain network model and dynamic programming
algorithms, we explicitly consider it). This is because, in this model we are observing a
relatively short time horizon (one cycle), the utilization in this time horizon cannot reflect
the long-term utilization. Instead, we set the objective of using up all the transmitted
data at the end of each cycle in the design of our algorithms, which ensures that the
long-term data utilization is near 100%.
4.2.1 Analytical Results for Generic Model
Under the established framework, we analyze the performance of our adaptive control al-
gorithm, the MRL based algorithm and the simple shaping algorithm with the quadratic
Chapter 4. Generic Network Model 31
Parameter Value
Time slot length 1 s
T1, T2 20 s, 20 s
R1 0.5:0.25:2.75 Mbps
R2 6:-0.25:3.75 Mbps
r0 6 Mbps
a1, b1 6.67, 3
a2, b2 6.67, 3
Table 4.1: Analysis parameters - 1
Parameter Value
Time slot length 1 s
T1, T2 20 s, 20 s
R1 0.5:0.25:2.75 Mbps
R2 6:-0.25:3.75 Mbps
r0 6 Mbps
a1, b1 13.33, 1.5
a2, b2 13.33, 1.5
Table 4.2: Analysis parameters - 2
variation metric. The simple adaptive algorithm is also evaluated as a comparison. To
avoid the cases when MRL based algorithm doesn’t perform intuitively (like the expo-
nential distribution case), we choose light-tail distributions of residence times to analyze
the performance. An example of the analytical results with Gamma-distributed residence
times is shown in Figure 4.3 and 4.4 along with the residence time distributions (the same
for T1 and T2) in Figure 4.2. We also simulate the algorithms in the same settings to val-
idate the analytical framework. The parameter β in the adaptive and simple algorithms
are selected based on simulations first to ensure the long-term utilization is above 90%.
Parameters are listed in Tables 4.1 and 4.2 (ai and bi are the two parameters in
the Gamma distribution). Proper values of these parameters (especially the average
residence times) depend on the density and coverage of T2Ns, the MT’s moving speed
and moving pattern. In Section 5.6 when we discuss the network model generated by
mobility modeling, we use practical values for moving speed and T2N coverage, and a
realistic distribution of the T2Ns. The mean residence times we get in that setting are
T1 = 35sandT2 = 25s. So we think the mean residence time of 20s here is also practical
in real scenarios.
As we can see, the adaptive control algorithm gives the best performance all the time,
Chapter 4. Generic Network Model 32
0 10 20 30 40 500
0.01
0.02
0.03
0.04
0.05
0.06
T1
Pd
f(T
1)
0 10 20 30 400
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
T1
Pd
f(T
1)
Gamma distribution of residence time -1 Gamma distribution of residence time -2
Figure 4.2: Distributions of residence times
even though we do not minimize the variation metric in the adaptive algorithm explic-
itly. In contrast, the MRL based algorithm tries to optimize through selecting the best
estimation, yet it fails because of the non-linear relationship between the residence times
and the variation. The simple shaping algorithm is not optimal either, because although
it tries to minimize the variation by shaping the quality curve under the assumption of
perfect estimation, it fails to deal with the uncertainty of the residence times.
Furthermore, the adaptive algorithm has more advantage if we compare the complex-
ity of the algorithms and the ability to deal with VBR cases. While the adaptive control
algorithm is designed for VBR cases and performs well in VBR cases as we show later,
the MRL based algorithm and the simple shaping algorithm are designed based on CBR
assumptions and would simply fail to deal with more uncertainty in the VBR cases.
We can also see that the simulation results and the analytical results are reason-
ably close to each other and exhibits the same trends, which validates our analytical
framework. However, there is a consistent bias in the analysis shown in both figures in
comparison to the simulation. Through experiments we found that by using finer quan-
Chapter 4. Generic Network Model 33
tization for TB and q0 we can reduce the gap between analytical results and simulation
results. Hence we believe the bias is caused by quantization errors in the analysis.
On the other hand, comparing between the results with differently shaped distribu-
tions (the first one more dispersed and the second one more centralized), we can also
notice some characteristics about the MRL algorithm: the MRL based algorithm works
better (nearer to the adaptive one) when the distribution of the algorithm is more cen-
tralized, because when the distribution is more centralized, the MRL will be nearer to
the real remaining residence time. The simple shaping algorithm has its own character-
istics as well. However, as they show poor performance in comparison to our adaptive
algorithm, we do not look further into the details of these algorithms.
Chapter 4. Generic Network Model 34
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.60
0.002
0.004
0.006
0.008
0.01
0.012
0.014
R1
V
Adaptive(A)
MRL(A)
Shaping(A)
Simple(A)
Adaptive(S)
MRL(S)
Shaping(S)
Simple(S)
Figure 4.3: Analysis vs simulation results: generic model, Gamma distribution - 1
Chapter 4. Generic Network Model 35
0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.60
0.002
0.004
0.006
0.008
0.01
0.012
R1
V
Adaptive(A)
MRL(A)
Shaping(A)
Simple(A)
Adaptive(S)
MRL(S)
Shaping(S)
Simple(S)
Figure 4.4: Analysis vs simulation results: generic model, Gamma distribution - 2
Chapter 5
Markov Chain Network Model
We can also model the heterogeneous wireless network channel as a Markov chain where
we assume exponential network residence time in each sub-network and memoryless tran-
sitions between sub-networks. Based on this Markov chain, we make decisions on video
source rate adaptation, turning the streaming process into a controlled Markov Decision
Process (MDP). We can then apply dynamic programming algorithm on this MDP, which
is assumed to achieve optimal performance.
5.1 Markov Decision Process Model
A common assumption on residence times in separated homogeneous wireless networks,
such as cellular and WLAN networks, is that they are exponentially distributed. Follow-
ing this assumption, we can assume the transitions between the two sub-networks to be
memoryless. The ABR in the channel is then modeled as a Markov chain with two states
{R1, R2} and transition probabilities
P =
p11 p12
p21 p22
Hence a part of the streaming session can be characterized as a Markov Decision
Process (S, T , Φ), where S is the set of possible states in the streaming session, T is
36
Chapter 5. Markov Chain Network Model 37
the transition probability matrix between states, and Φ is the set of possible decisions
(quality levels) we can choose at each state.
We define the system state as s = {R,A,L}, where R is the ABR at the current time
slot, A is the control action (quality level) chosen by the algorithm at the previous time
slot, and L is the buffered length of video stream (in seconds) at the beginning of the
current time slot. Thus the transitions take place as follows:
P (R(i + 1) = Rl|R(i) = Rk) = plk, l, k = 1, 2
A(i + 1) = φ(i) (5.1)
L(i + 1) = L(i) + R(i)/f−1(φ(i)) − 1
Note that the state variables A and L are completely determined given R and the current
control action.
For an arbitrary admissible control action φ at time slot i, we associate a cost with
the transition from i-th slot to (i + 1)th (from state s = {R(i), A(i), L(i)} to state
s′ = {R(i + 1), A(i + 1), L(i + 1)}):
V φi (s) = α · (L(i) + R(i)/f−1(φ) − 1)2 + g(φ(i) − A(i)), if i = N (5.2)
V φi (s) = g(φ − A(i)), i = 1, 2, ..., N − 1 (5.3)
where the g(φ − A(i)) part reflects the variation in the adapted video quality de-
termined by the algorithm, and the α · (L(i) + R(i)/f−1(φ) − 1)2 part reflects another
objective of our algorithms: high utilization of transmitted data, and α is a weight as-
sociated with this part. N is the total number of time slots in the part of the streaming
session we are observing. f−1(φ) is the inverse mapping from control action φ to the
adapted video source rate. The objective of the algorithm is to minimize the expected
cost over all transitions:
JΨi = E[
N∑
l=1
Vl(sΨ)] (5.4)
Chapter 5. Markov Chain Network Model 38
Thus the system becomes a finite-horizon controlled MDP. The process is not homoge-
neous, because its transition matrix vary from time to time. As a result, there exists no
optimal static control policy.
Now define the cost-to-go at time slot i for an admissible policy Ψ = (φ1, φ2, ...φN):
JΨi = E[
N∑
l=i
Vl(sΨ)] (5.5)
According to Bellman’s Principle of Optimality, the optimal control policy is given by
the following equation:
φi∗(s) = arg min{Vi(s) + E[Ji(s)]} (5.6)
5.2 Dynamic Programming Algorithm
Based on the system model described by Equations (5.1) ∼ (5.6), we use a backward
induction based algorithm to solve for the optimal control action at every state:
Algorithm 1 Find the optimal control policy Φ∗ = (φ1, φ2, ...φN).
Require: T ≥ 1
i ⇐ N
for all states s do
φN(s) = arg min{VN(s)]}
end for
while i ≥ 1 do
for all states s do
φi(s) = arg min{Vi(s) + E[Ji(s)]}
end for
i ⇐ i − 1
end while
After obtaining the optimal policy Φ∗, we can store it into a look-up table at the MT.
At the very beginning, we let the system start within T2N, from state s0 = {R2, q0, 0},
Chapter 5. Markov Chain Network Model 39
where R2 denotes the ABR in T2N, q0 denotes the quality level corresponding to the
average ABR in the network: q0 = f−1( R2T2+R1T1
T2+T1).
Then at the beginning of each following time slot, the MT chooses the optimal action
according to current system state. Throughout this time slot, the server sets the quality
levels of video sequences to be transmitted as the optimal one and transmits them to the
MT.
A major difference between the dynamic programming algorithm and the adaptive
control algorithm lies in their estimation horizons. The dynamic programming algorithm
has the information about the network throughout the whole length of N , and utilizes
this information in order to achieve global optimality during [0, N ]. Yet the adaptive
algorithm only utilizes the information in the current cycle (though we assume the dis-
tributions of residence times don’t vary from cycle to cycle in the model in simulation
and analysis, it is not an assumption in the algorithm, and is not used by the algorithm.)
Hence the adaptive algorithm only tries to achieve local optimality within the cycle.
5.3 Simulation Results
The dynamic programming algorithm is supposed to provide optimal performance on the
Markov chain network model. While our adaptive algorithm can work with all kinds of
residence time distributions, it also works on the Markov chain model. It is then of our
interest to see how far away is the adaptive algorithm from optimum.
In the simulation, we generate realizations of T2’s and T1’s for a fixed length of
time, within each residence interval the transmission rate is constant. We run our al-
gorithm with different parameters on the generated network rate trace and the (CBR)
video trace, then calculate the expected variation in a fixed length of time, i.e. V =
E(∑
0<i<Tg(|q(i+1)−q(i)|)
T) = E(
∑0<i<T
(q(i+1)−q(i))2
T) as performance metric, where T = N .
Another metric we care about is the utilization of the transmitted data, since one of our
Chapter 5. Markov Chain Network Model 40
Parameter Value
Time slot length 1 s
T 150 s
R1 1 Mbps
R2 6 Mbps
r0 6 Mbps
p11, p22 0.1
T1, T2 10 s
NA 8
NB 17
Table 5.1: Simulation parameters for 2-zone Markov model
objectives is to achieve high data utilization over time.
Some parameters we used in the simulation are listed in Table 5.1. NA denotes the
quantization level of control actions (number of control actions), and NB denotes the
quantization level of buffered video length. Notice that, NB has to be increasing with
NA, otherwise, two control actions on one state may lead to a transition to states with
the same buffered length of video, but different variations, so the control action (quality
level) with the smaller variation will always be selected, and the other one (usually the
largest one) will never be selected by the algorithm.
To ensure a fair comparison, the outputs of the adaptive algorithm and the simple
algorithm are quantized using the same quantization level.
The performance of dynamic programming algorithm as a function of α is shown in
Figure 5.1. As we can see, since α represents the weight of the residual buffer part in the
total cost, as α increases, the data utilization also increases, and the expected variation
increases slightly.
The performance of adaptive algorithm and simple algorithm as a function of β is
Chapter 5. Markov Chain Network Model 41
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80.018
0.02
0.022
0.024
0.026
0.028
0.03
0.032
α
V—
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8
0.97
0.975
0.98
0.985
0.99
0.995
1
Uti
liza
tio
n −
−
Figure 5.1: DP: variation and utilization vs. α
0.03 0.032 0.034 0.036 0.038 0.04 0.0420
0.02
0.04
0.06
0.08
0.1
0.12
0.14
β
V—
0.03 0.032 0.034 0.036 0.038 0.04 0.042
0.8
0.82
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
Uti
liza
tio
n −
−
Adaptive
Simple
Figure 5.2: Adaptive algorithm: variation and utilization vs. β
shown in Figure 5.2. β is the proportional gain between the input and the “error” in the
negative feedback system, hence adjusts the amount of variation at each step. From the
figure we can see that, there is an optimal operating point of β for a specific set of network
model parameters, when β is too small, the variation at each step is too small. Although
it offers a small total variation, the utilization can also be low because the cumulative
adjusted video rate cannot track the cumulative transmission rate quite closely.
Chapter 5. Markov Chain Network Model 42
0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.090.92
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1
V
Uti
liza
tio
n
DP
Adaptive
Simple
Figure 5.3: Comparison between algorithms: utilization vs. variation
While when β is too large, the variation at each step is also too large, thus may cause
a certain amount of oscillation in the adjusted video rate and significantly deteriorates
the performance in terms of both variation and data utilization. The data utilization
becomes lower because if the control output (q(i)) is too high due to the oscillation,
and the network transmission rate cannot sustain that quality level, only the highest
sustainable quality will be chosen, hence the cumulative adjusted video rate cannot track
the cumulative transmission rate closely. Generally, the choice of β is determined by a
lot of parameters, such as the distribution of the network, the MT’s moving speed and
moving pattern, the average video source rate and network transmission rates at different
zones, etc. Hence it is empirical and can be determined through experiments. Since the
distribution of networks is relatively stable, the proper choices of β under different MT
parameter sets can be stored in a look-up table in the coordinating BS or AP which is
providing other network-wise information to the MT and can be regularly updated.
To compare both types of algorithms, we draw the relationship between the time-
averaged variation and the data utilization in Figure 5.3.
Since our objectives are high utilization and low variation, the more upper-left the
curve is, the better the algorithm’s performance is. Dynamic programming algorithm
Chapter 5. Markov Chain Network Model 43
achieves very high utilization and lowest variation at the same utilization level. The
performance of adaptive control algorithm depends heavily on the choice of parameter,
yet the optimal operating point provide near-optimal variation performance at some
acceptable utilization levels. Also, the performance of adaptive algorithm is much better
than that of the simple algorithm.
5.4 More Realistic 3-Zone Network Model
In the previously used 2-zone Markov chain network model, we assume constant bit-rate
in each sub-network, and assume exponentially distributed residence times. While for
T1N which covers large area, we may make the assumption of constant bit-rate in the
current moving area of the MT, we can hardly make the same assumption for the T2N
network. This is because the T2N we are considering only covers local area, and the
supported transmission rates may vary dramatically with the distance between MT and
AP. Thus we consider a more realistic network model in this chapter, where the T2N is
assumed to support multiple transmission rates at different ranges (zones). We adopt a
simple 2-zone T2N model here, as shown in Figure 5.4. We denote the two zones within
T2N as T2No for outer zone, and T2Ni for inner zone. Hence our network model becomes
a 3-zone model.
Also, it is not generally true that the residence times are exponentially distributed (or
with other commonly used distributions) in a two-tier integrated wireless network such
as the one we are considering. Hence it is of our interest to explore how our proposed
algorithms perform under more realistic residence time distributions. We generate the
actual moving traces for MT in the network with a simple mobility model, and then
obtain the samples of the residence times within each zone along the traces.
One example of the generated map and MT’s moving trace is shown in Figure 5.5.
The map is repetitive so that the MT will reach the other end and continue moving if
Chapter 5. Markov Chain Network Model 44
T1N
BS
T2N
1r
2r
Low Rate
High Rate
Figure 5.4: Integrated two-tier network with 2-zone T2N
0 50 100 150 200 250 3000
50
100
150
200
250
300
Figure 5.5: An example of the generated user’s moving trace
it passes the border of the map. The moving trace is generated by a simple mobility
model: the MT picks a random point to start first, then at every step, it picks a random
direction and random speed (with in a reasonable range [Vmin, Vmax]), and moves toward
that direction for an exponentially distributed length of time (with mean Ts).
Chapter 5. Markov Chain Network Model 45
By simulating enough moving traces of the MT on each map, the distributions of
the samples of residence times approximate to the real distribution. As we can see in
Figure 5.6, the distribution for T1 is near exponential, while the distributions for T2i and
T2o are far from exponential. While the distributions of residence times are generally
affected by the mobility models and the coverage map, it is out of our scope to discuss
about the realistic types of distributions in this thesis. So we rely on the simplified model
for generating more realistic residence time distributions than exponential distribution.
Again, we want to claim that our adaptive algorithm does not rely on the structure of
the specific distribution hence can work on any type of distribution in the real world.
5.5 PH-Fitting of Residence Times
While the adaptive algorithm only requires the mean values of residence times and hence
can work directly on the generated distribution, the dynamic programming algorithm
requires a structure of Markov chain and more parameters such as the transition prob-
abilities between any two zones. We adopt Phase-Type (PH) distribution, as suggested
and evaluated by [34], as a modeling tool to convert the generated distributions into a
Markov chain.
In general, a PH distribution is the distribution of the time until absorption in a
Markov chain. The states in this Markov chain are called “phases”, which generally
have no physical meaning. PH distributions are highly versatile and can be used to
approximate any distribution of non-negative random variables. A PH distribution can
be described by (~τ ,Q), where ~τ is the initial distribution vector, and Q is the infinitesimal
generator. We use a fitting tool called EMPHT [4] to fit the PH distributions. An
example of the original and PH-fitted CDF’s for T1, T2o (residence time in outer region
of T2N) and T2i (residence time in inner region of T2N) are shown in Figure 5.6. Each
variable is fitted using a 2-phase PH distribution. As we can see, the CDF for T1 is quite
Chapter 5. Markov Chain Network Model 46
0 50 100 1500
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Distribution function
− input, − − fitted PH
0 10 200
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Distribution function
− input, − − fitted PH
0 10 20 30 400
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1Distribution function
− input, − − fitted PH
Figure 5.6: CDF’s of residence times in different zones
well approximated. Yet the fitted distributions of T2i and T2o are not that accurate.
While increasing the numbers of phases will increase the accuracy of the model, it will
also increase the state space and the complexity of the dynamic programming algorithm.
On the other hand, our experiments show that there will be no significant improvement
of accuracy in terms of the value of the log-likelihood function in the EMPHT algorithm,
when we increase the number of phases to 12 or more, in comparison to the 2-phase
model. Hence we keep using 2 phases to fit the residence times in each zone.
After fitting the residence time distributions individually, we need to combine them
together to obtain the Markov chain model of the integrated network. Figure 5.7 gives
an illustration of this Markov chain. Denote the transition probability matrix (between
different phases) within each zone and the initial distribution for each zone as {T1, ~τ1},
{T2o, ~τ2o} and {T2i, ~τ2i}, then the transition probability matrix for the Markov chain of
the integrated network would be:
T =
T1 p12to~τo 0
p21t1~τ1 To poiti~τi
0 pioto~τo Ti
where p12 = pio = 1, and the value of p21 and poi can be calculated as p21 =
v21
v21+voi, poi = voi
v21+voi, with v21 and voi representing the numbers of transitions from T2N
Chapter 5. Markov Chain Network Model 47
11P
12P
2 ,1oP
2 ,2oP
2 ,1iP
2 ,2iP
oip
21p
Figure 5.7: PH-fitted Markov chain network model
to T1N and from T2No to T2Ni in our collected traces, respectively.
Now we have finished constructing the Markov chain on which we can run the dynamic
programming algorithm.
5.6 Estimation in Adaptive Control Algorithm
The dynamic programming algorithm can directly run on the 3-zone model. However,
since the adaptive algorithm is designed for the 2-zone model, when it runs on the 3-zone
model we need some modifications on the residence time and transmission rate estimation
part. There are two types of information we can utilize to estimate the residence times.
5.6.1 Utilizing Statistical Information
In one approach, we can use the statistical information, i.e. the average residence times,
as estimations. To extend the adaptive algorithm to the new model, we still treat the
T2N as a homogeneous zone, but change the way to calculate the average residence time
and the average transmission rate in T2N. Since now we have two zones within T2N, the
estimation of residence time in T2N for the adaptive algorithm is not directly given as
the average residence time. Instead, we may have the average residence times with in
each zone T2i, T2o , as well as the transition probabilities between zones poi = voi
v21+voiand
Chapter 5. Markov Chain Network Model 48
pio = 1. Thus we can calculate the T2(0) in each cycle as follows:
T2(0) = T2 =∞∑
j=0
[T2o + j · (T2i + T2o)] · pjoi · p21
= p21[T2o ·1
1 − poi
+ (T2i + T2o) ·poi
(1 − poi)2 ]
= p21[T2i ·poi
(1 − poi)2 + T2o ·
1
(1 − poi)2] (5.7)
The estimations at the following time slots is similar to what we described in Chapter 4.
Similarly, we calculate the average transmission rate inside the entire T2N and use it as
the estimation of transmission rate in the adaptive algorithm.
5.6.2 Utilizing Localization Information
With the network model generated by actual mobility trace, we are able to obtain localiza-
tion information of the MT at every time slot, hence we can also utilize the localization
information to estimate the residence time. We make following assumptions when we
simulate the case of utilizing the localization information:
1. The MT’s moving direction and speed, as well as its position at current time slot
are known.
2. The position of each T2N AP and their coverage areas (including both T2No and
T2Ni) are known.
3. The MT moves in straight line most of the time.
When estimating the residence time in each zone, the MT is expected to move toward
current direction with current speed until it performs the next T1N-to-T2N VHO. Thus
we can calculate the residence time in each zone by dividing the distances in each zone
by the moving speed.
Chapter 5. Markov Chain Network Model 49
Parameter Value
Time slot length 1 s
T 150 s
R1 1 Mbps
R2i, R2o 3 Mbps, 6 Mbps
r0 6 Mbps
Vmin, Vmax 3.5 m/s, 5 m/s
r1, r2 30 m, 20 m
Ts 20 s
NA 8
NB 15
Table 5.2: Simulation parameters for 3-zone model
5.6.3 Simulation Results for 3-Zone Model
The parameters we used in 3-zone network model simulation are listed in Table 5.2. The
transition probability matrix we obtain from the above mentioned PH-fitting method
with 2 phases in each zone is as follows:
T =
0.8350 0.1506 0 0.0144 0 0
0 0.8350 0 0.1650 0 0
0.0877 0 0.7662 0 0.0651 0.0810
0.0112 0 0.2041 0.7662 0.0083 0.0102
0 0 0 0.0592 0.9073 0.0335
0 0 0 0.0115 0.0313 0.9572
The performance of each algorithm on the 3-zone model is shown in Figures 5.8- 5.10.
In Figure 5.9, we also draw the performance curve of the adaptive and simple algorithm
running on the actual network model instead of the PH-fitted one. We can see that the
Chapter 5. Markov Chain Network Model 50
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80.012
0.014
0.016
0.018
0.02
0.022
0.024
0.026
0.028
0.030 0.1 0.2 0.3 0.4 0.5
0.92
0.93
0.94
0.95
0.96
0.97
0.98
0.99
1
Uti
liza
tio
n −
−
V—
α
Figure 5.8: DP on 3-zone model: variation and utilization vs. α
two curves for the same algorithm are reasonably near each other and exhibit the same
trend, which shows that the PH-fitting process does not incur a significant amount of
error when comparing the performance of the algorithms. Furthermore, it shows that
the adaptive algorithm performs better on actual model than on the PH-fitted one, thus
it is advantageous in the realistic problem we are considering.
Similar trends exhibited for the 3-zone model as for the 2-zone model. The data
utilization of the adaptive algorithm is significantly lower. This is because we use the ex-
pected values of residence time and transmission rate over the whole T2N as estimations,
the estimations are more inaccurate.
For the dynamic programming algorithm running on the 3-zone model, it was difficult
for us to generate more results when V < 0.013, which is primarily because of the quan-
tization level we were using. The simulation was conducted on a Dell PowerEdge 1950
server with dual dual-core Intel Xeon 3.0 GHz processors and 2GB memory, and further
increasing the dimensionality in this simulation would give us a “OUT OF MEMORY”
error. However, judging from the trend of the curve we can still believe that the dynamic
programming generates less variation than the adaptive and simple algorithms at lower
Chapter 5. Markov Chain Network Model 51
0.015 0.02 0.025 0.03 0.035 0.04 0.0450
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
0.2
β
V—
,--
0.015 0.02 0.025 0.03 0.035 0.04 0.045
0.82
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
Utilization––
Adaptive(A)
Adaptive(M)
Simple(A)
Simple(M)
Figure 5.9: Adaptive algorithm on 3-zone model: variation and utilization vs. β
0 0.02 0.04 0.06 0.08 0.1 0.120.93
0.94
0.95
0.96
0.97
0.98
0.99
1
V
Uti
liza
tio
n
Adaptive
Simple
DP
Figure 5.10: Comparison DP and AA: utilization vs. variation
data utilization area.
Chapter 5. Markov Chain Network Model 52
Parameter Value
Time slot length 1 s
T 500 s
R1 0.453 Mbps
R2i, R2o 1.520 Mbps, 3.333 Mbps
Table 5.3: Simulation parameters for VBR network and VBR video
5.7 Simulating with VBR Network and VBR Video
Stream
As we mentioned before, our adaptive algorithm can work with VBR network and VBR
video as well. While it is theoretically possible for the dynamic programming based
algorithm to work with VBR cases (as long as we use Markov process to describe the
network ABR and the video source rate), it would be either extremely complex in terms
of computation, if the models are accurate and complex, or far from optimal, if the
models are simple and inaccurate. Hence we only simulate and compare with the simple
adaptive algorithm to show the effect of different types of long-term estimation. We
simulate the VBR cases on the 3-zone network model, where T2N supports two ranges
of transmission rates.
In every simulation, we use a random part of the “Tokyo Olympics” video trace
provided by the authors of [28] to evaluate our algorithm. The average bit rate of this
video trace is 3.71Mbps. The channel rates are generated randomly within each zone,
with different average rates. Parameters used in the simulation are listed in Table 5.3.
Figure 5.11 shows one instance out of many simulations when we run adaptive al-
gorithm and use statistical information to estimate the residence times. In this set of
simulation, β = 0.01.
Figure 5.12 shows the same simulation instance running the algorithm with local-
Chapter 5. Markov Chain Network Model 53
0 50 100 150 200 250 300 350 400 450 5000
0.5
1
1.5
2
2.5
3
3.5
4
Network Rate (Mbps)
Video Rate (Mbps)
Quality Level
t
Figure 5.11: VBR simulation: adaptive algorithm with statistical information
0 50 100 150 200 250 300 350 400 450 5000
0.5
1
1.5
2
2.5
3
3.5
4
Network Rate (Mbps)
Video Rate (Mbps)
Quality Level
t
Figure 5.12: VBR simulation: adaptive algorithm with localization information
ization information. We use similar map as the one in Figure 5.5 to generate the MT’s
moving traces and the localization information. In this set of simulation, we use the same
parameters as listed above, β = 0.01.
Figure 5.13 shows the result of simple algorithm (without estimation of the residence
Chapter 5. Markov Chain Network Model 54
0 50 100 150 200 250 300 350 400 450 5000
1
2
3
4
5
6
Network Rate (Mbps)
Video Rate (Mbps)
Quality Level
t
Figure 5.13: VBR simulation: simple adaptive algorithm
times) with same parameter β = 0.01.
Comparing the three results, we can see that the adjusted video source rate curves
are much smoother when we run the adaptive algorithm with either kind of estimation.
Averaging 1500 runs of experiments, we have the average performance with both types
of estimation under different parameters, shown in Figures 5.14 - 5.16. We can see that,
estimation with statistical information (i.e. mean values) leads to lower variations most
of the time, but also with lower data utilization. This is because the amount of buffer
left in the end depends on the accuracy of the estimation. Generally the localization
information provides more accurate estimation, hence utilizing localization information
leads to better data utilization but it also causes large variation when the MT changes
the direction abruptly.
Besides, the results indicate that either kind of information will be helpful in estimat-
ing the residence time and smoothing the video quality to overcome long-term variations
in video adaptation. In our specific setting, the localization information might be more
accurate on an average basis. Yet when the MT’s moving trace is more unpredictable,
utilizing localization information may not have so much advantage over using statistical
Chapter 5. Markov Chain Network Model 55
0 0.005 0.01 0.015 0.02 0.0251
2
3
4
5
6
7
8
9x 10
−3
β
V
Localization
Simple
Statistical
Figure 5.14: Simulating VBR case - variation vs. β
0 0.005 0.01 0.015 0.02 0.0250.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
β
Util
izat
ion
LocalizationSimpleStatistical
Figure 5.15: Simulating VBR case - utilization vs. β
information. With a proper choice of parameter, the adaptive control algorithm can
reduce the variation in quadratic form by 80% or more.
Chapter 5. Markov Chain Network Model 56
0 0.005 0.01 0.015 0.02 0.0250.88
0.89
0.9
0.91
0.92
0.93
0.94
0.95
0.96
V
Uti
liza
tio
n
Localization
Simple
Statistical
Figure 5.16: Simulating VBR case - utilization vs. variation
Chapter 6
Conclusion
In this thesis, we have proposed an adaptive control algorithm for video streaming over
heterogeneous wireless networks with dramatically different available channel bit rates.
The proposed approach combines source rate adaptation with buffering and employs
certain information of the network, such as the expected residence time in each network
and the location of mobile device and access points.
By modeling the streaming process over an integrated two-tier network using a frame-
work based on Markov chain with associated rewards (costs), the theoretical analysis for
CBR video, CBR channel case shows a significant reduction in video quality variation in
comparison to some intuitive and complex algorithms.
Modeling the network using a Markov chain allows us to run a dynamic programming
based algorithm to adjust the video source rate, the result of which is supposed to be
optimal under our assumptions. In order to model the more complicated and realistic
heterogeneous wireless network, we further introduced Phase-Type fitting of the resi-
dence times and extended the algorithms to a 3-zone network model. Simulations with
both 2-zone and 3-zone models indicate that our adaptive algorithm gives near-optimal
performance with acceptable data utilization. At the same time, it has the advantage of
low computation and requires less information of the network.
57
Chapter 6. Conclusion 58
Further in the simulations of the VBR cases, the result shows that by utilizing statisti-
cal or localization information we are able to reduce the quality variation of the streamed
VBR video by 80% or more under the quadratic form metric.
From these results, we believe that although the proposed approach is simple and
intuitive, it can improve the quality of streamed video significantly in terms of quality
variation, and is promising in achieving constant-quality high-rate video streaming over
heterogeneous wireless networks.
Bibliography
[1] T. Alpcan, J. P. Singh, and T. Basar. Robust rate control for heterogeneous net-
work access in multihomed environments. IEEE Transactions on Mobile Computing,
8(1):41–51, 2009.
[2] J. G. Apostolopoulos, W.-T. Tan, and S. J. Wee. Video streaming: Concepts,
algorithms, and systems. Technical report, HP Laboratories, 2002, 2002.
[3] S. Aramvith, I.-M. Pao, and M.-T. Sun. A rate-control scheme for video trans-
port over wireless channels. IEEE Transactions on Circuits and Systems for Video
Technology, 11(5):569–580, May 2001.
[4] S. Asmussen, O. Nerman, and M. Olsson. Fitting phase-type distribution via the
EM algorithm. Scand.J.Statist., 23:419–441, 1996.
[5] L. Atzori, M. Krunz, and M. Hassan. Cycle-based rate control for one-way and
interactive video communications over wireless channels. IEEE Transactions on
Multimedia, 9(1):176–184, Jan. 2007.
[6] M. Bernaschi, F. Cacace, R. Clementelli, and L. Vollero. Adaptive streaming on
heterogeneous networks. In WMuNeP ’05: Proceedings of the 1st ACM workshop on
Wireless multimedia networking and performance modeling, pages 16–23, New York,
NY, USA, 2005. ACM.
59
Bibliography 60
[7] H.-C. Chuang, C. Huang, and T. Chiang. Content-aware adaptive media playout
controls for wireless video streaming. Multimedia, IEEE Transactions on, 9(6):1273
–1283, oct. 2007.
[8] Allot Communications. The allot mobiletrends report. http://www.allot.com/
mobiletrends.html, 2009.
[9] P. de Cuetos and K. W. Ross. Adaptive rate control for streaming stored fine-grained
scalable video. In NOSSDAV ’02: Proceedings of the 12th international workshop
on Network and operating systems support for digital audio and video, pages 3–12,
New York, NY, USA, 2002. ACM.
[10] P. de Cuetos and K. W. Ross. Optimal streaming of layered video: joint scheduling
and error concealment. In MULTIMEDIA ’03: Proceedings of the eleventh ACM
international conference on Multimedia, pages 55–64, New York, NY, USA, 2003.
ACM.
[11] M.U. Demircin, P. van Beek, and Y. Altunbasak. Delay-constrained and r-d opti-
mized transrating for high-definition video streaming over wlans. Multimedia, IEEE
Transactions on, 10(6):1155 –1168, oct. 2008.
[12] P. Frossard and O. Verscheure. Joint source/fec rate selection for quality-optimal
mpeg-2 video delivery. Image Processing, IEEE Transactions on, 10(12):1815 –1825,
dec 2001.
[13] V.K. Goyal. Multiple description coding: compression meets the network. Signal
Processing Magazine, IEEE, 18(5):74 –93, sep 2001.
[14] M. Hassan and M. Krunz. Video streaming over wireless packet networks: An
occupancy-based rate adaptation perspective. IEEE Transactions on Circuits and
Systems for Video Technology, 17(8):1017–1027, Aug. 2007.
Bibliography 61
[15] G. Ji and B. Liang. Buffer schemes for vbr video streaming over heterogeneous
wireless networks. In IEEE International Conference on Communications (ICC),
Dresden, Germany, June 2009.
[16] G. Ji and B. Liang. Stochastic rate control for scalable vbr video streaming over
wireless networks. In Global Telecommunications Conference, 2009. GLOBECOM
2009. IEEE, pages 1 –6, nov. 2009.
[17] W. Li. Overview of fine granularity scalability in mpeg-4 video standard. IEEE
Transactions on Circuits and Systems for Video Technology, 11(3):301–317, March
2001.
[18] Y. Li, A. Markopoulou, J. Apostolopoulos, and N. Bambos. Content-aware playout
and packet scheduling for video streaming over wireless links. Multimedia, IEEE
Transactions on, 10(5):885 –895, aug. 2008.
[19] G. Liang and B. Liang. Effect of delay and buffering on jitter-free streaming over
random vbr channels. Multimedia, IEEE Transactions on, 10(6):1128 –1141, oct.
2008.
[20] K. McArthur. The future is here: Next is now. http://redboard.rogers.com/
2010/the-future-is-here-next-is-now/, 2010.
[21] Cisco VNI Mobile. Cisco visual networking index: Global mobile data traffic fore-
cast update, 2009-2014. https://www.cisco.com/en/US/solutions/collateral/
ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html, 2010.
[22] A. Ortega and M. Khansari. Rate control for video coding over variable bit rate
channels with applications to wireless transmission. In ICIP ’95: Proceedings of the
1995 International Conference on Image Processing (Vol. 3)-Volume 3, page 3388,
Washington, DC, USA, 1995. IEEE Computer Society.
Bibliography 62
[23] M. Qin and R. Zimmermann. Improving mobile ad-hoc streaming performance
through adaptive layer selection with scalable video coding. In MULTIMEDIA ’07:
Proceedings of the 15th international conference on Multimedia, pages 717–726, New
York, NY, USA, 2007. ACM.
[24] H. Schwarz, D. Marpe, and T. Wiegand. Overview of the scalable video coding
extension of the h.264/avc standard. IEEE Transactions on Circuits and Systems
for Video Technology, 17(9):1103–1120, September 2007.
[25] Y. Sun, I. Ahmad, D. Li, and Y.-Q. Zhang. Region-based rate control and bit
allocation for wireless video transmission. Multimedia, IEEE Transactions on, 8(1):1
– 10, feb. 2006.
[26] T. Taleb, K. Kashibuchi, A. Leonardi, S. Palazzo, K. Hashimoto, N. Kato, and
Y. Nemoto. A cross-layer approach for an efficient delivery of tcp/rtp-based mul-
timedia applications in heterogeneous wireless networks. IEEE Transactions on
Vehicular Technology, 57(6):3801–3814, Nov. 2008.
[27] Y.-H. Tseng, E.H.-K. Wu, and G.-H. Chen. Scene-change aware dynamic bandwidth
allocation for real-time vbr video transmission over ieee 802.15.3 wireless home net-
works. Multimedia, IEEE Transactions on, 9(3):642 –654, april 2007.
[28] G. Van der Auwera, P. T. David, M. Reisslein, and L. J. Karam. Traffic and quality
characterization of the h.264/avc scalable video coding extension. Adv. MultiMedia,
2008(2):1–27, 2008.
[29] M. van der Schaar and H. Radha. A hybrid temporal-snr fine-granular scalability
for internet video. IEEE Trans. on CSVT, 11:318–331, 2001.
[30] P. Vidales, J. Baliosian, J. Serrat, G. Mapp, F. Stajano, and A. Hopper. Autonomic
system for mobility support in 4g networks. Selected Areas in Communications,
IEEE Journal on, 23(12):2288 – 2304, dec. 2005.
Bibliography 63
[31] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the
h.264/avc video coding standard. IEEE Transactions on Circuits and Systems for
Video Technology, 13(7):560–576, July 2003.
[32] Wikipedia. PID controller. http://en.wikipedia.org/wiki/PID_controller#
PID_controller_theory.
[33] J. Xu, X. Shen, J.W. Mark, and J. Cai. Adaptive transmission of multi-layered
video over wireless fading channels. Wireless Communications, IEEE Transactions
on, 6(6):2305 –2314, june 2007.
[34] A. H. Zahran and B. Liang. Zone residence time mobility modeling and performance
evaluation framework for heterogeneous multi-tier wireless networks.
[35] A.H. Zahran and C.J. Sreenan. Cost efficient media streaming algorithms for rate-
dependent pricing strategies in heterogeneous wireless networks. In Next Generation
Mobile Applications, Services and Technologies, 2008. NGMAST ’08. The Second
International Conference on, pages 485 –491, 16-19 2008.
[36] A.H. Zahran and C.J. Sreenan. Pgms: Pseudo-optimal greedy media streaming al-
gorithm for heterogeneous wireless networks. In Networking and Communications,
2008. WIMOB ’08. IEEE International Conference on Wireless and Mobile Com-
puting,, pages 390 –396, 12-14 2008.
[37] A.H. Zahran and C.J. Sreenan. Threshold-based media streaming optimization
for heterogeneous wireless networks. Mobile Computing, IEEE Transactions on,
9(6):753 –764, june 2010.
[38] X. Zhu and B. Girod. Distributed rate allocation for video streaming over wireless
networks with heterogeneous link speeds. In IWCMC ’07: Proceedings of the 2007
international conference on Wireless communications and mobile computing, pages
296–301, New York, NY, USA, 2007. ACM.