Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet...

20
Hindawi Publishing Corporation International Journal of Digital Multimedia Broadcasting Volume 2013, Article ID 319594, 20 pages http://dx.doi.org/10.1155/2013/319594 Research Article Performance Evaluation of Concurrent Multipath Video Streaming in Multihomed Mobile Networks James Nightingale, Qi Wang, and Christos Grecos Centre for Audio-Visual Communications and Networks, School of Computing, University of the West of Scotland, Paisley PA1 2BE, UK Correspondence should be addressed to James Nightingale; [email protected] Received 30 December 2012; Revised 9 July 2013; Accepted 15 July 2013 Academic Editor: Fabrice Labeau Copyright © 2013 James Nightingale et al. is is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. High-quality real-time video streaming to users in mobile networks is challenging due to the dynamically changing nature of the network paths, particularly the limited bandwidth and varying end-to-end delay. In this paper, we empirically investigate the performance of multipath streaming in the context of multihomed mobile networks. Existing schemes that make use of the aggregated bandwidth of multiple paths can overcome bandwidth limitations on a single path but suffer an efficiency penalty caused by retransmission of lost packets in reliable transport schemes or path switching overheads in unreliable transport schemes. is work focuses on the evaluation of schemes to permit concurrent use of multiple paths to deliver video streams. A comprehensive streaming framework for concurrent multipath video streaming is proposed and experimentally evaluated, using current state- of-the-art H.264 Scalable Video Coding (H.264/SVC) and the next generation High Efficiency Video Coding (HEVC) standards. It provides a valuable insight into the benefit of using such schemes in conjunction with encoder specific packet prioritisation mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent multipath schemes are identified, and the challenges in realising HEVC based concurrent multipath streaming are highlighted. 1. Introduction One of the main challenges in video streaming is to intelli- gently adapt the stream in response to dynamically changing network conditions in a way that attempts to minimise the distortion effect on the received video of adverse network conditions. H.264 Scalable Video Coding (H.264/SVC) [1], the scalable extension to the H.264 Advanced Video Coding standard (H.264/AVC) [2], has emerged as a promising means of adapting a video stream to prevailing network conditions but has yet to be adopted on a large scale for delivery of streamed video content. In H.264/SVC, a video stream consists of a number of scalable layers, each of which can be dropped either partially or in its entirety to adapt to changing network path conditions. Adapting an H.264/SVC stream to a change in the available bandwidth on a single wired network path is a well-understood problem; solutions that drop entire scalable layers [3] or individual packets from a layer [4] have both been previously proposed. H.264/AVC is the current, widely deployed, video encod- ing standard, and its extensions such as H.264/SVC represent the current state of the art. However, work on a replacement for the H.264 family of encoding standards has reached the international standard stage with the next generation High Efficiency Video Coding (HEVC) standard published by the ITU as H.265 [5] in June 2013. HEVC offers the same visual quality as H.264/AVC while reducing the bandwidth requirement by 50%. e initial HEVC standard permits in- network adaptation of a video stream by employing temporal scalability by means of temporal prediction sublayers. A scalable extension to HEVC is also planned with work on the development having begun in December 2012 [6]. For situations where a single network path does not pro- vide sufficient bandwidth to deliver a video stream, schemes that use the aggregated bandwidth of multiple paths [712] have been proposed. Nomadic users of streamed video con- tent face the additional challenges associated with delivery over wireless mobile networks. A mobile network connected

Transcript of Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet...

Page 1: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

Hindawi Publishing CorporationInternational Journal of Digital Multimedia BroadcastingVolume 2013 Article ID 319594 20 pageshttpdxdoiorg1011552013319594

Research ArticlePerformance Evaluation of Concurrent Multipath VideoStreaming in Multihomed Mobile Networks

James Nightingale Qi Wang and Christos Grecos

Centre for Audio-Visual Communications and Networks School of Computing University of the West of ScotlandPaisley PA1 2BE UK

Correspondence should be addressed to James Nightingale jamesnightingaleuwsacuk

Received 30 December 2012 Revised 9 July 2013 Accepted 15 July 2013

Academic Editor Fabrice Labeau

Copyright copy 2013 James Nightingale et al This is an open access article distributed under the Creative Commons AttributionLicense which permits unrestricted use distribution and reproduction in any medium provided the original work is properlycited

High-quality real-time video streaming to users in mobile networks is challenging due to the dynamically changing nature ofthe network paths particularly the limited bandwidth and varying end-to-end delay In this paper we empirically investigatethe performance of multipath streaming in the context of multihomed mobile networks Existing schemes that make use of theaggregated bandwidth ofmultiple paths can overcome bandwidth limitations on a single path but suffer an efficiency penalty causedby retransmission of lost packets in reliable transport schemes or path switching overheads in unreliable transport schemes Thiswork focuses on the evaluation of schemes to permit concurrent use of multiple paths to deliver video streams A comprehensivestreaming framework for concurrent multipath video streaming is proposed and experimentally evaluated using current state-of-the-art H264 Scalable Video Coding (H264SVC) and the next generation High Efficiency Video Coding (HEVC) standardsIt provides a valuable insight into the benefit of using such schemes in conjunction with encoder specific packet prioritisationmechanisms for quality-aware packet scheduling and scalable streaming The remaining obstacles to deployment of concurrentmultipath schemes are identified and the challenges in realising HEVC based concurrent multipath streaming are highlighted

1 Introduction

One of the main challenges in video streaming is to intelli-gently adapt the stream in response to dynamically changingnetwork conditions in a way that attempts to minimise thedistortion effect on the received video of adverse networkconditions H264 Scalable Video Coding (H264SVC) [1]the scalable extension to the H264 Advanced Video Codingstandard (H264AVC) [2] has emerged as a promisingmeans of adapting a video stream to prevailing networkconditions but has yet to be adopted on a large scale fordelivery of streamed video content In H264SVC a videostream consists of a number of scalable layers each of whichcan be dropped either partially or in its entirety to adapt tochanging network path conditions Adapting an H264SVCstream to a change in the available bandwidth on a singlewired network path is a well-understood problem solutionsthat drop entire scalable layers [3] or individual packets froma layer [4] have both been previously proposed

H264AVC is the current widely deployed video encod-ing standard and its extensions such as H264SVC representthe current state of the art However work on a replacementfor the H264 family of encoding standards has reachedthe international standard stage with the next generationHigh Efficiency Video Coding (HEVC) standard publishedby the ITU as H265 [5] in June 2013 HEVC offers the samevisual quality as H264AVC while reducing the bandwidthrequirement by 50 The initial HEVC standard permits in-network adaptation of a video stream by employing temporalscalability by means of temporal prediction sublayers Ascalable extension to HEVC is also planned with work on thedevelopment having begun in December 2012 [6]

For situations where a single network path does not pro-vide sufficient bandwidth to deliver a video stream schemesthat use the aggregated bandwidth of multiple paths [7ndash12]have been proposed Nomadic users of streamed video con-tent face the additional challenges associated with deliveryover wireless mobile networks A mobile network connected

2 International Journal of Digital Multimedia Broadcasting

to multiple potential access networksrsquo paths (referred to as amultihomed mobile network) between the server and clientpresents a challenging environment for the delivery of delay-sensitive traffic such as video streaming Of the multipathstreaming schemes proposed in the literature to the best ofour knowledge only [12 13] are specifically designed for theuse with H264SVC scalable video streams in the NEMO[14] based multihomed mobile networks environment It isnoted that any scheme that can deliver high-quality videocontent such as streaming in (or near to) real time in thedownlink direction could potentially be explored in thereverse uplink direction for example to provide real-timepassenger safety monitoring over the aggregated bandwidthof all available network paths in public transportation sce-narios The possibility of such use is considered in thedesign of the proposed system The work in [12] is based onthe ldquoAlways Best Connectedrdquo (ABC) model for multihomedmobile networks proposed byWang et al in [15] In the ABCmodel an application flow for example a video stream isldquoswitchedrdquo from one network path to another based on apolicy-drivenmechanism that considers both the applicationflow characteristics and the current network path conditionsThe path selection component of the scheme proposed in[12] is an H264SVC-specific instance of the general modeldescribed in [15] Therefore although all available networkpaths can be used simultaneously (for the delivery of differentapplication flows) any given application flow is in factsequentially switched between paths by the path selectionand packet scheduling algorithm operating at the streamingserver

The Stream Control Transmission Protocol (SCTP) [16]is an alternative networking protocol that supports multi-homing SCTP following a host-to-host approach differsfrom NEMO in that it only supports individual multihomedhosts NEMO in contrast is able to support multihomedmobile routers that in turn provide mobility support for allof the mobile hosts within the mobile network in a muchmore efficient fashionMobile hosts inNEMOnetworks neednot be mobility aware or multihomed themselves whereasSCTP multihomed mobile hosts manage their own mobilityas well as multihoming A proposed extension [17] to theSCTP standard introduces a concurrent multipath transfermechanism cmt-SCTP In cmt-SCTP an application flowis split into a number of substreams each of which isconcurrently transmitted on a different end-to-end networkpathwithin an SCTP association Due to the reliable nature ofthe SCTP protocol the concurrent use ofmultiple paths leadsto out-of-sequence packet delivery and the ldquoreceiver bufferblockingrdquo problem [18 19] occurs

In [13] path switching overhead was established as beingthe largest single limiting factor imposed upon the viabilityof video streaming in multihomed mobile networks environ-ment and the CMT-NEMO scheme was proposed to over-come this limitation In this paper we significantly extend thework in [13] by providing a comprehensive analysis of con-currentmultipath streaming inmultihomedmobile networkswhich includes an investigation of the effects of backgroundtraffic on the CMT-NEMO scheme introduced in [13] Thiswork also extends the evaluation of concurrent multipath

streaming to include additional metrics of jitter and delaywhich have not previously been evaluated for concurrentmultipath transmission in mobile networks Additionallylooking to the future this work provides a timely empiricalinvestigation of the use of the newly standardised videoencoding technology HEVC in concurrent multipath videostreaming We also propose a simple HEVC specific packetweighting scheme for priority-based packet scheduling andscalable packet delivery (via selective packet dropping tomeetnetwork pathsrsquo bandwidth constraint) The results of thispilot HEVC implementation provide valuable insights intothe challenges that will face researchers seeking to furtherinvestigate HEVC streaming in both single and multipathenvironments

The rest of this paper is organized as follows In Section 2related work on multihomed mobile networks H264SVCHEVC and multipath transmission is reviewed The testbedimplementation of our framework is described in Section 3and numerical results of empirical experimentation arereported in Section 4 Section 5 concludes the paper

2 Related WorkDue to the multidisciplinary nature of this work that com-bines networking communications is and video signal pro-cessing the state of the art of a number of related researchareas are reviewed as follows

21 Multihomed Mobile Networks Working groups withinthe IETF have developed standards such as Mobile IPv4[20] and Mobile IPv6 [21] for managing the mobility ofmobile nodes BothMobile IPv4 andMobile IPv6 address themobility of an individual mobile host The emerging wirelessnetworking paradigm of mobile networks is based on theIETF Network Mobility (NEMO) standard [14] In NEMOnetworks groups of users travel together in unison (eg ona bus or train) and attach to the wider infrastructure via amobile router serving as a gateway for the mobile networkThe NEMO protocol [14] further considers the mobilityrequirements of an entire moving network by extending theMobile IPv6 protocol to include its use on a new entity themobile router (MR) which manages mobility on behalf ofall the individual mobile network nodes (MNNs) within themobile network

The mobile router is referred to as being multihomedwhen it is equipped with multiple network interfaces and isable to make simultaneous use of multiple network pathsThe components of a multihomedmobile network are shownin Figure 1 In this typical scenario a mobile network nodeaway from its home link is contacted by a correspondentnode (CN) via the NEMO-enabled home agent (HA) Thehome agent maintains a binding cache table entry linking themobile routerrsquos home address (HoA) to its current care ofaddress (CoA) In the case of multihomed mobile networkswhere the MR is equipped with multiple network interfacesa third identifier called the binding identifier (BID) is usedto differentiate between the different (HoA CoA) bindingsin the binding cache Each BID is associated with a networkinterface on the MR

International Journal of Digital Multimedia Broadcasting 3

IPv6 tunnelfrom HA (interface 2)

to MR (interface 2)

Accessnetwork 2

Home agent(HA)

IPv6 tunnelfrom HA (interface 1)

to MR (interface 1)Access

network 1Correspondent

node (CN)(media server)

Mobile networknode (MNN)

Mobilenetwork

Multihomedmobile router

(MR)

Figure 1 Topology of a two-path multihomed mobile network

When a mobile network node joins the mobile networkall traffic from a CN destined for the MNN is firstly directedto the HA of the MR The packet is encapsulated in an IPv6packet giving the destination address as the MRrsquos CoA Thispacket is then transmitted over an IPv6 tunnel between theHA and the MR When the packet arrives at the MR theencapsulation is removed and the packet is forwarded to theMNN Inmultihomedmobile networks each interface on theHA is connected to its corresponding interface on the MR bya separate IPv6 tunnel Figure 1 shows a typical multihomedmobile network where the media server is the CN and thereare two distinct paths between the HA and the MR It canbeen seen that the IPv6 tunnels connect the HA to the MR

With ABC implementations of multihomed mobile net-works such as the one proposed in the MULTINET project[15] an application flow is firstly described as an associationbetween the two end points (CN and MNN) using the(protocol source IP address source port destination IPaddress and destination port) quintuple to uniquely identifythe flow Consequently by associating this flow quintuplewith a NEMO BID the flow is bound to a particular pathbetween the HA and the MR A policy-driven networkselection algorithmusesmetrics from the application flow (itsbandwidth delay or other quality of service requirements)and the current network path conditions to decide the mostappropriate path between the HA and the MR for anygiven application flow An application flow is ldquoswitchedrdquo toanother network path by changing the association (at theHA)between its unique quintuple flow identifier and the (HoACoA BID) tuple that uniquely identifies a path from HA toMR In the ABC model a single flow can only be associatedwith one network path at any given time with the flow beingswitched among paths by the network selection algorithm

22 Scalable Video Coding Video streams encoded usingthe H264SVC [1] format comprise a number of substreams(layers) An H264AVC [2] compliant base layer providesa minimum quality of video while successive enhancementlayers improve the quality of the video stream H264SVCstreams offer a three-dimensional scalability Picture resolu-tion frame rate and signal-to-noise ratio enhancement layersprovide spatial temporal and quality scalability respectively

Qua

lity

PSN

R (d

B)

32

30

28

Tem

pora

l

QCIF CIF 4CIF 16CIF Spatial

20Hz25Hz

30Hz

lowast

lowast

lowast

Bitstream 20Hz CIF and 32dB PSNR

Figure 2 Extracting an H264SVC scalable bit stream

Figure 2 shows the extraction of a scalable substream match-ing a specified spatial (CIF) temporal (20Hz) and quality(32 dB PSNR) tuple H264SVC streams are readily adaptedto meet terminal requirements or in response to changingnetwork path conditions For example the sender will onlysend those layers that the receiver is capable of processingWhere there is insufficient bandwidth to transmit the entirestream a Media-Aware Network Entity (MANE) may drophigher enhancement layers thus ensuring the delivery of themore important base layer and lower enhancement layers

Meanwhile the user is still provided with an acceptable(albeit lower) quality of received video The userrsquos quality ofexperience (QoE) is therefore managed by providing a grace-ful degradation of the stream rather than its disruption As anexample the quality enhancement layer blocks (denoted bylowast) in the extracted bit stream in Figure 2 could be droppedto reduce the bandwidth requirement while maintaining anacceptable quality of received video with the desired spatialand temporal characteristics alternatively the stream couldhave been adapted by dropping either temporal or spatiallayer or indeed some combination of the different scalablelayers

23 High Efficiency Video Coding HEVC has been publishedin June 2013 by the ITU-T as H265 the next generationvideo coding standard [5] In common with the H264 familyof standards HEVC consists of two separate layers thevideo coding layer (VCL) and the Network Abstraction Layer(NAL)TheVCL layer provides coded representations of eachpicture in a video sequence while the NAL layer provides thebasic data unit (NAL unit) in which VCL data is encapsulatedfor storage or transmission

Compression efficiency in HEVC is improved (whencompared to H264AVC) by around 50 which is achievedby the inclusion of a number of new encoding tools andthe adoption of larger coding block sizes with an adaptivequadtree structure The only form of scalability offered inthe current HEVC specification [5] is temporal scalabilityThree temporal prediction modes are used the first of which(Intraprediction Only) does not permit temporal scalabilityThe LowDelay encodingmode has an InstantaneousDecoderRefresh (IDR) picture as its first picture with all subsequent

4 International Journal of Digital Multimedia Broadcasting

pictures being generalized P and B pictures These general-ized P and B pictures are only able to use pictures that areprior to themselves in output order (have lower picture ordercount (POC)) as references pictures In the third (RandomAccess) encodingmode a hierarchical B picture structure sim-ilar to that employed in H264SVC is employed The streamand IDR picture begin with intraencoded pictures beingadded approximately one per second these Clean RandomAccess (CRA) pictures provide random access points withinthe stream Pictures that follow a CRA picture in decodingorder but precede it in output order may use pictures thatcome before the CRA for reference

The experiments undertaken for this paper were con-ducted using version 6 (HM60) of the HEVC reference soft-ware JCT-VC regularly update the HEVC reference softwarethe current version [22] is version 110 (June 2013)

24 Multipath Streaming Algorithms Using the aggregatedbandwidth of multiple network paths between sender andreceiver has been proposed as a method of delivering high-quality video streams over links with a limited bandwidthChebrolu and Rao proposed the Earliest Delivery Path First(EDPF) scheme [7] where the arrival time of a packet isestimated for each available network path and the packettransmitted on the path offering the earliest delivery timeIn [8] the same authors extended their work by includinga frame-based selective drop mechanism Fernandez et al[9] further adapted EDPF to Time Division Multiple Access(TDMA) systems by inclusion of a time-slot policy mech-anism More recently Jurca and Frossard [10] proposed ascheme that while still using an earliest path first approachconsidered the dependencies between packets in a videostream and only transmitted those packets that can besuccessfully decoded at the receiver

This was achieved by dropping any packets which relyon a previously dropped packet for decoding purposes thuspreventing the wastage of network resources on packets thatare not viable The authors of [11] identified a potentialproblem in [10] which can lead to out-of-sequence packetdelivery to the client and proposed a new algorithm toovercome this limitationThe same observation is true for allthe earliest path-type schemes Our scheme in [12] includeda mechanism to prevent the out-of-sequence packet deliveryidentified in [11] In this work we incorporate an additionalcomponent at the client to manage any out-of-sequencearrival that may still occur due to imprecise bandwidthmeasurements or transient interferences on the wireless linksin our testbed

Whilst the authors of [10] considered a generic scalablevideo stream none of these schemes have addressed thehigher levels of granularity and complex packet dependenciesfound in H264SVC-encoded streams Our previous work[12 23] considered the specific challenges associated withdelivering H264SVC over multiple paths in multihomedmobile networks The HEVC standard [5] is a very recentdevelopment and no multipath streaming algorithms specif-ically targeted at HEVC have as yet been proposed in theliterature

The additional mobility-related overheads of NEMO andthe path switching cost of ABC-based NEMO networks weremitigated in our optimised path selection and schedulingalgorithm (OPSSA) [23] A further review [13] of the per-formance of H264SVC streaming in multihomed NEMOenvironments identified the ABC-related path switchingoverhead as being a dominant limiting factor in the deliveryof video streams in this context The CMT-NEMO algorithmdescribed in Section 25 was proposed in [13] to overcomethe limitations placed on ABC-NEMO by path switchingoverheads

25 Concurrent Multipath Transmission The majority ofexisting concurrent multipath transfer (CMT) schemes arebased on the SCTP transport-layer protocol SCTP is a reli-able transport protocol which retransmits packets failing toreach the client and incorporates a congestion controlmecha-nism [16] An end-to-end SCTP association is made betweentwo multihomed SCTP hosts A number of independentpaths exist within this association In SCTP multihomingonly one of these paths may be used as the primary path fordata transfer with the other being utilized for retransmissionsand primary path failure redundancy

Iyengar et al [17] proposed the cmt-SCTP scheme whereSCTP data chunks from the same application flow can beconcurrently transmitted using all of the available pathswithin the single end-to-end SCTP association Mechanismsare incorporated to reduce sender-initiated out-of-sequencepacket delivery to limit fast retransmissions and to managethe congestion-window update frequency However the reli-able sequence number driven nature of the SCTP protocolstill leads to the ldquoreceiver buffer blockingrdquo problem [18] aswhen a client has to wait for retransmission of a missingdata chunk it is unable to pass the other chunks in theacknowledgement window to the application

The authors of [24 25] proposed the use ofmultiple send-ing and receiving buffers to resolve the issue of receiver bufferblocking in cmt-SCTP Yuan et al [24] also introduced thefacility to differentiate subflows within an SCTP associationin terms of quality of service (QoS) requirements whilst in[25 26] the use of cmt-SCTP in wireless environments wasinvestigated mCMT [26] a cmt-SCTP variant for use inwireless networks employed multiple sending and receivingbuffers in a path-orientated multistreaming scheme whichalso incorporated the Media Independent Handover (MIH)scheme [27] for individual host mobility support

In addition to the many other cmt-SCTP based schemessuch as [28 29] for concurrent multipath transmission anumber of generic CMT schemes have also been proposedTsai et al [30] proposed a CMT scheme incorporatingforward error correction (FEC) and path interleaving withan adaptive FEC block size Liao et al in [19] proposedSMOS a sender-based multipath out-of-order scheduler forTCP-based multipath streaming whilst in [31] the authorsfurther considered the path correlation problem of sharedbottlenecks on end-to-end paths and proposed a new genericmultihoming sublayer

Although all of the above schemes whether SCTP or TCPbased offer solutions to some of the problems associated

International Journal of Digital Multimedia Broadcasting 5

with CMT they all only considered the case of individualmultihomed hosts There is a significant difference betweenthat environment and the one found in NEMO-based mobilenetworks In NEMO networks it is the mobility agents (HAand MR) that are multihomed rather than the end nodes(CN and MNN) In SCTP the end-to-end association isconducted over paths directly linking the network interfacesof the multihomed end hosts where the IP address of eachinterface that is used as the endpoint of the individual pathsin the association is known However in NEMO while theapplication flow is between the end nodes the independentpaths only exist between the interfaces of theHA and theMRIn CMT-NEMO [13] a mechanism is provided to enable theapplication to be split into subflows that are to be correctlyassociated with the interfaces at the HA and theMR to ensurethat they travel over the desired network path Each of theother CMT proposals described above was implemented atthe transport layer or in a cross-layer manner between thetransport layer and the application layer whilst the CMT-NEMO scheme operates at the RTPUDPNEMO protocolstack and is able to support multihomed mobile networks

In this work the components of CMT-NEMO are incor-porated into and become an integral part of our comprehen-sive multipath video streaming framework for multihomedNEMO environments Details of how the CMT componentsinteract with the other components of our framework areprovided in Section 3

3 Framework Implementation

In this work we introduce our comprehensive multipathstreaming framework for H264SVC and HEVC over multi-homed mobile networks The framework is a substantial fur-ther development of the scheme previously proposed in [12]Figure 3 gives an overviewof the framework (theH264SVC-specific implementation is shown) H264SVC and HEVCspecific implementations of the CMT-NEMO scheme [13]are combined with a modified version of the schedulingalgorithm from [12] which no longer needs to mitigate theNEMO path switching delay We also introduce a quality-layers-based packet weighting scheme for H264SVC asimple packet weighting scheme for HEVC and an improvedancestor checking schemewhich fully considers the granular-ity of H264SVC scalability In the framework the data planeprocesses and transmits video packets whilst the controlplane provides supporting functionality regarding networkpath conditions session setup and so forth Whilst the fullframework is shown in Figure 3 and has been implementedforH264SVC the current state of development of theHEVCstandard does not as yet permit full implementation ofcomponents that leverage multidimensional scalability suchas those responsible for target-rate derivation and extractionof substreams to a target bandwidth

Our framework includes CMT-NEMO-based compo-nents of a session setup control structure to determine thenumber of potential paths from the streamer to the clientand establish one application subflow (bound to a singlenetwork path) for each available path A flow disaggrega-tion mechanism overcomes the path switching limitation

observed in previous schemes and facilitates concurrentmul-tipath transmission Preprocessing modules use historicaldata from a target-bit-rate derivation subsystem for NEMO-based vehicular networks to determine target bit rates that areused by the encoder and bit stream extractor while a rate-distortion optimised stream is extracted using quality layersbased on [32]

Encoding and extraction functions employ the JSVMreference software [33] which is written in C++ The qualitylevel assigned to each NAL unit is also used to provide apacket prioritisation scheme that is specific to H264SVCand is used in our selective packet dropping mechanismin the case when there is insufficient aggregated bandwidthavailable

Each NAL unit in the stream is checked for decodingviability using a new more practical implementation of theancestor checking scheme used in [10] This mechanism isbetter suited to be used in real-time media-aware streamingsituations by replacing the time-consuming recursive search-ingmethod in [10] NAL units that pass the decoding viabilitytest are packetized for RTP transport over UDP A pathselection and packet scheduling algorithm then determinesif there is a viable path to the client and if more than onepath is found selects the path offering the earliest deliveryOut-of-sequence packet delivery to the client is mitigated atboth the streamer and the client in a manner that minimizesadded delay required to prevent out-of-sequence packetarrival at the client Finally successfully scheduled packetsare transmitted using the selected subflow with the subflowsbeing aggregated back to a single flow at the client prior topassing to the H264SVC decoder

31 Framework Design Considerations Placement of thestream adaptation path selection and packet schedulingagents within the network topology has a substantial effecton the viability of a multipath transmission scheme In [7ndash9] the agents were placed at a network proxy deployed at thepoint of path divergence (eg the HA in multihomed mobilenetworks) whilst in [10ndash12] the agents resided at the stream-ing server (CN) Both [10 11] only considered the case ofmultihomedhosts directly connected over independent pathsvia their multiple network interfaces A signaling schemein [12] sent control messages from the scheduling and pathselection agents at the CN to the actual point of divergence atthe HAwhere the path switching module directed the streamonto the desired path In this work we make the reasonableassumption that commercial multimedia content servers areunlikely to be equipped with interfaces that directly connectthem to multiple heterogeneous network technologies Thepoint of path divergence is more likely to be at a router withinthe network infrastructure In amultihomedmobile networkthe logical point of path divergence is the HA and the pointof path convergence is the MR

By considering the potential bidirectional use of the pro-posed streaming framework further constraints are imposedon the possible deployment of these agents Placement ofthe stream adaptation path selection and scheduling agentsat the point of divergence would put a substantial burdenon both the HA and for bidirectional use the MR Both of

6 International Journal of Digital Multimedia Broadcasting

Pathmonitoring

Controlplane

Target-ratederivation

Target base-

layer rate

Target

extractio

n

rate

Encoder

Bitstreamextractor

Packetweighting

Data plane

Subflow topath binding

lowast

lowast

lowastlowast

Flowdistribution

Pathselection

and packetscheduler

NAL unitpacketisation

Path metrics

Path m

etrics

Binding control andsession setup messages

Concurrent multipath transfer components

Figure 3 Overview of comprehensive streaming framework

these devices are already handling the mobility needs of theentire mobile network In the proposed scheme the agentsare placed at the streaming server All of stream adaptationpath selection and packet scheduling tasks are performed atthe CN or in the case of the potential bidirectional use at anappropriately resourced local fixed node (LFN) with a wiredconnection to the MR within the mobile network

32 Session Setup and Control The subflow to path bindingsubsystem is responsible for session setup and associatingsubflows with available paths The initial session setup con-sists of a series of control messages which are shown inFigure 4 A session setup agent at the MNN cooperateswith its counterpart at the CN to establish a streamingsession New streamlining sessions are initiated by themobilenetwork node which sends a ldquostream initiation requestrdquomessage (step 1 in Figure 4) to the CN The CN respondsby sending a ldquohome agent discoveryrdquo message addressed tothe MNN (step 2 in Figure 4) This message is interceptedby the HA that manages the mobility for the MR The HAreplies to the CN with a ldquoclient route identityrdquo messagecontaining the number of paths between the HA and the MRand the BIDs identifying each of its interfaces to the MR(step 3 in Figure 4) A reception ldquoport negotiationrdquo messageis then undertaken jointly by the CN and MNN The agreedreception port numbers are passed from the session setupagent to the subflow aggregation agent at the MNN (step4 in Figure 4) which opens listening ports to the CN andthen replies with a client ready message (step 5 in Figure 4)Upon the receipt of the ldquoclient readyrdquomessage the CN sends aldquosubflow bindingrdquo message for each available path (from HAto MR) to the HA (step 6 in Figure 4)

This message contains the (protocol source IP addresssource port destination IP address and destination port)quintuple identifying a subflow and the NEMO BID withwhich the subflow should be associatedTheHA then updatesthe binding cache and ensures that packets of the subflowdescribed by the quintuple in the ldquosubflow bindingrdquo message

Correspondentnode (CN)

(media server)

Homeagent (HA)

Multihomedmobile

router (MR)

Mobilenetwork

node(MNN)(1) Stream initiation request

(2) Request paths and BIDs

(3) Return paths and BIDs

(4) Port number negotiation

(5) Receiver ready status message

(6) Associate subflows to BIDs

(7) ACK subflow association

(8) Video stream transmission

Figure 4 Session setup

are delivered via the correct interface (BID) to the MRWhen the subflow associations are established the HA sendsa ldquosubflow acknowledgmentrdquo message to the CN (step 7 inFigure 4) which begins streaming to the client that is alreadyin the listening state The CN maintains a subflow to BIDbinding table for each streaming session Figure 5 showsthat application flow A is transmitted from the CN to theMNN Application subflows (A 1) and (A 2) belonging to thesame H264SVC HEVC or other application-type flows areassociated with NEMO BIDs 100 and 200 respectively at theHA

33 Packet Weighting and Ancestor Checking (PWAC)

331 H264SVC PWAC Scheme An H264SVC-encodedflow comprises a stream of logical data units called NetworkAbstraction Layer (NAL) units each of which has a one-byte H264AVC [2] compliant headerWhile base-layer NALunits only comprise this one-byte header and payload all

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 2: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

2 International Journal of Digital Multimedia Broadcasting

to multiple potential access networksrsquo paths (referred to as amultihomed mobile network) between the server and clientpresents a challenging environment for the delivery of delay-sensitive traffic such as video streaming Of the multipathstreaming schemes proposed in the literature to the best ofour knowledge only [12 13] are specifically designed for theuse with H264SVC scalable video streams in the NEMO[14] based multihomed mobile networks environment It isnoted that any scheme that can deliver high-quality videocontent such as streaming in (or near to) real time in thedownlink direction could potentially be explored in thereverse uplink direction for example to provide real-timepassenger safety monitoring over the aggregated bandwidthof all available network paths in public transportation sce-narios The possibility of such use is considered in thedesign of the proposed system The work in [12] is based onthe ldquoAlways Best Connectedrdquo (ABC) model for multihomedmobile networks proposed byWang et al in [15] In the ABCmodel an application flow for example a video stream isldquoswitchedrdquo from one network path to another based on apolicy-drivenmechanism that considers both the applicationflow characteristics and the current network path conditionsThe path selection component of the scheme proposed in[12] is an H264SVC-specific instance of the general modeldescribed in [15] Therefore although all available networkpaths can be used simultaneously (for the delivery of differentapplication flows) any given application flow is in factsequentially switched between paths by the path selectionand packet scheduling algorithm operating at the streamingserver

The Stream Control Transmission Protocol (SCTP) [16]is an alternative networking protocol that supports multi-homing SCTP following a host-to-host approach differsfrom NEMO in that it only supports individual multihomedhosts NEMO in contrast is able to support multihomedmobile routers that in turn provide mobility support for allof the mobile hosts within the mobile network in a muchmore efficient fashionMobile hosts inNEMOnetworks neednot be mobility aware or multihomed themselves whereasSCTP multihomed mobile hosts manage their own mobilityas well as multihoming A proposed extension [17] to theSCTP standard introduces a concurrent multipath transfermechanism cmt-SCTP In cmt-SCTP an application flowis split into a number of substreams each of which isconcurrently transmitted on a different end-to-end networkpathwithin an SCTP association Due to the reliable nature ofthe SCTP protocol the concurrent use ofmultiple paths leadsto out-of-sequence packet delivery and the ldquoreceiver bufferblockingrdquo problem [18 19] occurs

In [13] path switching overhead was established as beingthe largest single limiting factor imposed upon the viabilityof video streaming in multihomed mobile networks environ-ment and the CMT-NEMO scheme was proposed to over-come this limitation In this paper we significantly extend thework in [13] by providing a comprehensive analysis of con-currentmultipath streaming inmultihomedmobile networkswhich includes an investigation of the effects of backgroundtraffic on the CMT-NEMO scheme introduced in [13] Thiswork also extends the evaluation of concurrent multipath

streaming to include additional metrics of jitter and delaywhich have not previously been evaluated for concurrentmultipath transmission in mobile networks Additionallylooking to the future this work provides a timely empiricalinvestigation of the use of the newly standardised videoencoding technology HEVC in concurrent multipath videostreaming We also propose a simple HEVC specific packetweighting scheme for priority-based packet scheduling andscalable packet delivery (via selective packet dropping tomeetnetwork pathsrsquo bandwidth constraint) The results of thispilot HEVC implementation provide valuable insights intothe challenges that will face researchers seeking to furtherinvestigate HEVC streaming in both single and multipathenvironments

The rest of this paper is organized as follows In Section 2related work on multihomed mobile networks H264SVCHEVC and multipath transmission is reviewed The testbedimplementation of our framework is described in Section 3and numerical results of empirical experimentation arereported in Section 4 Section 5 concludes the paper

2 Related WorkDue to the multidisciplinary nature of this work that com-bines networking communications is and video signal pro-cessing the state of the art of a number of related researchareas are reviewed as follows

21 Multihomed Mobile Networks Working groups withinthe IETF have developed standards such as Mobile IPv4[20] and Mobile IPv6 [21] for managing the mobility ofmobile nodes BothMobile IPv4 andMobile IPv6 address themobility of an individual mobile host The emerging wirelessnetworking paradigm of mobile networks is based on theIETF Network Mobility (NEMO) standard [14] In NEMOnetworks groups of users travel together in unison (eg ona bus or train) and attach to the wider infrastructure via amobile router serving as a gateway for the mobile networkThe NEMO protocol [14] further considers the mobilityrequirements of an entire moving network by extending theMobile IPv6 protocol to include its use on a new entity themobile router (MR) which manages mobility on behalf ofall the individual mobile network nodes (MNNs) within themobile network

The mobile router is referred to as being multihomedwhen it is equipped with multiple network interfaces and isable to make simultaneous use of multiple network pathsThe components of a multihomedmobile network are shownin Figure 1 In this typical scenario a mobile network nodeaway from its home link is contacted by a correspondentnode (CN) via the NEMO-enabled home agent (HA) Thehome agent maintains a binding cache table entry linking themobile routerrsquos home address (HoA) to its current care ofaddress (CoA) In the case of multihomed mobile networkswhere the MR is equipped with multiple network interfacesa third identifier called the binding identifier (BID) is usedto differentiate between the different (HoA CoA) bindingsin the binding cache Each BID is associated with a networkinterface on the MR

International Journal of Digital Multimedia Broadcasting 3

IPv6 tunnelfrom HA (interface 2)

to MR (interface 2)

Accessnetwork 2

Home agent(HA)

IPv6 tunnelfrom HA (interface 1)

to MR (interface 1)Access

network 1Correspondent

node (CN)(media server)

Mobile networknode (MNN)

Mobilenetwork

Multihomedmobile router

(MR)

Figure 1 Topology of a two-path multihomed mobile network

When a mobile network node joins the mobile networkall traffic from a CN destined for the MNN is firstly directedto the HA of the MR The packet is encapsulated in an IPv6packet giving the destination address as the MRrsquos CoA Thispacket is then transmitted over an IPv6 tunnel between theHA and the MR When the packet arrives at the MR theencapsulation is removed and the packet is forwarded to theMNN Inmultihomedmobile networks each interface on theHA is connected to its corresponding interface on the MR bya separate IPv6 tunnel Figure 1 shows a typical multihomedmobile network where the media server is the CN and thereare two distinct paths between the HA and the MR It canbeen seen that the IPv6 tunnels connect the HA to the MR

With ABC implementations of multihomed mobile net-works such as the one proposed in the MULTINET project[15] an application flow is firstly described as an associationbetween the two end points (CN and MNN) using the(protocol source IP address source port destination IPaddress and destination port) quintuple to uniquely identifythe flow Consequently by associating this flow quintuplewith a NEMO BID the flow is bound to a particular pathbetween the HA and the MR A policy-driven networkselection algorithmusesmetrics from the application flow (itsbandwidth delay or other quality of service requirements)and the current network path conditions to decide the mostappropriate path between the HA and the MR for anygiven application flow An application flow is ldquoswitchedrdquo toanother network path by changing the association (at theHA)between its unique quintuple flow identifier and the (HoACoA BID) tuple that uniquely identifies a path from HA toMR In the ABC model a single flow can only be associatedwith one network path at any given time with the flow beingswitched among paths by the network selection algorithm

22 Scalable Video Coding Video streams encoded usingthe H264SVC [1] format comprise a number of substreams(layers) An H264AVC [2] compliant base layer providesa minimum quality of video while successive enhancementlayers improve the quality of the video stream H264SVCstreams offer a three-dimensional scalability Picture resolu-tion frame rate and signal-to-noise ratio enhancement layersprovide spatial temporal and quality scalability respectively

Qua

lity

PSN

R (d

B)

32

30

28

Tem

pora

l

QCIF CIF 4CIF 16CIF Spatial

20Hz25Hz

30Hz

lowast

lowast

lowast

Bitstream 20Hz CIF and 32dB PSNR

Figure 2 Extracting an H264SVC scalable bit stream

Figure 2 shows the extraction of a scalable substream match-ing a specified spatial (CIF) temporal (20Hz) and quality(32 dB PSNR) tuple H264SVC streams are readily adaptedto meet terminal requirements or in response to changingnetwork path conditions For example the sender will onlysend those layers that the receiver is capable of processingWhere there is insufficient bandwidth to transmit the entirestream a Media-Aware Network Entity (MANE) may drophigher enhancement layers thus ensuring the delivery of themore important base layer and lower enhancement layers

Meanwhile the user is still provided with an acceptable(albeit lower) quality of received video The userrsquos quality ofexperience (QoE) is therefore managed by providing a grace-ful degradation of the stream rather than its disruption As anexample the quality enhancement layer blocks (denoted bylowast) in the extracted bit stream in Figure 2 could be droppedto reduce the bandwidth requirement while maintaining anacceptable quality of received video with the desired spatialand temporal characteristics alternatively the stream couldhave been adapted by dropping either temporal or spatiallayer or indeed some combination of the different scalablelayers

23 High Efficiency Video Coding HEVC has been publishedin June 2013 by the ITU-T as H265 the next generationvideo coding standard [5] In common with the H264 familyof standards HEVC consists of two separate layers thevideo coding layer (VCL) and the Network Abstraction Layer(NAL)TheVCL layer provides coded representations of eachpicture in a video sequence while the NAL layer provides thebasic data unit (NAL unit) in which VCL data is encapsulatedfor storage or transmission

Compression efficiency in HEVC is improved (whencompared to H264AVC) by around 50 which is achievedby the inclusion of a number of new encoding tools andthe adoption of larger coding block sizes with an adaptivequadtree structure The only form of scalability offered inthe current HEVC specification [5] is temporal scalabilityThree temporal prediction modes are used the first of which(Intraprediction Only) does not permit temporal scalabilityThe LowDelay encodingmode has an InstantaneousDecoderRefresh (IDR) picture as its first picture with all subsequent

4 International Journal of Digital Multimedia Broadcasting

pictures being generalized P and B pictures These general-ized P and B pictures are only able to use pictures that areprior to themselves in output order (have lower picture ordercount (POC)) as references pictures In the third (RandomAccess) encodingmode a hierarchical B picture structure sim-ilar to that employed in H264SVC is employed The streamand IDR picture begin with intraencoded pictures beingadded approximately one per second these Clean RandomAccess (CRA) pictures provide random access points withinthe stream Pictures that follow a CRA picture in decodingorder but precede it in output order may use pictures thatcome before the CRA for reference

The experiments undertaken for this paper were con-ducted using version 6 (HM60) of the HEVC reference soft-ware JCT-VC regularly update the HEVC reference softwarethe current version [22] is version 110 (June 2013)

24 Multipath Streaming Algorithms Using the aggregatedbandwidth of multiple network paths between sender andreceiver has been proposed as a method of delivering high-quality video streams over links with a limited bandwidthChebrolu and Rao proposed the Earliest Delivery Path First(EDPF) scheme [7] where the arrival time of a packet isestimated for each available network path and the packettransmitted on the path offering the earliest delivery timeIn [8] the same authors extended their work by includinga frame-based selective drop mechanism Fernandez et al[9] further adapted EDPF to Time Division Multiple Access(TDMA) systems by inclusion of a time-slot policy mech-anism More recently Jurca and Frossard [10] proposed ascheme that while still using an earliest path first approachconsidered the dependencies between packets in a videostream and only transmitted those packets that can besuccessfully decoded at the receiver

This was achieved by dropping any packets which relyon a previously dropped packet for decoding purposes thuspreventing the wastage of network resources on packets thatare not viable The authors of [11] identified a potentialproblem in [10] which can lead to out-of-sequence packetdelivery to the client and proposed a new algorithm toovercome this limitationThe same observation is true for allthe earliest path-type schemes Our scheme in [12] includeda mechanism to prevent the out-of-sequence packet deliveryidentified in [11] In this work we incorporate an additionalcomponent at the client to manage any out-of-sequencearrival that may still occur due to imprecise bandwidthmeasurements or transient interferences on the wireless linksin our testbed

Whilst the authors of [10] considered a generic scalablevideo stream none of these schemes have addressed thehigher levels of granularity and complex packet dependenciesfound in H264SVC-encoded streams Our previous work[12 23] considered the specific challenges associated withdelivering H264SVC over multiple paths in multihomedmobile networks The HEVC standard [5] is a very recentdevelopment and no multipath streaming algorithms specif-ically targeted at HEVC have as yet been proposed in theliterature

The additional mobility-related overheads of NEMO andthe path switching cost of ABC-based NEMO networks weremitigated in our optimised path selection and schedulingalgorithm (OPSSA) [23] A further review [13] of the per-formance of H264SVC streaming in multihomed NEMOenvironments identified the ABC-related path switchingoverhead as being a dominant limiting factor in the deliveryof video streams in this context The CMT-NEMO algorithmdescribed in Section 25 was proposed in [13] to overcomethe limitations placed on ABC-NEMO by path switchingoverheads

25 Concurrent Multipath Transmission The majority ofexisting concurrent multipath transfer (CMT) schemes arebased on the SCTP transport-layer protocol SCTP is a reli-able transport protocol which retransmits packets failing toreach the client and incorporates a congestion controlmecha-nism [16] An end-to-end SCTP association is made betweentwo multihomed SCTP hosts A number of independentpaths exist within this association In SCTP multihomingonly one of these paths may be used as the primary path fordata transfer with the other being utilized for retransmissionsand primary path failure redundancy

Iyengar et al [17] proposed the cmt-SCTP scheme whereSCTP data chunks from the same application flow can beconcurrently transmitted using all of the available pathswithin the single end-to-end SCTP association Mechanismsare incorporated to reduce sender-initiated out-of-sequencepacket delivery to limit fast retransmissions and to managethe congestion-window update frequency However the reli-able sequence number driven nature of the SCTP protocolstill leads to the ldquoreceiver buffer blockingrdquo problem [18] aswhen a client has to wait for retransmission of a missingdata chunk it is unable to pass the other chunks in theacknowledgement window to the application

The authors of [24 25] proposed the use ofmultiple send-ing and receiving buffers to resolve the issue of receiver bufferblocking in cmt-SCTP Yuan et al [24] also introduced thefacility to differentiate subflows within an SCTP associationin terms of quality of service (QoS) requirements whilst in[25 26] the use of cmt-SCTP in wireless environments wasinvestigated mCMT [26] a cmt-SCTP variant for use inwireless networks employed multiple sending and receivingbuffers in a path-orientated multistreaming scheme whichalso incorporated the Media Independent Handover (MIH)scheme [27] for individual host mobility support

In addition to the many other cmt-SCTP based schemessuch as [28 29] for concurrent multipath transmission anumber of generic CMT schemes have also been proposedTsai et al [30] proposed a CMT scheme incorporatingforward error correction (FEC) and path interleaving withan adaptive FEC block size Liao et al in [19] proposedSMOS a sender-based multipath out-of-order scheduler forTCP-based multipath streaming whilst in [31] the authorsfurther considered the path correlation problem of sharedbottlenecks on end-to-end paths and proposed a new genericmultihoming sublayer

Although all of the above schemes whether SCTP or TCPbased offer solutions to some of the problems associated

International Journal of Digital Multimedia Broadcasting 5

with CMT they all only considered the case of individualmultihomed hosts There is a significant difference betweenthat environment and the one found in NEMO-based mobilenetworks In NEMO networks it is the mobility agents (HAand MR) that are multihomed rather than the end nodes(CN and MNN) In SCTP the end-to-end association isconducted over paths directly linking the network interfacesof the multihomed end hosts where the IP address of eachinterface that is used as the endpoint of the individual pathsin the association is known However in NEMO while theapplication flow is between the end nodes the independentpaths only exist between the interfaces of theHA and theMRIn CMT-NEMO [13] a mechanism is provided to enable theapplication to be split into subflows that are to be correctlyassociated with the interfaces at the HA and theMR to ensurethat they travel over the desired network path Each of theother CMT proposals described above was implemented atthe transport layer or in a cross-layer manner between thetransport layer and the application layer whilst the CMT-NEMO scheme operates at the RTPUDPNEMO protocolstack and is able to support multihomed mobile networks

In this work the components of CMT-NEMO are incor-porated into and become an integral part of our comprehen-sive multipath video streaming framework for multihomedNEMO environments Details of how the CMT componentsinteract with the other components of our framework areprovided in Section 3

3 Framework Implementation

In this work we introduce our comprehensive multipathstreaming framework for H264SVC and HEVC over multi-homed mobile networks The framework is a substantial fur-ther development of the scheme previously proposed in [12]Figure 3 gives an overviewof the framework (theH264SVC-specific implementation is shown) H264SVC and HEVCspecific implementations of the CMT-NEMO scheme [13]are combined with a modified version of the schedulingalgorithm from [12] which no longer needs to mitigate theNEMO path switching delay We also introduce a quality-layers-based packet weighting scheme for H264SVC asimple packet weighting scheme for HEVC and an improvedancestor checking schemewhich fully considers the granular-ity of H264SVC scalability In the framework the data planeprocesses and transmits video packets whilst the controlplane provides supporting functionality regarding networkpath conditions session setup and so forth Whilst the fullframework is shown in Figure 3 and has been implementedforH264SVC the current state of development of theHEVCstandard does not as yet permit full implementation ofcomponents that leverage multidimensional scalability suchas those responsible for target-rate derivation and extractionof substreams to a target bandwidth

Our framework includes CMT-NEMO-based compo-nents of a session setup control structure to determine thenumber of potential paths from the streamer to the clientand establish one application subflow (bound to a singlenetwork path) for each available path A flow disaggrega-tion mechanism overcomes the path switching limitation

observed in previous schemes and facilitates concurrentmul-tipath transmission Preprocessing modules use historicaldata from a target-bit-rate derivation subsystem for NEMO-based vehicular networks to determine target bit rates that areused by the encoder and bit stream extractor while a rate-distortion optimised stream is extracted using quality layersbased on [32]

Encoding and extraction functions employ the JSVMreference software [33] which is written in C++ The qualitylevel assigned to each NAL unit is also used to provide apacket prioritisation scheme that is specific to H264SVCand is used in our selective packet dropping mechanismin the case when there is insufficient aggregated bandwidthavailable

Each NAL unit in the stream is checked for decodingviability using a new more practical implementation of theancestor checking scheme used in [10] This mechanism isbetter suited to be used in real-time media-aware streamingsituations by replacing the time-consuming recursive search-ingmethod in [10] NAL units that pass the decoding viabilitytest are packetized for RTP transport over UDP A pathselection and packet scheduling algorithm then determinesif there is a viable path to the client and if more than onepath is found selects the path offering the earliest deliveryOut-of-sequence packet delivery to the client is mitigated atboth the streamer and the client in a manner that minimizesadded delay required to prevent out-of-sequence packetarrival at the client Finally successfully scheduled packetsare transmitted using the selected subflow with the subflowsbeing aggregated back to a single flow at the client prior topassing to the H264SVC decoder

31 Framework Design Considerations Placement of thestream adaptation path selection and packet schedulingagents within the network topology has a substantial effecton the viability of a multipath transmission scheme In [7ndash9] the agents were placed at a network proxy deployed at thepoint of path divergence (eg the HA in multihomed mobilenetworks) whilst in [10ndash12] the agents resided at the stream-ing server (CN) Both [10 11] only considered the case ofmultihomedhosts directly connected over independent pathsvia their multiple network interfaces A signaling schemein [12] sent control messages from the scheduling and pathselection agents at the CN to the actual point of divergence atthe HAwhere the path switching module directed the streamonto the desired path In this work we make the reasonableassumption that commercial multimedia content servers areunlikely to be equipped with interfaces that directly connectthem to multiple heterogeneous network technologies Thepoint of path divergence is more likely to be at a router withinthe network infrastructure In amultihomedmobile networkthe logical point of path divergence is the HA and the pointof path convergence is the MR

By considering the potential bidirectional use of the pro-posed streaming framework further constraints are imposedon the possible deployment of these agents Placement ofthe stream adaptation path selection and scheduling agentsat the point of divergence would put a substantial burdenon both the HA and for bidirectional use the MR Both of

6 International Journal of Digital Multimedia Broadcasting

Pathmonitoring

Controlplane

Target-ratederivation

Target base-

layer rate

Target

extractio

n

rate

Encoder

Bitstreamextractor

Packetweighting

Data plane

Subflow topath binding

lowast

lowast

lowastlowast

Flowdistribution

Pathselection

and packetscheduler

NAL unitpacketisation

Path metrics

Path m

etrics

Binding control andsession setup messages

Concurrent multipath transfer components

Figure 3 Overview of comprehensive streaming framework

these devices are already handling the mobility needs of theentire mobile network In the proposed scheme the agentsare placed at the streaming server All of stream adaptationpath selection and packet scheduling tasks are performed atthe CN or in the case of the potential bidirectional use at anappropriately resourced local fixed node (LFN) with a wiredconnection to the MR within the mobile network

32 Session Setup and Control The subflow to path bindingsubsystem is responsible for session setup and associatingsubflows with available paths The initial session setup con-sists of a series of control messages which are shown inFigure 4 A session setup agent at the MNN cooperateswith its counterpart at the CN to establish a streamingsession New streamlining sessions are initiated by themobilenetwork node which sends a ldquostream initiation requestrdquomessage (step 1 in Figure 4) to the CN The CN respondsby sending a ldquohome agent discoveryrdquo message addressed tothe MNN (step 2 in Figure 4) This message is interceptedby the HA that manages the mobility for the MR The HAreplies to the CN with a ldquoclient route identityrdquo messagecontaining the number of paths between the HA and the MRand the BIDs identifying each of its interfaces to the MR(step 3 in Figure 4) A reception ldquoport negotiationrdquo messageis then undertaken jointly by the CN and MNN The agreedreception port numbers are passed from the session setupagent to the subflow aggregation agent at the MNN (step4 in Figure 4) which opens listening ports to the CN andthen replies with a client ready message (step 5 in Figure 4)Upon the receipt of the ldquoclient readyrdquomessage the CN sends aldquosubflow bindingrdquo message for each available path (from HAto MR) to the HA (step 6 in Figure 4)

This message contains the (protocol source IP addresssource port destination IP address and destination port)quintuple identifying a subflow and the NEMO BID withwhich the subflow should be associatedTheHA then updatesthe binding cache and ensures that packets of the subflowdescribed by the quintuple in the ldquosubflow bindingrdquo message

Correspondentnode (CN)

(media server)

Homeagent (HA)

Multihomedmobile

router (MR)

Mobilenetwork

node(MNN)(1) Stream initiation request

(2) Request paths and BIDs

(3) Return paths and BIDs

(4) Port number negotiation

(5) Receiver ready status message

(6) Associate subflows to BIDs

(7) ACK subflow association

(8) Video stream transmission

Figure 4 Session setup

are delivered via the correct interface (BID) to the MRWhen the subflow associations are established the HA sendsa ldquosubflow acknowledgmentrdquo message to the CN (step 7 inFigure 4) which begins streaming to the client that is alreadyin the listening state The CN maintains a subflow to BIDbinding table for each streaming session Figure 5 showsthat application flow A is transmitted from the CN to theMNN Application subflows (A 1) and (A 2) belonging to thesame H264SVC HEVC or other application-type flows areassociated with NEMO BIDs 100 and 200 respectively at theHA

33 Packet Weighting and Ancestor Checking (PWAC)

331 H264SVC PWAC Scheme An H264SVC-encodedflow comprises a stream of logical data units called NetworkAbstraction Layer (NAL) units each of which has a one-byte H264AVC [2] compliant headerWhile base-layer NALunits only comprise this one-byte header and payload all

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 3: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 3

IPv6 tunnelfrom HA (interface 2)

to MR (interface 2)

Accessnetwork 2

Home agent(HA)

IPv6 tunnelfrom HA (interface 1)

to MR (interface 1)Access

network 1Correspondent

node (CN)(media server)

Mobile networknode (MNN)

Mobilenetwork

Multihomedmobile router

(MR)

Figure 1 Topology of a two-path multihomed mobile network

When a mobile network node joins the mobile networkall traffic from a CN destined for the MNN is firstly directedto the HA of the MR The packet is encapsulated in an IPv6packet giving the destination address as the MRrsquos CoA Thispacket is then transmitted over an IPv6 tunnel between theHA and the MR When the packet arrives at the MR theencapsulation is removed and the packet is forwarded to theMNN Inmultihomedmobile networks each interface on theHA is connected to its corresponding interface on the MR bya separate IPv6 tunnel Figure 1 shows a typical multihomedmobile network where the media server is the CN and thereare two distinct paths between the HA and the MR It canbeen seen that the IPv6 tunnels connect the HA to the MR

With ABC implementations of multihomed mobile net-works such as the one proposed in the MULTINET project[15] an application flow is firstly described as an associationbetween the two end points (CN and MNN) using the(protocol source IP address source port destination IPaddress and destination port) quintuple to uniquely identifythe flow Consequently by associating this flow quintuplewith a NEMO BID the flow is bound to a particular pathbetween the HA and the MR A policy-driven networkselection algorithmusesmetrics from the application flow (itsbandwidth delay or other quality of service requirements)and the current network path conditions to decide the mostappropriate path between the HA and the MR for anygiven application flow An application flow is ldquoswitchedrdquo toanother network path by changing the association (at theHA)between its unique quintuple flow identifier and the (HoACoA BID) tuple that uniquely identifies a path from HA toMR In the ABC model a single flow can only be associatedwith one network path at any given time with the flow beingswitched among paths by the network selection algorithm

22 Scalable Video Coding Video streams encoded usingthe H264SVC [1] format comprise a number of substreams(layers) An H264AVC [2] compliant base layer providesa minimum quality of video while successive enhancementlayers improve the quality of the video stream H264SVCstreams offer a three-dimensional scalability Picture resolu-tion frame rate and signal-to-noise ratio enhancement layersprovide spatial temporal and quality scalability respectively

Qua

lity

PSN

R (d

B)

32

30

28

Tem

pora

l

QCIF CIF 4CIF 16CIF Spatial

20Hz25Hz

30Hz

lowast

lowast

lowast

Bitstream 20Hz CIF and 32dB PSNR

Figure 2 Extracting an H264SVC scalable bit stream

Figure 2 shows the extraction of a scalable substream match-ing a specified spatial (CIF) temporal (20Hz) and quality(32 dB PSNR) tuple H264SVC streams are readily adaptedto meet terminal requirements or in response to changingnetwork path conditions For example the sender will onlysend those layers that the receiver is capable of processingWhere there is insufficient bandwidth to transmit the entirestream a Media-Aware Network Entity (MANE) may drophigher enhancement layers thus ensuring the delivery of themore important base layer and lower enhancement layers

Meanwhile the user is still provided with an acceptable(albeit lower) quality of received video The userrsquos quality ofexperience (QoE) is therefore managed by providing a grace-ful degradation of the stream rather than its disruption As anexample the quality enhancement layer blocks (denoted bylowast) in the extracted bit stream in Figure 2 could be droppedto reduce the bandwidth requirement while maintaining anacceptable quality of received video with the desired spatialand temporal characteristics alternatively the stream couldhave been adapted by dropping either temporal or spatiallayer or indeed some combination of the different scalablelayers

23 High Efficiency Video Coding HEVC has been publishedin June 2013 by the ITU-T as H265 the next generationvideo coding standard [5] In common with the H264 familyof standards HEVC consists of two separate layers thevideo coding layer (VCL) and the Network Abstraction Layer(NAL)TheVCL layer provides coded representations of eachpicture in a video sequence while the NAL layer provides thebasic data unit (NAL unit) in which VCL data is encapsulatedfor storage or transmission

Compression efficiency in HEVC is improved (whencompared to H264AVC) by around 50 which is achievedby the inclusion of a number of new encoding tools andthe adoption of larger coding block sizes with an adaptivequadtree structure The only form of scalability offered inthe current HEVC specification [5] is temporal scalabilityThree temporal prediction modes are used the first of which(Intraprediction Only) does not permit temporal scalabilityThe LowDelay encodingmode has an InstantaneousDecoderRefresh (IDR) picture as its first picture with all subsequent

4 International Journal of Digital Multimedia Broadcasting

pictures being generalized P and B pictures These general-ized P and B pictures are only able to use pictures that areprior to themselves in output order (have lower picture ordercount (POC)) as references pictures In the third (RandomAccess) encodingmode a hierarchical B picture structure sim-ilar to that employed in H264SVC is employed The streamand IDR picture begin with intraencoded pictures beingadded approximately one per second these Clean RandomAccess (CRA) pictures provide random access points withinthe stream Pictures that follow a CRA picture in decodingorder but precede it in output order may use pictures thatcome before the CRA for reference

The experiments undertaken for this paper were con-ducted using version 6 (HM60) of the HEVC reference soft-ware JCT-VC regularly update the HEVC reference softwarethe current version [22] is version 110 (June 2013)

24 Multipath Streaming Algorithms Using the aggregatedbandwidth of multiple network paths between sender andreceiver has been proposed as a method of delivering high-quality video streams over links with a limited bandwidthChebrolu and Rao proposed the Earliest Delivery Path First(EDPF) scheme [7] where the arrival time of a packet isestimated for each available network path and the packettransmitted on the path offering the earliest delivery timeIn [8] the same authors extended their work by includinga frame-based selective drop mechanism Fernandez et al[9] further adapted EDPF to Time Division Multiple Access(TDMA) systems by inclusion of a time-slot policy mech-anism More recently Jurca and Frossard [10] proposed ascheme that while still using an earliest path first approachconsidered the dependencies between packets in a videostream and only transmitted those packets that can besuccessfully decoded at the receiver

This was achieved by dropping any packets which relyon a previously dropped packet for decoding purposes thuspreventing the wastage of network resources on packets thatare not viable The authors of [11] identified a potentialproblem in [10] which can lead to out-of-sequence packetdelivery to the client and proposed a new algorithm toovercome this limitationThe same observation is true for allthe earliest path-type schemes Our scheme in [12] includeda mechanism to prevent the out-of-sequence packet deliveryidentified in [11] In this work we incorporate an additionalcomponent at the client to manage any out-of-sequencearrival that may still occur due to imprecise bandwidthmeasurements or transient interferences on the wireless linksin our testbed

Whilst the authors of [10] considered a generic scalablevideo stream none of these schemes have addressed thehigher levels of granularity and complex packet dependenciesfound in H264SVC-encoded streams Our previous work[12 23] considered the specific challenges associated withdelivering H264SVC over multiple paths in multihomedmobile networks The HEVC standard [5] is a very recentdevelopment and no multipath streaming algorithms specif-ically targeted at HEVC have as yet been proposed in theliterature

The additional mobility-related overheads of NEMO andthe path switching cost of ABC-based NEMO networks weremitigated in our optimised path selection and schedulingalgorithm (OPSSA) [23] A further review [13] of the per-formance of H264SVC streaming in multihomed NEMOenvironments identified the ABC-related path switchingoverhead as being a dominant limiting factor in the deliveryof video streams in this context The CMT-NEMO algorithmdescribed in Section 25 was proposed in [13] to overcomethe limitations placed on ABC-NEMO by path switchingoverheads

25 Concurrent Multipath Transmission The majority ofexisting concurrent multipath transfer (CMT) schemes arebased on the SCTP transport-layer protocol SCTP is a reli-able transport protocol which retransmits packets failing toreach the client and incorporates a congestion controlmecha-nism [16] An end-to-end SCTP association is made betweentwo multihomed SCTP hosts A number of independentpaths exist within this association In SCTP multihomingonly one of these paths may be used as the primary path fordata transfer with the other being utilized for retransmissionsand primary path failure redundancy

Iyengar et al [17] proposed the cmt-SCTP scheme whereSCTP data chunks from the same application flow can beconcurrently transmitted using all of the available pathswithin the single end-to-end SCTP association Mechanismsare incorporated to reduce sender-initiated out-of-sequencepacket delivery to limit fast retransmissions and to managethe congestion-window update frequency However the reli-able sequence number driven nature of the SCTP protocolstill leads to the ldquoreceiver buffer blockingrdquo problem [18] aswhen a client has to wait for retransmission of a missingdata chunk it is unable to pass the other chunks in theacknowledgement window to the application

The authors of [24 25] proposed the use ofmultiple send-ing and receiving buffers to resolve the issue of receiver bufferblocking in cmt-SCTP Yuan et al [24] also introduced thefacility to differentiate subflows within an SCTP associationin terms of quality of service (QoS) requirements whilst in[25 26] the use of cmt-SCTP in wireless environments wasinvestigated mCMT [26] a cmt-SCTP variant for use inwireless networks employed multiple sending and receivingbuffers in a path-orientated multistreaming scheme whichalso incorporated the Media Independent Handover (MIH)scheme [27] for individual host mobility support

In addition to the many other cmt-SCTP based schemessuch as [28 29] for concurrent multipath transmission anumber of generic CMT schemes have also been proposedTsai et al [30] proposed a CMT scheme incorporatingforward error correction (FEC) and path interleaving withan adaptive FEC block size Liao et al in [19] proposedSMOS a sender-based multipath out-of-order scheduler forTCP-based multipath streaming whilst in [31] the authorsfurther considered the path correlation problem of sharedbottlenecks on end-to-end paths and proposed a new genericmultihoming sublayer

Although all of the above schemes whether SCTP or TCPbased offer solutions to some of the problems associated

International Journal of Digital Multimedia Broadcasting 5

with CMT they all only considered the case of individualmultihomed hosts There is a significant difference betweenthat environment and the one found in NEMO-based mobilenetworks In NEMO networks it is the mobility agents (HAand MR) that are multihomed rather than the end nodes(CN and MNN) In SCTP the end-to-end association isconducted over paths directly linking the network interfacesof the multihomed end hosts where the IP address of eachinterface that is used as the endpoint of the individual pathsin the association is known However in NEMO while theapplication flow is between the end nodes the independentpaths only exist between the interfaces of theHA and theMRIn CMT-NEMO [13] a mechanism is provided to enable theapplication to be split into subflows that are to be correctlyassociated with the interfaces at the HA and theMR to ensurethat they travel over the desired network path Each of theother CMT proposals described above was implemented atthe transport layer or in a cross-layer manner between thetransport layer and the application layer whilst the CMT-NEMO scheme operates at the RTPUDPNEMO protocolstack and is able to support multihomed mobile networks

In this work the components of CMT-NEMO are incor-porated into and become an integral part of our comprehen-sive multipath video streaming framework for multihomedNEMO environments Details of how the CMT componentsinteract with the other components of our framework areprovided in Section 3

3 Framework Implementation

In this work we introduce our comprehensive multipathstreaming framework for H264SVC and HEVC over multi-homed mobile networks The framework is a substantial fur-ther development of the scheme previously proposed in [12]Figure 3 gives an overviewof the framework (theH264SVC-specific implementation is shown) H264SVC and HEVCspecific implementations of the CMT-NEMO scheme [13]are combined with a modified version of the schedulingalgorithm from [12] which no longer needs to mitigate theNEMO path switching delay We also introduce a quality-layers-based packet weighting scheme for H264SVC asimple packet weighting scheme for HEVC and an improvedancestor checking schemewhich fully considers the granular-ity of H264SVC scalability In the framework the data planeprocesses and transmits video packets whilst the controlplane provides supporting functionality regarding networkpath conditions session setup and so forth Whilst the fullframework is shown in Figure 3 and has been implementedforH264SVC the current state of development of theHEVCstandard does not as yet permit full implementation ofcomponents that leverage multidimensional scalability suchas those responsible for target-rate derivation and extractionof substreams to a target bandwidth

Our framework includes CMT-NEMO-based compo-nents of a session setup control structure to determine thenumber of potential paths from the streamer to the clientand establish one application subflow (bound to a singlenetwork path) for each available path A flow disaggrega-tion mechanism overcomes the path switching limitation

observed in previous schemes and facilitates concurrentmul-tipath transmission Preprocessing modules use historicaldata from a target-bit-rate derivation subsystem for NEMO-based vehicular networks to determine target bit rates that areused by the encoder and bit stream extractor while a rate-distortion optimised stream is extracted using quality layersbased on [32]

Encoding and extraction functions employ the JSVMreference software [33] which is written in C++ The qualitylevel assigned to each NAL unit is also used to provide apacket prioritisation scheme that is specific to H264SVCand is used in our selective packet dropping mechanismin the case when there is insufficient aggregated bandwidthavailable

Each NAL unit in the stream is checked for decodingviability using a new more practical implementation of theancestor checking scheme used in [10] This mechanism isbetter suited to be used in real-time media-aware streamingsituations by replacing the time-consuming recursive search-ingmethod in [10] NAL units that pass the decoding viabilitytest are packetized for RTP transport over UDP A pathselection and packet scheduling algorithm then determinesif there is a viable path to the client and if more than onepath is found selects the path offering the earliest deliveryOut-of-sequence packet delivery to the client is mitigated atboth the streamer and the client in a manner that minimizesadded delay required to prevent out-of-sequence packetarrival at the client Finally successfully scheduled packetsare transmitted using the selected subflow with the subflowsbeing aggregated back to a single flow at the client prior topassing to the H264SVC decoder

31 Framework Design Considerations Placement of thestream adaptation path selection and packet schedulingagents within the network topology has a substantial effecton the viability of a multipath transmission scheme In [7ndash9] the agents were placed at a network proxy deployed at thepoint of path divergence (eg the HA in multihomed mobilenetworks) whilst in [10ndash12] the agents resided at the stream-ing server (CN) Both [10 11] only considered the case ofmultihomedhosts directly connected over independent pathsvia their multiple network interfaces A signaling schemein [12] sent control messages from the scheduling and pathselection agents at the CN to the actual point of divergence atthe HAwhere the path switching module directed the streamonto the desired path In this work we make the reasonableassumption that commercial multimedia content servers areunlikely to be equipped with interfaces that directly connectthem to multiple heterogeneous network technologies Thepoint of path divergence is more likely to be at a router withinthe network infrastructure In amultihomedmobile networkthe logical point of path divergence is the HA and the pointof path convergence is the MR

By considering the potential bidirectional use of the pro-posed streaming framework further constraints are imposedon the possible deployment of these agents Placement ofthe stream adaptation path selection and scheduling agentsat the point of divergence would put a substantial burdenon both the HA and for bidirectional use the MR Both of

6 International Journal of Digital Multimedia Broadcasting

Pathmonitoring

Controlplane

Target-ratederivation

Target base-

layer rate

Target

extractio

n

rate

Encoder

Bitstreamextractor

Packetweighting

Data plane

Subflow topath binding

lowast

lowast

lowastlowast

Flowdistribution

Pathselection

and packetscheduler

NAL unitpacketisation

Path metrics

Path m

etrics

Binding control andsession setup messages

Concurrent multipath transfer components

Figure 3 Overview of comprehensive streaming framework

these devices are already handling the mobility needs of theentire mobile network In the proposed scheme the agentsare placed at the streaming server All of stream adaptationpath selection and packet scheduling tasks are performed atthe CN or in the case of the potential bidirectional use at anappropriately resourced local fixed node (LFN) with a wiredconnection to the MR within the mobile network

32 Session Setup and Control The subflow to path bindingsubsystem is responsible for session setup and associatingsubflows with available paths The initial session setup con-sists of a series of control messages which are shown inFigure 4 A session setup agent at the MNN cooperateswith its counterpart at the CN to establish a streamingsession New streamlining sessions are initiated by themobilenetwork node which sends a ldquostream initiation requestrdquomessage (step 1 in Figure 4) to the CN The CN respondsby sending a ldquohome agent discoveryrdquo message addressed tothe MNN (step 2 in Figure 4) This message is interceptedby the HA that manages the mobility for the MR The HAreplies to the CN with a ldquoclient route identityrdquo messagecontaining the number of paths between the HA and the MRand the BIDs identifying each of its interfaces to the MR(step 3 in Figure 4) A reception ldquoport negotiationrdquo messageis then undertaken jointly by the CN and MNN The agreedreception port numbers are passed from the session setupagent to the subflow aggregation agent at the MNN (step4 in Figure 4) which opens listening ports to the CN andthen replies with a client ready message (step 5 in Figure 4)Upon the receipt of the ldquoclient readyrdquomessage the CN sends aldquosubflow bindingrdquo message for each available path (from HAto MR) to the HA (step 6 in Figure 4)

This message contains the (protocol source IP addresssource port destination IP address and destination port)quintuple identifying a subflow and the NEMO BID withwhich the subflow should be associatedTheHA then updatesthe binding cache and ensures that packets of the subflowdescribed by the quintuple in the ldquosubflow bindingrdquo message

Correspondentnode (CN)

(media server)

Homeagent (HA)

Multihomedmobile

router (MR)

Mobilenetwork

node(MNN)(1) Stream initiation request

(2) Request paths and BIDs

(3) Return paths and BIDs

(4) Port number negotiation

(5) Receiver ready status message

(6) Associate subflows to BIDs

(7) ACK subflow association

(8) Video stream transmission

Figure 4 Session setup

are delivered via the correct interface (BID) to the MRWhen the subflow associations are established the HA sendsa ldquosubflow acknowledgmentrdquo message to the CN (step 7 inFigure 4) which begins streaming to the client that is alreadyin the listening state The CN maintains a subflow to BIDbinding table for each streaming session Figure 5 showsthat application flow A is transmitted from the CN to theMNN Application subflows (A 1) and (A 2) belonging to thesame H264SVC HEVC or other application-type flows areassociated with NEMO BIDs 100 and 200 respectively at theHA

33 Packet Weighting and Ancestor Checking (PWAC)

331 H264SVC PWAC Scheme An H264SVC-encodedflow comprises a stream of logical data units called NetworkAbstraction Layer (NAL) units each of which has a one-byte H264AVC [2] compliant headerWhile base-layer NALunits only comprise this one-byte header and payload all

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 4: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

4 International Journal of Digital Multimedia Broadcasting

pictures being generalized P and B pictures These general-ized P and B pictures are only able to use pictures that areprior to themselves in output order (have lower picture ordercount (POC)) as references pictures In the third (RandomAccess) encodingmode a hierarchical B picture structure sim-ilar to that employed in H264SVC is employed The streamand IDR picture begin with intraencoded pictures beingadded approximately one per second these Clean RandomAccess (CRA) pictures provide random access points withinthe stream Pictures that follow a CRA picture in decodingorder but precede it in output order may use pictures thatcome before the CRA for reference

The experiments undertaken for this paper were con-ducted using version 6 (HM60) of the HEVC reference soft-ware JCT-VC regularly update the HEVC reference softwarethe current version [22] is version 110 (June 2013)

24 Multipath Streaming Algorithms Using the aggregatedbandwidth of multiple network paths between sender andreceiver has been proposed as a method of delivering high-quality video streams over links with a limited bandwidthChebrolu and Rao proposed the Earliest Delivery Path First(EDPF) scheme [7] where the arrival time of a packet isestimated for each available network path and the packettransmitted on the path offering the earliest delivery timeIn [8] the same authors extended their work by includinga frame-based selective drop mechanism Fernandez et al[9] further adapted EDPF to Time Division Multiple Access(TDMA) systems by inclusion of a time-slot policy mech-anism More recently Jurca and Frossard [10] proposed ascheme that while still using an earliest path first approachconsidered the dependencies between packets in a videostream and only transmitted those packets that can besuccessfully decoded at the receiver

This was achieved by dropping any packets which relyon a previously dropped packet for decoding purposes thuspreventing the wastage of network resources on packets thatare not viable The authors of [11] identified a potentialproblem in [10] which can lead to out-of-sequence packetdelivery to the client and proposed a new algorithm toovercome this limitationThe same observation is true for allthe earliest path-type schemes Our scheme in [12] includeda mechanism to prevent the out-of-sequence packet deliveryidentified in [11] In this work we incorporate an additionalcomponent at the client to manage any out-of-sequencearrival that may still occur due to imprecise bandwidthmeasurements or transient interferences on the wireless linksin our testbed

Whilst the authors of [10] considered a generic scalablevideo stream none of these schemes have addressed thehigher levels of granularity and complex packet dependenciesfound in H264SVC-encoded streams Our previous work[12 23] considered the specific challenges associated withdelivering H264SVC over multiple paths in multihomedmobile networks The HEVC standard [5] is a very recentdevelopment and no multipath streaming algorithms specif-ically targeted at HEVC have as yet been proposed in theliterature

The additional mobility-related overheads of NEMO andthe path switching cost of ABC-based NEMO networks weremitigated in our optimised path selection and schedulingalgorithm (OPSSA) [23] A further review [13] of the per-formance of H264SVC streaming in multihomed NEMOenvironments identified the ABC-related path switchingoverhead as being a dominant limiting factor in the deliveryof video streams in this context The CMT-NEMO algorithmdescribed in Section 25 was proposed in [13] to overcomethe limitations placed on ABC-NEMO by path switchingoverheads

25 Concurrent Multipath Transmission The majority ofexisting concurrent multipath transfer (CMT) schemes arebased on the SCTP transport-layer protocol SCTP is a reli-able transport protocol which retransmits packets failing toreach the client and incorporates a congestion controlmecha-nism [16] An end-to-end SCTP association is made betweentwo multihomed SCTP hosts A number of independentpaths exist within this association In SCTP multihomingonly one of these paths may be used as the primary path fordata transfer with the other being utilized for retransmissionsand primary path failure redundancy

Iyengar et al [17] proposed the cmt-SCTP scheme whereSCTP data chunks from the same application flow can beconcurrently transmitted using all of the available pathswithin the single end-to-end SCTP association Mechanismsare incorporated to reduce sender-initiated out-of-sequencepacket delivery to limit fast retransmissions and to managethe congestion-window update frequency However the reli-able sequence number driven nature of the SCTP protocolstill leads to the ldquoreceiver buffer blockingrdquo problem [18] aswhen a client has to wait for retransmission of a missingdata chunk it is unable to pass the other chunks in theacknowledgement window to the application

The authors of [24 25] proposed the use ofmultiple send-ing and receiving buffers to resolve the issue of receiver bufferblocking in cmt-SCTP Yuan et al [24] also introduced thefacility to differentiate subflows within an SCTP associationin terms of quality of service (QoS) requirements whilst in[25 26] the use of cmt-SCTP in wireless environments wasinvestigated mCMT [26] a cmt-SCTP variant for use inwireless networks employed multiple sending and receivingbuffers in a path-orientated multistreaming scheme whichalso incorporated the Media Independent Handover (MIH)scheme [27] for individual host mobility support

In addition to the many other cmt-SCTP based schemessuch as [28 29] for concurrent multipath transmission anumber of generic CMT schemes have also been proposedTsai et al [30] proposed a CMT scheme incorporatingforward error correction (FEC) and path interleaving withan adaptive FEC block size Liao et al in [19] proposedSMOS a sender-based multipath out-of-order scheduler forTCP-based multipath streaming whilst in [31] the authorsfurther considered the path correlation problem of sharedbottlenecks on end-to-end paths and proposed a new genericmultihoming sublayer

Although all of the above schemes whether SCTP or TCPbased offer solutions to some of the problems associated

International Journal of Digital Multimedia Broadcasting 5

with CMT they all only considered the case of individualmultihomed hosts There is a significant difference betweenthat environment and the one found in NEMO-based mobilenetworks In NEMO networks it is the mobility agents (HAand MR) that are multihomed rather than the end nodes(CN and MNN) In SCTP the end-to-end association isconducted over paths directly linking the network interfacesof the multihomed end hosts where the IP address of eachinterface that is used as the endpoint of the individual pathsin the association is known However in NEMO while theapplication flow is between the end nodes the independentpaths only exist between the interfaces of theHA and theMRIn CMT-NEMO [13] a mechanism is provided to enable theapplication to be split into subflows that are to be correctlyassociated with the interfaces at the HA and theMR to ensurethat they travel over the desired network path Each of theother CMT proposals described above was implemented atthe transport layer or in a cross-layer manner between thetransport layer and the application layer whilst the CMT-NEMO scheme operates at the RTPUDPNEMO protocolstack and is able to support multihomed mobile networks

In this work the components of CMT-NEMO are incor-porated into and become an integral part of our comprehen-sive multipath video streaming framework for multihomedNEMO environments Details of how the CMT componentsinteract with the other components of our framework areprovided in Section 3

3 Framework Implementation

In this work we introduce our comprehensive multipathstreaming framework for H264SVC and HEVC over multi-homed mobile networks The framework is a substantial fur-ther development of the scheme previously proposed in [12]Figure 3 gives an overviewof the framework (theH264SVC-specific implementation is shown) H264SVC and HEVCspecific implementations of the CMT-NEMO scheme [13]are combined with a modified version of the schedulingalgorithm from [12] which no longer needs to mitigate theNEMO path switching delay We also introduce a quality-layers-based packet weighting scheme for H264SVC asimple packet weighting scheme for HEVC and an improvedancestor checking schemewhich fully considers the granular-ity of H264SVC scalability In the framework the data planeprocesses and transmits video packets whilst the controlplane provides supporting functionality regarding networkpath conditions session setup and so forth Whilst the fullframework is shown in Figure 3 and has been implementedforH264SVC the current state of development of theHEVCstandard does not as yet permit full implementation ofcomponents that leverage multidimensional scalability suchas those responsible for target-rate derivation and extractionof substreams to a target bandwidth

Our framework includes CMT-NEMO-based compo-nents of a session setup control structure to determine thenumber of potential paths from the streamer to the clientand establish one application subflow (bound to a singlenetwork path) for each available path A flow disaggrega-tion mechanism overcomes the path switching limitation

observed in previous schemes and facilitates concurrentmul-tipath transmission Preprocessing modules use historicaldata from a target-bit-rate derivation subsystem for NEMO-based vehicular networks to determine target bit rates that areused by the encoder and bit stream extractor while a rate-distortion optimised stream is extracted using quality layersbased on [32]

Encoding and extraction functions employ the JSVMreference software [33] which is written in C++ The qualitylevel assigned to each NAL unit is also used to provide apacket prioritisation scheme that is specific to H264SVCand is used in our selective packet dropping mechanismin the case when there is insufficient aggregated bandwidthavailable

Each NAL unit in the stream is checked for decodingviability using a new more practical implementation of theancestor checking scheme used in [10] This mechanism isbetter suited to be used in real-time media-aware streamingsituations by replacing the time-consuming recursive search-ingmethod in [10] NAL units that pass the decoding viabilitytest are packetized for RTP transport over UDP A pathselection and packet scheduling algorithm then determinesif there is a viable path to the client and if more than onepath is found selects the path offering the earliest deliveryOut-of-sequence packet delivery to the client is mitigated atboth the streamer and the client in a manner that minimizesadded delay required to prevent out-of-sequence packetarrival at the client Finally successfully scheduled packetsare transmitted using the selected subflow with the subflowsbeing aggregated back to a single flow at the client prior topassing to the H264SVC decoder

31 Framework Design Considerations Placement of thestream adaptation path selection and packet schedulingagents within the network topology has a substantial effecton the viability of a multipath transmission scheme In [7ndash9] the agents were placed at a network proxy deployed at thepoint of path divergence (eg the HA in multihomed mobilenetworks) whilst in [10ndash12] the agents resided at the stream-ing server (CN) Both [10 11] only considered the case ofmultihomedhosts directly connected over independent pathsvia their multiple network interfaces A signaling schemein [12] sent control messages from the scheduling and pathselection agents at the CN to the actual point of divergence atthe HAwhere the path switching module directed the streamonto the desired path In this work we make the reasonableassumption that commercial multimedia content servers areunlikely to be equipped with interfaces that directly connectthem to multiple heterogeneous network technologies Thepoint of path divergence is more likely to be at a router withinthe network infrastructure In amultihomedmobile networkthe logical point of path divergence is the HA and the pointof path convergence is the MR

By considering the potential bidirectional use of the pro-posed streaming framework further constraints are imposedon the possible deployment of these agents Placement ofthe stream adaptation path selection and scheduling agentsat the point of divergence would put a substantial burdenon both the HA and for bidirectional use the MR Both of

6 International Journal of Digital Multimedia Broadcasting

Pathmonitoring

Controlplane

Target-ratederivation

Target base-

layer rate

Target

extractio

n

rate

Encoder

Bitstreamextractor

Packetweighting

Data plane

Subflow topath binding

lowast

lowast

lowastlowast

Flowdistribution

Pathselection

and packetscheduler

NAL unitpacketisation

Path metrics

Path m

etrics

Binding control andsession setup messages

Concurrent multipath transfer components

Figure 3 Overview of comprehensive streaming framework

these devices are already handling the mobility needs of theentire mobile network In the proposed scheme the agentsare placed at the streaming server All of stream adaptationpath selection and packet scheduling tasks are performed atthe CN or in the case of the potential bidirectional use at anappropriately resourced local fixed node (LFN) with a wiredconnection to the MR within the mobile network

32 Session Setup and Control The subflow to path bindingsubsystem is responsible for session setup and associatingsubflows with available paths The initial session setup con-sists of a series of control messages which are shown inFigure 4 A session setup agent at the MNN cooperateswith its counterpart at the CN to establish a streamingsession New streamlining sessions are initiated by themobilenetwork node which sends a ldquostream initiation requestrdquomessage (step 1 in Figure 4) to the CN The CN respondsby sending a ldquohome agent discoveryrdquo message addressed tothe MNN (step 2 in Figure 4) This message is interceptedby the HA that manages the mobility for the MR The HAreplies to the CN with a ldquoclient route identityrdquo messagecontaining the number of paths between the HA and the MRand the BIDs identifying each of its interfaces to the MR(step 3 in Figure 4) A reception ldquoport negotiationrdquo messageis then undertaken jointly by the CN and MNN The agreedreception port numbers are passed from the session setupagent to the subflow aggregation agent at the MNN (step4 in Figure 4) which opens listening ports to the CN andthen replies with a client ready message (step 5 in Figure 4)Upon the receipt of the ldquoclient readyrdquomessage the CN sends aldquosubflow bindingrdquo message for each available path (from HAto MR) to the HA (step 6 in Figure 4)

This message contains the (protocol source IP addresssource port destination IP address and destination port)quintuple identifying a subflow and the NEMO BID withwhich the subflow should be associatedTheHA then updatesthe binding cache and ensures that packets of the subflowdescribed by the quintuple in the ldquosubflow bindingrdquo message

Correspondentnode (CN)

(media server)

Homeagent (HA)

Multihomedmobile

router (MR)

Mobilenetwork

node(MNN)(1) Stream initiation request

(2) Request paths and BIDs

(3) Return paths and BIDs

(4) Port number negotiation

(5) Receiver ready status message

(6) Associate subflows to BIDs

(7) ACK subflow association

(8) Video stream transmission

Figure 4 Session setup

are delivered via the correct interface (BID) to the MRWhen the subflow associations are established the HA sendsa ldquosubflow acknowledgmentrdquo message to the CN (step 7 inFigure 4) which begins streaming to the client that is alreadyin the listening state The CN maintains a subflow to BIDbinding table for each streaming session Figure 5 showsthat application flow A is transmitted from the CN to theMNN Application subflows (A 1) and (A 2) belonging to thesame H264SVC HEVC or other application-type flows areassociated with NEMO BIDs 100 and 200 respectively at theHA

33 Packet Weighting and Ancestor Checking (PWAC)

331 H264SVC PWAC Scheme An H264SVC-encodedflow comprises a stream of logical data units called NetworkAbstraction Layer (NAL) units each of which has a one-byte H264AVC [2] compliant headerWhile base-layer NALunits only comprise this one-byte header and payload all

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 5: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 5

with CMT they all only considered the case of individualmultihomed hosts There is a significant difference betweenthat environment and the one found in NEMO-based mobilenetworks In NEMO networks it is the mobility agents (HAand MR) that are multihomed rather than the end nodes(CN and MNN) In SCTP the end-to-end association isconducted over paths directly linking the network interfacesof the multihomed end hosts where the IP address of eachinterface that is used as the endpoint of the individual pathsin the association is known However in NEMO while theapplication flow is between the end nodes the independentpaths only exist between the interfaces of theHA and theMRIn CMT-NEMO [13] a mechanism is provided to enable theapplication to be split into subflows that are to be correctlyassociated with the interfaces at the HA and theMR to ensurethat they travel over the desired network path Each of theother CMT proposals described above was implemented atthe transport layer or in a cross-layer manner between thetransport layer and the application layer whilst the CMT-NEMO scheme operates at the RTPUDPNEMO protocolstack and is able to support multihomed mobile networks

In this work the components of CMT-NEMO are incor-porated into and become an integral part of our comprehen-sive multipath video streaming framework for multihomedNEMO environments Details of how the CMT componentsinteract with the other components of our framework areprovided in Section 3

3 Framework Implementation

In this work we introduce our comprehensive multipathstreaming framework for H264SVC and HEVC over multi-homed mobile networks The framework is a substantial fur-ther development of the scheme previously proposed in [12]Figure 3 gives an overviewof the framework (theH264SVC-specific implementation is shown) H264SVC and HEVCspecific implementations of the CMT-NEMO scheme [13]are combined with a modified version of the schedulingalgorithm from [12] which no longer needs to mitigate theNEMO path switching delay We also introduce a quality-layers-based packet weighting scheme for H264SVC asimple packet weighting scheme for HEVC and an improvedancestor checking schemewhich fully considers the granular-ity of H264SVC scalability In the framework the data planeprocesses and transmits video packets whilst the controlplane provides supporting functionality regarding networkpath conditions session setup and so forth Whilst the fullframework is shown in Figure 3 and has been implementedforH264SVC the current state of development of theHEVCstandard does not as yet permit full implementation ofcomponents that leverage multidimensional scalability suchas those responsible for target-rate derivation and extractionof substreams to a target bandwidth

Our framework includes CMT-NEMO-based compo-nents of a session setup control structure to determine thenumber of potential paths from the streamer to the clientand establish one application subflow (bound to a singlenetwork path) for each available path A flow disaggrega-tion mechanism overcomes the path switching limitation

observed in previous schemes and facilitates concurrentmul-tipath transmission Preprocessing modules use historicaldata from a target-bit-rate derivation subsystem for NEMO-based vehicular networks to determine target bit rates that areused by the encoder and bit stream extractor while a rate-distortion optimised stream is extracted using quality layersbased on [32]

Encoding and extraction functions employ the JSVMreference software [33] which is written in C++ The qualitylevel assigned to each NAL unit is also used to provide apacket prioritisation scheme that is specific to H264SVCand is used in our selective packet dropping mechanismin the case when there is insufficient aggregated bandwidthavailable

Each NAL unit in the stream is checked for decodingviability using a new more practical implementation of theancestor checking scheme used in [10] This mechanism isbetter suited to be used in real-time media-aware streamingsituations by replacing the time-consuming recursive search-ingmethod in [10] NAL units that pass the decoding viabilitytest are packetized for RTP transport over UDP A pathselection and packet scheduling algorithm then determinesif there is a viable path to the client and if more than onepath is found selects the path offering the earliest deliveryOut-of-sequence packet delivery to the client is mitigated atboth the streamer and the client in a manner that minimizesadded delay required to prevent out-of-sequence packetarrival at the client Finally successfully scheduled packetsare transmitted using the selected subflow with the subflowsbeing aggregated back to a single flow at the client prior topassing to the H264SVC decoder

31 Framework Design Considerations Placement of thestream adaptation path selection and packet schedulingagents within the network topology has a substantial effecton the viability of a multipath transmission scheme In [7ndash9] the agents were placed at a network proxy deployed at thepoint of path divergence (eg the HA in multihomed mobilenetworks) whilst in [10ndash12] the agents resided at the stream-ing server (CN) Both [10 11] only considered the case ofmultihomedhosts directly connected over independent pathsvia their multiple network interfaces A signaling schemein [12] sent control messages from the scheduling and pathselection agents at the CN to the actual point of divergence atthe HAwhere the path switching module directed the streamonto the desired path In this work we make the reasonableassumption that commercial multimedia content servers areunlikely to be equipped with interfaces that directly connectthem to multiple heterogeneous network technologies Thepoint of path divergence is more likely to be at a router withinthe network infrastructure In amultihomedmobile networkthe logical point of path divergence is the HA and the pointof path convergence is the MR

By considering the potential bidirectional use of the pro-posed streaming framework further constraints are imposedon the possible deployment of these agents Placement ofthe stream adaptation path selection and scheduling agentsat the point of divergence would put a substantial burdenon both the HA and for bidirectional use the MR Both of

6 International Journal of Digital Multimedia Broadcasting

Pathmonitoring

Controlplane

Target-ratederivation

Target base-

layer rate

Target

extractio

n

rate

Encoder

Bitstreamextractor

Packetweighting

Data plane

Subflow topath binding

lowast

lowast

lowastlowast

Flowdistribution

Pathselection

and packetscheduler

NAL unitpacketisation

Path metrics

Path m

etrics

Binding control andsession setup messages

Concurrent multipath transfer components

Figure 3 Overview of comprehensive streaming framework

these devices are already handling the mobility needs of theentire mobile network In the proposed scheme the agentsare placed at the streaming server All of stream adaptationpath selection and packet scheduling tasks are performed atthe CN or in the case of the potential bidirectional use at anappropriately resourced local fixed node (LFN) with a wiredconnection to the MR within the mobile network

32 Session Setup and Control The subflow to path bindingsubsystem is responsible for session setup and associatingsubflows with available paths The initial session setup con-sists of a series of control messages which are shown inFigure 4 A session setup agent at the MNN cooperateswith its counterpart at the CN to establish a streamingsession New streamlining sessions are initiated by themobilenetwork node which sends a ldquostream initiation requestrdquomessage (step 1 in Figure 4) to the CN The CN respondsby sending a ldquohome agent discoveryrdquo message addressed tothe MNN (step 2 in Figure 4) This message is interceptedby the HA that manages the mobility for the MR The HAreplies to the CN with a ldquoclient route identityrdquo messagecontaining the number of paths between the HA and the MRand the BIDs identifying each of its interfaces to the MR(step 3 in Figure 4) A reception ldquoport negotiationrdquo messageis then undertaken jointly by the CN and MNN The agreedreception port numbers are passed from the session setupagent to the subflow aggregation agent at the MNN (step4 in Figure 4) which opens listening ports to the CN andthen replies with a client ready message (step 5 in Figure 4)Upon the receipt of the ldquoclient readyrdquomessage the CN sends aldquosubflow bindingrdquo message for each available path (from HAto MR) to the HA (step 6 in Figure 4)

This message contains the (protocol source IP addresssource port destination IP address and destination port)quintuple identifying a subflow and the NEMO BID withwhich the subflow should be associatedTheHA then updatesthe binding cache and ensures that packets of the subflowdescribed by the quintuple in the ldquosubflow bindingrdquo message

Correspondentnode (CN)

(media server)

Homeagent (HA)

Multihomedmobile

router (MR)

Mobilenetwork

node(MNN)(1) Stream initiation request

(2) Request paths and BIDs

(3) Return paths and BIDs

(4) Port number negotiation

(5) Receiver ready status message

(6) Associate subflows to BIDs

(7) ACK subflow association

(8) Video stream transmission

Figure 4 Session setup

are delivered via the correct interface (BID) to the MRWhen the subflow associations are established the HA sendsa ldquosubflow acknowledgmentrdquo message to the CN (step 7 inFigure 4) which begins streaming to the client that is alreadyin the listening state The CN maintains a subflow to BIDbinding table for each streaming session Figure 5 showsthat application flow A is transmitted from the CN to theMNN Application subflows (A 1) and (A 2) belonging to thesame H264SVC HEVC or other application-type flows areassociated with NEMO BIDs 100 and 200 respectively at theHA

33 Packet Weighting and Ancestor Checking (PWAC)

331 H264SVC PWAC Scheme An H264SVC-encodedflow comprises a stream of logical data units called NetworkAbstraction Layer (NAL) units each of which has a one-byte H264AVC [2] compliant headerWhile base-layer NALunits only comprise this one-byte header and payload all

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 6: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

6 International Journal of Digital Multimedia Broadcasting

Pathmonitoring

Controlplane

Target-ratederivation

Target base-

layer rate

Target

extractio

n

rate

Encoder

Bitstreamextractor

Packetweighting

Data plane

Subflow topath binding

lowast

lowast

lowastlowast

Flowdistribution

Pathselection

and packetscheduler

NAL unitpacketisation

Path metrics

Path m

etrics

Binding control andsession setup messages

Concurrent multipath transfer components

Figure 3 Overview of comprehensive streaming framework

these devices are already handling the mobility needs of theentire mobile network In the proposed scheme the agentsare placed at the streaming server All of stream adaptationpath selection and packet scheduling tasks are performed atthe CN or in the case of the potential bidirectional use at anappropriately resourced local fixed node (LFN) with a wiredconnection to the MR within the mobile network

32 Session Setup and Control The subflow to path bindingsubsystem is responsible for session setup and associatingsubflows with available paths The initial session setup con-sists of a series of control messages which are shown inFigure 4 A session setup agent at the MNN cooperateswith its counterpart at the CN to establish a streamingsession New streamlining sessions are initiated by themobilenetwork node which sends a ldquostream initiation requestrdquomessage (step 1 in Figure 4) to the CN The CN respondsby sending a ldquohome agent discoveryrdquo message addressed tothe MNN (step 2 in Figure 4) This message is interceptedby the HA that manages the mobility for the MR The HAreplies to the CN with a ldquoclient route identityrdquo messagecontaining the number of paths between the HA and the MRand the BIDs identifying each of its interfaces to the MR(step 3 in Figure 4) A reception ldquoport negotiationrdquo messageis then undertaken jointly by the CN and MNN The agreedreception port numbers are passed from the session setupagent to the subflow aggregation agent at the MNN (step4 in Figure 4) which opens listening ports to the CN andthen replies with a client ready message (step 5 in Figure 4)Upon the receipt of the ldquoclient readyrdquomessage the CN sends aldquosubflow bindingrdquo message for each available path (from HAto MR) to the HA (step 6 in Figure 4)

This message contains the (protocol source IP addresssource port destination IP address and destination port)quintuple identifying a subflow and the NEMO BID withwhich the subflow should be associatedTheHA then updatesthe binding cache and ensures that packets of the subflowdescribed by the quintuple in the ldquosubflow bindingrdquo message

Correspondentnode (CN)

(media server)

Homeagent (HA)

Multihomedmobile

router (MR)

Mobilenetwork

node(MNN)(1) Stream initiation request

(2) Request paths and BIDs

(3) Return paths and BIDs

(4) Port number negotiation

(5) Receiver ready status message

(6) Associate subflows to BIDs

(7) ACK subflow association

(8) Video stream transmission

Figure 4 Session setup

are delivered via the correct interface (BID) to the MRWhen the subflow associations are established the HA sendsa ldquosubflow acknowledgmentrdquo message to the CN (step 7 inFigure 4) which begins streaming to the client that is alreadyin the listening state The CN maintains a subflow to BIDbinding table for each streaming session Figure 5 showsthat application flow A is transmitted from the CN to theMNN Application subflows (A 1) and (A 2) belonging to thesame H264SVC HEVC or other application-type flows areassociated with NEMO BIDs 100 and 200 respectively at theHA

33 Packet Weighting and Ancestor Checking (PWAC)

331 H264SVC PWAC Scheme An H264SVC-encodedflow comprises a stream of logical data units called NetworkAbstraction Layer (NAL) units each of which has a one-byte H264AVC [2] compliant headerWhile base-layer NALunits only comprise this one-byte header and payload all

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 7: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 7

Accessnetwork 2

Home agent(HA) Access

network 1

Correspondentnode (CN)

(media server)

Mobile networknode (MNN)

Multihomedmobile router

(MR)Path =1

BID=100

Path = 2

BID = 200

Appl

icat

ion

flow

(A)

Mob

ile n

etw

ork

Subflow (A 2)Appli

catio

nflo

w (A

)

Subflow (A 1)

Figure 5 Application subflow to NEMO BID binding

0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7

F NRI Type R I N DID QIDPRID TID U D O RR

NAL unit payload

1-byte header 3-byte extension header

Figure 6 Internal structure of an H264SVC NAL unit

enhancement layer NAL units and supplementary enhance-ment information (SEI) messages also carry a three-bytescalability extension header The extension header carriesscalability information in three high-level syntax elements(shown in Figure 6) These syntax elements (dependency id)(temporal id) and (quality id) respectively describe the spa-tial temporal and quality layer characteristics of the NALunit

To assign different weights to the packets for selectivedropping in the case of insufficient bandwidth we employ abitstream extraction method for H264SVC based on [32]where the rate-distortion impact of each NAL unit on thebitstream is calculated and then utilized to assign a qualitylevel (ranging from 0 to 63) to the NAL unit The qualitylevel information is carried in a fourth high-level syntaxelement (simple priority id) within the extension header (oroptionally in accompanying SEI messages) After a qualitylevel is assigned to each NAL unit in the H264SVC streama scalable bitstream is extracted at the target bit rate by thebitstream extractor (which also receives a target extraction bitrate from the target-rate derivation subsystem)

The extracted stream is then passed from the JSVM bitstream extractor to the path selection and packet schedulingsubsystem where stream adaptation takes place In ourquality-layers-based packet prioritisation scheme we use thesimple priority id field to weight the importance of NALunits when deciding which packets should be dropped at theMANE Previous schemes for multipath transmission [10 12]

used a less advanced frame-based and non-H264SVC-specific weighting approach where the I frames were givena higher weighting than the P and B frames and so forth Inour quality-layers-based scheme NAL units with the lowestdistortion impact are dropped to adapt to changes in networkpath conditions

On a group of pictures (GOP) basis NAL units are sortedin priority order The NAL units are then presented to theancestor checking component within the path selection andpacket scheduling subsystem where packets that rely on apreviously dropped packet for decoding are dropped from thestreamThis preserves bandwidth by not sending packets thatcannot be successfully decoded by the clientThoseNALunitsthat pass the ancestor checking are then packetized based onRFC 6190 [34] A ldquoone NAL unit per RTP packetrdquo strategy isapplied to enhancement layer NAL units (Type 20) whilst aldquosingle time aggregation packet (STAP)rdquo strategy is employedfor an AVC base-layer NAL unit (Type 5) packetized togetherwith an NAL unit (Type 14) carrying the scalability extensionheader associated with the base-layer NAL unit

From an example shown in Figure 7 it can be observedthat packets numbered 6 and 13 are dropped since theestimated delivery time at the client was later than required bythe decoding process Because of these ancestor packets beingdropped their descendant packets numbered 9 and 15 whichare dependent on them are also dropped since they cannotsuccessfully be decoded without packets 6 and 13 Sendingpackets 9 and 15 would have been a waste of bandwidth

The ancestor checking scheme proposed in [10] used atime-consuming recursive searching method requiring thata search of all path queues is made for every new packetarriving at the scheduler in order to determine the status ofa packetrsquos ancestors In this work we propose a significantlyless computation-intensive method of ancestor checking thatis better suited to be used in a real-time environment byleveraging H264SVC scalability information A prefetchwindow of variable size is employed although in the exper-iments conducted for this paper the size was fixed at oneGOP Based on the scalable layer structure of the H264SVCstream we determine which packets are dependent (fordecoding) on others within the GOP based on the framenumber and scalability data contained in the NAL unitextension header (Figure 7) A record is maintained for theframe number and scalability data of dropped packets forthe current prefetch window This record is reset at thestart of a new prefetch window For each packet arrivingat the ancestor checking component a simple comparisonis made of the NAL unitrsquos frame number and scalabilityinformation (dependency id temporal id and quality id) tothe known failures within the current window Given thatthe number of NAL units in a prefetch window is relativelysmall our scheme provides a considerably faster and moreefficientmethod of ancestor checking for H264SVC streamscomparedwith [10] Ancestor checking in this work is limitedto NAL units dropped by the scheduler the possibility ofincluding an out-of-band feedback mechanism to delivertimely information on packets dropped in transit to theancestor checking component will be considered in futurework

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 8: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

8 International Journal of Digital Multimedia Broadcasting

19 18 17 16 14 12 11 10

15

13

69

8

7

4

2

1

5

3

Flow disaggregation

Subfl

ow (A

2)

Subfl

ow (A

1)

Path

=2

BI

D=200

Path

=1

BI

D=100

Arrive ontime

Earliestpath

Ancestorcheck

Disc

ard

pack

et

Discard

pack

et

Anc

esto

r not

sche

dule

d

Will

not a

rrive

on tim

e

Figure 7 The flow of packets through the path selection and packet scheduling subsystem

Tid ReservedTypeF N

0 7 15

NAL unit payload

HEVC NAL units (HM6)

Figure 8 Internal structure of an HEVC NAL unit

332 HEVC PWAC Scheme HEVC NAL units (in versionHM6) consist of a two-byte header followed by the payloadThree fields within an HEVC header influence the mannerin which packets can be prioritised These fields could beemployed by a selective packet dropping mechanism such asthose employed by anMANEThe nal ref flag (N in Figure 8)is a one-bit flag denoting whether the picture to which thisNAL unit belongs is used as a reference picture for otherpictures in the sequence

The 6-bit nal type field indicates the NAL unit type inmuch the sameway as forH264 variants however the field isextended from 5 in H264 to 6 bits in HEVC to accommodatethe increased number of NAL unit types both currently usedby HEVC and expected to be required for the envisagedextensions to HEVC The Temporal Layer ID field identifiesthe temporal prediction layer to which the picture containedin the NAL unit payload belongs Substreams representingtemporal layers ofHEVCencoded streams can be successfullydecoded when NAL units from higher temporal layers aremissing or removed from the stream

In this work we propose and employ a simple packetweighting scheme for HEVCwhich provides the highest levelof importance toNAL units of pictures that are InstantaneousDecoder Refresh (IDR) pictures Clean Random AccessPictures (CRA) or are marked as used for reference bythe nal ref flag header field NAL units are then furtherprioritised according to the temporal layer to which they

belong with the highest level of importance being attachedto the lowest temporal prediction layer

Ancestor checking in this pilot implementation of con-current multipath HEVC streaming is restricted to ensuringthat within a group of pictures (GOP) NAL units of highertemporal layers are only transmitted if the NAL units of thelower temporal layers from which they are predicted havebeen successfully transmitted

34 Path Selection and Packet Scheduling A path state moni-toring subsystem previously described in [12] provides pathconditions to the path selection component This compo-nent is also provided with the full network size of thepacket including all NEMO-related tunneling overheadsTheestimated arrival time on each available network path iscalculated and if no path is able to deliver the packet tothe client on time to be of use in the decoding process it isdropped Where a single viable path is found the packet ispassed to the flow disaggregation subsystem which transmitsit on the subflow associated with the chosen network path inthe subflow toBIDbinding tableWheremore than one viablepath is found the packet is transmitted on the path offeringthe earliest estimated arrival time at the client

Figure 7 shows the flow of packets through the pathselection and packet scheduling subsystem to the flow disag-gregation subsystem Using the NEMO protocol results in anadditional network overhead due to IPv6 tunneling betweenthe HA and the MR In our streaming framework we takeaccount of all networking overheads when estimating thepacket arrival time at the client for path selection purposes

Network overheads of 100 bytes are added for every NALunit (or fragment of an NAL unit) sent in a single levelNEMO based mobile network (ie no other mobile networksuch as a personal area network nested within the mobilenetwork)These network overheads consist of the initial IPv6header containing the destination address of the MNN (40bytes) the tunnelling IPv6 header (40 bytes) containing the

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 9: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 9

care of address (CoA) of the mobile router an 8-byte UDPheader and a 12-byte (minimum)RTPheader On average thebandwidth required to transmit an HEVC encoded bitstreamis increased by 9 in the NEMO environment due to thesenetwork overheadsThis observation is based on a strategy ofone NAL unit per RTP packet

35 Out-of-Sequence Packet Handling Out-of-sequence de-livery of packets to the client is a significant problem inmultipath delivery systems It is especially prevalent inheterogeneous networks where each path has different char-acteristics such as available bandwidth and end-to-end delayResource constrained mobile devices may be limited in theamount of memory available for allocation to the receiverbuffer of the client application It is important to note thatthe nature of packet-switched networks often means thatsome level of out-of-sequence delivery to the client is oftenunavoidable

Where a router has the choice of more than one pathto a destination it may choose to send some packets of thesame application flow over different links to for exampleavoid congestion on a particular link or for load balancingpurposes Especially in busy wireless environments it ispossible that there will be sudden changes in the availablebandwidth or end-to-end delay due to nodes either leaving orjoining the network or contention in thewireless network thatrequires retransmissions Since the sender-based approachhas been proved to yield better performance in handlingout-of-sequence packets [19 35] we employ a sender-basedout-of-sequence mitigation scheme that is supplemented byadditional practical measures at the client

At the CN the estimated arrival time 119905119886

119894of a packet pi

on each available path is calculated using the full networksize of the packet and the end-to-end delay Each packet hasa decoding deadline 119905

119889

119894or time by which it must arrive at

the client in order to be of use in the decoding process Thedecoding deadline of a packet is calculated from the framerate of the video and is relative to decoding time of the firstpacket in the streamThe decoding window of a packet opensat the decoding deadline (relative to the first packet) andcloses at 119905119889

119894+ Δ where Δ is the playback delay at the client If

no available path can provide an estimated arrival time where119905119886

119894ge (119905119889

119894+ Δ) the packet is dropped at the streamer and the

failure points for the current prefetch window are updated toensure that any subsequent packet relying on this packet fordecoding is also dropped

If only one viable path is found the packet is passed tothe flow disaggregation subsystem and placed on the subflowassociated with the viable path found Where more than onepath can deliver the packet within the decoding window thepath offering the earliest arrival time is used In order toprevent out-of-sequence delivery of packets at the client twofactors are considered in the delay packet function describedin [13]

Firstly if 119905119886119894lt 119905119889

119894 the packet will arrive before the opening

time of its decoding window and will occupy the decoderbuffer until its decoding deadline arrives Dependent on thesize of the playback buffer at the client packets arriving before

the decoding window opens may lead to out-of-sequencedelivery Preventing a packet from arriving before the startof its decoding window (119905119886

119894= 119905119889

119894) may not prevent out-of-

sequence delivery which can only be ensured when 119905119886

119894gt 119905119886

119894minus1

Therefore when considering the estimated arrival time of apacket on any given path if 119905119886

119894lt 119905119886

119894minus1 the packet would need

to be delayed by 119889119894= (119905119886

119894minus1minus 119905119886

119894) on that path to ensure that it

would not arrive before the previous packetWhen it is necessary to delay a packet at the streamer

(CN) to prevent out-of-sequence packet delivery to the client(MNN) the path with the lowest added delay (119889

119894) is chosen

The added delay is taken into account when calculating theestimated arrival time of the next packet on the path wherethe delay was added thus preventing any temporal drift inthe estimated arrival time calculations

36 Flow Disaggregation The ABC approach to switchingan application flow among network paths in NEMO-basedmobile networks has previously been used in [12 15] Itswitches the application flow between the interfaces of theHA thereby changing the path used between HA and MR

An average delay of 137ms is introduced for each pathswitching operationThe delay consists of the time for the CNto send a path switch message to the HA the implementationof the path switch at the HA and the waiting time until theCN receives a path switch acknowledgement from the HATheoverall delaymeasured in our testbed is likely to be higherin real implementations where the path between CN and HAmay have a higher delay

Our interpretation of the results in [15] indicates pathswitching between 200 and 300ms The CMT-NEMO basedframework in this paper splits the application flow into anumber of subflows at the CN Each subflow is associatedwith a separate listening socket at the MNN and with aspecific network interface at the HA

Subflows travel across the path to which they are boundwith all of the mobility-related issues being handled by theexisting NEMO protocol running at both HA and MR

Table 1 compares CMT streaming over NEMO (CMT-NEMO) with both Always Best Connected streaming overNEMO (ABC-NEMO) and cmt-SCTP

The flow disaggregation scheme consists of softwareagents at the CN HA andMNNThe agents at the CNwouldbe replicated at the LFN and those at the HA replicated at theMR for bidirectional streaming An application flow is splitinto subflows by creating a listening socket for each availablepath at the MNN and providing a subflow aggregation andout-of-sequence mitigation agent at the MNN Such a designis applicable to applications beyond video streaming usingH264SVC or HEVC

37 Algorithm Summary Algorithms 1ndash3 highlight the mainalgorithms implemented in the streaming framework Atthe streamer CMT-NEMO from [13] is fully integratedwith updated path switching and packet scheduling modulesfrom [12] which now include a revised ancestor checkingscheme enhanced out-of-sequence mitigation and CMT-NEMO packet to subflow distribution

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 10: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

10 International Journal of Digital Multimedia Broadcasting

Table 1 Features of the three schemes

Always Best Connected(ABC) NEMO

Concurrent multipath transfer(CMT) NEMO cmt-SCTP

Use of paths Switched sequentialtransmission Concurrent transmission Concurrent transmission

Bandwidth A trade-off against pathswitching overhead

Effective bandwidthaggregation is achievedAggregation is suboptimal due tothe relatively small overheadrequired to preventout-of-sequence packet delivery(average 14ms)

Effective bandwidthaggregation

Aggregation

Higher levels of aggregationlead to increased switchingoverhead and lower QoEfor the user

The reliable nature of SCTPcan reduce throughput dueto both control messagesand retransmissions

Path switchingoverheads Average 137ms [12] No path switching delay incurred

No path switching delayincurred

Control plane overheadsInitial session setup controlmessages

Initial session setup controlmessages

Messaging foracknowledgement ofdelivery retransmissionsand congestion controlthroughout streamingsession

Path switching REQ andACK between CN and HAfor every path switchingoperation

No path switching messagesduring session

Multihomingrequirement

The HA and the MR in theinfrastructure aremultihomed

The HA and the MR in theinfrastructure are multihomed

All end hosts aremultihomed

Mobility support All mobile hosts in thewhole mobile network

All mobile hosts in the wholemobile network

An individual mobile hostonly

10 STREAM PRE-PROCESSOR (H264SVC)15 Get number of available paths119898 from Home Agent20 For each available network path 119899

30 Get current bandwidth 119887119888

119899from path monitoring sub-system

40 Get max bandwidth 119887ℎ

119899min bandwidth 119887

119897

119899from route history file

50 If 119887119888119899lt 119887119897

119899lowest known bandwidth 119887

119897119896

119899= 119887119888

119899 else 119887119897119896

119899= 119887119897

119899

60 If 119887119888119899gt 119887ℎ

119899highest known bandwidth 119887

ℎ119896

119899= 119887119888

119899 else 119887ℎ119896

119899= 119887ℎ

119899

65 Next path70 Base-layer target rate 119877bl =sum

119899=119898

119899=1(119887119897119896

119899)

80 Extraction target rate 119877ex =sum119899=119898

119899=1(119887ℎ119896

119899)

90 If real time streaming encode scalable bit stream using 119877bl

100 Perform rate distortion calculations to each NAL unit110 Assign quality level for each NAL unit to simple priority id in NAL header as defined in [32]120 Extract scalable bit stream at target 119877ex

Algorithm 1 Stream preprocessing algorithm (H264SVC version)

570 CLIENT SUB-FLOWAGGREGATOR580 Repeat590 For each available path600 Read path received buffer610 Next path620 Sort received packets by RTP timestamp630 Deliver aggregated flow to decoder640 Until stream finished or receiver time out

Algorithm 2 Client side aggregation of subflows

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 11: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 11

130 STREAMING SESSION SETUP (H264SVC)140 For each available path 119899

150 Get BID(119899) from Home Agent160 Create sub-flow 119878

119899

170 Create sub-flow binding table entry at streamer for 119878n 997888rarr BID(119899) association180 Signal 119878

119899997888rarr 119861119868119863(119899) association to Home Agent

190 Next 119899200 SCHEDULER210 Sort each packet in pre-fetch window by simple priority id and 119905

119889

119894

220 If start of new pre-fetch window230 Reset ancestor fail points240 End If250 For each packet in pre-fetch window260 ancestor check routine270 For each fail point in pre-fetch window280 If fail point is ancestor of this packet290 Drop packet300 Update fail points310 Break320 End If330 Next fail point340 Estimate arrival time350 Calculate mobility overheads as per [12]360 Add mobility overheads to packet size370 For each available path 119899

380 Calculate 119905119886119894(119899) as per [12]

390 If 119905119886119894(119899)le (119905119889

119894+Δ)

400 If 119905119886119894(119899)lt 119905

119886

119894minus1(119899)

410 119889119894(119899) = (119905119886

119894minus1(119899)minus 119905

119886

119894(119899))

420 End If440 End If450 Next n460 INTEGRATED SELECTION AND PACKET SCHEDULING470 If viable path found480 Use path with earliest 119905119886

119894

490 If multiple paths with same 119905119886119894use path with lowest 119889

119894

500 Lookup sub-flow to BID binding table510 Add packet to sub-flow queue associated with chosen path520 Else530 Drop packet540 Update fail points550 End If560 Next packet

Algorithm 3 Main algorithm of the streaming framework incorporating CMT-NEMO path selection and packet scheduling (H264SVCversion)

The preprocessing steps are detailed in Algorithm 1 usingH264SVC as an example where streams are preprocessedusing path metrics from the path monitoring and targetrate derivation subsystems The stream is matched to theprevailing network conditions and packets are prioritizedas described in Algorithm 3 Subflows are aggregated at theclient where additional out-of-sequence mitigation is alsoperformed as shown in Algorithm 2 For the HEVC pilotimplementation where the ability to extract a video streamto a target bandwidth is not yet available the bandwidths of

the network paths are set to match the requirements of theHEVC bitstream

38 Control Plane Subsystems Our framework contains threesubsystems within the control plane The path monitoringsubsystem provides path state information on available band-width and delay to the scheduling algorithm

Network emulation software running at a core routeron each network path between the home agent and themobile router changes the bandwidth and end-to-end delay

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 12: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

12 International Journal of Digital Multimedia Broadcasting

Figure 9 Testbed

on each path autonomously These changes are reported tothe streamer using a series of low overhead control messagesA default message frequency of one per second is usedhowever when the network emulator makes changes to apath (eg by reducing available bandwidth) to emulate theinsertion of background traffic a control message is triggeredimmediately Although not implemented in our testing sce-nario an out-of-band feedback mechanism for congestioncontrol is considered for future work For instance a schemefor H264SVC congestion control in UDP was developedin [36] which is compatible with and could be integratedwith our framework (an HEVC specific congestion controlscheme is yet to be designed though) The subflow to pathbinding subsystem is described in Section 32 A target-ratederivation subsystem uses a combination of current andhistorical path metrics to make an informed decision onthe bit rates at which a scalable stream should be bothencoded and extracted to best match the anticipated networkconditions These control plane subsystems and the dataplane subsystems described before constitute a practical andfully functional streaming system for concurrent multipathdelivery of H264SVC or HEVC video to mobile networkusers

39 Physical Testbed Implementation Our framework hasbeen implemented on a realistic hardware-based multi-homed mobile networks testbed for testing and evaluationpurposes Figure 9 shows the actual testbed The basic topol-ogy of the testbed is the same as that depicted in Figure 1 withthe addition of core routers providingWAN emulation on theaccess network paths betweenHAandMRThe access routersare modified Linksys WRT54GL wireless routers runningopen source software OpenWRT With the exception of theCN which is an Intel Core i5 PC with 4GB RAM and a solidstate drive all remaining nodes in the testbed are standardIntel Pentium 4 PCs with 1 GB RAM All PCs run the UbuntuLinux operating system with open source Network Mobility(NEMO) software installed at the HA and the MR

The components of our framework are implemented inLinux user space using C++ with Python for pre- and post-processing tasks The core routers run wide-area-network(WAN) emulation software and are configurable to changethe bandwidth or delay on each path

Options are available to run both static tests where thecharacteristics of a path remain unchanged during a stream-ing session or to dynamically and independently change thenature of each path (within defined upper and lower limits)

4 Experimental Evaluation

For H264SVC evaluation four well-known video testsequences (Bus Foreman Paris and Soccer) are encodedwith the JSVM reference software using a range of spatialresolutions (QCIF CIF and 4CIF) and temporal resolutions(15 30 and 60 fps) As a streampreprocessing step the qualitylevel data is generated using the QualityLevelAssignerStatictool in the JSVM reference software The bit stream is thenextracted at a bit rate equal to the highest available aggregatedbandwidth that will be configuredwithin the testbed environ-ment for the testing of that particular sequence

Aggregated bandwidths in a range from 64 kbps to3Mbps and path delays of 20ms to 250ms are used duringtesting Some tests are conducted using a static (for theduration of a streaming session) bandwidth and delay whilstin others the effects of competing background traffic (fromother nodes in the network) are emulated when bandwidthand delay change dynamically during a session

For HEVC evaluation two standard compliant HEVC testsequences (Racehorses BQSquare) are encoded using version6 of the HEVC reference software [22] Each sequence isencoded using the standard HEVC conformance configura-tion files for Intraprediction Only Low Delay and RandomAccess temporal prediction encoding modes Two spatial res-olutions (832 times 480 416 times 240) and two temporal resolutions(30 fps 60 fps) are employed Encoded video bitrates are notmanipulated by any means other than selection of differenttemporal prediction modes

In the HEVC experiments the bitrate at which thevideo sequence is encoded using the standard conformanceconfiguration is assumed to be the target bitrate and theWAN emulation software is provided with these values

Data from the CN HA CR1 CR2 and the MNN iscollected in log files and the network analysis toolWireshark[37] is running at the HA CR1 CR2 and the MR to allowcollection and inspection of packets ldquoin flightrdquo The log file atthe CN details the scheduling decision made for each packetincluding the data used to make that decision (packet sizepriority weighting scalability and frame number data andestimated arrival time on each available path) The log at theMNN records all packets received and their arrival time

The HA log shows subflow to BID binding data andthe CR logs contain details of path state changes and pathstate updates that have been transmitted as control messagesto the CN In the results highlighted in this paper com-parisons are made between Always Best Connected NEMO(ABC-NEMO) [12] and the video streaming frameworkwhich incorporates CMT-NEMO [13] with the describedsupporting subsystems of quality-layers-based prioritiza-tion streamlined ancestor checking and improved out-of-sequence packet handling

41 Picture Quality Comparison Using H264SVC In eachfigure (Figures 10 11 12 13 and 14) schemes are comparedwhen the available bandwidth has been allocated in a 50 50split between the paths (equal paths) and also with an 80 20split between the paths (differential or diff paths) In eachfigure the legend denoting CMT-NEMO is used for brevity

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 13: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 13

Aggregated bandwidth on paths (kbps)

400 600 800 1000

PSN

R (d

B)

24

26

28

30

32

34

36

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Bus CIF 30 fps

Figure 10 Comparison of schemes for the Bus sequence at 30 fps

Aggregated bandwidth on paths (kbps)

1500 1750 2000 2250 2500 2750 3000 3250

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Soccer CIF 30 fps

Figure 11 Comparison of schemes for the Soccer sequence at 60 fps

and identifies the results obtained using the comprehensivestreaming framework which incorporates CMT-NEMO

Concurrent multipath transmission performs best inequal path scenarios for CMT-NEMObecause of the reducedincidence of out-of-sequence packet delivery when paths areequal In CMT-NEMO based framework the sender-basedout-of-sequence packet mitigation mechanism ldquoholds backrdquopackets that would arrive before the previously sent packet

In addition Figure 14 shows the results of streaming theParis sequence at lower bit rates using an encoder targetbase-layer rate of 64 kbps Overall concurrent multipathtransmission performs best on equal paths providing aremarkable average PSNR improvement of 608 dB across all

Aggregated bandwidth on paths (kbps)1000 1500 2000 2500 3000

PSN

R (d

B)

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Paris CIF 30 fps

Figure 12 Comparison of schemes for the Paris sequence at 30 fps

750 1000 1250 1500 1750Aggregated bandwidth on paths (kbps)

PSN

R (d

B)

22

24

26

28

30

32

34

36

38

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different pathsABC-NEMO equal paths

Foreman CIF 30 fps

Figure 13 Comparison of schemes for the Foreman sequence at30 fps

testing scenarios with a maximum improvement of up to872 dB being achieved in some tests

Where the paths are differential CMT-NEMO still out-performs ABC-NEMO considerably by an average of 420 dBand a maximum of 833 dB in some test sequences

As the delay caused by out-of-sequence mitigation is lessin equal path scenarios it is possible to better utilize theavailable paths leading to an improvement in the number ofpackets arriving at the client on time and thus the resultantPSNR Figure 15 shows both the number of packets arrivingat the client and those that are usable in the decoding processfor each scheme

It can be seen from Figure 15 that more packets aredelivered over equal paths using the CMT-NEMO basedframework with the opposite being true for ABC-NEMO

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 14: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

14 International Journal of Digital Multimedia Broadcasting

64 128 192 256

PSN

R (d

B)

20

25

30

35

CMT-NEMO equal pathsCMT-NEMO different paths

ABC-NEMO different paths

Aggregated bandwidth on paths (kbps)

ABC-NEMO equal paths

Paris QCIF 30 fps

Figure 14 Comparison using the Paris sequence at low bandwidths

Packet delivery ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

50

60

70

80

90

100

Equal paths deliveredEqual paths usable

Different paths deliveredDifferent paths usable

Figure 15 A comparison of packet delivery ratios at the client

In ABC-NEMO the path switching frequency is bettercontrolled for differential path situations than for equal pathscenarios Path switching only takes place if the current(last used path) can no longer deliver packets within theirdecoding deadline

Where the characteristics (delay and bandwidth) of theavailable paths are equal ABC-NEMO is less effective aspackets are distributed more evenly across the paths whichcreates a higher path switching frequency Each path switch-ing adds a switching overhead thereby reducing the numberof packets that can be delivered in a given time ThereforeABC-NEMO performs better in terms of the number ofpackets delivered and the resultant PSNR in differential pathsituations In concurrent multipath transfer the oppositeapplies in that performance is better on equal paths

Out-of-sequence packet delivery to the application run-ning at the client is low for both the concurrent multipath

Out-of-sequence packet ratio

CMT-NEMO ABC-NEMO

Pack

et d

eliv

ery

ratio

()

00

02

04

06

08

10

12

Equal pathsDifferent paths

Figure 16 Comparison of out-of-sequence packet delivery rates

streaming framework and ABC-NEMO schemes at less than1However it is noted that in ABC-NEMOout-of-sequencepackets delivery mainly takes place during path switchingoperations only due to the sequential transmission styleand thus the out-of-sequence packet delivery ratio is low innature In contrast due to concurrent transmission in CMT-NEMO based streaming framework out-of-sequence packetdelivery could occur all the time and thus it must be tackledmore rigorously through both sender and receiver mitigationmechanisms

Due to the robust mitigation scheme the number ofpackets sent out-of-sequence to the client in CMT-NEMO isminimized even lower than that of ABC-NEMO as shownin Figure 16

The delay imposed by the out-of-sequence mitigationscheme at the sender is the primary cause of the reductionin throughput in differential path situations Our frameworkgives a packet delivery ratio at the client (Figure 15) thatis up to 24 higher than ABC-NEMO The quality-layers-based weightings are employed in selective packet droppingto ensure that packets are dropped in a rate-distortionoptimised manner The higher number of packets arrivingat the client contributes to a substantially improved PSNRvalue for all sequences The PSNR results for concurrentmultipath streaming represent a reduction of up to 68 inthe ldquoperformance gaprdquo identified in [38] for equal paths andup to 55 for differential paths

The results obtained are valid across the whole range ofvideo sequences and testing scenarios used in our experi-ments Table 2 provides results of additional testing usingalternative combinations of spatial and temporal resolutionsfor each of the test sequences

42 Picture Quality Comparison Using HEVC The HM6version of the HEVC reference software [22] deployed inour framework and experiments does not offer any inherent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 15: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 15

Table 2 Average PSNR and packets delivered for additional sequences

BusQCIF30Hz

Soccer4CIF30Hz

ForemanQCIF15Hz

ParisCIF60Hz

ABC-NEMOAvg PSNR (dB) 2722 2535 2697 2582

CMT-NEMOAvg PSNR (dB) 3243 3149 3316 3299

ABC-NEMOUsable packet delivery ratio 7632 7217 7358 7465

CMT-NEMOUsable packet delivery ratio 9416 9221 9437 9394

resilience to packet loss consequently we adopt a ldquoframecopyrdquo based error concealment method in which a missingpicture due to lost NAL units is copied either from the imme-diately previous picture (in the output order) or the nearestreference picture in the HEVC short term reference picturelist if available in the reference picture listThe packet priorityweighting scheme employed prioritises those pictures thatare used for reference Generally in the Low Delay and theRandom Access configurations of HEVC which are bothinterpicture prediction modes our experiments have shownthat pictures of the highest temporal layer are unused forreference and may be safely discarded to meet a bandwidthconstraint In the Intraprediction Only configuration modeall pictures are intraencoded with temporal prediction ordependencies

Figure 17 compares the PSNR achieved by ABC-NEMOand the CMT-NEMO based streaming framework when theRacehorses (832 times 480 30 fps) sequence was streamed overmultiple paths The aggregated bandwidth was set to theencoded bandwidth of 2253Mbps which was achieved byencoding using the Low Delay main profile configurationwith a quantisation parameter value of 27ThePSNR achievedwhen no packet loss is encountered is shown as the ldquoencodedbitraterdquo In Figure 18 the BQSquare sequence is shown (416 times

240 60 fps) The HEVC examples shown in Figures 17and 18 were conducted using equal path settings where theavailable bandwidth was equally split between the two pathsPacket loss ratios and out-of-sequence delivery ratios forHEVC encoded sequences were consistent with those for theH264SVC experiment showing that the framework behavedconsistently while delivering application flows are encodedunder the two video encoding standards Overall results froma small test sample of five test runs for each HEVC encoderconfiguration and test sequence combination showed thatthe PSNR losses relative to those of the encoded sequencebefore transmission were slightly higher for HEVC than forH264SVC

Losses when comparing ABC-NEMO to the originalsequence were on average 032 dB higher for HEVC thanH264SVC and on average 028 dB when comparing con-current multipath transmission We attribute this perfor-mance issue to the relatively unsophisticated packet weight-ing scheme and error concealment mechanisms used in this

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

Racehorses HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 17 Comparison using HEVC at 2252Kbps

pilot implementation for HEVC The results of our HEVCexperiments show that our comprehensive video stream-ing framework for use in multihomed mobile networkscan be readily adapted from its initial implementation forH264SVC to other video codecs and can potentially beapplied in a generalised form for the concurrent multipathtransmission of other types of application flow

43 Delay and Jitter In addition to PSNR-based video qualitymeasurement and packet loss statistics we also considerend-to-end delays and interpacket delay variation (IPDV) orjitter calculated based on RFC 1889 [39] Both metrics haveimportant impacts on a userrsquos QoE [40 41]

In Figures 19 to 21 we illustrate typical delay and IPDVcomparisons of ABC-NEMO and CMT-NEMO using theParis sequence at CIF resolution and 30 fps A fixed delay of25ms is introduced by the WAN emulation module on eachnetwork path The available bandwidth of 15Mbps has beenallocated in a 50 50 split between the paths (equal paths) andequates to the 1500Kbps testing point shown in Figure 14Therunning IPDV is shown in Figure 19

Although both schemes maintain a mean IDPV below50ms the CMT-NEMO based framework performs signif-icantly better at less than 10ms as shown in the inset The

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 16: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

16 International Journal of Digital Multimedia Broadcasting

27

28

29

30

31

32

33

34

35

36

37

PSN

R (d

B)

BQSquare HEVC-Low Delay

Encoded bitrateCMT-NEMOABC-NEMO

Figure 18 Comparison of HEVC at 763Kbps

Jitter-running IPDV

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Runn

ing

IPD

V (m

s)

0

10

20

30

40

50

60

70

ABC-NEMOCMT-NEMO

0 20 40 60 80 100456789

10

CMT-NEMO

Figure 19 Comparison of jitter (running IPDV)

instantaneous IDPV for each packet is plotted in Figure 20where the ldquospikesrdquo caused by path switching induced delaysin ABC-NEMO can be clearly identified Similar spikes canalso be observed in running IPDV (Figure 19) and delay(Figure 21) The improvement offered by CMT-NEMO sig-nificantly reduces delay and IPDV which can have beneficialimpact on client buffer sizing andmitigating potential buffer-ing problems that lead to degraded QoE Table 3 shows thatthemaximumand average IPDVs are reduced by 1521ms and237ms respectively when using CMT-NEMO comparedwith ABC-NEMO

In Figures 20 and 21 a spike in both IPDV and delayfor CMT-NEMO occurs in the region of packet number 10

Jitter IPDV

IPD

V (m

s)

0

0

20 40 60 80 100

2030

10

4050

CMT-NEMO

Packet number0 10 20 30 40 50 60 70 80 90 100 110

0

50

100

150

200

250

300

350

ABC-NEMOCMT-NEMO

Figure 20 Comparison of jitter (IPDV)

End-to-end delay

Packet number0 10 20 30 40 50 60 70 80 90 100 110

Del

ay (m

s)

0

50

100

150

200

250

300

350

400

ABC-NEMOCMT-NEMO

0 20 40 60 80 10020

40

60

80

CMT-NEMO

Figure 21 Comparison of end-to-end delays

This is due to the nature of an H264SVC stream wherethere can be a significant variation in NAL unit (and thuspacket) sizeThe first few packets in the stream are parameterset NAL units which are very small in comparison with theimmediately following first video coding layer (VCL) NALunits which are the instantaneous decoding refresh (IDR)units and as such are generally the largest in the stream

This difference in propagation time between the smallestand the largest NAL units in the stream results in a spike indelay and IPDV Table 3 shows that IPDV is further reducedin CMT-NEMO when the initial parameter set packets are

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 17: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 17

Table 3 IPDV comparison (milliseconds or ms)

MaxIPDV

AvgIPDV STDEV

ABC-NEMO 19800 2977 5355

CMT-NEMO 4690 607 733CMT-NEMOExcluding ParameterNALs

2842 567 541

filtered However it should be noted that filtering these pack-ets has the opposite effect on ABC-NEMO as discounting thevery small IPDV between each of the parameter set packetsincreases the mean IPDV for the testing range

44 Background Traffic Injection In addition when back-ground traffic is emulated by reducing the available band-width on a path within the network we observe that there isa short lag between the reduction taking place and the sched-uler reacting to the change (not illustrated here for brevity)Typically two or three packets experience an additional delayof between 20ms and 50ms and increased jitter (IPDV)before the system adapts to the new bandwidth when thereis a uniform increase in delay (but not jitter) reflecting thereduced bandwidth

A number of experiments were conducted in whichreal background video traffic was introduced between thestreamer server and the client node Performance of thestreaming framework was measured under a range of back-ground traffic rates The background traffic was transmittedover a single path and also simultaneously over multiplepaths For each of the 119899 available paths in the system thebandwidth set at theWANemulator is119901

119894(Kbps) and for each

of the119898 background flows in the system the bandwidth usedby the flow is 119891

119894(Kbps) The background traffic injection rate

119877 is the sum of all background traffic flows 120573119905divided by the

sum of all path bandwidths 120573119886

119877 =

120573119905

120573119886

=

sum (1198911 1198912 119891

119898)

sum (1199011 1199012 119901

119899)

(1)

The gross transmission efficiency119866 of a streaming systemis a measure of the utilisation of the aggregated bandwidth ofall available paths while the effective transmission efficiency119864 is a measure of how much of the aggregated bandwidth isbeing used to deliver useable video payload to the client sidedecoder For each packet 119894 the size (in Kbytes) of the NALunit(s) contained in the packet payload is 119904

119894 and the network

overheads required to encapsulate them for transmission byRTP over UDP in the NEMO environment is Δ

119894 The full

network resource (in Kbytes) required to deliver a packet is120593119894= 119904119894+ Δ119894 The number of packets successfully delivered to

the client is 119903 and the number of useable packets containingNAL units which could be successfully decoded (arrived ontime to be of use in the decoding process and had no unmet

dependencies or missing fragments) is 120583 119871119904is the duration

of the streaming session in seconds Consider

119866 =

(sum (1205931 1205932 120593

119903) 119871119904)

(120573119886minus 120573119905)

119864 =

(sum (1199041 1199042 119904

120583) 119871119904)

(120573119886minus 120573119905)

(2)

In (2) themaximumpotential throughput which could beachieved by an application flow is shown for the case where 119877remains constant for the duration of a streaming session Incases such as those shown in Figure 22 where119877was varied atevery 30 to 50 seconds during a streaming session the max-imum throughput potential used in efficiency calculationswas the sum of (120573

119886119909minus 120573119905119909)119871119904119909 where 119909 is a time slot during

which 119877 remained constant (eg in Figure 22 119877 is constantat 50 from elapsed time = 40 seconds until elapsed time =70 seconds)

Distortion efficiency 119863 is measured as the ratio ofachieved visual quality 120575

119886tomaximumpossible visual quality

120575119898for each encoded sequence

119863 = (

120575119886

120575119898

) (3)

From Figure 22 it can be seen that in both ABC-NEMOand CMT-NEMO the streaming mechanism respondsto changes in the rate of background traffic injectionCMT-NEMO delivers a consistently higher PSNR thanABC NEMO It was observed that for a short period afterany change in 119877 both schemes delivered a reduced PSNRon one or two frames as the streamer adapts to changes inbackground traffic This initial drop in PSNR after a changein 119877was consistently more pronounced in ABC-NEMO thanin CMT-NEMO

It can be seen from Figure 12 to Figure 16 that therelative difference in bandwidth and delay between pathsin the multipath streaming system leads to a differencein the measured quality of the received video stream Theinjection of background traffic when applied to a singlepath changes the bandwidth and delay ratios between thepaths This leads to an increased deviation from the meanPSNR for each test sequence For example in a systemwith two unequal paths if traffic is added to the higherbandwidth path this leads to an equalisation of the pathcharacteristics As CMT-NEMO performs better in equalpath situations the drop in PSNR due to added backgroundtraffic is offset in part by the increased effectiveness of thescheme whereas in ABC-NEMO the move towards equalpaths makes the scheme less efficient with the loss in PSNRbeing exacerbated by the reduction in scheme efficiencyThe reverse also holds true when traffic is injected into onepath in an equal path system In Figure 23 it can be clearlyseen that CMT-NEMO has a transmission efficiency that isapproximately 20higher than that of ABC-NEMOresultingin vastly increased distortion efficiency In both schemesthe relative difference between gross transmission efficiencyand effective transmission efficiency increases slightly as 119877

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 18: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

18 International Journal of Digital Multimedia Broadcasting

15

20

25

30

35

40

0 60 120 180

PSN

R (d

B)

Elapsed time (s)

10

0

20

30

40

50

CMT-NEMOABC-NEMOBackground traffic

Back

grou

nd tr

affic i

njec

tion

rate

R(

)

Figure 22 PSNR comparison of how each scheme responds to theinjection of background traffic

increases however the distortion efficiency (119863) decreases as119877 increases

5 Conclusions

In this paper we have presented and empirically evaluateda comprehensive concurrent streaming framework for opti-mised efficient delivery of H264SVC and HEVC videostreams over multiple paths in mobile networks The frame-work consists of and integrates multiple key componentsthat provide a means of firstly adapting a video stream tothe prevailing network conditions in a rate-distortion opti-mised manner and then using the aggregated bandwidth ofmultiple network paths concurrently to overcome bandwidthlimitations on any single path The framework comprisesa concurrent multipath transfer scheme for NEMO-basedmobile networks which can be generalised for the concurrentmultipath delivery of different types of application flows inNEMO environments Other components include a videocodec specific packet prioritisation scheme for optimisedselective packet dropping in low aggregated bandwidthsituations a new mitigation scheme to minimise out-of-sequence delivery an ldquoon-the-flyrdquo codec specific ancestorchecking scheme suitable for real-time applications includingscalable video streaming based on H264SVC and the nextgeneration video coding standard HEVC

The proposed CMT-NEMO streaming system has beenimplemented on a realistic hardware-based multipathmobile network testbed with experimental results forH264SVC and HEVC showing a substantial improvementin the quality of the received stream compared with apreviously proposed system ABC-NEMO In the H264SVCcase an average PSNR improvement of 608 dB and 420 dBis achieved across all four video sequences in equal anddifferential path situations respectively with a maximum

65

70

75

80

85

90

95

10 20 30 40 50

Effici

ency

()

D-CMT-NEMO D-ABC-NEMOE-CMT-NEMO E-ABC-NEMOG-CMT-NEMO G-ABC-NEMO

Background traffic injection rate R ()

Figure 23 A comparison of gross transmission efficiency effectivetransmission efficiency and distortion efficiency for each scheme

improvement of up to over 8 dB observed in some tests Theframework has been experimentally shown to improve theviability of multipath video streaming in mobile networksby delivering up to 24 more video packets over the samenetwork while reducing the average jitter by 237ms andthe maximum jitter by 151ms A pilot implementation ofthe framework for HEVC further demonstrates that it canbe readily adapted to the needs of emerging video encodingschemes and boasts clear-cut performance gains in contrastto the alternative ABC-NEMO scheme too

While these results show a remarkable performanceimprovement achieving themaximum userrsquos quality of expe-rience in the demanding multihomed mobile networkingenvironments is an area that would benefit from furtherresearchThe gross transmission efficiency of our frameworkat approaching 95 is almost 20 higher than that ofABC-NEMO but further work is still desired to providmore effective bandwidth aggregation and out-of-sequencedelivery mitigation to enable optimal transmission efficiencySimilarly while distortion efficiency is also over 90 furtherwork on improved packet weighting schemes for HEVC willraise this threshold The reduced efficiency observed whenusing HEVC compared with using H264SVC indicatesthat further research on not only packet weighting andselective dropping but also error concealment schemes isrequired Finally all of the work performed in this evaluationuses objective measurement of video quality future workwould consider quality of experience using subjective andorpseudosubjective evaluation techniques

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 19: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

International Journal of Digital Multimedia Broadcasting 19

Conflict of Interests

All authors declare that there is no potential conflict of inter-ests including financial interests relationships or affiliationsrelevant to the subject of this paper

Acknowledgments

This work was funded by the UK Engineering and Phys-ical Sciences Research Council (EPSRC) under Grant noEPJ0147291 Enabler for Next-Generation Mobile VideoApplications The authors wish to thank Dr Sergio Gomaof Qualcomm Inc who provided guidance and oversight onthis project

References

[1] H Schwarz D Marpe and T Wiegand ldquoOverview of thescalable video coding extension of the H264AVC standardrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1103ndash1120 2007

[2] T Wiegand G J Sullivan G Bjoslashntegaard and A LuthraldquoOverview of the H264AVC video coding standardrdquo IEEETransactions on Circuits and Systems for Video Technology vol13 no 7 pp 560ndash576 2003

[3] T Schierl T Stockhammer and T Wiegand ldquoMobile videotransmission using scalable video codingrdquo IEEE Transactionson Circuits and Systems for Video Technology vol 17 no 9 pp1204ndash1217 2007

[4] M Wien R Cazoulat A Graffunder A Hutter and P AmonldquoReal-time system for adaptive video streaming based on SVCrdquoIEEE Transactions on Circuits and Systems for Video Technologyvol 17 no 9 pp 1227ndash1237 2007

[5] International Telecommunications Union (ITU) ldquoHigh effi-ciency video codingrdquo ITU-T Recommendation H 265 httpwwwituintrecT-REC-H265-201304-I

[6] J Chen J Boyce Y Ye and M Hannuksela ldquoSHVC workingdraft 2rdquo JCT-VC Document JCTVC-M1008 April 2013

[7] K Chebrolu and R R Rao ldquoBandwidth aggregation for real-time applications in heterogeneous wireless networksrdquo IEEETransactions on Mobile Computing vol 5 no 4 pp 388ndash4032006

[8] K Chebrolu and R R Rao ldquoSelective frame discard for interac-tive videordquo in Proceedings of the IEEE International Conferenceon Communications (ICC rsquo04) pp 4097ndash4102 June 2004

[9] J C Fernandez T Taleb M Guizani and N Kato ldquoBand-width aggregation-aware dynamic QoS negotiation for real-time video streaming in next-generation wireless networksrdquoIEEE Transactions on Multimedia vol 11 no 6 pp 1082ndash10932009

[10] D Jurca and P Frossard ldquoVideo packet selection and schedulingformultipath streamingrdquo IEEETransactions onMultimedia vol9 no 3 pp 629ndash641 2007

[11] M-F Tsai N Chilamkurti J H Park and C-K Shieh ldquoMulti-path transmission control scheme combining bandwidth aggre-gation and packet scheduling for real-time streaming in multi-path environmentrdquo IET Communications vol 4 no 6 pp 937ndash945 2010

[12] J Nightingale Q Wang and C Grecos ldquoOptimised transmis-sion of H264 scalable video streams over multiple paths in

mobile networksrdquo IEEE Transactions on Consumer Electronicsvol 56 no 4 pp 2161ndash2169 2010

[13] J Nightingale Q Wang and C Grecos ldquoRemoving path-switching cost in video delivery over multiple paths in mobilenetworksrdquo IEEE Transactions on Consumer Electronics vol 58no 1 pp 38ndash46 2012

[14] V Devarapalli R Wakikawa A Petrescu and P ThubertldquoNetworkmobility (NEMO) basic support protocolrdquo RFC 39632005

[15] Q Wang T Hof F Filali et al ldquoQoS-aware network-supportedarchitecture to distribute application flows over multiple net-work interfaces for B3G usersrdquo Wireless Personal Communica-tions vol 48 no 1 pp 113ndash140 2009

[16] R Stewart ldquoStream control transmission protocolrdquo RFC 49602007

[17] J R Iyengar P D Amer and R Stewart ldquoConcurrent multipathtransfer using SCTP multihoming over independent end-to-end pathsrdquo IEEEACM Transactions on Networking vol 14 no5 pp 951ndash964 2006

[18] J R Iyengar P D Amer and R Stewart ldquoPerformance impli-cations of a bounded receive buffer in concurrent multipathtransferrdquoComputer Communications vol 30 no 4 pp 818ndash8292007

[19] J Liao J Wang T Li X Zhu and P Zhang ldquoSender-based multipath out-of-order scheduling for high-definitionvideophone in multi-homed devicesrdquo IEEE Transactions onConsumer Electronics vol 56 no 3 pp 1466ndash1472 2010

[20] C Perkins ldquoIP mobility support for IPv4rdquo RFC 3344 2002[21] D Johnson C Perkins and J Arko ldquoMobility support for IPv6rdquo

RFC 3775 2004[22] ldquoHEVC Reference Software Test Model (HM)rdquo httpshevchhi

fraunhoferdesvnsvn HEVCSoftware[23] J Nightingale Q Wang and C Grecos ldquoOPSSA a media-

aware scheduling algorithm for scalable video streaming oversimultaneous paths in NEMO-based Mobile Networksrdquo inProceedings of the IEEE 73rd Vehicular Technology Conference(VTC rsquo11) May 2011

[24] Y Yuan Z Zhang J Li et al ldquoExtension of SCTP for concurrentmulti-path transfer with parallel subflowsrdquo in Proceedings ofthe IEEE Wireless Communications and Networking Conference(WCNC rsquo10) pp 1ndash6 Sydny Australia April 2010

[25] B Wang W Feng S-D Zhang and H-K Zhang ldquoConcurrentmultipath transfer protocol used in ad hoc networksrdquo IETCommunications vol 4 no 7 pp 884ndash893 2010

[26] C-M Huang M-S Lin and L-H Chang ldquoThe design ofmobile concurrent multipath transfer in multihomed wirelessmobile networksrdquo Computer Journal vol 53 no 10 pp 1704ndash1718 2010

[27] ldquoStandard andmetropolitan area networks media independenthandover servicesrdquo IEEE Standard 802 21 2006

[28] T Dreibholz E P Rathgeb R Seggelmann and M TuxenldquoTransmission scheduling optimizations for concurrent multi-path transferrdquo in Proceedings of the 8th International Workshopon Protocols for Future Large-Scale and Diverse Network Trans-ports (PFLDNeT rsquo10) pp 2074ndash5168 Pa USA November 2010

[29] P Natarajan N Ekiz P D Amer and R Stewart ldquoConcurrentmultipath transfer during path failurerdquo Computer Communica-tions vol 32 no 15 pp 1577ndash1587 2009

[30] M-F Tsai N K Chilamkurti S Zeadally and A VinelldquoConcurrent multipath transmission combining forward errorcorrection and path interleaving for video streamingrdquo Com-puter Communications vol 34 no 9 pp 1125ndash1136 2011

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011

Page 20: Research Article Performance Evaluation of Concurrent ... · mechanisms for quality-aware packet scheduling and scalable streaming. e remaining obstacles to deployment of concurrent

20 International Journal of Digital Multimedia Broadcasting

[31] J Liao JWang T Li and X Zhu ldquoIntroducingmultipath selec-tion for concurrent multipath transfer in the future internetrdquoComputer Networks vol 55 no 4 pp 1024ndash1035 2011

[32] I AmonouNCammas S Kervadec and S Pateux ldquoOptimizedrate-distortion extraction with quality layers in the scalableextension of H264AVCrdquo IEEE Transactions on Circuits andSystems for Video Technology vol 17 no 9 pp 1186ndash1193 2007

[33] J Reichel H Schwarz and M Wien ldquoJoint Scalable VideoModel 11 (JSVM 11)rdquo Tech Rep no JVT-X202 Joint VideoTeam July 2007

[34] S Wenger Y-K Wang T Schierl and A Eleftheriadis ldquoRTPpayload format for scalable video codingrdquo RFC 6190 2011

[35] D Kaspar K Evensen A F Hansen P Engelstady P Halvorsenand C Griwodz ldquoAn analysis of the heterogeneity and IP packetreordering over multiple wireless networksrdquo in Proceedings ofthe IEEE Symposium on Computers and Communications (ISCCrsquo09) pp 637ndash642 July 2009

[36] D T Nguyen and J Ostermann ldquoCongestion control forscalable video streaming using the scalability extension ofH264AVCrdquo IEEE Journal on Selected Topics in Signal Process-ing vol 1 no 2 pp 246ndash253 2007

[37] Wireshark httpwwwwiresharkorgdocs[38] J Nightingale Q Wang and C Grecos ldquoPerformance eval-

uation of scalable video streaming in multihomed mobilenetworksrdquoPerformance EvaluationReview vol 39 no 2 pp 29ndash31 2011

[39] H Schulzrinne S Casner R Frederick and V Jacobsen ldquoRTPa transport protocol for real-time applicationsrdquo RFC 1889 1996

[40] J Asghar F Le Faucheur and I Hood ldquoPreserving video qualityin IPTV networksrdquo IEEE Transactions on Broadcasting 2009

[41] J Zhang and N Ansari ldquoOn assuring end-to-end QoE in nextgeneration networks challenges and a possible solutionrdquo IEEECommunications Magazine vol 49 no 7 pp 185ndash191 2011