IEE NETWORKING NET_2013_Sep

THE MAGAZINE OF GLOBAL INTERNETWORKING

®September/October 2013, Vol. 27, No. 5

®

www.comsoc.org

Cloud-Assisted MobileComputing and

Pervasive Services

A Publication of the IEEE Communications Societyin cooperation with theIEEE Computer Society and theInternet Society®

NETWORK_COVER-September2013_Layout 1 9/20/13 1:07 PM

http://www.comsoc.org

Free Access to papers on 100G from recent IEEE Communications technical conferences, magazines, and journals. Partial 100G list includes:

- Coherent 100G PM-QPSK Field Trial

- Beyond 100G Ethernet

- 100G Ethernet and OTU4 Testing Challenges

- Beyond 100G Client Optics

- Ultra-High-Capacity DWDM Transmission System

for 100G and Beyond

- Optical Transceivers for 100G Gigabit Ethernet

and Transport

- 100G Optical Transport Network and Beyond Employing

Digital Signal Processing

- Optical Transport Network Evolution with 100G Ethernet

- 100G Optical Access Based on Optical OFDM

- 40G and 100G Ethernet Short-Reach Optical

and Copper Host Board Channel Design

For this and other sponsor opportunities, please contact Eric Levine,

Associate Publisher. Phone: 212-705-8920, E-mail: [email protected]

www.comsoc.org/techfocus

FREE ACCESS COMPLIMENTS OF

TEC

HN

OLO

GY

FO

CU

Spre

sente

d b

y IE

EE

Co

mm

unic

atio

ns

Soci

ety

LIMITED TIME OFFER!

IEEE COMSOC

FREE ONLINE ARTICLES FROM IEEE COMMUNICATIONS SOCIETY

Register Now for Free Access!

100G100G

IENYCM3296.indd 1IENYCM3296.indd 1 8/26/13 9:18 PM8/26/13 9:18 PM

http://www.comsoc.org/techfocus

mailto:[email protected]

IEEE Network • September/October 2013 1


®

SEPTEMBER/OCTOBER 2013, VOL. 27, NO. 5

IEEE NETWORK ISSN 0890-8044 is published bimonthly by the Institute of Electrical and Electronics Engineers, Inc. Headquarters address: IEEE, 3 Park Avenue, 17th Floor, New York, NY 10016-5997, USA; tel: +1-212-705-8900; e-mail: [email protected]. Responsibility for the contents rests upon authors of signed articles and not the IEEE or its members. Unless otherwise specified,the IEEE neither endorses nor sanctions any positions or actions espoused in IEEE Network.

ANNUAL SUBSCRIPTION: $40 in addition to IEEE Communications Society or any other IEEE Society member dues. Non-member prices: $250. Single copy price $50.EDITORIAL CORRESPONDENCE: Address to: Xuemin (Sherman) Shen, Editor-in-Chief, IEEE Network, IEEE Communications Society, 3 Park Avenue, 17th Floor, New York, NY 10016-5997, USA;e-mail: [email protected]

COPYRIGHT AND REPRINT PERMISSIONS: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limits of U.S. Copyright law for private use of patrons:those articles that carry a code on the bottom of the first page provided the per copy fee indicated in the code is paid through the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA01923, USA. For other copying, reprint, or republication permission, write to Director, Publishing Services, at IEEE Headquarters. All rights reserved. Copyright © 2013 by the Institute of Electricaland Electronics Engineers, Inc.

POSTMASTER: Send address changes to IEEE Network, IEEE, 445 Hoes Lane, Piscataway, NJ 08855-1331, USA. Printed in USA. Periodical-class postage paid at New York, NY and at additionalmailing offices. Bulk rate postage paid at Easton, PA permit #7. Canadian GST Reg# 40030962. Return undeliverable Canadian addresses to: Frontier, P.O. Box 1051, 1031 Helena Street, Fort Eire,ON L2A 6C7.

SUBSCRIPTIONS: Orders, address changes should be sent to IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08855-1331, USA. Tel. +1-732-981-0060.

ADVERTISING: Advertising is accepted at the discretion of the publisher. Address correspondence to IEEE Network, 3 Park Avenue, 17th Floor, New York, NY 10016-5997, USA.

6

12

4

20

28

34

Guest EditorialVictor C. M. Leung, Min Chen, Mohsen Guizani, and Branka Vucetic

Mobile Cloud Computing Service Models: AUser-Centric ApproachThe authors provide a comprehensive study to lay outexisting mobile cloud computing service models and keyachievements, and present a new user-centric mobilecloud computing service model.Dijiang Huang, Tianyi Xing, and Huijun Wu

Follow Me Cloud: Interworking FederatedClouds and Distributed Mobile NetworksThe authors introduce the Follow-Me Cloud concept andproposes its framework. The proposed framework isaimed at smooth migration of all or only a required por-tion of an ongoing IP service between a data center anduser equipment of a 3GPP mobile network to anotheroptimal DC with no service disruption.Tarik Taleb and Adlen Ksentini

A Distributed Cloud Architecture for MobileMultimedia ServicesThe authors describe design requirements and an archi-tecture for Mobile Cloud Computing. The novelty in thisarchitecture is an integrated cloudlet and base stationsubsystem.Muhamad Felemban, Saleh Basalamah, and Arif Ghafoor

When Mobile Terminals Meet the Cloud:Computation Offloading as the BridgeThe authors identify the key issues in developing newapplications that effectively leverage cloud resources forcomputation-intensive modules, or migrating such mod-ules in existing applications to the mobile cloud.Xiaoqiang Ma, Yuan Zhao, Lei Zhang, Haiyang Wang, and Limei Peng

Toward a Unified Elastic Computing Platformfor Smartphones with Cloud SupportThe authors propose a unified elastic computing platformthat supports application offloading for mobile devices,reducing energy consumption on smartphones. Weiwen Zhang, Yonggang Wen, Jun Wu, and Hui Li

CitySee: Not Only a Wireless Sensor NetworkCitySee, an environment monitoring system with 1196sensor nodes and 4 mesh nodes in an urban area, is main-ly motivated by the needs of precise carbon emissionmeasurement and real-time surveillance for CO2 manage-ment in cities. The authors share some early lessonslearned from this project. Yunhao Liu, Xufei Mao, Yuan He, Kebin Liu, Wei Gong, and Jiliang Wang

Toward Cloud-Based Vehicular Networkswith Efficient Resource ManagementThe authors propose to integrate cloud computing intovehicular networks such that the vehicles can share com-putation resources, storage resources, and bandwidthresources. Rong Yu, Yan Zhang, Stein Gjessing, Wenlong Xia, and Kun Yang

Cloud-Enabled Wireless Body Area Networksfor Pervasive HealthcareWith the support of mobile cloud computing, wirelessbody area networks can be significantly enhanced formassive deployment of pervasive healthcare applications.However, several technical issues and challenges are asso-ciated with the integration of WBANs and MCC. Theauthors study a cloud-enabled WBAN architecture and itsapplications in pervasive healthcare systems. Jiafu Wan, Caifeng Zou, Sana Ullah, Chin-Feng Lai, MingZhou, and Xiaofei Wang

An Auto-Scaling Mechanism for VirtualResources to Support Mobile, Pervasive, Real-Time Healthcare Applications in CloudComputingThe authors propose a server-side auto-scaling mechanismto autonomously allocate virtual resources on an on-demand basis. The mechanism is tested in an AmazonEC2, and the results show how the proposed mechanismcan efficiently scale up and down the virtual resources,depending on the volume of requested real-time tasks. Yong Woon Ahn, Albert M. K. Cheng, Jinsuk Baek, Minho Jo, and Hsiao-Hwa Chen

42

48

56

62

Editor’s Note 2

Special IssueCloud-Assisted Mobile Computing and Pervasive Services

NETWORK_TOC-September_LYT/TOC/SEP 9/20/13 1:09 PM Page 1



Director of MagazinesSteve Gorshe, PMC-Sierra, Inc, USA

Editor-in-ChiefSherman Shen, University of Waterloo, Canada

Senior Technical EditorsTom Chen, Swansea University, UK

Peter O’Reilly, Northeastern Univ., USA

Technical EditorsJiannong Cao, Poly. Univ., HK

Jiming Chen, Zhejiang Univ., ChinaHan-Chieh Chao, National Ilan University, Taiwan

Michael Fang, Univ. of Florida, USAErol Gelenbe, Imperial College London, UK

Roch Glitho, Concordia Univ. CanadaMinho Jo, Korea Univ., Korea

Admela Jukan, Technische Univ. Carolo-Wilhelminazu Braunschweig, Germany

Nei Kato, Tohoku Univ., JapanXiaodong Lin, OUIT, Canada

Ying-Dar Lin, National Chiao Tung Univ., TaiwanIoanis Nikolaidis, Univ. of Alberta, CanadaRomano Fantacci, Univ. of Florence, Italy

Sudipta Sengupta, Microsoft Research, USANess Shroff, OSU, USA

Ivan Stojmenovic, Univ. Ottawa, CanadaJoe Touch,USC/ISI, USA

Anwar Walid, Bell Labs Research,Alcatel-Lucent, USA

Guoliang Xue, Arizona State Univ., USAMurtaza Zafer, IBM T. J. Watson Research

Center, USA

Feature Editors"New Books and Multimedia"

Yu Cheng, IIT, USA

IEEE Production StaffJoseph Milizzo, Assistant Publisher

Eric Levine, Associate PublisherSusan Lange, Online Production Manager

Jennifer Porcello, Production SpecialistCatherine Kemelmacher, Associate Editor

2013 IEEE Communications Society OfficersVijay K. Bhargava, President

Sergio Benedetto, Past President-ElectLeonard Cimini, VP–Technical Activities

Abbas Jamalipour VP–ConferencesNelson Fonseca, VP–Member Relations

Vincent Chan, VP–PublicationsAlex Gelman, VP-Standards Activities

Stan Moyer, TreasurerJohn M. Howell, Secretary

Board of GovernorsThe officers above plus Members-at-Large:

Class of 2013Gerhard Fettweis, Stefano Galli

Robert Shapiro, Moe WinClass of 2014

Merrily Hartman, Angel LozanoJohn S. Thompson, Chengshan Xiao

Class of 2015Nirwan Ansari, Stefano Bregni

Hans-Martin Foisel, David G. Michelson

2013 IEEE OfficersPeter W. Staecker, President

J. Roberto B. de Marca, President-ElectMarko Delimar, Secretary

John T. Barr, TreasurerGordon W. Day, Past-President

E. James Prendergast, Executive DirectorDoug Zuckerman, Director, Division III

IEEE Network • September/October 20132

ocial networks extending the social circles of users have become animportant integral part of our daily lives. With social networking tools,we are able to easily share information, images, and videos with our

friends, and search for desirable service information with recommendations. Asreported by ComScore, social networking sites such as Facebook and Twitterhave reached 82 percent of the world’s online population, representing 1.2 bil-lion users around the world. In the meantime, fueled by the dramatic advance-ments of smartphones and the ubiquitous connections of Internet, socialnetworking is further becoming available to mobile users and keeps them postedon up-to-date worldwide news and messages from their friends and families any-time, anywhere. The eMarketer estimates that up to 46 percent of mobile userswill access their social networks with smartphones in 2014, while this numberwas merely 16 percent in 2010. It is envisioned that, with the growing number ofsmartphone users, a pervasive and omnipotent communication platform, themobile social network (MSN), will become mainstream where smartphone usershave extensive methods, from browsing over the Internet to querying nearbypeers to obtain desired information.

The boom of mobile social network fosters a large volume of promising andsmart mobile applications. Apple Inc. has greatly increased the number ofmobile applications from 800 in July 2008 to over 825,000 in April 2013. Nowa-days, as many smartphone users indulge themselves in enjoying various mobilesocial applications, they can no longer live or work effectively without using theapplications. Despite the tremendous benefits brought by the MSN and applica-tions, the MSN still faces many security and privacy challenges. Since applica-tions normally require the access of users’ personal information to serve usersbetter, security and privacy preservation have not been paid much attention inmany application designs. For example, in most social applications, users need toregister with personal profiles, such as name, birthday, home address, and phonenumber, which are very likely to be disclosed due to the lack of protection. Inthe United Kingdom, the number of complaints and alleged crimes associatedwith Facebook and Twitter has increased by 780 percent in the last four years,resulting in about 650 people being charged in 2012. Besides, Internet Safetystates that 29 percent of Internet related sex crimes in 2012 originated from thesocial networking sites. As the mobile applications enable smartphone users tointeract with social networks pervasively, there will be more severe security andprivacy concerns for users. In the following, three unique security and privacyissues in mobile applications are discussed: information leakage, location priva-cy, and trust relation.

Information Leakage in Autonomous Mobile ApplicationsAutonomous mobile applications enable smartphone users to query neighboringusers and local service providers for the desired information through short-rangewireless communications such as Bluetooth and near field communication(NFC). Autonomous mobile applications are easy to set up and have much prac-

SS


®

Security and Privacy inMobile SocialNetwork

EDITOR’S NOTE

®

Xuemin (Sherman) Shen

NETWORK_EDIT-NOTE-Shen_Edit Note 9/20/13 12:40 PM Page 2

3IEEE Network • September/October 2013

tical value in our daily lives. A smartphone user can launcha local information search to consult other users nearby,who in turn will ask their friends, and so on, until the infor-mation is found. A smartphone user may search for a goodrestaurant by revealing her personal preferences to nearbyusers. If a smartphone user is looking for a carpool service,she may reveal her destination to local users who may thenprovide services to her if their destinations are in the vicini-ty. Also, patients can launch an application for healthcarepurposes (e.g., to find others with similar symptoms orexperiences). In all these applications, users are required toreveal their personal information (i.e., restaurant prefer-ences, travel destinations, and symptoms) to others. Suchinformation is highly privacy-sensitive, and maliciousattackers may track a target user’s behavior if they obtainthe information. The current effective solution is to requirea trustworthy mediator over the Internet to help the infor-mation requester receive the desired and accurate informa-tion from an information responder. However, autonomousmobile applications do not have such an Internet mediator,and it is very difficult to find a third user who has a wellestablished trust relationship with both the informationrequester and the information responder in physical prox-imity. As such, lightweight and energy-efficient authentica-tion schemes and secret handshake protocols need to beintegrated into the applications. Smartphone users shouldinteract only with other authenticated users in autonomousmobile applications to prevent information leakage.

Location Privacy in Location-Based ApplicationsIn addition to voice service available for any cellular tele-phone, smartphones distinguish themselves with powerfulcomputing resources and, most significantly, their capabili-ty to understand their surrounding environments throughmany built-in sensors. As a result, location-based applica-tions have become very popular. In such applications,selected information is downloaded from the Internet toassist location-based activities. Such applications are widelysupported by either social network giants such as Face-book, or specialized service providers such as Foursquareand Loopt. The main idea is as follows. The GPS chip in asmartphone detects its location coordinates, which are thenreported to Internet service providers for downloadinginformation related to local services. However, such a pro-cess raises a serious privacy issue: the continuously dis-closed location coordinates reveal where, when, or evenwhat the smartphone user has been doing. If the locationinformation of a user is revealed to malicious attackers, theattackers will know when the user is not at home and canbreak into the user’s house to commit criminal activities.To prevent any abuse of location information, smartphoneusers have to often manually switch on and off the localiza-tion function to self-control the access of their locationinformation. Another solution is to use cryptographicpseudonyms for mobile users such that the user’s behaviorsand locations cannot be easily linked. Another solution isto blur the location with an area of the vicinity or mix theiridentities with nearby other users. However, the former is

energy-consuming, while the latter may degrade the appli-cation performance in terms of accuracy of services. There-fore, dealing with location privacy is very critical inlocation-based applications.

Trust Relation in Mobile ApplicationsThe trust relation is fundamental to mobile applications,and affects user experiences of the applications. Mobileapplications can only be adopted by smartphone users ifthey have trust in the Internet service providers, local ser-vice providers, and other smartphone users. While smart-phone users enjoy conveniences brought by mobileapplications maintained by Internet service providers, theyrealize that more and more personal information isrevealed and start questioning how the service providerskeep the collected personal information (e.g., whether ornot the service providers will disclose the information forother purposes without proper consent). Doubts abouttrustworthiness will bother users in launching mobile appli-cations. Furthermore, the trust relation of customerstoward service providers is influenced by many social fac-tors. The trust relation of new customers toward a serviceprovider is tightly related to reviews from previous cus-tomers. Some mobile applications enable customers toquickly exchange their reviews. Strong recommendationsfrom close friends can effectively strengthen the trust rela-tion. However, in reality, the reviews can be forged, andcustomers do not want to reveal their identities in the rec-ommendation process. So far, how to build the trust rela-tion among smartphone users in mobile applicationsremains a challenging issue.

A trust relation also exists among users based on theircommon social communities. Users in a common commu-nity often have a certain trust level with each other, as theyeither share some common interests or recognize eachother to some extent. With the initial trust relation, smart-phone users can carry out local social activities via mobileapplications, and further strengthen the trust relation. Insuch a trust establishing process, mobile applications arevulnerable to notorious sybil attacks where an attackermanipulates bogus identities or abuses pseudonyms tocompromise the effectiveness of systems. Especially in theMSN, as smartphone users often adopt multiplepseudonyms to protect their location privacy, it is very chal-lenging to restrict sybil attackers who legally have multiplepseudonyms but maliciously use them. In addition, sybilattacks can be extended to the mobile domain and belaunched by mobile users anytime, anywhere. It is a com-plex task to promptly detect such attacks, due to difficultyin monitoring and characterizing their behaviors in amobile environment.

In conclusion, although the MSN brings tremendousbenefits to our daily lives, it introduces serious and emerg-ing security and privacy issues that permeate the cyber andphysical space around us. In the presence of many new andunique research challenges, long-term efforts in multi-disci-plinary research are necessary. I hope that you enjoy read-ing this Editor’s Note, and find it interesting and helpful.

EDITOR’S NOTE

NETWORK_EDIT-NOTE-Shen_Edit Note 9/20/13 12:40 PM Page 3


n order to provide rich mobile pervasive services toend users, advancing mobile communication technolo-gies and deployment of smart devices have become

key issues in both industry and academia. However, thelimited onboard computing, energy supply, and storagecapabilities of mobile devices are hampering their ability tosupport the increasingly sophisticated applications demand-ed by users. Recently, mobile cloud computing (MCC)technologies for mobile devices have emerged to overcomethese limitations. These technologies can minimize therequirements of computing power and storage to deploypervasive applications in mobile devices. Developments ofinnovative and pervasive mobile services (mobile videostreaming, rich media dissemination, surveillance, e-gam-ing, healthcare, etc.) can be greatly facilitated by cloudcomputing platforms employing advanced technologies.

In response to the call for contributions, we received 40submissions. After two rounds of careful reviews, nine out-standing papers have been collected for this Special Issue,which are classified into three categories: • New mobile cloud computing architectures• Computation offloading for mobile cloud computing• Cloud-assisted pervasive services and applications

The volume opens with a comprehensive survey article,“Mobile Cloud Computing Service Models: A User-Cen-tric Approach,” by Huang et al., which reviews diverseways of combining cloud computing and mobile platformstoward a new computing/communications paradigm so thatreaders will be able to have a holistic view on currentdevelopments and the vision of user-centric MCC. Thearticle provides a complete taxonomy of mobile cloudcomputing service model and a new genre of real-worldcommercial applications. The authors envision the seam-less integration of heterogeneous cloud platforms andmobile devices into a human-centric computing ecosystem.In the second article, “Follow Me Cloud: InterworkingFederated Clouds & Distributed Mobile Networks,” Taleband Ksentini present the Follow Me Cloud (FMC) concept

and propose its framework, aimed at smooth migration ofall or only a required portion of an ongoing IP servicebetween a data center (DC) and a user equipment (UE)device of a Third Generation Partnership Project (3GPP)mobile network to another optimal DC with no service dis-ruption. In the third article, “A Distributed Cloud Archi-tecture for Mobile Multimedia Services,” Felemban et al.present a distributed cloud architecture for mobile multi-media users. The key idea of this architecture is the inte-gration of base stations and cloudlets to guarantee qualityof service (QoS)-based services. The integration entailsresource management protocols which addresses variousQoS requirements such as aggregated radio channels andbuffer allocations.

Regarding the efficiency of MCC, the computationoffloading to the cloud effectively expands the usability ofmobile terminals beyond their physical limits, and alsogreatly extends their battery charging intervals throughpotential energy savings. In the fourth article, “WhenMobile Terminals Meet Cloud: Computation Offloadingas the Bridge,” Ma et al. present an overview of computa-tion offloading in MCC, and identify the key issues indeveloping new applications that effectively leveragecloud resources for computation-intensive modules, or inmigrating such modules in existing applications to themobile cloud. They analyze two representative applica-tions in detail from both macro and micro perspectives:cloud-assisted distributed interactive mobile applicationsand cloud-assisted motion estimation for mobile videocompression, to illustrate the unique challenges, benefit,and implementation of computation offloading in MCC.In the fifth article, “Toward a Unified Elastic ComputingPlatform for Smartphones with Cloud Support,” Zhangand Wen present a unified elastic computing platformthat supports application offloading for smartphones inorder to reduce energy consumption on smartphones,with an infrastructure-based cloud and an ad hoc virtualcloud formed by a cluster of smartphones. For this plat-

GUEST EDITORIAL

II

Cloud-Assisted Mobile Computing andPervasive Services

Victor C. M. Leung Min Chen Mohsen Guizani Branka Vucetic

NETWORK_GUEST_EDITORIAL-Chen_Layout 1 9/20/13 12:42 PM Page 4


GUEST EDITORIAL

form, the authors present both an offloading policy andan offloading mechanism under which applications aredelegated to the cloud for execution.

The remaining articles focus on cloud-assisted perva-sive services and applications. In the sixth article, “City-See: Not Only A Wireless Sensor Network,” Liu et al.share their early lessons learned from CitySee, the largestenvironment monitoring system (consisting of 1196 sensornodes and 4 mesh nodes) integrating both the underlyingwireless sensor network (WSN) techniques and the upper-layer cloud computing applications, such as a sensing as aservice (SaaS) cloudlet. While going through the chal-lenges (e.g., hardware, software, and protocols) of City-See, the authors concentrate on how to combine andevolve WSNs with cloud computing in order to providesatisfactory pervasive services to both network designersand users with respect to scalability, performance, privacy,cost savings, and so on, toward an innovative, pervasive,and easy-to-use cloud service platform. In the seventharticle entitled “Toward Cloud-Based Vehicular Networkswith Efficient Resource Management,” Yu et al. proposeintegrating mobile cloud computing technology into vehic-ular networks, such that vehicles and roadside units canshare computation resources, storage resources, andbandwidth resources. A hierarchical cloud architecture isproposed with a design of three-layered structure, includ-ing a vehicular cloud, a roadside cloud, and a centralcloud. A game-theoretic approach is proposed to effec-tively model and optimize the resource allocation schemeamong virtual machines (VMs) in cloud cites. Moreover,to support VM migration triggered by vehicle mobility, anoptimal resource reservation scheme is designed to con-serve virtual resources for migrated VMs and guaranteethe continuity of mobile vehicular services. Simulationsare conducted to verify the efficiency of the proposedcloud resource management strategies.

With the support of MCC, wireless body area networks(WBANs) can be significantly enhanced for massivedeployment of pervasive healthcare applications. Howev-er, several key issues and technical challenges are associ-ated with the integration of WBANs and MCC. In theeighth article, “Cloud-Enabled Wireless Body Area Net-works for Pervasive Healthcare,” Wan et al. study a cloud-enabled WBAN architecture and its applications inpervasive healthcare systems. They highlight the method-ologies for transmitting vital sign data to the cloud byusing energy-efficient routing, cloud resource allocation,semantic interactions, and data security mechanisms. Amethodology to manage private cloud infrastructures fore-health applications is addressed in the ninth article, “AnAuto-Scaling Mechanism for Virtual Resources to Sup-port Mobile, Pervasive, Real-Time, Healthcare Applica-tions in Cloud Computing,” in which Ahn et al. propose anew approach to scale virtual computing resources up anddown to process real-time tasks delivered from e-healthapplications. The workload prediction mechanism is usedto scale the virtual resources up to prevent missing dead-lines of real-time tasks. These e-health applications workas a real-time monitoring system to check on patient’shealth conditions periodically, and possibly increase theirsampling rates and bandwidth requirements to process all

real-time tasks whenever these applications detect abnor-mal symptoms.

In closing, we would like to thank all the authors whosubmitted their research work to this special issue. Wewould also like to acknowledge the contribution of manyexperts in the field who participated in the review process,and provided helpful suggestions to the authors to improvethe contents and presentation of the articles. We would inparticular like to thank Professor Xuemin “Sherman”Shen, Editor-in-Chief, for his support and very helpful sug-gestions and comments during the delicate stages of con-cluding the special issue.

BiographiesVICTOR C. M. LEUNG [S’75, M’89, SM’97, F’03] ([email protected]) is aprofessor of electrical and computer engineering and holder of the TELUSMobility Research Chair at the University of British Columbia (UBC). He hascontributed some 650 technical papers, 25 book chapters, and 5 books inthe areas of wireless networks and mobile systems. He was a DistinguishedLecturer of the IEEE Communications Society. He has served on the Editorial Boardsof IEEE Transactions on Computers, IEEE Wireless Communications Letters,and several other journals, and has contributed to the Organizing and Tech-nical Program Committees of numerous conferences. He was a winner of the 2012UBC Killam Research Prize and the IEEE Vancouver Section CentennialAward. He is a Fellow of the Canadian Academy of Engineering and theEngineering Institute of Canada.

MIN CHEN [M’08, SM’09] ([email protected]) is a professor at the School ofComputer Science and Technology of Huazhong University of Science andTechnology. He was an assistant professor at the School of Computer Scienceand Engineering of Seoul National University (SNU) from September 2009 toFebruary 2012. He worked as a post-doctoral fellow in the Department ofElectrical and Computer Engineering at UBC for three years. Before joiningUBC, he was a post-doctoral fellow at SNU for one and half years. He hasmore than 180 paper publications. He received the Best Paper Award at IEEEICC 2012 and Best Paper Runner-up Award at QShine 2008. He has been aGuest Editor for IEEE Network and IEEE Wireless Communications, amongother publications. He was Symposium Co-Chair for IEEE ICC 2012 and IEEEICC 2013. He is General Co-Chair for IEEE CIT 2012 and Tridentcom 2014.He is a TPC member for IEEE INFOCOM 2014. He was Keynote Speaker forCyberC 2012 and Mobiquitous 2012.

MOHSEN GUIZANI [S’85, M’89, SM’99, F’09] ([email protected]) is current-ly a professor and associate vice president for graduate studies at Qatar Uni-versity, Doha. He was chair of the Computer Science Department of WesternMichigan University from 2002 to 2006 and chair of the Computer Science Depart-ment of the University of West Florida from 1999 to 2002. He also served inacademic positions at the University of Missouri-Kansas City, University ofColorado-Boulder, Syracuse University, and Kuwait University. He receivedhis B.S. (with distinction) and M.S. degrees in electrical engineering, andM.S. and Ph.D. degrees in computer engineering in 1984, 1986, 1987, and1990, respectively, from Syracuse University, New York. His research inter-ests include computer networks, wireless communications and mobile comput-ing, and Optical Networking. He currently serves on the editorial boards ofsix technical journals, and is the founder and Editor-in-Chief of Wireless Com-munications and Mobile Computing (Wiley). He is the author of eight booksand more than 300 publications in refereed journals and conferences. Hehas guest edited a number of special issues in IEEE journals and magazines.He has also served as committee member, Chair, and General Chair of anumber of conferences. He served as the Chair of the IEEE CommunicationsSociety Wireless Technical Committee (WTC) and of the TAOS TechnicalCommittee. He was an IEEE Computer Society Distinguished Lecturer from2003 to 2005. He is a Senior Member of ACM.

BRANKA VUCETIC [M’83, SM’00, F’03] ([email protected]) receivedher B.S.E.E., M.S.E.E., and Ph.D. degrees in 1972, 1978, and 1982,respectively, in electrical engineering, from the University of Belgrade, Yugoslavia.During her career she has held various research and academic positions inYugoslavia, Australia, and the United Kingdom. Since 1986, she has beenwith the Sydney University School of Electrical and Information Engineering,Australia. She is currently director of the Centre of Excellence in Telecommuni-cations at Sydney University. Her research interests include wireless communi-cations, digital communication theory, coding, and multi-user detection. In thepast decade she has worked on a number of industry sponsored projects inwireless communications and mobile Internet. She has taught a wide range ofundergraduate, postgraduate, and continuing education courses worldwide. Shehas co-authored four books and more than 200 papers in telecommunicationsjournals and conference proceedings.

NETWORK_GUEST_EDITORIAL-Chen_Layout 1 9/20/13 12:42 PM Page 5





oday, the Internet web service is the main way weaccess information from fixed or mobile terminals.Some of the information is stored in Internet clouds,where computing, communication, and storage are

common services provided for Internet users. In the non-distantfuture, many of our queries will be beyond the current Internetscope and will be about the people, physical environments thatsurround us, and virtual environments with which we will beinvolved. With the Internet environment improving, mobilephones are overtaking PCs as the most common web accessentities worldwide by 2013 as predicted by Gartner [1]. Currentmobile devices have many advanced features such as mobility,communication, and sensing capabilities, and can serve as thepersonal information gateway for mobile users. However, whenrunning complex data mining and storing operations, the com-putation, energy, and storage limitations of mobile devicesdemand an integrated solution relying on cloud-based computa-tion and storage support. As a result, a new research field,called mobile cloud computing (MCC), is emerging.

In MCC, a mobile entity can be considered as either aphysical mobile device or a mobile computing/storage soft-ware agent within a virtualized cloud resource provisioningsystem. In the latter view of the cloud system, a softwareagent’s main functionality is the mobility associated with soft-ware codes. In other words, mobile cloud applications maymigrate or compose software codes in the distributed MCCresource provisioning environment. Mobile cloud services willaccount for delay, energy consumption, real-time entity pres-ence, information caching capabilities, networking and com-munication connectivity, data protection and sharingrequirements, and so on. By achieving these features, we areactually able to create a new world composed of both physi-cally networked systems and virtualized entities that aremapped to the physical systems, preserving and in some casesextending their functions and capabilities.

MCC distinguishes its research focuses on tight interactionbetween, and construction and integration of, the cyber physi-cal system (CPS) and cyber virtual system (CVS), in which theCPS is immensely composed by computational and physicalsmart and mobile entities, and the CVS is mainly formed by

cloud-based virtualized resources and services. Recent devel-opments in augmented reality (AR) have demonstrated someof the application capabilities of MCC.

This article first focuses on a comprehensive study of exist-ing MCC service models, and then a user-centric MCC serviceframework is presented. The rest of this article is arranged asfollows. We summarize current mobile cloud service modelsbased on the role of mobile devices. We illustrate the currentrepresentatives according to the different service models previ-ously defined. We state the transformation from the traditionalInternet cloud to the mobile cloud and highlight features ofMCC. The future research directions of MCC are proposed,focusing on a new user-centric service model and correspond-ing application scenarios. Finally, we conclude this article.

Current Mobile Cloud Service ModelsCurrent Internet clouds have been broadly classified in threeservice models: infrastructure as a service (IaaS), platform asa service (PaaS), and software as a service (SaaS). They areclassified according to the layers of virtualization. However,due to the involvement of both CPS and CVS, MCC’s servicemodels are more appropriately classified according to theroles of computational entities within its service framework,where the classification of MCC service models can use theroles and relations between mobile entities and their invokedcloud-based resource provisioning. Based on this view, exist-ing MCC services can be classified into three major models:mobile as a service consumer (MaaSC), mobile as a serviceprovider (MaaSP), and mobile as a service broker (MaaSB).These MCC service models are illustrated in Fig. 1, in whicharrows indicate service processing flows from service providersto service recipients.

MaaSC is originated from the traditional client-servermodel by introducing virtualization, fine-grained access con-trol, and other cloud-based technologies at the initial stage.Mobile devices can outsource their computation and storagefunctions onto the cloud in order to achieve better perfor-mance and more application capabilities. In this architecture,the service is one-way from the cloud to mobile devices and

T

6 IEEE Network • September/October 2013

AbstractMobile devices are rapidly becoming the major service participants nowadays.

However, traditional client-server based mobile service models are not able to meetthe increasing demands from mobile users in terms of services diversity, user expe-rience, security and privacy, and so on. Cloud computing enables mobile devicesto offload complex operations of mobile applications, which are infeasible onmobile devices alone. In this article, we provide a comprehensive study to lay outexisting mobile cloud computing service models and key achievements, and pre-sent a new user-centric mobile cloud computing service model to advance existingmobile cloud computing research.

Mobile Cloud Computing Service Models:A User-Centric Approach

Dijiang Huang, Tianyi Xing, and Huijun Wu, Arizona State University

T

0890-8044/13/$25.00 © 2013 IEEE

XING_LAYOUT_Layout 1 9/20/13 12:43 PM Page 6

mobile devices are service consumers. Most existingMCC services fall into this category.

MaaSP is different from MaaSC in that the role of amobile device is shifted from a service consumer to aservice provider. For example, with onboard sensors(GPS module, camera, gyroscope, etc.), mobile devicesare able to sense data from the devices and their neigh-boring environment, and further provide sensing ser-vices to other mobile devices through the cloud. In Fig.1, consumers receive services provided by both thecloud and mobile devices. The types of services provid-ed by mobile devices are diverse based on their sensingand processing capabilities.

MaaSB can be considered as an extension of MaaSP,where MaaSB provides networking and data forwardingservices for other mobile devices or sensing nodes.MaaSB is desired under some circumstances becausemobile devices usually have limited sensing capabilitycompared to sensors that are dedicated for speciallydesigned functionalities and sensing locations. For exam-ple, mobile phones can be used to collect users’ physicalactivities from Nike Fuelband [2]. MaaSB extends thecloud edges to mobile devices and wireless sensors. Thus,a mobile device can be configured as a gateway or proxyproviding networking services through various communi-cation approaches such as 3/4G, Bluetooth, and WiFi.Moreover, the proxy mobile device can also provide secu-rity and privacy protections to their interfaced sensors.

Existing Mobile Cloud ApplicationsWe summarize existing MCC services and applications inTable 1. We discuss four major MCC service types andcorresponding representatives. Each service or applica-tion can be categorized into one or multiple service mod-els. MaaSC is the most common MCC service modelbecause most existing mobile devices are still restrictedby their computation and energy capacities. As an exam-ple, clonecloud [3] provides computation task offloadingservice for mobile devices. In this case, the mobile device is theservice consumer since it only benefits from the service provid-ed by the cloud rather than providing services for other users.

Mobile Cloud ComputationComputation task offloading is a demanding feature formobile devices relying on Internet clouds to perform resource-intensive computation tasks. Partitioning computation tasksand allocating them between mobile devices and clouds canbe very inefficient during the application runtime consideringvarious performance metrics such as energy consumption,CPU usage, and network delay. How to efficiently and intelli-gently offload the computation tasks onto the cloud is one ofthe main research issues of MCC. CloneCloud [3] and MAUI[4] are two pioneer projects in this area. Both can automati-cally offload computing tasks to the cloud.

CloneCloud serves as an application partitioner as well asan execution runtime environment that allows unmodifiedmobile applications to seamlessly offload parts of the execu-tions from mobile devices onto a cloud server. The offloadingdecision is made by optimizing execution time and energyusage for mobile devices. In contrast to CloneCloud, MAUIallows modifying offloading applications at the coding level tomaximize the energy saving of mobile devices. Thinkair [5]demands dedicated virtual machines (VMs) in clouds as partof a complete smartphone system, and removes the restric-tions on applications/inputs/environmental conditions by usingonline method-level offloading.

Mobile Cloud StorageStorage capacity is another constraint of mobile devices.There are many existing storage services for mobile devices,such as Dropbox, Box, iCloud, Google Drive, and Skydrive[6]. Besides manually uploading the files or data onto thecloud, one desired feature of mobile cloud storage services isthe automatic synchronization between mobile devices andthe cloud. Multimedia data generated by mobile devicesdemands a stable and highly available storage solution. This isthe reason why many smartphone operating systems nativelyimplant the multimedia data synchronization feature (iCloudfor iOS, Skydrive for Windows Phone, Google Drive forAndroid, etc.). Moreover, mobile users’ behavior data, such aslocation traces, browsing history, personal contacts, and pref-erence settings, need to be kept in a reliable and protectedstorage space. Most existing commercial cloud storage solu-tions are built on a centralized data center, which is appropri-ate for Internet clouds.

Storage mobility has gradually become a current researchfocus. WhereStore [7] is a location-based data storage solu-tion for smartphones. It uses filtered replication (a filterexpressing the set of data items that are likely to be accessedin the near future) along with each device’s location history todistribute data items between smartphones and the cloud.STACEE [8] proposes a peer-to-peer (P2P) cloud storagewhere mobile phones, tablets, set-top-boxes, modems, andnetworked storage devices can all contribute as storage withinthese storage clouds. It provides a P2P cloud storage solution


Figure 1. Current service models of MCC.

Mobile as a service provider (MaaSP)

Mobile as a service broker (MaaSB)

Serviceprovider

Service recipient

Service AP

Mobile as a service consumer (MaaSC)

Cloud resource

Servicebroker

Sensornetworks

Service AP

Servicerecipient

Cloud resource

Cloud resource

Service AP

Servicerecipient


and addresses the storage issue for mobile users as a qualityof service (QoS)-aware scheduling problem.

Security and PrivacySecurity related services aim to provide data security protec-tions through the cloud. The security of mobile devices can beenhanced under the help of cloud security mechanism includ-ing cloud-based secure proxy, remote anti-virus, remote attes-tation, and so on.

CloudAV [9] advocates such a cloud-based security modelfor malware detection for end hosts by providing antivirus asan in-cloud security service. Secure web referral services [10]enable antivirus and antiphishing services through the cloud.Referral services depend on a secure search engine to validateURLs accessed by a mobile device to prevent mobile usersfrom accessing phishing websites.

Zscaler [11] is one of the most well-known commercialcloud-based security companies, providing policy-based secureInternet access for mobile devices. It provides a comprehensivecloud-based security solution including three main compo-nents: ZEN (proxy), central authority (CA), and Nanologsserver (log server). Various cloud-based security services arebuilt based on these components. For example, the ByteScanservice enables each ZEN to scan every byte of the webrequest, content, responses, and all related data to block mali-cious actions and data such as viruses, cross-site scripting(XSS), and botnets. The PageRisk service relies on the ZEN tocompute a PageRisk index for every page loaded and enablesthe administrator to control content served to their users basedon an acceptable risk evaluation. The NanoLog service enablesadministrators to access any transaction log in real time.

An increasing number of security features can be enabledin the cloud, in which a reliable and secure connectionbetween a mobile device and the cloud is the main challengefor this type of solution. Google Wallet [12] was developed ona cloud-mobile dual trust root model, where the cloud is incharge of the application-level security such as credit cardtransactions and user credential management, and the GoogleWallet enabled mobile device is protected by strong trust

computing elements on the board to prevent malicious attackson mobile devices.

MCC Context AwarenessNowadays, a smart mobile device usually serves as an infor-mation gateway for mobile users involving various personal-ized activities such as checking emails, making anappointment, surfing the web, locating some interesting spots,and analyzing personal behavior data based on data miningand machine learning. For example, in [13], each mobiledevice has a dedicated mobile cloud engine (MCE) includingthree modules: decision module, publish subscribe module,and context awareness module. The decision module handlesand regulates the transactions among the different parts ofthe MCE. The publish subscribe module is responsible forestablishing the data flow between the mobile application andthe MCE. Finally, the context awareness module providescontext information to the application. The state-of-the-artsolutions lack a unified approach suitable to support diverseapplications while reducing the energy consumption and pro-viding intelligent assistance to mobile users.

Transitions from Internet Clouds to User-Centric Mobile CloudsCurrent MCC Issues and Transition DirectionsFrom the service point of view, current MCC service providersand their customers (i.e., mobile devices) are clearly defined.Most existing computing models are similar to the traditionalclient-server service models. Several issues with existing MCCservices are explained, and the expected transition characteris-tics are also discussed below.

Symmetric MCC service model: Most current MCC servicemodels are asymmetric. As shown by the examples presentedin Table 1, mobile devices are usually considered as clients ofcloud services. The service (e.g., computing and storage ser-vices) direction is mainly unidirectional, from the cloud tomobile devices. With the increasing capability of mobile


Table 1. Summary of MCC services and applications.

MCC services and applications Service models

MCC service types Representative approaches MaaSC MaaSP MaaSB

Mobile cloud computation

CloneCloud [3]

MAUI [4]

ThinkAir [5]

Mobile cloud storage

Dropbox, Box, iCloud, GoogleDrive and Skydrive [6]

WhereStore [7]

STACEE [8]

Security and privacy

CloudAV [9]

Secure Web Referral Services for Mobile Cloud Computing [10]

Zscaler [11]

Google Wallet [12]

Context awareness An Integrated Cloud-Based Framework for Mobile Phone Sensing [13]


devices, mobile devices can also collaboratively execute theapplications’ tasks. Moreover, the virtualized environmentshould provide intelligent feedback to physical devices toadjust their behaviors or actions in order to provide better vir-tualized services. This virtualization-feedback loop modeldemands a symmetric MCC service model; that is, bothmobile devices and the virtualized cloud are service providersas well as clients at the same time.

Personalized situation awareness: In the current complicatedmobile cloud environment, data sources could be diverse (e.g.,a mobile device, the environment, or a social network). Some-times, a single data source is not sufficient to support MCCapplications in the cloud; moreover, data collected from het-erogeneous networks might be unstructured or unclassified.For example, in the physical world, there could be multiplenetworking interfaces and services that are available to auser’s device (a wireless sensor network, social network, vehic-ular network, personal and body area network, etc.). Thecloud should be able to get data from different source net-works and then cluster them together to make the data struc-tural and readable in the future. Thus, more work is expectedto construct situation awareness services that can be personal-ized according to individual users in the virtual environment.

User-centric trust model: Most current cloud trust modelsare centralized: all mobile entities need to trust the cloud ser-vice provider. Storing private data in the cloud environment isa big hurdle for most mobile cloud applications. It is desirableto establish a distributed or decentralized trust managementframework within the virtualized cloud system to address theprivacy concerns of mobile users. In the physical world, thevirtualized resource could be hosted in either public or privateclouds that are tailored according to users’ preference. Thisrequirement demands that the current centralized cloud betransferred in a distributed or decentralized fashion. Forexample, including mobile users’ computing and storageresources into the mobile cloud infrastructure without requir-ing (or even allowing) administrative privilege can significant-ly reduce the privacy concerns of mobile users.

User-Centric Mobile Cloud ComputingThe next-generation MCC applications demand tight integra-tion of the physical and virtual functions running on themobile devices and cloud servers, respectively. Moreover, dueto the mobility of mobile users and changes in the applicationrunning environment, the MCC application functions are notfixed on their running hosts. An illustrative vehicle trafficmanagement example is shown in Fig. 2a, in which a vehicle

may request video capture (VC) functions from other vehiclesdirectly (the dashed line) or through a centralized videofusion function to get a holistic view of the entire road inter-section. In this example, the VC providers are not fixed andare selected by their location. Moreover, a VC function maynot only be used for an individual vehicle; it can also be usedfor road traffic management, accident/hazard detection, andso on. The resources, including mobiles, cloud servers, andcorresponding networking, that form an ad hoc cloud applica-tion running environment can be customized for each individ-ual user; we refer such a customizable ad hoc cloudapplication running system as user-centric MCC. The basicfunctions used to form this MCC application (VC, display,data fusion, etc.) are called provisioning functions (PFs).

The user-centric mobile cloud application running environ-ment can be further illustrated in Fig. 2b, where mobiles (MA,MB, MC) and their responding cloud virtual resources (CVRA,CVRB, CVRC) construct a pairwise resource pool, RX = (MX,CVRX), including both physical and virtualized resources. RXrepresents user X constructing MCC applications formed by aset of PFs {PF}X running on local or remote resource pools.In this user-centric mobile cloud application running environ-ment, a PF can be highly mobile, and composed and used bymultiple applications at the same time.

Design Principles of User-Centric Mobile CloudComputingFuture MCC should be reconsidered as a new service model,where mobile agents (i.e., both physical and virtual entities)and related resources collectively operate as mobile cloudsthat enable computing, storage, and networking capabilities,context awareness modeling, content discovery, and data col-lection and dissemination. To build future user-centric MCCbased on the described concepts and requirements, mobileclouds should be shifted from the traditional Internet cloud byusing the following principles:• Principle 1: User-centric: MCC applications should be

designed in such a way that a user can control their owndata and activities with strong privacy and security protec-tion. Cloud resources should be collected and allocatedaccording to mobile applications customized for each indi-vidual user.

• Principle 2: Service-oriented application platform: Due to thesymmetric service model, every mobile node can potentiallyserve as an MCC service provider; thus, a service-orientedapplication platform is the natural choice for MCC.


Figure 2. An example of mobile cloud applications: a) MCC application scenario; b) user-centric MCC application model.

CVR: Cloud virtual resource (e.g., computing, communication, storage resources)

PF: Provisioning function (can be composed/offloaded to form MCC applications)

Integrated provisioning function group for mobile user.

MCC

RA

RC

RBMA

MC

MB

CVRA

CVRC

CVRB

{PF}A

{PF}C

{PF}C

(a) (b)

Video capture(VC) function

VC

VCCloud data

fusion functioncloud

servers

Physical-virtualenvironment integration(offloading, migration,

and composition)Traffic displayand advisingfunctions invehicle


• Principle 3: Mobility efficiency: MCC resources should bedynamically allocated and managed according to the needof mobile cloud applications. The mobility of MCC shouldbe confined through a set of mobile cloud application con-straints to maximize efficiency using a set of system perfor-mance evaluation metrics such as availability, computingpower, storage, and their spacial-temporal boundaries.

• Principle 4: Virtual representation: MCC maintains a trusted,reliable, and accessible virtual representation for each user.The virtualized representation can be considered as anassistant for mobile users and performs actions such assensing a user’s daily activity to build the user’s behaviorand activity profiles, and delegate the user’s activities in thevirtual environment.

Mobile as a Representer: A User-CentricApproachThe future mobile cloud service model should be deliveredbased on the principles illustrated. Besides previously presentedservice models (i.e., MaaSC, MaaSP, and MaaSB), we present anew user-centric MCC service model called mobile as a repre-senter (MaaR). The architecture of MaaR can be found in Fig.3. In MaaR, each user can be represented by a virtualized entityin the cloud through his/her physical entity (i.e., mobile device).Users’ behaviors and attributes can be collected from the realworld (people, environment, or mobile devices) in real time andsent to their corresponding virtual entities in the cloud to per-form further analysis and processing. Data mining and machinelearning algorithms can be used to analyze a mobile user’s situa-tion and perform actions proactively. MaaR can be regarded asthe next-generation MCC service model in that both physicalsystems and virtual systems are seamlessly integrated throughvirtualization technologies to provide services. In MaaR, themobile devices and clouds are highly interactive, and as a result,the service flow can be presented as bidirectional arrows. Inaddition to helping mobile entities execute tasks more efficient-ly, MaaR is able to accomplish some tasks that are impossible torealize in current MCC architecture.

MaaR model is presented to support the next-generationuser-centric MCC services and applications. A conceptualarchitecture of MaaR is presented in Fig. 4, where both CPSand CVS are integrated as a whole system. In the CPS, het-erogeneous networks coexist, and all these networks can bevirtualized at the CVS by performing operations includingpresenting, offloading, abstracting, caching, migration, and soon. All data with the spatial, temporal, and correlation infor-mation from the CPS will be submitted to the CVS. Amongall these three types of information, correlation information isessential in that it helps to fuse different types of data togeth-er into a well formatted one so that the CVS can further per-form context awareness, user-centric proactive, and securityprotection tasks. For example, the sensor network carries

sensed data, while the social network collects andgenerates the social relationship data. The correla-tion information helps the CVS to generate sensingdata with social attributes (e.g., personal data thatis only accessible from a specific social group, likepeople in the user’s friend list).

In MaaR, the CVS has three main types of pro-visioning resources: computing resource, storageresource, and networking resource. The user’s vir-tual entity is represented by maintaining seamlesscommunication between the CPS and the CVS,which also allows for establishing multiple person-alized MCC clouds due to different application

purposes. An MCC application is able to control integrationof CPS and CVS through a well defined application program-ming interface (API) and MCC tools. The traditional Internetcloud is one-way operational as users can only submit datafrom the CPS to the CVS, while it is possible to allow theCVS to further control the CPS functions in a highly adaptiveand dynamic fashion based on the MaaR model. Besidesphysical data being virtualized to a virtual environment, theCVS can provide feedback and control functions in the CPS.

To enable the service-oriented application running environ-ment, MaaR provides a personal on-demand execution envi-ronment for MCC (POEM) framework [14] to achieve theuser-centric MCC service running platform highlighted in Fig.2b. POEM is a mobile cloud application execution platformthat enables mobile devices to easily discover and composecloud resources for their applications. For mobile resourceproviders, they may not even know what applications and whomay call their provisioned functions beforehand. In this way,the mobile application design should not be application-ori-ented; instead, it should be functionality-oriented (or service-oriented). To achieve these features, we can consider thosePFs as the fundamental application components in the MaaRmodel, which can be composed by mobile cloud servicerequesters in runtime.

POEM takes a comprehensive approach by incorporatingthe OSGi-based [15] service-oriented architecture into MCC.It treats the offloading as part of service composition, and asa result, the codes (or computation tasks) are considered asservices provided by mobile devices and the cloud. In this way,offloading and migration operations can be multidirectional(i.e., among mobile devices and the cloud) compared to one-directional (i.e., from a mobile device to the cloud) in previ-ous solutions. Moreover, due to the popular Java-based OSGiframework, POEM can greatly improve the adoption of theSoA-based code reuse and composition for MCC.

An Application Scenario Based on the User-CentricMaaR ModelTo better understand the proposed future MaaR model, werevisit the vehicular video sensing and collaboration examplepresented in Fig. 2a. We assume that MaaR service modulesare already equipped on many users’ smartphones. When userAlice is driving, her smartphone uses onboard sensors like acamera or GPS to detect her location, driving speed, andimage/video captured on the road. The information can be col-lected and virtualized into the CVS to construct a virtual repre-sentation of the mobile device in the cloud for Alice, which isthe essence of MaaR in that the virtual representer representsthe real situation of the physical user. Practically, the represen-ter is implemented through a set of software agents (i.e., OSGibundles) on a dedicated VM allocated for Alice, where Alicehas the administrative privilege on the VM to decide whichdata can be shared and protected (by encryption). The dedicat-


Figure 3. Mobile as a representor (MaaR).

Real user

Physicalrepresentor

ServiceAP

Cloud resource

Virtualrepresentor

Mobile as a service representor (MaaR)


ed VM is the application holder for Alice to incorporate vari-ous data processing models and functions for security, datamining, and intelligent situation-aware decision making that arepersonalized for Alice’s use. In this model, the VM can behosted in a public or private cloud as Alice chooses.

User Bob may want to know the current traffic status aroundthe bridge five miles ahead where Alice is driving. Users withMaaR services running on their mobile devices near the bridgecan provide sensing functions (e.g., GPS, video/camera), whichare searchable by Bob so that Bob’s display function can callthose functions in real time through either direct P2P connec-tions or a centralized traffic monitoring function provided by athird party. In addition to the presented video capturing usageof the application, MaaR services and applications can alsomaintain social diagrams for each user. For example, when Bobis driving in the area during lunchtime, the MaaR service repre-senter of Bob can prepare for suggestions such as good nearbyrestaurants with high rates by Bob’s trusted friends. Other sug-gestions may relate to Bob’s daily activities and job functionsaccording to his current location, and provide resourcespromptly when Bob needs them. These personalized sugges-tions are based on correlating the location and various senseddata by the MaaR service representor.

SummaryThis article focuses on the introduction of MCC concepts and istutorial in nature so that readers are able to have a holistic viewof the current development of and vision for user-centric mobilecloud computing. We first provide a classification and represen-tative achievements of current MCC service models. Then wediscuss the transformation from the traditional Internet cloud tothe user-centric mobile cloud by listing the issues for currentMCC and presenting user-centric MCC and its design principles.Finally, an MaaR service model with an illustrative example ispresented for achieving the user-centric MCC.

AcknowledgmentThe presented work is sponsored by the ONR Young Investi-gator Program (YIP) award and NSF grant CPS1239396.

References[1] M. Walshy, “Gartner: Mobile to Outpace Desktop Web by 2013,”

Online Media Daily, 2010.[2] Nike Inc., http://www.nike.com.[3] B. Chun et al., “Clonecloud: Elastic Execution between Mobile Device and

Cloud,” Proc. 6th Conf. Computer Systems, 2011, pp. 301–14.

[4] E. Cuervo et al., “Maui: Making Smartphones Last Longer with CodeOffload,” Proc. Int’l. Conf. Mobile Sys., Applications, and Services, 2010.

[5] S. Kosta et al., “ThinkAir: Dynamic Resource Alloation and Parallel Execution inCloud for Mobile Code Offloading,” Proc. IEEE INFOCOM, 2012.

[6] A. Covert, “Google Drive, iCloud, Dropbox and More Compared: What’sthe Best Cloud Option?” Technical Review, 2012.

[7] P. Stued, I. Mohomed, and D. Terry, “Wherestore: Location-based DataStorage for Mobile Devices Interacting with the Cloud,” Proc. 1st ACMWksp. Mobile Cloud Computing & Services: Social Networks and Beyond,2010.

[8] D. Neumann et al., “Stacee: Enhancing Storage Clouds Using EdgeDevices,” Proc. 1st ACM/IEEE Wksp. Autonomic Computing in Economics,2010.

[9] J. Oberheide, E. Cooke, and F. Jahanian, “CloudAV: N-Version Antivirusin the Network Cloud,” Proc. 17th USENIX Security Symp., San Jose, CA,July 2008.

[10] D. H. Le Xu, V. Nagarajan, and W.-T. Tsai, “Secure Web Referral Ser-vices for Mobile Cloud Computing,” IEEE Int’l. Symp. Mobile Cloud, Com-puting, and Service Engineering, 2013.

[11] Zscaler Inc., http://www.zscaler.com.[12] Google Inc., http://www.google.com/wallet.[13] R. Fakoor et al., “An Integrated Cloud-Based Framework for Mobile

Phone Sensing,” Proc. ACM SIGCOMM MCC Wksp., 2012.[14] H. Wu and D. Huang, “Personal On-Demand Execution Environment for

Mobile Cloud Computing,” http://poem.mobicloud.asu.edu, 2013.[15] OSGi Alliance, http://www.osgi.org.

BiographiesDIJIANG HUANG ([email protected]) received his B.S. degree from Beijing Uni-versity of Posts and Telecommunications, China, in 1995. He received hisM.S. and Ph.D. degrees from the University of Missouri Kansas City in 2001and 2004, respectively. He is currently an associate professor in the School ofComputing Informatics and Decision System Engineering (CIDSE)at ArizonaState University. His current research interests are computer networking, securi-ty, privacy, and mobile cloud computing. He is an Associate Editor of theJournal of Network and System Management and an Editor of IEEE Communi-cations Surveys and Tutorials. He has served as an organizer for many inter-national conferences and workshops. His research has been sponsored byNSF, ONR, ARO, NATO, Hewlett Packard, and Consortium of EmbeddedSystems (CES). This research is a result of his ONR Young Investigator Pro-gram Award.

TIANYI XING ([email protected]) is currently a Ph.D. student in CIDSE at Ari-zona State University. He received a B.E. in telecommunications engineeringfrom Xidian University and an M.E. in electronic engineering from Beijing Uni-versity of Posts & Telecommunications in 2007 and 2010, respectively. Heworked in Microsoft Research Asia as a research intern from July to December2009. His research interests are future Internet, mobile cloud computing, andcomputer network security.

HUIJUN WU ([email protected]) is currently a Ph.D. student in CIDSE at Ari-zona State University. He received his B.E. and M.E. in electronics and infor-mation engineering from Huazhong University of Science & Technology in2007 and 2009, respectively. He worked in Alcatel-Lucent Shanghai Bell as aSIT engineer from July 2009 to July 2011. His research interests are mobilecloud application, mobile cloud service framework, and cloud computing.


Figure 4. MaaR conceptual architecture.

Correlation

APIs and tools

Coordination and integration

Mobile cloudresource provisioning

Time

Networkingresource

Storageresource

ComputingresourceSpace

Virtual-life

*Situation-aware*Human-centric*Proactive*Security andprivacyprotection

Control

Cyber virtual system: abstracted network systems

.....Mobile cloud application scenarios

Cyber physical system: complex heterogeneous network systems

Virtualization

Physicallife

*Presentation*Offloading*Abstraction*Caching*Migration*Data

Cellular and datanetworks Social networks Mobile ad hoc

networks

Vehicularnetworks

Personal and bodynetworks

Mission criticalnetworks


http://www.nike.com

http://www.zscaler.com

http://www.google.com/wallet

http://poem.mobicloud.asu.edu

http://www.osgi.org




obile traffic is increasing at a tremendouspace, exceeding far beyond the original capaci-ties of mobile operator networks. This hugeamount of mobile traffic is associated with a

wide plethora of emerging bandwidth-intensive mobile appli-cations popular among an ever growing community of mobileusers. The challenge presented by all of this mobile trafficstems particularly from the fact that current mobile networksare highly centralized, leading to high demand on centrallocations due to backhauling of all data traffic, to dramaticincreases in bandwidth requirements and processing loadresulting in undesirable bottlenecks, and, last but not least, tolong communication paths between users and servers. Theeffects are wasting core network resources, leading to undesir-able delays, and ultimately resulting in poor quality of experi-ence (QoE) for users.

A straightforward solution to these issues may consist ofhaving operators invest in speed or upgrade their core net-work nodes to comfortably accommodate traffic peak hours ofthese emerging bandwidth-intensive mobile applications.While this is technically and technologically possible, it eco-nomically represents a significant challenge for operators, par-ticularly due to the fact that the average revenue per user(ARPU) is not growing as quickly as traffic demands, particu-larly given the trend toward flat rate business models. Therehas thus been a need for cost-effective solutions that can helpoperators accommodate such huge amounts of mobile net-work traffic while keeping additional investment in the mobileinfrastructure minimal. In addition to application-type-basedtraffic admission control techniques (e.g., throttling video traf-fic), an important solution consists in selective IP trafficoffload (SIPTO) as close to the radio access network (RAN)as possible [1]. The key enabler of efficient SIPTO is to place

data anchors and mobility gateways close to RANs, essentiallyleading to a relatively decentralized mobile network deploy-ment [2].

On the other hand, cloud computing is gaining greatmomentum. Its market is expanding at a high speed, thanks tothe multiple features it supports (e.g., multi-tenancy support,pay as you go, elasticity, and cost-efficient scalability) and thenew business models it provides based on infrastructure shar-ing (infrastructure, platform, software as a service — IaaS,PaaS, and SaaS). In the telecommunications area, cloud com-puting has been gaining lots of attention. Indeed, there arealready many telcos and carrier providers deploying cloud-based services; some deployments are only for internal use,whereas others are being sold as a service. The fast growingbusiness of clouding computing is calling for distributedregional data centers (DCs) [3, 4], forming so-called federatedclouds.

Putting these two observations together, cloud providersare distributing their DCs due to growing business. As formobile operators, they need to decentralize their networks tocope with the growing number of smart phones and associatedbandwidth-intensive services. The expected outcome networkarchitecture is depicted in Fig. 1. Indeed, the figure shows thecase of a decentralized mobile network architecture wherebycore network gateways such as packet data network gateways(PDN-GWs) and serving GWs (S-GWs), in the context of theEvolved Packet System (EPS), are geographically distributed.Also shown is a federated cloud consisting of multiple region-al DCs, geographically distributed and interconnected.

In such decentralized mobile networks, the main objectiveof any mobile operator behind SIPTO is to ensure an optimalmobile connectivity service; that is, a user equipment (UE)device shall always be connected to the optimal data anchor

M


AbstractThis article introduces the Follow-Me Cloud concept and proposes its framework.The proposed framework is aimed at smooth migration of all or only a requiredportion of an ongoing IP service between a data center and user equipment of a3GPP mobile network to another optimal DC with no service disruption. The servicemigration and continuity is supported by replacing IP addressing with service iden-tification. Indeed, an FMC service/application is identified, upon establishment, bya session/service ID, dynamically changing along with the service being deliveredover the session; it consists of a unique identifier of UE within the 3GPP mobile net-work, an identifier of the cloud service, and dynamically changing characteristicsof the cloud service. Service migration in FMC is triggered by change in the IPaddress of the UE due to a change of data anchor gateway in the mobile network,in turn due to UE mobility and/or for load balancing. An optimal DC is then select-ed based on the features of the new data anchor gateway. Smooth service migra-tion and continuity are supported thanks to logic installed at UE and DCs thatmaps features of IP flows to the session/service ID.

Follow Me Cloud: Interworking FederatedClouds and Distributed Mobile Networks

Tarik Taleb, NEC EuropeAdlen Ksentini, IRISA/University of Rennes 1

M

0890-8044/13/$25.00 © 2013 IEEE

KSENTINI_LAYOUT_Layout 1 9/20/13 12:49 PM Page 12

and mobility gateways such as PDN-GWs and S-GWs. However, it is very likely to have a UE deviceconnected to an optimal data anchor gateway (asper its current location) but accessing a mobile ser-vice from a distant DC in a distant location (e.g.,UE being in location 2, having its data anchored atP-GW2 but receiving service from DC 1). This intu-itively results in inefficient mobile connectivity ser-vice given the absence of optimal end-to-end (E2E)connectivity. The objective of this article is toenable a user to always be connected to the optimaldata anchor and mobility gateway, and to access itsdata and/or service from the optimal DC, that is,geographically/topologically nearest (or in any othermetric, e.g., load and processing speed) DC. Fur-thermore, this optimal E2E mobile connectivityshall be ensured during the entire movement of theuser. It shall be noted that when the notion of“optimal” or “better” PDN-GW/PDN connectivity is used,this is always meant in comparison to a PDN-GW to the samecloud network to which the UE is already connected. Thedetailed criterion for optimality is defined by operator policy,but typically may be derived from geographical proximity (tothe UE) or load.

In this article, we describe how to achieve the above-men-tioned objectives through the Follow-Me Cloud (FMC1) con-cept, described below. An important restriction on which webase our work consists of the fact that we shall introduce nei-ther additional cost nor complexity to the network. The usageof software defined networking (SDN) technologies such asOpenFlow and the like is thus not considered. Changes toThird Generation Partnership Project (3GPP) standards,including those relevant to the nodes and interfaces of theEPS architecture or the underlying protocols, are not anoption either.

The remainder of this article is structured as follows. Wegive an overview on some related research work. The pro-posed FMC concept is described. We give preliminary perfor-mance results. The article is then concluded.

Related WorkSession/Service IdentificationGenerally speaking, migration of an IP service, due to move-ment of the receiving UE followed by a change in its IPaddress, would result in the breakdown of the session and theneed to reestablish a new one. This is intuitively due to thefact that IP addresses are, in practice, used for identifyingboth an endpoint and a network location. Session identifiersshould therefore be separated from location identifiers. Meth-ods for such separation have been devised before. DomainName Service (DNS) does realize such a separation, but itwas not designed to provide constant updates of current loca-tion. It is rather used only once at session establishment time.The Locator/Identifier Separation Protocol (LISP) [6] makessuch separation explicit, but does not natively support end-point mobility. There are some efforts to include mobility sup-port in LISP, but most of these approaches rely on using acentralized mechanism based on the map server (MS), whichmakes LISP deployment in architecture like 3GPP networksvery complicated. Serval [7] caters for user and service mobili-ty and provides identifier/location separation by introducingan additional layer in the networking stack. It makes use of

service identifiers, which require changes to applications usingthe system. To avoid the breakdown of an IP session betweentwo peers when the IP address of any of the two peers changesduring the course of a session, Network Address Translation(NAT) can be also used. In the context of mobile networks,the support of NAT would require changes to nodes of themobile network operator; also, many operators are not infavor of NAT, mainly due to the foreseen expansion of IPv6.Other research work has considered the usage of OpenFlowto hide, through its rules, any changes to the IP addresses[19]. For OpenFlow-based solutions, scalability represents themain challenge. Indeed, there are various dimensions for scal-ability, including the number of flows, the number of rules,the flow setup rate, number of packets, and the bandwidth ofthe control channel. Some ideas have been proposed to dealwith this issue. DevoFlow [8] reduces the number of controlpackets by moving some of the flow creation work from con-trollers to switches. In [9], the scalability of OpenFlow rules inan FMC scenario is assessed, and an approach to distributecontrol plane functions is proposed to enhance the systemscalability. As mentioned earlier, usage of OpenFlow andother SDN technologies is not an option in this article toavoid any additional complexity in mobile networks.

Information-centric network (ICN) architecture nativelysupports the separation between the user location and thecontent identifiers. Indeed, ICN shifts away from a host-cen-tric model toward an information-centric one, where contentis retrieved according to its name instead of its storage loca-tion (host address). Several ICN approaches have been pro-posed, such as data-oriented network architecture (DONA)[10] and content-centric networking (CCN) [11]. They sharethe same concepts: contents belonging to a service have aunique name and are cached at different locations in the net-work, where:• The name is independent from the location.• The communication is driven by a publish/subscribe model.These solutions differ in the way the contents are named/iden-tified. Naming in ICN could be hierarchical as in CCN, or aflat namespace as in DONA. In CCN the names are rooted ina prefix, unique for each publisher. The granularity of thenames is at the chunk level. The content name has severalcomponents delimited by a character (e.g., /FMC/Content1/chunk1.extension). Behind using hierarchical addressing is thefacility to achieve better routing scalability within name prefixaggregation. In fact, CCN names are used for both namingand transport. Meanwhile, names in DONA are in the formP:L, where P is the hash of the owner’s public key and L isthe owner assigned label. DONA requires another entity (res-olution handler) to perform name resolution by using a route-


1 While FMC is widely used to stand for fixed mobile convergence, thisabbreviation stands for Follow-Me Cloud throughout this article.

Figure 1. Distributed mobile networks and distributed clouds.

Distributedcloud

Motion

Datacenter 1

Datacenter 2

Datacenter 3

S-GW 3

Distributed EPCDistributedmobilenetwork

Consumerdevices

P-GW 3S-GW 2 P-GW 2S-GW 1

P-GW 1

Motion

Location 1 Location 2 Location 3


by-name paradigm. Service centric networking (SCN) [12]extends the principle of ICN to support service request inaddition to content request. Services are software elementslocated in the network infrastructure, hosted on dedicatedhardware placed alongside the routing infrastructure (as isdone with cloud services). Service naming is a mix betweenflat and hierarchical naming. The service name is composed oftwo parts, <service_owner, service_name>, and wild cardsare used (<*, service_name>) if users do not know the ser-vice provider in advance. It is worth mentioning that ICNnaming is very relevant for FMC context, where the aim is toachieve a clear separation between service location/mobilityand UE mobility (layers 2 and 3, L2 and L3).

Service Location/Migration in Federated CloudsFederated clouds refer to the connection of geographicallydistributed DCs together into a common resource pool todeliver a variety of cloud services. Upon reception of a servicerequest, one of these DCs is chosen to deliver the requestedservice over the network to the end user. The distribution ofcloud computing resources over different locations in the net-work is beneficial for different reasons such as increasingavailability, reducing bandwidth cost, and reducing latency bylocating resource near to users. To efficiently handle userrequests, there is a need to define a cloud management proce-dure. This procedure directs the user’s service request to theoptimal DC, which satisfies user constraints (cost), optimizesnetwork use (load balancing), and ensures application qualityof service (QoS)/QoE. Furthermore, this cloud managementprocedure must be able to migrate all or portions of servicesbetween DCs if one of the selected criteria is no longer satis-fied (QoS degradation). Obviously, redirecting a user requestto the geographically nearest DC seems to be the most effi-cient solution. However, for successful services (in a certainregion), redirecting all requests to the geographically nearestDC can overload it, causing degradation of QoS/QoE. There-fore, more sophisticated solutions need to be used for cloudmanagement.

In [13], a cloud management middleware is proposed tomigrate part of user service (constituted by a set of virtualmachines, VMs) between DC sites in response to workloadchange at the DC. Based on workload monitoring at each DC,the middleware initiates VM migration in order to moveapplication components (geographically) closer to the client.Volley [14] is an automatic service placement for geographi-cally distributed DCs based on iterative optimization algo-rithms. Volley migrates services to new DCs if the capacity ofa DC changes or the user changes location (chooses a DCnear the new location). The authors of [15] propose a DCselection algorithm for placing the requested VM by a usersuch that it minimizes the maximum distance between any twoDCs. The DC selection problem was formulated as a sub-graph selection problem. The demonstrator described in [16]shows how services can be placed according to informationretrieved from an application-layer traffic optimization(ALTO) network server. This work can be used to find opti-mal service locations. Note that most of these research teach-ings are orthogonal to the FMC framework described herein.

It is worth noting that there are technical issues to considerwhen migrating services (typically VMs) between two DCs.These issues pertain to the time needed to transfer a VMbetween DCs, which can disturb the service continuity. Thistime depends on:• The time required for converting a VM, particularly if DCs

are not using the same hypervisor• The time required for transferring the service (VM) over

the network

The latter intuitively depends on the object size, the con-nection speed, and the round-trip time (RTT) between theDCs. It is of high importance as VMs are transferred usingFTP/TCP-like applications, the performance of which largelydepends on RTT. To fix this issue, solutions such as file datatransfer (FDT) [17] can be used.

Follow Me CloudProblem StatementReferring again to Fig. 1, a user may be receiving an applica-tion/service from a server in DC 1 in location 1 via P-GW1 ata particular time instant. Later on, the user moves to a differ-ent location (i.e., location 2), and receives the remaining partof the service from a server in nearby DC 2 via an optimalanchor point, P-GW2. In this regard, two mobility scenarioscan be envisioned:• Connect-freeze-reconnect mobility: In this scenario, the

user temporarily pauses/freezes the cloud service whenmoving from location 1 to location 2 (e.g., a thin client useraccessing data from an office, then getting offline whilereturning home, and then accessing data from home).

• Always connected mobility: In this scenario, the userchanges P-GW and DC while being on the move and withno interruption in the service (e.g., a thin client user access-ing data while being onboard a high-speed train during along journey).The issues we aim to solve in this article are the following:

• In Fig. 1, when the UE moves from location 1 to location 2,both the IP address of the UE and the IP address of theserver may change. As discussed earlier, with current net-working solutions, an IP session between two peers willsimply be torn down if the IP address of any of the twopeers changes during the course of the session.

• The second issue pertains to the fact that for the sake ofsystem scalability, the system does not need to migrate thewhole service to the new location of the user, only therequired portion of service.

• The third issue pertains to when, how, and to which DC theservice migration shall be triggered, as well as how the UEshall become aware of the availability of optimal DCsand/or data anchor gateways.This article proposes a number of solutions that address all

these issues, defining a general framework that interworksbetween a distributed mobile operator network and a networkof regional DCs, a federated cloud, to enable the vision ofFMC whereby a cloud service follows the user along hermovement. As described herein, the key features of the pro-posed solutions are the following:• Replacing data anchoring at the network layer by service

anchoring• Replacing IP addressing by service/data identification• Decoupling session/service2 mobility from layers 2 ad 3

mobility

Network ArchitectureIn this article, we consider a network topology as shown inFig. 1, with additional components, namely FMC controllerand DC/GW (data center/gateway) mapping entity as illustrat-ed in Fig. 2. It shall be noted that, while in Fig. 2, these nodesare shown as two independent architecture components, theycan be functional entities collocated with existing nodes or run


2 Throughout this article, the terms service and session are used inter-changeably to refer to the same thing: a service being delivered over a ses-sion.

as software on any DC of the underlying cloud. Inboth figures, both the cloud network and the mobileoperator network are decentralized/distributed.

We first propose that a mobile network operatormaps access point names (APNs) to specific geo-graphical locations (or alternatively to P-GWs’identifiers). These geographical locations could beS-GW service areas, mobility management entities(MMEs) pool areas, P-GWs’ geographical loca-tions, and so on. The corresponding localizedAPNs would look like APN1=“Internet@loca-tion_1,” APN2=“Internet@location_2,” and so on.Admittedly, this is somehow against the originalprinciple of the APN, that is, to achieve locationindependence of access to a PDN. Indeed, the con-cept of APNs was designed for General PacketRadio Service (GPRS) (and carried over to Univer-sal Mobile Telecommunications Systems — UMTS and EPS)as a scheme to separate logical from physical points of inter-connection between a 3GPP operator’s IP network and exter-nally connected PDNs.

The APN allows one logical name to be associated with aparticular type of traffic and maps it flexibly (but constant forthe duration of an IP/PDN connection) to a route and pointof interconnection. The mapping is done by the networkbased on DNS, and while the UE may be aware of it, it is notconcerned with details of the backend connectivity. This wassuitable for typical highly centralized network deployments;however, with new traffic and load scenarios coming into play(especially traffic offload at decentralized points as mentionedearlier), this is no longer sufficient. The UE, not necessarilythe user, may become involved (partially) with network topol-ogy for the sake of its optimal backend connectivity (i.e., mini-mal network resource consumption, cost, and latency) evenwith active data transmission over relatively long durationsand with larger scale mobility.

In the envisioned network architecture, we also considerthat DCs are mapped to a set of P-GWs (i.e., data anchorpoints in EPS) based on some metric, e.g., location or hopcount. This mapping may be static or dynamic. In case of thelatter, it could be that the topology information is beingexchanged between an FMC service provider and a mobilenetwork operator (MNO). Alternatively, an MNO entity/func-tion could be in charge of updating the FMC service providerwith such information in either a reactive or proactive man-ner. Additionally, we assume that an FMC controller entityexists for managing distributed DC instances; alternatively,distributed DCs coordinate among themselves in a self-orga-nizing network (SON) manner.

It shall be noted that the cloud infrastructure and themobile network could belong to the same operator (i.e., MNO= FMC service operator) or be administered by two indepen-dent operators. The FMC controller and DC/GW mappingentity could be either in the premises of the MNO and/orFMC service provider, or owned and operated by a thirdparty.

In the envisioned FMC service, similar in spirit to CCN,content served by the FMC service has some predefined hier-archy; for example, content ID = FMCService/Application-Name.DataName.Characteristics. In the case of the movieTitanic, it could be that the content ID = Video.Titanic.30min;this means that this content is video content from Titanic, andthe frames to be played back are those from the 30th minutesince the beginning of the movie.

In this article, we mainly focus on the case of UE devices inEPS connection management (ECM)-active mode. The focuson UE devices in ECM-active mode is important due to the

fact that most Long Term Evolution (LTE) UE devices, suchas tablets and PCs equipped with an LTE modem, and evendevices similar to currently available 3G smart phones willhave ongoing background traffic due to many applications(Skype, Foursquare, etc.) that involve the frequent signalingof updates and keep alive messages, ultimately keeping UEdevices always actively connected to the network.

FMC Session/Service IdentificationTo replace IP addressing by service/data identification, a spe-cific application logic/plugin is installed at the UE and the DCservers. Indeed, requests from UE devices for an applicationor a service available in the cloud are mapped to a unique ses-sion/service identifier. In other words, any IP session betweena UE device and a cloud server is identified as follows:

Session/Service ID = Function(UE_ID ; Content_ID)

This session/service ID is generated by the end host (e.g., UE)that issues the service request and is communicated to thereceiving end host, which is the cloud server.

It shall be noted that the above proposed structure of thesession/service identification ensures that all sessions used bythe same UE or all sessions used by all UE devices belongingto any mobile network will be uniquely identified, and thereshall be no conflict in the session/service ID. Indeed, theusage of the UE ID (which is supposed to be unique withinand across different mobile operator networks) in the ses-sion/service ID serves to avoid any conflict in session/serviceID among UE devices, whereas the usage of content ID in thesession/service ID helps to differentiate sessions received bythe same UE device. As explained later, the latter also facili-tates a smooth migration of the session/service from a DC toanother one, achieving the concept of FMC. It is also impor-tant to note that since UE devices already have unique IDs,there is no need for a particular server to set up a session ID(e.g., as in the case of the Session Initiation Protocol, SIP).Indeed, in the context of EPS, a number of UE identifiers canbe used. The mobile subscriber integrated services digital net-work number (MSISDN, i.e., the phone number attached tothe subscriber identity module, SIM, card), internationalmobile subscriber identity (IMSI), international mobile phoneequipment identifier (IMEI), temporary mobile subscriberidentity (TMSI), and integrated circuit card ID (ICCID) areall potential alternatives. While it is outside the scope of thisarticle to decide which identifier to use, the following com-pares between MSISDN, IMSI, TMSI, ICCID, and IMEI.First of all, the main concern with MSISDN is the fact that anonly-packet-switched (PS) UE device does not need to havean MSISDN. As for IMSI, it is highly confidential, and mobile


Figure 2. Interworked cloud/mobile networks architecture.

FMC controller

DC/GW mapping entity

Mapping

Set 1 of anchor GW

Distributedcloud

Distributedmobilenetwork

Datacenter 1

Datacenter 2

Datacenter 3

Set 2 of anchor GW Set 3 of anchor GW


operators prefer not to expose it outside the mobile networkdomain. Regarding TMSI, as the name infers, it is a tempo-rary identifier that may change during the course of a ses-sion/service, mainly in case the UE goes idle for a while. It isthus not preferred for supporting “connect-freeze-reconnect”mobility scenarios. Using ICCID may be an interesting solu-tion as an ICCID with the minimum sized individual accountidentification number (IIN) (11 digits) provides approximately1011 or 100 billion unique identifiers per IIN. This amount perissuer (e.g., per MNO) would appear to be more than ade-quate to provide a new unique subscription identifier forFMC service/sessions from different UE types, including, say,machine-to-nachine/machine type communications(M2M/MTC) devices. It should also be noted that similar toMSISDN, the composition of ICCID contains enough routinginformation to be used to identify the home subscriber serv-er/home location register (HSS/HLR) of the UE/mobile sta-tion (MS) in case the FMC controller needs to contact theHSS/HLR. The IMEI and IMEISV (IMEI software version)are used to identify individual mobile devices. The total num-ber of devices that can be uniquely identified with an IMEI is1014, which seems to also be adequate for supporting FMCservice requests from different UE/MS types.

Triggering FMC Session/Service MigrationThe possible need for FMC service migration can be intuitive-ly noticed when a UE device changes its data anchor gateway(i.e., P-GW relocation); that is, changes its IP address. Achange of the IP address of the UE device can certainly benoticed by the corresponding DC. A preliminary decision hasto first be made by the UE and/or current DC on whether aservice migration is worthwhile or not. This decision may bebased on the service type (e.g., an ongoing video service withstrict QoS requirements may be migrated) [12], content size(e.g., when a user has been watching a movie and the movie isabout to finish at the time of P-GW relocation, the UE maydecide, at the FMC application layer, not to initiate the ser-vice migration), task type of the service (e.g., in case of MTC,in a session of emergency warning services, delay-sensitivemeasurement reporting services always have to be migrated tothe nearest DC), and/or user class. It is worth noting that theservice migration decision (to migrate or not) relies on severalattributes/criteria (could be conflicting) that depend on theuser’s expectation on the service (QoS/QoE, cost) and net-work/cloud provider policies (at each P-GW relocation, loadbalancing, maximize use of DC resources). Accordingly, tomigrate a service or not can be defined as a multi-attributedecision making (MADM) issue, and solved by any relevantalgorithm in this area.

Once it is deemed appropriate, by either UE or currentDC, to migrate the service, the FMC plugin available at theDC may request the FMC controller to select the optimal DCwith the right service and right content to serve the UE in itsnew location, and to initiate the service migration. As a ser-vice may consist of multiple cooperating sessions and pieces,the decision has to be made whether the service has to befully or partially migrated, while considering the servicemigration cost, such as the cost associated with the initiationof a new virtual machine at the target DC, the cost (if any)associated with the release of resources at the source DC, andthe cost associated with bandwidth consumption due to trafficto be exchanged between the DCs as well as the FMC con-troller. An estimate of the cost/overhead possibly incurredshall be compared against benefits to the cloud in terms oftraffic distribution and to end users in terms of QoE. It shallbe noted that there are different forms (e.g., state, data,images), different technologies (e.g., VMware), and different

approaches (e.g., SaaS, PaaS, or IaaS) for service migration.The latter decides the former.

Awareness of the Need for Data Anchor GatewayRelocationAs mentioned earlier, service migration may be triggered fol-lowing data anchor gateway relocation. Such relocation is fea-sible for UE devices in ECM-idle mode [1] and also for UEdevices in ECM-active mode [2]. These solutions work underthe assumption that a UE device is aware when an optimal P-GW becomes available and subsequently establishes a new IPsession via this optimal P-GW. An important question is how aUE device becomes aware of the availability of an optimaldata anchor gateway, so it will trigger relocation from the cur-rent data anchor gateway to the optimal one. In this subsec-tion, we provide a number of solutions that render a UEdevice aware of these things. Indeed, a UE device may use theS-GW change or MME change within existing handover proce-dures as a trigger. It should be noted that S-GW change andMME change could potentially indicate a change in the S-GWservice area and MME pool area, respectively. In the case ofan S-GW change for the cause of load balancing, this changeshall indicate that the current S-GW is no longer optimal, andthat another better S-GW has become available. Additionally,and especially in a distributed mobile operator network whereS-GWs could be potentially collocated with P-GWs, a changein S-GW could be an indication that a change of P-GW maybe desired; even with non-collocated S-GW and P-GW, thesame indication of non-optimality of the current P-GW can beutilized. It should be noted that according to current 3GPPspecifications [5], a UE device is aware of an MME change,but not of an S-GW change. Knowing of an MME changedoes not necessarily make a UE device aware of the distribut-ed network topology; the same can be said when the UEbecomes aware of an S-GW change. Indeed, a UE deviceneeds to know only about the optimality of the currently serv-ing P-GW, not the distributed network topology in full.

As mentioned earlier, while MME change is noticed by theUE, as it holds relevant context at its information storage,with the current standards, an S-GW change cannot benoticed by the UE. For this purpose, we propose that when anS-GW changes as part of a tracking area update (TAU) pro-cedure (which in turn occurs within an X2or S1-based hand -over procedure [5]), MME sends a corresponding flag in theTAU accept message to the UE. The UE shall interpret thisflag as an indication that the current P-GW may no longer beoptimal and that another optimal P-GW may have becomeavailable. Alternatively, the MME sends the optimal APN inthe TAU accept message to the UE so that the UE will use itto request PDN connectivity whenever it desires to initiate anew IP session to the same PDN.

Alternatively, a UE device may request APN informationfrom a configuration server (e.g., access network discoveryselection function, ANDSF) and subsequently requests PDNconnectivity indicating the “localized” APN. For the sake ofcomparison, Fig. 3 depicts the existing APN resolution mecha-nism (full lines and steps numbered from 1 to 4) and the pro-posed mechanism (dashed lines and steps numbered from Ato D). This option assumes that the ANDSF acquires local-ized APN information. The advantage is that existing non-access stratum (NAS) signaling can be kept unchanged (onlythe ANDSF information element is used differently ). Itshould be noted that while ANDSF was initially designed toprioritize for a UE device a list of currently available non-3GPP accesses, there is recent work in 3GPP that aims toenable ANDSF to provide UE devices with policies on which



PDN connection to select [7]. Based on an indicationfrom the MME that an S-GW has changed or an MMEchange notification, UE consults ANDSF or DNS orany other network node with defined policies. ANDSF(or the like) is assumed to maintain a table, mappingAPNs for each location. Upon receiving the currentlocation of the UE from the UE, ANDSF provides theUE with policies, based on which UE establishes newIP sessions to the same PDN via a new optimal P-GWusing the relevant APN indicated by the ANDSF. Usingthis indicated APN, the UE issues a PDN connectivityrequest to the MME [3]. MME uses the P-GW selec-tion function to select the optimal P-GW for the UE toconnect to the same PDN [16]. After the setup of thenew PDN connection, the UE stores the relevant APNinto its information storage and maps the relevant IPflows to the relevant PDN connection and APN. Theadded signaling steps between UE and ANDSF and thedifferent use of APN (now as a “localized” APN) isshown in Fig. 3, with dashed lines and steps numberedfrom A to D. The last two steps are identical to steps 1and 2 of the existing procedure. It is also assumed thatANDSF has its configuration data aligned with theDNS data; this is indicated by the double arrow betweenthe two entities. When IP sessions being delivered ontop of a given PDN connection are all off (e.g., if thetime of the last received/transmitted packet on the PDN con-nection is older than a certain threshold), the relevant APNsare deleted from the UE’s information storage.

FMC Session Establishment and MigrationFigure 4 shows the flow of signaling messages and procedurescarried out to establish an FMC session and migrate it to adifferent server via a different anchor point. In step 1, the net-work layer of the UE establishes PDN connectivity with anadequate P-GW, PGW1. The UE is then assigned an IPaddress, IP1, from within the range of IP addresses of PGW1.Later on, at step 2, the user of the UE decides to initiate asession/service to view content available at the cloud. Forexample, the user indicates the data she desires to view to theFMC controller (or to another appropriate node in the clouddomain) via a web portal or web interface. Based on the cur-rent IP address of the UE and DC/GW mapping informationavailable at the FMC controller (or another appropriate nodein the cloud domain), the FMC controller selects the appro-priate DC and issues a request for establishing the relevantsession in step 3. In step 4, the FMC controller indicates thecontent ID (i.e., the content name and relevant features), theUE identifier to identify the session, and the IP address of theUE, IP1. Afterward, the session is established and identifiedas a function of the content ID and the UE Identifier (step 5).In step 6, during the mobility of the user, the UE becomesaware of the availability of an optimal anchor gateway asdescribed above. In step 7, the UE establishes a new PDNconnection and receives a new IP address, IP2. Being awareof the change in the IP address, and once it deems that a ser-vice migration is worthwhile, the FMC application logic at theUE issues a service migration request to the FMC controller(or to another appropriate node in the cloud domain), indi-cating the session/service ID with new characteristics regard-ing the content/service (last played frame of a video content,last viewed page of an electronic book, etc). In step 9, oncethe FMC controller decides that it is worthwhile to enforcethe migration of the service to a different DC (i.e., comparingincurred overhead/cost vs. benefit), it carries out DC selectionbased on the DC/GW mapping information and the new IPaddress of the UE, IP2. In step 10a, if the content (e.g., code,

data, state) is not available at the newly selected DC, theFMC controller issues a content migration request message tothe source DC requesting that it forward required contentportion to the newly selected DC. In response, in step (10a′),the source DC forwards required content and/or exchangesadequate state information with the newly selected DC. Itshould be noted that depending on the data size, data migra-tion can be performed using one or more suitable robust andfast data delivery technologies. In step 10b, the FMC con-troller issues a session migration request message to theselected target DC (DC2 in Fig. 2) indicating the session/ser-vice ID, new characteristics of the content/service, and thenew IP address, IP2. In step 11a, the session/service migrationtakes place. In this way, despite a change in the IP addressesof both the UE and DC server, the session continues withoutbeing torn down as the session/service is identified by aunique identifier of the mobile terminal. In step 11b, if theold PDN connection to the old PGW (PGW1) was solely usedby the FMC session/service, it is released based on a triggerfrom the FMC application logic at the UE.

Regarding steps 8–10b, it may be that the UE issues a ses-sion/service migration request to the source DC server.Assuming the DC server is acquired with the DC/GW map-ping information (alternatively, the DC server may consult theDC/GW mapping node on demand), the source DC serverselects the target DC based on the new IP address of the UE.It then forwards the required portion of the content to thenewly selected DC and requests session migration indicatingthe new IP address of the UE and the session ID. Then steps11a and 11b take place. While Fig. 4 shows the case of UE-triggered session migration, session migration can also be trig-gered by the cloud (e.g., for maintenance of the current DC).Intuitively, in the case of cloud-triggered session migration,steps 6, 7, and 8 are omitted. Instead, the DC requests sessionmigration, as in step 8.

ResultsIn this section, we present preliminary results regarding theperformance of FMC. Further results based on an analyticalmodel of FMC are available in [18]. We used ns-3 to simulate


Figure 3. Overview of APN resolution mechanism (full lines: existing3GPP mechanism; dashed lines: proposed enhancement).

UE

S-GW P-GW

4

3

1

C

2 D

A

UE location

“Localized” APN

“Localized” APN

ANDSF

APN

P-GW@

DNS

MME

Resolve“localized” APNto P-GW@

ResolveAPNto P-GW@

B


the architecture of Fig. 1, adding one more location (location4 including one more DC, S-GW and P-GW). We used amobile UE, which remains in each P-GW service area for aduration of 5 min. The simulation runs for 20 min. We com-pare FMC against the case of triggering service migrationafter two P-GW relocations and the case of no service migra-tion (the service remains in the first affected DC). Here, weconsider no congestion in the used links. The data latencydepends only on the number of hops (communication pathlength) from P-GW to DC. We assume that the time requiredfor the service migration is short enough to ensure no distri-bution in the service.

Figure 5 shows the data latency during the simulation forthe three mechanisms. Clearly, we notice that FMC achievesthe lowest data latency as the service is always placed at theoptimal DC (geographically nearest). In contrast, if no servicemigration is used, the data latency increases along with theUE movement, as the UE is connected to new P-GWs thathave long communication paths to the initial DC hosting theservice. However, the gain of FMC has a cost in terms of sig-naling and number of objects migrated, which is higher thanthe other two mechanisms (Table 1). Effectively, for each ser-vice migration, the cost is incurred by the size of the migratedobjects and the number of exchanged signaling messages (typ-ically three messages; Fig 3). In FMC, service migration istriggered after each P-GW relocation; the final cost in thissimulation scenario is therefore three times (i.e., 3 P-GWrelocations) the cost of service migration.

These results clearly indicate a need for more sophisticatedalgorithms for service migration. Therefore, solutions such asthose based on MADM algorithms can efficiently balancebetween performance and incurred cost. Indeed, as stated inthe article, the decision to migrate a service or not is not triv-ial as there are several constraints to consider, which relate to

either operator policies or user quality needs. Therefore, anyunderlying decision making process needs to find a trade-offbetween these attributes. MADM techniques are usually usedto solve such problems. One of the most efficient MADMapproaches is based on the Technique for Order of Prefer-ence by Similarity to Ideal Solution (TOPSIS) solution. TOP-SIS assumes the availability of m alternatives (options) and nattributes/criteria as well as a score for each option withrespect to each criterion. We denote by xi,j the score(attribute) of option i with respect to criterion j. In our case,m represents the decision of migrating a service or not, whilen represents the number of criteria (e.g., QoE, P-GW reloca-tion, cost of migration) to be considered in the decision mak-


Figure 4. Flow chart for initial FMC session establishment and FMC session migration.

DC1server

DC2server

App L3 App L3 AppL3

UE

P-GW1 P-GW2 FMC controller

(1) PDN connectivity @IP1

(7) PDN connectivity @IP2

(5) Session establishment session ID=function(Content_ID, UE_ID, IP1)

(11a) Session migration with the same session ID=function(ContentID, UE_ID, + updates content characteristics)

(2) Initiate session request (ContentID, UE-ID)

(8) Initiate migration request (SessionID, new characteristics of content)

(6) Trigger foroptimal P-GW

availability

(3) DC selectionbased on IP1 and

mapping info

(9) DC selectionbased on IP2 and

mapping info

(4) Session request(ContentID, UE-ID)

(10a) Session migration request(new characteristics of content)

(10b) Session migration request(SessionID, new characteristics of content, IP2)

(11) Old PDN connectionis released if it was usedsolely by the FMC service

(10a’) State info/data migration

Figure 5. Data latency for a UE.

Simulation time (min)20

10

0

Late

ncy

(ms)

20

30

40

50

60

70

80

90

100

4 6 8

P-GW relocation

10 12 14 16 18 20

Service migration if two P-GW relocationsFMCNo service migration


ing process. According to the TOPSIS technique, the decisionto migrate a service or not will depend on the alternative thatreduces the gap with the ideal solution according to criteria aswell as the weight defined before. Employing MADM in FMCdefines one of the authors’ future research work directions.

ConclusionThe described FMC framework enables mobile cloud servicesto follow their respective mobile users during their journeysby migrating all or portions of services to the optimal DC toensure them the best QoE. A service migration decision isbased on user constraints and network operator policies, andparticularly on the P-GW relocation procedure. In fact, ateach P-GW relocation, the service migration procedure has todecide to migrate or not, or to migrate a portion (or none) ofservices to a new DC that is near the new P-GW location interms of communication path length. First results show thepotential of FMC to reduce the data latency when accessing aservice in the cloud for mobile users.

Furthermore, FMC implementation is possible without theuse of any SDN technology, avoiding any otherwise associatedscalability issues, only exploiting the already available uniqueidentifiers of mobile users and findings of ICN and CCN, par-ticularly those relevant to service/content naming, a topicincreasingly gaining tremendous interest. The framework doesnot add any major complexity to the current mobile networkarchitecture, and is thus highly feasible, practical, and stan-dards-compliant.

While the present article validates the FMC conceptthrough simulations, some of the authors’ recent researchwork has proven its feasibility using real tests, particularly anOpenFlow-based implementation of FMC. The findings ofthis implementation are available in [19].

References[1] K. Samdanis, T. Taleb, and S. Schmid, “Traffic Offload Enhancements for

eUTRAN,” IEEE Commun. Surveys & Tutorials, vol. 11, no. 3, Aug. 2012,pp. 884–96.

[2] T. Taleb, K. Samdanis, and F. Filali, “Towards Supporting Highly MobileNodes in Decentralized Mobile Operator Networks,” Proc. IEEE ICC 2012,Ottawa, Canada, June 2012.

[3] R. Miller, “AOL Gets Small with Outdoor Micro Data Centers,” Data Cen-ter Knowledge, July 2012.

[4] R. Miller, “Solar-Powered Micro Data Center at Rutgers,” Data CenterKnowledge, May 2012.

[5] 3rd Generation Partnership Project, “General Packet Radio Service (GPRS)Enhancements for Evolved Universal Terrestrial Radio Access Network (E-UTRAN) Access,” TS 23.401 (work in progress).

[6] D. Farinacci et al., “Locator/ID Separation Protocol (LISP),” IETF Internetdraft draft-ietf-lisp-13.txt, June 2011.

[7] E. Nordström et al., “Serval: An End-Host Stack for Service-Centric Net-working,” Proc. 9th USENIX Symp. Networked Sys. Design and Implementa-tion, San Jose, CA.

[8] A. R. Curtis et al., “DevoFlow: Scaling Flow Management for High-PerformanceNetworks,” Proc. ACM SIGCOMM 2011, Toronto, Canada, Aug. 2011.

[9] R. Bifulco et al., “Scalability of a Mobile Cloud Management System,”Proc. Wksp. Mobile Cloud Computing (MCC) in conjunction with ACM SIG-COMM 2012, Helsinki, Finland, Apr. 2012.

[10] T. Koponen et al., “A Data-Oriented (and Beyond) Network Architec-ture,” Proc. ACM SIGCOMM 2007, Kyoto, Japan, Aug. 27–31, 2007.

[11] V. Jacobson et al., “Networking Named Content,” Proc. ACM CoNEXT2009, Roma, Italy.

[12] T. Braun, A. Mauthe, and V. Siris, “Service-Centric Networking Exten-sions,” Proc. ACM Symp. Applied Computing 2013, Coimbra, Portugal.

[13] B. Malet and P. Pietzuch, “Resource Allocation Across Multiple CloudData Centres,” Proc. ACM MGC 2010, Bengalore, India.

[14] S. Agarwal et al., “Volley: Automated Data Placement for Geo-DistributedCloud Services,” Proc. 7th Symp. Networked Syst. Design and Implementa-tion 2010, San Jose, CA.

[15] M. Alicherry and T. V. Lakshman, “Network Aware Resource Allocationin Distributed Clouds,” Proc. IEEE INFOCOM 2012, Orlando, FL.

[16] M. Steiner et al., “Network-Aware Service Placement in a DistributedCloud Environment,” Proc. ACM SIGCOMM 2012, Helsinki, Finland, Aug.2012.

[17] File Data Transfer, http://monalisa.cern.ch/FDT/.[18] T. Taleb and A. Ksentini, “An Analytical Model for Follow Me Cloud,”

Proc. IEEE GLOBECOM 2013, Atlanta, GA, Dec. 2013.[19] T. Taleb, P. Hasselmeyer, and F. Mir, “Follow-Me Cloud: An OpenFlow-Based

Implementation,” Proc. IEEE GreenCom 2013, Beijing, China, Aug. 2013.

BiographiesTARIK TALEB ([email protected]) is currently working as a senior researcherand 3GPP standards expert at NEC Europe Ltd, Heidelberg, Germany. Priorto his current position and until March 2009, he worked as an assistant pro-fessor at the Graduate School of Information Sciences, Tohoku University,Japan, in a laboratory fully funded by KDDI, the second largest network oper-ator in Japan. From October 2005 to March 2006, he worked as a researchfellow with the Intelligent Cosmos Research Institute, Sendai, Japan. Hereceived his B.E. degree in information engineering with distinction, andM.Sc. and Ph.D. degrees in information sciences from GSIS, Tohoku Universi-ty, in 2001, 2003, and 2005, respectively. His research interests lie in thefield of architectural enhancements to mobile core networks (particularly3GPP’s), mobile cloud networking, mobile multimedia streaming, congestioncontrol protocols, handoff and mobility management, intervehicular communi-cations, and social media networking. He has been also directly engaged inthe development and standardization of the Evolved Packet System as a mem-ber of 3GPP’s System Architecture working group. He is a board member ofthe IEEE Communications Society Standardization Program DevelopmentBoard. As an attempt to bridge the gap between academia and industry, hefounded and has been the General Chair of the IEEE Workshop on Telecom-munications Standards: From Research to Standards, a successful event thatreceived the Best Workshop Award by IEEE the Communication Society. Heis/was on the Editorial Boards of IEEE Wireless Communications, IEEE Trans-actions on Vehicular Technology, IEEE Communications Surveys & Tutorials,and a number of Wiley journals. He is serving as Vice-Chair of the WirelessCommunications Technical Committee, the largest in IEEE ComSoc. He alsoserved as Secretary and then Vice Chair of the Satellite and Space Communi-cations Technical Committee of IEEE ComSoc (2006–2010). He has been onthe Technical Program Committee of different IEEE conferences, includingGLOBECOM, ICC, and WCNC, and has chaired some of their symposia.

ADLEN KSENTINI ([email protected]) is an associate professor at the Univer-sity of Rennes 1, France. He is a member of the INRIA Rennes team Dionysos.He received an M.Sc. in telecommunication and multimedia networking fromthe University of Versailles. He obtained his Ph.D. degree in computer sciencefrom the University of Cergy-Pontoise in 2005, with a dissertation on QoS pro-visioning in IEEE 802.11-based networks. His other interests include futureInternet networks, cellular networks, green networks, QoS, QoE, and multime-dia transmission. He is involved in several national and European projects onQoS and QoE support in future Internet networks. He is a co-author of over40 technical journal and international conference papers. He has been in thetechnical program commitee of major IEEE ComSoc conferences, includingICC/GLOBECOM, WCNC, and PIMRC.


Table 1. Cost incurred by service migration.

Mechanism Cost

FMC 3*(Migrated-Objects-size + 3* Signal-ing messages)

Service migration (iftwo P-GW relocation)

1* (Migrated-Objects-size + 3* Signal-ing messages)

No service migration 0


http://monalisa.cern.ch/FDT/



he rapid increase in the use of multimedia services andapplications on mobile devices has led IT companiesto evolve their technologies to cope with the multime-dia requirements. Cloud computing, which is a new

content-centric paradigm, can fulfill these requirements byproviding data and computing resources on demand. It allowsusers to access infrastructure, platforms, and software at lowcost. For example, Amazon provides its users personal storagespaces with Simple Storage Services (S3) and ability to per-form extensive computation on the data using Elastic Com-pute Cloud (EC2). Likewise, Google’s App Engine allowsusers to develop and deploy their applications on Google’splatform.

On the user’s side, the demand for mobile services is rapid-ly growing. It is expected that the number of mobile users willexceed 800 million by 2015 [1]. However, mobile devices haveseveral limitations, such as short battery life, and limited stor-age and computation power. To address these limitations,mobile cloud computing (MCC) is presented as an integrationof cloud computing and mobile technology. MCC is definedas the infrastructure where both data storage and processingare offloaded from mobile devices to the cloud, bringingmobile applications a much broader range of users [2]. MCCovercomes the limitations of mobile devices by moving thedata processing and storage to the powerful platforms locatedin the cloud.

MCC can provide an infrastructure for various mobile appli-cations such as emergency response management, large scaleevent planning such as Olympics, mobile gaming [3], and inter-active video streaming [4]. It can support multimedia servicesin scalable mobile environments. For example, MCC can beused along with urban transportation systems [5] to provideupdated traffic information for drivers. Traffic data and scenesare collected and processed on the cloud to make traffic deci-sions. Such information forms multimedia data that can beaccessed by mobile users. In large-scale event planning, dis-tributed cloud can provide a variety of multimedia data andservices to fulfill the needs of tourists. Such information can

include precomposed multimedia brochures, tour guide videos,and images. Such massive data can be archived and deliveredby distributed clouds constituting the MCC architecture.

The objective of this article is to address the challenges ofmobile services in terms of data management and networking,and develop an architecture that can lead to the design ofMCC. In particular, our focus is on the retrieval and commu-nication of preorchestrated multimedia data, which imposesseveral resource management challenges on designing anMCC architecture. The main technical challenges are high-lighted as follows,• Heterogeneous networks and QoS requirements: Multimedia

services may span multiple heterogeneous network proto-cols, such as second generation (2G), 3G, and Long TermEvolution (LTE), with different quality of service (QoS)requirements. Dynamic resource allocation protocols areneeded to meet these.

• Heterogeneous multimedia data: Distributed mobile servicessuch as video over IP, multimedia streaming, and photosharing can consist of various types of data including video,audio, and images. Such data may have different deliveryrequirements that need to be synchronized to providecoherent information to mobile users [6]. The proposed architecture entails multiple layers of func-

tionality and addresses the QoS requirements and resourcemanagement challenges in terms of end-to-end delay, jitter,buffering, and bandwidth. A novel feature of this architecture isthe integrated subsystem of cloudlet and base station, whichprovides a “close-to-the-user” proxy system functionality thatensures seamless delivery of data that meets QoS requirements.This functionality is achieved by dynamic allocation ofresources, including buffers and radio frequency (RF) channels,synchronization of multiple streams, and seamless handoff ofstreams among base stations. In this article, we first present acloud architecture for MMC and its components. Then we pro-vide a layered architecture of the MCC and the handoff proce-dure. Finally, we present resource management challenges andperformance assessment of the MCC architecture.

T


AbstractMobile cloud computing is emerging as a new paradigm for supporting a broadrange of multimedia services. MCC alleviates the burden of storage and computa-tion on mobile devices. In this article, we describe design requirements and anarchitecture for MCC. The novelty in this architecture is an integrated cloudlet andbase station subsystem that can meet application-level quality of service require-ments and allow mobile resource provisioning close to the user. We present a lay-ered architecture for MCC that elucidates the required functions and protocols. Wealso propose a connection handoff mechanism among cloudlets and discuss relat-ed resource management challenges for MCC.

A Distributed Cloud Architecture for Mobile Multimedia Services

Muhamad Felemban, Purdue UniversitySaleh Basalamah, Umm Al-Qura University

Arif Ghafoor, Purdue University

T

0890-8044/13/25.00 © 2013 IEEE

FELEMBAN_LAYOUT_Layout 1 9/20/13 12:51 PM Page 20

Cloud Architecture for Mobile MultimediaUsers

In Fig. 1, we perceive a distributed multimedia cloud architec-ture for mobile users that consists of distributed multimediadata centers, and integrated cloudlet and base station. The pro-posed MCC supports this environment. It can be noted thatMCC architecture has two major components; a set of dis-tributed multimedia data centers and an integrated cloudletand base station subsystem. We assume that the data centersact as repositories for multimedia information. A multimediadata center in the cloud retrieves the requested data from thedatabase and communicates it to the cloudlet over the Internet.The cloudlet then transmits the multimedia information tomobile users on the RF channels. The MCC architecture over-comes the high latency that results from the direct communica-tion between a large number of mobile users and multimediadata centers. The cloudlet ensures QoS to mobile users bymanaging the interface between the Internet and the mobilenetwork. It coordinates with allocation of RF resources throughits local base station. We now briefly discuss the two main com-ponents and accordingly discuss the layered architecture.

Virtual Multimedia Data Centers in MCCMultimedia data is not monolithic in nature and can be com-posed of several objects that are stored in different multime-dia cloud data centers, as depicted in Fig. 1. For example,real-time multimedia information can be streamed to userscontaining precomposed data including video, audio, and text.Multimedia information consisting of different data is repre-sented as a multimedia document. Figure 2 depicts the com-position of distributed multimedia objects into a singlemultimedia document. A document needs both spatial andtemporal composition. Temporal composition refers to theprocess of synchronizing multiple streams of multimedia data,whereas spatial composition allows superposition and overlayof multimedia data.

Temporal composition of multimedia objects requires syn-chronization among data streams of a document. Temporalsynchronization can be achieved at a level of fine-grained dataunit referred to as a synchronization interval unit (SIU). Asingle multimedia object is transmitted as a stream of SIUs.Since the SIU is the basic unit of playout, it is essential thatthe SIU’s playout deadline is met. Several document specifica-tion models that specify the temporal synchronization andquality of presentation (QoP) requirements exist in the litera-ture. One such model is the object composition Petri-net(OCPN), which uses an augmented Petri-net model [7].OCPN captures the synthetic relationships between theobjects and identifies media synchronization points. For theOCPN model, a schema to maintain the temporal relation-ships between objects can be constructed. The schema is usedfor storing and retrieving the objects from the distributed datacenters. OCPN also defines a specification of QoP require-ments for multimedia communication that includes speedratio, utilization, average delay, maximum jitter, maximum biterror rate, and maximum packet error rate. Multimedia appli-cations might tolerate some of the QoP based on delay sensi-tivity and error tolerance requirements. Real-time videostreaming, for example, requires a high data rate, and moder-ate delay and jitter. However, other multimedia applicationscannot tolerate high delay and jitter, such as interactive multi-media applications.

From the data centers perspective, the aforementionedtechnical challenges of providing heterogeneous network andQoS requirements can be addressed by employing distributedmultimedia data centers that store and deliver the requiredmultimedia information. To increase the performance, multi-media information can be replicated at various data centers,which requires virtualized access and retrieval mechanisms.Korotich and Samaan [8] proposed a service virtualizationarchitecture that hides the selection and configuration of themultimedia data center from end users. For the MCC archi-tecture, a similar virtualization mechanism can be used that iscomposed of a virtual data center (VDC) mapped to a single


Figure 1. Distributed cloud architecture for multimedia services.

Buffers

Integratedcloudlet andbase station

Distributedmultimediadata centers

VBM

RFchannels

Videocloud

Audiocloud

Imagecloud


or group of physical data center(s) that are used to retrievethe required multimedia object. The main component of thearchitecture is the virtualization manager (VM), which main-tains a hierarchy of VDCs and maps the end user’s request toa VDC. The request mapping is achieved using a service bro-ker that receives the request and locates the VDC where theobject is stored based on the input and output parameters.Once the VDC is located, the VM delivers the multimediaobject to the end user’s cloudlet.

Integrated Cloudlet and Base Station Architecture ofMCCHigh latency of packet and jitter delay are fundamental obsta-cles to developing mobile multimedia services. Due to the lim-ited bandwidth of 3G and 4G networks, widely dispersed,resource-rich, low-cost cloudlets can be deployed close to themobile devices. Cloudlets can perform distributed synchro-nization and composition of multimedia objects, as shown inFig. 3, to reduce the burden of computation from the mobiledevices.

The main function of the cloudlets is to provide a seamlessinterface between two diverse networks: the Internet andmobile networks. For this purpose, cloudlets can be integratedwith mobile base stations to form a logical entity that can pro-vide seamless end-to-end synchronization of multimediastreams to users, as shown in Fig. 3. Accordingly, in coordina-tion with its local cloudlet, a base station manages the out-bound RF channel to support multimedia connection for itsmobile users. For mobile multimedia services, the RF channelis a precious resource and therefore needs to be managedintelligently. The policy for channel allocation, however, canchange dynamically due to various factors such as the numberof users being served concurrently by the base station, thechanging level of concurrency of multimedia objects, andmanaging migrated “calls” from neighboring base stations.The channel allocation policy can be designed based on theassumption that the requested multimedia data are deliveredto the base station prior to the transmission to mobile users.However, this assumption may not be valid because of thenon-deterministic delays data packets may encounter over the

Internet. Therefore, buffering at the base station is requiredto compensate the jitter delays to avoid discontinuity of pre-sentation at the mobile devices. The buffering requirementcan be fulfilled by the cloudlet. The overall functionality ofintegrated cloudlet and base station is summarized as follows:• Providing synchronization and composition functionality for

multiple multimedia streams• Handling speed mismatch between the Internet and the

mobile network• Managing handoff calls and multimedia sessions• Dynamically allocating resources to mobile users

To manage the aforementioned functions, software-defined networking (SDN) technology can be utilized, whichcan allow separation of the control plane from the dataplane [9]. In essence, the base station, as part of the controlplane, implements session setup and teardown, paging, ses-sion handoff, and RF channel allocation protocols. On theother hand, the cloudlet manages data plane functions interms of performing stream synchronization, buffering, anddata forwarding to mobile devices. In this manner, SDN pro-vides flexible management of the integrated cloudlet andbase stations as a value-added service without interruptingbasic operations of the base stations. Customization of theintegrated cloudlet and base stations to support a wide rangeof mobile users, applications, and services can be realizedthrough virtualization [10]. We elaborate on the use of SDNand virtualization of cloudlets/base stations in the followingsection, where we propose a functional layered architecturefor MCC.

Functional Layered Architecture for MCCIn Fig. 4, the layered architecture of MCC is presented. Eachlayer includes a set of functions and protocols. The operationat each layer is performed in three phases: establishment, acti-vation, and termination. Initial setup of a multimedia sessionbetween a user and MCC takes place during the establish-ment phase. After this setup, the multimedia data is trans-ferred during the activation phase. Finally, the teardown ofthe session is done in the termination phase. In the followingsection, we discuss each layer in detail.


Figure 2. Effective bandwidth requirements of a multimedia document.

Effe

ctiv

e ba

ndw

idth

prof

ile (

kb/s

)

Multimediadocumentbandwidth profile

AudioVideo

TextImage

T1 T2 T3 T4 T5 T6Time (s)


End User Layer

The end-user layer provides a graphical user interface (GUI)to facilitate direct user interaction with multimedia applica-tions. The end-user layer identifies the objects and theirQoP parameters by processing a user’s requests. In addition,it allows end users to upload new multimedia data, and mod-ify the relevant QoP parameters and authorization informa-tion. Such operations are managed through the cloudlet

associated with the base station where the mobile user initi-ated the session.

In a multimedia document, multimedia objects may havevarying bandwidth requirements, as shown in Fig. 2. It can benoticed that the overall bandwidth of the document and theresource requirements may change considerably over timedepending on the concurrency level of the objects. In order toensure QoP requirements, the underlying network, includingthe Internet and mobile networks, must dynamically allocate


Figure 3. Session handoff between cloudlets. The figure depicts three data centers, originally S1, S2, andS3 in B1 before migration. S’1, S’2, and S’3 are the migrated sessions after the user migrates to B2.º

Integratedcloudlet andbase station

Distributedmultimediadata centers Video

cloudImagecloud

S3

S3

S’3

S2

S2

S’2

S1

S1S’1

Buffers

VBM

B2B1

User

RFchannels

Audiocloud

Figure 4. Functional layered architecture for MCC.

Esta

blis

hmen

t

Act

ivat

ion

Term

inat

ion

End-user layer(user interaction, object identification)

Data center management and virtualization layer(address resolution, VDC configuration, data availability and integrity)

Session management layer(sessions, streams, creates ASR)

Network configuration layer(virtual channels, resources allocation)

Cloudlet configuration andmanagement layer(SDN data plane)

(synchronization, maintain ASR,buffering, data forwarding)

Base station configuration andmanagement layer(SDN control plane)

(paging, session handoff,RF channel allocation)


sufficient resources. The requirements can be specified interms of end-to-end peak or average bandwidth needed fortransferring the objects. The document model can provide thebandwidth requirements of each object. It can be noticed thatthe profile of each object is maintained by its local data cen-ter. Accordingly, at the time of session establishment thebandwidth profile of the entire multimedia document becomesavailable to the cloudlet. In MCC, this profile is used by thecloudlet to allocate resources efficiently to ensure the desiredQoP. An end user may establish multiple sessions with varyingbandwidth profiles. Effectively, the integrated cloudlet andbase station serves as an interface between the Internet andthe mobile users. We assume that the Internet (the networkbetween data centers and cloudlets) is resource-sufficient andhas enough resources to guarantee the QoP required by themultimedia services.

Data Center Management and Virtualization LayerData center management and the virtualization layer providesthe management functionality for the distributed objects andmaintains their location in terms of data center IDs. As dis-cussed earlier, virtualization allows resolution of logicaladdresses to physical locations where services are invoked bythe user. In essence, the service broker performs objectaddress resolution by identifying the specific VDC once themultimedia object is identified. The configuration, manage-ment, and mapping of VDCs to the physical data centers inthe cloud is achieved in this layer. Moreover, VM managesdata integrity and availability among VDCs.

Session Management Layer

Once the distributed objects are identified by the previouslayer, session management layer establishes the correspondingstreams from data centers. Each stream is identified by aunique stream ID. Multiple streams can form a multimediasession, which in turn is assigned with a distinct session ID.The session management layer creates a record of all activesessions. This record, called an active session record (ASR), isa table that has entries of active sessions’ IDs, stream IDs thatform the sessions, and the data centers’ IDs of the streams, asshown in Fig. 5. In addition, an ASR contains an entry for thebandwidth profile of the objects in the session. The manage-ment of sessions is controlled by this layer. When sessionhandoff takes place, the lower layer requests that the sessionmanagement layer perform two functions:• Terminate the streams and sessions supporting the migra-

tion process• Reestablish sessions to the migrated cells and reroute data

streams accordinglyThe details of the handoff procedure are discussed later inthis article.

Network Configuration LayerThe function of this layer is to establish and maintain virtualchannels over the Internet. The establishment phase deter-mines the routes for the virtual channels between data centersand cloudlets, and allocates sufficient resources based on theobject’s bandwidth profile and QoP requirements to ensure


Figure 5. Handoff signaling procedure and active sessions record.

Device ID

Session ID Stream ID Data center ID Bandwidthprofile

1 S1 D1 r1

1 S2 D2 r2

1 S3 D3 r3

2 S1 D1 r1

3 S1 D2 r1

ASR

B1

Multimediadata center

Handoffrequest

ASRrequest

Establishconnections

Connectionsready

AcknowledgementOverlappingperiod

Stopconnections

Tim

e

Connectionto B1 is

terminated

Multimediadata center

B2


timely delivery of multimedia data to the cloudlets. Overlaynetworks are used in this layer to enhance multimedia QoP by:• Discovering redundant paths between data centers and

cloudlets• Implementing routing policies that allow customized media

delivery [11]

Cloudlet and Base Station ConfigurationManagement LayerThe cloudlet and base station configuration and managementlayer has two components. The base station management layerworks in the SDN’s control plane by performing basic base sta-tion operations such as establishing and tearing down sessionswith mobile devices, paging, allocating RF channels, and initi-ating a handoff procedure, as depicted in Fig. 5. On the otherhand, cloudlet configuration and management layer handlesinter-stream and intra-stream synchronizations to ensure conti-nuity of presentation to the user. The function of this layeralso includes aggregating the bandwidth profiles, allocatingbuffers, and forwarding data to mobile users. To manage thehandoff process, the cloudlet configuration and managementlayer maintains the ASR of each mobile device. To allowseamless migration of sessions across cloudlets, the user can beprovided with a virtualized service abstraction by the MCC. Inthe following section, we present the details of the handoffprocedure and function of each layer in the handoff process aspart of the virtualization service provided by MCC.

Session HandoffA session is composed of one or more streams that may origi-nate from different data centers. Routes between data centersand mobile users may change with the movement of usersacross multiple cells, as depicted in Fig. 3. To ensure smoothdelivery of multimedia data to users, handoff and resourcereservation mechanisms can be used to establish new sessionsamong data centers and a migrating user, as depicted by S¢1,S¢2, and S¢3 sessions in Fig. 3. The handoff process is initiatedwhen a mobile device moves out of reach of the base stationinto the coverage of the neighboring base station. There aretwo types of handoff procedures: hard handoff and soft hand-off. In the case of hard handoff, there can be a short interrup-tion time in the delivery of data streams during the migration.The interruption occurs when the first session terminates at,say, SIUn, and the second session starts at SIUn+1. However,any noticeable discontinuity is not favored by the user. On theother hand, soft handoff can avoid loss of data by allowingpartial overlapping of the split SIUs. The overlapped data isthen clipped according to SIUs from both streams.

A handoff procedure is initiated when the received signallevel from the current base station, B1, drops below a certainthreshold. Subsequently, the mobile device identifies the basestation, B2, with the highest received signal level. The handoffprocedure signaling is depicted in Fig. 5. The mobile devicesends a handoff request to B2 that includes B1’s ID. Once therequest is received by B2, it communicates with B1 through thecloudlet configuration and management layer and sends anASR request that contains the ID of the requesting device. B1replies back with the requested ASR to B2, which in turnrequests the session management layer to initiate the requiredstream connections to the multimedia data centers. In case ofsoft handoff, the original streams to B1 are retained to avoidinterruption. B1 continues transmitting the streams to themobile device until streams are established between B2 anddata centers. At that time, B2 sends an acknowledgment to B1.At the same time, B2 starts transmitting data streams to the

mobile device. When B1 receives the acknowledgment, it stopstransmitting data streams to the mobile device. B1 requeststhe sessions and management layer to terminate the datastreams of the migrated device. The overlapping periodensures a soft handoff and therefore continuity in the session.

Resource Management Challenges forIntegrated Cloudlet and Base StationAs mentioned earlier, several resource management chal-lenges need to be addressed while designing MCC architec-ture of Fig. 1. In this section, we present challenges related tomanaging two key resources, buffers and bandwidth at theintegrated cloudlet and base station subsystem. Managementof the resources to satisfy the QoP requirements can be for-mulated as an optimization problem, as illustrated in the fol-lowing sections.

Virtualized Dynamic Buffer AllocationTemporal intra-stream synchronization needs to be preservedin order to present the multimedia information correctly. Forexample, video objects require a certain playout rate to ensurecontinuity in the presentation. Jitter delays in multimediastreams occur when packets experience different delays whiletraversing from the multimedia cloud data centers to cloudletsover the Internet. To avoid discontinuity in presentation atthe mobile devices, buffering by the cloudlet is used to com-pensate for jitter delays.

Inter-stream synchronization, on the other hand, preservesthe timing relationship among multiple multimedia streams.Inter-stream synchronization is required to deliver a coherentmultimedia document to the user. Multimedia streams flowalong different routes over the Internet and experience differ-ent delays. Therefore, buffering is required to ensure inter-stream synchronization. However, buffer underflow andoverflow can affect the QoP. Buffering underflow occurs whenthe session management layer transmits sessions in a just-in-time (JIT) manner. Streams in a JIT flow might experienceunexpected network delay, leading to a playout deadline beingmissed, while buffering overflow occurs when the sessionmanagement layer dumps sessions at full speed. One of thechallenging issues in this regard is to provide an upper andlower buffering bound to support a large number of sessions.

In order to achieve inter-stream and intra-stream synchro-nization, cloudlets maintain virtual buffers for active sessionsmanaged by a virtual buffer manager (VBM), as shown in Fig.1. Buffers provide temporary storage for multimedia objectscommunicated over the Internet in each session in order tosmooth jitter delays and facilitate inter-stream synchronizationof multimedia data. As multimedia objects can arrive at acloudlet ahead of their playout deadlines, they are buffereduntil playout time. In this case, buffer underflow is prevented.The VBM assigns free RF channels to the SIUs with loomingdeadlines. The assigned RF channel capacity may not be suffi-cient due to resource constraints, causing a rate mismatchbetween the arrival rate of SIUs from the servers and the out-bound transmission rate onto the mobile networks. The VBMcan dynamically allocate buffer to compensate for the rate dif-ference between the Internet and the mobile network in orderto avoid buffer overflow, which can lead to loss of data.

Dynamic RF Channel Capacity AllocationMultimedia information, such as high-definition video, is char-acterized by high-bandwidth data transfers. Consequently, themanagement of RF channels in mobile networks is a signifi-cant challenge. This resource management problem can be



posed as an optimization problem. For example, in RF chan-nel allocation, let us assume that O1

Si, O2Si, …, On

Si representthe concurrently transmitted objects in a document within ses-sion Si. Let the corresponding bandwidth requirements be r1

Si,r2

Si, ..., rnSi. Figure 2 shows that the aggregate bandwidth

requirements changes with time at random transition pointsT1, T2, and so on. These transition points are called resourceallocation decision points (RADPs) [7]. The cloudlet deter-mines the resource requirements at these transition points.

Let I be the time interval between two consecutive RADPs,j and j + 1. The aggregate bandwidth requirement RSi of theobjects in interval I for session Si is given by

(1)

Assuming multiple sessions are initiated by user, the aggre-gate bandwidth requirement R for that user is given by

(2)

If the channel capacity C in the base station is greater thanR at a given time interval, the bandwidth requirements ofindividual session are guaranteed. However, the base stationsmight not be able to satisfy the bandwidth requirements dueto the establishment of new connections. Then at least (R –C). |I| amount of information is dropped. Let di

Si denote thedropping ratio of object Oi

Si in a session Si, given by

(3)

Shafiq et al. present a fair channel allocation policy for a singlesession and formalize it as a nonlinear programming problem[7]. However, the optimization problem dealing with multiplesessions for all the users can be solved in order to allocate RFchannel resources to individual sessions and users.

Resource Management for Migratory SessionsMobile traffic load at base stations may vary dynamicallyaccording to a number of factors such as the varying aggregat-ed bandwidth requirements of a session, the number ofincoming sessions, and the number of migrated sessions fromneighboring base stations. New sessions might be rejected if

the channel capacity in the base station is not sufficient toaccommodate more sessions. A migrated session is treated asa new session request that invokes the RF channel assignmentprocedure at the base station. To avoid dropping migratedsessions, channels are reserved in advance at all prospectivebase stations the user is expected to visit during the lifetime ofthe session. This can be done by delivering the bandwidth pro-file of a multimedia document to base stations in advance.

A user’s mobility profile can be used to estimate the arrivaland departure time at each base station using information suchas size of base station, geographic location of each cell, andmaximum speed of mobile users. The probability density func-tion of the residency time TR is given in [12] under the assump-tion that the session’s duration is greater than the residency timein the base station’s cell with radius R. Moreover, the mobileuser is assumed to be traveling at a constant speed in the inter-val [0, Vmax]. The density function of the residency time TR inthe base station in which the session is initiated is given by [12]

(4)

In addition, the probability density function of the TR in abase station where a handoff occurs is given by [11]

(5)

Using this function, the arrival and departure time ofmobile users within a base station can be estimated based onthe residency time TR. Accordingly, a set of tuples consistingof the estimated residency time and the expected visited basestation IDs can be maintained for each mobile user during thelifetime of the session.

Performance Assessment of MCCArchitectureFor management of resources in the MCC architecture, vari-ous techniques can be implemented with varying degrees ofperformance [6, 7]. Several criteria can be used for assessment

π

π

=− − ⎛

⎝⎜⎞⎠⎟

⎧⎨⎪

⎩⎪

⎫⎬⎪

⎭⎪

⎡

⎣

⎢⎢⎢

⎤

⎦

⎥⎥⎥

≤ ≤

≥

⎧

⎨

⎪⎪⎪

⎩

⎪⎪⎪

f

R

V t

tV

Rt

R

V

R

V tt

R

V

8

31 1

2if 0

2

8

3if

2T t( )

max2

max2 3

max

max2

max

R

=f t f t( )3

2( )T forhandoff T( )R R

δ =O

O

number of SIUs dropped in

total number of SIUs in iS i

S

iS

ii

i

∑==

R rSxS

x

n

1

i i

∑

∑∑

=

=

∀

=∀

R R

r

S

S

xS

x

n

S 1

i

i

i

i


Figure 6. Performance assessment of MCC architecture.

N High

Low

δbQoS parameter bound

(a)

Num

ber

of s

atis

fied

use

rs

Com

plexity

N High

Low

b1Bandwidth

(b)

Num

ber

of s

atis

fied

use

rs

Com

plexity

H3

H2

H1

HC

HB

HA


of such techniques. For example, one key performance crite-ria, given in the previous section (Eqs. 2 and 3), is to estimatethe number of users (N) whose QoS requirements are satis-fied by the heuristics employed at the bottom layer of theMCC architecture in Fig. 4. The general performance behav-ior of the architecture for such criteria is depicted in Fig. 6.As shown in Fig. 6a, for a given heuristic, the parameter Ntends to increase as the QoS threshold increases. Such athreshold can be specified by the application. Here we use thedata dropping ratio db to illustrate this point. However,heuristics yielding higher performance generally entail highercomplexity (e.g., H1 vs. H3). Figure 6b depicts another perfor-mance assessment of various heuristics in terms of change inN with varying degrees of availability of resources. It is intu-itive that with the increase in the amount of resources (e.g.RF bandwidth), N also increases. Again, heuristics yieldinghigh performance tend to have a high complexity. Note thathigh-complexity heuristics may not be desirable for real-timemultimedia applications, resulting in a trade-off between theguaranteed QoS and the real-time performance of the relatedheuristics.

ConclusionIn this article, we have proposed a novel mobile cloud com-puting architecture for supporting mobile multimedia applica-tions and services in mobile networks. The key part of thisarchitecture is the integrated cloudlet and base station subsys-tem that provides a “close-to-the-user” proxy functionality andperforms dynamic allocation of resources. In addition, wehave presented a functional layered architecture that includesa set of functions and protocols to support multimedia appli-cations and services. We have also presented the connectionhandoff mechanism among cloudlets and its related chal-lenges. In addition, we have discussed prospective challengesin managing resources including buffer and RF channels.

References

[1] S. Zeadally, H. Moustafa, and F. Siddiqui, “Internet Protocol Television(IPTV): Architecture, Trends, and Challenges,” IEEE Sys. J., vol. 5, no. 4,2011, pp. 518–27.

[2] H. Dinh et al., “A Survey of Mobile Cloud Computing: Architecture, Applica-tions, and Approaches,” Wireless Commun. and Mobile Computing, 2011.

[3] L. Garber, “GPUs Go Mobile,” Computer, vol. 46, no. 2, Feb. 2013, pp.16–19.

[4] G. Lawton, “Cloud Streaming Brings Video to Mobile Devices,” Computer,vol. 45, no. 2, Feb. 2012, pp. 14–16.

[5] R. Xue, Z.-S. Wu, and A.-N. Bai, “Application of Cloud Storage in TrafficVideo Detection,” 7th Int’l. Conf. Computational Intelligence and Security,2011, pp. 1294–97.

[6] T. D. C. Little and A. Ghafoor, “Spatio-Temporal Composition of Distribut-ed Multimedia Objects for Value-Added Networks,” Computer, vol. 24,no. 10, 1991, pp. 42–50.

[7] B. Shafiq et al., “Wireless Network Resource Management for Web-BasedMultimedia Document Services,” IEEE Commun. Mag., vol. 41, no. 3,2003, pp. 138–45.

[8] E. Korotich and N. Samaan, “A Novel Architecture for Efficient Manage-ment of Multimedia-Service Clouds,” IEEE GLOBECOM Wksps., 2011,pp. 723–27.

[9] L. E. Li, Z. M. Mao, and J. Rexford, “Toward Software-Defined CellularNetworks,” Proc. 2012 Euro. Wksp. Software Defined Networking, 2012,pp. 7–12.

[10] M. Satyanarayanan et al., “The Case for VM-Based Cloudlets in MobileComputing,” IEEE Pervasive Computing, vol. 8, no. 4, 2009, pp. 14–23.

[11] M. Venkataraman and M. Chatterjee, “Quantifying Video-QoE Degrada-tions of Internet Links,” IEEE/ACM Trans. Net., vol. 20, no. 2, 2012, pp.396–407.

[12] D. Hong and S. Rappaport, “Traffic Model and Performance Analysis forCellular Mobile Radio Telephone Systems with Prioritized and Non-Priori-tized Handoff Procedures,” IEEE Trans. Vehic. Tech., vol. 35, no. 3,1986, pp. 77–92.

BiographiesMUHAMAD A. FELEMBAN ([email protected]) received a B.S degree incomputer engineering from King Fahd University of Petroleum and Minerals,Saudi Arabia, in 2008, and an M.S degree in computer science from KingAbdullah University of Science and Technology, Saudi Arabia, in 2011. He iscurrently working toward a Ph.D. degree in the School of Electrical and Com-puter Engineering at Purdue University. His research interests include datastreams management and underwater acoustic networks.

SALEH BASALAMAH is an associate professor at Umm Al-Qura University. Hehas an M.Sc. from the University of Bristol and a Ph.D. from Imperial CollegeLondon. His research interests include computer vision and multimedia.

ARIF GHAFOOR [F] is a professor in the School of Electrical and Computer Engi-neering at Purdue University. His research interests include multimedia informa-tion systems, database security, and distributed computing.




he emergence of cloud computing [1] has been dra-matically changing the landscape of services for mod-ern computer applications. Through the Internet, itenables end users to conveniently access computing

infrastructure, platforms, and software provided by remotecloud providers (e.g., Amazon, Google, and Microsoft) in apay-as-you-go manner or with long-term lease contracts. Thisnew generation of computing paradigm, offering reliable, elas-tic, and cost-effective resource provisioning, can significantlymitigate the overhead for enterprises to construct and main-tain their own computing, storage, and network infrastruc-tures. With the aid of cloud resources, startup companies caneasily implement their ideas into real products with minimuminvestment in the initial stage and expand the system scalewithout much effort later on. A representative is Dropbox, atypical cloud storage and file synchronization service provider,which largely relies on Amazon’s S3 servers for file storageand leverages Amazon’s EC2 instances to provide such keyfunctions as synchronization and collaboration among differ-ent users. The existing content/service providers can alsomigrate their legacy applications to cloud platforms. Forexample, video on demand (VoD) providers (e.g., NetFlix),have leveraged cloud resources to handle burst traffic. Theypay by bytes for bandwidth and storage resources so that thelong-term costs become much lower than those with overpro-visioning in self-owned servers.

Mobile terminals, including smartphones and tablets, areincreasingly penetrating into people’s everyday lives as an effi-cient and convenient tool for communication and entertain-ment. With the advancements in computing, storage, andnetwork technologies, the diversity of applications on suchhandheld devices is now comparable to their counterparts ontraditional desktop PCs. The touch screen and all kinds ofsensors provide even richer user experiences that have yet tobe available on desktop PCs. Despite the fast development ofsuch key components as CPU, GPU, memory, and wireless

access technologies, the skyrocketing growth of market pene-tration, and the effort towards unifying handheld and desktopcomputers (e.g., through Windows 8), it remains widelyagreed that mobile terminals will not completely replace lap-top and desktop computers in the near future. Migrating pop-ular PC software to mobile platforms or developing similarsubstitutes for them is still confined by their limited computa-tion capability as well as the uniqueness of operating systemsand hardware architectures. To make it even worse, battery,as the only power source of most mobile terminals, has seenrelatively slow improvement in the past decade. Battery capac-ity is growing only 5 percent annually [2], which has become amajor impediment to providing reliable and sophisticatedmobile applications to meet user demands.

Empowered by cloud computing, both academic researchersand industrial pioneers have been striving to extend the capa-bilities of mobile terminals with cloud resources. Mobile cloudcomputing, which combines the strength of clouds and theconvenience of mobile terminals, naturally attracts tremen-dous attention. To date, cloud storage and synchronizationapplications, represented by Dropbox and iCloud, have beenembraced by most mobile platforms. In Apple’s popular Siri,after a piece of voice is recorded by an iPhone, a local recog-nizer will conduct speech recognition and decide whether toresort to the back-end clouds to make an appropriateresponse. Gaikai and Onlive further provide generic web-based platforms for distributed interactive applicationsthrough shifting the hardware/software requirements as wellas the excessive computing loads to cloud proxies. As such,mobile users can enjoy high-quality video games without per-forming the computation-intensive image rendering locally.The discrepancy among different operating systems can alsobe masked by such standard web development tools asHTML5, Flash, and JavaScript. Boosting resource-limitedusers to play the games that used to be exclusively for high-end PCs and gaming consoles, Gaikai has achieved great suc-

T


AbstractThe emergence of cloud computing has been dramatically changing the landscapeof services for modern computer applications. Offloading computation to the cloudeffectively expands the usability of mobile terminals beyond their physical limits,and also greatly extends their battery charging intervals through potential energysavings. In this article, we present an overview of computation offloading in mobilecloud computing. We identify the key issues in developing new applications thateffectively leverage cloud resources for computation-intensive modules, or migratingsuch modules in existing applications to the mobile cloud. We then analyze tworepresentative applications in detail from both the macro and micro perspectives,cloud-assisted distributed interactive mobile applications and cloud-assisted motionestimation for mobile video compression, to illustrate the unique challenges, bene-fit, and implementation of computation offloading in mobile cloud computing. Wefinally summarize the lessons learned and present potential future avenues.

When Mobile Terminals Meet the Cloud:Computation Offloading as the BridgeXiaoqiang Ma, Yuan Zhao, Lei Zhang, and Haiyang Wang, Simon Fraser University

Limei Peng, SooChow University

T

0890-8044/13/$25.00 © 2013 IEEE

MA_LAYOUT_Layout 1 9/20/13 12:55 PM Page 28

cess since launching in 2008. It reached 10 million monthlyservice users by late 2011 and targeted 100 million by 2012.

Such computation offloading effectively expands the usabili-ty of mobile terminals beyond their physical limits, and alsogreatly extends their battery charging intervals through poten-tial energy savings [3]. It serves as a key driving force of theconsumer and enterprise market for cloud-based mobileapplications, which is expected to reach $9.5 billion by 2014[4]. Intuitively, offloading is beneficial whenever a computa-tion-intensive task is not affordable by local resources. How-ever, the boundaries of the modules of different computationdemands are not always clear for implementing offloading.Even worse, moving the task to the remote cloud may incur alarge volume of data transfer too, which, although it may notbe a severe problem for users with high-speed wired networkconnections, can largely contradict the benefit for mobileusers with their energy-hungry wireless interfaces [5].

In this article, we present an overview of computationoffloading in mobile cloud computing. We identify the keyissues in developing new applications that effectively leveragecloud resources for computation-intensive modules, or migrat-ing such modules in existing applications to the mobile cloud.We then analyze two representative applications in detail,cloud-assisted distributed interactive mobile applications andcloud-assisted motion estimation for mobile video compres-sion, to illustrate the benefit, implementation, and uniquechallenges of computation offloading in mobile cloud comput-ing. We finally summarize the lessons learned and presentpotential future avenues.

Computation Offloading in Mobile CloudComputingCloud computing, which is readily accessible for mobileterminals with built-in wireless interfaces, is a natural solu-tion to augment the capabilities of mobile platforms at low

cost. The term mobile cloud computing was introduced notlong after the advent of cloud computing around 2007 [6].With the widespread penetration of third-/fourth-genera-tion (3G/4G) wireless networks, it offers a promising andviable solution that narrows the gap between the everincreasing demands of new applications and the limitedresources on even the latest mobile terminals, and hassince attracted significant attention from both academiaand industry.

Mobile Cloud Computing: An OverviewAccording to the Mobile Cloud Computing Forum, mobilecloud computing is defined as follows [7]:

“Mobile Cloud Computing at its simplest, refers to an infras-tructure where both the data storage and the data processinghappen outside of the mobile terminal. Mobile cloud applica-tions move the computing power and data storage away frommobile phones and into the cloud, bringing applications andmobile computing to not just smartphone users but a muchbroader range of mobile subscribers.”

Figure 1 shows a conceptual architecture that reflects theabove definitions of mobile cloud computing. The mobile ter-minals access the Internet via WiFi or cellular networks, andcoordinate with application servers to locally decide on theoffloading strategy. Then mobile terminals will offload thetasks to the cloud accordingly. Upon receiving the requestsfrom mobile terminals or mobile application servers, the cloudcontrollers will schedule the tasks on virtual machines, whichare rented by application service providers, and send back theresults. In some occasions, the application servers can also bedeployed in the cloud.

There have been plenty of previous works on mobile cloudcomputing [6, 8]. They suggest that mobile cloud computinginherits the salient benefits of general cloud computing. Inparticular, dynamic provisioning allows application providers


Figure 1. Architecture of mobile cloud computing.

Cloud provider A

Public cloud provider

Cloudcontroller

VMs

Mobile application server

Cloud provider B

Cloudcontroller

VMs

Internet

Accesspoint

Basestation

Mobile network B

Mobileterminals

Accesspoint

Basestation

Mobile network A

Mobileterminals


to handle the variation of service burden in a flexible andfine-grained way rather than overprovisioning in advance,which is a waste of money and resources. Applicationproviders can easily expand the system scale to meet increas-ing user demands. They can also integrate different servicesthrough the cloud for efficient management. Virtualization, akey feature in cloud computing, provides isolation and protec-tion for individual virtual machines, which significantlyimproves system reliability. It also offers higher utilization ofinfrastructure so that the running costs can be largely reduced.

Computation OffloadingDespite the similarities, mobile cloud computing is uniquewith wireless communications and mobile terminals. Themobile terminals truly enable anywhere and anytime computa-tion for any person. However, compared to their wired coun-terparts, mobile terminals are generally moreresource-constrained; in particular, the wireless communica-tion capacity and battery capacity are their inherent bottle-necks [3, 6]. As such, assembling local resources and remoteclouds organically to make offloading transparent to mobileusers requires nontrivial effort.

First, the key motivation of offloading for a mobile applica-tion must be determined: to save energy, to improve computa-tion performance, or both? This serves as a guideline for thefollowing system design.

Second, the potential offloading gain needs to be wellunderstood. There is no incentive to resort to clouds for a jobthat can easily and efficiently be executed locally. Even forone that can hardly be executed locally, moving it to theremote cloud may incur a large volume of data transfer,which, although it may not be a severe problem for users withhigh-speed wired network connections, can largely contradictthe benefit for mobile users with their energy-hungry wirelessinterfaces.

With such wireless data communication cost in considera-tion, often offloading the whole computation module of anapplication to the remote cloud is not necessary or effective.A profiling or breakdown analysis is needed to reveal thecomputation costs of different modules of the application. Butas we show later, the boundaries of these modules are notalways clear for implementing offloading. Even identified, thetrade-off depends on the specific application considered, andoptimal partitioning is known to be hard [9]. Furthermore, thecritical quality of service (QoS) requirements (e.g., latency,image/video quality, computation accuracy) should be satis-fied. As we show later, despite its benefit, offloading canadversely affect certain QoS measurements the impact ofwhich must be well considered.

In recent years, a substantial number of practical systemsand prototypes have been implemented, addressing the issueslisted above. For instance, MAUI enables fine-grained ener-gy-aware offloading of mobile codes to a cloud based on ahistory of energy consumption [11]. It achieves maximumenergy savings of 90, 45, and 27, and maximum performancespeedups of roughly 9.5, 1.5, and 2.5 for face recognition,chess, and video gaming, respectively. CloneCloud uses func-tion inputs and an offline model of runtime costs to dynami-cally partition applications between a weak device and thecloud [12]. It reports maximum speedups of 14.05, 21.2, and12.43 for virus scanning, image search, and behavior profil-ing, respectively.

Miettinen and Nurminen [13] provide an analysis of the criti-cal factors affecting the energy consumption of mobile clients incloud computing and show that the energy trade-offs heavilydepend on the workload characteristics, data communicationpatterns and technologies used. We next provide case studies to

closely examine the service partitioning in the mobile comput-ing context for offloading complex computation tasks.

Cloud as Computation Proxies forOffloadingWe start from a generic scenario that involves multiple mobileusers interacting with each other, which is common in manymodern applications (e.g., online gaming). In the traditionalclient-server model, almost all the complex tasks like real-timerendering for synthesized multiplayer scenes are completedlocally to relieve the burdens of the servers. To this end, thesedistributed interactive applications (DIAs) literally imposepeculiar hardware/software requirements on user consoles. Asan example, the recommended system configuration for Battle-field 3, a highly popular first-person shooter game, is a quad-core CPU with 4 Gbyte RAM, 20 Gbyte storage space, and agraphics card with at least 1 Gbyte RAM (e.g., NVIDIAGeForce GTX 560 or ATI Radeon 6950),1 which alone costsmore than $500. The newest tablets (e.g., Apple’s iPad withRetina display and Google’s Nexus 10) cannot even meet theminimum system requirements of a dual-core CPU over 2.4GHz, 2 GB RAM, and a graphics card with 512 MB RAM, notto mention smartphones, the hardware of which is limited bytheir smaller size and thermal control. Furthermore, mobileterminals have different hardware/software architectures fromPCs, such as ARM rather than x86 for CPU, lower memory fre-quency and bandwidth, power limitation, and distinct operatingsystems. As such, the traditional model, which has achievedgreat success in the traditional PC market, is not quite feasiblefor mobile terminals, which has largely hindered the penetra-tion of high-quality interactive games to mobile platforms, notto mention fantastic 3D games in the near future.

Recent pioneers, represented by Onlive and Gaikai, howev-er, have paved another avenue toward making effective use ofoffloading. These cloud-based distributed interactive applica-tions (CDIAs) allow multiple participants at different loca-tions to interact with each other. Different from traditionalDIAs, the actual game clients/consoles are deployed on cloudservers, and only the game screen and interactions arestreamed back to end users, which is particularly suitable formobile users.

To understand how CDIAs work in detail, we now useGaikai as a case study. We illustrate Gaikai’s underlyingframework in Fig. 2. When a user selects a game on Gaikai(step 1 in Fig. 2), an EC2 virtual machine will first deliverthe Gaikai game client to the user (step 2). After that, itforwards the IP addresses of game proxies that are ready torun the selected games to the user (step 3). The user willthen select one game proxy to run the game (step 4). Afterthat, the game proxy starts to run the game and the gamescreen will be streamed to the user via UDP (steps 5 and 6).For multi-player online games, these game proxies will alsoforward user operations to game servers (mostly deployedby the game companies) and send the relatedinformation/reactions back to the users (step 7). Such aCDIA implementation can remarkably relieve the hardwareand software requirements on the user side. The games arerun on cloud platforms with the game screen streamed tothe end users. This change enables users to play hardcoregames over less powerful devices (e.g., smartphones, tablets,and even digital TVs) as long as they are multimedia- andnetwork-ready.


1 Battlefield 3 faq. http://www.battlefield.com/battlefield3/1/bf3-faq.


http://www.battlefield.com/battlefield3/1/bf3-faq

Service Partitioning for ComputationOffloading

So far we assume that most complex tasks are offloaded toclouds. However, in many applications, this approach is notefficient or feasible, and it is necessary to partition the appli-cation at a finer granularity into local and remote parts, whichis a key step for offloading. As mentioned earlier, even if theapplication can be perfectly decomposed into relatively inde-pendent modules, and all the information is available, includ-ing mobile terminal and cloud capabilities, it is still not aneasy job to obtain an optimal partitioning. It would becomemore complex if the intrinsic QoS requirements or constraintsof applications (e.g., real-time or bandwidth requirements) areconsidered. Furthermore, the modules of some applicationshave inherent dependence on each other, which further com-plicates the partitioning.

We now use video encoding, an essential task in a broadspectrum of mobile applications, as a case study for partition-ing with data dependence. A mobile user uses his/her mobileterminal to capture video in real time, expecting to encodethe video and then stream it to others in real time as well.Directly uploading the raw video without efficient encodinginevitably leads to high bandwidth cost and large transmissionenergy consumption. On the other hand, video encoding oftenincurs heavy computation, which results in high energy con-sumption as well. For example, to encode a video of 5 s (30frames/s with resolution of 176 * 144 and pixel depth of 8bits) using an H.264 encoder needs almost 1 × 1010 CPUcycles [14], or 2 × 109 CPU cycles/s on average,which means that a 2 GHz CPU is required forreal-time encoding. Considering that the newestsmartphones and tablets are equipped with high-definition cameras, the CPU workload can be 5–10times higher than that in the above example.

Offloading the whole video compression task tothe cloud, however, is not practical because it isidentical to directly uploading the raw video data.The wireless transmission can be either too costlyor simply impossible with limited bandwidth. Aprofiling shows that motion estimation is the mostcomputation-intensive module, accounting foralmost 90 percent of the computation. While thismodule obviously should be the focus of offloading,it is not simple to decouple it from others given thedata dependence; that is, motion estimation of aframe depends on the data of the previous refer-ence frame. It is necessary to ensure that a mini-mum amount of data (not all the reference frame

data) are to be uploaded to the cloud and that estimation canstill be done accurately, which leads to the design of Cloud-Assisted Motion Estimation (CAME) [15]. CAME employs amesh-based motion estimation that synergizes the mobile ter-minal and the cloud. With CAME, a mobile terminal canupload reference frames and mesh data to the cloud for esti-mation (mesh node motion estimation), which are of muchsmaller data volume. It then downloads the estimated motionvectors (MVs) from the cloud server and completes theremaining video encoding steps (sub-block motion estima-tion). The CAME architecture is illustrated in Fig. 3. Thereare three transmission phases in CAME: initial referenceframe and mesh node data uploading, mesh node MV down-loading, and compressed video data uploading.

Figure 4a compares the total amount of transmitted datafor all three standard videos (Foreman, Mother, and Flower).The baseline here is All on Mobile (AoM), which executes theentire video encoding on a mobile terminal, and the transmis-sion energy consumption is converted into CPU cycles so thatthe total energy consumption can be quantified as well.Although Flower’s original video size is the smallest, bothAoM and CAME incur the highest transmission cost com-pared to the other two videos, because Flower has higher spa-tial details. It is not surprising that the transmission cost ofAoM is the lowest of the three, and raw uploading has thelargest cost. Compared to AoM, CAME introduces moretransmission because of the extra data transmission for meshnode uploading and mesh MV downloading. On the otherhand, compared to raw uploading, the CAME method stillsaves approximately 60 percent on total data transmission.

Although CAME consumes more energy on transmissionthan AoM does, it saves on total energy consumption throughoffloading the most computation-intensive task, motion esti-mation, to cloud servers. It spends nearly 40 percent less ener-gy on computation than AoM. Furthermore, Fig. 4b confirmsthe expectation that CAME can achieve up to 30 percent totalenergy savings on video encoding and transmission comparedto AoM.

Further DiscussionOur case studies in both macro and micro scopes have demon-strated that utilizing cloud resources provides great opportuni-ties to reduce the computation overhead on mobile terminals,which leads to considerable energy savings and thus longerbattery life. Cloud can simply take over the whole computa-tion task and send back the results in a mobile-friendly form,


Figure 2. Basic framework of Gaikai.

EC2 cloud

Game clientdelivery

Tracking proxyinfo

Load-balancers

Limelight cloud

Video encoder Game console

Cloud proxy

7

5

3

6

4

21

Gameservers

Userclients

Figure 3. CAME architecture.

Anchorframe

Cloud server

Cloud server

Predictive frames

Compressed video

Motion vectors


or handle the most computation-intensive subroutine throughsophisticated application partitioning. Although it is verytempting and promising to leverage the much cheaper andmore powerful resources on the cloud, the interaction betweena mobile terminal and the cloud needs careful examination toavoid excessive transmission overhead. As such, partitioningtailored to specific applications is often expected, as ourexample of motion estimation shows. The trade-off betweenthe energy for computation and that for transmission can befound in many other applications that rely on computationoffloading to extend battery lifetime. As part of our futurework, we will consider the coordination of wireless transmis-sion between the cloud and mobile devices, and the schedul-ing of offloading when the mobile device is moving.

One drawback for CAME and similar cloud-assisted solu-tions is the increased encoding delay introduced by the closedloop design (between the mobile user and the cloud). For asingle user, this delay is relatively small compared to the timeneeded for encoding locally. With multi-user interaction thatinvolves a series of cloud proxies, as in general CDIA, thedelay can be further amplified given the extra number of hopsalong an interactive path in the overlay network of proxies.We illustrate the interaction paths in both conventional DIAand CDIA architectures in Fig. 5, where L is the set of clients,S the set of service servers operated by CDIA providers, andC the set of cloud-based proxies. We can see that the path

between the two clients in CDIA is longer than that in DIA.For example, there are three hops between clients 1 and 4 inDIA (dotted lines), but five hops in CDIA (solid lines). Intu-itively, this would lead to increased user interactive latency inCDIA. We have carried out a real-world experiment usingPlanetLab. We select 588 PlanetLab nodes to run as CDIAclients to connect Gaikai’s cloud proxies. We have found 28Gaikai cloud proxies during the measurement process. Wemeasure the round-trip times (RTTs) between the clients andGaikai cloud proxies, as well as the RTTs between the serversand the cloud proxies. The sum of these two latencies can beused to calculate the client-server RTTs in the CDIA system.For comparison, we also measure the direct RTTs betweenthe servers and the PlanetLab clients, as in the traditionalDIA system. We plot the cumulative distribution function(CDF) of latency in different situations in Fig. 6. We can seethat most (over 80 percent) users in DIA have quite low inter-action latency, say, less than 60 ms. However, the averagelatency is much longer in CDIA, and 90 percent of users havean interaction latency over 200 ms in the worst case, which isalmost intolerable for smooth interaction.

In practice, however, adding extra nodes will not necessari-ly lead to longer path latency since triangle inequality doesnot always hold in the Internet. Cloud providers often have


Figure 4. Simulation results: a) total transmission volume (MB); b) total energy consumption in CPU cycles (billion cycles).

(a)Foreman

Tota

l tra

nsm

issi

on v

olum

e (M

B)

Tota

l ene

ry c

onsu

mpt

ion

in C

PU c

ycle

s (b

illio

n)

5

0Mother Flower

10

15

20

25

30

35

40

45

50

(b)Foreman

2

0Mother Flower

4

6

8

10

12

14

16AoMCAME

AoMRaw uploadingCAME

Figure 5. Path of client interaction.

S3 S4S2S1

L3 L4L2L1

C1 C2 C3 C4 C5

Interaction path in CDIA

Interaction path in DIA

Figure 6. Latency between user and server (DIA vs CDIA).

1000

0.2

0

0.4

0.6

0.8

1

200 300

RTT (ms)2000

0.1

0

CD

F

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

400 600 800 1000 1200 1400 1600

Client-server RTT of CDIA (best case)Client-server RTT of CDIA (worst case)Client-server RTT of DIA


better networks (e.g., near to the backbone and higher band-width) and better scalability than application servers. Thecloud providers also deploy their data centers at differentlocations to further reduce the latency. If the cloud proxiesand application servers are smartly assigned to user clients,the latency will not change a lot and may become even shorteron many occasions. As shown in Fig. 6, the maximum clientinteraction latency can be reduced by nearly 30 percent. Inour future work, we plan to investigate the user experience ofCDIA, especially whether image quality incurs significantdegradation. We also plan to examine the scalability of CDIA,for example, how many clients a server can support.

ConclusionIn this article, we have presented an overview of computationoffloading in mobile cloud computing, a new paradigm com-bining increasingly popular mobile terminals and powerfulcloud computing to provide seamless rich experiences tomobile users. This new paradigm brings opportunities as wellas challenges. New applications become viable, while existingones need careful redesign. We have used two case studies toillustrate how mobile applications can be enhanced with cloudengines to achieve improved performance or energy saving,and have examined trade-offs therein.

Although still in its infancy, mobile cloud computing pro-vides a promising model that could fundamentally reshapemobile applications in the future. As an interdisciplinary area, itfuses various areas such as mobile computing, cloud computing,networking, and program partitioning. Other kinds of mobileterminals, including wireless sensor nodes and RFID tags,could also become important components for which unique ser-vice partitioning and interactions are to be examined.

References[1] M. Armbrust et al., “A View of Cloud Computing,” Commun. ACM, vol.

53, no. 4, Apr. 2010, pp. 50–58. [2] S. Robinson, “Cellphone Energy Gap: Desperately Seeking Solutions,”

tech. rep., Strategy Analytics, 2009. [3] K. Kumar et al., “A Survey of Computation Offloading for Mobile Sys-

tems,” Mobile Networks and Applications, 2012, pp. 1–12. [4] S. Perez, “Mobile Cloud Computing: $9.5 Billion by 2014,” http://exo-

planet.eu/catalog.php, 2010. [5] K. Kumar and Y.-H. Lu, “Cloud Computing for Mobile Users: Can Offload-

ing Computation Save Energy?,” Computer, vol. 43, no. 4, Apr. 2010,pp. 51–56.

[6] H.T. Dinh et al., “A Survey of Mobile Cloud Computing: Architecture,Applications, and Approaches,” Wireless Commun. and Mobile Comput-ing, 2011.

[7] http://www.mobilecloudcomputingforum.com/. [8] N. Fernando, S. W. Loke, and W. Rahayu, “Mobile Cloud Computing: A Sur-

vey,” Future Gen. Computer Sys., vol. 29, no. 1, Jan. 2013, pp. 84–106.[9] Z. Li, C. Wang, and R. Xu, “Computation Offloading to Save Energy on

Handheld Devices: A Partition Scheme,” Proc. ACM CASES ’01, 2001,pp. 238–46.

[10] B. Hendrickson and R. Leland, “A Multilevel Algorithm for PartitioningGraphs,” Proc. ACM/IEEE Supercomputing ’95, 1995, pp. 28–41.

[11] E. Cuervo et al., “MAUI: Making Smartphones Last Longer with CodeOffload,” Proc. ACM Mobisys ’10, 2010, pp. 49–62.

[12] B.-G. Chun et al., “CloneCloud: Elastic Execution Between Mobile Deviceand Cloud,” Proc. EuroSys ’11, 2011, pp. 301–14.

[13] A. P. Miettinen and J. K. Nurminen, “Energy Efciency of Mobile Clientsin Cloud Computing,” Proc. USENIX HotCloud ’10, 2010.

[14] N. Imran, B.-C. Seet, and A. C. M. Fong, “A Comparative Analysis ofVideo Codecs for Multihop Wireless Video Sensor Networks,” MultimediaSys., vol. 12, 2012, pp. 373–89.

[15] Y. Zhao et al., “CAME: Cloud-Assisted Motion Estimation for MobileVideo Compression and Transmission,” Proc. ACM NOSSDAV ’12, 2012.

BiographiesXIAOQIANG MA ([email protected]) received his B.Eng degree from HuazhongUniversity of Science and Technology, China, in 2010, and his M.Sc. degreefrom Simon Fraser University, Canada, in 2012. He is now a Ph.D. student inthe School of Computing Science, Simon Fraser University, Canada. His areasof interest are wireless networks, social networks, and cloud computing.

YUAN ZHAO ([email protected]) received his B. Eng. degree from BeihangUniversity, China, in 2006, and his M.Sc. degree from Simon Fraser Universi-ty in 2013. He was a software engineer at the IBM China Development Labo-ratory. He is now a Ph.D. student in the School of Computing Science, SimonFraser University. His areas of research interest are multimedia communica-tions, social networking, and cloud computing.

LEI ZHANG ([email protected]) received his B. Eng. degree from Huazhong Uni-versity of Science and Technology, China, in 2011. He is now a Master’s stu-dent in the School of Computing Science, Simon Fraser University. His areasof research interest are multimedia communications, wireless networks, andcloud computing.

HAIYANG WANG ([email protected]) is currently a Ph.D. student in the Schoolof Computing Science, Simon Fraser University. He is working in the Multime-dia and Wireless Networking Group, and his research interests include cloudcomputing, peer-to-peer networks, multimedia systems/networks, IP routing,and QoS.

LIMEI PENG ([email protected]) received her B.S degree from the SouthCentral University for Nationalities in Wuhan, China, in 2004, and her M.S.and Ph.D. degrees from the Chonbuk National University in Jeonju, Chonbuk,South Korea, in 2006 and 2010, respectively. She has worked as a post-doc-toral fellow at the Grid Middleware Research Center, Korea Advanced Insti-tute of Science and Technology, South Korea. She is now an associateprofessor at the School of Electronic and Information Engineering, SooChowUniversity, P.R. China. Her research interests include optical communicationnetworks and protocols, data center networks, optical fiber sensor networks,and cloud computing networks.



http://www.mobilecloudcomputingforum.com/






http://exoplanet.eu/catalog.php

he tussle between resource-hungry applications andresource-poor smartphones is driving the evolution ofmobile application platforms. Recently, there hasbeen an explosion of mobile applications on smart-

phones. Many of these applications are computation-intensive,such as video streaming, data mining, and online gaming.However, these emerging applications are impeded by resourceconstraints on smartphones. First, smartphones are equippedwith a limited battery system that has become one of thebiggest complaints by users [1]. Second, smartphones still lagbehind their desktop counterparts in terms of computingpower and memory capacity. In addition, network connectivityis sporadic because of fading effects in the wireless channel.

Application offloading was proposed as an effectivescheme to address this tussle [2]. With the advent of cloudcomputing [3, 4], two approaches have been investigated forapplication offloading. The first approach is to offload anapplication to an infrastructure cloud for execution. In thiscase, each smartphone is associated with a system-level cloneor a delegated surrogate on the cloud, such as Cloudlet [5],Clone Cloud [6], and Weblet [7], which executes applicationson behalf of the smartphone. However, network connectivityis not always available. In addition, with the increasingrequests for application offloading to the cloud, communica-tion at base stations or access points could become the bot-tleneck. The second approach is to offload the application toa group of proximal smartphones [8]. These smartphones,connected to each other by a wireless radio, cooperatively

execute the application and can be viewed as a virtual cloudcomputing environment. However, this ad hoc virtual cloudcannot entertain computation-intensive mobile applicationsdue to limited aggregated onboard battery systems and com-puting resources on smartphones. Therefore, neither ofthese two approaches can achieve high scalability of smart-phones.

In this article, we propose a unified elastic computing plat-form by combining the ad hoc virtual cloud and infrastructure-based cloud for higher scalability. The infrastructure-basedcloud is empowered by execution engines and cloud clones.The ad hoc virtual cloud is formed by the cooperation ofsmartphones within the same coverage range. With the com-bined fabric of the infrastructure-based cloud and the ad hocvirtual cloud in our elastic computing platform, applicationoffloading can be conducted more efficiently.

Under the elastic computing platform, we first present thedecision-making policy of application offloading (i.e., offload-ing policy). The offloading policy determines each task of theapplication to be executed on the standalone smartphone oroffloaded to the cloud for execution. We build a directedacyclic graph model to represent the task execution of mobileapplication and define an optimization framework for theoffloading policy. In particular, we investigate four specialcases (i.e., a node, a linear chain, a tree, and a mesh in thegraph) and obtain the offloading policy for each case. We alsoinvestigate two implementation strategies for an offloadingmechanism, system-level and method-level offloading. In addi-

T


AbstractApplication offloading has been a popular approach to alleviate a tussle betweenresource-constrained smartphones and resource-hungry mobile applications. In thisarticle, for leveraging cloud computing, we propose a unified elastic computingplatform that supports application offloading for mobile devices, reducing energyconsumption on smartphones. The proposed computing fabric consists of an infra -structure-based cloud and an ad hoc virtual cloud formed by a cluster of smart-phones. We present both an offloading policy and a mechanism under whichapplications are delegated to the cloud for execution. For the former, we establisha unified optimization framework to decide where each task of the applicationshould be executed — on the standalone smartphone, in the ad hoc virtual cloud,or in the infrastructure-based cloud. For the latter, we provide implementationstrategies for application offloading. The proposed elastic computing platform canenhance the scalability of smartphones, fueling a new wave of innovative mobileapplications, for example, anti-virus and gaming on smartphones.

Toward a Unified Elastic ComputingPlatform for Smartphones with

Cloud SupportWeiwen Zhang and Yonggang Wen, Nanyang Technological University

Jun Wu, Tongji UniversityHui Li, Sichuan University

T

0890-8044/13/$25.00 © 2013 IEEE

ZHANG_LAYOUT_Layout 1 9/20/13 12:57 PM Page 34

tion, we study opportunities (i.e.,task delegation, cloud clone peer-to-peer [P2P] network, data back-up, and data staging) andchallenges (i.e., performance, secu-rity and energy issues) in this plat-form.

The unified elastic computingplatform can enhance the scalabilityof smartphones, fueling a new waveof innovative mobile applications(e.g., anti-virus and mobile cloudgaming).

The rest of the article is orga-nized as follows. We present anoverview of the unified elastic com-puting platform. We propose theoffloading policy in this computingplatform. We present implementa-tion strategies for the offloadingmechanism. We discuss opportuni-ties and challenges of the comput-ing platform. We highlight twomobile applications that can benefitfrom the computing platform.Finally, we summarize the articleand suggest future work .

An Overview of the Unified ElasticComputing PlatformThis section presents an overview of the unified elastic com-puting platform. We first propose a generic architecture ofthe platform and then present execution strategies of mobileapplications under the platform.

Generic Architecture of the Elastic Computing PlatformFigure 1 illustrates a generic architecture of the elastic com-puting platform, which consists of an ad hoc virtual cloud andan infrastructure-based cloud.

The infrastructure-based cloud is empowered by cloud clonesand remote execution engines, which extends the computingpower and reduces the energy consumption of smartphones. Inthe infrastructure-based cloud, there is an identical image ofthe system for each smartphone, which is referred to as thecloud clone. The cloud clone executes mobile applications onbehalf of the smartphone, thus reducing application delay andenergy consumption on the smartphone. The cloud clones arelogically connected, forming a cloud clone P2P network. Thereare also execution engines and data storage in the back-endthat open up more opportunities for application offloading.

The ad hoc virtual cloud is formed by a cluster of smart-phones nearby that work cooperatively to accomplish applica-tion offloading. A smartphone communicates with itsneighbors directly by a local wireless network interface (e.g.,Bluetooth). As the smartphone moves from one environmentto another, it will join a new cluster of smartphones and canstill benefit from application offloading seamlessly. As aresult, the ad hoc virtual cloud copes with the issue of spo-radic wireless network connectivity between the smartphoneand the infrastructure-based cloud.

In this elastic computing platform, the infrastructure-basedcloud and the ad hoc virtual cloud complement each other toaddress the issues of limited battery power on the smartphoneand sporadic network connectivity. Hence, the elastic comput-ing platform enhances the scalability of smartphones.

Execution Strategies under Elastic Computing PlatformIn this platform, computing resources, including any cloudclone and smartphone, are elastically allocated to the execu-tion of mobile applications. The application execution can bemade among three execution strategies, including:• Standalone execution by the individual smartphone• Cooperative execution by the cluster of smartphones• Cloud execution by cloud clone or execution engine

The standalone execution requires the computation of theapplication to be completed on the individual smartphone.The cooperative execution and the cloud execution consumecomputing resources in the cloud by application offloading viaa wireless network.

The Offloading Policy for the Unified ElasticComputing PlatformIn this section, we propose an optimization framework for theoffloading policy in the unified elastic computing platform.The offloading policy is to determine which execution strategy(i.e., standalone execution, cooperative execution, or cloudexecution) should be chosen for each task of the application.We first present an optimization framework, and then investi-gate four special cases (i.e., a node, a linear topology, a tree,and a mesh) of offloading policy.

The Optimization Framework of the Offloading PolicyWe construct a general directed acyclic graph model to repre-sent the task flow in the application, referred to as the task-flow graph. In the graph, a node represents a task, and a linkconnecting two nodes represents data dependence betweenthe corresponding tasks. Data dependence indicates that atask cannot be executed until it receives some required datafrom its precedent tasks. In addition, each link is labeled withthe cost between the adjacent nodes (e.g., energy consump-tion on the smartphone), which depends on the applicationprofile and network conditions. The total cost for the applica-tion execution is the summation over the cost of each link.


Figure 1. Generic architecture of the elastic computing platform is composed of an infrastruc-ture cloud and an ad hoc virtual cloud.

OS

©VMware

SmartphoneAd hoc virtual

cloud

Communication between cloud clones

Application offloading to the cloud

Cooperation between mobile phones

Task delegation

Data access

Executionengine

Datastorage

Infrastructurecloud

©VMware

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP

OS

APP OS

APP

©VMware, Inc.©VMware, In

c.

©VMware, Inc.

©VMware, Inc.

©VMware, Inc.©VMware, In

c.

©VMware, Inc.

©VMware, Inc.

Cloudphone

�VMware, Inc.

©

VMware, In

c. © VMware, In

c.


We propose an optimization framework as follows. Theobjective is to minimize the total cost (e.g., energy consump-tion on the smartphone) while respecting execution con-straints for all the tasks (e.g., application delay). We aim toderive optimal or near-optimal algorithms to allocate tasksinto computing resources (i.e., standalone smartphone, a clus-ter of smartphones, and cloud clone). However, due to itscombinatorial nature, the optimization problem is NP-com-plete. To obtain useful results and insights, we focus on a fewparticular cases that are computationally tractable, as shownin Fig. 2, including:• Only one active node, representing the whole application• A linear chain topology, representing a sequential list of

tasks• A tree structure, representing a tree-based hierarchy of

tasks• A regular mesh structure, representing a lattice-based topol-

ogy of tasksIn the following subsections, we provide solutions for these

particular cases.

Energy-Optimal Offloading Policy: One-Node CaseIn this subsection, we consider the energy-optimal applicationexecution policy when the task-flow graph has only one activenode, as illustrated in Fig. 2a. The decision is to determinewhether the entire application should be executed on thesmartphone or the cloud, with an objective to minimize theenergy consumption on the smartphone while meeting theapplication’s completion deadline.1

We can obtain the optimal energy for both the standaloneand cloud executions as presented in [9]. Consider an applica-tion profile (T, L), where the application with L bits of inputdata should be completed before time delay T. For standaloneexecution, the computation workload is completed on thesmartphone by varying CPU frequency for each workload. Forcloud execution, input data is transmitted to the cloud byadapting the transmission rate in response to network condi-tions. The minimum energy consumption of the smartphoneby standalone execution and cloud execution are Em* = ML3/T2

and Ec* = C(n) Ln/Tn–1, respectively. Herein, M is a constantdepending on the chip architecture of the smartphone, andC(n) is a function of monomial order n depending on theenergy model of data transmission over the wireless channel.Comparing Em* with Ec*, we can determine which execution ismore energy-efficient.

Figure 3 shows the results of optimal application executionfor the one-node case. First, for n < 3 and n > 3, optimalapplication execution regions of the standalone and cloud exe-cutions are separated by a line. Geometrically, the slope ofthe line can be interpreted as a threshold,

which is a constant. We define an effective data consump-tion rate as the ratio of input data size and delay deadline;that is, Re = L/T for the determination of the optimal exe-cution. For example, for n = 4 in Fig. 3c, if the point (T,L) is below the line, cloud execution is optimal; otherwise,standalone execution is optimal. Second, for the case of n =3, the energy consumption of standalone and cloud execu-tions have the same order of L and T; thus, the decisiondepends on the comparison between M and C(n). In the set-ting of this case, M < C(n); hence, standalone execution isthe optimal execution.

Energy-Efficient Offloading Policy: The Linear ChainCaseIn this subsection, we investigate offloading policy for the taskflow of a linear chain as illustrated in Fig. 2b.

We first construct a directed acyclic graph to model taskexecution in a linear chain in Fig. 4b. Two dummy nodes, Sand D, are introduced for application initiation and termina-tion. Node k indicates that task k has been completed on thesmartphone, while node k represents that task k has beencompleted by the cloud clone, where k = 1, 2, …, n, and n isthe total number of tasks in the application. A link betweenthe adjacent nodes represents data dependence between thetasks. In this case, data dependence requires that task k canonly be started after the completion of task k – 1, since theoutput data of task k – 1 is input data of task k. Also, eachlink is associated with a nonnegative cost to complete the cor-responding task (e.g., energy consumption on the smartphoneor completion time). In addition, if task k accesses localresources (e.g., GPS and sensors), it should be executed onthe smartphone, and hence the cost of the link connectingwith node k is infinite.

Based on the graph in Fig. 4, we then formulate the offload-ing policy as a constrained shortest path problem as presentedin [10]. The objective is to find the shortest path in terms ofenergy consumption between S and D in the graph, subject tothe constraint that total completion time of that path shouldbe less than or equal to time deadline Td. A path p is feasibleif total completion time satisfies the delay constraint. A feasi-ble path p* with minimum energy consumption is the optimalsolution among all the feasible paths. Mathematically, wehave

where P is the set of all possible paths, and eu,v and du,v areenergy consumption and completion time between any adja-

RM

C n( ),th

n

1

3=

⎡

⎣⎢

⎤

⎦⎥

−

E E

E E

Pe p e

s t d p d T

min [ ( )]

. . [ ( )] ,

pu v

u v p

u vu v p

d

,( , )

,( , )

∑

∑

=⎧⎨⎪

⎩⎪

⎫⎬⎪

⎭⎪

=⎧⎨⎪

⎩⎪

⎫⎬⎪

⎭⎪≤

∈ ∈

∈


Figure 2. Examples of task-flow graphs in different topologies: a)only one active node; b) linear chain topology; c) tree structure;d) regular mesh structure.

54

4

5

32

2 2 4

3 3

(c) (d)

11

1 1 5

(b)(a)

1 For ease of presentation, we focus on the analysis of cloud execution; thesame rationale can be applied for cooperative execution.

cent nodes u and v on the path p, respectively. Note that theexpectation is taken over the channel state due to fadingeffects in wireless networks. This constrained optimizationproblem, however, is NP-complete.

To solve this optimization problem efficiently, we adapta canonical algorithm, Lagrangian Relaxation BasedAggregated Cost (LARAC). We define the aggregated costfunction, L(l ) = E [e(p)+ld(p)] – lTd, where l i s aLagrange multiplier. By the Lagrange duality principle, wehave L(l) £ E [e(p*)], which gives a lower bound for theoptimal solution of offloading policy. More details of thealgorithm can be found in [10], and are here omitted dueto page limits.

Figure 5 illustrates an example of offloading policy for thelinear chain topology under a stochastic channel model wheredata transmission rate is independent and identically distribut-ed. We observe that the decision depends on the ratio betweendata size and workload. For example, for task 1, its input datasize is large while workload is small; hence, it is executed onthe smartphone. For task 3, its input data size is small andworkload is large; hence, it is efficient to offload task 3 to thecloud clone for execution.

Offloading Policy by Parallel Execution: The Tree andMesh CasesIn this subsection, we study offloading policy by parallel exe-cution for the task-flow graphs of tree and mesh. We observethat loose data dependence of tree and mesh and multiplewireless interfaces on the smartphone provide us opportuni-ties to offload tasks more efficiently.

First, tree and mesh structures have looser data depen-dence on tasks than linear chain topology. Hence, tasks with-out data dependence can be executed in parallel. For example,in Fig. 2c, tasks 2 and 3, at the same level of the tree, can besimultaneously executed by the cloud clone in the infrastruc-ture-based cloud or the cluster of smartphones in the ad hocvirtual cloud.

Second, using multiple wireless interfaces on the smart-phone can accelerate data transmission before parallel execu-tion of tasks on the cloud. Input and output data of tasks canbe concurrently transmitted via multiple wireless networkinterfaces (e.g., Bluetooth and 3G) to reduce data transmis-sion time. For example, in Fig. 2d, output data of task 1 canbe transmitted to the cloud clone and smartphones nearbysimultaneously by 3G and Bluetooth before the execution oftasks 2 and 3, respectively.

Based on these two opportunities, the offloading policy fortree and mesh task-flow graphs is to jointly allocate comput-ing resources for the execution of all the tasks and choosewireless network interfaces for data transmission. As anexample, we consider the mesh structure in Fig. 2d, aiming tominimize application completion time. It is equivalent tominimize the absolute value of difference between comple-tion time of the upper and lower paths between tasks 1 and 5in the graph.

The Offloading Mechanism of the UnifiedElastic Computing PlatformThere are two alternative implementation strategies ofoffloading mechanism: the method-level offloading approach(e.g., MAUI [11]) and system-level offloading approach (e.g.,Cloudlet [5] and CloneCloud [6]). In this section, we discussthe offloading mechanism of the infrastructure-based cloudand ad hoc virtual cloud, respectively.


Figure 3. Optimal application execution for the one node case.For n = 2 and n = 4, optimal application execution regions ofthe standalone execution and cloud execution are separated bya line. The slope of the line is Eq. 1, where M is a constantdepending on the chip architecture of the smartphone, andC(n) is a function of monomial order n depending on the ener-gy model of data transmission over a wireless channel. For n =3, the decision of optimal execution depends on the compari-son between M and C(n).

Delay deadline, T (ms)

(a)

500

200

0

Dat

a si

ze, L

(bi

ts)

400

600

800

1000

100 150

Standalone execution

Cloudexecution

200 250


(b)

500

200

0

Dat

a si

ze, L

(bi

ts)

400

600

800

1000

100 150


200 250


(c)

500

200

0

Dat

a si

ze, L

(bi

ts)

400

600

800

1000

100 150


Cloud execution

200 250


The Offloading Mechanism of the Infrastructure-BasedCloud

The method-level offloading approach can be applied toimplement application offloading in the infrastructure-basedcloud, due to its high execution efficiency. In the method-leveloffloading approach, an application is partitioned, and a partof code is executed by remote procedure call (RPC). Thisapproach provides a set of low-level programming interfacesfor remote execution, which achieves high execution efficien-cy. However, it requires programmers to decide which methodor module should be offloaded. For this implementation, wecan adopt an architecture similar to OpenMobster, which isan open source mobile cloud platform.

We can also choose the system-level offloading approach toimplement application offloading in the infrastructure-basedcloud due to its easy programmability. In this approach, animage of a smartphone is cloned in the cloud through virtualmachine technology (e.g., Xen). With a system identical tothat of the smartphone, the clone provides a set of high-levelprogramming interfaces, which is easy to program. However,completely cloning the system of the smartphone can increase

the complexity of security control in the infras-tructure-based cloud. An alternative solution is toset up a set of weblets [7] elastically via software-oriented architecture (SOA) to execute the appli-cation. For this implementation, we can adopt anarchitecture based on weblets.

Offloading Mechanism of the Ad HocVirtual CloudIn the ad hoc virtual cloud, the method-leveloffloading approach can be adopted to implementapplication offloading. As storage of smartphonesis limited, the system-level offloading approach

via VM clone is impractical in the ad hoc virtual cloud. As aresult, we choose the method-level offloading approach viaRPC for the mechanism.

Opportunities and Challenges of UnifiedElastic Computing PlatformIn this section, we discuss opportunities and challenges of theunified elastic computing platform.

OpportunitiesThe proposed elastic computing platform provides opportuni-ties to enhance the capability of smartphones, which includetask delegation, and cloud clone P2P network, data backup,and data staging. These are the advantages of our proposedplatform compared to previous work.

Task Delegation — The execution of computation-intensivetasks can be delegated to remote execution engines in thecloud infrastructure, which is referred to as task delega-tion. Some applications (e.g., media transcoding) can con-sume more computing resources than the cloud clone canafford. In this case, we can leverage back-end executionengines with sufficient computing resources to executethese tasks.

Cloud Clone P2P Network — A cloud clone P2P network isformed by a set of cloud clones in the infrastructure-basedcloud, which can mitigate sporadic network connectivityamong smartphones. Cloud clones are logically connectedwith more stable connectivity and higher bandwidth than theirassociated smartphones. Communication among smartphonescan be performed via cloud clones. In this way, there are newopportunities for energy-efficient collaborative mobile appli-cations among smartphones.

Data Backup — If the smartphone is lost accidentally, the datacan be recovered in another secure smartphone by data back-up from the cloud clone, which is similar to iCloud.

Data Staging — The cloud clone in the infrastructure-basedcloud can serve as a proxy for data staging, which can reducethe latency for the smartphone to fetch data. For example, anoriginal high-definition video, stored in the infrastructure, canbe transformed into HTTP streaming format with diverseplayback rates in any cloud clone [12]. When a mobile userrequests video, the cloud clone replies with a required stream-ing rate.

ChallengesThere are also challenges in the elastic computing platform,including performance, security, and energy issue.


Figure 5. Offloading policy under the IID channel model for lin-ear chain topology. There are 10 tasks. The horizontal axisdenotes the tasks to be executed, the left vertical axis representstask context, including normalized workload (M cycles) andinput/output data size (kb), and the right vertical axis representsthe optimal task execution location (0 for smartphone and 1 forcloud clone). For a particular task, the darker bar representsthe workload of the task, while its left and right lighter bars rep-resent the input and output data sizes of the task, respectively.The line represents the execution decision. The expected datatransmission rate over the wireless channel is 10 kb/s. Theapplication time deadline is 0.7s.

1

10

0

Task

con

text

Dec

isio

n

20

30

40

50 0

1

2 3 4 5 6 7 8 9 10

DataWorkload

Figure 4. A graph of task execution flow for linear chain topology. Node k rep-resents that task k has been completed on the mobile device, while node krepresents that task k has been completed on the cloud.

......

D

Cloud clone

Smartphone

nn-12 ......1

nn-11 ......2s


Performance — We refer to performance as the serviceresponse time of application offloading for mobile users. Toguarantee performance, we need a topology design of cloudclones in the infrastructure-based cloud. In [13], there arepeer-based, proxy-based, and clone-based models for thetopology design. A suitable topology of cloud clones should bechosen to achieve high performance for different types ofmobile applications.

Security — Security issues should be addressed in our proposedelastic computing platform. First, cloud clones should be trust-ed. The smartphone should identify a trusted cloud clone byitself (i.e., trust establishment [5]) or check the identity of thecloud clone based on trust measurements conducted by a thirdparty (i.e., reputation-based trust [5]). Second, cloud services(i.e., computation and storage) provided by the cloud infras-tructure should be trusted. We need to ensure that there areno other hidden programs running behind the service. In addi-tion, security mechanisms should be light-weight without incur-ring much energy consumption on smartphones.

Energy — Although we have mitigated the battery insufficien-cy of an individual smartphone by application offloading,energy consumption of the cloud side has not yet beenaccounted for, especially for ad hoc virtual clouds. Note thatthe battery systems of smartphones in an ad hoc virtual cloudis also limited. Therefore, we need a strategy to choose smart-phones and distribute workload based on their computingcapability and residual battery volume for application offload-ing in order to prolong the lifetime of all the smartphones.

ApplicationsIn this section, we discuss two mobile applications that canbenefit from offloading in our proposed elastic computingplatform: antivirus and mobile cloud gaming.

Mobile Antivirus ApplicationIn our elastic computing platform, we can exploit the cloudclone to scan for viruses and malicious content, as shown inFig. 6. Initially, a smartphone sends input files to the cloudclone for scanning. Then the detection engine in the cloudclone scans the content of the input files against a virusdatabase maintained by the infrastructure cloud. Finally, athreat report is sent back to the smartphone.

We can apply the threshold policy to decide whether virusscanning should be executed on the smartphone or on thecloud. The policy depends on the size of files to be scanned,completion time for scanning, and current network channel

status. We can evaluate the value of the effectivedata consumption rate to obtain the energy-opti-mal execution policy.

Mobile Cloud GamingOur proposed elastic computing platform can serveas a mobile cloud gaming system [14], which is apromising paradigm to eliminate the resource con-straint on the smartphone for mobile games. First,a cloud clone is delegated to complete the compu-tation-intensive tasks (e.g., 3D graphics rendering),while the smartphone as a thin client receivesresults displayed on its screen. Second, a cloudclone P2P network can enhance the performance ofmulti-user mobile games on smartphones. Cloudclones in the P2P network can communicate witheach other to obtain the information from mobileusers and complete the computation in the game.

We envision that the offloading policy can be applied inmobile cloud gaming. It is quite common for a complexmobile gaming application to be decomposed into multipletasks in a linear chain, tree, or mesh topology. The offloadingpolicy will optimize the execution of real-time interactivemobile games.

ConclusionWe combine the ad hoc virtual cloud and infrastructure-basedcloud as the fabric of the unified elastic computing platformto enhance the scalability of smartphones. Under this plat-form, we present an offloading policy by a unified optimiza-tion framework and an offloading mechanism to implementapplication offloading. We also discuss two applications (anti-virus and mobile cloud gaming) that can benefit from theelastic computing platform. In the future, we will considermore generic graphs for general offloading policy and buildreal mobile applications.

References[1] M. Satyanarayanan, “Fundamental Challenges in Mobile Computing,”

Proc. ACM Symp. Principles of Distrib. Computing, 1996, pp. 1–7. [2] R. Balan et al., “The Case for Cyber Foraging,” Proc. 10th ACM Special

Interest Group on Op. Sys. European Wksp., 2002, pp. 87–92. [3] K. Kumar and Y. H. Lu, “Cloud Computing for Mobile Users: Can Offload-

ing Computation Save Energy?,” IEEE Computer, vol. 43, no. 4, 2010,pp. 51–56.

[4] M. Chen et al., “Enabling Technologies for Future Data Center Network-ing: A Primer,” IEEE Network, 2013.

[5] M. Satyanarayanan, R. C. P. Bahl, and N. Davies, “The Case for VM-Based Cloudlets in Mobile Computing,” IEEE Pervasive Computing, vol. 8,no. 4, 2009, pp. 14-23.

[6] B. G. Chun et al., “Clonecloud: Elastic Execution Between Mobile Deviceand Cloud,” Proc. 6th Euro. Conf. Computer Sys., 2011, pp. 301–14.

[7] X. W. Zhang et al., “Towards an Elastic Application Model for Augment-ing the Computing Capabilities of Mobile Devices with Cloud Comput-ing,” Mobile Networks and Applications, vol. 16, no. 3, 2011, pp.270–84.

[8] G. Huerta-Canepa and D. Lee, “A Virtual Cloud Computing Provider forMobile Devices,” Proc. 1st ACM Wksp. Mobile Cloud Computing and Ser-vices: Social Networks and Beyond, 2010.

[9] Y. Wen et al., “Energy-Optimal Execution Policy for a Cloud-AssistedMobile Application Platform,” tech. rep., 2011.

[10] W. Zhang, Y. Wen, and D. Wu, “Energy-Efficient Scheduling Policy forCollaborative Execution in Mobile Cloud Computing,” 32nd Annual IEEEInfocom, 2013, pp. 190–94.

[11] E. Cuervo et al., “MAUI: Making Smartphones Last Longer With CodeOffload,” Int’l. Conf. Mobile Sys., Applications, and Services, 2010, pp. 49–62.

[12] W. Zhang et al., “QoE-Driven Cache Management for HTTP Adaptive BitRate Streaming Over Wireless Networks,” to appear, IEEE Trans. Multime-dia, 2013.

[13] G. Hu, W. P. Tay, and Y. Wen, “Cloud Robotics: Architecture, Chal-lenges and Applications,” IEEE Network Special Issue on Machine andRobotic Networking, vol. 26, no. 3, 2012, pp. 21–28.


Figure 6. Architecture of virus scanning by a cloud clone.

Input files

Threat report

Back-endstorage

Cloud clone

Mobiledevice

Virusdatabase

Detectionengine

�VMware, Inc.

OS

APP

�VMware, Inc.


[14] W. Cai, V. C. Leung, and M. Chen, “Next Generation Mobile CloudGaming,” Proc. IEEE Int’l. Symp. Mobile Cloud, Computing, and ServiceEngineering, 2013.

BiographiesWEIWEN ZHANG ([email protected]) received his Bachelor’s degree in soft-ware engineering and Master’s degree in computer science from South ChinaUniversity of Technology (SCUT) in 2008 and 2011, respectively. He is cur-rently a Ph.D. candidate in the School of Computer Engineering at NanyangTechnological University (NTU) in Singapore. His research interests includecloud computing and mobile computing.

YONGGANG WEN ([email protected]) received his Ph.D. degree in electricalengineering and computer science from the Massachusetts Institute of Technol-ogy (MIT) in 2008. He is currently an assistant professor with the School ofComputer Engineering at NTU. Previously, he worked at Cisco as a seniorsoftware engineer and system architect for content networking products. Healso worked as a research intern at Bell Laboratories and Sycamore Net-works, and served as a technical advisor to the chairman at Linear A Net-

works, Inc. His research interests include cloud computing, mobile computing,multimedia networks, cyber security, and green ICT.

JUN WU ([email protected]) received his B.S. degree in information engi-neering and M.S. degree in communication and electronic system from XidianUniversity in 1993 and 1996, respectively. He received his Ph.D. degree insignal and information procesing from Beijing University of Posts and Telecom-munications in 1999. He joined Tongji University as a professor in 2010. Hehas been a principal scientist at Huawei and Broadcom before joining Tongji.His research interests include wireless communication, information theory, andsignal processing.

HUI LI ([email protected]) received his M.S degree in computer sciencefrom Simon Fraser University in 1997. He received his Ph.D. degree in com-puter science from Sichuan University in 2008. He is currently a professor inthe College of Computer Science at Sichuan University, China. He worked atNorthern Telecom as a senior software developer. He also worked as a keysystem engineer in Wisesoft. His research interests include virtual reality, multi-media systems, computer graphics, cloud computing, and social networks.








http://www.milcom.org

ecently, natural disasters and extremely abnormal cli-mate situations happen frequently and globally, theculprit of which is the exacerbation of global warm-

ing. It is a consensus that both governments and individualsshould take action to control greenhouse gas emission, espe-cially emission of CO2, which is mainly caused by humanactivities. According to the statistical data, arguably more than80 percent of CO2 emissions originate in cities, while citiesonly occupy less than 2.4 percent of the global land mass [1].Hence, understanding the relationships between the form andpattern of urban development and the carbon cycle is crucialfor estimating future trajectories of greenhouse gas concentra-tions in the atmosphere and facilitating mitigation of climatedeterioration. There are generally two strategies admitted bymost countries for computing/estimating pollutant (e.g., CO2)emissions of factories, areas, or countries. The first one is toestimate CO2 emissions based on raw material (e.g., fossil oiland coal) consumption, which is adopted by the EuropeanEnvironment Agency when it ranks countries by their CO2emissions every year. The other one, called inference-basedcarbon sensing technology, is utilized to measure pollutant gasfluxes [2], like CO2 and NO2. Both estimation-based andinference-based methods are less accurate and do not satisfythe real-time requirements for pollutant gas monitoring. City-See is designed as an alternative approach to directly measurepollutant emissions of large-scale areas accurately and thor-oughly in a real-time and long-term manner. It integrates sev-eral thousands of sensors with mature technology and smallindividual wireless nodes to make us capable of interactingwith the physical world. In our first phase of implementation,1200 nodes (1196 sensor nodes and 4 wireless mesh nodes)are deployed in the urban area of Wuxi City, China, so as oftoday, July 2013, multidimensional environmental data (CO2,NO2, temperature, humidity, light level, location, etc.) havebeen continuously collected in a real-time manner sinceAugust 2011. CitySee faces many challenges, each of whichcorresponds to several issues that need to be properlyaddressed.• Hardware design: To facilitate large-scale and affordable

deployment, we need to carefully consider both the archi-

tecture design and the encapsulation design [3] for terminalnodes. Since all 1200 nodes are designed to be deployed inoutdoor environments without necessary protection, robust-ness is a big concern.

• Software design: CitySee contains several levels of software[4], including hardware drivers, embedded software for datacollection, data processing, and routing [5], as well as appli-cation software for providing rich and friendly services tousers. In addition, the software of CitySee should shield thedifference of heterogeneous operating system and hardwareplatforms according to compatibility and expansibility.

• Network design: To the best of our knowledge, CitySee iscurrently the largest working wireless sensor network(WSN). We have to address a series of key issues so as tokeep the entire system running healthily and smoothly, suchas sensor deployment [3], data loss [6], energy efficiency [5],network management and diagnosis [7].

• Pervasive services: The ultimate goal of CitySee is to providepervasive services to both governments and individual users.In other words, besides various of applications supported byall kinds of sensory data, CitySee also provides other typesof services, like localization service [8] utilizing wireless sig-nals of deployed sensors, or short video information byequipping small cameras into nodes.In this article, we briefly go through each part of CitySee

and take a first step toward how to combine WSNs with cur-rent leading technologies (e.g., cloud computing) in order toprovide satisfactory pervasive services to both network design-ers and users with respect to scalability, performance, privacy,cost savings, etc.

Related WorkOne of the first studies on WSNs for habitat monitoring wasreported by Mainwaring et al. in [9], in which an instance ofWSN for monitoring seabird nesting environments and behav-iors was described. The authors mainly investigate hardwaredesign of sensors, network architecture, capabilities forremote data access and management, etc. An early surveil-lance application using a power-constrained sensor network

R


AbstractCitySee, an environment monitoring system with 1196 sensor nodes and 4 meshnodes in an urban area, is mainly motivated by the needs of precise carbon emis-sion measurement and real-time surveillance for CO2 management in cities. Beingone of the largest working wireless sensor networks, CitySee faces several chal-lenges such as hardware design, software development, platforms, network proto-cols, and, most important, satisfactory services to users. We share some earlylessons learned from this project, illustrate the potential benefits and risks of currentsolutions, and discuss the possible extensions of CitySee applications.

CitySee: Not Only a Wireless Sensor Network

Yunhao Liu, Xufei Mao, Yuan He, Kebin Liu, Wei Gong, and Jiliang Wang, School of Software, Tsinghua University

R

0890-8044/13/25.00 © 2013 IEEE

MAO_LAYOUT_Layout 1 9/20/13 12:58 PM Page 42

was proposed by Vicaire and He et al. in [10]. The authorspresent the design and implementation of multidimensionalpower management strategies in VigilNet, and further intro-duce a novel tripwire service with effective sentry and dutycycle scheduling so as to increase the system lifetime. Thereare also some other important WSN applications [11, 12] forenvironmental surveillance.

In our early project GreenOrbs [6], we first deploy 330 wire-less TelosB nodes equipped with temperature, humidity, andlight sensors in the forest, for studying canopy closure of theforest, and later more sensors are employed. Sensor nodes inGreenOrbs are intensively dense due to the canopy measuringrequirements. Indeed, since there are not many vehicles andvery few people in the area, the environment of GreenOrbs isnot as complicated as in a city.

CitySee can also be viewed as the second phase ofGreenOrbs, and it has 1200 working nodes now, including 1196wireless sensor nodes (each has multiple sensors) and 4 wire-less mesh nodes. Besides the challenges brought by the largerscale, we focus more on coupling WSNs with other leadingtechnologies (e.g., cloud computing) in order to provide satis-factory services to both network designers and service users.

SolutionsSystem InfrastructureCitySee is currently designed as a pervasive service that inte-grates both the underlying wireless sensor network techniques(“sensing IaaS”; IaaS stands for infrastructure as a service)and the upper-layer cloud computing applications, that is, asensing as a service (SaaS) cloudlet. Here, the reason for us touse “cloudlet” rather than “cloud” is because our currentcloud computing architecture is not a full-fledged cloud facili-ty yet, and it is specially simplified/trimmed for our concen-trated sensing services. Meanwhile, we are making continuousefforts to strengthen it into a real cloud, as depicted in Fig. 1.

At the underlying layer, various kinds of sensors deployedacross buildings and plazas constitute the sensing IaaS. At theupper layer, the SaaS cloudlet consists of both cloudlet serversand sink nodes, as i CitySee sink nodes not only aggregate and

forward raw sensing data (i.e., sink nodes’ conventional func-tionality in a WSN), but also preprocess the data before syn-chronizing them to cloudlet servers. Finally, cloudlet serversform a series of SaaS application programming interfaces(APIs) that facilitate the access, use, and programming ofsensing service customers.

HardwareWe design and implement two types of nodes, wireless sensornodes equipped with different types of sensors (abbreviated tosensor nodes) and wireless mesh nodes (abbreviated to meshnodes), respectively. The hardware platforms of sensor nodesare based on TelosB. A wireless sensor node is equipped withan MSP430F1611 processor and a CC2420 radio chip compli-ant with the 802.15.4 protocol at 2.4 GHz. Each node has oneor multiple sensors, such as temperature, humidity, CO2, NO2,wind velocity and direction, according to various applicationrequirements. All nodes are encapsulated with industrialgrade design in order to adapt to hostile outdoor environ-ments. Sensor nodes mainly take charge of sensing the envi-ronments, packetizing sensing data, and further sending datapackets to one of sink nodes (each of which connects to amesh node directly) through one- or multi-hops (i.e., theyform a wireless ad hoc network). Due to the outdoor deploy-ment environments, during the past two years most of the sen-sor nodes have either had their batteries changed one or moretimes or have been replaced.

CitySee currently has four mesh nodes constructing a wirelessad hoc network, with one of them selected as the final sink. Amesh node mainly consists of an ARM7 processor and a net-work card of type WL017MP, using the 802.11a protocol at 5.8GHz. Considering the difficulty of power supply, a mesh nodecould be powered by either electricity, net wire, or solar panels.

Software on NodesWe develop software for different types of sensor nodes ontop of TinyOS 2.1.1, which consists of the following majorcomponents. First, we implement the link estimation compo-nent using the four-bit link estimation method to regularlymaintain a neighbor table. Second, we use the default Low


Figure 1. Three-layer service structure of CitySee.

Cloudletservers

Sink nodeSink node Sink node

SaaS APIs

Sensing servicecustomers

Sensing as a service(SaaS) cloudlet

Ad hoc sensornetworks


Power Listening medium access control (MAC) protocol ofTinyOS to reduce the energy consumption. Third, the multi-hop routing component is implemented based on the Collec-tion Tree Protocol (CTP) [13] for data collection. Fourth, weapply the Drip protocol to disseminate key system parametersin terms of the dissemination component, such as transmissionpower levels, sampling frequencies, and duty cycles. In City-See, a sensor node is programed to sample the environmentevery 10 minutes and then issue data packets to a sink. Thedetailed structure of the software running on a sensor node isshown in Fig. 2.

ConnectivityOne of the biggest challenges of CitySee is to maintain theconnectivity of all 1200 nodes, especially when harsh condi-tions exist. For CitySee, the exact locations for all CO2 sensornodes and some given types of environmental sensors havebeen pre-designated by environmentalists. As the sensingoperation of a CO2 sensor is very energy consuming, we nor-mally do not let the nodes equipped with a CO2 sensor (100of them) relay packets for others, while the other type ofnodes perform both sensing and relaying operations. In addi-tion, due to the complicated outdoor conditions and physicalconstraints of urban areas (e.g., buildings and artificial lakes),on one hand, there are some places where we cannot deployany nodes; on the other hand, two nearby sensor nodes maynot be able to communicate with each other due to signalblocking, reflection, and so on.

We formulate this issue as a geometric Group Steiner Treewith Holes problem and propose a 2-approximation algorithmto solve it [3]. Our main idea is to define and construct legalcomponents depending on the first deployment (determinedby the environmentalists), based on which a secondary deploy-ment is conducted while minimizing the total number of relaynodes needed as the optimization objective. We apply theproposed strategies to CitySee and part of the deployment sit-uation of CitySee as shown in Fig. 3.

Data CollectionThe fundamental goal of CitySee is to monitor and collectenvironment data frequently, so keeping the integrity andreal-time properties of sensory data is necessary and impor-tant. In addition, status data of nodes and the network arenecessary as well since grasping the running status of both asingle node and the network is critical for maintaining the sys-tem.

Protocols — In CitySee, we mainly employ CTP [13] for multi-hop data collection; that is, each node is planted with anagent, which senses the environment periodically and issuesdata packets to a sink node through one hop or multiple hops;all sensor nodes act as leaf nodes only. We collect three typesof data packets, each of which contains different types ofinformation. A packet with type C1 has two categories ofinformation:• Sensing data: temperature, humidity, light or CO2 concen-

tration values• Routing information: including path-ETX [13] from the

original source of the packet to some sink node, throughwhich we are able to recover the complete routing path ofany packet arriving at a sink node

A packet with type C2 records local information for eachnode, such as routing table information, including IDs andreceived signal strength indicator (RSSI) values, from itsneighbors, and sends ETX estimation values of links to itsneighbors. A packet with type C3 contains more detailedinformation on a single wireless node. For instance, the CPUcounter records the accumulated task execution time, the radiocounter indicates the accumulated radio-on time, the transmitcounter depicts the accumulated number of transmitted pack-ets, the receive counter describes the accumulated number ofreceived packets, and the loop counter tells us the accumulat-ed number of detected loops.


Figure 3. Part of the deployment situation of CitySee.

Figure 2. Architecture of embedded software.

Data collector Configurator Status viewer

MessageformatterLoggerDRIPCTPFTSP

Serialactive message

Flashreader/writerActive messageTimer

FTDI converterExternal flashCC2420 radio

Data flow

Oscillator

Control flow


Based on status data, CitySee pro-vides network management services tonetwork administrators by showing thereal-time running status of both theentire network and an individual node.For instance, the left part of Fig. 4plots the partial logical topology of1200 nodes during the last duty cycle;each individual node is shown as a cir-cle whose area indicates the number ofpackets it has transmitted for the last10 minutes. After clicking a node, thewhole path along which its last packetwalks is shown. The right part of Fig. 4indicates the measurement matrix(with more than 20 indices) of a singlenode, including the number of tasks ithas posted and executed, the numberof duplicated packets, and so on.

Potential Benefits of Using Cloud Com-puting Strategies — At a first glimpse,it seems that data collection in a WSNis kind of easy since every node onlyneeds to send data packets to somenode(s) by pre-designed rules. Howev-er, it is indeed quite complicated, espe-cially when resources of a wirelessnode are limited considering two bat-teries will support more than one year,while wireless links in the network arevery unstable, so many retransmissionsare necessary. In CitySee, the traffic loads of nodes with differ-ent roles (e.g., relay nodes and leaf nodes) are quite different,depending on the physical environment and routing protocols.For example, the number of tasks executed by a node close tothe sink node could be up to 8742 in 10 minutes, while around1/4 of wireless nodes have an average number less than 8. Fig-ure 5 shows the traffic load of nodes in CitySee, from whichwe can see that the traffic load of a node is not merely deter-mined by its physical location or assigned roles. In CitySee,two types of data collection strategies are designed: activereport and passive query. When the active report strategy isadopted, a node chooses one or more nodes as its relay nodeswhen it has packets to transmit. On the contrary, passivequery strategy is activated in some cases. For instance, when anode detects abnormal data packets from nearby nodes, itissues a query to the latter one and requires the node toresend its packets.

After packets arrive at the sink nodes, data synchronizationinto the cloudlet servers is distinct from conventional datasynchronization (e.g., Dropbox, Amazon S3, and MicrosoftAzure) in commercial cloud storage services due to the factthat data flows in WSNs are usually frequent, short, and loca-tion-sensitive. In the presence of such data flows, coarse-grained cloud synchronization mechanism design may lead tosevere traffic overuse. For example, even one byte’s sensingdata synchronization (from a sink node to a cloudlet server)would incur tens of kilobytes’ sync traffic (using the widelyadopted HTTP/HTTPS sync protocol). Naturally, a series offrequent, short sensing data may bring about several orders ofmagnitude more sync traffic than the size of the original sens-ing data.

As a result, for data synchronization, we adopt an adaptivetimer-triggered delta sync mechanism [14], which adaptivelytunes its sync defer timer threshold to match the latest sensingdata flow pattern and thus greatly reduces the sync traffic with

acceptable increment in sync delay. Here, sync defer is differ-ent from the commonly mentioned notation sync delay. Whena packet is generated, sync delay indicates how long it takes tofully sync this data packet to the cloud, while sync defer tellsus how long it takes for our proposed sync mechanism tointentionally defer the data sync process.

Data Processing and VisualizationData processing and visualization are the main matrices (e.g.,response time of a query) to measure the service quality ofCitySee to users.

When the Size of Data is Small — In the first two months, wecollect 8-Gbyte data traces from 1200 nodes including all envi-ronment-related data for the purpose of CO2 emission analy-sis and network status-related data for the purpose of networkmanagement and diagnosis. Combining all three types of datapackets, we depict the entire network at the base station usingthe real map where the geometric location of each node isobtained when deployed. Since we utilize an adaptive and dis-tributed routing strategy in CitySee, the network topologydynamically changes depending on network traffic and real-time link qualities. The longest hop distance observed from anode to a sink node is 21 hops, which means that some pack-ets are relayed at least 20 times before reaching a sink node.

Based on collected data, we design and implement a num-ber of visualization interfaces for both service users and net-work administrators through frequently recurringenvironmental and network events intuitively.

When the Size of Data Grows Larger — Unfortunately, thedisadvantages of utilizing several relatively independentservers emerge. The first issue we have to address is reliabilityand efficiency. When a user pushes the button to query 10months’ data of a sensor node on the server, he/she has to


Figure 4. The logical topology of part of 1200 nodes and the multidimensional matrix ofthe status of a single node.

3607

6803233

32333233

3233

700080

2209

2306

2324

2102

2326

2109

205020512039

2028

2209

10000070008

2051

2039

23342345

23432301

21002050

NoParentTaskSendFail

TableEvict

MacIBackoff

MacCBackoff

RadioOn

RadioOnTime

Receive

Transmit

OverflowDropSelfTransmit

NoACKRetransmit

NoACKDrop

Retransmit

Loop

Duplicate

SuccACK

ParentChange

TaskPost

TaskExe

Figure 5. Traffic load of nodes.

Heavy traffic nodewith stable load

Aug. 4 Aug. 4 Aug. 5 Aug. 6 Aug. 7 Aug. 8 Aug. 9 Aug. 4 Aug. 4 Aug. 5 Aug. 6 Aug. 7 Aug. 8 Aug. 9

Heavy traffic nodewith stable load


wait around 15 s, which is a terrible user experience without adoubt. In addition, a single database cannot satisfy the relia-bility requirements for CitySee since the power shutdown orphysical damage to hard disks leads to temporary or evenunrecoverable data loss. Clearly, utilizing the cloud can allevi-ate this situation. Here, we face two choices, a private cloud(i.e., we build our own cloud using a number of high-perfor-mance servers) or the public cloud (e.g., obtaining servicesfrom Amazon or Google directly). The question is that theadvantages and disadvantages of both private clouds and pub-lic clouds are obvious. For example, to design our own privatecloud, it seems that we only need to pay more for hardwareinstead of paying service fees to cloud service providers. How-ever, it is hard to achieve as good performance as the cloudservice providers do in terms of efficiency, reliability, and scal-ability since the latter have relatively mature technologies. Onthe contrary, if we totally trust the public cloud, besides thehigh cost, do we really trust the security of their services,especially for sensitive data?

After counting the cost, our next step for CitySee is to com-bine both a private cloud and the public cloud in order toboth obtain and provide the best cloud services, that is, build-ing our own private cloud and using the public cloud at thesame time.

Network Diagnosis and ManagementCitySee has a long-term running objective, and any physicalmodification of the network (e.g., replacing individual nodes)is pretty costly, so it is critical and necessary to learn the run-ning status of the entire network as well as each individualnode. In order to collect key metrics, such as radio dutycycles, and the number of packet transmissions and recep-tions, and provide visibility to the system, we further designand implement the network management and diagnosis com-ponent. We design and implement PAD [15], a probabilisticand passive diagnosis approach for inferring the root causes ofabnormal phenomena. PAD employs a packet marking algo-rithm for efficiently constructing and dynamically maintainingthe inference model without incurring additional traffic over-head for collecting desired information. Using PAD, wedesign a number of indices to evaluate the health of CitySee(e.g., data reception ratios, total number of tasks executed,routing loops detected, traffic analysis), which are shown inFig. 6.

Through continuously analyzing the running status of City-See for the last two years, we have many interesting observa-tions. According to the link level study, we seek to answerseveral fundamental questions: what are the temporal andspatial characteristics of links, what causes link performancedegradation, and what is the impact of link performance onwireless ad hoc network routing? The interesting key findingsinclude:• The performance of intermediate links is the most unpre-

dictable, and some links exhibit highly periodic patterns.• The width of the reception “transitional region” is much

larger than those reported in previous experiments, indicat-ing that an outdoor environment might have a greaterimpact on the link performance.

• Differing from previously reported results, link performancedegradation has a relatively weak correlation with the cor-responding RSSI values fluctuating near the noise floor.

• Although individual links exhibit high dynamics, the propor-tions of good, intermediate, and poor links are fairly stableover time.

• Low-performance links are often wrongly selected for rout-ing due to the inefficiency of current routing algorithms,rather than performance degradation of the majority oflinks.

Mobility and LocalizationAs mentioned earlier, CitySee is not simply designed as a wire-less sensor network, and we put much effort into its scalabilityand compatibility. For instance, besides the static deployedsensor nodes, we design mobile sink nodes (handsets based onARM and TelosB) to conduct mobile data collection. In addi-tion, by introducing the concept of node localizability andanalyzing the conditions for a node being uniquely positionedunder a certain sensor network topology, we utilize the infras-tructure of sensor nodes and mesh nodes of CitySee to providelocalization services to mobile users as well. Furthermore, weare combining one of our ongoing projects LiFS [8], a local-ization system based on off-the-shelf WiFi infrastructure andmobile phones, with CitySee in order to provide more accuratelocalization services to users. Our main idea is to use alldeployed nodes to provide the calibration of fingerprints in acrowdsourcing and automatic manner, as we did in LiFS. Con-


Figure 6. Data for network diagnosis and management: a) num-ber of parent changes; b) radio duty cycle; c) number of packetstransmitted.

400

0

550600

650700

750800

500010000150002000025000 25000

20000

15000

10000

5000

0

450

(a)

(b)

(c)

500 550 600 650

400

550600

650700

750800

0.070.060.050.040.030.020.01

0

0.08 0.080.070.060.050.040.030.020.010

450 500 550 600 650

400

550600

650700

750800

250000200000150000100000

50000

300000 300000

250000

200000

150000

100000

50000

450 500 550 600 650


sidering that the energy issue of WSNs is nontrivial, we enableboth sensor nodes and mesh nodes to be powered by solarenergy directly. With energy-harvesting technology, we areable to dig up more about the routing and other protocols inWSNs, which enhances the Sensing IaaS ability of CitySee.

ConclusionsWe present CitySee, an environment-monitoring system usinga large-scale WSN in an urban area in Wuxi, China. Webriefly go through the solutions of CitySee with respect tohardware design, software design, network management anddiagnosis, and pervasive services.

With the increment of mobile users, mobile crowdsourcinghas received much attention. Since both sensor deploymentand network maintenance often incur unacceptable cost, webelieve that to partially use the existing devices in people’shands under the concept of crowdsourcing and participatorysensing is going to be one of the further directions. Further-more, as WSN and Internet of Things techniques becomemore mature, sensing is expected to be industry-oriented andprogrammer-friendly (not just for domain professionals).Therefore, we have been enhancing and transforming theCitySee project into a service (i.e., sensing as a service) thatfacilitates its users/customers. In other words, CitySee is evolv-ing toward an innovative, pervasive, and easy-to-use cloud ser-vice platform.

AcknowledgmentThis work is supported by the NSFC Major Program61190110, the NSFC Distinguished Young Scholars Programunder grant 61125202, NSFC under grant 61272426, ChinaPostdoctoral Science Foundation funded project under grant2012M510029 and 2013T60119, National High-Tech R&DProgram of China (863) under grant 2011AA010100, andChina 973 Program under grant 2011CB302705.

References[1] G. Churkina, “Modeling the Carbon Cycle Of Urban Systems,” Ecological

Modelling, vol. 216, no. 2, 2008, 107–13. [2] A. Chédin et al., “The Feasibility of Monitoring CO2 from High-Resolution

Infrared Sounders,” J. Geophysical Research: Atmospheres, vol. 108, no.D2, 2003, pp. 1984–2012.

[3] X. Mao et al., “Citysee: Urban CO2 Monitoring with Sensors,” Proc. 31thIEEE Int’l. Conf. Comp. Commun., 2012, pp. 1611–19.

[4] W. Gong et al., “Quality of Interaction for Sensor Network Energy-Effi-cient Management,” Computer J., 2013.

[5] J. Wang et al., “Qof: Towards Comprehensive Path Quality Measurementin Wireless Sensor Networks,” Proc. 30th IEEE Int’l. Conf. Comp. Com-mun., 2011, pp. 775–83.

[6] Y. Liu et al., “Does Wireless Sensor Network Scale? A Measurement Studyon Greenorbs,” Proc. 30th IEEE Int’l. Conf. Comp. Commun., 2011, pp.873–81.

[7] Y. Liu, K. Liu, and M. Li, “Passive Diagnosis for Wireless Sensor Net-works,” IEEE/ACM Trans. Net., 2010, vol. 18, pp. 1132–44.

[8] Z. Yang, C. Wu, and Y. Liu, “Locating in Fingerprint Space: WirelessIndoor Localization with Little Human Intervention,” Proc. 18th ACM Annu-al Int’l. Conf. Mobile Computing and Networking, 2012, pp. 269–80.

[9] A. Mainwaring et al., “Wireless Sensor Networks For Habitat Monitor-ing,” Proc. 1st ACM Int’l. Wksp. Wireless Sensor Networks and Applica-tions, 2002, pp. 88–97.

[10] P. Vicaire et al., “Achieving Long-Term Surveillance in Vigilnet,” ACMTrans. Sensor Networks, vol. 5, no. 1, 2009.

[11] L. Selavo et al., “Luster: Wireless Sensor Network for EnvironmentalResearch,” Proc. 5th ACM Int’l. Conf. Embedded Networked Sensor Sys-tems, 2007, pp. 103–16.

[12] G. Tolle et al., “A Macroscope in the Redwoods,” Proc. 3rd ACM Int’l.Conf. Embedded Networked Sensor Systems, 2005, pp. 51–63.

[13] O. Gnawali et al., “Collection Tree Protocol,” Proc. 7th ACM Conf.Embedded Networked Sensor Systems, 2009, pp. 1–14.

[14] Z. Li, Z. Zhang, and Y. Dai, “ Coarse-Grained Cloud SynchronizationMechanism Design May Lead to Severe Traffic Overuse,” IEEE TsinghuaScience and Technology, 2013.

[15] X. Miao et al., “Agnostic Diagnosis: Discovering Silent Failures in Wire-less Sensor Networks,” Proc. 30th IEEE Int’l. Conf. Comp. Commun.,2011, pp. 1548–56.

BiographiesYUNHAO LIU [SM’06] received his B.S. degree in automation from TsinghuaUniversity, Beijing, China, in 1995, and M.S. and Ph.D. degrees in computerscience and engineering from Michigan State University, East Lansing, in2003 and 2004, respectively. He is a professor with the School of Software,Tsinghua National Lab for Information Science and Technology, and directorof the MOE Key Lab for Information Security, Tsinghua University.

XUFEI MAO [M’10] ([email protected]) received his Ph.D. degree in Com-puter Science from Illinois Institute of Technology, Chicago, in 2010. Hereceived his M.S. degree (2003) in computer science and Bachelor’s degree(1999) in computer science from Northeastern University and Shenyang Uni-versity of Technology, respectively. He is currently with the School of Soft-ware, Tsinghua University. His research interests span wireless ad hocnetworks, and pervasive computing.

YUAN HE received his B.E. degree from the University of Science and Technol-ogy of China, his M.E. degree from the Institute of Software, Chinese Acade-my of Sciences, and his Ph.D. degree from Hong Kong University of Scienceand Technology. His research interests include sensor networks, peer-to-peercomputing, and pervasive computing.

KEBIN LIU received his B.S. degree from the Department of Computer Scienceat Tongji University in 2004, and his M.S. and Ph.D. degrees from ShanghaiJiaotong University in 2007 and 2010 respectively. He is currently an assis-tant researcher in the School of Software at Tsinghua University. His researchinterests include sensor networks and distributed systems.

WEI GONG received his B.S. degree from the Department of ComputerScience and Technology, Huazhong University of Science and Technolo-gy, Wuhan, China, in 2003, and his M.S. and Ph.D. degrees from theSchool of Software and Department of Computer Science and Technologyat Tsinghua University in 2007 and 2012, respectively. His research inter-ests include wireless sensor networks, RFID applications, and mobile com-puting.

JILIANG WANG received his Ph.D. degree from the Department of ComputerScience and Engineering at Hong Kong University of Science and Technology.He received his B.E. degree from the Department of Computer Science fromthe University of Science and Technology of China. He is currently with theSchool of Software at Tsinghua University. His research interest includes wire-less sensor networks, network measurement, and pervasive computing.




ehicular networks are in the progress of mergingwith the Internet to constitute a fundamental infor-mation platform, which is an indispensable part ofan intelligent transport system (ITS) [1]. This will

eventually evolve into all vehicles connected in the era of theInternet of Things (IoT) [2]. By supporting traffic-related datagathering and processing, vehicular networks are able tonotably improve transport safety, relieve traffic congestion,reduce air pollution, and enhance driving comfortability [3]. Ithas been reported that, in Western Europe, deaths due to caraccidents could be reduced 25 percent by deploying safetywarning systems at highway intersections [4]. Another exam-ple is that real-time traffic information could be collected andtransmitted to data centers for processing, and in return,information could be broadcast to drivers for route planning.City traffic congestion would be alleviated and travel timereduced, leading to greener cities.

A variety of information technologies have been developedfor intelligent vehicles, roads, and traffic infrastructures suchthat all vehicles are connected. Smart sensors and actuatorsare deployed in vehicles and roadside infrastructures for dataacquisition and decisions. Advanced communication technolo-gies are used to interconnect vehicles and roadside infrastruc-tures, and eventually access to the Internet. For instance,dedicated short-range communications (DSRC) is specificallydesigned for vehicle-to-vehicle (V2V) and vehicle-to-roadside(V2R) communications. IEEE 802.11p, called Wireless Accessin Vehicular Environments (WAVE) [5], is currently a popu-lar standard for DSRC. Besides, the Long Term Evolution(LTE), LTE-Advanced, and cognitive radio (CR) [6, 7] are allfairly competitive technologies for vehicular networking [8, 9].

Despite the well developed information technologies, thereis a significant challenge that hinders the rapid developmentof vehicular networks. Vehicles are normally constrained by

resources, including computation, storage, and radio spectrumbandwidth. Due to the requirements of small-size and low-cost hardware systems, a single vehicle has limited computa-tion and storage resources, which may result in low dataprocessing capability. On the other hand, many emergingapplications demand complex computation and large storage,including in-vehicle multimedia entertainment, vehicularsocial networking, and location-based services. It is becomingincreasingly difficult for an individual vehicle to efficientlysupport these applications. A very promising solution is toshare the computation and storage resources among all vehi-cles or physically nearby vehicles. This motivates us to studythe new paradigm of cloud-based vehicular networks.

Recently, a few research projects have been reported thatstudy the combination of cloud computing and vehicular net-works. In [10], the concept of autonomous vehicular clouds(AVCs) is proposed to exploit the underutilized resources invehicular ad hoc networks (VANETs). A platform as a service(PaaS) model is designed in [11] to support cloud services formobile vehicles. The work in [12] proposes architectures ofvehicular clouds (VCs), vehicles using clouds (VuCs), andhybrid clouds (HCs). Vehicles act as cloud service providersand clients in VCs and VuCs, respectively, and as both inHCs.

In this article, we propose a hierarchical cloud architecturefor vehicular networks. Our work is different from previousresearch in three main aspects. First, we aim to create a per-vasive cloud environment for mobile vehicles by integratingredundant physical resources in ITS infrastructures, includingdata centers, roadside units, and vehicles. The aggregation ofthese sporadic physical resources potentially compose massiveand powerful cloud resources for vehicles. Second, we pro-pose a three-layered architecture to organize the cloudresources. The layered structure allows vehicles to select their

V


AbstractIn the era of the Internet of Things, all components in intelligent transportation sys-tems will be connected to improve transport safety, relieve traffic congestion,reduce air pollution, and enhance the comfort of driving. The vision of all vehiclesconnected poses a significant challenge to the collection and storage of largeamounts of traffic-related data. In this article, we propose to integrate cloud com-puting into vehicular networks such that the vehicles can share computationresources, storage resources, and bandwidth resources. The proposed architectureincludes a vehicular cloud, a roadside cloud, and a central cloud. Then we studycloud resource allocation and virtual machine migration for effective resource man-agement in this cloud-based vehicular network. A game-theoretical approach ispresented to optimally allocate cloud resources. Virtual machine migration due tovehicle mobility is solved based on a resource reservation scheme.

Toward Cloud-Based Vehicular Networkswith Efficient Resource Management

Rong Yu, Guangdong University of TechnologyYan Zhang and Stein Gjessing, University of Oslo

Wenlong Xia, Guangdong University of TechnologyKun Yang, University of Essex

V

0890-8044/13/$25.00 © 2013 IEEE

YU_LAYOUT_Layout 1 9/20/13 12:59 PM Page 48

cloud services resiliently. Centralclouds have sufficient cloudresources but large end-to-end com-munications delay. On the contrary,roadside and vehicular clouds havelimited cloud resources but satisfycommunications quality. Third, weemphasize the efficiency, continuity,and reliability of cloud services formobile vehicles. As a consequence,efficient cloud resource manage-ment strategies are elaborately pro-posed. Countermeasures to dealwith vehicle mobility are devised.

The remainder of the article is organized as follows. Weillustrate the proposed architecture that includes vehicular,roadside, and central clouds. Cloud deployment strategies arediscussed for these three layers. We envision several promis-ing applications for sharing different resources in cloud-basedvehicular networks. We focus on cloud resource allocationproblems, and a game-theoretical approach is presented tooptimally allocate cloud resources. We study virtual machinemigration due to vehicle mobility. Illustrative results indicateoptimized resource allocation and virtual machine migrationperformance. Finally, our conclusion is presented.

Proposed Cloud-Based Vehicular NetworksArchitectureFigure 1 shows the proposed cloud architecture for vehicularnetworks. It is a hierarchical architecture that consists of threeinteracting layers: the vehicular cloud, roadside cloud, andcentral cloud. Vehicles are mobile nodes that exploit cloudresources and services:• Vehicular cloud: A local cloud established among a group of

cooperative vehicles. An intervehicle network (i.e., aVANET) is formed by V2V communications. The vehiclesin a group are viewed as mobile cloud sites and coopera-tively create a vehicular cloud.

• Roadside cloud: A local cloud established among a set ofadjacent roadside units. In a roadside cloud, there are dedi-cated local cloud servers attached to roadside units (RSUs).A vehicle accesses a roadside cloud by V2R communica-tions.

• Central cloud: A cloud established among a group of dedi-cated servers in the Internet. A vehicle accesses a centralcloud by V2R or cellular communications.This architecture has several essential advantages. First,

the architecture fully utilizes the physical resources in anentire network. From vehicles to roadside infrastructures anddata centers, the computation and storage resources are allmerged into the cloud. All clouds are accessible to all vehi-cles. Second, the hierarchical nature of the architectureallows vehicles using different communication technologies toaccess different layers of clouds accordingly. Hence, thearchitecture is flexible and compatible with heterogeneouswireless communication technologies such as DSRC,LTE/LTE-Advanced, and CR technologies. Third, the vehic-ular and roadside clouds are small-scale localized clouds.Such distributed clouds can be rapidly deployed and provideservices quickly.

Vehicular CloudIn a vehicular cloud, a group of vehicles share their computa-tion resources, storage resources, and spectrum resources.Each vehicle can access the cloud and utilize services for its

own purpose. Through cooperation in the group, the physicalresources of vehicles are dynamically scheduled on demand.The overall resource utilization is significantly enhanced.Compared to an individual vehicle, a vehicular cloud hasmuch more resources.

Due to vehicle mobility, vehicular cloud implementation isvery different from a cloud in a traditional computer network.We propose two customization strategies for vehicular clouds:generalized vehicular cloud customization (GVCC) and speci-fied vehicular cloud customization (SVCC).

In GVCC, a cloud controller is introduced in a vehicularcloud. A cloud controller is responsible for the creation,maintenance, and deletion of a vehicular cloud. All vehicleswill virtualize their physical resources and register the vir-tual resources in the cloud controller. All virtual resourcesof the vehicular cloud are scheduled by the cloud con-troller. If a vehicle needs some resources of the vehicularcloud, it should apply to the cloud controller. In contrast toGVCC, SVCC has no cloud controller. A vehicle will speci-fy some vehicles as candidate cloud sites, and directly applyfor resources from these vehicles. If the application isapproved, the corresponding vehicles become cloud sites,which will customize virtual machines (VMs) according tothe vehicle demand.

These two strategies, GVCC and SVCC, are quite differ-ent. With respect to resource management, GVCC is similarto a conventional cloud deployment strategy in which cloudresources are scheduled by a controller. A vehicle is notaware of the cloud sites where the VMs are built up. Thecloud controller should maintain the cloud resources. Dur-ing a cloud service, if a cloud site is not available due tovehicle mobility, the controller should schedule a new siteto replace it. In SVCC, since there is no cloud controller, avehicle has to select other vehicles as cloud sites and main-tain the cloud resources itself. In terms of resource utiliza-tion, GVCC is able to globally schedule and allocate allresources of a vehicular cloud. GVCC has higher resourceutilization than SVCC. However, the operation of the cloudcontroller will need extra computation. Therefore, SVCCmay be more efficient than GVCC in terms of lower systemoverhead.

Roadside CloudA roadside cloud is composed of two main parts: dedicatedlocal servers and RSUs. The dedicated local servers virtual-ize physical resources and act as a potential cloud site. RSUsprovide radio interfaces for vehicles to access the cloud. Aroadside cloud is accessible only to nearby vehicles (i.e.,those located within the radio coverage area of the cloudsite’s RSU). This fact helps us recall the concept of acloudlet. A cloudlet is a trusted resource-rich computer orcluster of computers connected to the Internet and availablefor use by nearby mobile devices [13]. In this article, we pro-pose the concept of a roadside cloudlet. A roadside cloudlet


Figure 1. Proposed cloud-based vehicular network architecture.

Roadside cloudCentral cloud

RSU

Internet

Basestation

VehiclesPrivate cloud servers

Public cloud servers

Vehicular cloud

RSU

RSU

Localservers

Localservers


refers to a small-scale roadside cloud site that offers cloudservices to bypassing vehicles. A vehicle can select a nearbyroadside cloudlet and customize a transient cloud for use.Here, we call the customized cloud a transient cloud becausethe cloud can only serve the vehicle for a while. After thevehicle moves out of the radio range of the current servingRSU, the cloud will be deleted, and the vehicle will cus-tomize a new cloud from the next roadside cloudlet in itsmoving direction.

When a vehicle customizes a transient cloud from a road-side cloudlet, it is offered by virtual resources in terms of aVM. This VM consists of two interacting components: theVM-base in the roadside cloudlet and the VM-overlay in thevehicle. A VM-base is a resource template recording thebasic structure of a VM, while a VM-overlay mainly containsthe specific resource requirements of the customized VM.Before a cloud service starts, the vehicle will send the VM-overlay to the roadside cloudlet. After combining the VM-overlay with the VM-base, the roadside cloudlet completesthe customization of a dedicated VM. During a cloud ser-vice, as the vehicle moves along the roadside, it will switchbetween different RSUs. For the continuity of cloud service,the customized VM should be synchronously transferredbetween the respective roadside cloudlets. This process isreferred to as VM migration. VM migration scenarios will befurther elaborated.

Central CloudCompared to a vehicular cloud and a roadside cloud, a cen-tral cloud has much more resources. The central cloud canbe driven by either dedicated servers in vehicular networksdata center or servers in the Internet. A central cloud ismainly used for complicated computation, massive data stor-age, and global decisions. There are already mature opensource or commercial software platforms that could beemployed for the deployment of a central cloud. Openstackis an open source cloud platform using the infrastructure asa service (IaaS) model. Other potential commercial plat-forms are Amazon Web Services, Microsoft Azure, andGoogle App Engine.

Promising Applications of Cloud-BasedVehicular NetworksWith powerful cloud computing, cloud-based vehicular net-works can support many unprecedented applications. In thissection, we illustrate potential applications and explain theexploitation of a vehicular cloud, a roadside cloud, and a cen-tral cloud to facilitate new applications.

Real-Time Navigation with Comp utation ResourceSharingIn a real-time navigation application, the computationresources in the central cloud are utilized for traffic data min-ing. Vehicles may offer services that use resources beyondtheir own computing ability. Different from traditional naviga-tion, which can only provide static geographic maps, real-timenavigation is able to offer dynamic three-dimensional mapsand adaptively optimize routes based on traffic data mining.

In Fig. 2a, vehicle A is using real-time navigation during itstravel. It first requests cloud service from the central cloudand roadside cloud. Then a VM cluster and a VM are estab-lished in the central cloud and roadside cloud, respectively.VM cluster-A in the central cloud is in charge of traffic datamining and suggests several routes based on the current trafficconditions. Once a route is selected by A, real-time navigationstarts. VM-A in the roadside cloud acts as an agent to pushmessages to vehicle A, updating the driver with traffic condi-tions on the road. As vehicle A moves on, VM-A will migrateto different roadside cloud sites. During the entire trip, VMcluster-A in the central cloud keeps updating the route infor-mation based on real-time traffic conditions. Once there is anunexpected event (e.g., traffic congestion), VM cluster-A willreport the situation quickly and compute a new route.

Video Surveillance with Storage Resource SharingVideo surveillance is an important application that utilizesshared storage resources. Currently, many buses in cities haveinstalled high-definition (HD) camera systems to monitor in-bus conditions. A very large-volume hard drive is needed tostore video content for a couple of days. This video storagescheme has several disadvantages. First, to save HD videocontent for days, the hard drive should have very large stor-age, which leads to high cost and big size. Second, video con-tent can only be checked in an offline manner, so theDepartment of Transportation is not able to make timely andproper decisions immediately after an accident. In cloud-based vehicular networks, a new distributed storage paradigmcan address this problem. The storage capability of in-busvideo camera systems is significantly extended.

In Fig. 2b, bus A exploits the roadside cloud to facilitatestorage of in-bus video surveillance content. Specifically, thebus applies for cloud services and receives a VM in the road-side cloud. The video content is uploaded to guest VM-A inroadside cloudlet-1 in a real-time manner. When the busmoves along the road and is located in the coverage area ofroadside cloudlet-2, VM-A will be migrated accordingly. As aresult, the video content is divided into several segments andseparately stored in different roadside cloudlets along the


Figure 2. Applications of cloud-based vehicular network: a) traffic data mining for real-time navigation; b) distributed storage in videosurveillance; c) cooperative download of a large file.

(a) (b) (c)

Roadsidecloudlet-1

Roadsidecloudlet-1Roadside

cloudlet-1Internet

Roadsidecloudlet-2

Roadsidecloudlet-2

AA

B

CFile

segment-1

Videosegment-1Video

segment-2Local mapand route

Navigation agent

Traffic data mining

Route

Cooperativedownload

Video storage

Video storage

Cooperativedownload

Filesegment-3Roadsidecloudlet-2

Guest VM-A

Guest VMcluster-A

Guest VM-A

Guest VM-A

Video surveillanceapplication

File

segm

ent-2

Navigationapplication

Guest VM-A

Guest VM-A

Large-volume file

File downloadapplication


road. The video segments in the roadside cloudlets will betransmitted to a data center on demand. When an accident isreported, the Department of Transportation can request road-side cloudlets to send back video to the data center.

Cooperative Download/Upload with BandwidthSharingCooperative downloading and uploading services are interest-ing applications that share bandwidth resources. Many newapplications involve large-volume data upload or download.Typical examples include in-vehicle multimedia entertain-ment, location-based rich media advertisements, and big-sizeemail services. Due to limited wireless bandwidth and vehiclemovement, it is very difficult to download an entire large filefrom a specific RSU. While the vehicle drives by, there is notenough time to complete the download of large amounts ofdata. Here, we illustrate that usage of a vehicular cloud willmake such applications feasible.

In Fig. 2c, vehicle A is going to download a large file froma roadside infrastructure. The cooperative downloading hastwo phases. In the first phase, vehicle A observes neighboringvehicles B and C, and then sets up a vehicular cloud for coop-erative downloading. Then a guest VM is constructed in bothB and C. File downloading is carried out by the vehicularcloud that consists of vehicle A and the two VMs on B and C.Since the file is downloaded by three vehicles in parallel, thetotal transmission rate becomes much faster. In this way, vehi-cle A has a high possibility to finish downloading before mov-ing out of the range of the roadside infrastructure. In thesecond phase, the VMs in B and C further cooperativelytransmit two separate segments of the file to A. Since onlyV2V communications is involved, the second phase can beperformed without the roadside infrastructure. After that, Awill reassemble the file segments into an entire file.

Table 1 summarizes potential applications in cloud-basedvehicular networks. We also show the relevant cloud resourcesharing in each application.

A Game-Theoretical Approach to Resource AllocationVehicle and roadside clouds are both resource-intensive com-ponents. Resource management is very crucial for these twotypes of clouds. Resources in vehicle and roadside clouds arerepresented in the form of VMs. In the literature, VMresearch has been studied mainly in computer networks. In a

recent study [14], VM migration is considered for dynamicresource management in cloud environments. In [15], VMreplication and scheduling are intelligently combined for VMmigration across wide area network environments. However,there are few studies on VM resource management in mobilecloud environments. In [13], the cloudlet is discussed and cus-tomized in the mobile computing environments.

In this section, we mainly focus on VM resource allocationin vehicular and roadside clouds. In a roadside cloud, thereare multiple VMs since a cloud site provides services to sever-al vehicles simultaneously. In this case, the resources in acloud site should be appropriately allocated. VM resourceallocation should consider several aspects:• Efficiency: VM resource allocation strategy should be effi-

cient such that the limited resources are fully utilized.• Quality of service (QoS): The resources allocated to a spe-

cific VM should be sufficient for the accomplishment of theVM’s tasks to achieve its QoS requirements.

• Fairness: VMs with the same workload should be offeredstatistically equal resources. Here, we formulate the compe-tition among VMs for cloud resources as a non-cooperativegame.

Game-Theoretical ModelConsider a roadside cloudlet with N VMs (i.e., the players ofthe game). The VMs will apply to the cloud site and competefor resources. These VMs are selfish in the sense that theyaim to obtain as many resources as possible for their ownusage. The cloud will allocate the total available resources tothe VMs in proportion to the number of requested resources.

Let C and M represent the total available computation andstorage resources of the cloud site, respectively. Let ci (0 < ci≤ C) and mi (0 < mi ≤ M) denote the number of requestedresources from the ith VM in computation and storage,respectively. Define c–i = SN

n=1,nπicn and m–i = SNn=1,nπimn.

The ith VM will be allocated computation and storageresources

respectively. For the sake of fairness, the cloud site sets uptwo virtual resource counters (VRCs) for each VM. Thesetwo VRCs are used to record the accumulative number ofapplied resources, one for computation and the other for stor-

+ +− −

c C

c c

m M

m m and ,i

i i

i

i i


Table 1. Applications of cloud-based vehicular networks.

Potential applications

Relevant cloud assistance Resource sharing

Centralcloud

Roadsidecloud

Vehicularcloud Computation Storage Bandwidth

Real-time traffic condition analysis and broadcast

Real-time car navigation

Video surveillance

LBS commercial advertisement

Mobile social networking

In-vehicle multimedia entertainment

Intervehicle video and audio communications

Remote vehicle diagnosis

age. When a VRC reaches its maximal value, the VM is notallowed to apply for that type of resource. By using VRCs, thetotal amount of allocated resources are equal for all VMsfrom a long-term perspective. Let ai and bi (ai > 0, bi > 0),respectively, denote the predefined resource weights that indi-cate the importance of computation and storage resources inthe workloads of the ith VM, and let li and gi (li > 0, gi > 0)denote the pricing factors associated with applied computa-tion and storage resources, respectively, of the ith VM. Theutility function, or payoff, for the ith VM is given by

(1)

The proposed game-theoretical model is specially devisedfor mobile cloud applications in vehicular networks. In partic-ular, the resource weights a i and b i in the utility functionmake the game model adaptable to resources preference indifferent applications. The pricing factors li and gi are set toprevent resource waste imposed by excessive competition, andthus potentially enhance resource utilization. These keyparameters ai, bi, li, and gi are elaborately selected regardingthe mobile environment of the cloud-assisted vehicles. Forexample, vehicles may have different quality of radio links tothe cloud site. Their VMs should be provided with differentai, bi, li, and gi according to the link quality. Typically, in amobile multimedia application where scalable video coding(SVC) technique is involved, the VM is responsible for adap-tive video decoding in the cloud site. The required VMresource mostly depends on the link quality. Because the linkrate restricts the affordable quality of a video stream, it conse-quently determines the amount of VM resources for videoprocessing.

Nash EquilibriumIn a non-cooperative game, a Nash equilibrium is a balancedstate with a strategy profile from which no game player hasany incentive to deviate. In the proposed VM resource alloca-tion game, by computing the second order derivative of U(ci,mi) with respect to ci and mi, respectively, we get

This means that U(ci, mi) is a concave function with respect toci or mi. Therefore, the existence of a Nash equilibrium isproven in the VM resource allocation game model [16]. Given

the other VMs’ applications, say, c–i and m–i, we define (c*i,m*i) Œ arg max U(ci, mi) as the best response, or the optimalstrategy of the ith VM in each iteration. We have

(2)

To prove the uniqueness of Nash equilibrium in the VMresource allocation game, we can validate that the bestresponse function is a standard function, which has three fea-tures: positivity, monotonicity and scalability [16]. FollowingEq. 2, it is easy to prove that the sufficient conditions for theuniqueness of Nash equilibrium are i, ai ≥ 4(N – 1)li and bi≥ 4(N – 1)gi.

Figure 3 shows a numerical example of resource allocationin our game-theoretical model. In the example, there arethree VMs in a roadside cloud. The total available resourcesin computation and storage are set to 50 and 100 units,respectively. The VMs have different resource demands. VM-1 has the highest demand on computation, while VM-2 hasthe highest demand on storage. We randomly select initial val-ues of the resource applications for the three VMs, say, c{1,2,3}= {10, 5, 5} and m{1,2,3} ={5, 15, 10}. In the simulation, it isobserved that the game iteration converges fast. The gamereaches its Nash equilibrium after nearly 10 rounds of itera-tions. Results indicate that the resources are appropriatelyallocated based on demand. In particular, VM-1, VM-2, andVM-3 are allocated 21.4, 14.3, and 14.3 units of computation,respectively. VM-1, VM-2, and VM-3 are allocated 31.1, 37.8,and 31.1 units of storage, respectively.

A Resource Reservation Scheme for VirtualMachine MigrationVirtual Machine Migration ScenariosVM migration refers to the process through which an operat-ing VM is transferred along with its applications across differ-ent physical machines. In VM migration, a VM image has tobe copied from the source to destination roadside cloudlets.Different from traditional VM migration, VM migration incloud-based vehicular networks has several different scenariosdue to different deployments of roadside clouds and vehiclemovements:• Inter-cloudlet case: In Fig. 4a, when vehicle A moves from

the coverage area of RSU-1 to that of RSU-2, a VM migra-tion is needed. Since RSU-1 and RSU-2 connect to differ-ent cloudlets, guest VM-A should be transferred fromroadside cloudlet-1 to roadside cloudlet-2. After that, A willaccess cloudlet-2 via RSU-2 to resume its service.

• Intra-cloudlet case: In Fig. 4b, vehicle A moves from thecoverage area of RSU-1 to that of RSU-2. Since these twoRSUs connect to the same roadside cloudlet, there is noneed for VM migration. However, radio handoff fromRSU-1 to RSU-2 may still take a short period. Duringhandoff, the interaction between vehicle A and guest VM-A may be temporally suspended.

• Across roadside-vehicular cloud case: In Fig. 4c, vehicle Amoves from the coverage area of RSU-2 to that of RSU-1.Before A’s movement, nodes A, C, and D are connected inan ad hoc manner. Vehicle C access the roadside cloudthrough vehicle A. The movement of A will cause the dis-

U

c

c C

c c

U

m

m M

m m

2

( )0 and

2

( )0.

i

i i

i i i

i i

i i

2

2 3

2

2 3α β∂

∂= −

+< ∂

∂= −

+<−

−

−

−

αλ

βλ

= −⎛

⎝⎜⎜

⎞

⎠⎟⎟

= −⎛

⎝⎜⎜

⎞

⎠⎟⎟

⎧

⎨

⎪⎪

⎩

⎪⎪⎪

−−

−−

c Cc C

c

m Mm M

m

min , ,

min , .

ii i

ii

ii i

ii

*

*

α β λ γ=+ +

− +− −

U c mc C

c c

m M

m mc m( , ) + ( ).i i

i i

i i

i i

i ii i i i


Figure 3. Resource allocation result in roadside cloud.

Computation

5

0

10

15

20

25

30

35

40

Storage Utility

VM-1VM-2VM-3

connection of C from the roadside cloud. In this case, guestVM-C will be transferred from the roadside cloud to thevehicle cloud in D. Then vehicle C can continue its servicethrough D.

• Across roadside-central cloud case: The scenario in Fig. 4d issimilar to that in Fig. 4c, except that there is no direct linkbetween vehicles C and D. In this case, guest VM-C has tobe migrated from the roadside cloud to the central cloud.After that, C will access the central cloud to resume its ser-vice by long-distance communications (e.g., third-/fourthgeneration, 3G/4G, cellular).

A Resource Reservation SchemeThe discussion on VM migration indicates that the VMmigration process involves resource re-allocation in theroadside cloud. If the resources of the destination cloudhave been intensively occupied, after a VM migration andresource re-allocation, some VMs may not have sufficientresources and may not even resume their services. In orderto avoid resource over-commitment, the target cloud sitehas to deny the VM migration in order to maintain the ser-vices of the existing VMs. In this case, the cloud service ofa vehicle with VM migration is said to be dropped. Toreduce service dropping, we propose a resource reservationscheme. In this scheme, a small portion of the cloud siteresources are reserved only for migrated VMs, but not forlocal VMs. When there are dedicated resources for VMmigration, the dropping rate of cloud services is significant-ly decreased.

In the proposed resource reservation scheme, resources aredivided into two categories: reserved resources and commonresources. Let Cr and Mr denote the reserved resources, and

Cc = C – Cr and Mc = M – Mr the common resources in com-putation and storage, respectively. In VM migration, a VMarrival refers to the event in which a VM is created for eithera new local or migrated VM. A VM departure refers to arequest for a VM deletion, either for an ending of VM serviceor VM migration to another cloud site. The resource reserva-tion scheme operates as follows: • Local VM arrival: When there is a request for creating a

new local VM, resource allocation will be carried out (e.g.,using the proposed game-theoretic allocation scheme).Since some of the resources are reserved, the local VMscan only share the common resources. If the resource allo-cation result satisfies all existing VMs, the new local VM isadmitted; otherwise, it is blocked.

• Local VM departure: Resource allocation is also performedwhen the service of a local VM ends or migrates to anothercloud site.

• Migrated VM arrival: Upon a request for a VM migration,the target cloud site will re-allocate resources. In this case,the reserved resources will be also taken into account.Specifically, the existing local VMs and migrated VM willshare all available resources. After re-allocation, if all theVMs (including the migrated VM) resource requests aresatisfied, VM migration is approved; otherwise, the VMmigration request is rejected.

• Migrated VM departure: Resource allocation is also per-formed when the service of a migrated VM ends, or itmigrates to another cloud site. It is noticeable that if thereis no migrated VMs in a cloud site, the resource allocationcan only use common resources. The reserved resourceswill be conserved for further usage upon another VMmigration.


Figure 4. Virtual machine migration scenarios: a) inter-cloudlet; b) intra-cloudlet; c) across roadside-vehicular cloud; d) across road-side-central cloud.

Roadsidecloudlet-2

Guest VM-A New

Guest VM-CGuest VM-D

Roadsidecloudlet

Guest VM-AGuest VM-B

Roadsidecloudlet

Centralcloud

BS

VM migration

VM migration

VM migrationNew

Guest VM-AGuest VM-BGuest VM-C

Roadsidecloudlet

B

A

A

CD

RSU-1

B

A

A C

D

RSU-1

RSU-2

RSU-2

(b)

(d)

(a)

(c)

Roadsidecloudlet-1

B

A

A

CD

RSU-1

RSU-2

New

B

A

AC

D

RSU-1

RSU-2

Guest VM-C

Guest VM-C

No need for migration

Guest VM-A

Guest VM-C

Guest VM-BGuest VM-C

...

Guest VM-BGuest VM-A


Optimal Resource ReservationWe consider K classes of VMs. Let ck and mk represent theamount of required resources by the kth class of VMs incomputation and storage, respectively. Let nk

l and nkg denote

the number of local and migrated VMs of class k, respective-ly. Suppose that the arrivals and departures of both local andmigrated VMs follow a Poisson process model. The systemstate transition may be formulated as continuous-timeMarkov process. Let nl = (n1

l, …, nkl, …, nK

l) and ng = (nlg, …,

nkg, …, nK

g). We represent the system state by s = (nl, ng) andthe state space by S. Let ps denote the steady state probabili-ty of state s. Given the arrival and departure rates of newand migrated VMs, the steady state probability matrix P ={ps|s Œ S} will be derived by a 2K-dimension Markov chainmodel.

Let Rb and Rd denote the blocking rate and dropping rate,respectively. Then a new local VM is blocked if the totalamount of required resources of the local VMs (including thenew one) exceeds that of the common resources, that is,SK

k=1nkl ck > Cc or SK

k=1nkl mk > Mc. A migrated VM is dropped

if the total amount of required resources of all VMs (includ-ing the migrated one) is more than that of all resources, thatis, SK

k=1(nkl + nk

g)ck > C or SKk=1(nk

l + nkg)mk > M. Let lk

l , mkl ,

lkg, and mk

g denote the arrival and departure rates of local andmigrated VMs; then Sb and Sd the sets of states that encounterblocking and dropping, respectively. We can derive Rb(Cr, Mr)= S sŒSb Skp slk

l , and Rd(Cr, Mr) = S sŒSd Skp slkg. Let Rc

bdenote the constraint of the blocking rate. The optimal num-ber of reserved resources is derived by solving the followingoptimization problem:

min Rd(Cr, Mr),

s.t. Rb(Cr, Mr) ≤ Rcb. (3)

Figure 5 shows a performance comparison with and with-out resource reservation. The total resources of the road-side clouds are 50 and 100 units in computation and storage,respectively. Two classes of VMs are considered. VMs ofclass 1 are mainly for computation-type applications, whichneeds 20 units in computational resources and 15 units instorage resources. VMs of class 2 are mainly for storage-type applications, which need 10 units in computationalresources and 40 units in storage resources. The two classesof VMs are assumed to have identical arrival and departurerates. We set the range of local VM arrival rate from 0.1 to

0.3, the local VM departure rate by 2.0, the arrival anddeparture rates of migrated VM by 0.05 and 0.1, respective-ly. The simulation results show that the dropping rate ofmigrated VMs is significantly reduced with resource reser-vation, which demonstrates the efficiency of our proposedmechanism.

ConclusionsIn this article, we first discuss the opportunities and chal-lenges in exploiting cloud computing in vehicular networks.Then we present a hierarchical architecture for cloud-basedvehicular networks that facilitates sharing of computationalresources, storage resources, and bandwidth resources amongvehicles. Furthermore, we focus on efficient resource manage-ment in the proposed architecture. The resource competitionamong virtual machines is formulated and solved in a game-theoretical framework. Virtual resource migration due tovehicle mobility is addressed based on a resource reservationscheme. Finally, illustrative results indicate a significant reduc-tion of the service dropping rate during virtual machinemigration.

AcknowledgmentThis research is partially supported by program of NSFC(grant no. 61370159 U1035001, U1201253, 61203117), theOpening Project of Key Lab. of Cognitive Radio and Informa-tion Processing (GUET), Ministry of Education (grant no.2011KF06), the project 217006 funded by the Research Coun-cil of Norway, the European Commission FP7 Project EVANS(grant no. 2010-269323), and the European CommissionCOST Action IC0902, IC0905 and IC1004.

References[1] M. Miche, and T. M. Bohnert, “The Internet of Vehicles or the Second Genera-

tion of Telematic Services,” ERCIM News, vol. 77, 2009, pp. 43–45. [2] ITU Strategy and Policy Unit (SPU), ITU Internet Reports 2005: The Internet

of Things, Geneva, 2005. [3] J. Chen et al., “Measuring the Performance of Movement — Assisted Cer-

tificate Revocation List Distribution in VANET,” Wireless Commun. andMobile Computing, vol. 11, no. 7, 2011, pp. 888–98.

[4] WHO, World Health Report 2002, “Reducing Risks, Promoting HealthyLife,” Geneva, Switzerland, 2002.

[5] R. A. Uzcategui and G. Acosta-Marum, “WAVE: A Tutorial,” IEEE Com-mun. Mag., vol. 47, no. 5, May 2009, pp. 126–33.

[6] R. Yu et al., “Secondary Users Cooperation in Cognitive Radio Networks:Balancing Sensing Accuracy and Efficiency,” IEEE Wireless Commun., vol.19, no. 2, Apr. 2012, pp. 2–9.

[7] S. Xie et al., “A Parallel Cooperative Spectrum Sensing in Cognitive RadioNetworks,” IEEE Trans. Vehic. Tech., vol. 59, no. 8, 2010, pp. 4079–92.

[8] T. Wang, L. Song, and Z. Han, “Coalitional Graph Games for PopularContent Distribution in Cognitive Radio VANETs,” to appear, IEEE Trans.Vehic. Tech.

[9] T. Wang et al., “Popular Content Distribution in CR-VANETs with JointSpectrum Sensing and Channel Access,” to appear, IEEE JSAC.

[10] S. Olariu, M. Eltoweissy, and M. Younis, “Towards Autonomous Vehicu-lar Clouds,” ICST Trans. Mobile Commun. and Applications, vol. 11, no.7-9, pp. 1-11, 2011.

[11] D. Bernstein, N. Vidovic, and S. Modi, “A Cloud PAAS for High Scale,Function, and Velocity Mobile Applications — With Reference Applicationas the Fully Connected Car,” Proc. 5th Int’l. Conf. Systems and NetworksCommunications (ICSNC), 2010, pp. 117–23.

[12] R. Hussain et al., “Rethinking Vehicular Communications: MergingVANET with Cloud Computing,” Proc. IEEE 4th Int’l. Conf. Cloud Comput-ing Technology and Science, 2012, pp. 606–09.

[13] M. Satyanarayanan et al., “The Case for VM-based Cloudlets in MobileComputing,” IEEE Pervasive Computing, vol. 8, no. 4, 2009.

[14] M. Mishra et al., ”Dynamic Resource Management Using Virtual MachineMigrations,” IEEE Commun. Mag., vol. 50, no. 9, 2012, pp. 34–40.

[15] S. K. Bose et al., “Cloudspider: Combining Replication with Scheduling for Optimizing Live Migration of Virtual Machines Across Wide Area Net-works,” Proc. Int’l. Symp. Cluster, Cloud and Grid Computing, 2011, pp.13–22.

[16] D. Fudenberg, J. Tirole. Game Theory, MIT Press, Cambridge, MA,1991.


Figure 5. Dropping rate in terms of local VM arrival rate.

Arrival rate of local VMs0.15

x 10-3

0.1

0.1

0

R d

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

0.2 0.25 0.3

Reservation (Rcb=0.01)

Reservation (Rcb=0.02)

No reservation


BiographiesRONG YU [S’05, M’08] ([email protected]) received his Ph.D. from TsinghuaUniversity, China, in 2007. After that, he worked in the School of Electronicand Information Engineering of South China University of Technology (SCUT).In 2010, he joined the Institute of Intelligent Information Processing at Guang-dong University of Technology (GDUT), where he is now an associate profes-sor. His research interest mainly focuses on wireless communications andnetworking, including cognitive radio, wireless sensor networks, and homenetworking. He is the co-inventor of over 10 patents and author or co-authorof over 50 international journal and conference papers. He is currently serv-ing as the Deputy Secretary General of the Internet of Things (IoT) IndustryAlliance, Guangdong, China, and the deputy head of the IoT EngineeringCenter, Guangdong, China. He is a member of the Home Networking Stan-dard Committee in China, where he leads the standardization work of threestandards.

YAN ZHANG [SM’10] ([email protected]) received a Ph.D. degree fromNanyang Technological University, Singapore. He is working with SimulaResearch Laboratory, Norway, and is an adjunct associate professor at theUniversity of Oslo, Norway. He is an Associate Editor or Guest Editor of anumber of international journals. He serves as Organizing Committee Chairfor many international conferences. His research interests include resource,mobility, spectrum, energy, and data management in wireless communicationsand networking.

STEIN GJESSING ([email protected]) is a professor of computer science in theDepartment of Informatics, University of Oslo, and an adjunct researcher atSimula Research Laboratory. He received his Ph.D. degree from the Universityof Oslo in 1985. He acted as head of the Department of Informatics for fouryears from 1987. From February 1996 to October 2001 he was the Chair-man of the national research program Distributed IT-System, founded by theResearch Council of Norway. He participated in three European funded pro-jects: Macrame, Arches, and Ascissa. His current research interests are rout-ing, transport protocols, and wireless networks, including cognitive radio andsmart grid applications.

WENLONG XIA ([email protected]) received his M.S. degree in electronicsengineering from PLA Information Engineering University, China in 2011.Now he is pursuing his M.S. degree in signal and information processingfrom GDUT, China. His research interests include vehicular wireless networks,opportunistic networks, and cloud computing.

KUN YANG [SM] ([email protected]) received his Ph.D. from the Depart-ment of Electronic and Electrical Engineering of University College London,United Kingdom. He is currently a full professor in the School of ComputerScience and Electronic Engineering, University of Essex, United Kingdom, andhead of the Network Convergence Laboratory in Essex. His main researchinterests include wireless networks/communications, fixed mobile convergence,future Internet technology, and network virtualization. He has published over150 papers in the above research areas. He serves on Editorial Boards ofboth IEEE and non-IEEE journals. He is a Fellow of IET.








ue to several recent technological advances and newconcepts, such as wireless body area networks(WBANs) and low-power wireless communications,pervasive health monitoring and management ser-

vices are becoming increasingly popular. However, efficientmanagement of the large number of monitored data collectedfrom various WBANs is an important issue for their large-scaleadoption in pervasive healthcare services. Since WBANs havelimited memory, energy, computation, and communication capa-bilities, they require a powerful and scalable high-performancecomputing and massive storage infrastructure for real-time pro-cessing and data storage, as well as for online and offline dataanalysis [1]. Mobile cloud computing (MCC) is graduallybecoming a promising technology, which provides a flexiblestack of massive computing, storage, and software services in ascalable and virtualized manner at low cost [2]. The integrationof WBANs and MCC is expected to facilitate the developmentof cost-effective, scalable, and data-driven pervasive healthcaresystems, which must be able to realize long-term health monitor-ing and data analysis of patients in different environments.

In MCC, mobile devices do not need a powerful configura-tion, such as high CPU speed or large memory capacity, sincetheir data and complicated computing modules can be storedand processed in the cloud. The seamless integration of WBANsand MCC introduces several advantages, which include:• Richer functionalities and services: With MCC capabilities, a

wider range of services, including more medical videostreaming and medical data mining (MDM) can be provid-ed to meet richer application requirements.

• Performance efficiency: Since the resource constraints mobiledevices and network bandwidth limitations in WBANs havehampered further improvement of service quality and largedeployment of mobile pervasive services, the mobile ser-vices and cloud servers are considered as a whole toenhance the performance efficiency of pervasive healthcareapplications in terms of computation, storage, communica-tions, and energy.

• Patient-centric services: Since WBANs are evolving to enable ahighly flexible and scalable infrastructure for mobile servicesassisted by cloud computing, MCC applications can bedesigned in such a way that patients can control their owndata and activities with strong privacy and security protection.

• Reinforced reliability: Since medical data from a patient isstored on multiple servers in the cloud, even if some data islost in the mobile device there are still backup copies in thecloud. Furthermore, when the battery of a mobile devicedies, the applications can still continue running in the cloudwithout interruption.Nevertheless, the research into cloud-enabled WBAN plat-

forms (also called wMCC platforms) is still in its infancy, sev-eral technical issues and challenges are to be addressed inorder to fulfill the research promises. For example, the sup-porting functionalities of cloud services need deliberate cate-gories and designs according to application requirements indifferent environments. Current studies related to wMCCplatforms focus on architectural design to realize a healthmonitoring and analysis system.

The remainder of this article is organized as follows. Wefirst discuss the importance of WBANs with MCC support.We then study WBAs architecture with MCC capability,which must be able to provide convenient, reliable, and richercloud services. We further emphasize the crucial methodolo-gies for improving the quality of service (QoS) of the wMCCplatform. Finally, we offer concluding remarks and sugges-tions for future research.

Developing WBANs with MCC CapabilityWBANs have emerged as an indispensable technology forpervasive healthcare. They collect patients’ vital signs andmovements using miniaturized wearable or implantable sen-sors and forward this information to medical servers or physi-cians [3]. Because of MCC’s elasticity, scalability, andpay-as-you-go pricing model, it can potentially provide huge

D


AbstractWith the support of mobile cloud computing, wireless body area networks can besignificantly enhanced for massive deployment of pervasive healthcare applications.However, several technical issues and challenges are associated with the integrationof WBANs and MCC. In this article, we study a cloud-enabled WBAN architectureand its applications in pervasive healthcare systems. We highlight the methodolo-gies for transmitting vital sign data to the cloud by using energy-efficient routing,cloud resource allocation, semantic interactions, and data security mechanisms.

Cloud-Enabled Wireless Body AreaNetworks for Pervasive Healthcare

Jiafu Wan and Caifeng Zou, South China University of TechnologySana Ullah, King Saud University

Chin-Feng Lai, National Chung Cheng UniversityMing Zhou, Huazhong University of Science and Technology

Xiaofei Wang, University of British Columbia

D

0890-8044/13/$25.00 © 2013 IEEE

ZHOU_LAYOUT_Layout 1 9/20/13 1:02 PM Page 56

cost savings, flexible high through-put, and ease of use of WBAN ser-vices.

New mobile applications for per-vasive healthcare can be rapidlyprovisioned and released usingMCC with minimal effort. We con-sider MCC technology to highlightsome innovative applications,including mass medical data storageand MDM for pervasive healthcarewith richer mobile video streaming,more supporting functionalities,and more reliable QoS. The devel-opment of WBANs with MCCcapability is based on two observa-tions as follows:• MCC benefits: MCC inherits many

benefits of cloud computing suchas dynamic provisioning, scalabil-ity, and ease of integration, aswell as of a mobile network, suchas seamless mobility. We candevelop and deploy numerous mobile applications for per-vasive healthcare, which can access larger and faster datastorage services and processing power from the cloud. Fur-thermore, MCC can improve the reliability and security ofmobile applications, where the data and computation canbe backed up in the medical cloud.

• MCC applications: Many mobile applications includingmobile commerce, mobile learning, and mobile gaminghave been developed for diverse MCC environments.As an example, a prototype of a mobile healthcareinformation management system based on cloud com-puting and a mobile terminal running an Android oper-ating system is being implemented. This prototypeplatform is developing services that utilize the AmazonS3 c loud storage service to manage pat ient healthrecords and medical images [4]. All these exampleshave provided some useful references, and have estab-lished the foundation for incorporating MCC capabili-ties into WBANs. Figure 1 shows the conceptual architecture of WBANs with

MCC capability. The mobile devices serve as gateways forWBANs, and access the Internet via WiFi or cellular networksto coordinate with application servers or locally make deci-sions on the offloading strategy. The mobile devices will thenoffload the healthcare tasks to the cloud accordingly. Oncethe requests from patients or mobile application servers havebeen received, the cloud controllers will schedule the health-care tasks on virtual machines (VM), which are rented byapplication service providers, and return the results. In somesituations, the application servers can also be deployed in thecloud.

A Pervasive Healthcare System with MCCCapabilityThis article designs cloud-enabled WBANs to provide threetypes of scenarios (home, hospital, or outdoor environ-ment) for ambulatory monitoring, and support a point ofcare to patients, the elderly, and infants in different envi-ronments. In this section, we propose a framework for apervasive healthcare system with MCC capability, and focuson cloud-enabled WBAN architecture and system function-alities.

Cloud-Enabled WBAN Architecture

Figure 2 depicts a framework for a pervasive healthcaresystem with MCC capability. This system is composed offour main components: WBANs, wired/wireless transmis-sion, cloud services, and users. WBANs collect various vitalsignals such as body temperature or heart rate informationfrom wearable or implantable sensors. The collected moni-tored data are processed in the cloud and then selectivelytransmitted to the users. The medical video streaming fromcameras are transmitted to the adjacent routing equipmentvia wired or wireless transmission, and then to the cloudserver via the Internet. Cloud servers possess powerful VMresources such as CPU, memory, and network bandwidth inorder to provide all kinds of cloud services such as auto-matic diagnosis and alarm, geographical information system(GIS) services, location-based services, and medical deci-sion making (MDM). Different users such as hospitals,clinics, researchers, and even patients ubiquitously acquiremultiple cloud services by a variety of interfaces such aspersonal computers, TVs, and mobile phones. This enablesthe sharing of monitored data to authorized social net-works or medical communities to search for personalizedtrends and group patterns, offering insights into diseaseevolution, the rehabilitation process, and the effects ofdrug therapy.

While some attempts have been made to integrate WBANsand MCC to improve pervasive healthcare services, the exist-ing work has had limited success in developing a clinicallyeffective system; therefore, the full potential of this integratedtechnology remains unutilized. In the following section, weidentify several key issues we believe must be addressed inorder to enable large-scale pervasive healthcare services andapplications.

For the proposed architecture, we further stress the followingthree important aspects: the inclusion of communication stan-dards for WBANs, the use of hybrid clouds in the wMCC plat-form, and the authorized social networks for analyzing the trends.• Communication standards for WBANs: One of the critical

issues in WBANs is low power consumption, which givesrise to some constraints in the communication standards andprotocols. A number of protocols and standards are avail-able for communication between nodes in WBANs, such asBluetooth over IEEE 802.15.1, Zigbee over IEEE 802.15.4,


Figure 1. Conceptual architecture for WBANs with MCC capability.

Public cloud provider

Data centerCloud

controller

Mobile application server

Access point

Internet

Base station

Home

Hospital

Body sensor node

WBANs Transmission Cloud services

Mobile device


UWB over IEEE 802.15.6, Insteon, Z-Wave, ANT, RuBee,and radio frequency identification (RFID) [5].

• Hybrid clouds : In general, a hybrid cloud computingarchitecture can accelerate the migration from existingIT resources in hospitals to cloud computing, makefull use of WBAN resources, and reduce costs. Impor-tant medical data and applications such as GIS deploy-ment can be deployed on a loca l pr ivate c loud toguarantee security, while operations related to systemdevelopment, upgrade, and testing can be carried outon a publ ic c loud. Moreover , when there are not

enough resources on the local private cloud at thepeak load time, some work can be switched to the pub-lic cloud.

• Social networks: The wMCC platform will increasinglyincorporate the analysis of social networks into pervasivehealthcare analytics. The research contents of WBANsintroduced into social networks include not only epidemi-ological studies but also analysis modeling of patient com-munication and education, disease prevention, mentalhealth diagnosis and treatment, and the study of health-care organizations and systems.


Figure 2. A framework for a pervasive healthcare system with MCC capability.

WBA

Ns

Clo

ud s

ervi

ces

Use

rsW

ired

/wir

eles

str

ansm

issi

on

Emergency Patient Clinics Hospitals Immediatefamily

Social network

Public data

<hospitalID> <name>*</name> <tel>*</tel> <location>*</location> <level>*</level>...</hospitalID>......

<patientID> <name>*</name> <age>*</age> <tel>*</tel> <status>*</status><location>*</location>...</patientID>......

User interfacePublic cloudPrivate cloud

Private dataMDM

Firewall

Non-sensitivedata

Medicalknowledge

sharing

Automaticdiagnosis and

alarm

Real-timemonitoring ofpatient status

Location-based services,

GIS services

Internet

GPRS3Gwired or wireless(e.g., UWB or 60GHzmillimeter wave)

WLANBody sensor

Hospital Home Outdoor environment

User interface

Datarequest

Dataresponse

Cloud-Enabled WBANFunctionalities

As shown in Fig. 2, the medical con-dition of a patient can be monitoredby the corresponding healthcare sys-tem, and subsequently updated inthe cloud by means of a smart phone,a WiFi connection, or somethingsimilar, according to the patient’slocation (e.g., home, hospital, or out-door environment). Any abnormali-ties that do not require immediatetreatment may be logged into thecloud and registered by the patient’sID for future reference. Because ofMCC support and the improvedcommunication bandwidth, doctorsor other caregivers can communicatewith patients directly by mobiledevices in the form of medical videostreaming. If needed, the patient canthen be asked to visit the healthcarefacility.

The user’s profile and medical his-tory data are maintained by the man-agement center of the local privatecloud. According to a user’s servicepriority and/or doctor’s availability, the doctor may access theuser’s information as needed. At the same time, automatednotifications can be issued to his/her relatives based on thisdata via various telecommunication means. Besides thesebasic services, the cloud services also provide GIS deploy-ment, medical data storage, MDM, virtual resource optimiza-tion management, and so on. With cloud support, the mobiledevices of medical staff will easily exhibit richer mobile videostreaming from remote cameras.

From a cloud computing research perspective, we specifi-cally consider the following aspects: massive medical datastorage, optimization and management of virtual resources,semantic interactions, and search of medical information. Thekey technologies in semantic analysis include the establish-ment of medical knowledge bases, ontology technology,semantic reasoning, capturing the real intention of patientmessages, semantic data mining technology, and hidden infor-mation discovery by means of parallel semantic reasoning.MapReduce technology and distributed file system represent-ed by the Hadoop Distributed File System (HDFS) are usedin a parallel reasoning method based on cloud computing [6].

The logic flowchart of a pervasive healthcare system withMCC capability is given in Fig. 3. We focus on supportingfunction designs for patients in different locations and usingdifferent MCC services:• Home: For the patient at home, we can obtain real-time

location information by various wireless location methodssuch as time difference of arrival (TDOA) and time ofarrival (TOA), and further determine patient activity bydata fusion technology. To provide better services, weshould install cameras connected to diverse high-speedcommunications, such as ultra wideband (UWB) and 60GHz millimeter wave, to store medical video streaming inthe cloud and provide richer multimedia contents for users.

• Outdoor environment: In general, a smart phone is used togather and deliver the patient’s physiological data informa-tion to the cloud for outdoor patients. When an accidenthappens, immediate family, a doctor, or a nurse are imme-diately informed, and they attempt rescue according to the

GPS location. Also, if a patient falls seriously ill, they canconveniently request help.

• Hospital: Using GIS for hospitals with a local private cloud,doctors or caregivers can quickly obtain the location infor-mation and physiological data of the patient by varioussmart terminals such as smart phones or personal digitalassistants (PDAs).

Research Directions for QoS Improvement We propose four research directions for QoS improvement inwMCC platforms, including the development of routing pro-tocols to support efficient data transmission to the clouds,cloud resource allocation, semantic interactions, and datasecurity and privacy mechanisms.

Reliable and Energy-Efficient Routing Protocols forWBANsReliable routing protocols for WBANs must support multihopcommunication and provide low end-to-end delay, low packetdrop rate, and low energy consumption. Because patients’conditions change continuously and may cause massive mobil-ity issues, new routing protocols may offer numerous methodsto solve these issues. After investigating proactive, reactive,and hybrid routing protocols for WBANs, we focus on tem-perature, cluster-based, and cross-layer routing solutions.

Temperature routing focuses on the effects of tissue heat-ing on the human body and their consequences during multi-hop communication. One of the approaches to reduce tissueheating is to minimize the transmission power and traffic rate.Another approach is to always avoid high-temperature nodeswhen forwarding data packets. The Thermal-Aware RoutingAlgorithm (TARA) adopts this approach, but has a high pack-et drop rate and low reliability. An alternative solution is toselect the lowest-temperature route (not necessarily the nexthop) to forward the data packets, and provide low packet lossand high reliability.

For sparse WBANs, clustering is considered as done in


Figure 3. Logic flowchart of a pervasive healthcare system with MCC capability.

Doctors, patients, andfamily get cloud services

Get body area dataof monitored patients

Get the real-timelocation informationby wireless location

methods (e.g.,TDOA, TOA)

Cameras are connectedto the high-speed

communications (e.g.,UWB and 60GHz

millimeter wave) to storemedical video streaming

in the cloud

Doctors get the locationinformation and

physiological data of thepatient by camera

video, GPS location, andGIS system

Reckon the patientactivity by data fusion

technology (e.g.,kalman filter)

Family anddoctors are

notifiedautomatically andcarry out rescue

YesYes

Use smart phonesto gather and

deliver the patient’sphysiological data

to the cloud

Yes

NoRequire

immediatetreatment?

Requireimmediatetreatment?

Login the cloud andregister by the patient’sID for future reference

Family and doctorsare informed andcarry out rescueaccording to the

GPS location

Hospital

Where isthe patient?

Requireimmediatetreatment?

No

Outdoor Home

Yes


Low Energy Adaptive ClusteringHierarchy (LEACH) architecture.Although conventional LEACHaggregates data from cluster heads, itis extremely unreliable for WBANs.Cross-layer solutions are proposed tosolve this problem. Much of theresearch on cross-layer routing solu-tions considers IEEE 802.15.4 at themedium access control (MAC) andphysical layers [7]. However, IEEE802.15.4 is not suitable for WBANsfor many reasons, including its limit-ed data rate and high energy con-sumption. The future cross-layerrouting protocol may partially consid-er the IEEE 802.15.6 MAC protocol. Because the IEEE802.15.6 MAC protocol is solely designed for WBANs, addingan efficient routing protocol at the network layer will increasenetwork performance by connecting the nodes that are in theline of sight. This may ultimately decrease the packet droprate and average energy consumption. Because these proto-cols can decrease the packet drop rate and end-to-end delay,they must be able to ensure efficient data transmission to thewMCC platform.

Cloud Resource Allocation MechanismsDeveloping scalable, energy-efficient, and cost-effective VMresource allocation mechanisms is a hot research topic. Thebasic concern of VM allocation is that a physical machinemust have sufficient capacity to host VMs [8]. Generally,two methods are used to solve the VM resource allocationproblem. The first method is to design a linear program-ming (LP) model that optimizes VM allocation using costobjectives and constraints on the resource utilization condi-tion, CPU utilization, energy consumption, and delay of ser-vices. The second approach is to use different heuristics bygenerating candidate allocation schemes and selecting thebest among them.

Semantic InteractionsIn extreme heterogeneous cloud-enabled WBANs, all kindsof resources, such as bandwidth, computing, storage, soft-ware, and data resources, are integrated to provide infor-mation and application service to users. Semantic modelscan provide a set of generic standard protocols for hetero-geneous and distributed computing [9]. It is necessary tobuild the semantic modeling analysis for pervasive health-care service in cloud-enabled WBANs in order to extract

the desired medical data from mass data, realize the realintelligence, and improve the functional portability, expan-sibility, and QoS.

It is difficult to move data from a schema-less data store toa schema-driven data store, such as a relational database,because of the uncertain platform data model. It would be asignificant advantage to set up the semantic modeling of datato provide a platform independent data representation in thepervasive healthcare cloud space. Semantic models are usefulfor pervasive healthcare cloud service from the aspects of func-tional definition, data model, service description enhancement,and so on. The information with the semantic is easy to under-stand and process for the computer in the pervasive healthcareservice. The ontology technology is used to implement infor-mation semantic interaction in cloud-enabled WBANs.

The semantic annotations can be used to generate machinecomprehensible semantics for web resources in a cloud-enabled pervasive healthcare system. Figure 4 shows that aweb service level agreement (WSLA) representation is creat-ed from an HTML webpage.

Data Security in the wMCC PlatformThe security of patient-related data is an indispensable com-ponent of the wMCC platform. In [10], Li et al. looked intotwo important data security issues for WBANs: secure anddependable distributed data storage, and fine-grained dis-tributed data access control for sensitive and private patientmedical data. In this article, since the MCC is introduced intoWBANs, some new issues on data security are emerging.

In designing a secure wMCC platform, a number of designfactors including encryption, scalability, access control, datapartitioning, user diversity, and mobile access should be con-sidered. The current research on the security of a wMCC plat-


Figure 4. Extracting the WSLA representation from an HTML webpage.

Extracted WSLA representation

Text annotations

<wsla:Contant name- “BodyTemperature” > <wsla:Float>39.8</wsla:Float></wsla:Constant>

Patient

Base-information

Bloodpressure

Vital-signs

Pulse

Contacts

SLA ontology

Age Name

If the body temperature of patient exceeds39.8degree celsius, or the pulse of patient exceeds160times per minute, the family and doctors should be notifiedautomatically to carry out rescue if necessary.

Body-temperature

Table 1. Comparison of key management approaches to mobile cloud security.

Responsibility for storageof keys Advantages Disadvantages

Centralized in the cloudprovider

Utilizes the scalable computational and networkresources of the cloud. Relies upon the directuser-to-cloud link.

Requires trust in the cloud provider to not decodeencrypted user data stored on its servers.

Centralized in a trustedauthority that is outside ofthe cloud domain

Does not require trust in the cloud provider. Maycontrol access to cloud data as an intermediarynode.

Requires maintenance of a scalable authority serverby the client, or trust in a third-party guardian as apaid service.

Fully decentralized amongusers

Requires no additional network elements. Keysharing may utilize cheap local links such as Wi-Fior Bluetooth.

Obtaining keys may require arbitration by anauthority which entails additional traffic. Revoca-tion is inefficient.

form includes key management and encrypted storage. Acomparison of the key management approaches is given inTable 1. The shortcomings of traditional key management aregiving way to advancements in completely secure cloud stor-age ideas. The goal is to encrypt medical data stored in thecloud so that the provider can access it at any time. Figure 5illustrates a typical proxy re-encryption scenario: a medicaldata owner uploads encrypted content to the medical clouddata store using a shared public key; another user requests it,and a trusted authority invokes a re-encryption process eitherwithin the medical cloud or inside the authority itself; the con-tent is then downloaded directly from the medical cloud or viathe authority, and read by the recipient using his or her pri-vate key. If the recipient’s access rights are revoked, contentwill not be re-encrypted to a form that can be read by thatuser.

ConclusionsThe seamless integration of WBANs and MCC providestremendous opportunities for pervasive healthcare systems. Inthis article, we provide a brief review and outlook of thispromising field, and discuss a cloud-enabled WBAN architec-ture for pervasive healthcare systems. In particular, we studythe functionality and reliability of MCC services. We also sug-gest some future research directions to improve performanceand QoS of cloud-enabled WBANs. We believe cloud-enabledWBANs will attract enormous attention and research effort inthe near future.

Acknowledgment This project was supported in part by grants from the Nation-al Natural Science Foundation of China (No. 61262013,11104089), and the Research Fund for the Doctoral Programof Higher Education of China (No. 20110142120095).

References[1] J. Liu et al., “Towards Key Issues of Disaster Aid Based on Wireless Body

Area Networks,” KSII Trans. Internet and Info. Sys., vol. 7, no. 5, 2013,pp. 1014–35.

[2] X. Wang et al., “AMES-Cloud: A Framework of Adaptive Mobile VideoStreaming and Efficient Social Video Sharing in the Clouds,” IEEE Trans.Multimedia, 10.1109/TMM.2013.2239630, Feb. 2013.

[3] M. Chen et al., “Body Area Networks: A Survey,” ACM/Springer MobileNetworks and Applications, vol. 16, no. 2, 2011, pp. 171-193.

[4] C. Doukas, T. Pliakas, and I. Maglogiannis, “Mobile Healthcare Informa-tion Management utilizing Cloud Computing and Android OS,” Proc. IEEEEng. Med. Bio. Soc., 2010, pp. 1037–40.

[5] M. Chen et al., “A 2G-RFID based Ehealthcare System,” IEEE WirelessCommun., vol. 17, no. 1, 2010, pp. 37–43.

[6] K. Shvachko et al., “The Hadoop Distributed File System,” Proc. IEEE 26thSymp. Mass Storage Sys. and Technologies, 2010.

[7] S. Ullah et al., “On PHY and MAC Performance in Body Sensor Net-works,” EURASIP J. Wireless Commun. and Net., 2009.

[8] P. T. Endo et al., “Resource Allocation for Distributed Cloud: Concepts andResearch Challenges,” IEEE Network, vol. 25, no. 4, 2011, pp. 42–46.

[9] J. Macías, “Enhancing Interaction Design on the Semantic Web: A CaseStudy,” IEEE Trans. Sys., Man, and Cybernetics, Part C: Applications and.Reviews, vol. 42, no. 6, 2012, pp. 1365–73.

[10] M. Li, W. Lou, and K. Ren, “Data Security and Privacy in Wireless BodyArea Networks,” IEEE Wireless Commun., vol. 17, no. 1, 2010, pp.51–58.

BiographiesJIAFU WAN [M] ([email protected]) is an associate research fellow in theSchool of Computer Science and Engineering, South China University of Tech-nology. He has authored/co-authored one book and more than 40 peer-reviewed papers. He is also Workshop Chair of M2MC ’12, M2MC ’13, andMCC ’13. His research interests include wireless body area networks, cloudcomputing, cyber-physical systems, machine-to-machine communications, andthe Internet of Things. He is a member of ACM.

SANA ULLAH [M] ([email protected]) is an assistant professor in the Collegeof Computer and Information Science, King Saud University, Riyadh. He cur-rently serves as an Editor for KSII Transactions on Internet and Information Sys-tems, Wiley’s Security and Communication Network (SCN), Journal of InternetTechnology, and others. His research interests include body area networks,wireless sensor networks, low-power communication protocols, the Internet ofThings, and cloud computing.

CHIN-FENG LAI ([email protected]) is now an assistant professor at the Depart-ment of Computer Science and Information Engineering, National ChungCheng University since 2013. His research interests include multimedia commu-nications, sensor-based healthcare, and embedded systems. After receiving hisPh.D. degree, he has authored/co-authored over 80 refereed papers in jour-nals, conferences, and workshop proceedings about his research areas withinfour years. He is also a member of the IEEE Circuits and Systems Society andIEEE Communications Society.

MING ZHOU ([email protected]) is a lecturer in the School of Energyand Power Engineering at Huazhong University of Science and Technology(HUST). He received his Ph.D. degree in control theory and control engineer-ing from the Department of Control Science and Engineering at HUST in2011. His research interests include network communication, intelligent con-trol, detection technology, and automatic equipment.

XIAOFEI WANG ([email protected]) is currently a post-doctoral research fel-low in Department of Electrical and Computer Engineering, University of BritishColumbia, Canada. He received M.S. and Ph.D degrees from the School ofComputer Science and Engineering, Seoul National University in 2008 and2013 respectively. His current research interests are social-aware multimediaservice in cloud computing, cooperative backhaul caching, and traffic offload-ing in mobile content-centric networks.

CAIFENG ZOU ([email protected]) is currently a Ph.D candidate in theSchool of Computer Science and Engineering, South China University of Tech-nology. She received her M.S. degree in computer application technologyfrom Sun Yat-sen University, Guangzhou, China, and her B.S. degree in com-puter science and technology from Shanghai University, China. Her currentresearch interests are cloud computing, cyber-physical systems, the Internet ofThings, and social computing.


Figure 5. Scenario illustrating the process of proxy medical datare-encryption.

Firewall

Wireless mediumand/or Internet

... ...

Trustedauthority

Data store

Controller

Re-encryptiontaskPublic key

directory

Private keystore

Dataowner User User








he world is moving toward a cloud computingparadigm, where mobile pervasive services will beintegrated with peoples’ daily lives. The developmentof innovative mobile pervasive services can be greatly

facilitated by publicly available cloud computing infrastruc-tures that employ virtualization technologies [1]. In order totake advantage of this facility, many organizations have start-ed to relocate their server groups and software to cloud com-puting infrastructures [2]. This new trend has provided greatadvantages, such as reduced operation expenses and energyconsumption, while achieving high utilization of computingresources.

Cloud computing infrastructures create a virtual comput-ing environment, providing service interfaces for their usersto launch applications for importing/exporting virtualmachine (VM) images with a variety of operating systems(OSs). On these public infrastructures, it is common for auser’s VMs to be collocated with other anonymous VMsbelonging to other users on the same physical machine.Also, operations of the VMs and their associated usage of

virtual resources are controlled by a shared virtual machinemonitor (VMM).

Our ultimate goal is to develop a cloud-assisted mobile per-vasive system with medical software as a service (SaaS) and itsback-end real-time application server stacks. It should storeand manage patient health records. A possible first technolog-ical evolution to this ultimate system is addressed in this arti-cle. We consider deadline-critical real-time medical datagenerated by sensor-based medical devices, such as wirelesselectrocardiogram (ECG), as an example of a need for a morestreamlined computing platform. In order to handle the time-sensitive and mission-critical medical data in a public cloudcomputing infrastructure, a real-time application (RTA) serv-er is required and should be operated as a VM.

However, before this goal can be successfully realized,there are issues that need to be resolved due mainly to thefact that the amount of data generated by a sensor-basedmedical device tends to fluctuate over time, depending on thephysical condition of a patient. Generally, multiple sensors areattached to each medical device, and once a medical devicedetects an abnormal event, it is supposed to dramaticallyincrease its data transmission rate to accommodate data fromthe sensors detecting the aforementioned abnormalities. Theserver side of the platform then has to launch a new virtual

T


AbstractCloud computing with virtualization technologies has become an important trend inthe information technology industry. Due to its salient features of reliability and costeffectiveness, cloud computing has changed the paradigms of development formobile pervasive services, effectively permeating the market. While most types ofbest effort mobile pervasive applications can be seamlessly migrated to cloud com-puting infrastructures, we need to consider specialized elements to make cloudcomputing infrastructures more effective in real-time healthcare applications. Theclient side of those applications dramatically increases its transmission rate whenev-er it detects an abnormal event. However, the existing server side mechanismshave limitations in adaptively allocating necessary computing resources in order tohandle these various data volumes over time. In this article, we propose a novelserver-side auto-scaling mechanism to autonomously allocate virtual resources onan on-demand basis. The mechanism is tested in an Amazon EC2, and the resultsshow how the proposed mechanism can efficiently scale up and down the virtualresources, depending on the volume of requested real-time tasks.

An Auto-Scaling Mechanism forVirtual Resources to Support Mobile, Pervasive,

Real-Time Healthcare Applications inCloud Computing

Yong Woon Ahn and Albert M. K. Cheng, University of HoustonJinsuk Baek, Winston-Salem State University

Minho Jo, Korea UniversityHsiao-Hwa Chen, National Cheng Kung University

T

0890-8044/13/$25.00 © 2013 IEEE

Minho Jo and Hsiao-Hwa Chen are the corresponding authors for thisarticle.

CHEN_LAYOUT_Layout 1 9/20/13 1:00 PM Page 62

RTA server to process the increased data volume. However,this process always introduces a delay to load disk images tothe new VM due to boot-up latency. While delay varies induration depending on what OS and software are loaded froma disk image, the pending real-time tasks cannot be processeduntil the boot-up process is completed.

Therefore, we cannot consider a fair resource sharingmechanism available at existing cloud computing infrastruc-tures to support our target system because such a solutionevenly assigns the limited virtual hardware resources to everyreal-time and non-real-time VM. Of course, scaling mecha-nisms at a certain level are supported by some of the publiccloud computing infrastructures, functioning to scale up ordown the amount of virtual resources by taking into accountthe current data volume. Let us first acknowledge that thosemechanisms are only designed to support best effort tasks,requiring a relatively conservative scaling with predefined stat-ic thresholds. In addition to this, most of these mechanismsrequire frequent human intervention in preparation for anemergency case.

In this article, we propose a novel auto-scaling mechanismin order to dynamically adjust the number of VMs to handledeadline-critical real-time data, which varies in size over time.In consequence, the resizing of the virtual resources for pro-cessing the given data is achieved on an on-demand basis. Thekey mechanism is to predict the volume of future data.Although it is not necessarily trivial to predict the exactmoment at which a large volume of data will be deliveredfrom sensor-based medical devices, most objects monitored bysensor-based devices typically show symptoms before transi-tioning to an abnormal state. The proposed mechanism isimplemented in Amazon EC2 [3], and our evaluation resultsverify that it can reliably support real-time data by efficientlyscaling up or down the number of VMs using the proposedprediction mechanism. We also need to mention that weachieve this effect without introducing any performancedegradation in other non-real time applications. That is, wedo not modify the existing virtual resource sharing modules inthe VMM to support real-time applications. Instead, weimplement an independent and specific session manager oper-ating only for the RTAs.

The rest of this article is organized as follows. First, weintroduce the existing auto-scaling mechanisms employed inpublic cloud computing infrastructures. We then explain oursystem model for the client and server sides, respectively. Next,e propose an auto-scaling mechanism that takes into accountthe size of future data volume sent by sensor-based medicaldevices. We conduct a performance evaluation of the proposedmechanism, followed by the conclusion of this article.

Related WorkIn many public cloud computing infrastructures, available vir-tual hardware resources are fairly shared by all VMs in aphysical machine. This fairness fails to support RTAs requir-ing differentiated levels of available virtual resources fromother non-real-time applications. Although some certaininfrastructure [3] provides a mechanism to statically increaseor decrease virtual resources for each VM, the allowable scal-ing period (typically several minutes) is too optimistic to sup-port RTAs. Even just a few minutes can be too risky in aperiod for sensor-based RTAs such as remote structural orpatient monitoring systems.

In order to efficiently support the RTAs with currentlyavailable hardware resources, a mechanism predicting a futuredata volume is essential. The workload prediction model wasrecently introduced in [4] for cloud services. The prediction

model was designed for best effort data generated by human-controlled behaviors without considering processing timelinessconstraints. Therefore, its simplicity introduces a limitation tobe applied to sensor-based RTAs. This more intuitiveapproach works well with best effort web services. Otherresearch [5] proposed a virtual resource scaling mechanismthat considers both timeliness and resource constraints. How-ever, the aforementioned timeliness is not adaptively deter-mined based on the dynamic transmission rate generated bysensor-based medical devices. Although the approach allowsthe deadline to be changed depending on the observed datavolume, the adjustment is still manually controlled by ahuman system administrator who has to modify a configura-tion file.

An autonomous computing system without human interven-tion was considered in [6, 7]. The system defined multiplestrategic steps to develop an autonomous system, and showedhow to apply the proposed development steps to implement aJava EE application server running in a cloud computinginfrastructure. Unfortunately, the approach is more appropri-ate for designing best effort applications in cloud computinginfrastructures, supported with limited computing capacity.Another autonomous management solution was proposed in[8]. In order to reduce the energy consumption of batterypowered user devices, this approach detects and localizesthermal hotspots in cloud data centers. Obviously, thisapproach has completely different goals and methods fromours. However, due to its real-time sensing concept, it pro-vides meaningful clues to help solve our problems.

In summary, to the best of our knowledge, no mechanismhas yet been designed to support sensor-based RTAs that alsoprocesses deadline-critical real-time data generated by medi-cal devices. More important, all aforementioned approachesneglect to consider the booting-up delay that occurs whenlaunching a new VM. Likewise, the cooling-down mechanism,which configures the proper moment to decrease the numberof running VMs, is missing. The aforementioned limitations ofthe existing approaches are taken into consideration in ourproposed system without disturbing normal operations ofother VMs in the same physical machine.

System ModelClient SideLet us consider a sensor-based medical device, such as anECG equipped with a local controller and multiple sensors asa client entity. A local controller periodically collects the sam-pled analog signals from its sensors, performs an analog-to-digital conversion, and compresses the digitized signals. Thesampling rate is dynamically managed to determine the datatransmission rate. For example, when the attached sensorsdetect an abnormal event, the local controller transmits thecollected sampled data to its local outgoing queue at anincreased transmission rate with a predetermined and speci-fied deadline. The transmission deadline represents the maxi-mum allowed time until the sampled data should betransferred to the outgoing queue of the medical device. Assuch, this deadline should be determined by comprehensivelytaking into account various delay factors, including time over-head for digitization and compression. Usually, a transmissiondeadline of data in each sampling period is set to the startingtime of the following sampling period. If no abnormal event isdetected, the sampled data does not need to be immediatelytransmitted for emergency treatment. In such a case, reliabletransmission is more important than fast transmission. Oncethe sampled data is transferred to the outgoing queue, thedata are mapped into one of the appropriated sub-queues



having a specific processing deadline dn, where n is the indexof the sub-queue. This deadline-based mapping is also man-aged by the local controller. The controller now forms a groupof consecutive real-time tasks T having the same processingdeadline d. When s different sensors are involved in datatransmission, variable T has four properties:• The deadline for a group of tasks• A task type• Network address of the device• The total amount of virtual or physical resources required

to finish T within the specific group deadlineThe server side will reference the task type to figure out virtu-al resource requirements for task T. For example, if the tasktype is set to be deadline-critical and CPU-intensive, the RTAserver will reserve more virtual CPU (VCPU) resources forthe task. Otherwise, if the task type is set to be mission-criticaland I/O intensive, the server only needs to pass task T to vir-tual I/O devices. More detailed procedures to convert thesampled raw signals to ordinary real-time tasks were discussedin [9]. Hereafter, the sensor-based medical device is referredto as a client node.

Server SideOn the server side, the RTA server parses sampled data sentby client nodes. After that, it extracts, processes, and storesthem to a shared data repository. Also, it transmits the pro-cessed data to remote client nodes if necessary.

Figure 1 shows that the RTA server installed in multipleVMs possibly coexists with multiple other independent gener-al-purpose VMs. Therefore, each VM is isolated and protect-ed from external malfunctions. A shared VMM cooperateswith a virtual resource manager to control all VMs forresource allocation purposes.

With the given architecture, our inclination is to provide anon-stop service by allowing the multiple RTA servers toshare common client sessions. Without this consideration,each client node has to attach its detailed session informationto every packet header, causing unnecessary network band-width consumption.

As such, we design an independent session manager locatedin another VM. The session manager controls and synchro-nizes the sessions of all real-time client nodes connected to

the RTA servers. This eventually allows the sampled data sentby the same client node to be processed by different RTAservers. To addressing fault tolerance and scalability issues, asession manager can be duplicated to multiple VMs.

We utilize one designated root RTA server to process thesampled data. More RTA servers will be launched and definedas child RTA servers upon receiving the request to launchmore RTA servers. The root RTA server has an incoming andoutgoing queue to buffer the requested tasks by its clientnodes. If the root RTA server does not have enough comput-ing resources to finish all of the real-time tasks within theirspecified deadlines, it assigns those tasks to its child RTAserver.

In order to check the available computing resources againsta given real-time task Ti, it calculates projected systemresponse time Ri for the real-time task Ti and compares itwith a given absolute processing deadline di. For the calcula-tion, we consider:• Expected processing time for task Ti• Waiting time to de-queue task Ti from an incoming queue• Waiting time to de-queue task Ti from an outgoing queueThe expected processing time again consists of:• Time for scheduling task Ti in a root RTA server• Time for computation for task Ti• Time for completing I/O operations for task TiTo meet the processing deadline, the projected response timeRi should be shorter than absolute time difference betweentwo consecutive absolute processing deadlines di and di+1.

Auto-Scaling MechanismDue largely to our hierarchical structure among the RTAservers, our system does not need to be governed by the cen-tralized conventional auto-scaling mechanism provided by theVMM. Instead, the root RTA server acts as an auto-scalingcontroller to launch a new child RTA server or terminate anexisting child RTA server. In order to design the auto-scalingmechanism, we partially adopt four iterative, stepwise, andfunctional concepts of autonomous computing proposed in[10], as follows:• Monitor: It collects, filters, and reports condition of the

managed resources.


Figure 1. Physical machine architecture with virtual RTA servers as guest domains.

VMM

Common best effort traffic

TT

T

Othertraffic

VM(others)

VM running a sessionmanager

Sync.session

T

Virtualresourcemanager VM

running a RTAserver

Othertraffic

Othertraffic

Clientnode

Clientnode Client

node


• Analyzer: It analyzes collected data of managed resourcesand predicts future states based on these data.

• Planner: It generates an appropriate plan to achieve techni-cal goals.

• Executer: It controls managed resources based on a recom-mended plan received from the planner.

MonitorThe simplest way to monitor resource usage of multiple RTAservers can be achieved by straightforwardly adopting adefault resource monitor [11] provided by a public cloudinfrastructure. However, the predefined optimistic monitoringinterval does not work well with our RTA servers. This isbecause in our system each RTA server may need to have dif-ferent monitoring intervals and performance metrics, whichshould be dynamically adjusted for better system-wise perfor-mance. Therefore, we implement an independent real-timeresource monitor as a sub-component on a guest OS runningin the root RTA server.

The developed monitoring module initially collects systemparameters, such as associated private and public IP address-es, associated instance IDs, initial monitoring interval, andperformance metrics. Within the monitoring system, welaunch various monitoring software such as the one reportedin [12]. The monitor can now capture and parse the on-screenresults from the software. The results are stored in a sharedknowledge repository for other functional components. Each

component has peer-to-peer communication modules torequest and respond to virtual resources usage and queuestates. Another important role of the monitoring system is tomonitor the child RTAs in the same way as the other man-aged resources. The monitored results will be used by the ana-lyzer.

AnalyzerThe analyzer predicts future states of the RTA servers basedon the collected performance metrics. It decides whether theroot RTA server will take on a new real-time task after check-ing available resources against the deadline of the task. If theroot RTA server has insufficient resource capacity to run thetask, it shifts the task to the child RTA that has the smallestnumber of buffered tasks in its incoming queue. If there is noavailable child RTA server, a root RTA server launches a newVM to run an additional child RTA server. It introducesboot-up delay D, which is the required time to launch a newVM with a guest OS image.

In order to assign the task to a new RTA, the projectedresponse time requirement for the new RTA server should berevised to include the boot-up delay. That is, the calculatedand projected system response time Ri should be even shorterthan the time difference between two consecutive absolutedeadlines, di and di+1, plus boot-up delay D. This deadlinechecking procedure is required to automatically assign real-time tasks to available RTA servers. A new child RTA servershould be launched only when absolutely necessary. However,if D is too long to satisfy the deadline requirement, a properprediction mechanism is required. Our prediction is per-formed with a moving average filter (MAF) module. Let ussuppose that Avgi is the ith moving average value of VCPUusage, CUi is the amount of the ith VCPU usage, and k is thenumber of observed intervals. When we calculate the value ofSavg by subtracting Avgi–1 from Avgi, Savg becomes the currentslope of Avgi, which will then be used to predict Avgi+1.Accordingly, if Savg is larger than zero, the root RTA serverdetermines that the tasks may require more computingresources in the next interval. Otherwise, it requires less com-puting resources.

The existence of an independent resource monitor with anadjustable monitoring interval allows us to reference real-timeVCPU usage records to predict the system states for the nextinterval more accurately. However, increasing or decreasingthe number of VMs only depending on the observed Savgwould introduce suboptimal resource utilization, because thevalue Savg is likely to be oscillated drastically even within avery short time interval. Therefore, the system should definemultiple logical states to prevent frequent but unnecessaryvariations of the number of VMs. Let N be the total numberof states that each RTA server has, and D[i] is the degree rep-resenting a current state i of a RTA server, which is rangedfrom –90° to 90°. The D[i] value can be calculated by convert-ing Savg to the angular value of each timing point.

Our system initially indicates that the root RTA server is in“State 1” in normal operational mode. In order to providemore accurate predictions, the number of transitions shouldvary depending on the value of D[i]. Let us assume that thereare M sections between –90° and 90°. If the value of D[i] isequal to or greater than w90°/M/2, but smaller than(w+1)90°/M/2, and D[i] is bigger than 0°, the analyzer movesthe state w transitions forward. If the value of D[i] is equal toor smaller than w(–90°)/M/2, but still smaller than(w+1)(–90°)/M/2, and D[i] is smaller than 0°, the analyzermoves the state w transitions backward. Figure 2 shows anexample, when there are four different sections, including F1,F2, B1, and B2.


Figure 2. State diagram for transitions of the proposed auto-scal-ing mechanism.

F2

F1

B1

B2

D[1]

D[0]

90°

0°

-90°


If the root RTA server reaches State N, it determineswhether it needs more computing resources or not by check-ing the states of other child RTA servers. If it is needed, itlaunches a new child RTA server, where the overloaded pend-ing real-time tasks will be assigned. On the other hand, if D issmaller than 0°, it now makes w backward transitions. When itreaches at State 1, it terminates one of its child VMs andintercepts the workload of the terminating child RTA. Notethat the actual launching and termination of child RTAs willbe performed in executer.

In order to figure out appropriate parameters such as N,we need to run a certain number of iterations with fourautonomous computing concepts. The parameter values caneventually be obtained by repeatedly referencing a knowl-edge repository, where the previous parameter values andhistory of missing deadlines are stored. The parameter val-ues obtained at the current iteration are passed to theplanner.

PlannerThe planner makes a plan for the next iteration based on theforwarded parameter values from the analyzer. If the analyzerreports that a root RTA server reaches State N, the plannersends a launch command to the executer. In addition to this,the planner will make a plan to assign overloaded pendingtasks to the newly launched child RTA server. If the analyzerindicates that a VM reaches State 1, the planner needs tomake a plan to terminate the child VM running the fewesttasks. The running tasks at the child RTA that are supposedto be terminated will shift to the root RTA server or anotherchild RTA server. In such a case, the system notifies theseactivities for the client node. We require that the root RTAserver take over most of these shifted tasks as long as the

resource is available because this mechanism allows the childRTA servers to have smaller workloads and be terminatedsooner. In our system, this module is implemented using AWSJava SDK [13].

ExecuterThis module executes a plan recommended by the planner. Ifthe root RTA needs to launch a new child RTA, it is executedby a remote procedure call. In the case of child RTA termina-tion, the executer must complete a similar procedure by send-ing the terminating message to the cloud. In order to usethese remote procedure calls, the executer must collect theinstance ID of the child RTA along with its private and publicIP address.

PerformanceWe set up our experiment environment in Amazon EC2 thatprovides 1.7 Gbyte memory space and moderate I/O perfor-mance. To observe the operations of the proposed auto-scal-ing mechanism, we turned off the default resource managerprovided by the VMM. Instead of actual workload generatedby the client nodes, we used a virtual workload indicatingrequired VCPU capacities to process all real-time tasks withinthe specified deadlines. It allows us to eliminate any possibleimpact of the shared VMM for our evaluation. It is necessaryto provide reliable and generic evaluation results, which canlater be applied to other publicly available cloud computinginfrastructures equipped with a general-purpose VMM. Theproposed mechanism was implemented in Java 1.6 and includ-ed in the Fedora 16 image [14]. It starts automatically as adaemon process while booting up the image.

Figure 3 shows VCPU workload requirements for 300 minto process the requested real-time tasks. As we reflect oncases of erroneous environments such as packet losses, thecurves show a faintly audible level of noise, which makes itdifficult to predict the workload shouldered by the next inter-val. The maximum VCPU capacity of each RTA is set to 50.Therefore, the first existing root RTA experiences two pend-ing groups of real-time tasks at 28 min and 196 min, respec-tively. The workload also shows two peaks. The first peakoccurs at 55 min, and approximately 68 units of the VCPUcapacity are required to process the given real-time tasks.Therefore, the root RTA has to launch a new child RTAbefore dropping the real-time tasks. The boot-up delay for the64-bit Fedora 16 image is usually about 1 min.

Figure 4 shows values D[i] with the VCPU workload. Weset the interval value k to 10, and N to 90°. As we can see, theroot RTA reaches the final state N at 36 and 198 min, and isrequired to launch a new child RTA. Since these results wereevaluated based on the prediction mechanism, we can ensurethat the mechanism allows a new RTA to be launched beforethe real-time tasks are overloaded. We also can simultaneous-ly launch multiple child RTAs by analyzing the amount ofincoming tasks in the queue.

When the analyzer detects that the root RTA reaches StateN, it calculates the number of RTAs still to be booted up. Fig-ure 5 shows the number of running child RTAs with the pro-posed auto-scaling mechanism. We can see that the root RTAlaunches the first running child RTA at about 20 min. On theother hand, the root RTA terminates all child RTAs at about93 min in order to scale down. As a result, the root RTA doesnot need to have a child RTA until the 197-min mark.

However, at 198 min, two child RTAs become available. Itis crucial to note that these two child RTAs have already beenlaunched at the 181-min mark, showing an achievement thesuccess of which is due to our prediction mechanism.


Figure 3. VCPU workload used for evaluation.

Time (min)500

50

0

Requ

ired

VC

PU c

apac

ity

100

150

200

250

300

350

400

100 150 200 250 300

Figure 4. States of root RTA server over time with a given VCPUworkload.

Time (min)500

200

Root

RTA

sta

tes

40

60

80

100

120

140

160

180

200

100 150 200 250 300


Since the workload includes unstable and nonlinear val-ues of required VCPU capacities, it is possible that the VMpool has more child RTAs than it actually needs. However,this is still acceptable to run RTA servers in the publiccloud infrastructures, due to the fact that the VM poolmaintains only a small number of VMs when the objectssensed by the client node are in a normal condition. There-fore, meeting the specific deadlines is much more importantthan saving the computing resources for RTA servers, espe-cially when time-critical real-time tasks are requested by theclient nodes.

Figure 6 shows VCPU usage of the root RTA with the pro-posed mechanism. After launching its child RTAs, the VCPUusage of the root RTA immediately drops down to around 10percent. As described in the previous section, we allow theroot RTA to process as many tasks as it can. In most cases,the child RTAs complete a relatively small amount of taskswhen compared to their root RTA. Also, the child RTAs aresupposed to be terminated as soon as their incoming queuesare empty. This can maximize VCPU utilization of the systemand minimize the number of child RTAs. Accordingly, ataround 40 min, our auto-scaling mechanism determines thatthe root RTA can process more tasks and tries to terminateits child RTA.

As shown in Fig. 5, there are two running child RTAs atthat moment. As a result of these terminations, the VCPUusage of the root RTA increases over 50 percent. Immediatelyafterward, the system detects that the tasks are overloadedonce again. The root RTA unfortunately needs to launchchild RTAs again at the 51-min mark. However, from around100 to 190 min, the workload shows stabilization due to thenormal condition of the detected object. Our system accurate-ly recognizes this condition and minimizes the number of run-ning RTAs during this period.

ConclusionWe investigate limitations of the existing scaling mecha-nisms implemented in publicly available cloud computinginfrastructures. In order to overcome the limitations, wepropose a novel auto-scaling mechanism supported by ses-sions used to support RTAs. The reliability and efficiency ofthe proposed mechanism come from cooperating with anindependent real-time resource monitor, a virtual sessionmanager, and a workload prediction algorithm. The evalua-tion was performed with workload in terms of VCPU usagein Amazon EC2 with Fedora 16 image. The results verifythat the proposed mechanism can efficiently scale the num-ber of RTA servers up and down by considering the avail-able computing resources against the given workload. In thefuture, we will define new parameter groups to consider andcategorize subject (or patient) groups to differentiate ourstate transition mechanism. For example, if a group’s severi-ty is higher than others, the RTA server’s state would bemoved to another state relatively faster with differentiatedparameters, which can be determined by physicians or medi-cal professionals.

References[1] S. Ahn et al., “Isolation Schemes of Virtual Network Platform for Cloud

Computing,” KSII Trans. Internet and Info. Sys., vol. 6, no. 11, Nov.2012, pp. 2764–83.

[2] W. Hui, C. Lin, and Y. Yang, “MediaCloud: A New Paradigm of Multime-dia Computing,” KSII Trans. Internet and Info. Sys., vol. 6, no. 5, May2012, pp. 1153–70.

[3] Amazon Elastic Compute Cloud, http://aws.amazon.com/ec2/, lastretrieved June 2013.

[4] N. Roy, A. Dubey, and A. Gokhale, “Efficient Autoscaling in the CloudUsing Predictive Models for Workload Forecasting,” Proc. 2011 IEEE Int’l.Conf. Cloud Computing, July 2011, pp. 500–07.

[5] M. Mao, J. Li, and M. Humphrey, “Cloud Auto-Scaling with Deadline andBudget Constraints,” Proc. 2010 IEEE/ACM Int’l. Conf. Grid Computing,Oct. 2010, pp. 41–48.

[6] B. Solomon et al., “Decentralized Predictive Control of Autonomic Comput-ing Environments,” Proc. 2006 Int’l. Info. and Telecommun. TechnologiesSymp., Dec. 2006, pp. 94–103.

[7] B. Solomon et al., “A Real-Time Adaptive Control of Autonomic ComputingEnvironments,” Proc. 2007 Centre for Advanced Studies Conf., Oct.2007, pp. 1–6.

[8] H. Viswanathan, E. K. Lee, and D. Pompili, “Self-Organizing SensingInfrastructure for Autonomic Management of Green Datacenters,” IEEENetwork, vol. 25, no. 4, Aug. 2011, pp. 34–40.

[9] Y. W. Ahn et al., “Improving QoS for ECG Data Transmission withEnhanced Admission Control in EDCA-Based WLANs,” Proc. IEEE GLOBE-COM, Dec. 2011, pp. 1–5.

[10] “SMART (Self Managing and Resource Tuning),” IBM Research, 2003.[11] Xen, http://www.xen.org/products/, last retrieved in June 2013.[12] Mpstat, http://www.linuxcommand.org/man pages/mpstat1.html, last

retrieved June 2013.[13] AWS Java SDK, http://aws.amazon.com/sdkforjava/, last retrieved June

2013.[14] Linux, Fedora 16, http://fedoraproject.org/, last retrieved June 2013.

BiographiesYONG WOON AHN ([email protected]) received B.S. and M.S. degrees incomputer science and engineering from the Hankuk University of ForeignStudies, Korea, in 2001 and 2003, respectively. He is currently pursuing aPh.D. degree in computer science with the Department of Computer Sci-ence, University of Houston, Texas. His current research interests includecloud computing, real-time systems, fault-tolerant computing, ubiquitouscomputing with embedded devices, and middleware for scalable networkenvironments.

ALBERT MO KIM CHENG ([email protected]) received B.A., M.S., and Ph.D.degrees, all in computer science, from the University of Texas, Austin. He is afull professor and former interim associate chair of the Department of Comput-er Science at the University of Houston, where he is also the founding directorof the Real-Time Systems Laboratory. The author of the popular textbook Real-Time Systems (Wiley), he has published over 180 refereed publications inleading venues in the area of power and reliability-aware real-time, embed-ded, and cyber-physical systems.


Figure 5. The number of child RTA servers over time.

Time (min)500

2

0

Num

ber

of r

unni

ng c

hild

RTA

s

4

6

8

10

12

100 150 200 250 300

Figure 6. VCPU usage of the root RTA.

Time (min)500

100

VC

PU u

sage

(%

)

20

30

40

50

60

70

80

90

100

100 150 200 250 300


http://aws.amazon.com/ec2/

http://www.xen.org/products/

http://aws.amazon.com/sdkforjava/

http://fedoraproject.org/



http://www.linuxcommand.org/man_pages/mpstat1.html

JINSUK BAEK ([email protected]) received B.S. and M.S. degrees in computerscience and engineering from the Hankuk University of Foreign Studies in1996 and 1998, respectively, and a Ph.D. degree in computer science fromthe University of Houston in 2004. He is currently an associate professor ofwith the Department of Computer Science, Winston-Salem State University,North Carolina. His current research interests include multimedia communica-tions, scalable reliable multicast protocols, mobile wireless communications,and network security.

MINHO JO [M’07] ([email protected]) received his Ph.D. from theDepartment of Industrial and Systems Engineering, Lehigh University, in1994. He is a professor with the College of Information and Communica-tion at Korea University, Seoul. He is the founder and Editor in-Chief ofKSII Transactions on Internet and Information Systems. He is an Editor ofIEEE Network and IEEE Wireless Communications, respectively He has pub-lished many refereed academic publications in very high-quality journals

and magazines. Areas of his current interest include cognitive radio, net-work algorithms, optimization and probability in networks, network securi-ty, wireless communications, energy efficient wireless communications,WBAN, and cloud computing.

HSIAO-HWA CHEN [S’89, M’91, SM’00, F’10] ([email protected]) iscurrently a Distinguished Professor in the Department of Engineering Science,National Cheng Kung University, Taiwan. He obtained his B.Sc. and M.Sc.degrees from Zhejiang University, China, and a Ph.D. degree from the Univer-sity of Oulu, Finland, in 1982, 1985, and 1991, respectively. He is thefounding Editor-in-Chief of Wiley’s Security and Communication NetworksJournal (www.interscience.wiley.com/journal/security). He was the recipient ofthe Best Paper award at IEEE WCNC 2008 and the IEEE Radio Communica-tions Committee Outstanding Service Award in 2008. Currently, he is alsoserving as Editor-in-Chief of IEEE Wireless Communications. He is a Fellow ofIET and a Fellow of BCS.






http://www.interscience.wiley.com/journal/security

Ad Page 13-IENYCM1463 10/7/09 3:47 PM Page 13

http://www.comsoc.org/adv


http://www.comsoc.org/commag/author-guidelines

http://mc.manuscriptcentral.com/commag-ieee

Instant Access to IEEE Publications

Start today to maximize your research potential.

Contact: [email protected]/digitalsubscriptions

Enhance your IEEE print subscription with online access to the IEEE Xplore® digital library.

Download papers the day they are published

Discover related content in IEEE Xplore

Signifi cant savings over print with an online institutional subscription

“IEEE is the umbrella thatallows us all to stay currentwith technology trends.”

Dr. Mathukumalli VidyasagarHead, Bioengineering Dept. University of Texas, Dallas



http://www.ieee.org/digitalsubscriptions

CALL FOR PAPERS

CONTEXT-AWARE NETWORKING AND COMMUNICATIONS

Context-aware communication and computing have attracted increasing attention since it allows automatic adaptation ofdevices, systems, and applications to the changing user's context. The context is the information characterizing the situation ofan entity and providing information about the present status of people, places, things and devices in the environment. An enti-ty is a person, device, place, or object relevant to the interaction between a user and an application, such as location, time, activ-ities, and services. Context awareness allows for customization or creation of the application to match the preferences of theindividual user, based on current context such as enterprise environment or home network.

A first area of interest concerns the Person Context Awareness. The recent emergence of the so-called social networks, thewidespread presence of smartphones equipped by heterogeneous sensors, such as GPS receivers, accelerometers, compasses,microphones and cameras, and the availability of geo-referenced information enable analysis of new context definitions thatmay concern individual, social, and urban scenarios. Indeed, recently, the available information may include mobility patternsof people and also physical activities (movements), physical status, and emotional conditions. This information is often acquiredand shared, in real time, by users. Allowing the reliable extraction and sharing of that information is a fundamental researchissue with important applications. It could improve the experience of individual, communities, organizations, and societies byadapting context to the environment (home, hospitals, campuses, offices, etc.).

Another area in this field deals with the Object Context Awareness. Context awareness may be implemented using quite dif-ferent aspects under different environments, conditions, and layers, such as layered context-aware architecture for middleware,context awareness for connecting entities of network components, and infrastructure (Internet protocol, handoff management,sensing, network requirements, network controls and network implementation).

This feature topic's scope will include both computing and communications networks, especially mobile computing networks.This topic will focus on more recent relevant topics, such as green context awareness (which would be supported by TechnicalSubcommittee on Green Communications and Computing [TSCGCC] of the IEEE Communications Society), context-aware secu-rity, new context-aware network architecture, and context-awareness for connecting entities (which would be supported byTechnical Committee on Satellite and Space Communications [SSC] of the IEEE Communications Society), and context-awaresocial networks.

The papers in this feature topic will focus on state-of-the-art research and emerging industry technologies in Context-AwareNetworking and Communications. We solicit papers covering various topics of interest that include, but are not limited to, thefollowing:

•Context-aware protocols, algorithms, architecture •Context-aware communications services and applications•Context-aware green communications and computing •Location-aware services and/or context-aware location tracking

networking •Context-aware messaging and/or addressing and/or routing•Context-aware modeling and analysis methods •Mobile phone sensing•Context-aware security approaches •Personal awareness in smart environments•Context-aware distributed systems •Social context understanding and/or social interaction

among peers•Context-awareness in the Internet of Things •Context-aware social networks•Context-aware semantic networking, including •Urban awareness for communications and networking

semantic Web •Social agents and avatars•Context-aware data storage and cloud computing •Virtual humans for communications and networking•Context-aware recommender systems •Standardizations and regulations for context-aware information

networking and communicationsProspective authors should follow the IEEE Communications Magazine manuscript format described in the Authors Guidelines

(http://www.comsoc.org/commag/paper-submission-guidelines). All articles to be considered for publication must be submittedthrough the IEEE Manuscript Central (http://commag-ieee.manuscriptcentral.com, select "June 2014/Context-Aware Networkingand Communications" from the drop-down menu), according to the following timetable:

SCHEDULE

Submission Deadline: November 1, 2013Notification of Acceptance: February 1, 2014Final Manuscript Due: April 1, 2014Publication Date: June 2014

SERIES EDITORS

Jinsong Wu Igor Bisio Haibo Li Ekram HossainBell Laboratories, China University of Genoa, Italy Royal Institute of Technology, Sweden Univ. of Manitoba, [email protected] [email protected] [email protected] [email protected]

Chris Gniady Massimo VallaUniversity of Arizona, USA Telecom Italia S.p.A., [email protected] [email protected]

http://www.comsoc.org/commag/paper-submission-guidelines

http://commag-ieee.manuscriptcentral.com







COMMUNICATIONSTECHNOLOGY

Visit www.wiley.com and quote code VB781* when you order

20% Discount on These New and Bestselling Books

www.twitter.com/WBComms

Sign up for email

alerts. Visit:

www.wiley.com/commstech

*Discount valid until 31/05/13

Wiley

Communications

Technology

Connect with us:

13-51102

Global Networks: Engineering, Operations

and DesignISBN: 978-1-119-94340-2

Radio Propagation Measurement and Channel Modelling

ISBN: 978-0-470-75184-8

LTE Security: Second Edition

ISBN: 978-1-118-35558-9

LTE-Advanced and Next Generation

Wireless Networks: Channel Modelling and

PropagationISBN: 978-1-119-97670-7

Modern Lens Antennas for Communications

EngineeringISBN: 978-1-118-01065-5

WiFi, WiMAX and LTE Multi-hop Mesh

Networks: Basic Communication Protocols and

Application AreasISBN: 978-0-470-48167-7

Nonlinear Inverse Problems in Imaging

ISBN: 978-0-470-66942-6

Radio Protocols for LTE and LTE-Advanced

ISBN: 978-1-118-18853-8

LTE and the Evolution to 4G Wireless: Design

and Measurement Challenges, 2nd EditionISBN: 978-1-119-96257-1

Mobile Ad Hoc Networking: The

Cutting Edge Directions, 2nd Edition

ISBN: 978-1-118-08728-2


http://www.wiley.com

http://www.twitter.com/WBComms

http://www.wiley.com/commstech

#GalaxyS4

IENYCM2374ndd 1IENYCM2374_415730 5/15/13 3:53 PM5/15/13 3:53 PM

IEE NETWORKING NET_2013_Sep

Documents

Transcript of IEE NETWORKING NET_2013_Sep