1

SDN/NFV Empowered Future IoV with EnhancedCommunication, Computing, and Caching

Weihua Zhuang, Fellow, IEEE, Qiang Ye, Member, IEEE, Feng Lyu, Member, IEEE,Nan Cheng, Member, IEEE, Ju Ren, Member, IEEE

Abstract—Internet-of-vehicles (IoV) connects vehicles, sensors,pedestrians, mobile devices, and the Internet with advancedcommunication and networking technologies, which can enhanceroad safety, improve road traffic management, and supportimmerse user experience. However, the increasing numberof vehicles and other IoV devices, high vehicle mobility,and diverse service requirements render the operation andmanagement of IoV intractable. Software-defined networking(SDN) and network function virtualization (NFV) technologiesoffer potential solutions to achieve flexible and automatednetwork management, global network optimization, and efficientnetwork resource orchestration with cost effectiveness, andare envisioned as a key enabler to future IoV. In thispaper, we provide an overview of SDN/NFV-enabled IoV, inwhich SDN/NFV technologies are leveraged to enhance theperformance of IoV and enable diverse IoV scenarios andapplications. In particular, the IoV and SDN/NFV technologiesare first introduced. Then, the state-of-the-art research worksare surveyed comprehensively, which are categorized into topicsaccording to the role that SDN/NFV technologies play inIoV, i.e., enhancing the performance of data communication,computing, and caching, respectively. Some open research issuesare discussed for future directions.

Index Terms—SDN/NFV, Internet-of-vehicles, 5G, mobile edgecomputing, resource slicing, edge caching, vehicle mobility,quality of service.

I. INTRODUCTION

As fundamental technologies for information disseminationand Internet access, communication networks haveexperienced a long-term evolution from the first generation(1G) systems to the fifth generation (5G) systems foraccommodating dramatically increased and diversified datacommunication demands. The evolving trend is getting fasterand broader in terms of the unprecedented growth speedof mobile data traffic volume and the shifting demandsfrom increasing service data rate to multi-dimensionalservice quality enhancement (e.g., bandwidth, connectivity,latency, and energy efficiency). According to the IMT-2020specifications [1], the 5G is envisaged to support a diversity

This work was supported by a research grant from the Natural Sciences andEngineering Research Council (NSERC) of Canada. (Corresponding author:Qiang Ye.)

W. Zhuang and F. Lyu are with the Department of Electrical and ComputerEngineering, University of Waterloo, Waterloo, ON, N2L 3G1, Canada(emails: {wzhuang, feng.lyu}@uwaterloo.ca).

Q. Ye is with the Department of Electrical and Computer Engineering andTechnology, Minnesota State University, Mankato, MN, USA, 56001 (email:qiang.ye@mnsu.edu).

N. Cheng is with the School of Telecom Engineering, Xidian University,Xi’an 710071, China (email: nancheng@xidian.edu.cn).

J. Ren is with the School of Computer Science and Engineering, CentralSouth University, Changsha 410083, China (email: renju@csu.edu.cn).

of use cases in three broad service categories, namelyenhanced mobile broadband (eMBB), ultra-reliable and lowlatency communications (URLLC), and massive machine-typecommunications (mMTC) [2]. The goal of eMBB is toachieve immerse experience for highly mobile users, suchas virtual reality (VR) and augmented reality (AR), byproviding higher and more stable data rates. The URLLC willsupport mission-critical applications, such as e-healthcare andindustrial automation, while the mMTC provides seamlessinterconnection of a large number of Internet-of-Things (IoT)devices. These three 5G application categories capture keyperformance indicators (KPIs) for 5G and even beyond 5G(B5G) networks.

Internet-of-vehicles (IoV) is one of typical 5G networkingscenarios to support different vehicle-related applications (e.g.,safety and infotainment, and on-board sensing and processingservices), which may require ultra-high reliability and low-latency data transmissions, high throughput, or massiveconnectivity [3]. An advanced mobile communication systemis required for IoV to enable timely information sharing andcomputing among vehicles and between vehicles and other“things” using vehicle-to-everything (V2X) communications,including vehicle-to-vehicle (V2V), vehicle-to-infrastructure(V2I), vehicle-to-pedestrian (V2P), and vehicle-to-sensor(V2S) communications. During the past decade, manyresearch and industrial standardization activities related tovehicular communications have significantly promoted thedevelopment of IoV and facilitated new vehicular andtransportation services and applications, such as real-timetraffic-aware navigation, collision warning, and infotainment.More importantly, the rapid development of IoV also bringsautonomous driving into reality [3]. However, the diversifiedservice demands of IoV applications not only require advancedvehicular communication techniques, but also pose significantchallenges on the management of communication, computingand caching resources.

In order to meet various computing requirements fromdiverse services, offloading computation tasks to the cloudhas been a dominant trend for IoV systems [4]. However,due to the increasing data traffic load in core networks, thelong computing response latency and unstable connectivityin a vehicular environment make cloud computing unsuitablefor supporting delay-sensitive and context-aware computingtasks. This consequently leads to the emergence of edgecomputing/caching that can exploit the computing andcaching capabilities of “near-the-edge” infrastructures anddevices (e.g., edge servers, base stations (BSs), and accesspoints (APs) at road side). However, the edge computing

2

requires augmented capital and operational expenditures(CapEx and OpEx) for efficiently managing different types ofsystem resources and flexibly provisioning different servicesin a complex and highly-dynamic vehicular environment[5]. For managing vehicle access to edge servers, theincreasing numbers of vehicles and distributed edge serverspose challenges on the allocation of communication andcomputation resources to meet various task requirements andbalance the computation resource distribution. Meanwhile, dueto high movement speed of vehicles causing a highly dynamicnetwork topology, task offloading needs to cope with taskmigration among different edge servers, and caching strategiesshould be tailored to improve the pre-fetching accuracy formobile vehicles [6].

Software-defined networking (SDN) is an emerging andpromising solution to simplify network-layer data trafficforwarding and optimize network-level resource orchestrationby decoupling the control plane from the data plane, whichhas been utilized along with mobile edge computing (MEC)for building advanced IoV systems [7]. It uses a centralizedcontrol module (called SDN controller) to collect networkinformation, such as the vehicle density, vehicle locationand mobility, supported service types and traffic load, andlocal resource allocation policies. With the gathered networkinformation, the SDN controller can make network-leveldecisions for radio resource allocation, access control policiesfor end vehicles, and traffic routing path configuration amongnetwork elements, including BSs/APs, edge servers, andnetwork switches in both access and core networks, to enhanceservice quality and improve the overall resource utilization.On the other hand, network function virtualization (NFV)offers a potential solution to use the virtualization technologyfor flexibly programming service functionalities, e.g., firewall,domain name system (DNS), network address translation(NAT), and video transcoding, as software instances referredto as virtual network functions (VNFs) at edge servers withoutadding significant costs [8]. The NFV can be combined withMEC to realize computation-oriented service provisioningat the network edge. By employing NFV, applications withdifferent requirements of computation-intensive functionalitiescan be flexibly supported in a cost-effective way. Theintegration of SDN and NFV can potentially achieve network-level resource allocation and flexible service provisioningto enhance the performance of IoV systems, in terms ofend-to-end quality-of-service (QoS) guarantee and servicecustomization.

In leveraging the potential advantages, SDN/NFV-enabled IoV faces technical challenges that require furtherresearch for performance enhancement techniques from thecommunication, computing, and caching perspectives. First,since the SDN controller needs to have both local and globalcontrol views, a challenge lies in how to apply SDN toIoV without significant signaling overhead in the presenceof high network dynamics. Second, with SDN/NFV-basedMEC, a set of VNFs need to be flexibly operated onedge servers at different locations near BSs to balance thecomputation burdens and enhance computation efficiency. Itbrings challenging technical issues such as how to determine

the optimal amount of offloaded computation tasks betweenedge and cloud servers, and how to optimize task migrationamong different edge servers, by taking vehicle mobilityinto consideration. Third, the existence of SDN/NFV canfacilitate content caching at the network edge to reducethe service delay and alleviate backhaul link congestion.However, further research on caching strategies is requiredfor optimizing cache deployment, improving cache hit ratio,and minimizing cache update cost.

This paper provides a comprehensive survey on varioustechniques for SDN/NFV-enabled IoV. The remainder ofthis paper includes an overview of IoV, and the state-of-the-art SDN/NFV enabling technologies for enhancing theperformance of IoV services, ranging from reliable real-timeservices and data-intensive services to computing and cachingoriented applications. We also discuss some open researchissues for further studies and draw a conclusion at the endof this survey.

II. INTERNET OF VEHICLES: AN OVERVIEW

As an evolution of vehicular ad-hoc networks (VANETs)in the era of 5G and IoT, the IoV integrates the emergingtechnologies such as cloud/edge computing, SDN, andartificial intelligence (AI) with the conventional VANETs tofurther improve road safety, efficiency, and comfort. In IoV,vehicles act as multi-sensor smart objects, equipped withcommunication and computing capabilities. The IoV shouldbe capable of sensing and processing in-car and surroundinginformation, communicating with nearby vehicles and otherIoT devices, and have IP-connectivity to the cloud serversand Internet. The advanced communication and networkingtechnologies in IoV facilitate new vehicular and transportationservices and applications.

A. Architecture and Communication Paradigms

The IoV relies on efficient, fast, reliable, and cost-effective communication systems to guarantee ubiquitousnetwork connectivity. V2X communications are a fundamentalfunction of IoV, which allow vehicles to communicatewith other vehicles, human, road sensors, transportationinfrastructure, IoT devices, access networks, and the Internet[9]. V2X communications include different types of vehicularcommunications, such as V2V, V2I, V2P, and V2S, each ofwhich employs various wireless and networking technologiesto facilitate data transmission between vehicles and othernetwork components.

The dedicated short-range communication (DSRC) andIEEE wireless access in vehicular environments (WAVE)protocol stack are the key enabler of V2V and V2Icommunications. The IEEE 802.11p was released in 2010[10], which specifies technical details of medium accesscontrol (MAC) and physical layers of WAVE. In DSRC,one distinctive feature is the distributed and uncoordinatedchannel access, thanks to the 802.11p carrier sensing multipleaccess with collision avoidance (CSMA/CA). It avoids theneed of a centralized network coordinator for access controland resource allocation and, therefore, reduces the cost and

3

improves the network scalability. However, low radio resourceutilization, especially at a high vehicle density, is evident dueto frequent channel access transmission collisions. Therefore,DSRC technologies are suitable for safety applicationsand light-weight proximate services such as advertisementbroadcasting, rather than data-craving applications.

Cellular V2X (C-V2X) is a series of advanced technologiesto provide IoV with coverage, guaranteed QoS, reducedinfrastructure deployment cost, and enhanced network security.The recent long term evolution (LTE) and 5G new radio (NR)cellular networks adopt sophisticated wireless transmissiontechnologies to enhance the network capacity, and can thusoffer high data rates to IoV applications. In addition, thecentralized resource allocation, such as orthogonal frequency-division multiple access (OFDMA), guarantees the QoS ofeach IoV node in terms of data rate, delay, and reliability,crucial in many IoV services. The international standardorganization, 3rd Generation Partnership Project (3GPP),has been working on the standardization of C-V2X, anddefining its architecture, fundamental functions, and use cases[11]. The architecture of C-V2X is also specified and twofundamental operation modes are discussed, i.e., traditionaluplink/downlink communication via Uu interface and directV2X communication via PC5 interface [12]. There are manyworks investigating the theoretical and practical performanceof C-V2X. The reliability of DSRC is shown to be better atshorter distances while LTE-V2V is more reliable at longerdistances [13]. The transmission performance measures ofDSRC and C-V2X are characterized and compared throughfield tests in Ann Arbor and San Diego. The results show thatC-V2X outperforms DSRC in terms of coverage, reliability,and interference [14].

Although C-V2X communication has high performance, itis often an expensive approach to support the exponentiallyincreasing IoV data traffic. In addition, the limited radioresources in the cellular system renders it vulnerable tonetwork congestion. In order to reduce the data servicecosts and offload the cellular IoV data traffic, alternativecommunication spectrum bands and paradigms are proposedfor IoV, which include cognitive radio (CR), roadside wirelesslocal area networks (WLANs), and vehicular device-to-device(D2D) communications. These vehicular data offloadingparadigms can employ much richer spectrum resourcesto provide cost-effective data pipes. Due to the primaryusage of radio spectrum bands, limited coverage, andvehicle mobility, these communication modes are typicallyavailable tempo-spatially, which are therefore referred to asopportunistic communication methods. For instance, TV whitespaces (TVWS) are considered as a potential solution toaccommodate the heavy IoV data traffic by employing CRtechnologies. In roadside WLANs, the communication takesplace only when the IoV terminals are within the networkcoverage area, which depends on the vehicle speed and thedeployment of roadside WLANs. Consequently, an importantresearch issue is how to efficiently harvest the availablespectrum and communication opportunities in order to improvethe network performance of IoV, which has been investigatedfor CR-IoV [15], roadside WLAN traffic offloading [16], and

vehicular D2D communications [17].Space-air-ground integrated networking (SAGIN) is another

promising solution to provide IoV with ubiquitous networkcoverage, pervasive communication and computing, scalablenetwork deployment, and flexible network services andmanagement [18]. The satellite communication network canprovide IoV users with telecommunication services, especiallyon highway or in rural areas where a terrestrial networkis not available. With technical advances in high-throughputsatellites (HTS) and low-earth orbit (LEO) constellation, thesatellite network can offload the heavy data traffic burden fromterrestrial cellular networks. Another important category ofcomponents, such as communication balloons and unmannedaerial vehicles (UAVs), forms an airborne network which canbe deployed on demand and provide flexible services to IoV.Compared to the solution of deploying ultra-dense terrestrialcellular networks [19], UAVs can be deployed according to thevarying demands, serve as flying relays, BSs, or caching unitsfor different services, which is more flexible and cost-effective[20].

With the development of intelligent transportation systems,smart city, and autonomous vehicles, a wide range ofcomputation-intensive IoV services have emerged, such asVR/AR, self-driving, and crowdsourcing. To support theseservices, a comprehensive cloud/edge computing system isrequired. Cloud computing technologies are mature, and manycommercial cloud computing services are available, includingAmazon Web Services (AWS), AliCloud, and MicrosoftArezu. IoV users can offload the computation-intensive tasksto a cloud computing platform through a variety of V2Icommunication techniques to reduce the latency and energyconsumption [21]. Yet, the latency of cloud computing canbe unacceptable for some delay-critical applications, such asautonomous driving. In this case, edge computing shouldbe utilized where the computing resources are deployedat the network edge, much closer to the IoV users thancloud computing. Recently, another novel IoV computingparadigm, referred to as vehicular cloud, has attractedmuch attention. Since connected vehicles are equipped withincreasingly powerful on-board computing capabilities, theycan form a local cloud through reliable V2V communicationsto cooperatively execute their computing tasks [22]. Toreduce the latency for content retrieval, edge caching canbe applied by caching popular contents at edge servers toimprove the network performance when data-intensive servicesare delivered. Fig. 1 shows a comprehensive networkingarchitecture of IoV systems, where V2X communications areprovided by the space, aerial, and terrestrial communicationnetworks, which can cooperatively offer seamless, efficient,and secure wireless connections to IoV users. A three-tier(vehicle-edge-cloud) computing system with edge cachingis enabled to realize flexible and cost-effective IoV serviceprovisioning.

B. IoV Applications and ServicesA wide range of IoV applications and services are at the

core of IoV, the requirements of which should be fully satisfiedby IoV sensing, communication, networking, and computing

4

Table I: Characteristics and requirements of IoV applications

Application Communication support Bandwidth Req Delay Req Computation ReqVehicle collision warning V2V, V2P low high none

Real-time navigation V2R, V2I high high lowAR/VR V2I, V2V very high high low to high

Autonomous driving V2V, V2I, V2S high very high very high

V2V

IoV Sensing

V2P V2I

IoV Sensing

Network edge

Cloud

Cellular Base Station

Infrastructures

Satellite

UAV

V2I

Network edge

Cellular Base Station

V2I

V2I

V2I

Earth station

Aerial connectionSpace

connection Edge cacheEdge computing

Edge cacheEdge computing

Cloud computing

Vehicular cloud

V2I

Fig. 1: An illustration of IoV system.

technologies. In the following, several typical IoV applicationsare described, and their requirements are listed and comparedin Table I.

1) Vehicle collision warning: Collision warning is animportant road safety application, aiming at reducing therisk of accidents caused by collisions among vehicles, roadstructures, and pedestrians. Collision warning includes manydifferent use cases, such as forward collision warning,emergency vehicle warning, signal violation warning, pre-crash sensing warning, and so on [23]. Periodical and event-driven broadcasting is usually employed to warn the nearbyvehicles with vehicle movement status, traffic condition, andpotential danger. Such an application has strict requirementson message delivery latency and reliability, ranging between50 ms and 100 ms for delay and 10−5 for maximalmessage delivery failure rate. Short-range V2V and V2Pcommunications are proper candidates for collision warningapplications.

2) Real-time vehicle traffic management: It is to manageand optimize the traffic flow in a timely manner based onreal-time vehicle traffic and road information [24]. In IoV,both vehicles and roads can be equipped with cameras,light detection and ranging (LIDAR) devices, and othersensors. These cameras and sensors can collect rich road andtraffic information, which is then transmitted to a centralizedITS server via V2I communications. A city-wide trafficoptimization is executed in the ITS server, and navigationguidelines are then disseminated to vehicles considering thetraveling time, safety, fuel, and comfort. To guarantee thereal-time traffic optimization and management, efficient IoVdata collection, traffic information transmission, and trafficmanagement optimization are required.

3) VR and AR: One important category of IoV applicationsis infotainment, i.e., to provide users with a variety ofinformation and entertainment contents and services to make

road travelling more pleasant and comfortable. Among them,VR and AR have attracted much attention since they canfacilitate immersive experience, windshield road informationdisplay, and driving assistance. However, supporting IoVVR/AR services requires ultra-high transmission rates andcomputing resources, which can pose severe challengeson the IoV infrastructure. Although cloud computing canprovide ultra-high computing capabilities, the latency is likelyunacceptable for AR/VR applications. Therefore, mobile edgecomputing/caching technologies are a potential solution tojointly provide low latency and alleviate the data traffic burdenof the backbone network [2].

4) Autonomous driving: Autonomous vehicles undoubtedlyrepresent the future of transportation systems. The Societyof Automotive Engineers (SAE) International has definedsix levels of car autonomy, from none to full automation[25]. The low levels of autonomy, i.e., driving assistancesystem, can be solely supported by the conventional V2V andV2I communications. The high levels of autonomy, such aslevel-3 and level-4, are currently in development based onindividual vehicle perception, data fusion, machine learning,and manoeuvre control. However, the individual perception-based autonomy suffers from the limited perception capabilityand the lack of vehicle cooperation, and thus cannot supportthe level-5 full autonomy. IoV and V2X communications arepromising approaches to address these limitations [26].

III. SDN/NFV FOR IOV INFRASTRUCTURE

The conventional vehicular networking architectureemploys V2V and V2I communications to interconnectdifferent vehicles for supporting both road safety applicationsand infotainment services. With an increasing number ofvehicles and the diversity of services, the overall networkresource utilization needs to be enhanced substantially in orderto accommodate differential QoS demands [27]. For example,in autonomous driving, different computation-intensiveapplications, including high-definition (HD) map downloadingand on-board sensing, require ultra-low processing andcommunication delays, whereas other data-hungry servicesdemand high throughput. To achieve more efficient resourceusage for diversified service accommodation, it is necessaryto have a multi-tier of network coverage deployment (i.e.,a macro-cell deployment underlaid by different tiers ofsmall cells) to exploit the spatial multiplexing gain ofradio resources. In addition, a variety of wireless accesstechnologies (e.g., LTE-V, DSRC) will coexist to offera wide range of radio spectrum opportunities. However,the current networking architecture, i.e., an integration ofheterogeneous wireless networking technologies with multiplelayers of BS deployment, requires distributed coordination

5

among communication infrastructures (e.g., BSs) for resourceallocation, leading to increased signaling overhead when thenumber of BSs becomes large [28]. In the core network,traffic flows of different services aggregated from thewireless access networks need to traverse different sets offunctionalities to fulfill E2E service delivery. For instance,traffic from a video streaming service may need to go througha firewall function and a transcoding function before reachingend clients for a secured E2E video downloading. Thesefunctionalities are often provided and operated in differentfunction-specific servers. When supporting more diversifiedservices, an increasing number of hardware servers are addedinto the network, forming a cloud computing platform withaugmented capital and operational expenditures.

Network topology Cell loads

Data Plane

SDN/NFV Controller: Open-Sourced Programmable Control Platform (e.g., Opendaylight, NoX)

Physical Network Infrastructures

Access control policy

Mobility management policy

Resource slicing policy

Network deployment policy

Firewall

Traffic statistics

Vehicle mobility

UAV position & trajectory

Network information

Virtualized Network Functions

Policies/Algorithms Customization

Decision making

OpenFlow (or extension ) API API

Northbound Interfaces

OBUs & RSUs resource

Policy enforcement

Application Plane

Driving Safety

Automated Driving Infotainment Traffic

Monitoring&Scheduling……

IoV Applications

Communication Computing offloading Caching

……

Policy enforcement

Control Plane Southbound Interfaces

DNS NAT Transcoding Retransmission……

Fig. 2: A layered SDN/NFV architecture supporting IoV.

By enabling a flexible and programmable networkconfiguration, SDN is a promising networking technology tosimplify network-layer data forwarding and optimize network-level resource allocation [29]. With SDN, the network controlplane is decoupled from the data plane and is migratedto a centralized controller placed in a network server. TheSDN controller is physically connected to underlying networkelements, including BSs in wireless networks and networkswitches/servers in core networks, through programmablelinks operating OpenFlow protocol [30]. In the core networks,the control function for routing configuration is migratedfrom each forwarding device to the SDN controller whichdetermines and optimizes an edge-to-edge routing path foreach service flow based on global network information. Inthis way, the underlying data forwarding structure of eachdevice is greatly simplified, and the SDN control modulecustomizes routing configurations for different service flowsto meet different QoS requirements. In the wireless networkdomain, an SDN controller can be physically installed at anedge server connecting BSs through programmable controllinks. As shown in Fig. 2, with the information collected fromunderlying physical networks (e.g., the number of vehicles,vehicle location and mobility, and data traffic load) through the

southbound application program interface (API), the controllercan make network-level decisions for radio resource slicingand access control, to improve the overall service performanceand resource utilization [31]. On the other hand, to reduceCapEx and OpEx for service provisioning, NFV is to decouplenetwork (or service) functions from underlying hardwareservers, and to represent the functions as programmablesoftware instances (i.e., VNFs) using virtualization technology[32]. This virtualization makes it possible to use genericservers instead of the conventional function-specific servers,and the VNFs managed by a centralized NFV controlmodule can be flexibly placed at different network locationswith optimal computing resource allocation to achieve costeffectiveness. As a matter of fact, NFV provides a wayof flexibly programming an increasing number of VNFsat appropriate network servers with minimal hardwareinstallation cost. This flexibility in function provisioningsupports service customization, especially in the 5G era,when newly emerging applications require different setsof computation-intensive functionalities to be installed andoperated in the network. Therefore, SDN and NFV arecomplementary technologies that can be integrated, as shownin Fig. 2, to achieve cost-effective VNF placement and service-oriented traffic routing, and network-level resource slicing withproper control of channel access from vehicles under differentIoV scenarios.

With the new SDN/NFV integrated networking architecture,the overall network performance can be enhanced interms of satisfying E2E QoS requirements (e.g., highthroughput, low latency, and high reliability) and realizing asequence of computation-intensive functionalities (e.g., dataclassification, video transcoding) with low response delayfor different services. First, a new network-level resourceallocation framework with proper BS-vehicle access controlis necessary, in which resource combinations are created andcustomized for different services to satisfy their unique QoSrequirements. This process is called resource slicing. The SDNcontroller has direct programmability on resource allocationfor heterogeneous BSs, and the process of resource slicingaims at improving the overall resource utilization. Second,with SDN/NFV control, a set of VNFs can be flexibly executedin network servers at different network locations to enhancethe computation efficiency. With the emergence of edge/fogcomputing technology, computation tasks can be offloadedfrom cloud servers to the edge servers connecting to BSsor even to end vehicles to reduce the two-way responsedelay. How to optimize computation offloading with theconsideration of high vehicle mobility needs investigation.Lastly, using SDN/NFV integrated control for content cachingat network edge becomes promising in terms of optimizingthe caching deployment and minimizing the cost of cachingcontent updates. In the following, we provide detail discussionon how SDN/NFV can enhance communication, computing,and caching for IoV applications.

IV. IOV COMMUNICATION SERVICES

To enhance road safety, transportation efficiency, andinfotainment services to users on roads, many location-based

6

CSC: Central SDN Controller

RSU Wi-Fi

RSU: Roadside Unit TVWSS: TV White Space Station CBS: Cellular Base Station Control area of LSC

DSRC Spectrum

TVWSS

CBS

TVWS Spectrum CBS SpectrumRSU Wi-Fi Spectrum

ES

CSC

RSU Wi-Fi

LSC

Satellite

UAV

Urban area Suburban area

RSU Wi-Fi

...

SatelliteLSC

ES

Core networks

LSC: Local SDN Controller

UAV Connection Satellite ConnectionES: Earth Station

Fig. 3: An SDN based IoV communication system illustration.

and Internet services are essential, which rely on real-time,reliable, and data-intensive information exchanges amongvehicles as well as other road-side equipments [33]. With theadvanced SDN techniques [34], the communication paradigmis geared for better performance in the following aspects.

A. Latency

Ensuring low-latency communication is of paramountimportance for delivering road-hazard messages, which areexchanged among nearby vehicles and RSUs. The 3GPPstandard (release 15) has specified latency requirements tosupport V2X services. Specifically, for vehicular cooperationapplications in terms of platooning information exchange,sensor information sharing, and information sharing forautomated driving, the end-to-end latency levels are requiredto be lower than 20 ms, 50 ms, and 100 ms, respectively [35],which is challenging to achieve in highly-dynamic vehicularenvironments. The delivery latency mainly consists of theover-the-air transmission delay and channel access delay. Asthe transmission delay is deterministic when the data rate isgiven, and negligible due to the high light traveling speed, theexisting studies in the VANET literature focus primarily onguaranteeing channel access delay [36].

In the literature, the low-latency communications have beenstudied in terms of balancing the tradeoff between throughputand delay, such as the delay-constrained link capacity andthe use of effective network capacity [37], [38]. These worksaim at minimizing the long-term time-averaged latency, whilefalling short of guaranteeing the tail and worst case latency,which may fail to deliver critical messages for danger warning.To essentially guarantee the access delay for delivering road-hazard messages by broadcasting, efficient MAC protocoldesign is requisite [39]. Even though IEEE 802.11p-basedMAC protocol has been standardized, it suffers from longaccess delay and unfairness in support of periodical broadcastdue to the contention-based distributed random access [10].To overcome the limitations, time division multiple access(TDMA)-based MAC protocols are proposed to supportperiodical broadcasting [40], [41]. The pre-defined frame andslot channel structure naturally support delay guarantee forurgent messages. Based on the time-slotted medium structure,

other centralized and distributed TDMA-based MAC protocols[40], [42] are proposed for efficient time slot assignment andtransmission collision avoidance.

Advanced SDN techniques potentially can help to reducethe random access delay when a vehicle fails to obtain ashare of medium resources after contending or negotiating.First, by applying the network softwarization and slicing, avirtual dedicated network can be created and customized forthe periodical broadcast service, whereby other bandwidth-intensive services are completely isolated out. Hence, thedelay performance of broadcasting emergency messages isindependent of the bursty heavy loads from other bandwidth-hungry services. Even though the resource utilization in thededicated network may degrade without multiplexing withother services in resource usage, it is worthwhile as fastchannel access has high priority in road safety applications. Toenable isolated V2X services, Campolo et al. justify the roleof network slicing and provide suggestions to design dedicatedV2X slices [43]. Ye et al. present a dynamic radio resourceslicing framework for a two-tier heterogeneous wirelessnetwork to determine the optimal bandwidth slicing ratios[31]. In a vehicular scenario, the challenge is to guaranteethe stable performance for each network slice as vehiclesmove dynamically, while the allocation of physical radioresources is relatively stationary in interference managementwith frequency reuse, leading to the mismatch betweensupply and demand. Therefore, to deal with the mismatch,a dynamic and observant algorithm is in need to update thebandwidth slicing ratios frequently, in order to achieve bothQoS satisfaction and efficient resource management. Given theavailable spectrum resources in the dedicated virtual network,the SDN controller can be utilized to control the mediumaccess from vehicles, and dynamically update the lengthof time frames and the time slot assignment parameters inaccordance with vehicle density dynamics in both time andspace domains.

B. Reliability

Communication reliability is another hurdle to overcomefor IoV service satisfaction, especially for safety and onlinevehicle-traffic management applications. The wireless channel

7

fading and network topology vary dramatically in a vehicularenvironment. For safety message delivery, the reliabilityis generally measured in the probability that a packet issuccessfully received by the target vehicle within a timelimit [44]. As specified in the 3GPP standard [35], thereliability for most V2X applications is required to be above99.99%, which call for developing new engineering solutionsfor V2X transmission and networking in presence of rapid-changing network topology and fast-varying wireless channels.In vehicular scenarios, two main factors can significantlydegrade the reliability: 1) channel access collisions byconcurrent transmissions [45]; and 2) multi-path fadingdue to both mobile (high-speed vehicles) and stationary(buildings) scatterers, as well as shadowing due to in-channel obstacles such as tress, slopes, trucks, and buses. Thechannel impairments can significantly degrade transmissionperformance at the physical layer and link layer.

To reduce channel access collisions among nearby vehicles,various TDMA-based MAC protocols are proposed to assigndisjoint time slots to vehicles in a neighborhood [40], [42]. Inaddition to transmission collisions caused by vehicle mobility,link-layer transmission performance deteriorates in a highvehicle density scenario. Transmission power control [46]and transmission rate control [47] are some approaches todeal with transmission congestion. In particular, cooperativecommunication at the link layer can enhance transmissionreliability. For instance, Bharati et al. propose a cooperativeMAC, named CAH-MAC, to mitigate the channel impairmentsand improve the network throughput [48]. In cooperativecommunication, how to select the helper vehicle for aretransmission is critical, and there are receiver-oriented [49]and sender-oriented [23] approaches to determine the helpervehicle.

Under the SDN architecture, with the knowledge of real-time network status (e.g., the number of vehicles, vehiclemobility, QoS requirements, and available medium resources)via OpenFlow (or extension), the logically centralized controlplane is able to update the channel access and transmissionpolicy in a timely manner. The programmability of protocol-agnostic digital transceivers in the data plane makes theunderlying hardware adaptable to all channel access andtransmission policies by enabling/disabling transmissions,varying the transmission power and transmission rate [50].Under this working model, the transmission collisions causedby mobility and data traffic congestion can be reduced viaefficient control implemented at a local SDN controller. Forinstance, without affecting other applications, the transmissionpolicy can require a transceiver to lower the data rate andenlarge the transmission power for reliability enhancement.In addition, in cooperative communication, the regional viewof local controller benefits the helper vehicle selection sincethe rich information about candidate helpers (e.g., position,historical transmitting/receiving status, velocity) is available.However, to achieve the centralized control, the overhead ofcollecting the network status from vehicles is considerableas all vehicles under the control have to update the statuswith a high frequency, which limits the coverage area ofeach controller. In addition, as the controller deployment

is costly, it is impossible to deploy local controllers tocover all road segments. Fig. 3 illustrates an SDN-basedenhanced IoV communication architecture for both urban andsuburban areas. In urban areas with busy and complicatedvehicle traffic, the centralized control can potentially playan important role: 1) As the radio spectrum resources arelikely insufficient in such a high vehicle density scenario,the controller with a regional view is to enhance the overallefficiency of resource management. Taking account of thesignaling overhead, multiple local SDN controllers should beplaced to work cooperatively for an area of interest; 2) Asthe vehicle traffic condition is more complicated and changesmore dramatically in the scenario (in comparison with that insuburban and rural areas), a robust policy is needed to work inall conditions for service performance guarantee. On the otherhand, in suburban areas where the vehicle traffic density is lowand traffic condition is relatively simple, distributed networkoperation and control (via direct interactions among nearbyvehicles) are potentially applicable, which avoids deploymentof local SDN controllers.

C. Ubiquitous and efficient connections

In addition to the preceding light-weight V2Vcommunications, data-intensive communication is essential toaccommodate high-throughput Internet services for vehicleusers and for big data informatics, where vehicles equippedwith various sensors are information source nodes. As theDSRC radio spectrum is insufficient, the radio channels arelikely congested when supporting data-craving applications[47]. To this end, the heterogeneous vehicular networkarchitecture is required. As shown in Fig. 3, in addition toDSRC and cellular BS, road-side WiFi and TVWS shouldbe leveraged to provide IoV connections [51]. However,the terrestrial networks suffer from two inherent limitationsin supporting IoV applications. First, the vehicle densityvaries dramatically with time while the terrestrial networkresources are more or less tied to geographical areas, leadingto both resource wastage and shortage in different times andlocations [18]. Even with an optimized deployment strategy,it is difficult to achieve both efficient resource utilization andservice quality satisfaction in the long run. Second, for theterrestrial network, coverage holes exist especially in ruraland remote areas, where radio signals are not sufficientlystrong. The coverage holes restrain the ubiquitous connectionsfor vehicles. Note that a full connection to and from a vehicleanywhere and anytime is required for future autonomousdriving.

To address the two limitations, aerial and space networksare advocated to provide connections for vehicles, whereon-demand deployment of aerial networks (i.e., UAVs)can potentially make the network adaptive to a diverseenvironment, and a space satellite network can providethe ubiquitous wireless coverage [52]. To support IoVapplications, how to determine in real time which network forvehicles to connect to (referred to as radio access technology(RAT) selection) is a critical yet challenging task, as variousaccess technologies have unique advantages in terms of delay,

8

Central SDN/NFV controller

Core networks

Local SDN/NFV

controller

Cloudlet 1 Cloudlet 2

MEC

servers

Virtual cloud

computing

resource pool

Edge computing

layer

Infrastructure

computing layer

Remote cloud

computing

Edge

computing

Vertical

collaboration

Local computing

Op

era

tio

n

Off

loa

din

g

Op

era

tio

n

Off

loa

din

g

Vertical

collaboration

Cellular

networks

VANETs Vehicular clouds

Local SDN/NFV

controller

Fig. 4: A three-tier vehicular computing architecture.

throughput, and coverage range. For instance, road-side WiFican provide a high data rate when a vehicle is nearby, butthe data rate is unstable as the vehicle moves away and theconnection duration is short. Cellular networks demonstratestable performance but its high service cost may not beacceptable to users. Although a satellite network can providethe ubiquitous coverage, the transmission delay from theground to the space is long, making it unsuitable for delay-sensitive applications [18]. In addition, complex algorithmsare required to dispatch on-demand UAVs and schedule theirtrajectories [53].

SDN can potentially enhance the IoV connectionperformance by determining the access mode of eachvehicle in real time. The SDN architecture decouples theunderlying data packet forwarding from network control, andcan utilize protocol-agnostic hardware to support dynamicallycustomized control policies. At the application plane,the precise application-specific requirements can be pre-determined and delivered to the control plane via NorthboundAPI. Based on the network status information includingavailable resources in network segments (i.e., the terrestrial,aerial, and space network segments), vehicle density, QoSrequirements, vehicle mobility statistics, etc., the control planecan customize the access policy and parameters for differentvehicles towards differentiated applications [54]. However,to simultaneously manage all resources in the terrestrial,aerial, and space networks is difficult as the networks havedisparate interference ranges, while the coverage of eachcontroller is limited due to the signaling overhead. Therefore,a hierarchical control architecture is requisite, with a numberof local controllers, each in charge of V2X communicationsin its coverage area (e.g., via a terrestrial network), and acentral controller in charge of the local controllers and tocoordinate service provisioning in a large area (includingmanaging resources in the satellite network), as shown in Fig.3. Under the hierarchical architecture, the local controllerscan be deployed at network edge to provide quick responsesto service demands from vehicles, while the central controllercan be deployed at the core/backhaul network for large timeand geographical scale (global) network management andoperation.

V. IOV COMPUTING SERVICES

Data traffic flows from an increasing number ofcomputation-intensive IoV applications (e.g., on-board videostreaming, environment sensing, HD map downloading)are required to traverse different sets of network/servicefunctionalities (e.g., firewall, transcoding, data classificationand data fusion) for service customization, which isdifferent from conventional communication-oriented vehicularnetworking for road safety applications. For instance, asecure HD video streaming service requires video traffictraverse a sequence of firewall function and transcodingfunction before being displayed on end vehicles supportingdifferentiated video file formats. In a vehicular networkingscenario, access and mobility management function (AMF)is another important network-level functionality in the 5Gcore network, which deals with user location registration,connection management, and vehicle-to-BS or vehicle-to-new-network (vertical) handover management [55]. By usingthe NFV technology, these network-level and service-levelfunctionalities are decoupled from hardware servers and arevirtualized as VNFs that can be flexibly placed by an NFVcontrol module on generic servers in the network core,reducing the function execution and operational cost. Forexample, the AMF can be instantiated near the edge of thecore network to improve the efficiency of connection andmobility management for vehicles moving among differentBS coverage areas. Additionally, with SDN, virtual networktopology configured for both routing and VNF placement canbe optimized and customized for different types of services.However, executing VNFs only at cloud servers incurs a two-way communication delay. For some real-time applications, alow response delay is required for task computation, such asobject identification in environment sensing for autonomousdriving [56], HD map data downloading and processing[57], and VR gaming. Therefore, MEC emerges to offloadparts of the computing tasks from cloud servers to edgeservers, in order to reduce the task transmission delay onbackhaul links [4]. As shown in Fig. 4, a three-tier vehicularcomputing architecture is proposed to accommodate variousIoV services, where nearby vehicles provide opportunisticcomputing, edge nodes (e.g., MEC servers and cloudlets)

9

support fast computing and processing with limited resources,and the remote cloud guarantees sufficient resources forsustainable computing. Many existing studies present differentintegrated architecture for IoV by combining MEC/cloudcomputing with SDN/NFV [50], [58], to enable the flexibilityof VNF orchestration with computing task offloading nearthe network edge and to improve the QoS performancefor different computation-intensive applications. Several mainresearch issues under this integrated architecture are discussedin the following.

A. Computation offloading with edge-cloud interplay

Combining NFV with MEC for vehicular networking isa promising approach to extend flexible VNF placementwith computation offloading to the network edge forfulfilling the communication delay requirements for computingtask execution. The system architecture has already beenstandardized by ETSI in 2018 [59]. Although the computationoffloading reduces communication resource consumption overbackhaul links and shortens the response latency for delay-sensitive computing task execution, it increases architecturalcomplexity (i.e., more MEC servers need to be deployed)and computation redundancy at the network edge. Moreover,some delay tolerant computing tasks from certain IoVapplications require more powerful computation capabilityand should be placed at cloud servers for execution. Hence,there exists an edge-cloud interplay in terms of computationoffloading. A key issue is how to balance the trade-offbetween reducing communication delay for task processingand achieving desired computation performance with lowcost. The SDN architecture provides a potential solution toenhance the NFV and MEC integrated vehicular networkingarchitecture, while dealing with the edge-cloud interplay. Thelocal SDN control can be implemented on a virtualizedMEC server as a control system to handle the computingtask scheduling among the edge server and end vehicles,or a central SDN controller can be implemented in a clouddata center to execute the computation task migration amongdifferent edge servers. By decoupling and migrating thecontrol functionalities for data delivery from vehicles toedge servers, the underlying vehicles equipped with radiotransceivers are abstracted as SDN switches [50]. Then, eachlocal SDN controller can collect the vehicular network stateinformation, the communication and computing requirementsof IoV applications through either DSRC or cellular networksoperating the OpenFlow protocol in the application layer [4].Based on the collected information, an optimized computingtask scheduling policy can be executed as an application-plane program over the local SDN controller through thenorthbound API. On the other hand, the central SDN controllerin the core network manages computing task offloadingand migration policies among different edge servers basedon the collected data traffic information and the predictionon vehicle mobility for service satisfaction and servicecontinuity. Many existing works investigate the edge-cloudinterplay problem in an SDN/NFV based MEC architecture.For example, a task scheduling framework is proposed

for IoV based on an SDN/NFV-enabled MEC architecture,where challenges of the edge-cloud interplay problem areidentified in terms of resource and energy limitation on edgeservers and mobility management [60]. With the considerationof resource, energy, and delay constraints, developing acomprehensive computation offloading scheme to satisfydifferentiated service requirements falls into the categoryof multi-objective optimization problems. Specifically, toobtain optimal decisions for task offloading, the edge-cloudinterplay problem can be formulated to optimize the trade-offbetween energy efficiency and backhaul bandwidth resourceconsumption under the latency constraints for task execution.For tractability, multi-objective evolutionary algorithms can beused to solve the problem based on decomposition methods.

B. Mobility-aware computing task migration and dataoffloading

Under the NFV and MEC integrated architecture, anotherimportant research problem is how to conduct computingtask migration among nearby edge servers to maximizeservice completion probability within a certain time limitwhen a vehicle moves across different BS coverage areas.Each MEC server is physically connected to a set of BSsand is responsible for computing task offloading (fromthe vehicles) and execution under the BS coverages, asshown in Fig. 4. The SDN can be incorporated into thearchitecture to obtain computing resource usage informationon edge servers and facilitate task migration over a properlyconfigured path. There exist proposals for SDN-based wirelessconnection handover and inter-MEC server computing taskmigration in SDN/NFV based vehicular network architectures[61], [62], in which a local SDN controller manages thewireless connection re-association when vehicles move outof the previous BS coverage, and determines computing taskmigration among MEC servers. Based on the collection ofvehicular network context information, how to optimize theedge server deployment and computing resource allocationbased on vehicle mobility models is essential to maximize thesuccess probability of task completion. In an IoV scenario,the SDN can help to establish an end-to-end routing pathbetween two vehicles at different network areas through V2Vand V2I communication modes for data offloading. Becauseof the increasing number of vehicles and densified networkdeployment, how to route data traffic with low communicationoverhead, and to accommodate high source/destination nodemobility is a key issue. In an SDN-based edge computingarchitecture, the SDN controller can assist to optimize thetraffic routing paths to achieve load balancing and adaptthe path selection to customized services with differentiateddelay requirements [63]. V2V-based traffic routing for dataoffloading from cellular networks is a cost-effective solutionin an SDN-based IoV scenario due to its simplicity, where alocal SDN controller can calculate an optimal V2V routingpath across different RSU coverages based on the collectedvehicular network information [64] and vehicle mobilityprediction [65].

10

C. Vehicular edge cloud

Because of the deployment cost of MEC servers, anotherway is to explore computation opportunities among nearbyvehicles for computing task offloading, which can reducethe response delay for task execution to satisfy strictlatency requirements and relieve the computation burden onedge servers. Therefore, by employing vehicular networkingtechnologies, some computation powerful vehicles can forma vehicular edge cloud, referred to as vehicular cloud (VC),to cooperatively execute various computing tasks [4], [22],[56]. The concept of VC has become relevant, especiallyin the area of autonomous driving where vehicles need toconsistently collect a large amount of sensing data fromthe environment for prompt processing and proper actions.Forming an effective VC for cooperative computing cansignificantly reduce the two-way response delay for taskexecution on MEC servers, and improve the computingperformance by efficient task dissemination. Due to vehicledynamics in joining and leaving a VC, the computingresource availability for a VC can be intermittent. Based onthe computation requirements from different services, howto select vehicles to form an effective VC and how toschedule tasks among the selected vehicles to ensure thetask completion within the vehicle contact duration need tobe customized, which is technically challenging. CombiningSDN with MEC is a promising architecture to facilitateVC establishment and make locally centralized decisions forcomputing task execution and data traffic routing amongvehicles within a VC. In literature, how to leverage SDN todynamically form a VC and distribute tasks among vehiclesfor collaborative computing has been studied based on theinformation of resource availability, where the discrepancy ofresource availability and task inter-dependency are consideredfor minimizing the average response time for task completion.The local SDN controller deployed at each MEC server helpsto group nearby vehicles to form a VC, and the central SDNcontroller deployed at a cloud server coordinates distributedMEC servers for centralized management of VC groups. Howto design the hierarchical controller architecture (in terms ofdetermining the placement locations, the necessary number ofcontrollers, and the networking topology among controllers)remains a key and challenging research issue, as it depends onthe size, stability, resource availability, and task computationload of each VC. The controller operational cost also needs tobe specified by considering the information exchange betweeneach VC and the controllers, including the types, amount, andinteraction frequency of network state information. Therefore,the objective is to determine an appropriate SDN controllerarchitecture to improve the VC formation and computing taskallocation performance under certain controller deploymentand operational cost.

VI. IOV CACHING SERVICES

With more and more on-board mobile devices, IoVis envisioned to support ever-increasing infotainmentapplications for drivers and passengers to improve their roadexperience. However, these infotainment applications such

as HD video streaming, online gaming, and future AR/VR,can consume a huge amount of spectrum resources and haverigorous real-time service delivery requirements, which poseschallenges on vehicular networking. In addition, to enableautonomous driving, delivering real-time and high-volumeHD map to vehicles is essential, which helps to localize exactvehicle positions (with the precision of 10-20 centimeters) onthe road with the consumption of large amounts of computingand communication resources. Mobile edge caching playsan important role to provide satisfactory vehicular serviceprovisioning, upon which contents can be cached at edgenodes close to end vehicles. It can significantly reducethe service latency and alleviate the backhaul pressure byallowing users to fetch contents within one transmission hopwithout traversing multi-hop core/backhaul networks to reachthe Internet or cloud servers for contents. However, to createan effective caching system is not easy, as the problems ofcaching deployment, caching content update, and contentdissemination are significant yet unsolved. The cachingdeployment problem deals with where to deploy cachingfunctions and how much cache space to allocate, which isessential to increase the probability of successful downloadingof cached data but is also labor-intensive with high capitaland operational expenditure [66]. The performance of acaching system can be measured by three metrics: 1) cachinggain – the amount of successfully downloaded data overthe total amount of requested data; 2) content hit ratio –the number of content hits over the number of requests; 3)caching diversity – the types of cached contents at differentBSs. Therefore, an effective caching deployment aims tomaximize the caching gain under constraints of deploymentcost (including installation cost and hardware maintenancecost) [66]; Considering vehicle mobility and differentiatedspatio-temporal content request patterns from vehicles, howto update the cached contents to achieve a high content hitratio is critical to improve the caching performance [67];When a requested content from a vehicle is hit, how todisseminate the content to the end vehicle also affects servicequality. Caching duplicated contents at multiple locationsimproves the transmission efficiency by exploiting highchannel qualities at BSs close to vehicles which, however,lowers the caching diversity. Achieving efficient contentdissemination is to balance the trade-off between transmissionefficiency and caching diversity. SDN/NFV technologies canpotentially be leveraged to deal with the aforementionedproblems with caching performance enhancement.

A. Caching deployment

The edge caching deployment is a non-trivial problemas content demands from vehicles are unpredictable andvehicle locations are time-varying. In a typical vehicularscenario, moving vehicles, stationary RSUs and BSs arepossible executors for caching functions. Vehicles have limitedcapability for computing and data storage, and their mobilityleads to difficulties to fully utilize their caching functions,especially with unstable communication links. RSUs arelightweight, with potential large-scale deployment. To achieve

11

satisfactory caching performance, the deployment strategy forcaching locations among all candidate RSUs aims to balancethe tradeoff between deployment cost and caching gain. Forcellular BSs, the cache sizes should be properly determined.Insufficient cache sizes can fail to fully unleash the cachingpotentials, while an excessive cache space leads to resourcewastage.

In order to maximize the long-term caching gain, a statisticalanalysis on user association and data traffic should be carriedout for developing a strategy in a large-scale WiFi system [66].The spatio-temporal vehicle traffic variations can be learnedby mining the existing large-scale GPS traces [68] for cachingdeployment at RSUs and BSs. Likewise, to deploy cachingfunctions at moving vehicles, the wireless interconnection timebetween a pair of vehicles [69] and their social behaviors(e.g., vehicle traveling routes) [70] can be studied based on theGPS trace, to improve the caching performance. However, theinherent one-shot deployment approach cannot accommodateall vehicle traffic conditions with maximized caching gain.The NFV technology can help to facilitate dynamic cachingdeployment. With NFV, computing and storage resources onvehicles are virtualized to support different functionalitiesas programmable software instances. In this way, cachingfunctions can be flexibly programmed on different vehicles toexploit their resource availability, especially at rural or remoteareas where both RSU and BS coverages are insufficient[71]. In the areas where vehicle traffic is light, some virtualresources used for sensing and computing can be furtherreleased to support more caching function instances to servenearby vehicles with on-demand content requests [72]. Inurban areas, superfluous caching function instances at RSUsand BSs can be deactivated during off-peak hours, and thereleased virtual resources can be utilized to support othernetwork functions such as firewall, retransmission, and soforth. Therefore, the resource virtualization technology canimprove caching deployment performance by accommodatingmore caching function instances in different spatio-temporalscenarios [73]. Further, for dynamic caching deployment, theSDN can be leveraged to make decisions on when and where toactivate/deactiavte caching function instances and how manyvirtualized storages to allocate to support caching functioninstances, by collecting real-time network state informationand customizing dynamic caching policies [74].

B. Cached content update

In addition to HD maps that are required to be cached withpriority, each user may have a unique preferable file list forother Internet contents. Given certain cache resources, whatcontents to cache and when to update become an importantresearch problem, the goal of which is to maximize the cachedcontent hit ratio. There are two conventional caching strategiesin the literature: 1) Least recently used (LRU): when the cacheis full, the least recently requested contents are replaced; 2)Least frequently used (LFU): when the cache is full, the leastfrequently requested contents are replaced. Both strategies aregeneral approaches but their granularity is not small enoughto meet user request demands with spatio-temporal patterns.

For instance, by learning the real-world data set that stores 0.3million unique video viewing records by 2 million users, Maet al. investigate the user request patterns and confirm theirspatio-temporal variation features [67]. Some works aim atpredicting content popularity in order to enhance the contenthit ratio, based on a noisy linear model [75] or human-centricinformation (e.g., visited locations, requested contents) [76].

To adapt to the spatio-temporal popularity variations ofInternet contents, the big data [3] and emerging AI techniques[77] are essential methodology. The centralized SDNarchitecture provides a suitable platform for implementation.Particularly, with the OpenFlow (or its extension) protocol,SDN controllers can facilitate the collection of contentrequests with user IDs, times, and locations and generatea statistically sufficient data set. Given a location, the timeseries of file requests can be learned and modeled off-line,based on which the location-aware online popularity predictionis enabled [78]. Such information is useful for a dynamiccontent placement policy, to enhance the overall content hitratio. With off-line pre-learning for the dynamic network statusas well as other features, such as location and time, theconvergence speed for making real-time learning decisionscan be improved to keep pace of the fast-changing vehiculartraffic conditions. Even though the information of historicalfile requests can be obtained, thanks to the SDN controller,the random nature of content demands reduces the cachingperformance. For instance, the requested content distributionfor gaming can change significantly, which deviates fromthe prediction results. Likewise, if a traffic accident happens,bursty and unpredictable file requests may be generated, thusreducing the accuracy of AI models. Therefore, a backupscheme should be in place when the trained model losesefficacy. It should react to unexpected events in time, and runas an alternative to compensate for the degraded performancefrom AI decisions. In addition, the popular file list normallychanges rapidly with time [67], which requires the SDNplatform to continuously collect the up-to-date user requestrecords, in order to update the training model. The large-scaledata collection and computationally-costly model training canput a high burden on the platform.

C. Cached content dissemination

The main objective of SDN/NFV-based cached contentdissemination in IoV is to minimize the content retrieval time[73]. However, it is technically challenging to achieve this goalas the transmission efficiency and the caching diversity areconflicting objectives. Caching duplicated contents at multipleBSs improves the transmission efficiency by successfullyfetching the content over better channels, but degrades thecontent caching diversity with underutilized cache spaces; Onthe other hand, caching different contents at multiple locations,referred to as cooperative caching, increases the contentcaching diversity but degrades the transmission efficiencyfor the cached contents when the channel quality betweena BS and a vehicle is low. The essence of cached contentdissemination is to investigate a multi-dimensional resource(e.g., caching resources and radio resources) management

12

problem to balance the tradeoff between caching diversityand transmission efficiency, with the consideration of availablecaching locations, cache spaces and radio resources on BSs,and vehicle mobility. Mathematically, a multi-dimensionalresource allocation problem can be formulated to jointlyoptimize two caching performance metrics: 1) content hitratio and 2) content delivery time. Due to the problemcomplexity, it is difficult to develop a simple and accuratealgorithm for making real-time decisions in a highly dynamicvehicular network. Alternatively, model-free reinforcementlearning (RL) methods remain a tractable online-learningapproach to solve the problem with fast convergence and highaccuracy [18]. Specifically, the multi-dimensional resourceallocation problem in a dynamic vehicular scenario can bedescribed as a Markov decision process (MDP), and the RLmethod can be applied to solve the MDP by making real-time decisions on content placements and user associations,in order to maximize the time-averaged reward. A rewardfunction can be expressed as a weighted function of contentdiversity and content delivery time. Upon interactions withthe network environment, the RL-based approach tries tobalance the exploration (randomly trying an action) andexploitation (choosing the most promising action based oncurrent knowledge) processes to achieve time-averaged rewardmaximization. Due to vehicle mobility and limited wirelesscommunication range, the connections between vehicles andcaching places are intermittent, leading to possible failureof downloading a complete cached content. Therefore, withthe consideration of mobility prediction, how to cooperativelyprefetch portions of contents at multiple locations to maximizethe content hit ratio is important to enhance the contentdissemination.

VII. OPEN RESEARCH ISSUES

There are many open technical issues for IoV in terms ofimproving the network performance with the considerationof high vehicle mobility, increased data traffic load, anddiverse service requirements. In this section, some potentialresearch issues are discussed from the aspects of enhancingIoV performance by applying SDN/NFV technologies. Fromthe communication perspective, we identify the importanceand effectiveness of leveraging the SDN/NFV technologies inIoV scenarios for conducting network-level resource slicingin a multi-tier heterogeneous communication infrastructure(e.g., macro-cell BSs, small-cell BSs, RSUs/APs) for servicecustomization. From the computing perspective, a VNFchaining and placement problem at the edge of vehicularnetworks is discussed. Another potential research issue is howto jointly allocate communication, computing, and cachingresources in SDN/NFV-enabled vehicular networks. We alsodiscuss hierarchical SDN/NFV controller deployment formulti-dimensional resource allocation.

A. Resource slicing and vehicle access control for IoVSDN and NFV are the key technologies to enhance the

integration of network-level communication and computingresource orchestration and end device association. An

SDN/NFV integrated control module is often placed at theedge of the core network and is physically connected toa set of BSs/APs through backhaul links [79]. Throughprogrammable control links, the integrated control modulecollects important network state information from underlyingvehicular networks, such as resource reservation informationon BSs, vehicle density and mobility, data traffic load, statisticsof different service demands, and supported service typeswith QoS requirements. Accordingly, the control modulemakes network-level decisions to update vehicle access controlpolicies and reconfigure the overall network resources amongBSs/APs to create resource “slices” for different services.Due to the inter-dependence of radio resource allocationand BS-user association, how to jointly deal with theresource slicing and vehicle access control to maximize thenetwork-level resource utilization is a key research issue.The 3GPP standards release 15 [80] and release 16 [81]specify the split network functionalities between 5G radioaccess networks and core networks, such as AMF, networkslice selection function (NSSF), to support network slicing,and introduce new interfaces (e.g., Xn) to facilitate inter-gNodeB resource sharing. A general description of slicecharacteristics is provided in terms of QoS requirements(e.g., bit rate, latency) for service differentiation. Althoughthe network slicing concept has been proposed in 3GPPstandards, and the radio resource slicing problem has beenstudied for both 5G wireless and core networks [31], [82],designing an optimal resource slicing framework to supportdiversified IoV applications is still in its infancy, as theproblem becomes much more complex in an IoV scenario withthe consideration of mobility patterns of vehicles and smallmoving cells (e.g., covered by UAVs), resource heterogeneity,and differentiated IoV service requirements in terms of latency,reliability, and data rate. SDN/NFV-based resource slicingframeworks are proposed for IoV in existing works [83]. Ajoint communication and computing resource slicing problemis studied to minimize the connection outage probability ofvehicles for edge-assisted vehicular networks [84]. In [85], aresource slicing problem with content pushing and cachingis studied to support different vehicular network applicationsin air-assisted IoV, with a focus on how much computingtasks and popular contents that can be offloaded onto groundvehicles for differentiated QoS guarantee. Further studies areneeded to determine proper resource slices at heterogeneousBSs to maximize the overall resource utilization via BS-vehicle association.

B. VNF chaining and placement for computation offloading

Under the SDN/NFV-based MEC architecture as in Fig.4, some computing tasks can be executed on network edgeservers near BSs or even on individual vehicles with sufficientcomputing resources. As available computing resources areoften unevenly distributed among network elements, offloadingcomputation-intensive missions to the network edge for taskprocessing improves the computation efficiency, especially forapplications requiring ultra low latency for task execution.For many IoV applications, their computing tasks need to

13

go through a sequence of VNFs for E2E traffic delivery.For instance, HD map data traffic needs to traverse dataclassification and data fusion functionalities before beinganalyzed on vehicles. Therefore, the computing offloadingalso comes with VNF migration and corresponding computingresource allocation. For a vehicular cloud network, as shownin Fig. 4, a local SDN/NFV integrated control moduledeals with how VNFs are chained and placed at differentvehicles, with the consideration of vehicular mobility, toensure service continuity and differentiated QoS satisfaction.The coordination between a central SDN/NFV controller andlocal SDN/NFV controllers also needs to be considered forVNF migration when vehicles move among different BScoverages. For IoV scenarios, computation offloading withVNF chain orchestration is a challenging research issue, wherethe VNF chaining and placement and resource allocation onthe edge servers and vehicles depend on dynamic resourceavailability, function provisioning cost, and vehicle mobility.Preliminary studies on VNF placement and resource allocationinclude a resource-oriented VNF scheduling based on trafficclassification [86] and VNF chain mapping and resourceprovisioning to maximize the overall service acceptanceratio [87]. A key issue in jointly determining computationoffloading and VNF placement is how to maximize overallresource utilization, under the constraints of VNF offloadingand service provisioning costs.

C. Joint communication, computing, and caching resourceallocation

With the emergence of edge computing and edgecaching, future vehicular networks are expected to integratecommunication, computing, and caching resources to supporta variety of IoV applications. The SDN/NFV frameworkintegrated with MEC and edge caching provides centralizednetwork control over different types of resources, whichcan improve the efficiency of multi-resource orchestration.However, diverse demands for the multi-dimensional resourcesfrom end devices pose technical challenges over conventionalmodel-based network optimization approaches [88], [89].First, the model-based methods require a large amountof network state information obtained from the underlyingphysical vehicular networks in the 5G (and B5G) era,which increases the complexity of resource allocation andcomputation overhead at SDN/NFV control modules; Second,the heterogeneity and inter-dependence of resource demandsmake it difficult to determine the trade-off among multi-dimensional resources. For example, a computing taskrequest sent from a vehicle to an edge server consumesa combination of radio bandwidth resources for wirelesstransmission and computing resources on the edge serverfor task execution; Requesting a cached video content ina BS needs transcoding before being fetched by an endvehicle, which consumes a sequence of caching, computing,and communication resources. Modeling the relation amongmultiple types of resource consumption is essential, leadingto complex resource allocation problems. To overcome thecomplexity, an alternative approach is RL-based methods to

learn an efficient multi-dimensional resource allocation policyby including the present action’s impact on future networkstates, to achieve long-term accumulated maximal rewards[90]. The features of available network state information (e.g.,user demands, vehicle mobility, network service load) can beextracted and predicted to some extent, which can facilitatethe RL module design to improve the learning convergenceand accuracy.

D. Hierarchical SDN/NFV controller deployment

The placement of SDN/NFV control modules is ofparamount importance to improve the communication,computing, and caching performance in terms of optimizingthe multi-resource allocation with minimal controllerdeployment cost and control message exchange overhead.One way to deal with scalability issue of a centralizedcontroller is to have a hierarchical SDN/NFV controllerarchitecture. Local controllers are placed at each MEC serverconnecting to a set of BSs, and MEC servers are managed by ahigher level central controller placed at a cloud server or a datacenter node through backhaul and core network connections,as shown in Fig. 4. This controller architecture is promisingin distributing the overall controlling tasks at different levelsof control modules to reduce the implementation complexitywith increased efficiency and accuracy and, at the sametime, to reduce signaling delays to/from vehicles. A localcontroller is in charge of radio resource slicing among itsconnected BSs, vehicle access control and traffic routingamong vehicles, and computation offloading with VNF chainorchestration among end vehicles. From the coordination withlocal controllers, a central controller determines the policyupdates for computing task migration among neighboringedge servers, VNF placement with edge-cloud computationoffloading, and content caching placement. Many existingstudies present different conceptual approaches for thecontroller architectural design in different IoV networkingscenarios (e.g., MEC-assisted hierarchical control for VCgrouping [4]). Several important research aspects need furtherinvestigation for the controller deployment: 1) Given the largescale of a physical vehicular network, we need to determinethe minimum number of local/central controllers, theirlocation placement and networking topology, and associationpatterns between each local controller and its connectedBSs to achieve the optimal allocation of multiple types ofresources. The solution depends on the types, amount, andupdate frequency of collected network state information byeach controller, and the cost of bi-directional informationexchange between each control module and underlyingnetwork elements; 2) An efficient networking protocol isrequired to interconnect the control modules for control-planemessage exchanges, which affects the E2E QoS performanceof supported IoV applications.

To justify the proposed SDN/NFV-enabled IoV architecture,it is necessary to build a real testbed to conduct experimentsin terms of performance evaluation of different resourceallocation algorithms/schemes for IoV. MEC servers withboth computing and caching capabilities can be deployed

14

to physically connect to cellular, WiFi, or TVWS BSs, andthe SDN/NFV control modules can be realized in edge andcloud servers to provide centralized hierarchical control overdifferent types of resources (for communication, computing,and caching). Some high-performance deep learning serversare expected to connect to BSs, for data traffic tracetraining and RL algorithm implementation to make learningdecisions in terms of multi-dimensional resource allocation.The inter-working among the deployed network elementsfor synchronization, algorithm/scheme implementation, andhierarchical control needs to be properly specified to achievesystem-level stability.

VIII. CONCLUSION

SDN/NFV technologies are a key enabler of future IoVsystems in terms of multi-dimensional resource allocation andservice provisioning. This paper introduces the emerging trendof enhancing IoV with SDN/NFV technologies. Accordingto various QoS requirements of IoV applications andservices, three important roles of the SDN/NFV networkingframework in IoV are described and surveyed, respectively,for enhancing the communication, computing, and cachingcapabilities. Network softwarization, resource slicing, andVNF orchestration can improve service delivery, reliability,and enhance ubiquitous and efficient connectivity; On the otherhand, MEC and NFV technologies can be utilized to offloadIoV computing tasks to edge and cloud servers, and SDN canbe integrated to configure routing paths for data offloadingand computing task migration; Further, the SDN/NFV-based framework can facilitate adaptive caching deployment,cached content update and dissemination. To achieve theSDN/NFV/MEC potentials for IoV applications, there aremany open research issues, including joint resource slicing andaccess control, VNF chaining and placement for computationoffloading, joint multi-dimensional resource orchestration, andhierarchical SDN/NFV controller deployment.

REFERENCES

[1] “ITU towards IMT for 2020 and beyond,” https://www.itu.int/en/ITU-R/study-groups/rsg5/rwp5d/imt-2020/Pages/default.aspx, accessed: 2019-05-31.

[2] “The top 5G use cases,” https://www.sdxcentral.com/5g/definitions/top-5g-use-cases/, accessed: 2019-05-31.

[3] N. Cheng, F. Lyu, J. Chen, W. Xu, H. Zhou, S. Zhang, and X. Shen,“Big data driven vehicular networks,” IEEE Netw., vol. 32, no. 6, pp.160–167, 2018.

[4] X. Huang, R. Yu, J. Kang, Y. He, and Y. Zhang, “Exploring mobile edgecomputing for 5G-enabled software defined vehicular networks,” IEEEWireless Commun., vol. 24, no. 6, pp. 55–63, 2017.

[5] K. Wang, H. Yin, W. Quan, and G. Min, “Enabling collaborative edgecomputing for software defined vehicular networks,” IEEE Netw., no. 99,pp. 1–6, 2018.

[6] X. Wang, Z. Ning, and L. Wang, “Offloading in internet of vehicles:A fog-enabled real-time traffic management system,” IEEE Trans. Ind.Informat., vol. 14, no. 10, pp. 4568–4578, 2018.

[7] X. Ge, Z. Li, and S. Li, “5G software defined vehicular networks,” IEEECommun. Mag., vol. 55, no. 7, pp. 87–93, 2017.

[8] C. Liang, F. R. Yu, and X. Zhang, “Information-centric network functionvirtualization over 5G mobile wireless networks,” IEEE Netw., vol. 29,no. 3, pp. 68–74, 2015.

[9] H. T. Cheng, H. Shan, and W. Zhuang, “Infotainment and roadsafety service support in vehicular networking: From a communicationperspective,” Mech. Syst. Signal Process., Special Issue Integr. Veh. Dyn.,vol. 25, no. 6, pp. 2020–2038, 2011.

[10] “Standard for information technologyłspecific requirementsłpart 11:Wireless LAN medium access control (MAC) and physical layer (PHY)specifications,” IEEE Std. no. 802.11-2016, 2016.

[11] “Study on LTE Support for Vehicle to Everything (V2X) Services,”3GPP TS 22.885, 2016.

[12] “Architecture enhancements for V2X services,” 3GPP TS 23.285, 2017.[13] A. Bazzi, B. M. Masini, A. Zanella, and I. Thibault, “On the performance

of IEEE 802.11p and LTE-V2V for the cooperative awareness ofconnected vehicles,” IEEE Trans. Veh. Technol., vol. 66, no. 11, pp.10 419–10 432, 2017.

[14] 5GAA, “V2X Technology Benchmark Testing,”https://www.fcc.gov/ecfs/filing/109271050222769, 2018.

[15] M. Li, L. Zhao, and H. Liang, “An SMDP-based prioritized channelallocation scheme in cognitive enabled vehicular ad hoc networks,” IEEETrans. Veh. Technol., vol. 66, no. 9, pp. 7925–7933, 2017.

[16] G. S. Aujla, R. Chaudhary, N. Kumar, J. J. Rodrigues, and A. Vinel,“Data offloading in 5G-enabled software-defined vehicular networks: Astackelberg-game-based approach,” IEEE Commun. Mag., vol. 55, no. 8,pp. 100–108, 2017.

[17] J. Mei, K. Zheng, L. Zhao, Y. Teng, and X. Wang, “A latencyand reliability guaranteed resource allocation scheme for LTE V2Vcommunication systems,” IEEE Trans. Wireless Commun., vol. 17, no. 6,pp. 3850–3860, 2018.

[18] N. Cheng, F. Lyu, W. Quan, C. Zhou, H. He, W. Shi, and X. Shen,“Space/aerial-assisted computing offloading for IoT applications: Alearning-based approach,” IEEE J. Sel. Areas Commun., vol. 37, no. 5,pp. 1117–1129, 2019.

[19] M. Kamel, W. Hamouda, and A. Youssef, “Ultra-dense networks: Asurvey,” IEEE Commun. Surveys Tuts., vol. 18, no. 4, pp. 2522–2545,2016.

[20] H. Wang, G. Ding, F. Gao, J. Chen, J. Wang, and L. Wang, “Powercontrol in UAV-supported ultra dense networks: Communications,caching, and energy transfer,” IEEE Commun. Mag., vol. 56, no. 6, pp.28–34, 2018.

[21] I. Jang, S. Choo, M. Kim, S. Pack, and G. Dan, “The software-definedvehicular cloud: A new level of sharing the road,” IEEE Veh. Technol.Mag., vol. 12, no. 2, pp. 78–88, 2017.

[22] F. Sun, F. Hou, N. Cheng, M. Wang, H. Zhou, L. Gui, and X. Shen,“Cooperative task scheduling for computation offloading in vehicularcloud,” IEEE Trans. Veh. Technol., vol. 67, no. 11, pp. 11 049–11 061,2018.

[23] F. Lyu, H. Zhu, N. Cheng, H. Zhou, W. Xu, M. Li, andX. Shen, “Characterizing urban vehicle-to-vehicle communicationsfor reliable safety applications,” IEEE Trans. Intell. Transp. Syst.DOI:10.1109/TITS.2019.2920813, to appear.

[24] Y. Wang, Z. Su, Q. Xu, T. Yang, and N. Zhang, “A novel chargingscheme for electric vehicles with smart communities in vehicularnetworks,” IEEE Trans. Veh. Technol. DOI: 10.1109/TVT.2019.2923851,to appear.

[25] SAE International, “Taxonomy and definitions for terms relatedto driving automation systems for on-road motor vehicles,”https://www.sae.org/standards/content/j3016 201806/, Jun. 2018.

[26] L. Hobert, A. Festag, I. Llatser, L. Altomare, F. Visintainer, andA. Kovacs, “Enhancements of V2X communication in support ofcooperative autonomous driving,” IEEE Commun. Mag., vol. 53, no. 12,pp. 64–70, 2015.

[27] H. Jiang, W. Zhuang, X. Shen, and Q. Bi, “Quality-of-serviceprovisioning and efficient resource utilization in CDMA cellularcommunications,” IEEE J. Sel. Areas Commun., vol. 24, no. 1, pp. 4–15,Jan. 2006.

[28] C. Liang and F. R. Yu, “Wireless network virtualization: A survey, someresearch issues and challenges,” IEEE Commun. Surv. Tutor., vol. 17,no. 1, pp. 358–380, 2015.

[29] B. A. A. Nunes, M. Mendonca, X. N. Nguyen, K. Obraczka, andT. Turletti, “A survey of software-defined networking: Past, present, andfuture of programmable networks,” IEEE Commun. Surv. Tutor., vol. 16,no. 3, pp. 1617–1634, 2014.

[30] N. McKeown, T. Anderson, H. Balakrishnan, G. Parulkar, L. Peterson,J. Rexford, S. Shenker, and J. Turner, “Openflow: enabling innovationin campus networks,” ACM SIGCOMM Comput. Commun. Rev., vol. 38,no. 2, pp. 69–74, 2008.

[31] Q. Ye, W. Zhuang, S. Zhang, A. Jin, X. Shen, and X. Li, “Dynamicradio resource slicing for a two-tier heterogeneous wireless network,”IEEE Trans. Veh. Technol., vol. 67, no. 10, pp. 9896–9910, 2018.

[32] F. Bari, S. R. Chowdhury, R. Ahmed, R. Boutaba, and O. C. M. B.Duarte, “Orchestrating virtualized network functions,” IEEE Trans.Netw. Serv. Manage., vol. 13, no. 4, pp. 725–739, 2016.

15

[33] H. A. Omar, N. Lu, and W. Zhuang, “Wireless access technologies forvehicular network safety applications,” IEEE Netw., vol. 30, no. 4, pp.22–26, 2016.

[34] D. Kreutz, F. M. V. Ramos, P. E. Verłssimo, C. E. Rothenberg,S. Azodolmolky, and S. Uhlig, “Software-defined networking: Acomprehensive survey,” Proc. IEEE, vol. 103, no. 1, pp. 14–76, 2015.

[35] “Service requirements for enhanced V2X scenarios,” 3GPP TS 22.186V15.3.0 Release 15, pp. 1–17, Jul. 2018.

[36] J. He, L. Cai, P. Cheng, and J. Pan, “Delay minimization for datadissemination in large-scale VANETs with buses and taxis,” IEEE Trans.Mobile Comput., vol. 15, no. 8, pp. 1939–1950, 2016.

[37] G. Zhang, Y. Xu, X. Wang, X. Tian, J. Liu, X. Gan, H. Yu, andL. Qian, “Multicast capacity for vanets with directional antenna anddelay constraint,” IEEE J. Sel. Areas Commun., vol. 30, no. 4, pp. 818–833, 2012.

[38] M. J. Neely and E. Modiano, “Capacity and delay tradeoffs for ad hocmobile networks,” IEEE Trans. Inf. Theory, vol. 51, no. 6, pp. 1917–1937, 2005.

[39] P. Wang and W. Zhuang, “A token-based scheduling scheme for WLANssupporting voice/data traffic and its performance analysis,” IEEE Trans.Wireless Commun., vol. 7, no. 5, pp. 1708–1718, May 2008.

[40] H. A. Omar, W. Zhuang, and L. Li, “VeMAC: A TDMA-basedMAC protocol for reliable broadcast in VANETs,” IEEE Trans. MobileComput., vol. 12, no. 9, pp. 1724–1736, 2013.

[41] M. Hadded, P. Muhlethaler, A. Laouiti, R. Zagrouba, and L. A. Saidane,“TDMA-based MAC protocols for vehicular ad hoc networks: A survey,qualitative analysis, and open research issues,” IEEE Commun. SurveysTuts., vol. 17, no. 4, pp. 2461–2492, 2015.

[42] X. Jiang and D. H. C. Du, “PTMAC: A prediction-based TDMA MACprotocol for reducing packet collisions in VANET,” IEEE Trans. Veh.Technol., vol. 65, no. 11, pp. 9209–9223, 2016.

[43] C. Campolo, A. Molinaro, A. Iera, and F. Menichella, “5G networkslicing for vehicle-to-everything services,” IEEE Wireless Commun.,vol. 24, no. 6, pp. 38–45, 2017.

[44] F. Bai, D. D. Stancil, and H. Krishnan, “Toward understandingcharacteristics of dedicated short range communications (DSRC) from aperspective of vehicular network engineers,” in Proc. ACM MobiCOM’10, Sep. 2010.

[45] W. Alasmary and W. Zhuang, “Mobility impact in IEEE 802.11 pinfrastructureless vehicular networks,” Ad Hoc Netw., vol. 10, no. 2,pp. 222–230, 2012.

[46] M. Torrent-Moreno, J. Mittag, P. Santi, and H. Hartenstein, “Vehicle-to-vehicle communication: Fair transmit power control for safety-criticalinformation,” IEEE Trans. Veh. Technol., vol. 58, no. 7, pp. 3684–3703,2009.

[47] F. Lyu, N. Cheng, H. Zhou, W. Xu, W. Shi, J. Chen, and M. Li, “DBCC:Leveraging link perception for distributed beacon congestion control inVANETs,” IEEE Internet Things J., vol. 5, no. 6, pp. 4237–4249, 2018.

[48] S. Bharati and W. Zhuang, “CAH-MAC: Cooperative ADHOC MACfor vehicular networks,” IEEE J. Sel. Areas Commun., vol. 31, no. 9,pp. 470–479, 2013.

[49] N. Wisitpongphan, O. K. Tonguz, J. S. Parikh, P. Mudalige, F. Bai, andV. Sadekar, “Broadcast storm mitigation techniques in vehicular ad hocnetworks,” IEEE Wireless Commun., vol. 14, no. 6, pp. 84–94, 2007.

[50] J. Liu, J. Wan, B. Zeng, Q. Wang, H. Song, and M. Qiu, “A scalable andquick-response software defined vehicular network assisted by mobileedge computing,” IEEE Commun. Mag., vol. 55, no. 7, pp. 94–100,2017.

[51] K. Abboud, H. A. Omar, and W. Zhuang, “Interworking of DSRCand cellular network technologies for V2X communications: A survey,”IEEE Trans. Veh. Technol., vol. 65, no. 12, pp. 9457–9470, 2016.

[52] J. Liu, Y. Shi, Z. M. Fadlullah, and N. Kato, “Space-air-ground integratednetwork: A survey,” IEEE Commun. Surveys Tuts., vol. 20, no. 4, pp.2714–2741, 2018.

[53] Y. Zeng, R. Zhang, and T. J. Lim, “Wireless communications withunmanned aerial vehicles: opportunities and challenges,” IEEE Commun.Mag., vol. 54, no. 5, pp. 36–42, 2016.

[54] N. Zhang, S. Zhang, P. Yang, O. Alhussein, W. Zhuang, and X. Shen,“Software defined space-air-ground integrated vehicular networks:Challenges and solutions,” IEEE Commun. Mag., vol. 55, no. 7, pp.101–109, 2017.

[55] 3GPP, “3GPP TS 23.501, system architecture for the 5G system (release15),” 3rd Generation Partnership Project: Sophia-Antipolis, france,2017.

[56] Z. Su, Y. Hui, and T. H. Luan, “Distributed task allocation to enablecollaborative autonomous driving with network softwarization,” IEEE J.Sel. Areas Commun., vol. 36, no. 10, pp. 2175–2189, 2018.

[57] H. Peng, Q. Ye, and X. Shen, “Spectrum resource management for multi-access edge computing in autonomous vehicular networks,” IEEE Trans.Intell. Transp. Syst., to appear, DOI: 10.1109/TITS.2019.2922656.

[58] R. Bruschi, F. Davoli, P. Lago, and J. F. Pajo, “A multi-clusteringapproach to scale distributed tenant networks for mobile edgecomputing,” IEEE J. Sel. Areas Commun., vol. 37, no. 3, pp. 499–514,2019.

[59] “ETSI GR MEC 022 V2.1.1 multi-access edge computing(MEC); study on MEC support for V2X use cases,” EuropeanTelecommunications Standards Institute, Tech. Rep., 2018. [Online].Available: https://www.etsi.org/deliver/etsi gr/MEC/001 099/022/02.01.01 60/gr MEC022v020101p.pdf

[60] K. Kaur, S. Garg, G. S. Aujla, N. Kumar, J. J. Rodrigues, andM. Guizani, “Edge computing in the industrial Internet of Thingsenvironment: Software-defined-networks-based edge-cloud interplay,”IEEE Commun. Mag., vol. 56, no. 2, pp. 44–51, 2018.

[61] Y. Kim, N. An, J. Park, and H. Lim, “Mobility support for vehicularcloud radio-access-networks with edge computing,” in Proc. IEEECloudNet’ 18, Oct. 2018, pp. 1–4.

[62] S. Choo, J. Kim, and S. Pack, “Optimal task offloading and resourceallocation in software-defined vehicular edge computing,” in Proc. IEEEICTC’ 18, Oct. 2018, pp. 251–256.

[63] X. Li, D. Li, J. Wan, C. Liu, and M. Imran, “Adaptive transmissionoptimization in SDN-based industrial Internet of Things with edgecomputing,” IEEE Internet Things J., vol. 5, no. 3, pp. 1351–1360, 2018.

[64] C. Huang, M. Chiang, D. Dao, W. Su, S. Xu, and H. Zhou, “V2V dataoffloading for cellular network based on the software defined network(SDN) inside mobile edge computing (MEC) architecture,” IEEE Access,vol. 6, pp. 17 741–17 755, 2018.

[65] Z. Zhao, L. Guardalben, M. Karimzadeh, J. Silva, T. Braun, andS. Sargento, “Mobility prediction-assisted over-the-top edge prefetchingfor hierarchical VANETs,” IEEE J. Sel. Areas Commun., vol. 36, no. 8,pp. 1786–1801, 2018.

[66] F. Lyu, J. Ren, N. Cheng, P. Yang, M. Li, Y. Zhang, and X. Shen,“Demystifying traffic statistics for edge cache deployment in large-scaleWiFi system,” in Proc. IEEE ICDCS’ 19, Jul. 2019.

[67] G. Ma, Z. Wang, M. Zhang, J. Ye, M. Chen, and W. Zhu, “Understandingperformance of edge content caching for mobile video streaming,” IEEEJ. Sel. Areas Commun., vol. 35, no. 5, pp. 1076–1089, 2017.

[68] H. Huang, P. Luo, M. Li, D. Li, X. Li, W. Shu, and M. Wu, “Performanceevaluation of SUVnet with real-time traffic data,” IEEE Trans. Veh.Technol., vol. 56, no. 6, pp. 3381–3396, 2007.

[69] Y. Zhang, C. Li, T. H. Luan, Y. Fu, W. Shi, and L. Zhu, “A mobility-aware vehicular caching scheme in content centric networks: Model andoptimization,” IEEE Trans. Veh. Technol., vol. 68, no. 4, pp. 3100–3112,2019.

[70] H. Zhu, L. Fu, G. Xue, Y. Zhu, M. Li, and L. M. Ni, “Recognizingexponential inter-contact time in VANETs,” in Proc. IEEE INFOCOM’10, Mar. 2010, pp. 1–5.

[71] A. Abdrabou and W. Zhuang, “Probabilistic delay control and roadside unit placement for vehicular ad hoc networks with disruptedconnectivity,” IEEE J. Sel. Areas Commun., vol. 29, no. 1, pp. 129–139, 2011.

[72] P. Hao, L. Hu, J. Jiang, J. Hu, and X. Che, “Mobile edge provisionwith flexible deployment,” IEEE Trans. Serv. Comput., to appear,DOI:10.1109/TSC.2018.2842227.

[73] C. Liang, Y. He, F. R. Yu, and N. Zhao, “Enhancing QoE-aware wirelessedge caching with software-defined wireless networks,” IEEE Trans.Wireless Commun., vol. 16, no. 10, pp. 6912–6925, 2017.

[74] J. Chen, S. Yan, Q. Ye, W. Quan, P. Do, W. Zhuang, X. Shen, X. Li,and J. Rao, “An SDN-based transmission protocol with in-path packetcaching and retransmission,” in Proc. IEEE ICC’ 18, May 2019.

[75] P. Yang, N. Zhang, S. Zhang, L. Yu, J. Zhang, and X. Shen, “Contentpopularity prediction towards location-aware mobile edge caching,”IEEE Trans. Multimedia, vol. 21, no. 4, pp. 915–929, 2019.

[76] M. Chen, M. Mozaffari, W. Saad, C. Yin, M. Debbah, and C. S. Hong,“Caching in the sky: Proactive deployment of cache-enabled unmannedaerial vehicles for optimized quality-of-experience,” IEEE J. Sel. AreasCommun., vol. 35, no. 5, pp. 1046–1061, 2017.

[77] Z. Chang, L. Lei, Z. Zhou, S. Mao, and T. Ristaniemi, “Learn to cache:Machine learning for network edge caching in the big data era,” IEEEWireless Commun., vol. 25, no. 3, pp. 28–35, 2018.

[78] T. Trzciski and P. Rokita, “Predicting popularity of online videos usingsupport vector regression,” IEEE Trans. Multimedia, vol. 19, no. 11, pp.2561–2570, 2017.

16

[79] S. Wang, X. Zhang, Y. Zhang, L. Wang, J. Yang, and W. Wang, “Asurvey on mobile edge networks: Convergence of computing, cachingand communications,” IEEE Access, vol. 5, pp. 6757–6779, 2017.

[80] “3rd Generation Partnership Project; Technical Specification GroupServices and System Aspects; Release 15 Description; Summary of Rel-15 Work Items (Release 15),” 3GPP TR 21.915 V2.0.0, pp. 1–118, Sept.2019.

[81] “3rd Generation Partnership Project; Technical Specification GroupServices and System Aspects; Release 16 Description; Summary of Rel-16 Work Items (Release 16),” 3GPP TR 21.916 V0.1.0, pp. 1–23, Sept.2019.

[82] Q. Ye, J. Li, K. Qu, W. Zhuang, X. Shen, and X. Li, “End-to-end qualityof service in 5G networks: Examining the effectiveness of a networkslicing framework,” IEEE Veh. Technol. Mag., vol. 13, no. 2, pp. 65–74,2018.

[83] C. Campolo, A. Molinaro, A. Iera, and F. Menichella, “5G networkslicing for vehicle-to-everything services,” IEEE Wireless Commun.,vol. 24, no. 6, pp. 38–45, 2017.

[84] K. Xiong, S. Leng, J. Hu, X. Chen, and K. Yang, “Smart network slicingfor vehicular fog-RANs,” IEEE Trans. Veh. Technol., vol. 68, no. 4, pp.3075–3085, 2019.

[85] S. Zhang, W. Quan, J. Li, W. Shi, P. Yang, and X. Shen, “Air-groundintegrated vehicular network slicing with content pushing and caching,”IEEE J. Sel. Areas Commun., vol. 36, no. 9, pp. 2114–2127, 2018.

[86] J. Wang, B. He, J. Wang, and T. Li, “Intelligent VNFs selection basedon traffic identification in vehicular cloud networks,” IEEE Trans. Veh.Technol., vol. 68, no. 5, pp. 4140–4147, 2019.

[87] J. Wang, Q. Qi, S. Qing, and J. Liao, “Elastic vehicular resourceproviding based on service function-group resource mapping of smartidentify network,” IEEE Syst. J., vol. 12, no. 2, pp. 1897–1908, 2018.

[88] C. Wang, Y. He, F. R. Yu, Q. Chen, and L. Tang, “Integration ofnetworking, caching, and computing in wireless systems: A survey, someresearch issues, and challenges,” IEEE Commun. Surveys Tuts., vol. 20,no. 1, pp. 7–38, 2018.

[89] Q. Chen, F. R. Yu, T. Huang, R. Xie, J. Liu, and Y. Liu, “Joint resourceallocation for software-defined networking, caching, and computing,”IEEE/ACM Trans. Netw., vol. 26, no. 1, pp. 274–287, 2018.

[90] Y. He, N. Zhao, and H. Yin, “Integrated networking, caching, andcomputing for connected vehicles: A deep reinforcement learningapproach,” IEEE Trans. Veh. Technol., vol. 67, no. 1, pp. 44–55, 2018.

Weihua Zhuang (M’93-SM’01-F’08) has beenwith the Department of Electrical and ComputerEngineering, University of Waterloo, Canada, since1993, where she is a Professor and a Tier ICanada Research Chair in Wireless CommunicationNetworks. She is the recipient of 2017 TechnicalRecognition Award from IEEE CommunicationsSociety Ad Hoc & Sensor Networks TechnicalCommittee, and a co-recipient of several best paperawards from IEEE conferences. Dr. Zhuang was theEditor-in-Chief of IEEE Transactions on Vehicular

Technology (2007-2013), Technical Program Chair/Co-Chair of IEEE VTCFall 2017 and Fall 2016, and the Technical Program Symposia Chair of theIEEE Globecom 2011. She is a Fellow of the IEEE, the Royal Society ofCanada, the Canadian Academy of Engineering, and the Engineering Instituteof Canada. Dr. Zhuang is an elected member in the Board of Governors andVP Publications of the IEEE Vehicular Technology Society.

Qiang Ye (S’16-M’17) has been an assistantprofessor with the Department of Electrical andComputer Engineering and Technology, MinnesotaState University, Mankato, MN, USA sinceSeptember 2019. He received his Ph.D. degreein electrical and computer engineering from theUniversity of Waterloo, Waterloo, ON, Canada,in 2016. He had been with the Department ofElectrical and Computer Engineering, University ofWaterloo, as a post-doctoral fellow from December2016 to November 2018 and a research associate

from December 2018 to September 2019. His current research interestsinclude 5G networks, SDN and NFV, network slicing, AI and machinelearning for future networking, protocol design and end-to-end performanceanalysis for IoT.

Feng Lyu (S’16-M’18) received his Ph.D degreein the Department of Computer Science andEngineering from Shanghai Jiao Tong University,Shanghai, China, in 2018, and BS degree insoftware engineering from Central South University,Changsha, China, in 2013. Since Sept. 2018, hehas worked as a postdoctoral fellow in BBCRGroup, Department of Electrical and ComputerEngineering, University of Waterloo, Canada. Hisresearch interests include vehicular ad hoc networks,space-air-ground integrated network, cloud/edge

computing, and big data driven application design. He is a member of theIEEE Computer and IEEE Communication Society.

Nan Cheng (S’12-M’16) received the Ph.D.degree from the Department of Electrical andComputer Engineering, University of Waterloo in2016, and B.E. degree and the M.S. degree fromthe Department of Electronics and InformationEngineering, Tongji University, Shanghai, China,in 2009 and 2012, respectively. He is currentlya professor with School of TelecommunicationEngineering, Xidian University, Shaanxi, China.He worked as a Post-doctoral fellow with theDepartment of Electrical and Computer Engineering,

University of Toronto, from 2017 to 2018. His current research focuses onspace-air-ground integrated system, big data in vehicular networks, and self-driving system. His research interests also include performance analysis,MAC, opportunistic communication, and application of AI for vehicularnetworks. He is also interested in space-air-ground integrated networks.

Ju Ren (S’15, M’16) received the B.Sc. (2009),M.Sc. (2012), Ph.D. (2016) degrees all in computerscience, from Central South University, China.During 2013-2015, he was a visiting Ph.D. studentin the Department of Electrical and ComputerEngineering, University of Waterloo, Canada.Currently, he is a professor with the Schoolof Computer Science and Engineering, CentralSouth University, China. His research interestsinclude Internet-of-Things, wireless communication,network computing and cloud computing. He is a

co-recipient of the best paper award of IEEE IoP’18 and the most popularpaper award of Chinese Journal of Electronics (2015-2018). He currentlyserves/served as an associate editor for IEEE Transactions on VehicularTechnology and Peer-to-Peer Networking and Applications, and a TPCmember of many international conferences including IEEE INFOCOM’19/18,Globecom’17, WCNC17, WCSP16, etc. He also served as the TPC chair ofIEEE BigDataSE 2019, a poster co-chair of IEEE MASS’18, a track co-chairfor IEEE/CIC ICCC’19, IEEE VTC’17 Fall and IEEE I-SPAN’18, and anactive reviewer for over 20 international journals. He is a member of IEEEand ACM.