IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019 953

An End-to-End Load Balancer Based on DeepLearning for Vehicular Network Traffic Control

Jinglin Li , Member, IEEE, Guiyang Luo , Nan Cheng , Member, IEEE,

Quan Yuan , Zhiheng Wu, Shang Gao, and Zhihan Liu

Abstract—The infrastructure to vehicle (I2V) communicationboosts a large number of prevailing vehicular services, which canprovide vehicles with external information, storage, and comput-ing power located at both mobile edge server (MES) and remotecloud. However, vehicle distribution is imbalanced due to the spa-tial inhomogeneity and temporal dynamics. As a consequence, thecommunication load for MES is imbalanced and vehicles may suf-fer from poor I2V communications where the MES is overloaded.In this paper, we propose a novel proactively load balancingapproach that enables efficient cooperation among MESs, whichis referred to as end-to-end load balancer (E2LB). E2LB sched-ules the cached data among MESs based on the predicted roadtraffic situation. First, a convolutional neural network (CNN)is applied to efficiently learn the spatio-temporal correlation inorder to predict the road traffic situation. Then, we formulatethe load balancing problem as a nonlinear programming (NLP)problem and a novel framework based on CNN is adopted toapproximate the NLP optimization. Finally, we connect the aboveneural networks into an end-to-end neural network to jointlyoptimize the performance, where the input is the historical traf-fic situation while the output is the balanced scheduling solution.E2LB can guarantee the real-time scheduling, since the calling ofa well-trained neural network only requires a small number ofsimple operations. Experiments on the trajectories of taxis andbuses in Beijing demonstrate the efficiency and effectiveness ofE2LB.

Index Terms—Convolutional neural network (CNN), deeplearning, end-to-end, load balance, network traffic control.

I. INTRODUCTION

MOBILE edge computing (MEC) is a system-level hor-izontal architecture that distributes resources and ser-

vices of computing, storage, control, and networking anywhere

Manuscript received April 25, 2018; revised July 8, 2018; acceptedAugust 11, 2018. Date of publication August 21, 2018; date of current versionFebruary 25, 2019. This work was supported in part by the National Scienceand Technology Major Project of the Ministry of Science and Technologyof China under Grant 2016ZX03001025-003, in part by the Natural ScienceFoundation of Beijing under Grant 4181002, in part by the Natural ScienceFoundation of China under Grant 91638204 and Grant 61876023, in part bythe BUPT Excellent Ph.D. Students Foundation under Grant CX2018210, andin part by the Natural Sciences and Engineering Research Council, Canada.(Corresponding author: Guiyang Luo.)

J. Li, G. Luo, Q. Yuan, Z. Wu, S. Gao, and Z. Liu are withthe State Key Laboratory of Networking and Switching Technology,Beijing University of Posts and Telecommunications, Beijing 100876, China(e-mail: jlli@bupt.edu.cn; luoguiyang@bupt.edu.cn; yuanquan@bupt.edu.cn;zh_woo@bupt.edu.cn; gshtime@bupt.edu.cn; zhihan@bupt.edu.cn).

N. Cheng is with the Electrical and Computer Engineering Department,University of Waterloo, Waterloo, ON N2L 3G1, Canada (e-mail:n5cheng@uwaterloo.ca).

Digital Object Identifier 10.1109/JIOT.2018.2866435

along the continuum from cloud to things, which can providecomputing and networking ability to things [1], [2]. MEC willbring about new characteristics to current vehicular ad hocnetworks (VANETs) [3], [4].

1) The Couple of Data Transmission and Process: Thelarge quantity of data generated by end sensors willbe aggregated at the MEC for real-time analysis, min-ing and process. These sensors include various sensorsmounted on autonomous cars (e.g., camera, Lidar, GPS,etc.), infrastructures (e.g., inductive loops, roadside cam-eras, roadside communication devices, etc.) of intelligenttransportation system (ITS), and even hands-on devicesof passengers. These data will be processed at a mobileedge server (MES) and only the necessary results willbe transmitted into the remote cloud [5].

2) Edge Caching for Better Quality of Service (QoS): Mostof the data communication can be predicted based onthe analysis, mining, and prediction of various senseddata [6], [7]. In order to provide a better QoS, the MESwould precache the data to lower down the latency andprovide better services.

3) Services Provision at Network Edge: For example, thehigh definition (HD) maps will be updated and storeddirectly at the MES.

Autonomous car is in desperate needs of accessing to anMES in order to equip itself with additional computation andstorage power [8], obtain necessary information for efficientand safe driving, and enrich the experience of passengers.The infrastructure to vehicle (I2V) communication would meetthese needs and exhibit a promising future for ITS [9], [10].Each vehicle benefits a lot from I2V communication. Forexample, HD maps contribute to precise localization, real-time planning and control, and high-performance driving atthe limits [11]. Each vehicle can obtain the HD maps whenconnect to MES, which could be maintained and real-timeupdated at MES. Besides, I2V can broaden the informationand entertainment range of passengers and enrich the wonder-ful travel experience. As a consequence, I2V communicationplays a critical role in Internet of Vehicles, in improving thesafety and efficiency for autonomous car, and in enabling theefficient management and control of ITS [8].

The most popular way for I2V is directly connecting toinfrastructures, such as base station to vehicle by cellularcommunication, roadside unit to vehicle by 802.11p, accesspoint to vehicle by WiFi, and so forth [12]. These infrastruc-tures have a fixed coverage and nodes within coverage share

2327-4662 c© 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

954 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

Fig. 1. Architecture and the information processing procedures of E2LB.

the communication resources. However, vehicle distribution ishighly imbalanced [13], leading to a unreliable QoS for theI2V communication. In a heavy vehicle density scenario, itis hard to satisfy the I2V requirements due to heavy trafficdemand and limited wireless resources, as there exists a largenumber of vehicles, while in a light vehicle density scenario,the network resources cannot be fully utilized. There havebeen a lot of researches focusing on balancing the load forcellular network [14]. Most prior load balancing techniqueswere based on cell association [15], relays on top of cellu-lar networks [16], and orchestrated device-to-device (D2D)communication [17], etc. These techniques focus on balancingthe load within the coverage of a single macro base station(MBS). These techniques have the following disadvantageswhen adopted to balancing load for VANETs.

1) Considering the small coverage of MBS in urban envi-ronment and vehicle distribution characteristics, vehicledensities have a tremendous difference on differentMBSs. Balancing load among adjacent MBSs can sig-nificantly improve the network utilization.

2) Traditional schemes are of low efficiency and willcause heavy overhead, since the rapid mobility of vehi-cles require considerable communication overhead toupdate the balancing strategy. Meanwhile, road traf-fic presents high spatio-temporal correlation, therefore,vehicle trace is highly predictable, which can be elab-orately orchestrated to support balancing the load ofadjacent MBSs.

Therefore, in this paper, we consider balancing the loadproactively among adjacent MBSs from the perspective of

network operator. Its objective is to make sure the load for dif-ferent MBSs are more balanced, making full use of networkresources, and improving the QoS perceived by nodes. Thebasic idea is that we could choose to download the acquireddelay-insensitive data in a network with a better QoS, ratherthan a network with poor QoS. These delay-insensitive datainclude the HD maps for autonomous driving cars, the popularmultimedia content for passengers, and even the products andinformation on sales promotions. These data will be precachedat the MES to lower down the latency and every vehicle isinterested in it. As shown in Fig. 1, there exist two adjacentMESs, MES1 and MES2. MES2 will experience a higher loadas there will be a large number of vehicles accumulated in itscoverage. The large number of data requests and little timefor data transmission would lead to poor QoS in MES2. Toaddress the problem, we can balance the load among adjacentMESs by scheduling proactive data downloading. For example,some vehicles passing MES1 to MES2 can download the HDmaps for MES2 region proactively from MES1. These vehicleshave already obtained their required data when they arrive atMES2, resulting that other vehicles in MES2 also experiencea better QoS. Accordingly, part of MES2 cached data shouldbe scheduled to MES1.

To put this idea into action, we present end-to-end loadbalancer (E2LB), a novel scheme that deals with the loadbalance among adjacent MBSs in VANETs based on end-to-end deep learning. E2LB consists of two major modules,one is the transportation traffic situation prediction, whichaims at learning the spatio-temporal correlation of the traffic.Convolutional neural network (CNN) is applied to learn the

LI et al.: E2LB BASED ON DEEP LEARNING FOR VEHICULAR NETWORK TRAFFIC CONTROL 955

spatial correlation. As for the temporal correlation, multiplehistorical observations are inputted to the CNN, learning thetemporal correlation in a supervised manner. The other mod-ule is load balancing based on the predicted traffic situation.This can be formulated as a nonlinear programming (NLP)problem. The formulated NLP requires a considerable num-ber of iterations for convergence, which poses challenges forreal-time processing. To handle this, a novel neural network isproposed, which enables the real-time scheduling by approx-imating the optimization with heavy computing power in thetraining stage. During the execution, this well-trained networkcan come up with the load balancing results in real-time.Meanwhile, these two modules are combined to an end-to-end neural network, which is fine-tuned to enhance itsperformance. The E2LB is demonstrated to be very effective,especially for its real-time processing. In particular, the maincontributions of this paper are outlined as follows.

1) We propose an E2LB that directly outputs the scheduleof vehicle traffic download and MES caching from his-toric traffic information, which is proactive rather thanreactive load balancing approach. This load balancercan handle imbalanced QoS experienced by differentMBSs resulting from the large scale imbalanced vehi-cle distribution, thus improving the network resourceutilization.

2) We propose an efficiently traffic prediction scheme thatcan accurately predict the inflow and outflow traffic ofa specific region based on historical traffic informa-tion. We treat the traffic situation as image and adoptthe CNN for the prediction, which can better learn thespatio-temporal correlation.

3) We formulate the load balancing among MESs by anNLP problem. A CNN-based method is proposed toapproximate the optimization problem very quickly tosatisfy the real-time requirement of the scheduling invehicular environments. We design a sophisticated deepneural network (DNN), which first extract the commonfeatures, and then learn the directional caching strategiesthrough separated neural network models.

4) We use real-world data sets to verify our proposed loadbalancer for vehicular delay-insensitive services. Thereal-world traffic data sets can accurately reflect the traf-fic conditions such as spatio-temporal variations in realcity scenarios, and our devised schemes can thereforebe directly applied to practice.

The remainder of this paper is organized as follows. Therecent literatures are reviewed in Section II. Then, we for-mulate the traffic situation prediction and the load balanceproblem in Section III. The details of our proposed deeplearning-based algorithms are discussed in Section IV, fol-lowed by extensive experiments in Section V. Finally, thispaper is concluded in Section VI.

II. LITERATURE REVIEW

E2LB balances the load based on the predicted traffic sit-uation. The load balancing problem is formulated as an NLPand solved by applying a deep learning network to directly

approximate the optimization. In this section, we investigatethe related works on load balance, traffic situation prediction,and optimization approximation by deep learning, respectively.

A. Load Balance

Load balancing distributes a workload across multiple enti-ties, which can achieve optimal utilization, maximize through-put, minimize response time, and avoid overload. Therehas been a lot of researches on load balancing for cellu-lar networks. Sarma et al. [18] dealt with load-imbalanceproblem by distributing the traffic load among the overlap-ping access points in a WiFi hotspot, where the WiMAXnetwork is used to distribute traffic load among the underlyingWiFi hotspots, as it has access to all the users information.Franco and de Marca [19] proposed a self-optimizing cellrange expansion (CRE) scheme based on a statistical learn-ing approach for an LTE heterogeneous network. The proposescheme dynamically expands the small cell coverage accord-ing to traffic conditions. Abbas et al. [20] adopted reversefrequency allocation scheme along with CRE-based user asso-ciation to abate heavy intercell interference. This schemeschedules users from overloaded macro-cells to underloadedsmall cells, thus balancing the load. Liu et al. [17] proposeda D2D load balancing paradigm, which exploits intercell D2Dcommunication and dynamically relay traffic of a busy cellto adjacent under-utilized cells to improve spectrum tempo-ral efficiency. Following this paper, Hajiesmaili et al. [21]investigated incentive mechanism to encourage device partic-ipation based on an online procurement auction framework.Nagaraj et al. [22] proposed a new tool Libra to effectivelyassess the impact of load balancing in cellular networks. Theycaptured imbalance measure using Gini score, which is generic(applicable to a wide variety of load metrics), scale-free (inde-pendent of the number of neighbors), and automated (enablesrapid analysis across a large number of network elements).

These load balancing approaches focus on balancing theload within coverage of an MBS, which cannot be directlyapplied to large scale load balancing in VANETs. Besides,most of these approaches are reactive rather than proactive. InITS, trajectory of vehicle can be highly predicted, therefore,the load balance can be more proactive to better improve thenetwork utilization.

B. Traffic Situation Prediction

Traffic situation prediction is significant to a wide range ofapplications, including traffic forecasting, vehicle navigation,vehicle routing, and congestion management. Therefore, thisproblem has gained extensive researches. These approachescan be divided into two categories: 1) model driven and 2) datadriven. Model-driven approaches try to computationally modelthe road network and, usually through simulation, analyze theperformance and behavior of the drivers. These approaches areespecially useful when planning changes to existing infras-tructure. More related works on model driven can be foundat [23]. The availability of large scale transportation dataand huge computation power allow the direct application of

956 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

data-driven approaches. Abadi et al. [24] adopted an autore-gressive model that adapts itself to unpredictable events fortraffic prediction, where traffic flow data is generated througha dynamic traffic simulator. Ma et al. [25] proposed a CNN-based method that learns traffic as images. This model consistsof two consecutive steps: 1) abstract traffic feature extractionand 2) network-wide traffic speed prediction. Yang et al. [26]adopted stacked autoencoder Levenberg–Marquardt model toimprove prediction accuracy. This model is designed usingthe Taguchi method to develop an optimized structure andto learn traffic flow features through layer-by-layer featuregranulation with a greedy layerwise unsupervised learningalgorithm. Lv et al. [27] proposed a novel deep-learning-basedtraffic flow prediction method, which considers the spatialand temporal correlations inherently. They employ a stackedautoencoder model to learn generic traffic flow features andthen predict traffic flow based on the extracted features.However, the autoencoder model cannot well capture the spa-cial correlation, as it needs sequence input. Zhang et al. [28]forecast the crowds of a city and employ the residual neu-ral network framework to model the temporal closeness,period, and trend properties of crowd traffic. Inspired by thesedata driven approaches, we exploit the CNN to capture thespatio-temporal correlation to predict the traffic. In order toefficiently learn the spatial-temporal correlation, we treat thetraffic situation as image, which can better reserve the spatialcorrelation.

C. Solving Optimization by Deep Learning

Deep learning has been applied to a variety of fields includ-ing computer vision, speech recognition, natural language pro-cessing, audio recognition, social network filtering, machinetranslation, bioinformatics and drug design, where they haveproduced results comparable to and in some cases superior tohuman experts. Deep learning adopts a cascade of multiplelayers of nonlinear processing units for feature extraction andtransformation. The effectiveness on deep learning can beensured by the universal approximation theorem, which statesthat a feed-forward network with a single hidden layer contain-ing a finite number of neurons, can approximate continuousfunctions on compact subsets of Rn, under mild assumptionson the activation function [29]. Andrychowicz et al. [30]showed how the design of an optimization algorithm can becast as a learning problem, allowing the algorithm to learn toexploit structure in the problems of interest in an automaticway. They implemented learned algorithms by long short-termmemory networks, which outperforms generic, hand-designedcompetitors on the tasks for which they are trained, andalso generalizes well to new tasks with similar structure.Vinyals et al. [31] proposed pointer net, which can be usedto learn approximate solutions to solve the traditional NPproblem Travelling Salesman Problem (TSP). The proposedmodel is proved to be effective and have good generalizationability. Similar work can be found in [32], they proposed asingle meta-learning algorithm, which combines reinforcementlearning and graph embedding to solve TSP problem.

Aside from the enormous success for deep learning inthe field of computer vision, speech recognition, natural lan-guage processing, and even for tradition optimization problem,deep learning has been widely applied in wireless networks.Sun et al. [33] treated the input and output of a resource allo-cation algorithm as an unknown nonlinear mapping and usea deep learning network to approximate it. Mao et al. [34]adopted deep reinforcement learning (DRL) for the task ofpacking tasks with multiple resource demands, which is formu-lated as a optimization problem. The training data is generatedby the optimization algorithm, thus, it is actually to learnthe optimization through a DRL. Liu et al. [35] exploiteddeep learning to find the latent relationship between flowinformation and link usage in optimal solution to reduce com-putational cost. Tang et al. [36] proposed a deep CNN tolearn routing protocol, which learns experiences regardingnetwork abnormalities such as congestion from the trainingdata. Wang et al. [37] proposed a deep learning approach forrealizing device-free wireless localization and activity recog-nition, which is achieved by a sparse autoencoder networkto automatically learn discriminative features from the wire-less signals and a softmax-regression-based machine learningframework to merge the learned features. Inspired by all thesuccess, in this paper, we design a novel deep CNN to approx-imate the formulated NLP problem to efficiently balance theload.

III. SYSTEM MODEL AND PRELIMINARIES

In this section, we give an overview of the proposed E2LB,which includes traffic situation prediction and a load balancer.Then, we formulate the traffic situation prediction problem andtransform the load balancer problem into an NLP problem.

A. Overview of E2LB

E2LB actually balance the load of adjacent MBSs (schedul-ing the cached data among MBS) based on the real-timesensed traffic flow, which is also termed as traffic situation.The architecture of E2LB is shown in Fig. 1, which consistsfour phases.

1) Data Crowdsensing: At first, a large number of sensors,e.g., the various sensors mounted on vehicles, the cam-era at the roadsides, the inductive loops, the hands-ondevices of passengers, etc., will participate in sensingthe traffic [38]. These sensors will generate large volumeand heterogeneous data.

2) Data Process at MES: Transmitting these data toremote cloud for processing would congest the backhaulnetwork and induce longer latency. As a consequence,these data will be processed and analyzed at the MESand only the necessary results will be transmitted intoremote cloud.

3) Predict the Traffic Situation and Schedule the LoadBalancing at the Remote Cloud: In E2LB, we need toreal-time construct the traffic situation, which requiresthe sensed traffic information from all over the city.Therefore, each MES is required to upload the processedresults to the remote cloud, and then, the traffic situation

LI et al.: E2LB BASED ON DEEP LEARNING FOR VEHICULAR NETWORK TRAFFIC CONTROL 957

(a) (b) (c)

(d) (e) (f)

Fig. 2. Traffic situation of Beijing at different times, which exhibit spatio-temporal variations and correlation. (a) 6:00. (b) 9:00. (c) 12:00. (d) 15:00.(e) 16:00. (f) 19:00.

can be real-time obtained and updated in remote cloud.Based on the real-time traffic situation information, wecould predict the traffic situation. Afterward, the remotecloud balance the large scale load-based the predictedsituation.

4) Data Caching at the MES: MESs cache the data basedon the scheduling strategy of E2LB. Consequently, tar-get vehicles can download the required data in a lesscongested network since their interested data is cachedin advance.

The traffic situation exhibits highly spatio-temporal vari-ations and correlation [28], as shown in Fig. 2. From theperspective of temporal variations, the traffic at 6:00 is verylight, while that at 9:00 becomes more heavy. As for the spatialvariations, traffic distributions at different times are distinct.For example, the traffic at 6:00 congregates in the northeastof the city, while at 15:00, and the traffic congregates in thedowntown area. Therefore, the traffic distribution is imbal-anced both at spatial and temporal dimensions. Due to thespatio-temporal correlation of traffic situation, we can predictthe traffic. Based on this predicted traffic, we could balancethe load for MESs. As shown in Fig. 1, there exist two adja-cent MESs, MES1 and MES2. Based on the predicted trafficsituation, MES2 will experience a higher load as there will bea large number of vehicles accumulated within the coverage ofMES2. These vehicles could request more data transmission,which might exceed the capacity and lead to a poor QoS.The target vehicles that will pass from MES1 to MES2, canchoose to download the interested data in MES1, experiencinga better QoS. A part of vehicles have already obtained theirinterested data when they arrive at MES2, consequently, othervehicles in MES2 also experience a better QoS. Therefore, weneed to precache the interested data for the target vehicles inMES1 in advance. Each MES should precache content basedon the load balancing strategy from remote cloud. After theprecaching, each MES schedules the download for the tar-get vehicles. Taking the HD map as an example [39], due tolocation-based service attributes, each MES maintains the HDmaps of the road within its coverage. In general, each vehi-cle obtain the HD maps of a region when they arrive at theregion. However, in the above case, we need to offload partof HD maps from MES2 to MES1 and schedule these targetvehicles to download the HD maps of MES2 when they are

(a) (b)

Fig. 3. Region: partition the area (inner area of Beijing Fourth Ring) intomultiple grids. Each grid is the coverage of an MBS and an MES is connectedto each MBS. Based on region partition algorithm, we illustrate the inflow ofBeijing at 9:00 on September 22, 2011. (a) Grid. (b) Inflow matrix.

within communication range of MES1, as shown in Fig. 1. Theload balancing strategy in this case is the amount of offloadedHD maps between neighboring MESs. Each MES schedulesthe target vehicles to download the interested HD maps inadvance.

B. Formulation of Traffic Situation Prediction Problem

Vehicular traffic flows exhibit highly spatio-temporal corre-lation, as vehicle trajectory is constrained by road topologyand travel demand presents an evident periodicity. Thanks tothis correlation, traffic flows are highly predictable. We firstdefine the region and traffic flow, then formulate the trafficsituation prediction problem.

The considered area is partitioned into multiple regions, andin each region, the inflow and outflow can be crowdsensed byfloating cars, induced loops, cameras, and so forth. There aremany definitions of a region in terms of different granularitiesand semantic meanings. In this paper, we adopt the regionpartition scheme [28] as described in Definition 1.

Definition 1 (Region): In this paper, we partition the con-sidered area into M × N grids based on the longitude andlatitude, where a grid (m, n) that lies at the mth row and thenth column denotes a region, as shown in Fig. 3(a).

Then, the inflow and outflow for each region (m, n), 0 ≤m < M, 0 ≤ n < N, are adopted to represent the trafficcharacteristic for the region, which are defined as follows.

Definition 2 (Inflow and Outflow): Let St(m, n) be the col-lection of vehicles that lie in grid (m, n) at the tth time interval.The inflow and outflow of grid (m, n) at the tth time intervalare defined, respectively, as

fin,t(m, n) = St(m, n) − St−1(m, n)

fout,t(m, n) = St+1(m, n) − St(m, n) (1)

where S1 −S2 is the difference set between set S1 and set S2,which is defined as S1 − S2 = {x|x ∈ S1, x /∈ S2}.

At the tth time interval, inflow and outflow in all M × Nregions can be denoted by a tensor Ft ∈ R

2×M×N ,where (Ft)in,m,n = fin,t(m, n) and (Ft)out,m,n = fout,t(m, n).Therefore, in each time interval, we could obtain the inflowand outflow Ft for the considered area. Based on this inflowand outflow, the traffic situation prediction problem can beformulated as follows.

958 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

Definition 3 (Traffic Situation Prediction): For the consid-ered area, which can be partitioned into regions according toDefinition 1, the historical observations are {Fh|h = t − l, t −l + 1, . . . , t}. Predict the {Fk|k = t + 1, t + 2, . . . , t + s} basedon the Fh.

C. Formulation of Load Balancer

For clearness, we choose an arbitrary region (m, n) anddenote it by ri. Therefore, the neighboring regions of ri canbe denoted by R(ri) = {(x1, x2)| max {|m − x1|, |n − x2|} =1, 0 ≤ x1 < M, 0 ≤ x2 < N}. For each region ri, thereexists an MBS B(ri). This base station can be regardedas an MES according to the white paper of The EuropeanTelecommunications Standards Institute [1]. The MES canprovide a surplus of storage and computing resources. Inorder to guarantee better QoS perceived by vehicles, a lot ofdata should be precached at B(ri), for example, the popularmultimedia and the HD maps for autonomous driving. Everyvehicle passing through the region ri could request the data byaccessing B(ri). The network may be congested at (t + 1)thtime interval as the number of vehicles within B(ri) is pre-dicted to be very large. The congested network will in turnseverely degrade QoS perceived by vehicles. Consequently,the requested data cannot be fully downloaded. Therefore,we need to transfer a part of data to adjacent base stationB(rj), rj ∈ R(ri), enabling efficient cooperation among adja-cent MESs, which can be shown in Fig. 4. Under this scenario,the vehicles that travel from rj at tth time interval to ri at(t + 1)th time interval will download the data in B(rj) at tthtime interval, as in tth time interval, the number of vehicles inB(rj) is less, and the network is idle. During the (t+1)th timeinterval at the base station B(ri), as a part of vehicles havedownloaded the data in advance, other vehicles experience abetter QoS.

The traffic flow from region ri to rj at tth time interval,rj ∈ R(ri), can be denoted by

St,ri→rj = fin,t(ri) ∩ St(ri) ∩ St(rj) + St(ri) ∩ St+1

(rj). (2)

The number and average speed of vehicles for flow St,ri→rj

can be defined, respectively, as

Nt,ri→rj = ∣∣St,ri→rj

∣∣

Vt,ri→rj =∑

α∈St,ri→rj

vt(α)

Nt,ri→rj

(3)

where vt(α) is the average speed for vehicle α at tth timeinterval.

For each vehicle α, it needs βα(ri) data (HD maps forautonomous driving) from the network when it passes throughthe region ri. Therefore, the total amount of data that need tobe delivered to vehicles at tth interval is denoted by CVt(ri),which is the sum of required data for vehicles of all Inflowand can be calculated by

CVt(ri) =∑

rj∈R(ri)

∑

α∈St,rj→ri

βα(ri). (4)

For B(ri), the data can be downloaded at tth time interval arerelated with the coverage of the station, the attribute of the

Fig. 4. Example for load balancing, the cached data in the center edgenode (1) can be scheduled to adjacent edge nodes (2–8).

station, and the distribution of vehicles, which can be denotedby CSt(ri). As the vehicle distribution is highly dynamic andimbalanced, B(ri) may be not able to deliver all the requesteddata CVt(ri), i.e., CVt(ri) > CSt(ri). The reason behind thisis that, if B(ri) cannot afford all data delivery, then, B(rj), rj ∈R(ri), can afford a part of data for B(ri). The traffic flowSt,rj→ri can download the part of data from B(rj), rather thandownload it from B(ri), which is shown in Fig. 4. Vehicles inSt,rj→ri need to download data that belong to B(ri) in B(rj),therefore, these data are required to be transmitted from B(ri)

to B(rj) in advance. The quantity of data transferred from B(ri)

to B(rj), ri ∈ R(ri) is denoted by Tt,ri→rj . Then, the quantityof data that need to be downloaded from B(ri) after balancingthe load is

ht(B(ri)) = CVt(ri) +∑

rj∈R(ri)

(Tt,rj→ri − Tt,ri→rj

). (5)

The utility of this scheduling for B(ri) is

h∗t (B(ri)) =

{0, ht(B(ri)) < CSt(ri)

ht(B(ri)) − CSt(ri), otherwise.(6)

The load balancer problem can be formulated as

min∑

ri

h∗t (B(ri))

︸ ︷︷ ︸Total unsatisfied download

+∑

ri

∑

rj∈R(ri)

∣∣Tt,rj→ri

∣∣

︸ ︷︷ ︸Penalty: L1 regularization

s.t.

{Tt,ri→rj ≥ 0, rj ∈ R(ri)

Tt,ri→rj ≤ Nt,ri→rj ∗ D(ri, rj

)/Vt,ri→rj ∗ C0

(7)

where C0 is the maximum data rate of the connection betweenthe MES and a vehicle, and D(ri, rj) is the distance betweenri and rj (Euclidean distance between the centers of tworegions). The first constraint demands that the offloaded dataTt,ri→rj is greater than zero and the second constraint restrictsthat the data offloaded Tt,ri→rj to the region rj should notexcess maximum capacity for the vehicle flow St,ri→rj . Thisflow capacity is the sum of the capacity of vehicles in flowSt,ri→rj . The objective is to minimize the total amount ofdata which fails to be downloaded, which is

∑ri

h∗t (B(ri)).

At the same time, we also need to minimize the numberof data that is loaded into neighbors. If the objective isonly to minimize the

∑ri

h∗t (B(ri)), there may exist multiple

optimal solutions. Among these solutions, a lot of scheduleddata Tt,rj→ri is not necessary. For example, two adjacentregions, r1 and r2, the network capacity is all 5, and theload for these two regions are 2 and 7, respectively. Then,

LI et al.: E2LB BASED ON DEEP LEARNING FOR VEHICULAR NETWORK TRAFFIC CONTROL 959

Fig. 5. End to end deep learning for E2LB, which consists of two modules, the PCNN for traffic situation prediction and LCNN for load balancing.

under this scenario, these exist multiple optimal solutions,(Tt,r1→r2 = 0, Tt,r2→r1 = 2), (Tt,r1→r2 = 0, Tt,r2→r1 = 2.5),(Tt,r2→r1 = 0, Tt,r1→r2 = 3), . . . However, the solutions exceptfor the first one are not necessary. By adding the penalty∑

ri

∑rj∈R(ri)

|Tt,rj→ri |, which is the L1 regularization, thesolution is unique and will avoid unnecessary data scheduling.The formulated load balancing problem is an NLP, and willbe approximated by a sophisticated designed deep learningnetwork in the following section.

IV. PROPOSED APPROACH

A. Traffic Prediction

Vehicles on roads are influenced by road topology, speedlimit, traffic congestion, etc. Therefore, vehicles in currentregion will appear in adjacent regions and rather than in avery far region in the next time interval. As a result, inflowand outflow are highly spatio-temporal correlated. The traf-fic prediction problem is actually to learn this correlation. Invirtue of this correlation, CNN is applied for this learningtask. Convolution kernel is a basic makeup for CNN. Thiskernel connects each output with only local values, as shown inFig. 6. This local connectivity is very suitable for learning thetemporal correlation in transportation traffic flow. Besides, thenumber of parameters can be greatly reduced due to the sharedweights by adopting CNN. In order to capture the temporalcorrelation, multiple historical observations are combined as atensor ({Fh|h = t − l, t − l + 1, . . . , t}). The CNN learns thetemporal correlation from this tensor in a supervise manner.The network model for the prediction CNN (PCNN) is shownin Fig. 5.

CNN is a kind of representation learning and is a veryspecial case for DNNs. DNN can approximate the class ofcompositional functions with the same accuracy as shallownetworks but with exponentially lower number of trainingparameters [40]. Moreover, the features at higher layers canlearn the high level representations, which is of great bene-fit for classification and prediction. As a result, DNN should

be considered for this prediction task rather than a shallownetwork. A DNN would have more parameters, which is hardto train. Besides, the convolution kernel is good at learningthe inherent spatio-temporal correlation. Therefore, we choosedeep CNN to predict future traffic situation, where weights areshared for certain kernel. The CNN will output predicted traf-fic flow {F∗

k |k = t + 1, t + 2, . . . , t + s} based on the Fh. Themean squared error between ground truth traffic situation Fk

and predicted traffic situation F∗k is defined as the loss for

deep CNN, which can be denoted by

LPCNN =M∑

i=0

N∑

j=0

t+s∑

o=t+1

∣∣Fo(i, j) − F∗

o (i, j)∣∣2

M ∗ N ∗ s. (8)

B. Load Balancer

The load balancer problem (7) is an NLP problem. Such anNLP program is hard to be optimized. For the past couple ofdecades, numerical optimization has played a central role inaddressing such problems. However, optimization algorithmsusually need multiple iterations before convergence and entailconsiderable complexity, which creates a serious gap betweentheoretical design/analysis and real-time processing. As for thecase of E2LB, the scheduling strategy which is related to thehistorical traffic flow observations should be executed just atthe beginning of next time interval. As a consequence, theload balancer problem has very urgent requirements on real-time processing. To address this challenge, we propose a noveldeep learning-based approach. The key idea is to treat the inputand output of load balancer NLP as an unknown nonlinearmapping and exploit a deep CNN to approximate it, which isreferred to as load balancing CNN (LCNN). If the nonlinearmapping can be learned accurately by a deep CNN of moderatesize, then scheduling can be done in nearly real-time, sincepassing the input through a DNN only requires a small numberof simple operations.

At this point, it remains unclear whether a DNN can beused to approximate the behavior of the formulated NLP (7).The answer to such a question is by no means trivial. The

960 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

Fig. 6. Kernel for CNN, which connects each output with only local values,is good at learning the spatial correlation.

constraints of (7) are all linear inequalities, which define apolyhedra. The objective of (7) is continuous. The formu-lated NLP can be regarded as a continuous mapping fromthe defined polyhedra to the objective. Actually, any continu-ous function on compact subsets of Rn can be approximatedto arbitrary accuracy by a feed-forward network with a sin-gle hidden layer, as long as we have sufficient neurons withnonlinear activation for that hidden layer [29]. Therefore, themapping can be approximated by a single hidden layer neu-ral network with sufficient neurons, not to mention the DNN.Sun et al. [41] showed that for any algorithm whose itera-tions represent continuous mappings, its input/output relationcan be approximated by a well trained neural network. Theintuition to learn the iterative algorithm is to learn a con-tinuous mapping from the problem parameter to the optimalsolution. Inspired by their work, we design a novel deep CNNto approximate the load balancing problem.

The load balancing problem has two distinctive featureswhich render deep CNN a suitable solution. First, the cacheddata in each region ri could be transferred only to neighborsR(ri), which we refer to as neighbor transferring. The rea-sons behind this are, on one hand, the traffic flows are incontinuous evolution. The network for certain region which iscongested in the current time interval would be idle for thenext time interval. Therefore, there is no need to transfer thedata in current region to very far regions. On the other hand,the number of variables for this problem is fixed. Once the loadbalancing for edge caching in VANET is deployed as a service,the number of regions should be matched with the number ofedge caching nodes (MES). Each MES precaches the amountof latency-insensitive data for the vehicles. The cached datain each region could be transferred to eight neighbor regions.As a consequence, the number of variable is M ∗ N ∗ 8 for theformulated load balancer problem. The cached data in eachregion could only be transfered to neighbors, which resem-bles the kernel in CNN, as shown in Fig. 6. Taken the kernel3 ∗ 3 as an example, this kernel output y = g(

∑xiwi + b),

where xi is the traffic flow information in the region, wi and bare the learnable variables. This kernel outputs the value basedon the local 3 ∗ 3 regions, which is in accordance with decid-ing the data transferring for the center region. As for the fixednumber of variables, which is of great benefit to CNN, theoutput of CNN is fixed. Besides, the number of parameterscan be greatly reduced by shared weights of CNN, which lay

the foundation for efficient and successful training of neuralnetwork by adopting deep CNN.

Although the output is fixed, it has a very high dimen-sion (M ∗ N ∗ 8). We encode it into an 8-channel figure, andeach channel is the scheduling load at the certain direction,which can be shown in Fig. 5. Such encoding enables the effi-cient representation for output of load balancer problem. Thereexists eight branches, and each branch has a unique struc-ture πi. To reduce the number of parameters, these branchesshare a common feature extraction framework, as shown inFig. 5. This shared framework can lower down the complex-ity and speed up the training process. The common featuresare connected with a framework πi for the scheduling resultat direction i, which consists of a dimension reduction oper-ation and mapping operation. The dimension reduction canextract the most suitable features for the load balancer at cer-tain direction. Then, these directional features are mapped tothe scheduling results at that direction. As there exists eightbranches, the loss can be calculated by

LLCNN =8∑

d=1

M∑

i=0

N∑

j=0

∣∣∣Tt+1,ri→rj(d) − T∗t+1,ri→rj

(d)

∣∣∣2

8 ∗ M ∗ N(9)

where Tt+1,ri→rj(d) is the quantity of data transfered from ri

to rj, rj is the closest neighbor at direction d and T∗t+1,ri→rj

isthe output of the deep CNN.

C. End-to-End Load Balancer

The E2LB learns the load balancing strategy Tt+1,ri→rj ,ri ∈ R(ri) based on historical traffic flow {Fh|h = t − l, t −l + 1, . . . , t}, which consists of a deep CNN for traffic sit-uation prediction (PCNN) and another deep CNN for loadbalancer (LCNN). Therefore, E2LB is a very DNN, whichgenerally have more parameters and is hard to train. Since thecomputational complexity of the neural network is measuredby the number of weights and bias, if we have exponen-tially fewer parameters, we could exponentially speed up thecomputational time in testing stage, which gives us a wayto design a run-time algorithm based on DNNs. Meanwhile,deep networks can approximate the class of compositionalfunctions with the same accuracy as shallow networks butwith exponentially lower number of training parameters [40].Directly training such a deep network require a large numberof training dataset and huge amount computation resources,and moreover, it is very hard to converge due to the gradi-ent vanishing and gradient explosion [42]. Therefore, we firsttrain the PCNN and LCNN individually. Then, we connectthese two CNNs and fine-tune it in an end-to-end manner.The training procedures is shown in Algorithm 1.

V. PERFORMANCE EVALUATION

To illustrate our methodology, experiments are conductedbased on the bus and taxi GPS data in Beijing. These dataconsist of nearly 30 000 taxi and 20 000 buses, with a periodof three months. These data are collected by GPS installed onvehicles, and each vehicle records the GPS location, veloc-ity, time stamp, and so forth. Considering the mobility of

LI et al.: E2LB BASED ON DEEP LEARNING FOR VEHICULAR NETWORK TRAFFIC CONTROL 961

Algorithm 1 Algorithm to Train E2LBStep 1: Train PCNNInput: Input {Fh|h = t − l, t − l + 1, · · · , t}, {Fk|k = t + 1,

t + 2, · · · , t + s}, output the predicted F∗k .

1: Setup the PCNN network and initialize weight matricesand bias vectors randomly.

2: Input Fh to the PCNN, output F∗k . Compute the loss

according to (8). Train the network by gradient descendalgorithm.

Step 2: Train LCNNInput: Input Fh, F∗

k , output the load balancing strategyT∗

t,ri→rj.

3: Setup the LCNN network and initialize weight matricesand bias vectors randomly.

4: Get Tt,ri→rj , rj ∈ R(ri)by solving (7). Encode it into an 8channel image.

5: Input Fh and F∗k to the LCNN, output T∗

t,ri→rj. Compute

the loss according to (9). Train the network by gradientdescend algorithm.

Step 3: Fine-tuning E2LBInput: Input {Fh|k = t − l, t − l + 1, · · · , t}, output the load

balancing strategy T∗t,ri→rj

.6: Load the weights of PCNN and LCNN.7: Input Fh to PCNN, and output F∗

k . Input Fk and F∗k to

LCNN and output T∗t,ri→rj

.8: Fine-tune the E2LB by gradient descend algorithm.

vehicles and coverage of edge server, we set the durationbetween two successive scheduling to 2 min. Therefore, weconvert the traffic data within 2 min into an image, includinginflow and outflow, which is shown in Fig. 3(b). Besides, thereexists a very small number of vehicles on the road between0:00 − 6:00. These data are not suitable to train the CNN,therefore, we eliminate these figures during the training. Forthe traffic prediction problem in E2LB, we only need to predict{Fk|k = t + 1} (i.e., s = 1) based historical observations{Fh|h = t − l, t − l + 1, . . . , t}. The reason is that the loadbalancer only requires the traffic situation in the next timeinterval, as the balancing is executed periodically. Therefore,these images {Fq|q = t − l, t − l + 1, . . . , t, t + 1} act as atraining or test sample. Due to the temporal relation, differentsamples should not overlap in time with each other. For exam-ple, the data in one hour has 60 images, and for l = 2, thereexist 60/(l+2) = 15 samples. For all data, we generate nearly19 000–24 000 samples as different l are considered. 80% ofthe samples are taken as the training sets while the rest 20%are regarded as test sets. All the samples are normalized undersame scale. Experiments are conducted on a single CPU core(2.40 GHz) of an Intel Xeon E5-2630 server with 64GB ofRAM. An NVIDIA Titan GPU is used for CNN computations.

A. Evaluation of PCNN

We compare PCNN with the following three baselines.1) Random Forest (RF): RF makes predictions based on

branches of decision trees.

TABLE IAVERAGE LOSS FOR PCNN UNDER DIFFERENT HIDDEN LAYERS AND

DIFFERENT l. FC: FULLY CONNECTED LAYERS

2) k-Nearest Neighbors Algorithm (KNN): KNN performsregression using the nearest points.

3) Ordinary Least Squares (OLS): OLS is a method forestimating the unknown parameters in a linear regressionmodel.

4) Stacked Autoencoder Model (SAE) [27]: They employan SAE model to learn generic traffic flow features andthen predict traffic flow based on the extracted features.The SAE model is trained through a greedy layerwiseunsupervised learning algorithm.

5) Deep Spatio-Temporal Residual Networks (ST-ResNet) [28]: A deep-learning-based predictionmodel for spatio-temporal data, employing the residualneural network framework to model the temporalcloseness, period, and trend properties of crowd traffic.

PCNN predicts future traffic Ft+1 based on historical obser-vation {Fh|h = t− l, t− l+1, . . . , t}. PCNN needs to learn thespatio-temporal correlation exhibited in the traffic flow data.We present the traffic flow of the whole city as an image,which makes it easy for the PCNN to mine the spatial corre-lation. As for the temporal correlation, we have tried differentnetwork layers and different l to maximize the performanceof PCNN. We have adopted the common architecture for acomputer vision task, where several convolutional layers arefirst applied to extract features. Then, a fully connected layerpredicts the future traffic based on the extracted features. Asshown in Table I, PCNN achieves a minimum average loss atfive hidden layers. PCNN achieves an average loss of 6.86 atfive hidden layers, which is smaller than that of four hiddenlayers. The reason behind this is that a neural network withmore layers can learn a more complex and nonlinear functionthan a shallow neural network, thus performing much betterin function approximation problems [43]. However, with theneural network becoming deeper, the average loss increases.The average loss for six hidden layers is greater than that offive hidden layers. The reason is as follows: 1) deeper networkintroduces more parameters that the network needs to learn,hence increasing the chances of over-fitting and 2) deepernetwork runs the chance of each layer just memorizing whatyou want the output to be, and ends up with a neural networkthat fails to generalize to new data. At the same time, wehave inputted different length (l) of historical observation{Fh|h = t − l, t − l + 1, . . . , t} to PCNN. PCNN learns thetemporal correlation in a supervised manner and achieves thebest performance at five hidden layers and l = 3. We use thisnetwork for the following experiments as it achieves the bestperformance.

962 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

(a)

(b) (c)

Fig. 7. Evaluation of PCNN in the region (14,15) as compared with RF,OLS, KNN, SAE, and ST-ResNet. (a) Traffic flow/of three days. (b) Trafficflow of one day. (c) Traffic flow of 40 min.

Fig. 8. Average loss of PCNN, RF, OLS, KNN, SAE, and ST-ResNet in fivedays.

Fig. 7 shows the prediction of traffic flow of the region(14, 15) as compared with five baselines, i.e., RF, KNN, OLS,SAE, and ST-ResNet. Fig. 7(a) shows the traffic flow ofBeijing over three days. The traffic flow exhibits a strong peri-odicity. This periodicity is related to the living style of humansand travel demands. The traffic reaches the highest during therush hours and is very small before dawn. Fig. 7(b) and (c)shows the prediction performance of PCNN as compared withRF, OLS, KNN, SAE, and ST-ResNet. PCNN and ST-ResNetcan better predict future traffic flow, while RF, OLS, KNN,and SAE cannot capture the temporal correlation, resultingin a large bias over the ground truth traffic flow. This canbe further shown in Fig. 8, which shows the average loss offour methods in five days. ST-ResNet performs slightly betterthan PCNN, as it has taken the temporal closeness, period,and trend properties into consideration, which complicates theneural network and requires more computing power to trainand run the neural network. However, PCNN is more effi-cient in terms of computing resources. PCNN achieves theminimum average loss as compared with RF, OLS, KNN, andSAE, which proves the effective and accuracy of PCNN forthe task of traffic flow prediction.

Fig. 9. Normalized size of HD maps for each region.

B. Evaluation of E2LB

We take the HD maps as the popular content to demon-strate the efficiency of E2LB. HD maps contains 3-D locationof all crucial aspects of a roadmap (e.g., lane markings,crosswalks, signs, barriers) and dynamic information thatfacilitates driving (e.g., traffic snarls, road conditions, acci-dents, lane closures). These maps specifically have extremelyhigh precision at centimeter-level accuracy. This is becauseautonomous vehicles need very precise instructions on howto maneuver themselves, how to navigate themselves aroundthe 3-D space. Therefore, the HD maps are of very large vol-ume in order to present highly detailed static and dynamicelements [38], [44]. We assume the size of HD maps in eachregion is directly proportional to the length and area of roadsin this region. Therefore, we get the normalized size of HDmaps for each region, which is shown in Fig. 9. In addi-tion, before vehicles travel to a region, every vehicle needsto update HD maps periodically in order to keep pace withthe changing driving conditions. Each region has a differentsize of HD maps. Meanwhile, with the rapid changing of traf-fic flow in each region, the total amount data required to bedownloaded from each MES is constantly changing. If therequired amount is larger than the network throughput, then,a part of vehicles can never download the required HD maps.We assume that 80% of the required data can be satisfiedat rush hours. Then, we could formulate the load balancingas an NLP problem. For the LCNN, based on the histori-cal observations {Fh|h = t − l, t − l + 1, . . . , t} and predictedtraffic flow F∗

t+1 by PCNN, we get Tt,ri→rj , rj ∈ R(ri) by solv-ing the NLP. Therefore, we could obtain the training samplesfor LCNN. For efficiently representing the Tt,ri→rj and fit-ting for the characteristics of CNN, we decode Tt,ri→rj into an8-channel image. The network architecture of LCNN is shownin Fig. 5. We first extract the common features by applyingthe feature extraction network, which consist of several convo-lutional layers. Then, we design a special network π to learnthe load balancing scheduling at certain direction based onthe extracted features. The π first reduce the dimension of theinput, followed by a convolution layer. We have tried variousconnection and depth to obtain the best performance, which isshown in Table II. According to the average loss of differentdepth and l, we select the [256, 128, 64, 32], πi : [16, 1] andl = 3 as the final network for LCNN.

Then, we derive the amount of data that cannot be down-loaded for each region, which is shown in Fig. 10. As shown

LI et al.: E2LB BASED ON DEEP LEARNING FOR VEHICULAR NETWORK TRAFFIC CONTROL 963

(a) (b) (c) (d)

Fig. 10. Performance of E2LB. The bottom is the traffic situation, the middle is the amount of traffic that cannot be downloaded, while the top is the resultof E2LB, which is the amount of traffic that cannot be downloaded after the scheduling of E2LB. We may draw the conclusion that most of data requestscan be satisfied by E2LB. (a) 6:00. (b) 9:00. (c) 15:00. (d) 20:00.

TABLE IIAVERAGE LOSS FOR LCNN UNDER DIFFERENT

HIDDEN LAYERS AND DIFFERENT l

in these figures, where the bottom is traffic situation (Inflow)and the middle is the amount of requested HD maps thatcannot be downloaded, the undownloaded data are relatedto the flow traffic. Such a problem cannot be efficientlysolved by improving the network capacity, since the reasonsare the changing requested data size and the dynamic num-ber of vehicles within each region in VANETs, resulting inimbalanced vehicle distribution. Imbalanced vehicle distribu-tion leads to imbalanced load for each MES. Therefore, wecould proactively balance the load based on the predictedtraffic distribution to solve this problem. Fig. 10 shows the

Fig. 11. Performance of E2LB as compared with LCNN, NLP optimization,without load balancing and CRE in one day. The inner figure is a combinationof line graph and stacked area chart, where the line graph is the total amount ofundownloaded data, while the stacked area charts show the absolute differenceamong LCNN, optimization, and E2LB.

performance of E2LB at different times. In these figures, themiddle is the amount of traffic that cannot be downloaded,while the top is the result of E2LB, which is the amountof traffic that cannot be downloaded after the scheduling ofE2LB. Through the efficient cooperation of adjacent MESs, theload of each MBS can be balanced. This load balancing canincrease the utilization of resources and improve the QoS ofeach vehicle.

Fig. 11 evaluates the performance of E2LB as comparedwith the results of formulated NLP optimization, LCNN andbias-based CRE in one day [45]. LCNN is the result in step 2and E2LB is the result of step 3, which connects two separatetrained networks into one neural network and trains it in an

964 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

end-to-end manner. Furthermore, we have compared our deeplearning approach with bias-based CRE [45]. As for the exam-ple in Fig. 1, MES1 will expand its communication range toserver a part of users within MES2, thus balance the load.As can be seen from this figure, without load balancing, alarge amount of data cannot be downloaded as the requesteddata excess the throughput of the network, which means thata lot of data requests cannot be satisfied, leading to a verylow QoS perceived by vehicles. In the case of HD maps,the driving experience can be harshly affected, as the HDmaps are of significance to safe and smooth driving. As forCRE, it can reduce the amount of data cannot be downloaded,however, it performs worse than our proposed deep learning-based approach. Most of the requested data can be satisfied bybalancing the load, which is achieved by the efficient coop-eration of adjacent MESs. Both E2LB and LCNN achievea good performance, as most of the requested data can bedownloaded. The performance of E2LB is better than thatof LCNN. The reason is that the end-to-end training (fine-tune) of E2LB has further improved the LCNN. The goodperformance of E2LB demonstrates the efficiency and accu-racy of applying neural network in the task of approximatingthe optimization.

VI. CONCLUSION

In this paper, we proposed a novel proactive load balancingapproach, namely E2LB, which enables efficient cooperationamong MBSs based on the predicted traffic situation. We haveadopted a deep CNN to learn the spatio-temporal correlationto predict the traffic situation. Then, we have formulated theload balancing problem as an NLP problem and treated it asnonlinear mapping between the input (traffic situation) and theoutput (load scheduling results through solving the NLP). Wehave proposed a special neural network to approximate themapping. Finally, we fine-tune these two networks in an end-to-end manner. Extensive experiments on the trajectories oftaxies and buses in Beijing have demonstrated the efficiencyand real-time scheduling of E2LB. We have demonstrated thepower of deep learning in the area of wireless communica-tion in this paper. To fully leverage the power of the artificialintelligence (AI) algorithms, some future research issues areprovided as follows.

A. Provisioning Resources for AI Algorithm by Integration ofComputation, Storage, and Communication

Currently, AI is powered by extensive big data and hugecomputation resources. Constrained by the capacity of a singlecomputation devices, a high bandwidth connection is requiredto exchange temporary states when several devices are usedin training large neural models. Besides, transmission of thesebig data generated by the end things to computing center poseschallenges to backhaul network. Therefore, networking hasbecome a bottleneck for AI systems. To handle this problem,integration of computation, storage, and communication isneeded to provision a better runtime platform.

B. Network Status Monitoring and NetworkKnowledge Discovery

AI can be used to quickly analyze all the network data (i.e.,radio access data, WLAN data, routers and switches data, etc.)within different network layers. This is something that is notpractically possible with people trying to manually correlateit. With the help of these AI algorithms, we are able to iden-tify anomalous build-ups of traffic and activity which may bethe result of malicious activities such as distributed denial-of-service attacks and attempted hacks. AI will be able to betterpredict traffic as it collects and analyzes data in real-time, sothat network managers are better prepared for big events suchas the Olympics, Black Friday, and Valentines Day, whichoften put the Internet under pressure.

C. Self-Configuration, Self-Optimization, and Self-Healingfor Next Generation Network

As these AI algorithms become more intelligent, it willfind faster and more foolproof methods of anticipating threatsand cleaning the network. Through real-time monitoring andanalysis, these AI algorithms are able to take efficientlymeasures to deal with encountered problems and create anautonomic response to illegitimate behaviors and attack vec-tors. For example, these AI algorithms can respond effectivelyto block incursions without requiring human analysts to pro-vide input. These algorithms help to build a self-configured,self-optimized, and self-healed next generation network.

REFERENCES

[1] Y. C. Hu, M. Patel, D. Sabella, N. Sprecher, and V. Young, “Mobile edgecomputing—A key technology towards 5G,” Sophia Antipolis, France,ETSI, White Paper, 2015.

[2] N. Cheng et al., “Air-ground integrated mobile edge networks:Architecture, challenges and opportunities,” IEEE Commun. Mag.,vol. 56, no. 8, pp. 26–32, Aug. 2018.

[3] S. Zhang et al., “Vehicular communication networks in automated driv-ing era,” arXiv preprint, arxiv: 1805.09583, 2018. [Online]. Available:https://arxiv.org/abs/1805.09583

[4] G. Luo, J. Li, Z. Liu, X. Tao, and F. Yang, “Physical layer securitywith untrusted relays in wireless cooperative networks,” in Proc. IEEEWireless Commun. Netw. Conf. (WCNC), Mar. 2017, pp. 1–6.

[5] W. Xu et al., “Internet of Vehicles in big data era,” IEEE/CAA J.Automatica Sinica, vol. 5, no. 1, pp. 19–35, Jan. 2018.

[6] X. Cheng, L. Fang, L. Yang, and S. Cui, “Mobile big data: Thefuel for data-driven wireless,” IEEE Internet Things J., vol. 4, no. 5,pp. 1489–1516, Oct. 2017.

[7] N. Cheng et al., “Big data driven vehicular networks,” IEEENetw., to be published. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8450539/

[8] H. Zhou et al., “Toward 5G spectrum sharing for immersive-experience-driven vehicular communications,” IEEE Wireless Commun., vol. 24,no. 6, pp. 30–37, Dec. 2017.

[9] G. Luo, S. Jia, Z. Liu, K. Zhu, and L. Zhang, “sdnMAC: A soft-ware defined networking based MAC protocol in VANETs,” in Proc.IEEE/ACM Int. Symp. Qual. Service (IWQoS), 2016, pp. 1–2.

[10] G. Luo et al., “sdnMAC: A software-defined network inspired MACprotocol for cooperative safety in VANETs,” IEEE Trans. Intell. Transp.Syst., vol. 19, no. 6, pp. 2011–2024, Jun. 2018.

[11] H. G. Seif and X. Hu, “Autonomous driving in the iCity—HD maps asa key challenge of the automotive industry,” Engineering, vol. 2, no. 2,pp. 159–162, 2016.

[12] N. Lu, N. Cheng, N. Zhang, X. Shen, and J. W. Mark, “Connectedvehicles: Solutions and challenges,” IEEE Internet Things J., vol. 1,no. 4, pp. 289–299, Aug. 2014.

LI et al.: E2LB BASED ON DEEP LEARNING FOR VEHICULAR NETWORK TRAFFIC CONTROL 965

[13] J. Barrachina et al., “Road side unit deployment: A density-basedapproach,” IEEE Intell. Transp. Syst. Mag., vol. 5, no. 3, pp. 30–39,2013. [Online]. Available: https://ieeexplore.ieee.org/document/6565484/

[14] X. Wang, M.-J. Sheng, Y.-Y. Lou, Y.-Y. Shih, and M. Chiang, “Internetof Things session management over LTE—Balancing signal load, power,and delay,” IEEE Internet Things J., vol. 3, no. 3, pp. 339–353,Jun. 2016.

[15] C. Vlachos and V. Friderikos, “Optimal device-to-device cell associationand load balancing,” in Proc. IEEE Int. Conf. Commun. (ICC), Jun. 2015,pp. 5441–5447.

[16] E. Yanmaz and O. K. Tonguz, “Dynamic load balancing and shar-ing performance of integrated wireless networks,” IEEE J. Sel. AreasCommun., vol. 22, no. 5, pp. 862–872, Jun. 2004.

[17] J. Liu, Y. Kawamoto, H. Nishiyama, N. Kato, and N. Kadowaki,“Device-to-device communications achieve efficient load balancing inLTE-advanced networks,” IEEE Wireless Commun., vol. 21, no. 2,pp. 57–65, Apr. 2014.

[18] A. Sarma, S. Chakraborty, and S. Nandi, “Deciding handover pointsbased on context-aware load balancing in a WiFi-WiMAX heteroge-neous network environment,” IEEE Trans. Veh. Technol., vol. 65, no. 1,pp. 348–357, Jan. 2016.

[19] C. A. S. Franco and J. R. B. de Marca, “Load balancing in self-organizedheterogeneous LTE networks: A statistical learning approach,” in Proc.IEEE Latin–Amer. Conf. Commun. (LATINCOM), Nov. 2015, pp. 1–5.

[20] Z. H. Abbas, F. Muhammad, and L. Jiao, “Analysis of load balancingand interference management in heterogeneous cellular networks,” IEEEAccess, vol. 5, pp. 14690–14705, 2017.

[21] M. H. Hajiesmaili, L. Deng, M. Chen, and Z. Li, “Incentivizingdevice-to-device load balancing for cellular networks: An online auc-tion design,” IEEE J. Sel. Areas Commun., vol. 35, no. 2, pp. 265–279,Feb. 2017.

[22] K. Nagaraj et al., “Libra: Impact assessment of cellular load balancing,”in Proc. IEEE Int. Conf. Comput. Commun. (INFOCOM), Apr. 2016,pp. 1–9.

[23] E. Ko, J. Ahn, and E. Y. Kim, “3D Markov process for traffic flowprediction in real-time,” Sensors, vol. 16, no. 2, p. 147, 2016.

[24] A. Abadi, T. Rajabioun, and P. A. Ioannou, “Traffic flow predictionfor road transportation networks with limited traffic data,” IEEE Trans.Intell. Transp. Syst., vol. 16, no. 2, pp. 653–662, Apr. 2015.

[25] X. Ma et al., “Learning traffic as images: A deep convolutionalneural network for large-scale transportation network speed prediction,”Sensors, vol. 17, no. 4, p. 818, 2017.

[26] H.-F. Yang, T. S. Dillon, and Y.-P. P. Chen, “Optimized structure ofthe traffic flow forecasting model with a deep learning approach,”IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 10, pp. 2371–2381,Oct. 2017.

[27] Y. Lv, Y. Duan, W. Kang, Z. Li, and F. Y. Wang, “Traffic flow predictionwith big data: A deep learning approach,” IEEE Trans. Intell. Transp.Syst., vol. 16, no. 2, pp. 865–873, Apr. 2015.

[28] J. Zhang, Y. Zheng, and D. Qi, “Deep spatio-temporal residual networksfor citywide crowd flows prediction,” in Proc. AAAI Conf. Artif. Intell.,Feb. 2017, pp. 1655–1661.

[29] G. Cybenko, “Approximation by superpositions of a sigmoidal function,”Math. Control Signals Syst., vol. 2, no. 4, pp. 303–314, 1989.

[30] M. Andrychowicz et al., “Learning to learn by gradient descent by gra-dient descent,” in Proc. Int. Conf. Neural Inf. Process. Syst. (NIPS),2016, pp. 3988–3996.

[31] O. Vinyals, M. Fortunato, and N. Jaitly, “Pointer networks,” in Proc. Int.Conf. Neural Inf. Process. Syst. (NIPS), vol. 2. Cambridge, MA, USA,2015, pp. 2692–2700.

[32] E. Khalil, H. Dai, Y. Zhang, B. Dilkina, and L. Song, “Learning combi-natorial optimization algorithms over graphs,” in Proc. Adv. Neural Inf.Process. Syst., 2017, pp. 6348–6358.

[33] H. Sun et al., “Learning to optimize: Training deep neural networksfor wireless resource management,” in Proc. IEEE Int. Workshop SignalProcess. Adv. Wireless Commun. (SPAWC), Jul. 2017, pp. 1–6.

[34] H. Mao, M. Alizadeh, I. Menache, and S. Kandula, “Resource manage-ment with deep reinforcement learning,” in Proc. 15th ACM WorkshopHot Topics Netw., 2016, pp. 50–56.

[35] L. Liu, Y. Cheng, L. Cai, S. Zhou, and Z. Niu, “Deep learning basedoptimization in wireless network,” in Proc. IEEE Int. Conf. Commun.(ICC), May 2017, pp. 1–6.

[36] F. Tang et al., “On removing routing protocol from future wirelessnetworks: A real-time deep learning approach for intelligent traf-fic control,” IEEE Wireless Commun., vol. 25, no. 1, pp. 154–160,Feb. 2018.

[37] J. Wang, X. Zhang, Q. Gao, H. Yue, and H. Wang, “Device-free wirelesslocalization and activity recognition: A deep learning approach,” IEEETrans. Veh. Technol., vol. 66, no. 7, pp. 6258–6267, Jul. 2017.

[38] Q. Yuan et al., “Toward efficient content delivery for automated driv-ing services: An edge computing solution,” IEEE Netw., vol. 32, no. 1,pp. 80–86, Jan. 2018.

[39] G. Luo et al., “Cooperative vehicular content distribution in edge com-puting assisted 5G-VANET,” China Commun., vol. 15, no. 7, pp. 1–17,Jul. 2018.

[40] S. Liang and R. Srikant, “Why deep neural networks for functionapproximation?” in Proc. ICLR, 2017, pp. 1–17.

[41] H. Sun et al., “Learning to optimize: Training deep neural networks forwireless resource management,” arXiv preprint arXiv:1705.09412, 2017.

[42] R. Pascanu, T. Mikolov, and Y. Bengio, “Understanding the explod-ing gradient problem,” CoRR, vol. abs/1211.5063, 2012. [Online].Available: https://www.semanticscholar.org/paper/Understanding-the-exploding-gradient-problem-Pascanu-Mikolov/728d814b92a9d2c6118159bb7d9a4b3dc5eeaaeb

[43] H. N. Mhaskar and T. Poggio, “Deep vs. shallow networks: An approx-imation theory perspective,” Anal. Appl., vol. 14, no. 6, pp. 829–848,2016.

[44] S. Zhang et al., “Air-ground integrated vehicular network slicing withcontent pushing and caching,” arXiv preprint, arXiv:1806.03860, 2018.

[45] I. Guvenc, “Capacity and fairness analysis of heterogeneous networkswith range expansion and interference coordination,” IEEE Commun.Lett., vol. 15, no. 10, pp. 1084–1087, Oct. 2011.

Jinglin Li (M’16) received the Ph.D. degree incomputer science and technology from the BeijingUniversity of Posts and Telecommunications,Beijing, China, in 2004.

He is currently an Associate Professor of com-puter science and technology and the Director of theSwitching and Intelligent Control Research Centerwith the State Key Laboratory of Networking andSwitching Technology, Beijing University of Postsand Telecommunications. His current researchinterests include mobile Internet, Internet of Things,

Internet of Vehicles, convergence network, and service technologies.

Guiyang Luo received the B.E. degree in elec-trical and communications from Beijing JiaotongUniversity, Beijing, China, in 2015. He is currentlypursuing the Ph.D. degree at the Beijing Universityof Posts and Telecommunications, Beijing.

His current research interests include softwaredefined networks, medium access control, and intel-ligent transportation systems.

Nan Cheng (S’12–M’16) received the Ph.D. degreefrom the Department of Electrical and ComputerEngineering, University of Waterloo, Waterloo, ON,Canada.

He is currently a Post-Doctoral Fellow with theDepartment of Electrical and Computer Engineering,University of Toronto, Toronto, ON, Canada, andalso with the Department of Electrical and ComputerEngineering, University of Waterloo. His cur-rent research interests include performance analy-sis and opportunistic communications for vehicular

networks, unmanned aerial vehicles, and cellular traffic offloading.

Quan Yuan received the Ph.D. degree in computerscience and technology from the Beijing Universityof Posts and Telecommunications (BUPT), Beijing,China, in 2018.

He is currently a Post-Doctoral Fellow with theState Key Laboratory of Networking and SwitchingTechnology, BUPT. His current research interestincludes crowdsensing, connected vehicle, mobileInternet, and intelligent transportation systems.

966 IEEE INTERNET OF THINGS JOURNAL, VOL. 6, NO. 1, FEBRUARY 2019

Zhiheng Wu is currently pursuing the M.S.degree in computer science and technology atthe State Key Laboratory of Networking andSwitching Technology, Beijing University of Postsand Telecommunications, Beijing, China.

His current research interests include Internet ofVehicle, mobile Internet, and intelligent transporta-tion system.

Shang Gao is currently pursuing the M.S.degree in computer science and technology atthe State Key Laboratory of Networking andSwitching Technology, Beijing University of Postsand Telecommunications, Beijing, China.

His current research interests include Internet ofVehicle, mobile Internet, and intelligent transporta-tion systems.

Zhihan Liu received the M.S. degree in computerscience and technology from the Beijing Universityof Posts and Telecommunications (BUPT), Beijing,China, in 2004.

He is a Researcher with the State Key Laboratoryof Networking and Switching Technology, BUPT.His current research interests include mobileInternet, Internet of Things, and Internet of Vehicles.