0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 1
A Reinforcement Learning-based Data Storage
Scheme for Vehicular Ad Hoc NetworksCelimuge Wu, Member, IEEE, Tsutomu Yoshinaga, Member, IEEE, Yusheng Ji, Member, IEEE,
Tutomu Murase, Member, IEEE, and Yan Zhang, Senior Member, IEEE
Abstract—Vehicular ad hoc networks (VANETs) have beenattracting interest for their potential roles in intelligent transportsystems (ITS). In order to enable distributed ITS, there is a needto maintain some information in the vehicular networks withoutthe support of any infrastructure such as road side units. Inthis paper, we propose a protocol which can store the data inVANETs by transferring data to a new carrier (vehicle) beforethe current data carrier is moving out of a specified region. Forthe next data carrier node selection, the protocol employs fuzzylogic to evaluate instant reward by taking into account multiplemetrics specifically throughput, vehicle velocity, and bandwidthefficiency. In addition, a reinforcement learning-based algorithmis used to consider the future reward of a decision. For the datacollection, the protocol uses a cluster-based forwarding approachto improve the efficiency of wireless resource utilization. We usetheoretical analysis and computer simulations to evaluate theproposed protocol.
Index Terms—Vehicular ad hoc networks, data storage scheme,reinforcement learning, fuzzy logic.
I. INTRODUCTION
For a large event like Olympic game, it is particularly
important to design an efficient navigation system to guide
visitors to/from the stadium. Since existing navigation systems
like VICS (Vehicle Information and Communication System)
in Japan are dependent on pre-installed infrastructure and
centralized control, they cannot attain expected real-time and
accurate information dissemination. It is very likely that all
people will be guided to the same route (resulting in traffic
congestion on that route) because the existing systems do
not take into account the load balancing and user-behavior
based adjustment. In order to solve this problem, we propose
a protocol which can store the data in distributed vehicular
networks, specifically vehicular ad hoc networks (VANETs),
without any support from infrastructure.
As shown in Fig. 1, some data, such as the change of
vehicle density with time domain, road status, camera sensor
Copyright (c) 2016 IEEE. Personal use of this material is permitted.However, permission to use this material for any other purposes must beobtained from the IEEE by sending a request to [email protected].
C. Wu and T. Yoshinaga are with the Graduate School ofInformatics and Engineering, The University of Electro-Communications,1-5-1, Chofugaoka, Chofu-shi, Tokyo, 182-8585 Japan (e-mail:{clmg,yosinaga,kato}@is.uec.ac.jp).
Y. Ji is with the Information Systems Architecture Research Division,National Institute of Informatics, 2-1-2, Hitotsubashi, Chiyoda-ku, Tokyo 101-8430 Japan (e-mail: {kei}@nii.ac.jp).
T. Murase is with the Information Technology Center, Nagoya University,Chikusa-ku, Nagoya (e-mail: [email protected]).
Y. Zhang is with University of Oslo, Norway (e-mail: [email protected]).Manuscript received 2016.
Fig. 1. Storing data in a distributed vehicular network.
information etc. [1], [2], can be stored locally in their interest
region (the rectangular area in the figure shows the interest
region; at least a vehicle in the region should have the data).
This can be usable when there is a request to know the
current traffic information and the future traffic estimation
of the corresponding area. By storing this information in a
distributed network, it is possible to provide more accurate
local information to intelligent transport systems without the
support of road side units or Internet connection. While some
recent works [3]–[9] discuss about the cloud-based resource
management and vehicular cloud computing, this paper de-
scribes the problem of how to store information in a vehicular
network, and propose a protocol which can efficiently maintain
information in VANETs.
There have been many studies discussing about data transfer
in VANETs [10]–[25]. The existing approaches can be clas-
sified into two main categories: broadcast protocols [11]–[17]
and unicast protocols [18]–[27]. Unicast protocols conduct
data transfer between one sender node and one receiver node.
In contrast, broadcast protocols are used to disseminate data
to multiple intended receivers. A unicast protocol is more
suitable for storing data in VANETs because it is low-cost (the
number of data carrier nodes is fixed to one) and bandwidth
efficient (efficient modulation and coding scheme can be used
to transmit data as compared with the broadcast scheme).
In this paper, we consider using unicast transmissions to
handoff data between vehicles in order to keep the data always
in the interest region. Only one data carrier node is required
for storing the data from the same source node with the
same interest region. However, a data carrier node has to
handover the data to the next data carrier node before moving
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 2
out of the interest region. We propose a protocol which can
select the next data carrier node efficiently. The proposed
protocol takes into account throughput, vehicle mobility, and
bandwidth efficiency by employing a fuzzy logic algorithm,
and considers long-term outcome by using a reinforcement
learning algorithm. Considering the possibility of multiple data
sources, we also propose an efficient route selection algorithm
to transfer data from each data source to the data carrier
node. We use theoretical analysis and computer simulations
to evaluate the proposed protocol.
The paper is an extension of our previous conference paper
[28]. While [28] only discussed the handoff of data between
data carrier nodes, the paper describes a solution which con-
ducts efficient data transmission from the data source nodes to
the data carrier node. We also present new theoretical analysis
and simulation results that have not been reported previously.
The remainder of the paper is organized as follows. Section II
gives a brief outline of related work. In section III, we give a
detailed description of the proposed data carrier node selection
algorithm. Next, we propose an efficient data transmission
scheme from the data source nodes to the data carrier node
in section IV. Theoretical analysis and simulation results are
presented in section V and section VI respectively. Finally, we
present our conclusions and future work in section VII.
II. RELATED WORK
There have been many studies related to the cloud comput-
ing (including data storage management) and routing protocols
for VANETs. However, the data storage issue in VANETs is
an under-explored research problem.
A. Storage management in VANETs
There have been some studies discussing about the man-
agement of data storage resources in vehicular environment.
Yu et al. [3] have studied the bandwidth and computing
resource (CPU, memory, and storage) sharing issues in cloud-
enabled vehicular networks, and proposed a coalition game
model to solve the idle resource sharing problem among
cloud service providers. In [4], the authors discussed the
opportunities and challenges in exploiting cloud computing
in vehicular networks, and proposed an integrated cloud
computing architecture which facilitates sharing of computa-
tional resources, storage resources, and bandwidth resources
among vehicles. Mershad and Artail [5] have presented a
cloud service discovery protocol for VANETs which can be
used to discover mobile cloud services provided by nearby
vehicles. The system is not a totally distributed approach
because the system depends on RSUs (roadside units) which
are used to register mobile cloud services. Lee et al. [6]
have reviewed emerging VANET applications and state-of-
the-art computing and networking models for vehicular cloud
networking systems. Bitam et al. [7] have proposed VANET-
Cloud, a cloud computing model for VANETs. VANET-Cloud
extends the conventional cloud infrastructure by introducing
vehicles as edge computing resources in order to allow drivers
and other users to access computing resources of vehicles.
Liu et al. [8] have proposed a cloud-assisted downlink safety
message dissemination framework where wireless networking
and cloud computing technologies are integrated to minimize
packet loss and redundancy. Kim et al. [9] have discussed
the data dissemination problem of providing reliable data
delivery services from a cloud data center to vehicles through
roadside wireless access points (APs) with local data storage,
and proposed two algorithms to prefetch a set of data from a
data center to roadside wireless APs. However, none of these
approaches discusses the problem of storing the vehicle data
in a totally distributed VANET.
B. Routing protocols for VANETs
The routing protocols for VANETs can be classified into
two categories specifically broadcast protocols and unicast
protocols. The aim of broadcast protocols is to provide an
efficient data dissemination for one-to-many communications.
Most broadcast protocols focus on reducing the broadcast
redundancy to improve the efficiency and packet dissemination
ratio. Yoo and Kim [11] have proposed a multi-hop broadcast
protocol called RObust and Fast Forwarding (ROFF) to miti-
gate the unnecessary contention delay and redundant packets.
Chuang and Chen [12] have resolved the broadcast storm
problem by finding the appropriate parameters for different
car densities via a mathematical model. Suthaputchakun et
al. [13] have proposed a trinary partitioned black-burst-based
broadcast protocol which takes into account packet priorities,
and uses the farthest possible vehicle to forward the data
packets in order to reduce the number of hops required for
data dissemination. However, the farthest vehicle could have
poor link quality. In [14], the inter-vehicle distance, signal
quality, and route length are jointly considered to improve the
efficiency without sacrificing the reliability. Bi et al. [15] have
proposed a multi-hop broadcast protocol which combines bi-
directional broadcast and directional broadcast. Ucar et al. [16]
have proposed a hybrid architecture which integrates LTE
with IEEE 802.11p-based multi-hop communication. Liu et
al. [17] have explored vehicle-to-vehicle data dissemination in
VANETs based on network coding.
The unicast routing protocols handle the packet forwarding
problem of one-to-one communications. Since the acquisition
and utilization of location information [29]–[31] are possi-
ble in VANETs, geographic protocols have attracted a great
interest. Shafiee and Leung [18] have proposed CMGR, a
connectivity-aware minimum-delay geographic routing proto-
col which changes route selection policy according to the
network connectivity. In a sparse network, CMGR gives a
higher weight to the connectivity of routes. In contrast, the
protocol chooses a route which has the minimal delay and
adequate connectivity in a high-density network. Yang et
al. [19] have proposed a protocol where the next forwarder
node is selected based on a metric which minimizes the packet
error rate of route. Eiza and Ni [20] have proposed an evolving
graph-based reliable routing protocol which can find a reliable
route without broadcasting the route request messages for each
route change. Al-Rabayah and Malaney [21] have proposed
a protocol which integrates the geographic routing approach
and reactive routing approach. Rak [22] has proposed an
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 3
opportunistic routing protocol which takes into account the
packet delivery ratio and link stability information for the route
selection. He et al. [23] have investigated the problem of delay
minimization for data dissemination in a large-scale vehicular
ad network, and proposed a routing strategy to minimize the
expected path delay with fixed-scheduled buses and random-
scheduled taxis. Zhang et al. [24] have proposed a street-
centric opportunistic routing protocol in which packets can
be dynamically forwarded at the intersections based on the
information of adjacent streets. Togou et al. [25] have proposed
SCRP, a distributed routing protocol that computes end-to-
end delay for the entire routing path before sending data
messages. SCRP is a distributed geographic source routing
scheme which takes advantage of the global network topology
to select routing paths. Zhu et al. [26] have proposed a
protocol which integrates both contact-level and social-level
mobility for fast routing. In [27], a cooperative protocol for
Roadside-Vehicle communication was proposed. The protocol
minimizes an average delivery delay of each user request while
maximizing the amount of data packets downloaded from the
RSU. However, all these protocols [11]–[25] do not discuss
the problem of collecting and maintaining vehicle data in
VANETs.
III. DATA CARRIER NODE SELECTION
A. Assumption
Each node (vehicle) is equipped with a positioning device.
Each node knows the road map information, and sends own
location information and velocity information using beacon
messages with a predefined interval (1 second by default).
We assume a connected network topology where at least one
multi-hop path exists between any two nodes.
B. System model and problem definition
For each block of data, there is an interest region for the
data. The data should be maintained in this region (some nodes
in this region have to maintain this information). We say that
the data are lost if the vehicles that storing the data go outside
of the interest region. The source node of the data sends the
interest region information with the data.
The data carrier node selection problem can be defined as
maximizeV1,V2,...,Vj
∑i=j−1
i=1TH(Vi,Vi+1)
j·(j−1)
subject to TH(Vi, Vi+1) ≥Sdata
CT (Vi,Vi+1), i = 1, ..., j − 1
d(Vi, Loc) ≤ RINT , i = 1, ..., j − 1. (1)
where Vi is the ith data carrier node, j is the number of
data carrier nodes used for the whole time domain, and
TH(Vi, Vi+1) is the throughput of data transmission between
Vi and Vi+1. The size of data that should be maintained in
the network is expressed as Sdata, and CT (Vi, Vi+1) is the
connection time between Vi and Vi+1. Here, d(Vi, Loc) is the
distance between Vi and the center location of interest region
(Loc is the center of interest region, and RINT is the radius of
interest region). The objective is to reduce the time required
[equivalent to increase the throughput and reduce the number
of data handoffs; this is why the objective function of (1)
considers the average throughput (∑i=j−1
i=1TH(Vi,Vi+1)
(j−1) ) and the
number of data carrier nodes (j)] while keeping the data in the
interest region (each data carrier node should handover the data
to the next carrier node before going out of the interest region).
In order to solve the problem, we first have to take into account
the throughput between the current data carrier node and the
next data carrier node because low throughput connection
could increase the channel time required for transmitting the
data. It is also important to consider vehicle velocity as a
metric because the position and velocity of the next carrier
node affects the future outcome. Considering there could be
multiple traffic source nodes in the network, the MAC layer
contention efficiency is another important metric should be
addressed. Therefore, in the paper, we use a heuristic approach
to solve the problem. We take into account throughput, vehicle
velocity, and bandwidth efficiency of the whole network by
using a fuzzy logic algorithm, and employ a reinforcement
learning approach to evaluate the long-term outcome of a
decision. The fuzzy logic is used to evaluate the next data
carrier node, and the reinforcement learning is used to evaluate
the possible future reward after selecting the next data carrier
node.
C. Fuzzy logic-based instant decision evaluation and rein-
forcement learning-based future reward evaluation
The source node selects its next data carrier node. We take
into account multiple metrics specifically throughput factor
(dependent on the inter-vehicle distance), vehicle stability
factor, and the bandwidth efficiency factor. Throughput factor
considers the achievable throughput between the sender and
the next data carrier node. Vehicle stability factor is used to
select a slowly moving vehicle in order to reduce the frequency
of data exchange. The consideration of bandwidth utilization
efficiency is also important especially when the number of
data blocks is large. Fuzzy logic is used to combine these
three factors to conduct an evaluation on the instant reward of
the selection (basically the efficiency of transmission from the
current sender node to the next data carrier node). In addition
to this, we have to consider long-term reward of the decision
as well. More specifically, the goodness of a next carrier node
selection is also dependent on the actions of the following data
carrier nodes. Here, we take into account this by considering
how much does the next carrier node close from the center of
interest region (the best action).
D. Fuzzy logic-based instant decision evaluation
1) Procedure: The sender node calculates the evaluation
value for each neighbor as follows.
• Step1: Fuzzification Use predefined linguistic variables
and membership functions to convert throughput factor
(TPF), vehicle stability factor (VSF), and bandwidth
efficiency factor (BEF) to fuzzy values (see Fig. 2, Fig. 3,
and Fig. 4).
• Step2: Mapping and combination of IF/THEN rules
Map the fuzzy values to predefined IF/THEN rules (see
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 4
Table II) and combine the rules to get the rank of the
neighbor as a fuzzy value.
• Step3: Defuzzification Use a predefined output member-
ship function (see Fig. 5) and defuzzification method (we
use the Center of Gravity method) to convert the fuzzy
output value to a numerical value (evaluation value).
2) Throughput factor and membership function: We take
into account the achievable throughput between the sender
node and the next carrier node. Since an accurate estimation
of throughput is difficult if not impossible in dynamic environ-
ment, for simplicity, we use inter-vehicle distance to estimate
throughput. We define a distance metric as
DMc(x) =
{
d(x,c)R
, d(x, c) <= R
1, d(x, c) > R(2)
where d(x, c) denotes the distance between the current node
(c) and node x. The throughput factor membership function is
defined based on the distance metric as shown in Fig. 2.
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Deg
ree
DM
High Medium Low
Fig. 2. Throughput factor membership function.
3) Vehicle stability factor and membership function: The
vehicle stability factor is calculated as
VSFc(x) =
{
ζ + (1− ζ)× (1 − |V (x)|maxy∈Nc |V (y)| ), C1
(1− ζ)× (1 − |V (x)|maxy∈Nc |V (y)| ), otherwise
(3)
where C1 denotes the case when the node is moving toward
the center of interest region. Nc is the set of one-hop neighbors
of c. ζ is set to 13 . The vehicle stability factor intends to give
a lower speed vehicle a higher evaluation. The factor also
takes into account the relative moving direction of vehicles
in relation to the center of interest region by applying the
condition C1. Therefore, by using this factor, the absolute
vehicle velocity and relative moving direction of vehicle can
be evaluated jointly. The value of ζ determines the weight
of the moving direction. A higher ζ intends to give a higher
evaluation for the vehicles moving toward the center of interest
region as compared with those leaving the center. The corre-
sponding fuzzy membership function for the vehicle stability
factor is defined as shown in Fig. 3.
4) Bandwidth efficiency factor and membership function:
Since IEEE 802.11p uses contention-based channel access
scheme, the channel utilization efficiency (bandwidth effi-
ciency) decreases as the number of sender nodes increases.
Therefore, it is important to reduce the number of sender
nodes in the network as far as possible. Here, we take into
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Deg
ree
VSF
Bad Medium Good
Fig. 3. Vehicle stability factor membership function.
account bandwidth efficiency factor by considering how much
the carrier node selection algorithm can reduce the number
of sender nodes in the network. Bandwidth efficiency factor
(BBF) is calculated as
BEFc(x) =
{
1, maxy∈NcCnt(y) = 0
Cnt(x)maxy∈NcCnt(y) , otherwise
(4)
where Cnt(x) denotes the number of data blocks that are
using node x as the data carrier. The information about
Cnt(x) is exchanged through hello messages among one-
hop neighbors. The bandwidth efficiency factor membership
function is defined as shown in Fig. 4.
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
Deg
ree
BEF
Bad Medium Good
Fig. 4. Bandwidth efficiency factor membership function.
5) Fuzzy rules: The fuzzy rules are defined as shown in
Table II.6) Defuzzification: The output function is defined as Fig. 5.
We use Center of Gravity (COG) method to defuzzify the
fuzzy result.
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
VeryBad Bad Unpreferable Acceptable Good Perfect
Fig. 5. Output membership function for the next data carrier node selection.
E. Reinforcement learning-based future reward evaluation
We use a fuzzy logic-based approach to evaluate each
neighbor as explained in §III-D, and then use a reinforce-
ment learning approach, specifically Q-Learning, to evaluate
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 5
TABLE IFUZZY RULES
Throughput Stability Bandwidth efficiency Rank
Rule1 High Good Good PerfectRule2 High Good Medium GoodRule3 High Good Bad UnpreferableRule4 High Medium Good GoodRule5 High Medium Medium AcceptableRule6 High Medium Bad BadRule7 High Bad Good UnpreferableRule8 High Bad Medium BadRule9 High Bad Bad VeryBadRule10 Medium Good Good GoodRule11 Medium Good Medium AcceptableRule12 Medium Good Bad BadRule13 Medium Medium Good AcceptableRule14 Medium Medium Medium UnpreferableRule15 Medium Medium Bad BadRule16 Medium Bad Good BadRule17 Medium Bad Medium BadRule18 Medium Bad Bad VeryBadRule19 Low Good Good UnpreferableRule20 Low Good Medium BadRule21 Low Good Bad VeryBadRule22 Low Medium Good BadRule23 Low Medium Medium BadRule24 Low Medium Bad VeryBadRule25 Low Bad Good BadRule26 Low Bad Medium VeryBadRule27 Low Bad Bad VeryBad
whether the decision is a long-term good decision or not. The
Q-learning model is defined as follows. Each node (vehicle)
is an agent. Each possible selection of the next carrier node
is considered a state of the agent. The set of all possible
candidate nodes for the next data carrier is the state space.
The learning task is to find the best data carrier node for
the corresponding network environment in relation to the
feedback. An action is to select the next data carrier node
that would be used for storing the data.
Each agent updates its Q-table after specifying the next data
carrier node, or reception of a hello message from a neighbor
node. Q-table is updated as
Qc(Loc, x)
← α× ls(c, x)× {Rwd+ γ ×maxy∈NxQx(Loc, y)}
+ (1− α)×Qc(Loc, x). (5)
where Loc denotes the center of interest region, and x is a
possible action (a neighbor node which can be used as the
next data carrier node). c is the current node, and ls(c, x) is
the instant evaluation value for the decision (this is basically
the evaluation of the link between the current node and the
next data carrier node as calculated in §III-D). Nx denotes
the one-hop neighbor set of x. The learning rate α is set to
0.7. Since the hello messages are exchanged periodically with
interval of 1 second by default, the value of 0.7 is enough to
reflect the network topology changes. The discount factor γ is
set to 0.9. The reward is calculated as
Rwd =
{
1, d(Loc, x) < RINT
2
0, otherwise(6)
where d(Loc, x) is the distance between the next data carrier
node and the center of interest region. The reward is 1 only
if d(Loc, x) is smaller than RINT
2 where RINT is the radius
of interest region. This is to guide the agent to select a node
which is closer to the center of interest region.
Each Q-value [Qc(Loc, x)] shows the evaluation value
for a selection of the next data carrier node. Each node
attaches its position information, and maximal Q-values
[maxy∈NxQx(Loc, y)] for active interest regions to the hello
messages. Here, we use “active interest regions” to denote
the interest regions which cover the position of the current
vehicle). Each node only needs to maintain the information
about the active interest regions. After reception of a hello
message, a node can update its knowledge about the distance
to the center of interest region by using the corresponding
maximal Q-value extracted from the hello message. As shown
in (12), the reward Rwd is discounted with the increase of
the distance from the center of interest region. The rationale
behind this is that we want to maintain the data at the center
of interest region in order to reduce the probability of data loss
due to vehicle mobility (in contrast, if the data are maintained
at the border of the interest region, there is a high probability
of failing to find a vehicle to perform the data handoff,
resulting in data loss).
F. Some considerations
The handoff timing is decided as follows. Based on the
selected data carrier node, the current data carrier node will
decide when to handover the data to the next data carrier
node. As explained before, we assume a connected network
topology. This means that there will be always at least one
node can be a candidate for maintaining the data in the
interest region. Each node maintains a table which shows the
relationship between the distance and the throughput, and then
makes a decision based on this table. Since each data carrier
node knows the link quality between itself and the next data
carrier node, the data carrier node can adjust the handoff
timing in order to ensure that the data would be always in
the interest region. The parameters used for this paper are set
based on our simulation results. Note that different application
requirements and network environment could require different
sets of parameters. However, the problem of how to efficiently
tune these parameters automatically with the environment
change is considered as our future work.
IV. ROUTE SELECTION TO THE DATA CARRIER NODE:
COLLECTION OF DATA USING DYNAMIC CLUSTERING
There could be multiple data sources sharing the same
interest region. This happens often especially in high-density
networks because the information collected from multiple
vehicles could be required to be maintained in the same area.
Typically, the size of interest region would be much larger than
the transmission range. Therefore, a multi-hop communication
could be required to transmit data from multiple data source
nodes to the data carrier node. In order to overcome the
problem of MAC layer performance degradation due to the
increase of the number of sender nodes, the proposed protocol
conducts efficient data transmissions from the data source
nodes to the data carrier node by employing a routing approach
based on dynamic clustering.
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 6
A. Virtual clustering of vehicles and packet forwarding with
cluster heads
In IEEE 802.11p standard, with the increase of the number
of sender nodes, the MAC contention efficiency decreases
significantly. Therefore, it is important to reduce the number
of data sender nodes as far as possible. In the proposed
protocol, we use a cluster-head based forwarding where data
are forwarded by cluster heads which are generated using
a distributed approach (this will be explained in the next
Subsection). As shown in Fig. 6, different source nodes (S1
and S2) could use the same forwarder nodes (CH1 and CH2)
to deliver packets, which are efficient in the utilization of
wireless resources by reducing the number of forwarder nodes
especially when the number of traffic flows is large.
Fig. 6. Packet forwarding to the data carrier node (S1 and S2 are the datasource nodes; D is the data carrier node).
B. Criteria for selecting cluster heads
The proposed protocol selects the cluster heads using a
distributed approach. In order to generate a stable cluster, the
protocol takes into account the vehicle velocity, the number
of neighbors driving to the same direction, and the channel
condition between the cluster head and cluster members for
the cluster head selection. The vehicle velocity is considered
in order to select slow vehicles as cluster heads, which is
efficient in terms of avoiding the frequent change of cluster
heads. The number of neighbors moving to the same direction
can reflect the long-term vehicle velocity (in a two-way road,
the cluster heads should be selected from the vehicles which
have more vehicles moving toward the same direction). The
channel condition is also an important metric because a cluster
head which has better link with cluster members (for example,
higher antenna height) is preferred. We use a fuzzy logic-based
approach to jointly consider these three metrics for evaluating
the fitness for a cluster head.
In the proposed protocol, each node sends the required
information (vehicle velocity and the number of neighbor
vehicles driving to the same direction) using hello messages.
For each hello interval, each node calculates a competency
value (as being a cluster head) for itself and each neighbor
vehicle. If the node has the largest competency value in its
vicinity (R2 where R is the average transmission range), the
node announces itself as a cluster head node using the next
hello message. Cluster head selection is conducted on road
basis. This ensures that the selected cluster heads can generate
a connected network.
C. Calculation of competency value based on fuzzy logic
Each node evaluates its one-hop neighbors to determine
which node should be the cluster head. The evaluation is
conducted by using a fuzzy logic-based algorithm.
1) Procedure: For each neighbor vehicle, each node calcu-
lates a competency value as follows.
• Acquisition of the three factors: Get the vehicle veloc-
ity, the number of vehicles moving to the same direction,
and the channel condition information mentioned before.
• Fuzzification: Use predefined linguistic variables and
membership functions to convert these factors to fuzzy
values.
• Mapping and combination of IF/THEN rules: Map the
fuzzy values to predefined IF/THEN rules and combine
the rules to get the rank of the neighbor as a fuzzy value.
• Defuzzification: Use a predefined output membership
function and defuzzification method to convert the fuzzy
output value to a numerical value.
2) Calculation of multiple factors: Upon reception of a
hello message from a neighbor x, node s calculates the
following factors.
Velocity Factor (V F ): Node s extracts the velocity of
node x, υ(x), and calculates V F (s, x) (the velocity factor for
node x calculated at node s) as
V F (s, x) =|υ(x)| −miny∈Ns
|υ(y)|
maxy∈Ns|υ(y)|
(7)
where Ns denotes the neighbor set of node s. A lower V F
indicates a lower velocity. V F is updated every hello interval
using a weighted exponential moving average as
V F (s, x) ← (1−α)×V Fi−1(s, x)+α×V Fi(s, x), (8)
where V Fi−1(s, x) and V Fi(s, x) denote the previous V F
value and current V F value respectively. V F is initialized to
1. The coefficient α is set to 0.7 which is the best value for
many cases according to our simulation results.
Follower Density Factor (FDF ): Node x announces the
number of neighbor vehicles [c(x)] driving to the same direc-
tion by using hello messages. FDF of node x is calculated
as
FDF (s, x) =c(x)
maxy∈Nsc(y)
. (9)
FDF indicates the vehicle density for the same direction. A
higher FDF means that the node is more suitable for being
a cluster head. FDF is updated every hello interval using a
weighted exponential moving average (FDF is initialized to
0) as
FDF (s, x) ← (1−α)×FDFi−1(s, x)+α×FDFi(s, x).
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 7
(10)
Channel Condition Factor (CCF ): We use the hello
packet reception ratio to infer the channel Condition Factor
(CCF ). We calculate the number of hello messages received
from the nodes located in R where R is the average transmis-
sion range. The hello messages are sent for each predefined
time interval (1 second by default). If a vehicle has better
channel quality than other vehicles (for example, a truck with
higher antenna), the CCF is larger. The CCF is initialized
to 0, and updated as
CCF (s)← (1− α)CCFi−1(s) + α× CCFi(s). (11)
3) Fuzzification: Figure 7 shows the fuzzy membership
functions for the velocity factor, follower density factor and
channel condition factor. The velocity membership function
defines what degree the velocity factor belongs to {Slow,
Medium, Fast}. Similarly, the follower density membership
function defines what degree belongs to {Heavy, Medium,
Light} and what degree the channel condition factor belongs
to {Good, Medium, Bad}.
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
VF
Slow Medium Fast
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
FDF
Light Medium Heavy
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
CCF
Bad Medium Good
Fig. 7. Fuzzy membership functions (left: V F , middle: FDF , right: CCF ).
TABLE IIRULE BASE
Velocity Follower density Channel Condition Rank
Rule1 Slow Heavy Good Perfect
Rule2 Slow Heavy Medium Good
Rule3 Slow Heavy Bad Unpreferable
Rule4 Slow Medium Good Good
Rule5 Slow Medium Medium Acceptable
Rule6 Slow Medium Bad Bad
Rule7 Slow Light Good Unpreferable
Rule8 Slow Light Medium Bad
Rule9 Slow Light Bad VeryBad
Rule10 Medium Heavy Good Good
Rule11 Medium Heavy Medium Acceptable
Rule12 Medium Heavy Bad Bad
Rule13 Medium Medium Good Acceptable
Rule14 Medium Medium Medium Unpreferable
Rule15 Medium Medium Bad Bad
Rule16 Medium Light Good Bad
Rule17 Medium Light Medium Bad
Rule18 Medium Light Bad VeryBad
Rule19 Fast Heavy Good Unpreferable
Rule20 Fast Heavy Medium Bad
Rule21 Fast Heavy Bad VeryBad
Rule22 Fast Medium Good Bad
Rule23 Fast Medium Medium Bad
Rule24 Fast Medium Bad VeryBad
Rule25 Fast Light Good Bad
Rule26 Fast Light Medium VeryBad
Rule27 Fast Light Bad VeryBad
4) Mapping and combination of IF/THEN rules: Each node
uses the IF/THEN rules (see Table II) to calculate the rank of
the vehicle as being a cluster head. Since there can be multiple
rules applying at the same time, we use the Min-Max method
to combine their evaluation results (the same method as used
in [14]).
5) Defuzzification: For the defuzzification, we use the out-
put membership function as shown in Fig. 8, and the Center of
Gravity (COG) method where the x coordinate of the centroid
is the defuzzified value which shows the competency value of
the node.
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1
VeryBad Bad Unpreferable Acceptable Good Perfect
Fig. 8. Output membership function for the calculation of cluster-headcompetency value.
D. Reinforcement learning-based last two-hop optimization
Fig. 9. Last two-hop optimization.
The proposed route selection algorithm uses cluster heads
to forward packets. As a result, different traffic flows could
use the same cluster heads to forward packet. This is efficient
for utilizing wireless resources by reducing the number of
sender nodes. However, this could increase the number of hops
when the source (destination) node is very close from the next
(previous) cluster head. In order to make the routing more
efficient, we utilize a reinforcement learning-based algorithm
to optimize the last 2-hops from/to the source/destination node.
As shown in Fig. 9, since CH1, CH2, CH3, and CH4 are the
cluster heads, the default route from the source node (S) to the
destination node (D) is “S→CH1→CH2→CH3→CH4→D”.
By using the last two-hop optimization, the route can be
optimized to “S→F1→CH2→CH3→F2→D” which is more
efficient than the default route. The cluster-head based for-
warding can improve the MAC layer contention efficiency
while the last two-hop optimization can improve the effi-
ciency of a multi-hop route. Therefore, considering the trade-
off between cluster-head based forwarding and last two-hop
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 8
optimization, we only conduct the last two-hop optimization
when the distance between the source (destination) node and
the next (previous) cluster head is smaller than 14R where R
is the average transmission range.
1) Q-Learning model: We define the following Q-learning
model for the last two-hop optimization.
TABLE IIIQ-LEARNING MODEL
Environment The entire vehicular ad hoc network
Agent Each packet P (o, r)State of the agent Each node in the network
State space The set of all nodes in the network
Actions Selecting an one-hop neighbor as the next hop
We define the Q-Learning algorithm as follows. The entire
network is the environment. Each packet P (o, r), indexed by
its originator node o and the reference node r (the destination
node or a cluster head node) is an agent. A node selects
the next hop that it should forward a packet to. Hence the
possible set of actions allowed at the node is the set of one-
hop neighbors. Every node maintains a Q-Table which consists
of Q-value (Q(r, x)) whose value ranges from 0 to 1, where
x is the next hop to the reference node.
2) Update of Q-values: For each one-hop neighbor, a node
maintains a value lq(c, x) which shows the link status between
node c and x. For simplicity, we use hello reception ratio
to estimate the link status. However, the estimation can be
improved by taking into account the vehicle velocity in the link
status evaluation. The Q-Table is updated upon the reception
of hello messages. Each node needs to maintain a Q-value
for each one-hop neighbor, the destination (the traffic source
node for the TCP ACK messages) node, and the cluster head
nodes located in two-hop distance. Q-values are broadcasted
by each node using hello messages. Each Q-value is initialized
to 0. Upon reception of a hello message from node x, node c
updates the corresponding Q-value to the node r as
Qc(r, x)
← α× lq(c, x)×{
ˆRwd+ γ ×maxy∈NxQx(r, y)
}
+ (1− α)× Qc(r, x). (12)
The learning rate (α) is 0.7, and the discount factor (γ) is
0.9. maxy∈NxQx(r, y) is the maximal Q-value of x to node
r. The reward ˆRwd is calculated as
ˆRwd =
{
1, if c ∈ Nr
0, otherwise(13)
where Nr denotes the one-hop neighbor set of node r. When
node c is a neighbor of the node r, the reward is 1 and
otherwise 0. Note that there is only one Q-value for each
pair of state and action. Upon reception of hello messages,
the corresponding Q-value is updated as shown in (12). As
shown in (12), the algorithm discounts the reward when the
number of hops increases. This means that a larger number
of hops results in a smaller reward and smaller Q-value. The
reward is also discounted depending on the quality of each
link which constitutes the communication path. As a result, a
Q-value represents the quality of a next packet forwarder node
considering the multi-hop performance. This ensures that the
proposed protocol can optimize the route to the reference node.
V. THEORETICAL ANALYSIS
A. Impact of velocity
If the length of the interest region is L (L = 2RINT ), the
maximum data transfer (handoff) interval is Lυ
where υ is the
vehicle velocity (the data transfer should be conducted at least
once for each Lυ
interval). Fig. 10 shows the maximum data
transfer intervals for different lengths of interest regions and
velocities. We can observe that selecting a vehicle with low
velocity is particularly important for reducing the data transfer
frequencies.
0
20
40
60
80
100
120
30 40 50 60 70 80
Max
imum
dat
a tr
ansf
er i
nte
rval
(se
cond)
Velocity (km/h)
L=200mL=400m
L=600mL=800m
L=1000m
Fig. 10. Maximum data transfer interval for different lengths of interestregions and velocities.
B. Impact of the inter-vehicle link quality and the number of
sender nodes
In the IEEE 802.11p standard, the backoff time is a random
number which is drawn from a uniform distribution over the
interval [0,CW] where CW is the current contention window. If
multiple sender nodes are located closer than the sensing range
and they choose the same contention window size, there will
be collisions at some receiver nodes. Since a transmission is
successful only when all sender nodes choose different backoff
values, we can calculate the probability of collisions as
Pc(N) =
{
1, if CW+ 1 ≤ N
1−∏N−1
k=0 (CW+1−k
1 )(CW+1)N
, otherwise(14)
where N is the number of sender nodes. Since a packet canbe lost due to either collisions or weak signal strength, thepacket loss probability can be calculated as
P (N) = Pc(N) + Pl(N) − Pc(N) × Pl(N) (15)
where Pl(N) is the packet loss probability due to weak
signal strength. Based on (15), we show the packet loss
probability for different numbers of sender nodes and link loss
probabilities in Fig. 11.
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 9
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Pac
ket
loss
pro
bab
ilit
y
Link loss probability
N=1N=2
N=3N=4
N=5N=6
N=7N=8
Fig. 11. Packet loss probability for different numbers of sender nodes andlink loss probabilities.
Since TCP is the most commonly used transport layer pro-
tocol, here we analyze the TCP congestion window size which
basically determines the throughput of a TCP connection. In
the TCP slow start phase, congestion windows is increased
[by 1 MSS (maximum segment size)] upon reception of an
ACK. When the end-to-end loss probability is Pl, the average
congestion window in the slow start phase for the first 5 RTTs
(round trip time) is
Pl +
∑5RTTn=1
(2RTTn−1)∑
i=1
(1− Pl)i× (i + 1) (16)
where RTTn is the number for RTT. Figure 12 shows the
average TCP congestion window for different numbers of
sender nodes and link loss probabilities. This shows that the
consideration of link quality and the number of sender nodes
in the data carrier selection is very important.
0
10
20
30
40
50
60
70
80
90
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Conges
tion w
indow
(M
SS
)
Link loss probability
N=1N=2
N=3N=4
N=5N=6
N=7N=8
Fig. 12. Average TCP congestion window size for different numbers ofsender nodes and link loss probabilities (with window scaling).
C. Impact of cluster-head based forwarding
When the number of source nodes is N , the end-to-end
packet delivery probability for H hop transmission is
Pe2e(N) =
H∏
i=1
(1− iP (N)) (17)
where iP (N) is the packet loss probability for ith hop.
Based on (17), Fig. 13 shows the end-to-end packet delivery
probability. Since the proposed protocol uses cluster head
nodes to deliver data to the data carrier node, the number
of contending nodes can be reduced significantly. As a result,
the proposed protocol can improve the end-to-end delivery
probability especially when the number of hops is large.
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
2 3 4 5 6
End-t
o-e
nd p
acket
del
iver
y p
robab
ilit
y
Number of hops
Conventional(N=1)Proposed(N=2)
Conventional(N=3)
Proposed(N=3)Conventional(N=4)
Proposed(N=4)
Fig. 13. End-to-end packet delivery probability (without retransmission; thelink loss probability was 0.1).
VI. SIMULATION RESULTS
We used ns-2.34 [32] to conduct simulations in freeway
scenarios (see Table IV). We used a freeway which had two
lanes in each direction [33]. The distance between any two
adjacent lanes was 5m. Nakagami propagation model was used
to simulate channel fading (see Table V) [34]. The proposed
protocol was compared with “Velocity” and “Velocity + Link
quality” where “Velocity” chooses the vehicle which has the
slowest moving speed as the next data carrier node, and
“Velocity + Link quality” chooses the slowest vehicle from the
vehicles that are strongly connected (hello message reception
ratio is larger than 70%). The maximum transmission range
was 250m. We evaluated the protocol performance in terms of
data transfer throughput (§VI-A,§VI-B, and §VI-C) and data
collection efficiency (§VI-D) for various vehicle velocities and
various numbers of source nodes. In the following simulation
results, the error bars indicate the 95% confidence intervals.
A. Number of data handoffs for various vehicle velocities
Fig. 14 shows the number of data handoffs (transfers) for
various vehicle velocities. There was one data source (see
§VI-C for the performance with multiple data sources). The
volume of the data was 1 MB. The length of the interest
region was 600m. Since “Velocity” (velocity only approach)
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 10
TABLE IVSIMULATION ENVIRONMENT
Topology 2000m, 4lanes
Number of nodes 200
Maximum velocity 100 km/h
Mobility generation Ref. [33]
MAC IEEE 802.11p MAC (27 Mbps)
Propagation model Nakagami Model
Simulation time 1500 s
TABLE VPARAMETERS OF NAKAGAMI MODEL
gamma0 gamma1 gamma2 d0 gamma d1 gamma
1.9 3.8 3.8 200 500
m0 m1 m2 d0 m d1 m
1.5 0.75 0.75 80 200
takes into account the velocity information for the data carrier
node selection, the protocol could select a node either located
far away from or located very close to the sender node. A
decision of selecting a too close node can result in a high
number of data handoffs. In contrast, as will be shown later, the
selection of a far node deteriorates the data transfer throughput.
“Velocity + Link quality” is likely to select a vehicle in
close proximity, resulting in a large number of data handoffs.
The proposed protocol shows the best performance by taking
into account the vehicle velocity and the distance to the best
action (how much the next carrier node is closer to the center
of interest region) by using the reinforcement learning-based
carrier node selection approach.
0
10
20
30
40
50
60
70
80
60 65 70 75 80 85 90 95 100
Num
ber
of
dat
a han
doff
s
Maximum velocity (km/h)
VelocityVelocity + Link quality
Proposed
Fig. 14. Number of data handoffs for various vehicle velocities.
B. Average data transfer throughput for various vehicle ve-
locities
Fig. 15 shows the average data transfer throughput (the
throughput between a sender node and the next data carrier
node) for various vehicle velocities. “Velocity” also could
select a vehicle which is weakly connected to the current node,
which is inefficient in terms of bandwidth utilization (this
will be explained later). By considering inter-vehicle distance,
“Velocity + Link quality” performs better than “Velocity”.
Since the fuzzy logic algorithm takes into account throughput
factor, the average transmission time of the proposed protocol
is lower than other approaches.
0
500
1000
1500
2000
2500
60 65 70 75 80 85 90 95 100
Thro
ughput
(kbps)
Maximum velocity (km/h)
VelocityVelocity + Link quality
Proposed
Fig. 15. Average data transfer throughput for various vehicle velocities.
C. Average data transfer throughput for various numbers of
data sources
Fig. 16 shows the average data transfer throughput for vari-
ous numbers of data sources. The data volume for each source
node was 500 KB. Since the fuzzy logic takes into account
throughput factor, the average required time for handoff is
lower than other approaches. As shown in the figure, with
the increase of the number of data sources, the advantage
of the proposed protocol over other approaches becomes
more notable. This is due to the consideration of bandwidth
efficiency factor which can significantly improve the MAC
layer contention efficiency resulting in a lower collision ratio
and higher TCP throughput.
0
500
1000
1500
2000
2500
0 2 4 6 8 10
Thro
ughput
(kbps)
Number of data sources
VelocityVelocity + Link quality
Proposed
Fig. 16. Average data transfer throughput for various numbers of datasources.
D. Transmission throughput from the data source nodes to the
data carrier node
Fig. 17 shows the average data collection throughput from
the data source nodes to the data carrier node for various
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 11
source-carrier node distances (the distance between the data
source node and the data carrier node). The number of
source nodes was 6. The proposed protocol shows a notable
improvement in the throughput over other approaches due
to the cluster-based forwarding. The improvement is more
significant when the source-carrier distance is larger because
the cluster-based forwarding approach can improve the MAC
layer contention efficiency.
0
100
200
300
400
500
600
700
800
900
1000
400 600 800 1000 1200 1400 1600
Thro
ughput
(kbps)
Distance (m)
VelocityVelocity + Link quality
Proposed
Fig. 17. Average data collection throughput for various source-carrierdistances.
Fig. 18 shows the required time for collecting 1MB data
from each source node to the data carrier node for various
numbers of source nodes. The conventional approaches (“Ve-
locity + Link quality” and “Velocity”) selects routes based on
each traffic flow. This is inefficient when the number of source
nodes (number of traffic flows) is large. Since the proposed
protocol utilizes a cluster-based forwarding approach which
conducts packet forwarding based on cluster-heads, the MAC
layer contention efficiency and the transmission throughput
are improved. As a result, the data collection delay is reduced
significantly.
0
50
100
150
200
250
300
350
2 4 6 8 10 12
Req
uir
ed t
ime
(s)
Number of data sources
VelocityVelocity + Link quality
Proposed
Fig. 18. Required time for collecting 1MB data from each source node tothe data carrier node for various numbers of source nodes.
VII. CONCLUSIONS
We proposed a protocol which can store data in vehicular
ad hoc networks. The protocol takes into account throughput,
vehicle velocity, and bandwidth efficiency by using fuzzy
logic to conduct short-term evaluation and using a Q-learning
algorithm to consider long-term efficiency. The protocol also
employs a cluster-based forwarding approach to collect vehicle
data to the data carrier node. Through computer simulations,
we confirmed the advantages of the proposed protocol over
possible alternatives.
ACKNOWLEDGMENT
This research is partially supported by the projects
240079/F20 funded by the Research Council of Norway,
the project IoTSec – Security in IoT for Smart Grids, with
number 248113/O70 part of the IKTPLUSS program funded
by the Norwegian Research Council, and JSPS KAKENHI
Grant Number 16H02817.
REFERENCES
[1] S. He, D.-H. Shin, J. Zhang, J. Chen and Y. Sun, “Full-View AreaCoverage in Camera Sensor Networks: Dimension Reduction and Near-optimal Solutions,” IEEE Trans. Veh. Technol., vol.65, no.9, pp.7448–7461, 2016.
[2] S. He, J. Chen, X. Li, X. Shen and Y. Sun, “Mobility and IntruderPrior Information Improving the Barrier Coverage of Sparse SensorNetworks,” IEEE Trans. Mobile Comput., vol.13, no.6, pp.1268–1282,2014.
[3] R. Yu, X. Huang, J. Kang, J. Ding, S. Maharjan, S. Gjessing, and Y.Zhang, “Cooperative Resource Management in Cloud-enabled VehicularNetworks,” IEEE Trans. Ind. Electron., vol.62, no.62, pp.7938–7951,2015.
[4] R. Yu, Y. Zhang, S. Gjessing, W. Xia and K. Yang, “Toward Cloud-based Vehicular Networks with Efficient Resource Management,” IEEE
IEEE Netw., vol.27, no.5, pp.7938–7951, 2013.[5] K. Mershad and H. Artail, “Finding a STAR in a Vehicular Cloud,”
IEEE Trans. Intell. Transp. Syst., vol.5, no.2, pp.55–68, 2013.[6] E. Lee, E.-K. Lee, M. Gerla, and S. Y. Oh, “Vehicular cloud networking:
architecture and design principles,” IEEE Commun. Mag., vol.52, no.2,pp.148–155, 2014.
[7] S. Bitam, A. Mellouk, and S. Zeadally, “VANET-cloud: a genericcloud computing model for vehicular Ad Hoc networks,” IEEE Wireless
Commun., vol.22, no.1, pp.96–102, 2015.[8] B. Liu, D. Jia, J. Wang, K. Lu, and L. Wu, “Cloud-Assisted Safety
Message Dissemination in VANET-Cellular Heterogeneous WirelessNetwork,” IEEE Systems Journal, DOI: 10.1109/JSYST.2015.2451156,2016.
[9] R. Kim, H. Lim, and B. Krishnamachari, “Prefetching-Based DataDissemination in Vehicular Cloud Systems,” IEEE Trans. Veh. Technol.,vol.65, no.1, pp.292–306, 2016.
[10] Jiujun Cheng, Junlu Cheng, M. Zhou, F. Liu, S. Gao, and C. Liu,“Routing in Internet of Vehicles: A Review,” IEEE Trans. Intell. Transp.
Syst., vol.16, no.5, pp.2339–2352, 2015.[11] H. Yoo and D. Kim, “ROFF: RObust and Fast Forwarding in vehic-
ular ad-hoc networks,” IEEE Trans. Mobile Computing, vol.14, no.7,pp.1490–1502, 2015.
[12] M.-C. Chuang and M. C. Chen, “DEEP: Density-Aware EmergencyMessage Extension Protocol for VANETs,” IEEE Trans. Wireless Com-munications, vol.12, no.10, pp.4983–4993, 2013.
[13] C. Suthaputchakun, M. Dianati, and Z. Sun, “Trinary Partitioned Black-Burst based Broadcast Protocol for Time-Critical Emergency MessageDissemination in VANETs,” IEEE Trans. Veh. Technol., vol.63, no.6,pp.2926–2940, 2014.
[14] C. Wu, S. Ohzahata, and T. Kato, “VANET Broadcast Protocol Basedon Fuzzy Logic and Lightweight Retransmission Mechanism,” IEICETrans. Commun., vol.E95-B, no.2, pp.415–425, 2012.
[15] Y. Bi, H. Shan, X. S. Shen, N. Wang, and H. Zhao, “A Multi-HopBroadcast Protocol for Emergency Message Dissemination in UrbanVehicular Ad Hoc Networks,” IEEE Trans. Intell. Transp. Syst., vol.17,no.3, pp.736–750, 2016.
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 12
[16] S. Ucar. S. C. Ergen, and O. Ozkasap, “Multihop-Cluster-Based IEEE802.11p and LTE Hybrid Architecture for VANET Safety MessageDissemination,” IEEE Trans. Veh. Technol., vol.65, no.4, pp.2621–2636,2016.
[17] F. Liu, Z. Chen, and B. Xia, “Data Dissemination With Network Codingin Two-Way Vehicle-to-Vehicle Networks,” IEEE Trans. Veh. Technol.,vol.65, no.4, pp.2445–2456, 2016.
[18] K. Shafiee and V. C.M. Leung, “Connectivity-aware minimum-delaygeographic routing with vehicle tracking in VANETs,” Ad Hoc Netw.,vol.9, no.2, pp.131–141, 2011.
[19] Q. Yang, A. Lim, S. Li, J. Fang and P. Agrawal, “ACAR: AdaptiveConnectivity Aware Routing for Vehicular Ad Hoc Networks in CityScenarios,” Mobile Netw. Appl., vol.15, pp.36–60, 2010.
[20] M.H. Eiza and Q. Ni, “An Evolving Graph-Based Reliable RoutingScheme for VANETs,” IEEE Trans. Veh. Technol., vol.62, no.4, pp.1493–1504, 2013.
[21] M. Al-Rabayah,and R. Malaney, “A New Scalable Hybrid Routing Pro-tocol for VANETs,” IEEE Trans. Veh. Technol., vol.61, no.6, pp.2625-2635, 2012.
[22] J. Rak, “LLA: A New Anypath Routing Scheme Providing Long PathLifetime in VANETs,” IEEE Commun. Letters, vol.18, no.2, pp.281–284,2014.
[23] J. He, L. Cai, P. Cheng, and J. Pan, “Delay Minimization for DataDissemination in Large-Scale VANETs with Buses and Taxis,” IEEE
Trans. Mobile Comput., vol.15, no.8, pp.1939–1950, 2016.[24] X. Zhang, X. Cao, L. Yan, and D. K. Sung, “A Street-Centric Op-
portunistic Routing Protocol Based on Link Correlation for UrbanVANETs,” IEEE Trans. Mobile Comput., vol.15, no.7, pp.1586–1599,2016.
[25] M. A. Togou, A. Hafid, and L. Khoukhi, “SCRP: Stable CDS-BasedRouting Protocol for Urban Vehicular Ad Hoc Networks,” IEEE Trans.
Intell. Transp. Syst., vol.17, no.5, pp.1298–1307, 2016.[26] H. Zhu, M. Dong, S. Changm, Y. Zhu, M. Li, and S. Shen, “ZOOM:
Scaling the mobility for fast opportunistic forwarding in vehicularnetworks,” Proc. IEEE INFOCOM, pp.2832–2840, 2013.
[27] K. Ota, M. Dong, S. Chang, and H. Zhu, “MMCD: CooperativeDownloading for Highway VANETs,” IEEE Transactions on Emerging
Topics in Computing, vol.3, no.1, pp.34–43, 2015.[28] C. Wu, T. Yoshinaga, Y. Ji, T. Murase, and Y. Zhang, “Reinforcement
Learning-based Data Storage Scheme in Vehicular Ad Hoc Networks,”Proc. IEEE ICC, pp.1–6, 2016.
[29] G. Han, J. Jiang, C. Zhang, T. Q. Duong, M. Guizani, and G. K. Kara-giannidis, “A Survey on Mobile Anchor Node Assisted Localization inWireless Sensor Networks,” IEEE Communications Surveys & Tutorials,vol.18, no.3, pp.2220–2243, 2016.
[30] G. Han, L. Wan, L. Shu, and N. Feng, “Two Novel DOA Estimation Ap-proaches for Real-Time Assistant Calibration Systems in Future VehicleIndustrial,” IEEE Systems Journal, DOI:10.1109/JSYST.2015.2434822,2016.
[31] G. Han, L. Liu, J. Jiang, L. Shu, and G. Hancke, “Analysis of Energy-Efficient Connected Target Coverage Algorithms for Industrial WirelessSensor Networks,” IEEE Transactions on Industrial Informatics, DOI:10.1109/TII.2015.2513767, 2016.
[32] The Network Simulator - ns-2, http://www.isi.edu/nsnam/ns/, Accessedon Sep. 23, 2015.
[33] F. Bai, N. Sadagopan, and A. Helmy, “Important: A Framework toSystematically Analyze The Impact of Mobility on Performance ofRouting Protocols for Adhoc Networks,” Proc. 22nd Annual Joint Conf.of the IEEE Computer and Communications Societies, pp.825–835,2003.
[34] A. Khan, S. Sadhu, and M. Yeleswarapu, “A comparative analysis ofDSRC and 802.11 over Vehicular Ad hoc Networks,” Project Report,University of California, Santa Barbara, pp.1–8, 2009.
Celimuge Wu received his ME degree from theBeijing Institute of Technology, China in 2006, andhis PhD degree from The University of Electro-Communications, Japan in 2010. He has been anassistant professor with the Graduate School ofInformation Systems, The University of Electro-Communications since 2010, where he is currentlyan associate professor. His current research interestsinclude vehicular ad hoc networks, sensor networks,intelligent transport systems, IoT, 5G, and mobilecloud computing. He has been a track co-chair
or workshops co-chair of several international conferences including IEEEPIMRC 2016, IEEE CCNC 2017, ISNCC 2017 and WICON 2016.
Tsutomu Yoshinaga received the BE, ME, andDE degrees from Utsunomiya University in 1986,1988, and 1997, respectively. From 1988 to July2000, he was a research associate of the Faculty ofEngineering, Utsunomiya University. He was also avisiting researcher at Electro-Technical Laboratoryfrom 1997 to 1998. Since August 2000, he hasbeen with the Graduate School of Information Sys-tems, The University of Electro-Communications,where he is now a professor. His research interestsinclude computer architecture, interconnection net-
works, and network computing. He is a member of ACM, IEEE, IEICE andIPSJ.
Yusheng Ji received the BE, ME, and DE degreesin electrical engineering from the University ofTokyo, Japan. She joined the National Center forScience Information Systems, Japan in 1990. Sheis currently a professor with the National Instituteof Informatics, Japan, and Graduate University forAdvanced Studies. Her research interests includenetwork architecture, resource management, and per-formance analysis for quality of service provisioningin wired and wireless communication networks. Sheis a member of IEICE and IPSJ.
Tutomu Murase was born in Kyoto, Japan in1961. He received his M.E. degree from GraduateSchool of Engineering Science, Osaka University,Japan in 1986. He also received his Ph.D. degreefrom Graduate School of Information Science andTechnology, Osaka University, Japan in 2004. Heworked at NEC Corporation for 1986–2014. He wasa visiting professor of Tokyo Institute of Technol-ogy for 2012–2014. He is currently a professor inNagoya University. He has been engaged in researchon QoS control and traffic management for high-
quality and high-speed Internet. His current interests include wireless networkQoS control, MAC, transport and session layer traffic control, and networksecurity. He received Best Tutorial Paper Award on his invited paper in IEICEtransaction on communication in 2006. He is an IEEE member and an IEICEfellow.
0018-9545 (c) 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TVT.2016.2643665, IEEETransactions on Vehicular Technology
IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. XX, NO. XX, XXX 2016 13
Yan Zhang is a Full Professor at the Depart-ment of Informatics, University of Oslo, Norway.He received a PhD degree in School of Electrical& Electronics Engineering, Nanyang TechnologicalUniversity, Singapore. He is an Associate TechnicalEditor of IEEE Communications Magazine, an Edi-tor of IEEE Transactions on Green Communicationsand Networking, an Editor of IEEE CommunicationsSurveys & Tutorials, an Editor of IEEE Internet ofThings Journal, and an Associate Editor of IEEEAccess. He serves as chair positions in a number of
conferences, including IEEE GLOBECOM 2017, IEEE VTC-Spring 2017,IEEE PIMRC 2016, IEEE CloudCom 2016, IEEE ICCC 2016, IEEE CCNC2016, IEEE SmartGridComm 2015, and IEEE CloudCom 2015. He servesas TPC member for numerous international conference including IEEEINFOCOM, IEEE ICC, IEEE GLOBECOM, and IEEE WCNC. His currentresearch interests include: next-generation wireless networks leading to 5G,green and secure cyber-physical systems (e.g., smart grid, healthcare, andtransport). He is IEEE VTS (Vehicular Technology Society) DistinguishedLecturer. He is also a senior member of IEEE, IEEE ComSoc, IEEE CS,IEEE PES, and IEEE VT society. He is a Fellow of IET.
Top Related