Ieeepro techno solutions 2013 ieee embedded project driver behavior

4264 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 62, NO. 9, NOVEMBER 2013

Context-Aware Driver Behavior Detection Systemin Intelligent Transportation Systems

Saif Al-Sultan, Ali H. Al-Bayatti, and Hussein Zedan

Abstract—Vehicular ad hoc networks (VANETs) have emergedas an application of mobile ad hoc networks (MANETs), whichuse dedicated short-range communication (DSRC) to allow ve-hicles in close proximity to communicate with each other or tocommunicate with roadside equipment. Applying wireless accesstechnology in vehicular environments has led to the improvementof road safety and a reduction in the number of fatalities causedby road accidents through development of road safety applicationsand facilitation of information sharing between moving vehiclesregarding the road. This paper focuses on developing a novel andnonintrusive driver behavior detection system using a context-aware system in VANETs to detect abnormal behaviors exhibitedby drivers and to warn other vehicles on the road to preventaccidents from happening. A five-layer context-aware architectureis proposed, which is able to collect contextual information aboutthe driving environment, to perform reasoning about certain anduncertain contextual information, and to react upon that informa-tion. A probabilistic model based on dynamic Bayesian networks(DBNs) in real time, inferring four types of driving behavior(normal, drunk, reckless, and fatigue) by combining contextualinformation about the driver, the vehicle, and the environment,is presented. The dynamic behavior model can capture the staticand the temporal aspects related to the behavior of the driver, thusleading to robust and accurate behavior detection. The evaluationof behavior detection using synthetic data proves the validity ofour model and the importance of including contextual informationabout the driver, the vehicle, and the environment.

Index Terms—Context-aware system, driver behavior, dynamicBayesian networks (DBNs), safety application, vehicular ad hocnetworks (VANETs).

I. INTRODUCTION

A T THE present time, cars and other private vehicles arebeing used daily by large numbers of people. The biggest

problem regarding the increased use of private transport is therising number of fatalities that is occurring as a consequenceof accidents on the roads; the associated expense and relateddangers have been recognized as a serious problem that is beingconfronted by modern society. According to the U.K. Depart-ment of Transport’s report for road casualties in Great Britainfor the first quarter of 2011, there were 24 770 people killedor seriously injured due to road accidents. This number rep-resents a small decrease of 5%, as compared with the previous12 mo period [1]. Driver errors due to being affected by fatigue,

Manuscript received September 28, 2012; revised February 19, 2013;accepted May 5, 2013. Date of publication May 16, 2013; date of current versionNovember 6, 2013. The review of this paper was coordinated by Dr. F. Bai.

The authors are with the Software Technology Research Laboratory,De Montfort University, Leicester LE2 7EW, U.K. (e-mail: [email protected];[email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TVT.2013.2263400

being drunk, or being reckless are the main factors responsiblefor most road accidents.

Wireless communications and mobile computing have led tothe enhancement of and improvement in the intelligent trans-portation systems (ITS) that focus on road safety applications[2], [3]. As a core component of ITS, vehicular ad hoc networks(VANETs) have emerged as an application of mobile ad hocnetworks (MANETs), which uses dedicated short-range com-munication (DSRC) to allow nearby vehicles to communicateeither with each other or with roadside equipment. These formsof communication offer a wide range of safety applicationsto improve road safety and traffic efficiency and to providea clean environment. VANET safety applications are consid-ered to represent a vital step toward enhancing road safetyand improving traffic efficiency by preventing accidents fromoccurring, e.g., intersection collision avoidance, warning aboutviolating traffic signal, approaching emergency vehicle warn-ing, etc. [4]. Many researchers have been working in the area ofdriver monitoring and detection over recent decades; therefore,multiple systems have been proposed to monitor and detectthe status of drivers. Some researchers have tried to monitorthe behavior of the vehicle or the driver in isolation, whereasothers have focused on monitoring a combination of the driver,the vehicle, and the environment, to detect the status of thedriver in an attempt to prevent road accidents. However, thereis still no comprehensive system that can effectively monitorthe behavior of a driver, the vehicle’s state, and environmentalchanges to perform effective reasoning regarding uncertain con-textual information (driver’s behavior) to alert other vehicleson the road by disseminating warning messages in time tothe relevant vehicles in the vicinity, including implementingpractical corrective actions to avoid accidents.

In this paper, we propose a five-layer context-aware archi-tecture for a driver behavior detection system in VANETs thatcan detect four types of driving behavior in real-time driving:normal, fatigued, drunk, and reckless driving. It will then alertthe driver and other vehicles on the road by operating invehicle alarms and sending corrective action, respectively. Thefunctionality of the architecture is divided into three phases,which are the sensing, reasoning, and acting phases. In thesensing phase, the system collects information about the driver,the vehicle’s state, and environmental changes. The reasoningphase is responsible for performing reasoning about uncertaincontextual information to deduce the behavior of the driver.The behavior of the driver is considered as an uncertain context(high-level contextual information); therefore, effective reason-ing techniques about uncertain contextual information mustbe performed. Driver behavior is developed over the course

0018-9545 © 2013 IEEE

AL-SULTAN et al.: CONTEXT-AWARE DRIVER BEHAVIOR DETECTION SYSTEM IN ITS 4265

of driving; therefore, we have designed a dynamic Bayesiannetwork (DBN) model to perform a probabilistic reasoning toinfer the behavior of the driver. Our model combines infor-mation from different kinds of sensors to capture the staticand temporal aspects of behavior and to perform probabilisticinference to deduce the driver’s current driving style. The actingphase is responsible for operating in vehicle alarms and sendingcorrective actions to other vehicles, via wireless technologyprovided by VANETs.

The remainder of this paper is organized as follows.Section II introduces the work that has been done in the fieldof driver behavior detection. Our definition to the normal andabnormal driving behaviors is given in Section III. The pro-posed context-aware architecture for driver behavior detectionis explained in Section IV. Section V proposes the reasoningmechanism based on the DBN to collect and analyze the be-havior of the driver. System validation is shown in Section VI,and the conclusion is given in Section VII.

II. RELATED WORK

Several researchers have examined the development of drivermonitoring and detection systems using range of methods.Some have attempted to measure the driver’s state or the vehi-cle’s behavior to detect fatigued and drunk drivers. Meanwhile,other researchers have tried to monitor the driver, the vehicle,and the environment to detect the state of the driver. The mainstudies are summarized in the following.

In [5], the focus of the paper was on building a context-awaresmart car by developing a hierarchical model that is able tocollect, to reason about, and to react to contextual informationabout the driver, the vehicle, and the environment, providinga safe and comfortable driving environment. However, thissystem is restricted to warning the driver and controlling thevehicle and does not warn other vehicles on road by sendingwarning messages. In [6], a context-aware system is proposedthat is used to collect and analyze contextual information aboutthe driver, the vehicle, and the environment in real-time driving.It also collects information from questionnaires completed bythe drivers to create driving situations. The Bayesian networkis used to reason about this contextual information, which isrelatively uncertain information, by using a learning processto observe and predict the future behavior of the driver. Thesystem was able to predict the future behavior of the driverand cannot detect the current state of the driver and warn othervehicles on the road.

In [7]–[9], the detection of the fatigue level of the driverusing a video camera to extract different cues such as eye state,eyelid movement, gaze movement, head movement, and facialexpression is attempted to measure the fatigue level and warnthe driver via in-vehicle alarms.

In [10], a program that works on a mobile phone and thatcontains an accelerometer and orientation sensors placed inthe vehicle to detect a drunk driver in real time is developed.The program compares current accelerations with typical drunkdriving patterns. When the program indicates that the driveris influenced by alcohol, warning messages are generated toalert the driver, and a message is sent to inform police. In

[11], a drunk and drowsy driver detection system combiningbreath and alcohol sensors in a single device is developed. Thisdevice is able to measure the degree of alertness of the driver todetect charged water clusters in the driver’s breath to detect thepresence of alcohol using breath and alcohol sensors.

In [12], a system for drowsy driver detection in real-timedriving by collecting information about the driver’s behavior,such as the speed of the vehicle, the vehicle’s lateral position,the yawing angle, the steering wheel angle, and the vehicle’slane position is proposed. Their system uses artificial neuralnetworks to combine different indications of drowsiness andto predict whether a driver is drowsy and to issue a warningif required. In [13], a noncontact system to prevent driverdrowsiness by detecting the eyes of the driver and checkingwhether they are opened or closed using a charged-coupled de-vice (CCD) camera has been developed. The system is based oncapturing the face of the driver and on using image processingtechniques to check if the eyes are closed for long intervals. Ifthe eyes are closed, the driver is drowsy, and the system willissue a warning to the driver.

The driver behavior detection systems described earlier focuson the detection of driver’s status (drunk, affected by fatigue,drowsy) by monitoring the driver or the vehicle and by issuingwarning messages to the driver to prevent road accidents. Whilethese systems have achieved good results in terms of improvingroad safety, they are limited to alerting the driver or control-ling the vehicle itself. Moreover, they have not consideredthe behavior of the driver as a high-level context (uncertaincontext). This paper attempts to construct a comprehensivesystem that is able to detect normal and abnormal drivingbehavior using a context-aware system to collect and analyzecontextual information about the driver, the vehicle’s state,and environmental changes and to perform reasoning aboutcertain and uncertain contexts. The driver and other vehicles arethen alerted by operating an in-vehicle alarm and by sendingwarning messages containing corrective actions via wirelesstechnology provided by VANETs, thus providing a flexible yetmore accurate proactive driver behavior detection system.

III. OVERVIEW OF DRIVER BEHAVIOR

Several definitions of driver behavior have been proposed inthe literature. In [14], normal driving behavior is defined asthe majority of behavior exhibited by each driver during theirdaily driving, whereas it defined abnormal driving behavioras the behavior of a driver while influenced by mental orphysical factors. In [15]–[17], the authors referred to the taskof driving as a complex dynamic environment and defineddriving as the interaction between the driver, the vehicle, andthe environment (surrounding road information and traffic).In [18], aggressive driving is defined as when the driver com-mits a combination of moving traffic offenses that may cause adanger to other drivers or property. Aggressive drivers are thosewho exceed the speed limit, who follow the front vehicle tooclosely, who perform unsafe lane change, and who fail to obeytraffic control rules, e.g., traffic signals. In [19], normal behav-ior is defined as a situation in which driver is concentrating ondriving.


In [20], driver behavior is defined as a sequence of actions,each of which is associated with the specific state of the driver,the vehicle, and the environment, which can be characterizedby a set of contextual information. In [21], driver behavior isreferred to as a sequence of internal states of the driver, eachof which may be observed by capturing associated observablefeatures (contextual information). In [22], it is stated that thedriver has a large number of internal mental states and that atransition from one state to another occurs during driving.

In this paper, the behavior of the driver is defined from theperspective of context awareness as follows. Driver behavior isa complex and dynamic interaction between three entities: thedriver, the vehicle, and the environment. It is described as atransition between a sequence of states (e.g., normal, affectedby fatigue, drunk, or reckless); over the course of driving, adriver will be in a particular state, which he or she may remainin for a period of time and then potentially changing to adifferent state. Each state can be characterized by capturinga large amount of contextual information of relevance to theinteracting entities. The behavior of the driver is considered tobe normal (safe) if his or her actions associated with the currentstate will not lead to an accident; it is otherwise considered tobe unsafe (abnormal).

The behavior of the driver can be represented as follows:

B = {St=1, St=2, . . . , St=n}

where B is the behavior of the driver, S is the state, and t is thetime. The states of the driver were classified into four classes:normal driving Sn, drunk driving Sd, fatigued driving Sf , andreckless driving Sr. As defined, each state may be characterizedby capturing observable context C. The state may be referredto as:

(St=i) = {C1, C2, C3, . . . , Ck}.

In conclusion, the behavior of the driver is considered as thecurrent unobservable state St=i that can be characterized bycapturing a set of observable context Cj , where St=i is the stateat time = i, and Cj is the context that need to be captured tocharacterize the state.

Based on the previous definitions [15]–[23] of the drivingbehavior, we have defined four categories of driving behavior.

1) Normal behavior: Behavior is considered to be normalwhen driver concentrates on the driving task. This canbe characterized by controlling the speed of the vehicle,avoiding sudden acceleration, driving without alcoholintoxication, maintaining a proper position between lanemarkers, and the driver having his or her eyes open whiledriving. When the driver matches the aforementionedcriteria, behavior is considered normal.

2) Drunk behavior: This refers to driving while intoxicatedby alcohol and is characterized by a set of observableactions such as sudden acceleration, driving withoutmaintaining the proper lane position, driving with outcontrolling the speed, and usually having closed eyes formore than 80% for a period of time.

3) Fatigue Behavior: In [24], fatigue is defined as an evolv-ing process that increase during driving and is associatedwith a loss of effectiveness in driving. In [24]–[26], it isstated that a driver driving after a period of 17 h withno sleep behaves exactly as a driver who has 0.05%intoxication of alcohol. A driver driving after a periodof 24 h with no sleep behaves exactly as one who has0.1% intoxication of alcohol. Based on this argument,fatigue driving was defined as driving that exhibits thesame characteristics as drunk driving but without alcoholintoxication in the driver’s blood.

4) Reckless behavior: In [27], the reckless driver is definedas a driver who drives at high speed and a high degreeof acceleration and puts other traffic participants at risk.The driver is classified as driving in this category whenthere is no alcohol intoxication and the driver’s eyes areopened, but the following behaviors are exhibited: drivingwith sudden acceleration, not maintaining the proper laneposition, and not controlling the vehicle’s speed.

IV. CONTEXT-AWARE-BASED ON-BOARD

UNIT ARCHITECTURE

Context-aware systems are those systems that are capable ofadapting their operations to the current context without userinteraction and are thus aimed at augmenting usability andeffectiveness by taking into account the environment’s contex-tual information [28]. Context-aware systems incorporate thefollowing three main subsystems [29].

• Sensing subsystem: the phase for gathering contextualinformation by sensors;

• Reasoning (thinking) subsystem: the phase for employingreasoning techniques to contextual data to obtain high-level contextual information (e.g., user situation);

• Acting subsystem: Depending on the current situation, thesystems that provide services to users.

Our architecture, as shown in Fig. 1, is divided into threemain phases, i.e., a sensing phase, a reasoning phase, and anapplication phase, which represent the three main subsystemsof a context-aware system, i.e., the sensing, reasoning, andacting subsystems, respectively. Sending corrective actions oroperating the in-vehicle alarms in the third layer depends on theresult of the second layer, which in turn depends on receivingthe information of the first layer.

A. Sensing Phase

The sensing phase is responsible for gathering contextualinformation about the driver, the vehicle, and the environmentand for transferring the collected information into a machine-executable form to be processed in the next phase. It is dividedinto two layers as follows.

• Sensors Layer: This layer is responsible for acquiring thecontext data. It consists of a set of different sensors in-tegrated into the driving environment in which the systemoperates. Different types of sensors provide different typesof information according to the system requirements. Twotypes of data sources (sensors) gather context data. The


Fig. 1. Driver behavior detection system architecture.

internal data sources (physical sensors) refer to the set ofsensors within the vehicle, such as cameras, speed sensor,GPS, alcohol, and the accelerometer sensor, which provideinformation about the vehicle’s speed, acceleration infor-mation, the direction of driver’s eyes, the position in lane,and the level of alcohol in the driver’s blood.

It also incorporates information from external datasources (virtual sensors), including traffic managementcenters (TMCs), which provide information relating totraffic, weather, and road conditions, based on the website,dynamic message signs, and highway auditory radio data[2]. External data sources also include information aboutother vehicles (i.e., speed, current position, and direction)collected through received hello messages.

• Raw data retrieval layer: The purpose of applying thislayer is to separate low-level sensing details from thesensors for the upper layer of the system and the abstractcontextual information received from the sensor layer.This layer contains the following two components.

– Data acquisition unit: responsible for controlling andcoordinating all the sensors in the sensor layer;

– Context interpreter: modeling process that is donein this component in terms of transferring the datareceived from the data acquisition unit into a machine-executable form. Several types of modeling algorithmscan be used to abstract the received sensory data (e.g.,ontology modeling). The received data may come from

different types of sensors, such as cameras, GPS, andspeed sensors. This component transfers the data intoa form that can be processed by the reasoner.

B. Reasoning Phase

This phase is responsible for extracting the situation of thedriver and calculating corrective actions for other vehicleson the road. There are two types of contextual information:certain information, which is obtained from a single sensor, anduncertain contextual information, which cannot be acquired bya single sensor and may be incomplete or inexact. The behaviorof the driver is categorized as uncertain contextual information(high-level contextual information). In this phase, the behav-ior detection algorithm performs reasoning about uncertainty(driver behavior) by combining data acquired from differentsensors to detect the state of the driver during real-time driving.The corrective action algorithm is responsible for calculat-ing the appropriate corrective action to other vehicles on theroad. The reasoning phase consists of two layers as follows.• Reasoning Layer: This layer is responsible for extracting

the current state of the driver (e.g., normal, fatigued, drunk,or reckless), and it generates corrective actions for othervehicles to avoid road accidents. This layer comprises thefollowing components.– Processor: The onboard unit (OBU) processor is respon-

sible for managing all the components of the OBU andcontrolling all the tasks and activities it performs. Theprocessor performs the following two algorithms.

• Behavior detection algorithm: This algorithm isdesigned to reason about uncertain contextual in-formation to detect the current behavior of thedriver using a DBN algorithm to combine the datacollected from a sensing layer and to detect thetype of behavior. If the behavior of the driveris normal, no action is needed. In the case ofabnormal driving behavior (e.g., drunk, fatigued,or reckless), the processor performs the correctiveaction algorithm. In this paper, we will focus onthe driver behavior detection algorithm.

• Corrective action algorithm: The aim of perform-ing this algorithm is to choose the appropriatein-vehicle alarm and to calculate the proactivecorrective action for other vehicles on the roadaccording to their positions, velocities, and direc-tions with the use of predefined digital road mapsand the information collected from the adaptivehello messages. The corrective action algorithmwill be out of the scope of this paper.

– Control unit 1: This unit is responsible for controllingin-vehicle alarms such as seat vibration and audio alarmto attract the driver’s attention. This unit receives the sig-nal from the processor in the case of abnormal drivingbehavior.

– Control unit 2: After receiving the signal from the pro-cessor indicating abnormal driving behavior, this unitsends signals to the DSRC/wireless access in vehicu-lar environment (WAVE) device to transmit corrective


Fig. 2. Driver behavior detection system mechanism.

messages to other vehicles on the road or to the roadsideunit.

– DSRC/WAVE network device: The OBU contains aDSRC/WAVE network device based on IEEE 802.11p[30]. It is responsible for connecting the vehicle to othervehicle’s OBUs or with the roadside unit through thewireless radio frequency based on IEEE 802.11p. TheOBU can send or receive messages via this networkdevice.

– Power supply: The power supply is responsible forproviding power to the OBU. It is rechargeable andprovides power to the OBU without any constraints.

– User interface: This contains the audio and video inter-face that allows the user to interact with the servicesprovided by the OBU.

• Storage Layer: In this layer, the database stores predefineddigital maps of the road and the historical data (past drivingsituations).

C. Application Phase

This phase represents the acting subsystem in a context-aware system. It is responsible for disseminating warning mes-sages that includes corrective actions for other vehicles on theroad. It also operates in-vehicle alarms to warn the driver toprevent the occurrence of accidents and to decrease the numberof potential fatalities.

Fig. 2 shows the mechanism of detecting the behavior ofthe driver and calculating corrective action for other vehicleson the road. The vehicle sense contextual information aboutthe vehicle, the driver, and the environment from sensors thatinclude both physical and logical sensors, such as speed, ac-celerometer, TMC, adaptive hello message, camera, GPS, andalcohol sensors, which are connected to the OBU. After collect-ing this information from the sensors, the interpreter transferthe different kinds of data to a form that can be processed bythe processor by applying one of the modeling techniques, such

as ontology modeling [31]. This will be out of the scope of thispaper.

The OBU processor performs the DBN algorithm to per-form reasoning about an uncertain context (driver behavior)by combining data received from the interpreter using theprobabilistic inference. If the output of the inference is a normaldriving behavior that satisfies all normal driving criteria, noaction will be taken by the processor, and the vehicle willsense new information. If the output of the inference is anabnormal driving behavior, such as being drunk, fatigued, orreckless, the processor performs the algorithm of calculatingthe corrective action for other vehicles on the road and choosingthe appropriate in-vehicle alarm according to the position ofother vehicles and their velocity and direction. After calculatingcorrective actions for other vehicles and choosing the in-vehiclealarm, the processor will send a signal to control units 1 and2 to operate an in-vehicle alarm and to send the correctivemessage to other vehicles through the DSRC network device.This process is based on a context-aware system and is a self-organizing process in which information sensing, reasoning,and acting upon this information occur instantly.

V. DYNAMIC BAYESIAN NETWORK DRIVER BEHAVIOR

DETECTION MODEL

As stated previously in the definition driver behavior, thisrefers to a transition between a set of states during the courseof driving. For example, the alcohol level in the driver’s bloodmay be low at the beginning of the driving but will becomehigher if the driver is drinking while driving; the level of fatiguemay also increase during driving [24]. This fact indicates that,in addition to the observable context at the current time slice,the driver’s state at the previous time slice is also consideredan indicator for the state at the current time slice. Moreover,the driver may exhibit different behaviors at different times. Itis therefore very important to capture the temporal aspect ofbehavior and to integrate the evidences over time.

As a result, the accurate and effective detection of differenttypes of behavior requires different types of context to becombined. This information may be incomplete or inaccuratedue to the inaccurate reading of some sensors and the fact thatdifferent variables need to be combined. Several informationfusion methods have been proposed such as fuzzy logic, theDampester–Shapher theory, neural networks, and the Kalmanfilter. These methods do not provide efficient expressive capa-bilities to capture incomplete data, uncertainties, dependencebetween the variables, and the temporal aspect exhibited bythe behavior. This system uses DBNs to combine data fromdifferent types of sensors to deduce driver behavior due to thefollowing reasons. First, they are considered to be the mostreliable method for dealing with inaccurate data and unobserv-able physical values. Second, they are able to model time-seriesdata. Third, they are efficient at combining uncertain contextualinformation from a wide range of sensors to deduce high-levelcontextual information (reason about uncertain context) and areable to combine prior data with current data [32]–[39].

A DBN is a directed acyclic graph that represents the con-ditional independence between a set of random variables and


deals with uncertain information and probabilistic inferenceupon receiving evidences. It consists of a set of nodes thatrepresent the random variables and a set of arcs representing theconditional independence between variables. It is considered asa set of static Bayesian networks interconnected by sequentialtime slices. The relationship between two neighboring timeslices can be modeled using the first-order hidden Markovmodel, which means that the random variables at time slicet are affected by the variables at time slice t and by thevariables at time slice (t− 1) only [35], [37], [40], [41]. ADBN can be defined as a pair of (S,

−→S ), where S is a static

Bayesian network that defines the prior P (Z1), and−→S defines

P (Zt|Zt−1), which is a two-slice temporal Bayesian network,as shown in the following [42]:

P (Zt|Zt−1) =

N∏

i=1

P(Zit |Pa(Zi

t))

(1)

where N is the number of nodes in the network, Zit is the ith

node at time slice t, and Pa(Zit) are the parents of Zi

t .The first step in designing a DBN is to identify the hypothesis

variables and group them into mutually exclusive events. Thesecond step is to identify the information variables that denotesomething about the hypothesis. After defining the hypothesisvariable and information variables, the next step is to cre-ate the directed links between the variables in the networks,which reflect the conditional independence between variablesto construct the static Bayesian network at time t = 1. Finally,the conditional probability table (CPT) for each node in thenetwork and the conditional probability over time have to becreated given its parent [43]. In this system, the DBN is treatedas a singly connected static Bayesian networks in which thehypothesis node at time slice t depends on the observations attime slice t and the hypothesis node at time slice t− 1 only.

A. Defining Network Variables (Nodes)

The hypothesis node in this network is the state node,which includes four mutually exclusive states: fatigue, normal,drunk, and reckless. This reflects the state of the driver at thecurrent time. The information variables were divided into twogroups: the first group representing variables that may affectthe behavior and the second group representing the informationthat results from a specific behavior type. The two groups aredefined as follows.

• Group 1 is composed of information variables that affectthe behavior, including the circadian rhythm and the driv-ing environment.

– Circadian rhythm: This refers to the human sleep–awakecycle, which is considered a cause of driver fatigue.There are two periods during the day (3:00–5:00 P.M.and 3:00–5:00 A.M.) during which human reach theirpeak level of fatigue [24], [32], [35]. The circadianrhythm was considered as one of the causes that affectthe hypothesis nodes. The circadian rhythm node isaffected by two nodes, i.e., time and time zone nodes.

Fig. 3. Static Bayesian network structure at time t = 1.

– Driving environment: Noise and temperature are con-sidered to have a high influence on the driving environ-ment, which can in turn cause fatigue. Fatigue is morelikely to occur when noise and high temperature occurinside or outside the vehicle [32], [35]. The drivingenvironment node was selected as one of the effects thatcause the hypothesis nodes.

• Group 2 is composed of information variables that resultfrom specific behavior, including vehicle-related informa-tion (vehicle speed, position between lane markers, andacceleration) and driver-related information (the state ofthe driver’s eyes and the level of alcohol in the driver’sblood).

– Controlling the speed: Drunk and fatigued driversstruggle to control their speed due to the mental stateof the affected driver. According to the definition ofthe reckless driver use in this paper, the driver mayviolate the speed limit [10].

– Position in the lane: In [10], it is stated that the drunkdriver has a problem in maintaining the lane position(vehicle position between lane markers).

– Acceleration: The driver is considered to use normalbehavior while driving with normal acceleration andconsidered to exhibit abnormal behavior, such asbeing drunk, fatigued, or reckless while driving withsudden acceleration [10].

– State of the driver’s eyes: The eyes of the fatigueddriver are usually measured by PERCLOS, which isconsidered to be the most accurate measure of driverfatigue. PERCLOS is the measure of eyes that are 80%covered by the eyelid for a period of time [8], [9], [32],[44], [45]. Eye movements affect eyelid movements,which can be measured via PERCLOS and AECS.

– Intoxication: This refers to the amount of alcohol inthe driver’s blood. This node has two states (less than0.05% and more than 0.05%). The driver is consideredto be a drunk driver if there is alcohol intoxication ofmore than 0.05% [11], [46].


Fig. 4. DBN structure.

TABLE IHYPOTHESIS NODE STATES

B. Network Graph

After deciding the variables included in the network, the nextstep is to decide the conditional independence between them(drawing the directed acyclic graph). Fig. 3 shows the staticBayesian network structure at time (t = 1) and the conditionalindependence between the variables. The DBN is shown inFig. 4, where the hypothesis node at time slice t is affected bythe information variables at time slice t and the hypothesis nodeat time slice t− 1 only.

C. Parameterizing the Network

Parameterizing the network (choosing the values of CPT) isthe step that the prior probability of the root nodes and the con-ditional probabilities of the links in the network are determined.First, all the nodes in the network and their possible statesmust be specified before the value (probability) for each stateis chosen. Tables I and II show the state space for each node.

Deciding the value for each node is important for the networkto be useful. Defining the probability of each node in thenetwork refers to the probability of node X being in state Awhen the evidence is received. The following two methods maybe used to obtain the probabilities of the states for each node inthe network.

• The values can be obtained by performing statistical anal-ysis of a huge amount of training data. Training dataare obtained by performing several tests in a test bedspecifically designed for the system and by collecting theoutput for each test.

• Parameterizing the network can be done using severalpublished papers and studies that are related or similar tothe system.

TABLE IINETWORK VARIABLES AND THEIR STATES

TABLE IIIPRIOR PROBABILITY FOR THE TIME NODE

It was too difficult to acquire a large amount of training datafor this paper as no test bed is equipped with all the sensorsrequired for the model. No previous study provides the datarequired to parameterize the system. Hence, in this model,the values for CPTs and the transition distributions betweentime slices from a wide range of published papers and reportswere maintained [10], [12], [24], [25], [32], [35], [37], [46]–[48]. This enabled all the probabilities required to parameterizethe DBN model to be obtained. Tables III–XVIII show theconditional probabilities for each node in the network.

Having specified the nodes, the conditional independencebetween them, and the CPT for each node in the graph andhaving configured the static Bayesian network at time t = 1,


TABLE IVPRIOR PROBABILITY FOR THE TIME ZONE NODE

TABLE VPRIOR PROBABILITY FOR THE NOISE NODE

TABLE VIPRIOR PROBABILITY FOR THE TEMPERATURE NODE

TABLE VIICONDITIONAL PROBABILITIES FOR THE

CIRCADIAN NODE GIVEN ITS PARENTS

TABLE VIIICONDITIONAL PROBABILITIES FOR THE ENVIRONMENT

NODE GIVEN ITS PARENTS

TABLE IXCONDITIONAL PROBABILITIES FOR THE LANE

MAINTENANCE NODE GIVEN ITS PARENT

TABLE XCONDITIONAL PROBABILITIES FOR THE ACCELERATION

NODE GIVEN ITS PARENT

TABLE XICONDITIONAL PROBABILITIES FOR THE CONTROLLING

SPEED NODE GIVEN ITS PARENT

TABLE XIICONDITIONAL PROBABILITIES FOR THE INTOXICATION


TABLE XIIICONDITIONAL PROBABILITIES FOR THE EYE MOVEMENT


TABLE XIVCONDITIONAL PROBABILITIES FOR THE EYELID

MOVEMENT NODE GIVEN ITS PARENT

TABLE XVCONDITIONAL PROBABILITIES FOR THE AECS NODE GIVEN ITS PARENT

the next stage involves preforming the inference process tocalculate the conditional probability for the hypothesis nodeover time after receiving a set of evidences via sensors todetect the behavior of the driver. Inference is the process ofcombining the low-level data collected by different sensorsand of deducing high-level contextual information (e.g., driverbehavior). The proposed model combines data about the driver,the vehicle, and the environment to deduce the driver behavior.The behavior of the driver depends on the sensory data, withdifferent sensor readings leading to different driving behaviors.

VI. EVALUATION

Here, we evaluate the performance of the proposed driverbehavior detection system using synthetic data. Given the pa-rameterized DBN, the driver behavior inference process startsupon reception of evidences via sensors. As shown in Fig. 4, thenetwork consists of ten evidence nodes (root nodes and leavesnodes), each of which has two possible states. The total numberof all the possible combinations of evidences is 210. As, thecircadian node is affected by the time and time zone nodes andthe environment node is affected by the noise and temperature


TABLE XVICONDITIONAL PROBABILITIES FOR THE STATE

NODE AT TIME t− 1 GIVEN ITS PARENTS

TABLE XVIICONDITIONAL PROBABILITIES FOR THE STATE

NODE AT TIME T GIVEN ITS PARENTS

TABLE XVIIICONDITIONAL PROBABILITIES FOR THE

PERCLOS NODE GIVEN ITS PARENT

nodes, we will treat the circadian and the environment nodes asevidence nodes during this evaluation. The eye movement nodeis treated as an evidence node as it affects the eyelid movementnode, which affects the PERCLOS and AECS nodes. After con-sidering the circadian, the environment and the eye movementas evidence nodes, the total number of all possible combina-tions attained is 27, which is equal to 128 possible inputs.

Here, we have instantiated all the possible combinationsof inputs, and the system has been able to detect thestate of the driver (fatigue, drunk, reckless, and normal)

over time accurately, applying all possible combinations. Itis impossible to detail all the possible combinations here;only samples of the combinations of evidence are shown inTable XIX. This shows the states of the evidence nodes, theinference results (hypothesis node’s state), and the degreeof belief. (Eye_movements = EM, Controlling_speed = CS,Acceleration = A, Intoxication = I, Lane_maintenance = LM,Circadian = C, and Environment = E).

It is shown in Table XIX that the system is able to detect fourdriving behaviors, which are fatigue, drunkenness, reckless-ness, and normal driving behavior. The system is able to detectnormal driving behavior in all possible states for the circadianand environment nodes (see 1–4 in Table XIX). The detectionof drunken driving is more accurate when the eyes of the driverare closed as this provide a combination of intoxication andother evidences (see 5–12 in Table XIX), as compared with thecase where the eyes are opened, combined with intoxicationand other evidences (see 13–20 in Table XIX). When weinstantiated a single evidence regarding eye movement, thefatigue level of the driver (see 28 in Table XIX) was foundto be less accurate than when we instantiated more than oneevidence in combination with eye movement evidence (see 27in Table XIX). In the same situation with the case of recklessdriving behavior, when we instantiated a single evidence suchas sudden acceleration (see 35 in Table XIX), the detectionaccuracy was less than when we instantiated more than oneevidence (see 29–34 in Table XIX).

Fig. 5 shows a comparison between all the possible com-binations of evidences, which lead to assessment of fatiguebehavior. As shown in the figure, four curves represent the levelof fatigue in all possible states of circadian and environmentnodes. The level of driver fatigue in the case of Circadian =awake and environment=good is the lowest level in the chart,whereas in the case of Circadian = fatigue and environment =bad, the fatigue reaches its highest level. This demonstrates theeffects of the environment and the circadian rhythm on thedriver’s level of fatigue. This further validates our driver be-havior detection system model.

Fig. 6 shows a comparison between all the possible com-binations of evidence, which lead to assessment of recklessbehavior. Four curves in the figure characterize the belief ofreckless behavior in all possible states for the circadian andenvironment nodes. Reckless behavior is more highly presentwhen the environment = good and circadian = awake, whichagain validates our driver behavior detection model.

Fig. 7 shows a comparison between all the possible com-binations of evidences, which lead to an assessment of drunkbehavior. As shown in the figure, there are four curves demon-strating the drunk behavior in all possible states of circadianand environment nodes. The belief of drunk behavior is approx-imately the same in all cases. It reaches its highest degree whencircadian = fatigue and the environment = bad.

The aforementioned inference results reveal the fact that thepresence of more than one evidence guarantees the occurrenceof a specific behavior, and explain the importance of combiningdifferent types of contextual information to deduce the behaviorof a driver. These results show the utility of the proposed driverbehavior detecting system.


TABLE XIXSAMPLES OF THE COMBINATIONS OF EVIDENCES

Fig. 5. Comparison between all possible evidence of fatigue behavior.

All data used in the evaluation of our system are syntheticdata since there was no test bed equipped with all sensorsrequired for collecting the data for the proposed model. If atest bed or real vehicle equipped with required sensors wereavailable, the CPTs for the network could be reparameterizedbased on the real data. Using one of the available learningalgorithms, such as expectation–maximization algorithm, can

Fig. 6. Comparison between all possible evidences of reckless behavior.

lead to parameterize the network with real data to representreal-time cases.

VII. CONCLUSION

Because it is a promising area of VANETs, safety ap-plications are attracting increasingly more consideration.


Fig. 7. Comparison between all possible evidences of drunk behavior.

Monitoring and detecting the behavior of drivers is vital toensuring road safety by alerting the driver and other vehicleson the road in cases of abnormal driving behavior. Driverbehavior is affected by many factors that are related to thedriver, the vehicle, and the environment, and over the courseof driving, a driver will be found to be in a particular state;the driver can then stay in this state for a period of time orshift to another state. Hence, it is important to capture thestatic and dynamic aspects of behavior and take into accountthe contextual information that relates to driver behavior. Inthis paper, we have presented a driver behavior detectionsystem in VANETs from the viewpoint of context awareness.Our contributions are threefold: 1) A five-layer context-awarearchitecture, which can detect the behavior of the driver, ispresented by capturing information about the driver, the vehicle,and the environment; 2) a DBN algorithm for inferring driverbehavior from different kind of sensors under uncertainty hasbeen formulated to capture the static and dynamic aspects of thebehavior; and 3) definitions for normal and abnormal drivingbehaviors are given. The evaluation result has demonstrated thedetection accuracy of the proposed model under uncertaintyand the importance of including a great amount of contextualinformation within the inference process. Our future workcomprises designing a corrective action algorithm to calculatethe appropriate corrective actions for other vehicles on theroad. Modeling techniques for transferring the data collectedfrom sensors into a machine-processable format will also bedeveloped.

REFERENCES

[1] Dep. for Transp., London, U.K., Reported Road Casualties in GreatBritain: Quarterly Provisional Estimates Q1 2011, 2011. [Online]. Avail-able: http://www.dft.gov.uk/statistics/releases/road-accidents-and-safety-quarterly-estimates-q1-2011

[2] S. Olariu and M. C. Weigle, Vehicular Networks: From Theory to Practice.London, U.K.: Chapman & Hall, 2009.

[3] Y. Qian and N. Moayeri, “Design of secure and application-orientedVANETs,” in Proc. IEEE VTC Spring, May 2008, pp. 2794–2799.

[4] “Vehicle Safety Communications Project Task 3 Final Rep., Identify In-telligent Vehicle Safety Applications Enabled by DSRC,” U.S. Dep. ofTrans., Washington, DC, USA, Tech. Rep. DOT HS 809 859, 2005.

[5] J. Sun, Y. Zhang, and K. He, “Providing context-awareness in the smartcar environment,” in Proc. IEEE CIT , Jul. 2010, pp. 13–19.

[6] A. Rakotonirainy, “Design of context-aware systems for vehicle usingcomplex systems paradigms,” in Proc. Workshop Safety CONTEXT Conj.,Paris, France, 2005, pp. 464–475.

[7] M. S. Devi and P. R. Bajaj, “Driver fatigue detection based on eye track-ing,” in Proc. IEEE ICETET , Nagpur, Maharashtra, India, Jul. 2008,pp. 649–652.

[8] H. Singh, J. S. Bhatia, and J. Kaur, “Eye tracking based driver fatiguemonitoring and warning system,” in Proc. IEEE IICPE, New Delhi, India,Jan. 2011, pp. 1–6.

[9] Z. Zhu and Q. Ji, “Real time and non-intrusive driver fatigue monitoring,”in Proc. IEEE ITSC, Oct. 2004, pp. 657–662.

[10] J. Dai, J. Teng, X. Bai, Z. Shen, and D. Xuan, “Mobile phone baseddrunk driving detection,” in Proc. PervasiveHealth, Munich, Germany,Mar. 2010, pp. 1–8.

[11] M. Sakairi and M. Togami, “Use of water cluster detector for preventingdrunk and drowsy driving,” in Proc. IEEE Sensors, Kona, HI, USA,Nov. 2010, pp. 141–144.

[12] D. Sandberg and M. Wahde, “Particle swarm optimization of feedforwardneural networks for the detection of drowsy driving,” in Proc. IEEEIJCNN, Hong Kong, Jun. 2008, pp. 788–793.

[13] H. Ueno, M. Kaneda, and M. Tsukino, “Development of drowsiness de-tection system,” in Proc. Veh. Navig. Info. Sys. Conf., Yokohama, Japan,Aug. 1994, pp. 15–20.

[14] T. Imamura, H. Yamashita, Z. Zhang, M. R. Bin Othman, and T. Miyake,“A study of classification for driver conditions using driving behaviors,”in Proc. IEEE SMC, Oct. 2008, pp. 1506–1511.

[15] N. Oliver and A. P. Pentland, “Driver behavior recognition and predictionin a smartcar,” in Proc. SPIE Aerosense–Enhanced Synth. Vis., 2000,pp. 280–290.

[16] D. Mitrovic, “Reliable method for driving events recognition,” IEEETrans. Intell. Transp. Syst., vol. 6, no. 2, pp. 198–205, Jun. 2005.

[17] N. Oliver and A. P. Pentland, “Graphical models for driver behaviorrecognition in a smartcar,” in Proc. IEEE Intell. Veh. Symp., Dearborn,MI, USA, 2000, pp. 7–12.

[18] Nat. Hwy. Traffic Safety Admin., Define Aggressive Driving, Washington,DC, USA, 2012. [Online]. Available: http://www.nhtsa.gov/people/injury/enforce/aggressdrivers/aggenforce/define.html

[19] M. Miyaji, M. Danno, and K. Oguri, “Analysis of driver be-havior based on traffic incidents for driver monitor systems,” inProc. IEEE Intell. Veh. Symp., Eindhoven, The Netherlands, Jun. 2008,pp. 930–935.

[20] M. Helander, “Applicability of drivers’ electrodermal response to thedesign of the traffic environment,” J. Appl. Psychol., vol. 63, no. 4,pp. 481–488, Aug. 1978.

[21] A. Pentland and A. Liu, “Modeling and prediction of human behavior,”Neural Comput., vol. 11, no. 1, pp. 229–242, Jan. 1999.

[22] H. Berndt, J. Emmert, and K. Dietmayer, “Continuous driver intentionrecognition with hidden Markov models,” in Proc. IEEE ITSC, Beijing,China, Oct. 2008, pp. 1189–1194.

[23] T. Wakita, K. Ozawa, C. Miyajima, K. Igarashi, K. Itou, K. Takeda, andF. Itakura, “Driver identification using driving behavior signals,” in Proc.IEEE ITSC, Sep. 2005, pp. 396–401.

[24] P. Jackson, C. Hilditch, A. Holmes, N. Reed, N. Merat, and L. Smith,“Fatigue and road safety: A critical analysis of recent evidence,” U.K.Dep. for Transp., London, U.K., Tech. Rep. RSWP21, 2011.

[25] T. A. Commiss., Melbourne, Vic., Australia, Reducing Fatigue—A CaseStudy, 2012. [Online]. Available: http://www.tac.vic.gov.au

[26] D. Dawson and K. Reid, “Fatigue, alcohol and performance impairment,”Nature, vol. 388, no. 6639, p. 235, Jul. 1997.

[27] T. Bar, D. Nienhuser, R. Kohlhaas, and J. M. Zollner, “Probabilisticdriving style determination by means of a situation based analysis of thevehicle data,” in Proc. IEEE ITSC, Washington, DC, USA, Oct. 2011,pp. 1698–1703.

[28] M. Baldauf, S. Dustdar, and F. Rosenberg, “A survey on context-awaresystems,” Int. J. Ad Hoc Ubiq. Comput., vol. 2, no. 4, pp. 263–277,Jun. 2007.

[29] S. Loke, Context-Aware Pervasive Systems: Architectures for a New Breedof Applications. New York, NY, USA: Auerbach, 2006.

[30] M. M. Al-Doori, A. H. Al-Bayatti, and H. Zedan, “Context aware archi-tecture for sending adaptive hello messages in VANET,” in Proc. ACMCASEMANS, Copenhagen, Denmark, 2010, pp. 65–68.

[31] H. Chen and T. Finin, “An ontology for context-aware pervasive com-puting environments,” Knowl. Eng. Rev., vol. 18, no. 3, pp. 197–207,Sep. 2003.

[32] Q. Ji, P. Lan, and C. Looney, “A probabilistic framework for modeling andreal-time monitoring human fatigue,” IEEE Trans. Syst., Man, Cybern. A,Syst., Humans, vol. 36, no. 5, pp. 862–875, Sep. 2006.

[33] G. Rigas, C. D. Katsis, P. Bougia, and D. I. Fotiadis, “A reasoning-based framework for car driver’s stress prediction,” in Proc. Med. ControlAutom. Conf., Ajaccio, France, Jun. 2008, pp. 627–632.

[34] K. Livescu, O. Cetin, M. Hasegawa-Johnson, S. King, C. Bartels,N. Borges, A. Kantor, P. Lal, L. Yung, A. Bezman, S. Dawson-Haggerty,B. Woods, J. Frankel, M. Magami-Doss, and K. Saenko, “Articulatoryfeature-based methods for acoustic and audio-visual speech recognition:Summary from the 2006 JHU summer workshop,” in Proc. IEEE ICASSP,Honolulu, HI, USA, Apr. 2007, vol. 4, pp. 621–624.


[35] G. Yang, Y. Lin, and P. Bhattacharya, “A driver fatigue recognition modelbased on information fusion and dynamic Bayesian network,” Inf. Sci.,vol. 180, no. 10, pp. 1942–1954, May 2010.

[36] V. Pavlovic, J. M. Rehg, T. Cham, and K. P. Murphy, “A dynamic Bayesiannetwork approach to figure tracking using learned dynamic models,” inProc. IEEE Comput. Vis., 1999, vol. 1, pp. 94–101.

[37] X. Li and Q. Ji, “Active affective state detection and user assistance withdynamic Bayesian networks,” IEEE Trans. Syst., Man, Cybern. A, Syst.,Humans, vol. 35, no. 1, pp. 93–105, Jan. 2005.

[38] C. Bettini, O. Brdiczka, K. Henricksen, J. Indulska, D. Nicklas,A. Ranganathan, and D. Riboni, “A survey of context modelling andreasoning techniques,” Perv. Mobile Comput., vol. 6, no. 2, pp. 161–180,Apr. 2010.

[39] H. Wu, “Sensor fusion for context-aware computing using Dempster-Shafer theory,” Ph.D. dissertation, Robot. Inst., Carnegie Mellon Univ.,Pittsburgh, PA, USA, Dec., 2003.

[40] Y. Zhang, Q. Ji, and C. Looney, “Active information fusion for decisionmaking under uncertainty,” in Proc. Info. Fus., Annapolis, MD, USA,2002, vol. 1, pp. 643–650.

[41] Z. Ghahramani, “An introduction to hidden Markov models and Bayesiannetworks,” Int. J. Pattern Recognit. AI, vol. 15, no. 1, pp. 9–42, Feb. 2001.

[42] K. P. Murphy, “Dynamic bayesian networks: Representation, infer-ence and learning,” Ph.D. dissertation, Dep. Comput. Sci., Univ. Calif.,Berkeley, CA, USA, 2002.

[43] F. V. Jensen and T. D. Nielsen, Bayesian Networks and Decision Graphs.New York, NY, USA: Springer-Verlag, 2007.

[44] I. Daza, N. Hernandez, L. Bergasa, I. Parra, J. Yebes, M. Gavilan,R. Quintero, D. Llorca, and M. Sotelo, “Drowsiness monitoring basedon driver and driving data fusion,” in Proc. IEEE ITSC, Washington, DC,USA, Oct. 2011, pp. 1199–1204.

[45] L. M. Bergasa, J. Nuevo, M. A. Sotelo, R. Barea, and M. E. Lopez, “Real-time system for monitoring driver vigilance,” IEEE Trans. Intell. Transp.Syst., vol. 7, no. 1, pp. 63–77, Mar. 2006.

[46] W. Qing and Y. WeiWei, “A driver abnormality recondition modelbased on dynamic bayesian network for ubiquitous computing,” in Proc.ICACTE, Chengdu, China, Aug. 2010, vol. 1, pp. 320–324.

[47] E. L. Co, K. B. Gregory, J. M. Johnson, and M. R. Rosekind, “CrewFactors in Flight Operations XI: A Survey of Fatigue Factors in RegionalAirline Operations,” NASA Ames Res. Cent., Moffett Field, CA, USA,Tech. Rep., 1999.

[48] D. H. Harris, “Visual detection of driving while intoxicated,” HumanFactors, J. Hum. Factors Ergonom. Soc., vol. 22, no. 6, pp. 725–732,Dec. 1980.

Saif Al-Sultan received the B.Sc. degree from theUniversity of Technology, Baghdad, Iraq, in 2004.

He is currently a Doctoral Researcher withthe Software Technology Research Laboratory, DeMontfort University, Leicester, U.K. His currentresearch interests include context-aware systems, ve-hicular ad hoc networks, and intelligent transporta-tion systems.

Ali H. Al-Bayatti received the B.Sc. degree fromthe University of Technology, Baghdad, Iraq, in 2005and the Ph.D. degree from the Software Technol-ogy Research Laboratory, De Montfort University,Leicester, U.K., in 2009.

He is currently a Research Fellow and Head ofthe Intelligent Transportation theme at the SoftwareTechnology Research Laboratory, De Montfort Uni-versity. His research deals with vehicular (e.g., Ve-hicular Ad hoc Networks) and smart technologies(e.g., Context-aware Systems) that promote collec-

tive intelligence. Applications range from promoting comfort to enabling safetyin critical scenarios. The goal of his research is to improve the effectiveness,efficiency, mobility, and safety of transportation systems. His current researchinterests include intelligent transportation, vehicular ad hoc networks, mobilecomputing, wireless computing, context-aware systems; pervasive computingand computer/mobile security.

Hussein Zedan received the Ph.D. degree in mathe-matics from the University of Bristol, Bristol, U.K.

He worked as a Researcher and a Lecturerwith the University of Bristol. Then, he workedas a Research Fellow and as a University Lec-turer with the University of Oxford, Oxford, U.K.,and was a Reader, a Professor, and a Head ofDepartment with the University of York, York,U.K. Finally, he set up the Software TechnologyResearch Laboratory at De Montfort University,Leicester, U.K. He is currently the Technical Direc-

tor of a university technology center in software evolution, which is fundedby Software Migration Ltd. He is a Professor of software engineering withover 30 years experience as an academic, a consultant, and a practitioner in thecomputer science and information technology industry. He is the author of fourbooks and over 190 technical papers and articles in highly reputable journalsand international conferences. His research interests include software engineer-ing techniques involving object technology, rapid application development andcomponent-based development, grid technology, computer security, trust andcritical systems, and context-aware systems.

Ieeepro techno solutions 2013 ieee embedded project driver behavior

Engineering

Transcript of Ieeepro techno solutions 2013 ieee embedded project driver behavior