Final Report Project Title: Driver Models for both Human and Autonomous Vehicles

DISCLAIMER

The contents of this report reflect the views of the authors, who are responsible for the facts and the accuracy of the information presented herein. This document is disseminated under the sponsorship of the U.S. Department of Transportation’s University Transportation Centers Program, in the interest of information exchange. The U.S. Government assumes no liability for the contents or use thereof.

i

OVERVIEW

This is the Final Report of the Project

Driver Models for Both Human and Autonomous Vehicles with Different Sensing Technologies and Near-crash Activity

Project Lead: Umit Ozguner

Director, University Transportation Center (UTC) Professor, Department of Electrical & Computer Engineering Transportation Research Center (TRC) Inc. Chair on Intelligent Transportation Systems (ITS) Email: ozguner.1@osu.edu Office: 614-292-5940 Fax: 614-292-7596 205 Dreese Laboratories; 2015 Neil Avenue, Columbus OH, 43210

Other Investigators: Donald Fisher (UMass), Abdollah Homaifar (NCA&T), John Lee (UW), David Woods (OSU)

Project Description

The goal of this project was to understand how multi-agent models of the driver and vehicle can inform design principles for optimized autonomous vehicle systems. In this project, a computational model for human behavior in pre-crash scenarios was developed and investigated.

A multi-agent model with both human drivers and autonomous and semi-autonomous vehicles was considered. The model was build upon successful models used in our Defense Advanced Research Projects Agency (DARPA) Grand Challenge vehicles, and also incorporates results from our experience in automotive industry project. This model takes dynamic inputs about the changing situation and behavior of others, and uses mathematical or symbolic processing to carry out the functions required to simulate the perception, attention, cognition, and control behavior of interest. We integrate different component models, including control theory models, decision and judgment models; learning classifier systems, joint human-automation system models, and attention models, to build a comprehensive model needed to make predictions in pre-crash situations, and needed to make quantitative estimates of hypothesized safety improvements.

These models have especially been used for investigating lane-change and merge type of activities, where crashes can occur. We have also considered convoy type of operations, a phenomena that we expect to be of increasing importance as driverless fleets become a possibility in the near future.

A number of researchers in the Consortium have contributed to the regular joint discussions held. The majority of the activity under this Project was concentrated at OSU and NC A&T, and the Project Report combines two Chapters, summarizing the activities at both locations.

1

Driver Models for both Human and Autonomous Vehicles Activity-1 Authors: Arda Kurt (kurta@ece.osu.edu) Umit Ozguner (ozguner.1@osu.edu) Peng Liu (liupbit@gmail.com )

2

1. Introduction

OSU CrIS UTC researchers are investigated different ways of capturing driver behavior in

computational and functional models. The purpose of developing these models was to provide a

means of understanding and quantifying human driving behavior for the computational aspects of

current and future transportation systems, ranging from Advanced Driver Assistance Systems

(ADAS) to partial/full automation applications.

There are multiple viable methodologies in modeling the decision-making and low-level driving

aspects of the various tasks that are undertaken by human drivers. Earlier studies (Liu2007) in

cognitive driving models mostly utilized the division of various subtasks such as perception,

implementation and decision-making that operate concurrently during regular human driving. One

example illustration of such a cognitive model can be seen below.

Figure 1. Human Driver Model

The most recent, CrIS-funded effort at OSU was a continuation of the earlier efforts on capturing

parts of this driver model in Hybrid-State Systems (HSS) and probabilistic state machines

3

(Kurt2010). These earlier models dealt specifically with intersection approach scenarios, as

illustrated below, and involved ad-hoc models tested and verified with real driving data.

Figure 2. Intersection approach decisions

The driver modeling effort as part of CrIS was carried out in two parallel tracks. OSU researchers

investigated probabilistic models of decision for crash-imminent scenarios through Hidden

Markov Models (HMM) and Hierarchical Hidden Markov Models (HHMM), which have shown

promise in both capturing driver modes/intents, and also factoring in stochastic nature of such

decisions. On the other hand, NCAT researchers investigated Support Vector Machines (SVM)

and fuzzy models for a similar purpose.

Common to both of the above-mentioned research tracks is a need for real or realistic driving data

covering relevant scenarios. Early on, a number of larger-scale data collection efforts such as VTTI

100-Car Naturalistic Driving Study and NHTSA SHRP2 were explored as candidate data sources.

Since the publicly available portion of the 100-Car study is de-identified (GPS locations and other

pieces of information stripped off), and the SHRP2 data took considerable time being processed,

our initial attempts at identifying useful driving data has focused on our internal resources. For

this purpose, data collected with an instrumented OSU research vehicle for an earlier project was

examined to pinpoint near-crash situations. Even though this data set is not extensive as the above-

4

mentioned naturalistic driving studies, it provided a starting point for the initial model-building

efforts and help get our group of researchers from multiple institutions onto the same page.

The rest of this report is organized into specific studies that were carried out under Project 2, with

brief overviews and representative illustrations from the publications generated as part of CrIS

UTC effort.

2. Prediction a. Trajectory Prediction Using Driver Models

In (Peng2014), OSU researchers focused on accurate trajectory prediction of a lane-changing

vehicle as the main application for driver modeling and estimation. This is a key issue for risk

assessment and early danger warning in advanced driver assistance systems(ADAS). The

particular study presented in (Peng2014) proposed a trajectory prediction approach for a lane

changing vehicle considering high-level driver status.

Figure 3. Lane change scenario description and process partition.

5

A driving behavior estimation and classification model was developed based on Hidden Markov

Models(HMMs). The lane change behavior was estimated by observing the vehicle state emissions

in the beginning stage of a lane change procedure, and then classified by the classifier before the

vehicle crosses the lane mark.

Furthermore, the future trajectory of the lane changing vehicle was predicted in a statistical way

combining the driver status estimated by the classifier. The classifier was trained and tested using

naturalistic driving data, which shows satisfactory performance in classifying driver status. The

trajectory prediction method generated different trajectories based on the classification results,

which is important for the design of both autonomous driving controller and early danger warning

systems.

The overall framework for the trajectory predictor is modeled as a combination of Hybrid-State

Systems, and Hidden Markov Models. The hybrid framework, shown below, consists of three

parts: the hybrid system representing the driver/vehicle dynamic of the lane changing vehicle

(target vehicle); the driver behavior classifier estimating the current driver status of the target

vehicle; and the trajectory prediction module generating the future vehicle trajectories of the

cutting-in stage. The driving behavior classifier takes the vehicle state emission from the target

vehicle and calculate the probabilities of two HMM models representing normal and dangerous

driving behavior, respectively. The trajectory prediction module generates the statistic trajectory

based on the real lane-change data set when considering the driver behavior classification result.

6

Figure 4. Lane change trajectory prediction framework, this framework assumes that the state

emissions of the cut-in vehicle could be observed via V2V network or sensors mounted on the host

vehicle.

In order to know whether a driver is under dangerous driving, it is essential to get the driver’s

current decision state. However, it’s difficult to directly estimate the current high-level driving

states using a hybrid state system. To solve this problem, the high-level driver state transition is

formulated as a stochastic process and is estimated using HMM by considering the historical states

and emissions. A Hidden Markov Model (HMM) can be expressed as a tuple λ = {N,M,π,T,e},

which consists of a series of N discrete hidden states and the corresponding observations for each

state, as illustrated below. The observations are dependent on the hidden states, and each state has

a probability distribution over each possible output. An HMM expresses the driver/vehicle

dynamic system in a stochastic way properly while eliminating the low-level vehicle dynamic

evolution. In addition, the modeled system will be in one of the states at any given time k, and the

hidden states are the same as that of the finite state machine in HSS.

7

Figure 5. A Diagram of the Hidden Markov Model

For the classifier, two HMM models are trained using Expectation Maximization (EM) method,

one model for normal driving, and the other for dangerous driving. The data of the lane changing

vehicle obtained from the observation interval are tested in both HMM models, and the conditional

probabilities P(x|λi) are calculated using forward algorithm. The classification result is computed

by comparing the probabilities returned from the two HMMs, as shown below. Further vehicle

trajectory prediction is conducted in a statistical way based on the classification results.

Figure 6. HMM-based classifier

Given the driving behavior classification results of a lane changing vehicle, the following vehicle

trajectory can be predicted in a statistical way using the correlated driving data set. The predicted

absolute trajectory offset with 50 and 85 percentage for both normal and dangerous driving are

8

shown below along with absolute offsets of the real trajectories. To use the obtained driving data

reasonably, the longitudinal traveling speed range for the data samples are set from 20 m/s to 30

m/s. While a smaller speed range division can make the lane change data more consistent, too

small speed range could cause excessive screening of the driving data, which causes a demand for

very large data sets. The results below shows that the dangerous lane change absolute trajectory

offsets have larger variances comparing with the normal lane change instances, which corresponds

to the statistical results in section V-A. In addition, the 50 percentage and 85 percentage absolute

trajectory offsets have large differences for both normal and dangerous lane change cases.

Figure 7. Absolute trajectory offset prediction for both normal and dangerous driving.

b. Decisive Features

Continued work along the HMM-based classification lines, presented in (Liu2015Classification),

introduced the concept of “decisive features.” These are features that are more likely to appear in

dangerous lane change processes. A feature detection module was proposed specifically

considering decisive features correlated to dangerous lane change. Furthermore, the feature

detection module was integrated into the HMM classifier to enhance classification ability. The

9

proposed classifier was verified with a separate test data set, and showed satisfactory results in

reducing false negative rate of misclassification.

The system framework for this more complex classifier is given in the figure below. The system

structure of the driver behavior classifier is developed in a cascade form, as shown in Figure 2.

The behavior classifier consists of a prior data filtering module, a maneuver predictor, a behavior

classifier, and a posterior feature detection module. The maneuver predictor consists of several

HMMs of which each state for one driver maneuver, e.g. move forward, turn left, change lane,

etc., in the given traffic context. In general, the maneuver predictor is a nonlinear function that

maps from the observation sequence of vehicle states to a driver maneuver.

Figure 8. System framework of the behavior classifier with decisive driving feature detection.

Using Gaussian observation emission models, the HMM classifier is utilized as shown below. In

general, driver states of the target vehicle during the first stage of lane change, i.e. stage before the

target vehicle crossing lane mark, are denoted by the N hidden states of an HMM. A driver is

assumed to be within one of these states during this stage. The vehicle state measurements of this

stage is treated as the emission sequence of the hidden regime states generated by the driver. With

this expression, HMMs representing normal and dangerous lane changes could be trained using

labeled driving data, respectively. With the two trained HMMs, for an observed vehicle state

sequence, the classification result is given by comparing how well the given models fit the

observation sequence. The behavior classification problem under the HMM modeling is then

turned into identifying the most likely hidden regime sequence using the observed emission

sequence.

10

Figure 9. HMM applied to model driver-vehicle interaction in highway lane change behavior

classifier.

11

3. Control

a. Model Predictive Control Using Lane Change Behavior Models

Expanding the classification and prediction work into a driver-model-aware controller, in the study

(Liu2015Predictive) a model predictive control approach was proposed for a vehicle convoy

traveling on homogeneous highway. Specifically, the lane change behavior of a cut-in vehicle in

front of the vehicle convoy was estimated and predicted for the predictive control of the leading

vehicle in the convoy. In order to capture state differences of the preceding vehicle in different

lane change manners, a cost function considering riding comfort and spacing safety was designed

for the leading vehicle based on the free headway set ahead. The proposed cost function integrated

trajectory differences of the cut-in vehicle, which makes the controller sensible to different lane

change manners. The space keeping and velocity tracking performance of the controller was tested

and compared with a conservative controller under different lane change behavior of the cut-in

vehicle. Furthermore, the control effects on the spacing and velocity of the followers in the convoy

were analyzed. The simulation results showed that the controller considering lane change behavior

difference has smaller velocity and spacing fluctuations dealing with lane change disturbance.

Figure 10. Vehicle convoy formation with a preceding cut-in vehicle, assuming convoy members

have V2V communication.

12

For this Cooperative Adaptive Cruise Control (CACC) setting, the vehicle convoy is formed by

one leader (the host vehicle) and m followers. All the followers in the convoy try to keep a

preferred traveling velocity while keeping certain spacing df to the vehicle in front of it. The host

vehicle also needs to maintain desired velocity and free spacing in front of it, and deal with sudden

cut-in events, as shown below. Here we focused on the scenario when a lane changing vehicle cuts

in front of the host vehicle coercively. In this scenario, only the vehicles in the lane change

behavior classification zone of the adjacent lane are of interest. In order to control the host vehicle

in a predictive way, the vehicle state of the cut-in vehicle, xc, is needed. Specifically, the future

position and velocity of the cut-in vehicle is got by predicting its future trajectory after crossing

the lane mark. The high-level lane change behavior is considered in the trajectory prediction.

The behavior-aware Model Predictive Controller (MPC) was compared to a more conservative

MPC, and the effects of the cut-in vehicle on the convoy host were comparatively analyzed, as

briefly illustrated below.

Figure 11. Longitudinal distance and velocity of the host vehicle (left), Force input of the host

vehicle (right).

The tracking performance of the three followers in the convoy is also analyzed. The figure below

shows the velocity profiles of the three followers when the host vehicle is using different tracking

strategies. The velocity profiles of the followers have smaller fluctuations when the host vehicle

uses the controller that predicts the trajectory of the preceding cut-in vehicle considering lane

13

change behavior prediction results. Furthermore, the followers avoid velocity overshoot when the

host vehicle considers the lane change behavior difference of the cut-in vehicle.

Figure 12. Longitudinal velocities of the followers in the convoy when the host vehicle is using

different tracking strategies.

Figure 13. Distance to the preceding vehicle, with the shadowed region states for the lane change

process of the preceding vehicle.

14

The CACC setting and the MPC controller were further studied in (Liu2017Synthesis), with the

development of a nonlinear bound to state the performance of the proposed controller. Simulations

of a cut-in scenario were conducted using the CarSim simulation environment to show the

effectiveness of the proposed controller, as snapshot below.

Figure 14. Screenshots of the test scenario in CarSim, the white vehicle is the cut-in (target)

vehicle, the dark vehicle is the lead vehicle of a convoy.

Most recently, a Distributed Model Predictive Control methodology was developed and presented

in (Liu2017Distributed). The method takes into account macroscopic traffic management and

microscopic vehicle dynamics to achieve efficiently cooperative highway driving. Critical traffic

information beyond the scope of human perception is obtained from connected vehicles

downstream to establish necessary traffic management mitigating congestion. With

backpropagating traffic management advice, a connected vehicle having an adjustment intention

exchanges control-oriented information with immediately connected neighbors to establish

potential cooperation consensus, and to generate cooperative control actions. The performance of

the distributed control scheme and the energy-saving potential of conducting such cooperation are

tested in a mixed highway traffic environment by the means of microscopic simulations.

15

References

[1] Liu, Y.; Ozguner, U., "Human Driver Model and Driver Decision Making for Intersection Driving," Intelligent Vehicles Symposium, 2007 IEEE , vol., no., pp.642,647, 13-15 June 2007.

[2] Kurt, Arda, John L. Yester, Yutaka Mochizuki, and Ümit Özgüner. "Hybrid-state driver/vehicle modelling, estimation and prediction." In Intelligent Transportation Systems (ITSC), 2010 13th International IEEE Conference on, pp. 806-811. IEEE, 2010.

[3] Liu, Peng, and Arda Kurt. "Trajectory prediction of a lane changing vehicle based on driver behavior estimation and classification." In Intelligent Transportation Systems (ITSC), 2014 IEEE 17th International Conference on, pp. 942-947. IEEE, 2014.

[4] Liu, Peng, Arda Kurt, Keith Redmill, and Umit Ozguner. "Classification of highway lane change behavior to detect dangerous cut-in maneuvers." In The Transportation Research Board (TRB) 95th Annual Meeting, vol. 2. 2015.

[5] Liu, Peng, and Ümit Özgüner. "Predictive control of a vehicle convoy considering lane change behavior of the preceding vehicle." In American Control Conference (ACC), 2015, pp. 4374-4379. IEEE, 2015.

[6] Liu, Peng, Arda Kurt, and Umit Ozguner. "Synthesis of a behavior-guided controller for lead vehicles in automated vehicle convoys." Mechatronics (2017).

[7] Liu, Peng, Umit Ozguner, and Yeqing Zhang. "Distributed MPC for cooperative highway driving and energy-economy validation via microscopic simulations." Transportation Research Part C: Emerging Technologies 77 (2017): 80-95.

16

Driver Models for both Human and Autonomous Vehicles Activity-2 Authors:

Saina Ramyar (sramyar@aggies.ncat.edu)

Seifemichael B. Amsalu (amsalu@aggies.ncat.edu)

Zihao Wang (zwang1@aggies.ncat.edu)

Keyvan Majd (kmajd@aggies.ncat.edu)

Abdollah Homaifar* (homaifar@ncat.edu)

Address:

Autonomous Control & Info. Tech. (ACIT) Center North Carolina A&T State University 1601 East Market Street Fort IRC Bldg. Greensboro, NC 27411

Corresponding author*: Abdollah Homaifar (email)

17

INTRODUCTION The North Carolina Agricultural and Technical (N.C A & T) state university team was

involved in both project 2 and project 3. Our goal in project 2 was to understand how multi-agent models of the driver and vehicle can inform design principles for optimized autonomous vehicle systems. In this project, we have developed and refined a computational model for human behavior in pre-crash scenarios. Our team also focused on using machine-learning tools to improve the performance of Advanced Driver Assistance Systems (ADASs). We studied the sensitivity of previously developed Hidden Markov driver models to increase the accuracy of the developed models. Moreover, Support Vector Machine models are developed for normal and dangerous driving that help the controller plan safe maneuvers, as well as create a basic situational awareness of the surrounding vehicles.

Trajectory planning is the final stage of the four-hierarchal-class motion planning platform, which is first suggested in [1] for the autonomous on-road driving. In order to directly apply the output of our computational model for human behavior into the motion-planning task, we expanded our research by investigating a novel trajectory-planning framework in which we employ the extracted knowledge from the human driver behavioral model. In our framework, we proposed a general problem formulation in which the surrounding environment’s limitations and any future estimated trajectory of other traffic participants can be formulated as inequality constraints on vehicle kinematic model configurations to guarantee the safe maneuverability of the motion task.

References [1] P. Varaiya, "Smart Cars on Smart Roads: Problems of Control," IEEE Transactions on

Automatic Control, vol. 38, no. 2, pp. 195-207, 1993.

18

RESEARCHES UNDER PROJECT 2: DRIVER MODELS FOR BOTH HUMAN AND AUTONOMOUS VEHICLES

1. Modeling Drivers’ Actions at Intersections The capability to estimate a driver's intention leads to the development of advanced driver

assistance systems that can assist the drivers in complex situations. Developing precise driver behavior models near intersections can considerably reduce the number of accidents at road intersections.

Figure 1: A simple intersection scenario. Given the right of way is for the.

Several models of driver behavior in different scenarios are developed by the team at N.C A & T. These models are based on Support Vector Machines (SVM), Hidden Markov Models (HMM) and Takagi-Sugeno (TS) Fuzzy models, and they are able to estimate a driver’s intention at an intersection.

A model combining Support Vector Machine (SVM) and Hybrid State System (HSS) is developed for driver behavior estimation near intersections. It provides accurate results that perform close to that of a human observer. This work combines the HSS framework with multi-class SVM based on statistical feature extraction. The statistical feature extraction enables the SVM to represent the historical information as the HMM represents sequence. A one-vs-one multi-class SVM classification method with the RBF kernel is used here. The proposed method is also compared with HMM based on HSS in estimating the state of the driver at an intersection. In many applications, SVM outperforms HMM in generalizing the pattern recognition problems. Also, the SVM has shown a higher performance rate above 97% is achieved in driver behavior estimation near an intersection.

In another study, a fuzzy based modeling technique is introduced for estimation of driver behavior at intersections. The required observations for modeling are velocity, acceleration and yaw-rate which are selected according to vehicle motion equations. In the proposed method, velocity and yaw rate are nonlinear functions of their values at one and two-time steps before. These nonlinear functions are approximated with local Takagi-Sugeno models using a Gath-Geva fuzzy clustering technique. The model is trained and then tested with naturalistic driving data from the OSU. The simulation results show that the model has a good estimation performance. One of the contributions of this fuzzy technique is its consideration of the order of the data in time-series,

19

as it employs a second order difference equation for the maneuver contrary to other classification techniques. The SVM and k nearest neighbor only consider data point by point without any order. These modifications have resulted in more accuracy and less computationally expensive estimations. The next contribution in this study is the two step evaluation process. First it eliminates the stopping intent, and later determines the trajectory (straight, right or left). The implementation of this process reduces the computational complexity by reducing the unnecessary estimation of the driver’s direction if he/she is stopping first. Consequently, the reaction time can be reduced in situations that everyone follows traffic regulations.

References [2] S. Amsalu, A. Homaifar, Afagh, F., S. Ramyar and A. Kurt, "Driver behavior modeling

near intersections using support vector machines based on statistical feature extraction," in Intelligent Vehicles Symposium (IV), 2015.

[3] S. Ramyar, A. Homaifar, A. Anzagira, A. Karimoddini, S. Amsalu and A. Kurt, "Fuzzy modeling of drivers' actions at intersections," in World Automation Congress (WAC), 2016.

[4] S. Ramyar, A. Homaifar, A. Karimoddini and E. Tunsel, "Identification of anomalies in lane change behavior using one-class SVM," in Systems, Man, and Cybernetics (SMC), 2016.

[5] S. Ramyar, M. Sefidmazgi, S. Amsalu, A. Anzagira, A. Homaifar, A. Karimoddini and A. Kurt, "Modeling driver behavior at intersections with takagi-sugeno fuzzy models," in Intelligent Transportation Systems (ITSC), 2015.

2. Personalized Automated Highway Driving System

The technology of automated driving has improved significantly over the last two decades. Vehicles implemented with some partial automation systems such as adaptive cruise control are already in existence, and there are several fully automated vehicles in the prototype stage.

One of the most popular scenarios in the autonomous driving field is highway driving. However, because of the high speed, there are many latent hazards on highways that will result in significant loss. Thus far, the partial and full autonomy available in vehicles are designed based on the system engineers’ knowledge, with the focus on safety. But to increase the confidence in autonomous vehicles, they must be able to cater to the individual needs of drivers and satisfy different styles.

In this study, a path-planning framework for autonomous highway driving is proposed which performs according to an individual driver’s preference. The proposed system can switch between different highway driving maneuvers and plan safe trajectories for all stated maneuvers while considering the driver’s satisfaction. The control system implemented on the system ensures the safety of the vehicle, however, there are some situations in which various drivers behave differently. For instance, in semi-congested traffic, less aggressive drivers prefer to remain in the same lane and follow the lead vehicle while more aggressive drivers prefer to alternate between lanes. The novel algorithm proposed in this study can make decisions about such arbitrary maneuvers with the driver’s preference in mind.

20

Figure 2: Summary of the personalized highway driving system.

The proposed system consists of a RF-based driver model, which estimates the driver’s preferred acceleration with respect to the vehicle’s features and traffic conditions. Another component of this system is a model predictive controller, which tracks the desired acceleration while maintaining safety constraints. In addition, a one-class SVM classifier is trained with the driver’s lane change history and is able to identify satisfactory lane change trajectories. We propose a novel decision-making algorithm in this work, which employs the driver model, the controller and the lane change inspector and results in a personalized maneuver decision and trajectory.

All the components of the system are tested separately and together and, the results showed the accurate and rational performance of the proposed system. The comparison results with a personalized adaptive cruise controller and a general MPC controller also showed the superior performance of the proposed system. An example of the car following performance of the proposed system is shown below. Based on the results, Driver A usually prefers a higher speed given a certain inter vehicle gap compared to Driver B. It can be concluded that Driver A is relatively more aggressive than Driver B. Moreover, the results confirm that different drivers react differently in various scenarios and this system is able to detect the difference in style and follow that while ensuring the safety of the vehicle.

References

[6] S. Ramyar, A. Homaifar, S. M. Salaken, S. Nahavandi and A. Kurt, "A personalized

highway driving assistance system," in Intelligent Vehicles Symposium (IV), 2017.

[7] S. Ramyar, A. Homaifar and A. Kurt, "Strategic decision-making algorithm with personalized features for highway driving," in IEEE Transaction on Intelligent Vehicles. (Submitted)

21

Figure 3: Car following behavior of the two drivers in personalized highway driving.

3. A Simplified Matrix Formulation (SMF) for Sensitivity Analysis of Hidden Markov Models (HMMs) Sensitivity analysis is a general technique for investigating the robustness of the output of a

system model. Sensitivity analysis of probabilistic networks has recently been studied extensively. This has resulted in the development of mathematical relations between a parameter and an output probability of interest and methods for establishing the effects of parameter variations on decisions. Sensitivity analysis in HMMs has usually been performed by taking small perturbations in parameter values and re-computing the output probability of interest. As recent studies show, the sensitivity analysis of an HMM can be performed using a functional relationship that describes how an output probability varies as the network’s parameters of interest change. To derive this sensitivity function, existing Bayesian network algorithms have been employed for HMMs. These algorithms are computationally inefficient as the length of the observation sequence and the number of parameters increases. Our goal in this research was to propose a simplified efficient matrix-based algorithm for computing the coefficients of the sensitivity function for all hidden states and all-time steps.

In this research, we proposed a sensitivity analysis algorithm for HMMs using a simplified matrix formulation directly from the model representation based on a recently proposed technique called the Coefficient-Matrix-Fill procedure [8]. Until recently, sensitivity analysis in HMMs has been performed using Bayesian network sensitivity analysis techniques. The HMM is represented as a dynamic Bayesian network unrolled for a fixed number of time slices, and the Bayesian sensitivity algorithms are used. However, these algorithms do not utilize the HMMs’ recursive probability formulations. In this work, a simple algorithm based on a simplified matrix formulation is proposed. In this algorithm, to calculate the coefficients of the sensitivity functions, a series of forward matrices 𝐹𝐹𝑘𝑘.𝑘𝑘 = 1. … . 𝑡𝑡. are used, where 𝑘𝑘 represents the time slice in the observation

22

sequence. The matrices (Initial, Transition, and Observation) where the corresponding model parameter 𝜃𝜃 varies are decomposed into the parts independent of and dependent on θ for mathematical convenience. This enables us to compute the coefficients of the sensitivity function at each iteration in the recursive probability expression and implement the algorithm in a computer program. These matrices are computed based on a proportional co-variation assumption. The Observation Matrix 𝑂𝑂 is represented as 𝑛𝑛 × 𝑛𝑛 diagonal matrices 𝑂𝑂 𝑡𝑡whose vth diagonal entry is 𝑃𝑃(𝑦𝑦𝑒𝑒𝑡𝑡|𝑥𝑥𝑣𝑣𝑡𝑡) for each state 𝑣𝑣 and whose other entries are 0 at the time 𝑡𝑡 of the observation sequence.

The computational complexity of our method is linear time, 𝑂𝑂(𝑙𝑙), where 𝑙𝑙 is the length of the observation sequence, whereas the Coefficient-Matrix-Fill procedure is quadratic time, 𝑂𝑂(𝑙𝑙2). In our method, the sensitivity function coefficients are computed in 𝑜𝑜𝑛𝑛𝑜𝑜 𝑓𝑓𝑜𝑜𝑓𝑓 𝑙𝑙𝑜𝑜𝑜𝑜𝑙𝑙 that runs for the length of the observation sequence, whereas in the Coefficient-Matrix-Fill procedure 𝑡𝑡𝑡𝑡𝑜𝑜 𝑛𝑛𝑜𝑜𝑛𝑛𝑡𝑡𝑜𝑜𝑛𝑛 𝑓𝑓𝑜𝑜𝑓𝑓 𝑙𝑙𝑜𝑜𝑜𝑜𝑙𝑙𝑛𝑛 are used. It computes the sensitivity coefficients in the forward matrices element-by-element in the 𝑖𝑖𝑛𝑛𝑛𝑛𝑜𝑜𝑓𝑓 𝑙𝑙𝑜𝑜𝑜𝑜𝑙𝑙, which runs in increasing time up to the maximum of the length of the observation sequence, and the 𝑜𝑜𝑜𝑜𝑡𝑡𝑜𝑜𝑓𝑓 𝑙𝑙𝑜𝑜𝑜𝑜𝑙𝑙 runs for the length of the observation sequence.

Figure 4: Time in seconds to compute the sensitivity coefficients for an observation sequence length from 1 to 1000 with a step size of 10.

This research has shown that it is more efficient to compute the coefficients for the HMM sensitivity function directly from the HMM representation. The proposed method exploits the simplified matrix formulation for HMMs. A simple algorithm is presented which computes the coefficients for the sensitivity function of filtering and smoothing probabilities for transition, initial and observation parameter variation for all hidden states, as well as all time steps. This method differs from the other approaches in that it neither depends on a specific computational architecture nor requires a Bayesian network representation of the HMM.

23

The future extension of this work will include sensitivity analysis of predicted future observations 𝑙𝑙(𝑦𝑦𝑒𝑒𝑡𝑡|𝑦𝑦𝑒𝑒1:𝑇𝑇)(𝜃𝜃). 𝑡𝑡 > 𝑇𝑇 , and the most probable explanation for the corresponding parameter variations. Future research on the sensitivity analysis of HMM, where different types of model parameters are varied simultaneously, as well as the case of continuous observations will be considered.

References

[8] S. Renooij, "Efficient sensitivity analysis in hidden Markov models," International Journal

of Approximate Reasoning, vol. 53, no. 9, pp. 1397-1414, 2012.

[9] S. B. Amsalu , A. Homaifar, A. Karimoddini and A. Kurt, "Driver intention estimation via discrete hidden Markov model," in IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, Canada , 2017.

[10] S. B. Amsalu, A. Homaifar and A. C. Esterline, "A Simplified Matrix Formulation for Sensitivity Analysis of Hidden Markov Models," Algorithms , vol. 10, no. 3, p. 97, 2017.

[11] A. Seifemichael B., "Driver Behavior Modeling and Hidden Markov Models Sensitivity Analysis," Available from ProQuest Dissertations & Theses Global., 2017.

4. Novel kinematic-based Framework for Autonomous Ground Vehicle

Trajectory Planning Using the Variational Approach Trajectory planning is seen as a key component for the emerging autonomous vehicles as it

provides safe maneuver and dynamically feasible trajectory for the vehicle's control system. Depending on the application, the trajectory planning algorithms are generally divided into two main categories. First, the kinematic based algorithms are developed for the low speed scenarios where the wheel-road slippage is not an important factor. On the other hand, the dynamic models are capable of accounting for more factors, such as side-slip angle, tire forces, and moment of inertia, which makes them especially suitable for the high speed and harsh road conditions. While the dynamic models are more general in nature, they demand for concurrent estimation of many unknown and random variables, such as wheels’ friction, stiffness, and mass. Consequently, they suffer from high computational complexity and impose significant overhead to the autonomous vehicle control system.

To take advantage of the generality of the dynamic-based algorithms as well as the simplicity and efficiency of the kinematic based models, combining these two approaches in a cascade structure has recently attracted considerable attention in the literature. The cascade structure, as shown in Fig. 1, illustrates a general form of such a combination, where a kinematically feasible trajectory planning module is embedded in the second level, and a dynamic-based controller tracks the trajectory in the third level.

24

Figure 5: Cascade motion planning and control scheme.

The goal in this research was to propose a novel trajectory planning framework: 1. minimizing the trajectory tracking error (position, velocity, and acceleration) 2. transforming motion constraints into the form of vehicle configuration (states)

limitations (road boundaries, limitations on velocity, etc.) 3. considering the limitations on control inputs of a vehicle (steering angle,

acceleration) 4. not presuming linearity of the trajectory planner and not restricting the input space

and trajectory to any certain parametric class of functions, e.g. Bezier curves, splines, and polynomials which indeed avoid the sub-optimal solution.

We started with defining a novel problem formulation which minimizes the tracking error subject to the car-like vehicle model as shown in Fig. 6.

Figure 6: Car-like vehicle model.

25

This problem is formulated as

In the cost function, the terms 𝑜𝑜𝑣𝑣𝑒𝑒𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑣𝑡𝑡𝑣𝑣2 and 𝑜𝑜𝑎𝑎𝑣𝑣𝑣𝑣𝑒𝑒𝑣𝑣𝑒𝑒𝑎𝑎𝑎𝑎𝑡𝑡𝑣𝑣𝑣𝑣𝑎𝑎2 guarantee the continuity of the

velocity and the acceleration by tracking the first and the second derivatives of reference trajectory. Moreover, it allows us to extract the optimal control 𝑜𝑜∗(𝑡𝑡) since the control inputs are appeared explicitly in the cost function. We assumed that the reference trajectory is given by a higher planning module, which is called path planner in the literature. The variational approach is used to solve the optimization and the corresponding optimal control law is obtained. The optimal trajectory and control inputs are found by solving a set of two-point boundary value (TPBV) nonlinear differential equations numerically. The proposed framework can track any arbitrary reference with continuous acceleration profile, which is shown in the simulation part. More detailed information about the problem formulation can be found in [6]. At the end, we evaluated the performance of the proposed framework in two scenarios of lane changing and multi-curvature road which verified the success of this framework.

We expanded our trajectory planning framework by consideration of mechanical and physical constraints of the vehicle, and boundaries of the road in the form of state and control inequality constraints. Thus, the problem formulation expanded as

Fig. 3, 4, and 5 illustrate the performance of the proposed framework in a multi-curve road

scenario, and the constraint and unconstraint planning techniques are also compared.

Figure 7: Optimal vehicle trajectory.

0 5 10 15 202

4

6

8

10

Vehicle constrained trajectory

Reference trajectory

Vehicle unconstrained trajectory

26

Figure 8: Multi-curve road scenario: (a) heading angle: 𝜃𝜃∗(𝑡𝑡) and 𝜃𝜃𝑎𝑎(𝑡𝑡), (b) velocity: 𝑣𝑣∗(𝑡𝑡) and 𝑣𝑣𝑎𝑎(𝑡𝑡).

Fig. 3 illustrates the vehicle trajectory, which satisfies the boundary conditions of 𝑥𝑥(𝑡𝑡) and 𝑦𝑦(𝑡𝑡). It can be seen that the tracking error converges to zero after 10 seconds. The transient shows that the vehicle cannot instantly track the reference trajectory due to the error (initial velocity and heading angle errors) between the initial configuration of the vehicle and the reference trajectory. Moreover, the tracking error takes longer to become zero compared to the case that there is no constraint on states and control inputs.

In Fig. 4, heading angle and velocity are shown. As demonstrated in Fig. 4 (b), velocity satisfies the constraints. Fig. 6 illustrates the optimal control inputs, steering angle and acceleration. As it can be seen in Fig. 6, the controller selects the lower bound as the optimal point in some in-stances.

Figure 9: Optimal control inputs (multi-curve road scenario): (a) steering angle: 𝜙𝜙∗(𝑡𝑡) 𝑎𝑎𝑛𝑛𝑛𝑛 𝜙𝜙𝑎𝑎(𝑡𝑡), (b) acceleration: 𝑎𝑎∗(𝑡𝑡)

and 𝑎𝑎𝑎𝑎(𝑡𝑡).

0 5 10 15 20-1

0

1

0 5 10 15 20

2

4

6

3

0 5 10 15 20-2

-1

0

1

0 5 10 15 20

-4

-2

0

27

In our latest work, we obtained the analytical solution for the unconstrained problem by applying the input-output linearization technique. The results show more than 380% cost function value reduction in analytical case ( [6], [7], and [8]).

Figure 10: Comparison of analytical and numerical methods.

References

[12] K. Majd, M. Razeghi-Jahromi and A. Homaifar, "Optimal Kinematic-based Trajectory

Planning and Tracking Control of Autonomous Ground Vehicle Using the Variational Approach," in Intelligent Vehicles (IV) Symposium, Changshu, Suzhou, China, 2018. (Submitted)

[13] K. Majd, M. Razeghi-Jahromi, S. Ramyar and A. Homaifar, "Model-based autonomous ground vehicle trajectory optimization and tracking using the Variational approach," in American Control Conference (ACC), 2019. (Submitted)

[14] K. Majd, M. Razeghi-Jahromi and A. Homaifar, "Closed-form Solution for Rear-wheel Car-like Robot Minimum-error Trajectory Tracking and Optimization Using the Variational Approach," IET Control Theory & Applications, 2018.

5. A Collision Avoidance System with Fuzzy Danger Level Detection

In this study, [15], a situation assessment (SA) algorithm is proposed which estimates the driver’s behavior and then interacts with the collision avoidance (CA) system to initiate earlier brake interventions when there is a threat.

The goal is to develop a fuzzy danger-level detection system based on naturalistic driving data to perform risk assessment in lane-change scenarios, as well as implement a collision avoidance system to modify the driving performance and reduce the level of danger. We are studying driving data based upon 100 cars to identify common features in different near-crash/ crash scenarios. There are external factors such as slow or fast surrounding vehicles, unexpected exits or dark roads, as well as internal factors such as a distracted or drowsy driver contributing to a pre-crash event.

0 0.5 1 1.5 2 2.50

50

100

150

200

250

300

350

400

28

To ensure the safety of the vehicle in near-crash or crash scenarios, we propose a collision avoidance system for safe trajectory planning of lane change events in our study. This collision avoidance system modifies the lane change trajectory according to the danger level of the event. A fuzzy danger level detection system is designed using naturalistic near-crash events to determine a realistic risk level of a dangerous scenario. The training data is derived from the 100 car naturalistic driving data. The data is pre-processed, and the samples with missing or invalid data are removed. We chose two groups of sample data, as described by one of the features as “fault belong.” The driving behavior of the ego vehicles and surrounding vehicles are extracted, then statistic results are used to develop the structure of the fuzzy danger level detection model. The model is using to produce a danger level basis for the crash avoidance system by using the measurements of the ego vehicles: velocity, yaw rate, accelerations in lateral and longitudinal, also the surrounding vehicles’ relative distance and velocity. Afterward, as the figure show, a model predictive controller will adjust the safety-gap (SG) constrain and generates safe longitudinal and lateral trajectories for the lane change maneuver based on the computed danger level.

Moreover, an additional feature in the proposed collision avoidance system is a fault determination classifier, which is trained by the naturalistic data to determines whether the subject vehicle or the surrounding vehicle is responsible for the near-crash/crash event. In this work, the problem is formulated as a classification task, and an Extreme Gradient Boosting algorithm is used for identifying the responsible driver who is at fault. The classifier is tested, and the results show that the technique can provide an additional feature as a fault belonging with high accuracy. The results of this classifier also can help to adjust the performance of the proposed collision avoidance system.

Figure 11: Schematic diagram of the proposed system

The system shows the ability to evaluate the driving behavior of the subject vehicle and provide a reasonable estimation of the danger of the maneuver, and the ability to plan the collision-free trajectory according to the percentage of the danger-level to ensure the safety of the subject vehicle. The ability to evaluate the current driving scenario also could be used for monitoring for the operating company or the third party. One of the advantages of this data-driven model is its adaptive nature feeding the newer naturistic driving, which collected as time goes to fit the personalized needs of the subject vehicles. In the future study, the more complicated driving maneuver will be used in the study to enhance the performance of the proposed system.

29

References

[15] Z. Wang, S. Ramyar, S. M. Salaken, A. Homaifar and S. Nahavandi, "A Collision

Avoidance System with Fuzzy Danger Level Detection," in Intelligent Vehicles Symposium (IV), 2017.