# Markov Switching Model for Driver Behavior Prediction: Use cases on
Smartphones

Ahmed B. Zaky
Faculty of Engineering (Shoubra), Benha University, Cairo 11240, Egypt
Big Data Institute, College of Computer Science and Software Engineering
Shenzhen University, Shenzhen 518060, Guangdong, China
<EMAIL_ADDRESS>
<EMAIL_ADDRESS>
&Mohamed A. Khamis
Cyber-Physical Systems Lab
Egypt-Japan University of Science and Technology (E-JUST)
New Borg El-Arab City, Postal Code 21934, Alexandria, Egypt
<EMAIL_ADDRESS>
Walid Gomaa
Cyber-Physical Systems Lab
Egypt-Japan University of Science and Technology (E-JUST)
New Borg El-Arab City, Postal Code 21934, Alexandria, Egypt
Faculty of Engineering, Alexandria University, Alexandria 21544, Egypt
<EMAIL_ADDRESS>

###### Abstract

Several intelligent transportation systems focus on studying the various
driver behaviors for numerous objectives. This includes the ability to analyze
driver actions, sensitivity, distraction, and response time. As the data
collection is one of the major concerns for learning and validating different
driving situations, we present a driver behavior switching model validated by
a low-cost data collection solution using smartphones. The proposed model is
validated using a real dataset to predict the driver behavior in short
duration periods. A literature survey on motion detection (specifically
driving behavior detection using smartphones) is presented. Multiple Markov
Switching Variable Auto-Regression (MSVAR) models are implemented to achieve a
sophisticated fitting with the collected driver behavior data. This yields
more accurate predictions not only for driver behavior but also for the entire
driving situation. The performance of the presented models together with a
suitable model selection criteria is also presented. The proposed driver
behavior prediction framework can potentially be used in accident prediction
and driver safety systems.

_K_ eywords Driver Behavior $\cdot$ Markov Switching Model $\cdot$ Auto-
Regression Model

##  1 Introduction

Models for collision avoidance systems, congestion assistants, crash
prediction software, etc. have been developed to support a vehicle driver
while doing complicated tasks. Accordingly, these systems increase the safety
limit for both the drivers and pedestrians. Fully automated autonomous driving
systems still need more attention in building models for different driving
assistance tasks. Developing a driving behavior model that can be adapted to
different driving situations and be able to cover most driving behaviors, is
still a challenging task.

Machine learning (ML) is one of the fastest growing areas of science. It has
been used in many applications; e.g., traffic signal control Khamis and Gomaa
(2014); Khamis et al. (2012), classification, and recognition. ML techniques
such as regression models Angkititrakul et al. (2013), neural networks (NNs)
Panwai and Dia (2007), and fuzzy systems Ma (2006) have been used recently in
modeling patterns of driving situations. However, such models face the
complication of understanding different driving situations (especially the
unexpected ones). Driving tasks can be segmented into driving regimes mapped
to different driving situations with different response for each driver. A
driver usually switches between different behaviors such as car following,
lane changing, mobile messaging, sign reading, etc. It is normal to see a
driver perform more than one task at the same time, e.g., following a car
while switching radio channels or messaging, etc. In this article, we propose
a stochastic model that is suitable for detecting and classifying different
driving regimes.

Preliminary results of the work presented in this paper have been published in
Zaky and Gomaa (2014); Zaky et al. (2015). In this paper, the Expectation
Maximization (EM) and Markov Chain Monte Carlo (MCMC) are used for estimating
the proposed model parameters. Moreover, we calibrate the model for car
following driver behavior using our own data set collected by smartphones (as
a low cost solution for collecting driving data) plus a naturalistic driving
data set presented in Manstetten et al. (1997). We also present a brief survey
for different machine learning models employed for driver behavior and data
collection based on smartphones.

The rest part of this paper is organized as follows. Section 2 presents the
state of the art literature review. Specifically, we focus on motion detection
with use cases on driving behavior detection using smartphones. Also, this
section provides the necessary background for the work presented in this
paper. This includes detailed description for driver behavior models Section 3
introduces Markov Switching Vector Auto-Regressive model and Bayesian Gipps
sampling for model parameters estimation. Section 4 depicts the data
collection process using smartphones, the car following dataset, and the
adopted driving behavior model. Section 5 presents the results of using both
of the data collected using smartphones and the naturalistic driving data.
Finally, Section 6 concludes the work presented in this article and provides
directions for future research.

##  2 Related Work

###  2.1 Driver behavior models

Recently, machine learning approaches have been proposed for driver behavior
modeling. Car following is the most popular behavior for evaluating these
approaches. Three models are mainly used: Hidden Markov Models (HMM), Gaussian
Mixture Models (GMM), and Piece-Wise Auto Regressive Exogenous models (PWARX).
These models have achieved remarkable results in simulating driving scenarios.
Additionally, these models divide each complex driving pattern into sub-
patterns using mixture components. Models introduce different methods for
calculating the latent classes, the relationship between the observed
variables and each class, the estimation of class parameters, and the number
of latent classes.

####  2.1.1 Hidden Markov models

HMM has been used for driver behavior modeling in different situations such as
the model implemented in Dapzol (2005). This model uses sensor data evolution
to predict the real current driving situation. The results achieved a
prediction accuracy of 80% of driver behavior recognition from the initial
driver movements. In Ikeda et al. (2011), the authors presented a collision
warning system based on HMM. Traffic models based on HMM have been reviewed in
Dapzol (2005) and Sathyanarayana et al. (2008).

####  2.1.2 Gaussian mixture models

A stochastic driver behavior modeling framework based on GMM is presented in
Angkititrakul et al. (2013). The model calculates the joint probability
distribution for a number of driving signals (following distance, vehicle
velocity, brake and gas pedal forces, and vehicle dynamics). The model
implements two GMMs as a representation of gas and brake pedals, and their
relation with the follower velocity and the gap distance. A main issue with
the GMM is the selection of the model components, the authors proposed the use
of a Dirichlet process as a non-parametric Bayesian approach that selects the
optimal number of model components. The model fits the driving observations
for each driver. In addition, a general driver model is implemented based on
fitting the observations of several drivers. The authors used different
mixtures (4, 8, 16, and 32) for evaluating the model performance.

####  2.1.3 Piece-wise auto-regressive exogenous models (PWARX)

PWARX have been presented in Sekizawa et al. (2007) and Okuda et al. (2009) to
model human driving behavior as a Hybrid Dynamical System (HDS). The proposed
approach is switching between simple linear behavior models instead of
modeling a non-linear complex model. The driver behavior recognition model
introduced in Sekizawa et al. (2007) is a standard HMM extended by embedding
an auto-regressive exogenous model (ARX) in each discrete state. The authors
introduced a simulation of a collision avoidance system.

In Akita et al. (2008), a car following model classification approach has been
introduced based on PWARX as a segmentation approach and a $K$-means
clustering for the input vector. The classification between modes is done
using a Support Vector Machine (SVM). The PWARX models have impressive results
in modeling driver behavior. However, the PWARX models have two problems
Takeda (2016). First, the model can not classify and estimate the behavior
simultaneously. Second, it is unable to handle a probabilistic time varying
data. The Probability weighted Auto-Regressive model (PrARX) proposed in Okuda
et al. (2013) is an extension to PWARX. The PrARX model addresses these two
issues by composing multiple ARX models by a probabilistic weighting function.

The PWARX and GMM models are compared in Takeda (2016) for car following
behavior modelling. These models predict a pattern of brake and gas pedal
response of the driver as a feedback to the current vehicle velocity and gap
distance between vehicles. The results showed that the PWARX approach
outperforms GMM in all cases. The prediction of gas pedal behavior was better
than the prediction of brake pedal in both models. The results also showed
that the operation of gas pedal was smoother than that of the brake pedal. In
addition, the gas pedal took longer control time than the pulse-wise shorter
period brake pedal.

###  2.2 Driving behavior detection using smartphones

Motion detection in traffic networks (e.g., anomaly detection, car following
driver behavior, etc.) has gained much concern in the last few years. In Zaky
and Gomaa (2014), the authors proposed a Markov regime switching-based model
to estimate the driver behavior and extract different driving regimes. The
proposed model analyzes a sequence of observations of driving time series
data. Trajectory data such as velocity, acceleration, and space gap between
the leader and follower drivers were used in model learning. The results
showed that by using real car following data sets, the model was able to
classify normal car following driving behavior, rare events, and short-time
events. In addition, the system was able to determine the switching dynamics
among different regimes by applying maximum likelihood estimates and Hamilton
filter. Moreover, the proposed model can infer regime specific characteristics
such as expected duration, the probability of transferring from one regime to
another, switching parameters and driving patterns.

In Ahmed et al. (2007), the authors use two networks; road traffic network and
IP network. They recorded images by six cameras over a period of four days.
The discrete wavelet transform (DWT) algorithm is used to process the images.
The DWT is known for its ability to extract spatially localized frequency
information. The authors perform 2D DWT on every image and average the energy
of transformation coefficients within each sub-band; a sudden change in the
power in the frequency content of the vector of sub-band intensities may
represent an anomaly.

The IP network data constitutes a time-series of entropy of four main packet
header fields (source IP address, destination IP address, source port number,
and destination port number) in each of 11 $\times$ 11 backbone flows in every
time step. Anomalies represent changes in the distributions of packets. The
authors use two anomaly detection algorithms: One-Class Neighbor Machine
(OCNM) and Kernel-based Online Anomaly Detection (KOAD). OCNM uses the kth
nearest neighbor Euclidean distance as a sparsity measure. KOAD is associated
with a kernel function where the features corresponding to the normal traffic
measurements are clustered. The region of normality is represented using a
relatively small dictionary of approximately linearly independent elements.

In Srivastava et al. (2011), the authors proposed a method to detect anomalies
in the trajectory of a vehicle by observing the patterns in its velocity. The
authors proposed a look-up table for mapping the trajectory from image co-
ordinates to a hypothetical coordinate system in which the axes are selected
with respect to the road. In this co-ordinate system, the (axial) velocity is
a one dimensional quantity. This is a spatial representation of the velocity
which helps localizing the normal behavior definition. The authors estimate
the normal modes in the velocity after scaling the velocities by the average
speed of the vehicle. The authors detect deviations from the normal velocity
for individual sections of the roadway where the normal velocity was modeled
using a mixture of Gaussians (obtained by training). The authors use the shape
of the trajectory to determine turns and other significant maneuvers, an
anomaly is also detected if the velocity of a vehicle falls in a path model
which is inconsistent with the shape of its trajectory. The authors
characterize the shape of the trajectory using template matching in order to
detect turns and other maneuvers. Template matching was performed using
sliding windows which does not require pre-segmentation of the trajectory.
Thus, it is very suitable for real-time operation.

In Johnson and Trivedi (2011), the authors proposed a novel system that uses
dynamic time warping (DTW) and smart phone sensors (accelerometer, gyroscope,
magnetometer, GPS, and video) in order to detect and record driving style
activities. The proposed system gathers related inter-axial data from multiple
sensors into a single classifier. The proposed system utilizes the Euler
representation of device attitude (based on the gathered data) to aid the
driving activity classification. All the proposed processing is done entirely
on the smart phone.

##  3 Methodology

In this section, we introduce the proposed model framework including model
formula, parameter estimation, and characteristics.

###  3.1 Markov switching vector auto-regressive model

MSVAR Krolzig (2013) is non-linear model which joins the vector auto-
regressive models with hidden Markov chain models. The MSVAR model builds a
non-linear data model as piece-wise linear model; this is achieved by modeling
the process to be linear in each regime. The main objective of such a model is
to find the specification of each regime using variables as switches between
each regime. Such models use intercept, mean, or both used as switches. For
instance, the model presented in Hamilton (1989) utilizes MSVAR with mean
switches to study business cycles using the U.S. GDP series. The major
difficulty of using the mean as a switching parameter is the estimation of the
switching parameters due to their interrelation and the latent variable. On
the other side, models that use the intercept as switches require less effort
in estimation, it can be estimated using Monte Carlo methods.

The model introduced is based on multivariate time-series
$Y=(y_{1},\dots,y_{t})$ consisting of $t$ observations, where $y_{t}$
represents an N-dimensional vector and a stochastic process that depends on a
latent discrete stochastic process, $S_{t}$, having discrete state-space with
state variable $s_{t}$ which indicates the dominant regime at time $t$. The
reduced form of the model is presented in Krolzig (2013) and is known by
MSIAH-VAR(p) as in (1). The reduced model uses three types of switches:
intercept $A_{s_{t}}^{(0)}$, regression coefficient $A_{s_{t}}^{(i)}$, and co-
variance matrix $U_{t}$.

$y_{t}=A_{s_{t}}^{(0)}+\sum\limits_{s=1}^{p}A_{s_{t}}^{(i)}y_{t-i}+U_{t},$ (1)

$\hskip 28.45274pt\text{where}\hskip 28.45274ptU_{t}|s(t)~{}\text{i.s}\
N(0,\sum\limits_{s_{t}})$.

The state variable $s_{t}$ is evolved over time as a discrete time, discrete
space Markov process, assuming $s_{t}=1,2,\dots k$ for $k$ regimes. Let $M$
represents the number of appropriate regimes, so that $s_{t}\in{1,...,M}$. The
conditional probability density of the observed time series vector $y_{t}$ is
given by (2) where $\theta_{M}$ is the VAR model parameter vector for regime
$M$ and $Y_{t-1}$ are the observations from time $T=0$ to time $T=t-1$.

$p(y_{t}|Y_{t-1},s_{t})=\begin{cases}f(y_{t}|Y_{t-1},\theta_{1})\hskip
28.45274pt\text{if}\ s_{t}=1,\\\ \cdots\\\ f(y_{t}|Y_{t-1},\theta_{M})\hskip
28.45274pt\text{if}\ s_{t}=M.\end{cases}$ (2)

The stochastic transition of states is determined by a Markov transition
matrix $p$ which determines the dynamics of the switching process where
$p_{i,j}$ = $Pr(s_{t}=i|s_{t-1}=j)$ is the probability of switching from state
$j$ to state $i$ and $\sum\limits_{j=1}^{M}p_{i,j}=1$.

The model parameter state vector is defined as follows:

$\begin{split}\theta=\\{A_{1}^{(0)},A_{2}^{(0)},..A_{M}^{(0)},A_{1}^{(1)},A_{2}^{(1)}..A_{M}^{(1)},\\\
U_{1},U_{2},..,U_{M},p_{i,j}\\}.\end{split}$ (3)

This model parameter state vector can be estimated by maximum likelihood as in
Eq. (4). The estimation process is performed through the Expectation
Maximization (EM) algorithm that is presented in Hamilton and Susmel (1994).
EM iteratively calculates the next step of the state vector $\theta_{t+1|t}$
given the previous observation and the previous state vector using the log-
likelihood function of the data. The algorithm proceeds in two steps:
expectation and maximization steps. The expectation step uses the parameter
estimated from the previous maximization step to compute both filtered
probability vectors.

The likelihood is proportional to the probability of observing the data given
the estimated parameter. The minimization of the log-likelihood in Eq. (4) can
be used as an objective for parameter estimation and for comparing the
different model fitting schemes.

$\ln
L=\sum\limits_{t=1}^{T}\ln\sum\limits_{j=1}^{K}P(y_{t}|s_{t}=j,\theta).P(s_{t}=j).$
(4)

There are three main terminologies for gaining information about different
driving regimes: driving regime inference, regime classification, and regime
expected duration.

####  3.1.1 Regime inference

The objective of the regime inference process is the identification of the
latent regime variable $s_{t}$ from the observations $x_{t}$. This process
requires two main steps: filtering and smoothing.

Filtered probability estimates The filter step aims at estimating $\xi_{jt}$
which represents the probability of the unobserved state vector $s_{t}$. The
probability of being under regime $j$ at time $t$ given the model parameters
is given by Eq. (5):

$\xi_{jt}=P(s_{t}=j|\Omega_{t};\theta).$ (5)

where $\Omega_{t}$ is the sequence of observations over time given by:

$\Omega_{t}=\\{y_{t},y_{t-1},...,y_{2},y_{1}\\}$.

and $\theta$ is the population parameter vector which is given by:

$\theta=\\{\sigma_{1},\sigma_{2},...,\sigma_{k},c_{1},c_{2}...,c_{k},\phi_{1},\phi_{2},...,\phi_{k},p_{i,j}\\}$.

with the constraints that $\xi_{jt}>0$ and $\sum\limits_{\forall
j}\xi_{jt}=1$.

Smoothing The filter step will generate estimates for state $s_{t}$ where
$t=1....,T$ using observations up to time $t$. The smoothing task improves
regime inference by taking into consideration the future observation $t+1<T$.
The smoothed probability is defined as $P(s_{t}|y_{t+1,...,T})$. The smoothed
algorithm is a backward filter that starts from the last observation point
$t=T$. The smoothing algorithm starts by calculating the smoothed probability
of the last observation point $P(s_{t}|y_{T})$, and then iterates backward to
$t=1$; the algorithm steps are as follows:

  1. I

The smoothed probability of the last observation point:

$\displaystyle P(s_{t}|y_{T})$
$\displaystyle=\sum\limits_{s_{t+1}}P(s_{t},s_{t+1}|y_{T})$
$\displaystyle=\sum\limits_{s_{t+1}}P(s_{t}|s_{t+1},y_{T})\times
P(s_{t+1}|y_{T}).$

  2. II

According to the Markovian assumption, the $s_{t+1}$ depends only on $s_{t}$:

$\displaystyle P(s_{t}|s_{t+1},y_{T})$
$\displaystyle=P(s_{t}|s_{t+1},y_{t},y_{t+1,...,T})$
$\displaystyle=\frac{P(y_{t+1,...,T}|s_{t},s_{t+1},y_{T})\times
P(s_{t}|s_{t+1},y_{t})}{P(y_{t+1,...,T}|s_{t+1},y_{t})}$
$\displaystyle=P(s_{t}|s_{t+1},y_{t}).$

  3. III

The calculation of the smoothed probability $P(s_{t}|y_{T})$ is done by using
the last term of the previous iteration $P(s_{t+1}|y_{T})$ where

$\displaystyle P(s_{t}|s_{t+1},y_{T})$
$\displaystyle=\frac{P(s_{t+1}|s_{t},y_{t})\times
P(s_{t}|y_{t})}{P(s_{t+1}|y_{t})}$ $\displaystyle=\frac{P(s_{t+1}|s_{t})\times
P(s_{t}|y_{t})}{P(s_{t+1}|y_{t})}.$

  4. IV

The recursion is initialized with the final filtered probability vector
$P(s_{t}|y_{T})$. The following equation shows how the future observation
$y_{t+1,...,T}$ improves the inference of the unobserved state $s_{t}$.

$\displaystyle P$
$\displaystyle(s_{t}|y_{T})=\sum\limits_{s_{t+1}=1}^{M}\frac{P(s_{t+1}|s_{t})\times
P(s_{t}|y_{t})}{P(s_{t+1}|y_{T})}\times P(s_{t+1}|y_{T}).$

####  3.1.2 Regimes classification

The classification of regimes begins by assigning each observation $y_{t}$ to
a regime $S$. The classification is achieved by mapping each observation to
the winning regime with the highest smoothed probability as in Eq. (6).

$\hat{s_{t}}=\operatorname{arg\,max}_{1,...M}P(s_{t}=m|y_{T}).$ (6)

The smoothed regime probabilities are calculated using the dataset. Then each
observation is assigned to the highest regime filtered probability.

####  3.1.3 Regime expected duration

The expected length of stay in a specific regime (state $i$) can be derived
from the regime transition matrix; this is achieved by using the probability
of staying in the same regime $P_{ii}$. Let $D_{i}$ be the time period in
which the system stays at regime $i$. Equation (7) is the probability to stay
time period $k$ in regime $i$. The expected duration can be specified by the
formula presented in Eq. (8), and according to the formula the expected
duration depends only on the transition probability $P_{ii}$ of the same
regime, so the expected duration remains constant over time and the higher the
transition probability, the longer the stay for the regime.

$P(D_{i}=k)=P_{ii}^{k-1}(1-P_{ii}).$ (7)

$E(D_{i})=\sum\limits_{k=0}^{\infty}(k\times P(D_{i}=k))=\frac{1}{1-P_{ii}}.$
(8)

####  3.1.4 Gipps bayesian parameter estimation

The estimation of the MSVAR model is a difficult task. Parameter estimation
can be done easily for Bayesian models having known closed form posterior
distributions. The main objective of our proposed model is to adopt the MSVAR
in driver behavior modeling where we cannot determine the model parameters of
prior distributions directly. MCMC can be used for finding the posterior
distribution for the model parameters. This is attributed to the ability of
MCMC to generate samples from the posterior distribution. The estimation of
the proposed MSVAR model has been implemented based on Gipps sampler for
posterior distribution sampling.

The Gipps sampler is a popular and efficient MCMC sampling algorithm Bay
(2013). The Gipps sampler is like the Metropolis Hastings (MH) component-wise
implementation in sampling each dimension. However, instead of sampling each
dimension from an independent proposed distribution, Gipps samples from
variable full conditional distribution
$P(y_{j}|y_{-j})=P(y_{j}|y_{1},y_{2},\dots,y_{j-1},y_{j+1},\dots,y_{n})$. The
algorithm accepts all drawn samples, thus Gipps has lower computation
requirements and converges faster Gipps (1981). Like the component-wise
implementation, the Gipps algorithm step-samples through each variable while
the other variables are fixed.

If the target conditional distribution is belonging to standard distributions,
then the sampling can directly be done from these distributions, otherwise
Metropolis–Hastings algorithm can be used for sampling the target
distribution. The Bayesian parameter estimation approach assumes that both of
the regime $S$ and the model parameters $\theta$ are random variables. The
Gipps sampler has been used in the inference of the state-space models, even
it has been used for classifying the states without estimating the model
parameters. The Gipps sampler can be used for sampling the parameters of the
posterior distributions.

The sampler draws samples from the latent states and samples the model
parameters from the full conditional distribution. The sampler starts with
sampling $\theta^{i}$ from $p(\theta|S^{i-1},\Omega_{T})$ where $\Omega_{T}$
is the observed data, then sampling $S^{i}$ from $p(S|\theta^{i},\Omega_{T})$.
The prior specification of the state-space sampling for known number of states
is followed (as presented in Richardson and Green (1997)) where the standard
distribution families selected for model parameters are implemented and the
model parameters are conditionally independent. The parameters prior
distribution are estimated as follows:

  * •

The joint transition probabilities use independent Dirichlet distribution
prior $Dir(1,1,1,\dots,1)$ for each state.

  * •

Each regression coefficient mean $\mu_{i}$ has independent Gaussian prior.

  * •

Each regression coefficient standard deviation has gamma prior.

The presented prior specification has full state conditional distribution
which follows a Dirichlet distribution as in Eq. (9) where $I\\{.\\}$ is an
indicator function for indicating the current state.

$\begin{split}p(s_{1,..,k}|\theta,\Omega)\backsim Dir(I\\{s_{1}=1\\}+1,\hskip
56.9055pt\\\ I\\{s_{2}=1\\}+1,\dots,I\\{s_{k}=1\\}+1).\end{split}$ (9)

The joint transition probabilities $p_{ij}$ for each state has full
conditional distribution which follows a Dirichlet distribution as in Eq. (10)
where $n_{ij}$ is the number of transitions from state $i$ to state $j$.

$p(p_{1,i,...,k}|\theta,\Omega,s)\backsim
Dir(n_{11}+1,n_{12}+1,\dots,n_{1k}+1).$ (10)

The Gipps sampler iterates on two steps. Step (a) for updating the parameters
and Step (b) for Markov chain revised as follows:

  * •

Step a

    * –

Update the mean by sampling from the Gaussian prior.

    * –

Update the standard deviation gamma prior.

    * –

Update the transition probabilities for each state independently by sampling
from the proposed distribution in Eq. (10).

    * –

Update the $p(s_{1,..,k}|\theta,\Omega)$ from the proposed distribution in Eq.
(9).

  * •

Step b

    * –

Update the filtered probability $P(s_{t}=j|\Omega,\theta)$.

    * –

Update the transition probabilities
$P(s_{t}=j|s_{t-1}=i)$.

##  4 Experimentation

###  4.1 Car following data set

The Robert Bosch GmbH Research Group Manstetten et al. (1997) floating car
dataset (FCD) is used to validate our model. This dataset represents a car
following behavior of vehicle speed under stop-and-go traffic conditions
during an afternoon peak on a single lane in Stuttgart, Germany. A car with a
frontal radar sensor based on a Doppler ultrasound is used to measure the
relative speed and distance between a leader and a follower drivers. The used
datasets are sampled at 100 ms with duration of 250, 400, and 300 seconds.
Data set 1 gap distance, speed, and acceleration are shown in Fig. 1.

Figure 1: Dataset_1 gap distance, velocity difference, and acceleration.

These datasets have complex situations in daily urban traffic with lots of
acceleration and deceleration periods. Due to the existence of traffic lights
in the recorded scenario, there are some standstill periods. The velocity
varies in the range between 0 km/h and 60 km/h. These datasets are used in
modeling, evaluating, and calibrating car following models such as the
Intelligent Driver Model (IDM) in Treiber and Kesting (2013), the neural
network models Panwai and Dia (2007), and the state-space models Hoogendoorn
et al. (2007).

###  4.2 Data collection

The driving data collection is a complex task. Most vehicle data collection
experiments consist of high quality recording of driver’s behavior. Lots of
sensors have to be equipped within the vehicle equipment in order to record
the various driver behavior signals. Sensors such as microphones, video
camera, steering wheel angle, gas pedal, brake pedal, GPS, speed,
acceleration, and heart rate can be used according to the objective behind the
study.

The collection of driving behavior with this procedure has a high cost. Thus,
a low cost solution using smartphones to collect car following behavior data
has been introduced. Sensor data from both follower and leader vehicles is
highly beneficial and accordingly used to fit the proposed model (using
smartphones iPhone 6 and 6 plus). We converted the GPS latitude and longitude
to the actual distance between the two vehicles considering the spherical
shape of the earth using the Haversine formula Nitin R.Chopde1 (2013).

The driving experiment has the following characteristics:

  * •

There are follower and leader drivers with predefined set of behaviors.

  * •

Every driver has one smartphone in his vehicle.

  * •

Every smartphone has SensorLog application running (an application for logging
sensors data).

  * •

All sensors are being logged; e.g., GPS, accelerometer, gyroscope, compass
(location heading).

  * •

Sampling rate of GPS is 1 Hz (1 sample per second).

  * •

Sampling rate of acceleromoter is 100 Hz.

  * •

We depend on GPS samples for localization. We obtain distances, velocities,
and accelerations from GPS readings. The accelerometer does not readily
provide acceleration data since it is relative to free fall.

  * •

Velocity is calculated by SensorLog (using 2 consecutive GPS readings).

  * •

Acceleration is calculated from the velocity difference.

  * •

We need to estimate, and accordingly classify, the follower velocity based on
follower acceleration, velocity difference, and gap distance.

###  4.3 Adopted driving behavior model

MSVAR models presented in Eq. (1) are adopted using driving signals that
represent the car following behaviors. The signals used are presented in Eq.
(11) where the observation vector consists of four observation signals
$y_{t}=[v_{t},a_{t},\Delta v_{t},h_{t}]$. These signals represent the car
following driver behavior where $v_{t}$ is the follower velocity, $a_{t}$ is
the follower acceleration, $\Delta v_{t}$ is the difference in velocity
between the leader and follower, and $h_{t}$ is the gap distance between the
leader and follower. The prediction of the model has $n$ interval forecasts
which can be evaluated using the conditional mean $\widehat{y}_{t+n}$ and the
mean square prediction error (MSPE).

The objective is to find the conditional density of $y_{t+n}$ given the model
parameters and the previous observation $\Omega_{t}$. The prediction density
given in Eq. (12) is a mixture of normals, and
$p(y_{t+n}|s_{t+n}=j,\Omega_{t})$ is the probability of each predicted regime.

${\begin{split}v_{t}=\begin{cases}A_{1,1}^{(0)}+A_{1,1}^{(1)}v_{t-1}+\cdots+A_{1,1}^{(p)}v_{t-p}+U_{1,1}\hskip
28.45274pt\text{if }s_{t}=1,\\\ ...\\\
A_{1,M}^{(0)}+A_{1,M}^{(1)}v_{t-1}+\cdots+A_{1,M}^{(p)}v_{t-p}+U_{1,M}\hskip
28.45274pt\text{if }s_{t}=M.\end{cases}\\\ \\\
a_{t}=\begin{cases}A_{2,1}^{(0)}+A_{2,1}^{(1)}a_{t-1}+\cdots+A_{2,1}^{(p)}a_{t-p}+U_{2,1}\hskip
28.45274pt\text{if }s_{t}=1,\\\ ...\\\
A_{2,M}^{(0)}+A_{2,M}^{(1)}a_{t-1}+\cdots+A_{2,M}^{(p)}a_{t-p}+U_{2,M}\hskip
28.45274pt\text{if }s_{t}=M.\end{cases}\\\ \\\ \Delta
v_{t}=\begin{cases}A_{3,1}^{(0)}+A_{3,1}^{(1)}\Delta
v_{t-1}+\cdots+A_{3,1}^{(p)}\Delta v_{t-p}+U_{3,1}\hskip 28.45274pt\text{if
}s_{t}=1,\\\ ...\\\ A_{3,M}^{(0)}+A_{3,M}^{(1)}\Delta
v_{t-1}+\cdots+A_{3,M}^{(p)}\Delta v_{t-p}+U_{3,M}\hskip 28.45274pt\text{if
}s_{t}=M.\end{cases}\\\ \\\
h_{t}=\begin{cases}A_{4,1}^{(0)}+A_{4,1}^{(1)}h_{t-1}+\cdots+A_{4,1}^{(p)}h_{t-p}+U_{4,1}\hskip
28.45274pt\text{if }s_{t}=1,\\\ ...\\\
A_{4,M}^{(0)}+A_{4,M}^{(1)}h_{t-1}+\cdots+A_{4,M}^{(p)}h_{t-p}+U_{4,M}\hskip
28.45274pt\text{if }s_{t}=M.\end{cases}\end{split}}$ (11)

$\begin{split}p(y_{t+n}|\Omega_{t})=\sum\limits_{j=1}^{M}Pr(s_{t+n}=j)|\Omega_{t})\\\
\times p(y_{t+n}|s_{t+n}=j,\Omega_{t}).\\\ \end{split}$ (12)

$\begin{split}Pr(s_{t+n}=j|\Omega_{t})=\sum\limits_{i=1}^{M}Pr(s_{t+n}=j|s_{t}=i)\\\
\times Pr(s_{t}=i|\Omega_{t}).\end{split}$ (13)

##  5 Results and Discussion

###  5.1 Results of data collected using smartphones

The data collected from the smartphones have been fitted using the Markov
Regime Switching model presented in Eq. (14), where the driving signals
selected as features are: follower velocity $v$, acceleration $a$, velocity
difference $dv$, and gap distance $h$. The model is introduced in our previous
work Zaky and Gomaa (2014) with the model ability to classify the different
car following driving regimes.

$v_{t+1}=\phi_{1,s_{t}}a_{t}+\phi_{2,s_{t}}dv_{t}+\phi_{3,s_{t}}h_{j,t}+\epsilon_{t}.$
(14)

The estimated model parameters $\sigma_{s_{k}},\phi_{1},\phi_{2}$ and
$\phi_{3}$ are presented in Table 2 for each regime, where the log-likelihood
approach is used. Markov transition matrix $p$ estimated, where
$P(s_{t}=i|s_{t-1}=j)$ is the probability of moving from driving regime $j$ to
driving regime $i$:

$p=\begin{pmatrix}0.94&0.05&0.01&0.00\\\ 0.00&0.85&0.00&0.01\\\
0.00&0.00&0.81&0.51\\\ 0.06&0.10&0.18&0.48\end{pmatrix}$

Table 1 presents the observed information of each driving regime in the
dataset. We can observe the expected duration in which a driver can stay in
each regime based on Eq. (8). This means that the driver will stay for a time
around the expected duration time driving in that regime. Other
characteristics of regimes are shown such as the number of occurrence which
counts the samples inside each regime, the number of observations that belongs
to each regime, and the percentage of driving under each regime overall time.

Regime Characteristics
---
State |  | Expected
---
duration (ms)
Occurrence | Observations | Percentage
Regime 1 | 17.50 | 4 | 276 | 67%
Regime 2 | 6.79 | 8 | 42 | 10%
Regime 3 | 5.34 | 10 | 34 | 8.25%
Regime 4 | 1.91 | 11 | 60 | 14.5%

Table 1: Driving regimes contained in the adopted dataset.

Parameter | Regime 1 | Regime 2 | Regime 3 | Regime 4
---|---|---|---|---
$\sigma_{s_{k}}$ | 1.3346 | 1.2511 | 1.6234 | 1.9769
$\phi_{1}$ | 1.2163 | -0.4069 | 0.0671 | -1.5901
$\phi_{2}$ | 1.5228 | 1.2641 | 0.3858 | 0.9247
$\phi_{3}$ | 0.6563 | 0.5074 | 0.2625 | 0.3031

Table 2: Estimates of car following Markov regime switching model parameters.

We have conducted several trials and due to the noise of the sensors specially
the GPS (which has up to 4 meters error with a sampling rate of 1 sample per
second), our experiment is limited by driving in only three car following
situations: acceleration, braking, and normal following. A manual tagging for
the car following situations is done. The interpretation of the results is as
following:

  * •

Tag 0: is the acceleration behavior which the model classifies as Regime 2.

  * •

Tag 2: is the braking behavior which the model classifies as Regime 3.

  * •

Tag 4: is the stable following behavior which the model classifies as Regime
1.

The results comply with the labels taken by the follower driver that is out of
412 records (seconds):

  * •

320 records are tagged as label 4 (stable following). The model classifies 276
observations with mis-classification of 44 observations, i.e., accuracy of
86.25%.

  * •

49 records are tagged as label 0 (acceleration). The model classifies 42
observations with mis-classification of 7 observations, i.e., accuracy of
85.42%.

  * •

43 records are tagged as label 2 (braking). The model classifies 34
observations with mis-classification of 9 observations, i.e., accuracy of 79%.

###  5.2 Results of the naturalistic driving data

We have implemented different MSVAR models with different regimes, varying
from 2 to 5 regimes and different time lag varying from 2 to 5 ms. The maximum
log-likelihood values are shown in Table 3 for different lags and regimes. As
shown in Table 3 the best model fit is for lag 1 having 5 regimes with the
maximum likelihood value. The using of the lag can be useful for modeling the
driver sensitivity factor which is presented in all GM models Rakha et al.
(2009). The following driver responds to the leading driver action by
acceleration or deceleration depending on the driver perception, reaction
time, and driver sensitivity factors. An average reaction time is estimated in
a range between 1.0 to 2.2 seconds, and an average driver sensitivity factor
of 0.37 second as introduced in May (1990).

Dataset_1
---
Regime Lag | 1 | 2 | 3 | 4 | 5
2 | 10036.68 | 10642.67 | 10739.58 | 11057.45 | 11090.25
3 | 9908.593 | 10869.07 | 10992 | 10837.91 | 10889.22
4 | 10151.84 | 10400.08 | 10981.64 | 10515.56 | 10499.64
5 | 10377.99 | 10244 | 10359.58 | 10341.47 | 10005.32
6 | 10191.81 | 9348.61 | 9985.96 | 10054.52 | 10119.44

Table 3: MSVAR log likelihood values.

Most VAR models are estimated using symmetric lags, i.e., the same lag length
is used for all variables of the model. The model lag length $P=1,...,p$ can
be determined based on a specific selection criteria. Model selection criteria
such as Akaike Information Criterion (AIC), Bayesian Information Criterion
(BIC), and Hannan-Quinn Information Criterion (HQC) are used. The selection of
an inappropriate lag length may affect the model performance and fitting.

The results of applying each criteria on the adopted dataset are shown in Fig.
2. This figure shows the results for different lags between 1 and 60 ms. This
helps us to exploit the history of the driving situation in the model. The
dataset is sampled at 10 Hz, thus 1 lag means 0.1 second while 60 lag means 6
seconds. The minimum values of the three selection criteria falling between 5
and 13 (0.5 and 1.3 seconds) can be observed which represent the different
reaction times of the driver.

(a) AIC values for lag between 1 and 60 ms.

(b) BIC values for lag between 1 and 20 ms.

(c) HQ values for lag between 1 and 20 ms.

Figure 2: MSVAR(P) model selection criteria.

The MSVAR model presented in Table 3 is used for fitting the naturalistic
driving data. The presented results are for the dataset which contains 2529
observations. As shown in the table, the best fitting is for the model with a
lag of 5 and 2 regimes. The lag selection criteria described above uses 2520
observations, and the remaining 9 observations have been used for models
forecasting evaluation.

The forecasting results based on Eq. (12) represent a comparison of the
prediction performance of the two selected models. The highest max log-
likelihood model is selected. The first model has 1 lag and 5 regimes Model I
(p=1,r=5), while the second model has 5 lags and 2 regimes Model II (p=5,r=2).
Table 4 shows a comparison between the Mean Square Error (MSE) of the
observation vector for the two models. The error is calculated between the
dataset observed values and the predicted values for 9 samples (0.9 second).
As shown, Model II has a lower MSE for all observation vector elements.

Model MSE | $a$ | $\Delta v$ | $h$ | $v$
---|---|---|---|---
I (p=1,R=5) | 0.1788184 | 0.013395644 | 0.05679842 | 0.02309714
II (p=5,R=2) | 0.1481768 | 0.010264159 | 0.05339007 | 0.01554496

Table 4: Mean square error of the two MSVAR models.

Fig. 3 presents the prediction of the two models for each point. The red
points represent the naturalistic real driving data samples, blue points
represent Model I, and green points represent Model II. Both models are
accurate for the first forecasting steps, afterwards the models start
deviating due to the accumulated error of the forecasting process (as shown in
the velocity and velocity difference figures). The models are able to predict
not only the driver behavior represented by its observation (velocity and
acceleration), but also the entire driving situation represented by the
relation with the leading vehicle observation (gap distance and velocity
difference).

(a) DVP

(b) HP

(c) VP

Figure 3: MSVAR model predicted values of car following observations.

###  5.3 MSVAR versus PrARX

The proposed MSVAR framework, as a switching linear regression model for
driving behavior, has features listed in Table 5. The table also lists the
features of PrARX as a switching framework based on linear regression. Both
frameworks are able to provide behavior/mode segmentation extracted from
driving signals. PrARX uses K-means clustering and the proposed framework uses
a probabilistic classifier based on selecting the maximum state filtering
value for each observation. The probabilistic classification approach is able
to identify the membership probability of a new observation to each state
(finds the best state that represents the observation with the highest
probability). The advantage of probabilistic classifier over non-probabilistic
one is that the former behaves like a confidence weighted classifier which
helps avoiding error propagation. The classifier adds value for the driving
behavior problem by allowing a smooth transition between each regime allowing
a mixed mode representation to understand the current behaviors that the
driver may behave.

|  PrARX |  Proposed MSVAR framework
---|---|---
Behavior (Mode) segmentation |  Extension to k-means clustering |  Probabilistic classification
Learning process |  Cannot simultaneously classify and estimate |  Classify and estimate in a single recursive process
Parameter estimation method |  Steepest descent |  ML (Hamilton) and Bayesian (Gipps Sampling)
Computational cost |  High due to the processing of the classification and estimation independently |  MCMC minimize computational time
Assumptions |  Human driving behavior does not exhibit an abrupt change |  Abnormal behavior detected and can capture short time events - behavior and switching processes are stationary
Mixed mode |  Can |  Can
Table 5: Main features of PrARX model and the proposed MSVAR framework

The learning process of PrARX has two stages; behavior classification and
parameters estimation. PrARX framework cannot simultaneously classify and
estimate. The proposed framework classifies and estimates in a recursive
process by using Hamilton filter for classification and maximum likelihood for
parameter estimation. Computational cost of PrARX is higher due to the
processing of classification and estimation independently. For the proposed
model, a Bayesian parameter estimation based on MCMC minimizes computational
time over Expectation Maximization (EM). As a future work, performance
measurements for the framework evaluation will be conducted.

PrARX has a major assumption for relaxing the problem of parameter estimation.
The assumption is that human driving behavior does not exhibit an abrupt
change which allows the framework to use the parameters of the previous
behavior as initial values for the next one. The proposed MSVAR framework does
not have this assumption as the model parameter estimation process generates
independent parameters for each regime, which allows the framework to handle
abnormal behavior. The proposed MSVAR framework has an assumption that the
driver behavior is stationary, so the stochastic process representing each
regime and the switching process (transition matrix) have the same parameters
over time. Releasing this assumption requires a variable time transition
probabilities and more complex stochastic models for driver behavior modeling
such as Gaussian Mixture Models (GMM).

##  6 Conclusions and Future Work

The prediction of driving behavior based on multiple Markov Switching Variable
Auto-Regression (MSVAR) is introduced. More than one model are implemented
with different parameters (lag and regime) and with different evaluation
criteria (AIC, BIC, HQC). The best fitted models are selected for the
prediction process. Additionally, the model is capable of fitting driving data
and data segmentation into regimes by estimating the different driving
behavior change points. One limitation is the long calibration time of the
model parameters. This is attributed to the learning of each model (depending
on the model configuration the learning takes up to 3 days). The best fitted
model is achieved at lag of 13 (1.3 s). We have implemented models with lags
down to 5 (0.5 seconds). The computational efficiency of the prediction is
reasonable; it takes only few seconds, however, it needs more adaptation to be
more accurate. We also presented a low-cost data collection solution using
smartphones validated with another naturalistic driving data set for
predicting the driver behavior for short periods of time. The proposed driver
behavior detection model can potentially be used in systems such as accident
prediction and driver safety.

##  7 Acknowledgements

This work is mainly supported by the Ministry of Higher Education (MoHE) of
Egypt through PhD fellowship awarded to Dr. Ahmed Zaky. This work is supported
in part by the Science and Technology Development Fund (STDF), Project ID
12602 ”Integrated Traffic Management and Control System”, and by E-JUST
Research Fellowship awarded to Dr. Mohamed A. Khamis.

## References

  * Bay (2013) In: Bayesian Data Analysis third edition. London : Chapman and Hall; 2013. .
  * Ahmed et al. (2007) Ahmed T, Oreshkin B, Coates M.  Machine learning approaches to network anomaly detection.  In: Proceedings of the 2nd USENIX workshop on Tackling computer systems problems with machine learning techniques USENIX Association; 2007\. .
  * Akita et al. (2008) Akita T, Suzuki T, Hayakawa S, Inagaki S.  Analysis and synthesis of driving behavior based on mode segmentation.  In: Control, Automation and Systems, 2008. ICCAS 2008. International Conference on IEEE; 2008. p. 2884–2889.
  * Angkititrakul et al. (2013) Angkititrakul P, Miyajima C, Takeda K.  Stochastic Mixture Modeling of Driving Behavior During Car Following.  Journal of information and communication convergence engineering 2013;11(2):95–102.
  * Dapzol (2005) Dapzol N.  Driver’s behaviour modelling using the Hidden Markov Model formalism.  In: ECTRI Young researchers seminar, The Hague, the Netherlands, vol. 2; 2005. p. 2–1.
  * Gipps (1981) Gipps PG.  A behavioural car-following model for computer simulation.  Transportation Research Part B: Methodological 1981;15(2):105–111.
  * Hamilton (1989) Hamilton JD.  A new approach to the economic analysis of nonstationary time series and the business cycle.  Econometrica: Journal of the Econometric Society 1989;p. 357–384.
  * Hamilton and Susmel (1994) Hamilton JD, Susmel R.  Autoregressive conditional heteroskedasticity and changes in regime.  Journal of Econometrics 1994;64(1):307–333.
  * Hoogendoorn et al. (2007) Hoogendoorn SP, Ossen S, Schreuder M.  Adaptive car-following behavior identification by unscented particle filtering.  In: Transportation Research Board 86th Annual Meeting No. 07-0950; 2007\. .
  * Ikeda et al. (2011) Ikeda K, Mima H, Inoue Y, Shibata T, Fukaya N, Hitomi K, et al.  An Adaptive Rear-End Collision Warning System for Drivers That Estimates Driving Phase and Selects Training Data.  Transactions of the Institute of Systems, Control and Information Engineers 2011;24:193–199.
  * Johnson and Trivedi (2011) Johnson DA, Trivedi MM.  Driving Style Recognition Using a Smartphone as a Sensor Platform.  In: Proceedings of the 14th International IEEE Conference on Intelligent Transportation Systems Washington, DC: IEEE; 2011. .
  * Khamis and Gomaa (2014) Khamis MA, Gomaa W.  Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework.  Journal of Engineering Applications of Artificial Intelligence 2014 March;29:134–151.
  * Khamis et al. (2012) Khamis MA, Gomaa W, El-Mahdy A, Shoukry A.  Adaptive Traffic Control System Based on Bayesian Probability Interpretation.  In: Proc. IEEE 2012 Japan-Egypt Conference on Electronics, Communications and Computers (JEC-ECC 2012) Alexandria, Egypt; 2012. p. 151–156.
  * Krolzig (2013) Krolzig HM.  Markov-switching vector autoregressions: Modelling, statistical inference, and application to business cycle analysis, vol. 454.  Springer Science & Business Media; 2013.
  * Ma (2006) Ma X.  A neural-fuzzy framework for modeling car-following behavior.  In: Systems, Man and Cybernetics, 2006. SMC’06. IEEE International Conference on, vol. 2 IEEE; 2006. p. 1178–1183.
  * Manstetten et al. (1997) Manstetten D, Krautter W, Schwab T.  Traffic simulation supporting urban control system development.  In: Mobility For Everyone. 4Th World Congress On Intelligent Transport Systems, 21-24 October 1997, Berlin.(Paper No. 2055); 1997. .
  * May (1990) May AD.  Traffic flow fundamentals.  Prentice-Hall; 1990.
  * Nitin R.Chopde1 (2013) Nitin R Chopde1 MMKN.  Landmark Based Shortest Path Detection by Using A* and Haversine Formula.  International Journal of Innovative Research in Computer and Communication Engineering 2013;92:298–302.
  * Okuda et al. (2013) Okuda H, Ikami N, Suzuki T, Tazaki Y, Takeda K.  Modeling and analysis of driving behavior based on a probability-weighted ARX model.  Intelligent Transportation Systems, IEEE Transactions on 2013;14(1):98–112.
  * Okuda et al. (2009) Okuda H, Suzuki T, Nakano A, Inagaki S, Hayakawa S.  Multi-hierarchical modeling of driving behavior using dynamics-based mode segmentation.  IEICE transactions on fundamentals of electronics, communications and computer sciences 2009;92(11):2763–2771.
  * Panwai and Dia (2007) Panwai S, Dia H.  Neural agent car-following models.  Intelligent Transportation Systems, IEEE Transactions on 2007;8(1):60–70.
  * Rakha et al. (2009) Rakha H, Pasumarthy P, Adjerid S.  A simplified behavioral vehicle longitudinal motion model.  Transportation letters 2009;1(2):95–110.
  * Richardson and Green (1997) Richardson S, Green PJ.  On Bayesian analysis of mixtures with an unknown number of components.  Journal of the Royal Statistical Society Series B (Methodological) 1997;p. 731–792.
  * Sathyanarayana et al. (2008) Sathyanarayana A, Boyraz P, Hansen JH.  Driver behavior analysis and route recognition by hidden Markov models.  In: Vehicular Electronics and Safety, 2008. ICVES 2008. IEEE International Conference on IEEE; 2008. p. 276–281.
  * Sekizawa et al. (2007) Sekizawa S, Inagaki S, Suzuki T, Hayakawa S, Tsuchida N, Tsuda T, et al.  Modeling and recognition of driving behavior based on stochastic switched ARX model.  Intelligent Transportation Systems, IEEE Transactions on 2007;8(4):593–606.
  * Srivastava et al. (2011) Srivastava S, Ng KK, Delp EJ.  Co-ordinate mapping and analysis of vehicle trajectory for anomaly detection.  In: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) IEEE; 2011. .
  * Takeda (2016) Takeda K.  Modeling and Detecting Excessive Trust from Behavior Signals: Overview of Research Project and Results.  In: Human-Harmonized Information Technology, Volume 1 Springer; 2016.p. 57–75.
  * Treiber and Kesting (2013) Treiber M, Kesting A.  Microscopic Calibration and Validation of Car-Following Models–A Systematic Approach.  Procedia-Social and Behavioral Sciences 2013;80:922–939.
  * Zaky and Gomaa (2014) Zaky AB, Gomaa W.  Car Following Regime Taxonomy Based on Markov Switching.  In: Proceedings of the IEEE 17th International Conference on Intelligent Transportation Systems (ITSC 2014) Qingdao, China: IEEE; 2014. p. 1329–1334.
  * Zaky et al. (2015) Zaky AB, Gomaa W, Khamis MA.  Car Following Markov Regime Classification and Calibration.  In: Proceedings of the IEEE 14th International Conference on Machine Learning and Applications (ICMLA 2015) Miami, Florida, USA: IEEE; 2015. .