Computers and Electrical Engineering 43 (2015) 66–75

Contents lists available at ScienceDirect

Computers and Electrical Engineering

journal homepage: www.elsevier .com/ locate /compeleceng

iBump: Smartphone application to detect car accidents q

http://dx.doi.org/10.1016/j.compeleceng.2015.03.0030045-7906/� 2015 Elsevier Ltd. All rights reserved.

q Reviews processed and approved for publication by the Editor-in-Chief.⇑ Corresponding author.

E-mail addresses: faloul@aus.edu (F. Aloul), izualkernan@aus.edu (I. Zualkernan), g00035203@aus.edu (R. Abu-Salma), b00032391@aus.edu (Hg00029113@aus.edu (M. Al-Merri).

1 Computer Science Department, University College London, UK.

Fadi Aloul ⇑, Imran Zualkernan, Ruba Abu-Salma 1, Humaid Al-Ali, May Al-MerriComputer Science & Engineering Department, American University of Sharjah, Sharjah, United Arab Emirates

a r t i c l e i n f o

Article history:Received 14 April 2014Received in revised form 1 March 2015Accepted 5 March 2015

Keywords:Dynamic Time Warping (DTW)Hidden Markov Models (HMMs)Pattern recognitionAccident detectionSmartphone application

a b s t r a c t

Traffic accidents are a fact of life. While accidents are sometimes unavoidable, studies showthat the long response time required for emergency responders to arrive is a primary rea-son behind increased fatalities in serious accidents. One way to reduce this response time isto reduce the amount of time it takes to report an accident. Smartphones are ubiquitousand with network connectivity are perfect devices to immediately inform relevant author-ities about the occurrence of an accident. This paper presents the development of a systemthat uses smartphones to automatically detect and report car accidents in a timely manner.Data is continuously collected from the smartphone’s accelerometer and analyzed usingDynamic Time Warping (DTW) and the Hidden Markov Models (HMMs) to determinethe severity of the accident, reduce false positives and to notify first responders of theaccident location and owner’s medical information. In addition, accidents can be viewedon the smartphone over the Internet offering instant and reliable access to the informationconcerning the accident. By implementing this application and adding a notificationsystem, the response time required to notify emergency responders of traffic accidentscan potentially reduce the response time which may help in reducing fatalities.

� 2015 Elsevier Ltd. All rights reserved.

1. Introduction

Smartphones are becoming more advanced and complex, and support a large number of sensors including audio recor-ders, Global Positioning Systems (GPSs), accelerometers and light and temperature sensors in addition to many others [1].There are many opportunities of implementing consumer applications that intelligently exploit the built-in sensors ofsmartphones.

In addition, most smartphones support wireless data services which provides additional opportunities for building con-sumer applications that exploit the sensors and the network connectivity afforded by the various types of connectivity rang-ing from SMS, GPRS and 3G/4G.

While there has been considerable progress in the use of advanced driver-assistance systems (ADAS), lane departurewarning system, and collision avoidance systems, the high cost of these systems has prompted researchers to consider usingsensors on smartphones to warn a driver and to prevent unsafe driving behavior [2]. Sathyanarayana et al. [3] propose theuse of a variety of sensors in the car including accelerometers to measure driver distraction. Threshold-based techniques

. Al-Ali),

F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75 67

using accelerometer data have been previously proposed to detect and report accident in motorcycles [4]. However, in amotorcycle accident since the motorcycle typically falls after an accident, it is much easier to use thresholds on accelerom-eter data to detect an accident. The situation is more complex in a car where the smartphone is typically inside the car andthere is a great possibility of false positives if such a simplified approach is used. Another closely related problem of detectingan accident is that of fall detection which has attracted significant research in the past few years [5]. For example, Tamuraet al. [6] used an accelerometer and gyro data to detect falling behavior based on simple thresholding techniques. Shi et al.[7] describe a more advanced system that applied a support vector machine (SVM) classifier to accelerometer data to detectfalling behavior. Naïve Bayes classifiers have also been used to detect falling behavior [8]. Similarly, Abbate et al. [9] describea system that uses a smartphone base accelerometer data with neural networks to successfully detect falls and reduce falsepositives. Finally, Kerdegari et al. [10] conducted a comparative analysis of a variety of pattern classification techniques likeMultilayer Perceptron, Naive Bayes, Decisions trees, Support Vector Machine, ZeroR and OneR in conjunction withaccelerometer data to detect falling behavior.

There are also many built-in systems that are used to detect car accidents using the electronic control units in the car,such as the OnStar AACN, and the ODB-II. Built-in sensors in the car can be used to detect changes in acceleration, or evento detect whether an airbag was ejected, which is a clear indication that a car accident has occurred [11]. WreckWatch is awireless smartphone-based application that detects and reports traffic accident [11]. The system uses an accelerometer andaudio data from the smartphone. The system uses a rule-based approach that combines thresholding on accelerometer andaudio data to detect accidents and to reduce false positives. Similarly, Dai et al. [12] describe a system that uses various typesof thresholding on acceleration data to detect several categories of drunk driver behavior including weaving, drifting, swerv-ing and turning with wide radius. Another approach has been to combine data from car’s OBD-II networks with those fromsmartphones [13].

The contribution of this paper is the development of a hybrid approach that uses Dynamic Time Warping (DTW) andHidden Markov Models (HMM) in conjunction with the built-in accelerometer in smartphones to detect and report caraccidents using existing telecom and Internet infrastructure. The approach is practical as it only requires the end user todownload the application on their smartphones without the need to buy expensive car built-in accident detection systems.The proposed approach has been successfully tested in a variety of situations using simulation data.

The remainder of this paper is organized as follows: Section 2 describes the components of the proposed system. Section 3describes the system implementation. Section 4 presents experimental results. Finally, Section 5 concludes the paper.

2. The proposed system

Fig. 1 shows the proposed system. The system is based on a smartphone application that continuously detects if there isan accident using the built-in accelerometer. In the case of an accident, the severity is detected and the location is identifiedusing the built-in GPS. The system then sends an SMS to emergency services and registered emergency contacts notifyingthem of the user’s information, accident, its severity, and its location. The system consists of two main parts: an Androidapplication to be downloaded and an application server. Each part is described below.

2.1. Android application

An Android application, as shown in Fig. 2(a), is downloaded on to a smartphone with a built-in accelerometer and sup-porting smartphone location services like a built-in GPS and/or GSM triangulation. As Fig. 2(b) shows, the application allowsa user to enter their personal information including name, ID, blood type, and phone numbers of individuals to inform in caseof an accident. The application runs in the background and if an accident occurs, the application immediately sends an SMSto the police, emergency services, and registered emergency contacts with user’s information and geo-location. After sendingSMSes, the application gives the user an option to register a false alarm which if done, sends SMSes to the various partiesindicating that the previous SMS was a false alarm (Fig. 2(c)).

2.2. Application server

The application server is a web-based application built using Apache, PHP, MySQL and JQuery. The application server pro-vides the following services.

� Real-time reporting of accidents with geolocation.� Various report showing current accidents and their location and trends.� User registration and tracking.

A screen shot of the application server is shown in Fig. 3.

Fig. 1. Overview of the proposed system.

Fig. 2. Screen shots of the accident detection smartphone application.

68 F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75

3. System implementation

The system was developed based on crash data generated using a physical apparatus as shown in Fig. 4. A 3-axisaccelerometer interfaced to a microcontroller was mounted on a metal surface of the testing model. Various severities ofcrashes were simulated by extending the spring to various lengths and by letting the accelerometer crash into the fixed sur-face. Accelerometer data during this movement was stored and retrieved for later analysis. The accelerometer embedded inthe smartphone, which usually has a range of 3G, allowing for more accurate results when determining between fallaccidents and daily user activities, since falls and daily user activities, such as running or walking, usually have accelerationmagnitude between 0 Gs and 6 Gs. However, the accelerometer values that are recorded during a car accident are usuallyhigher than 6 Gs, which is difficult to detect using a smartphone accelerometer that has a maximum reading of 3 Gs.Therefore, the apparatus used accelerometers with a range of ±16 Gs and this data was clipped to a threshold of 3 Gs in orderto verify that the proposed algorithm worked accurately in detecting car accidents on the smartphone.

The data retrieved from the apparatus was used to construct two different algorithms for crash detection. The develop-ment of each algorithm is described next.

Fig. 3. Screen shot from the application web server.

Fig. 4. Screen shots from the accident simulator apparatus.

F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75 69

3.1. Using Dynamic Time Warping (DTW)

Dynamic Time Warping (DTW) is a time series alignment algorithm developed originally for speech recognition. It align’stwo sequences of feature vectors by warping the time axis iteratively until an optimal match (according to a suitable metrics)between the two sequences is found [14]. The standard DTW has a time and space complexity of O(n2) where n is the lengthof the sequences being compared [15]. DTW has been successfully used to compare data streams [16].

As Fig. 4 shows, the apparatus described earlier was used to collect 30 samples for each of the three severity states byvarying the length with which the spring was pulled. Fig. 5 shows sample accident data from the apparatus. The no-accidentstate data was collected by including actual data from a car including cases of sudden acceleration, sudden brake, anduneven road. Fig. 6 shows actual no-accident data reflecting a car driving on an uneven road.

The raw data thus collected from the accelerometer consisted of ax, ay, and az as the acceleration on x-axis, y-axis and z-axis, respectively, and was transformed into single magnitude of acceleration (MA) as shown in Eq. (1).

MA ¼ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffia2

x þ a2y þ a2

z

qð1Þ

Fig. 5. Sample data readings reflecting an accident from the developed apparatus. Different colors represent the x, y, and z-axis readings. (For interpretationof the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 6. Sample data readings reflecting a no-accident from an actual car driving on an uneven road. Different colors represent the x, y, and z-axis readings.(For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

70 F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75

In order to apply this technique, it was assumed that a vehicle can be in one of four states; no-accident, low severity, med-ium severity and high severity accident. Training data was collected for each of the four states using the apparatus and sub-sequently DTW was then used to distinguish between the four states [17].

3.2. Using Hidden Markov Models (HMMs)

HMM is a modeling tool that can be reliably employed for modeling and analyzing time-series with spatial or temporalvariability and has been applied in many gesture and speech recognition projects with good results. For example,GestureWrist is a wrist-watch that recognizes distinct hand gestures using capacitance and acceleration sensors [18].

Fig. 7 shows the basic approach for developing the HMM-based algorithm. Sensing step consists of recording theaccelerometer data. Since the smartphone accelerometers currently only measure a range of [�3 Gs,3 Gs], the original testdata which has a range of [�16 Gs,+16 Gs] is clipped to this 3G range. Weighted Moving Average (WMA) filtering was usednext to smooth out the data (a = 0.2) as shown in Eq. (2) where a is the ax, ay, and az.

aiþ1 ¼ a � ai�1 þ ð1� aÞ � ai ð2Þ

Different values of a were used and empirically the value of 0.2 provided the best results.As Fig. 7 shows, the filtered data is submitted next to a vector quantization unit. Vector Quantization is mainly used to

convert three-dimensional sampled data to one-dimensional sequences of discrete symbols or prototype vectors, such thatthese vectors are used as inputs for both the training and recognition phases of the HMMs. The collection of these vectors isreferred to as the codebook.

Two codebook encodings were used. The first codebook encoding used the K-means algorithm. The K-means algorithm isa well-known clustering algorithm [19]. The K-means algorithm was used with the squared Euclidean distance d2 as shownin Eq. (3).

d2ða1; a2Þ ¼ a1x � a2xð Þ2 þ a1y � a2y� �2 þ a1z � a2zð Þ2 ð3Þ

Fig. 7. Overview of the HMM process.

F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75 71

After applying the K-means algorithm with a cluster size of 8, the original continuous acceleration vector a(t) is reduced to adiscrete codebook scalar cb(t) where cb(t) can take one of eight discrete states. Cluster size of 8 was chosen empirically.

The second codebook called Slope-codebook encoding used the slope of MA (see Eq. (1)) to encode the original a(t) intothree states of (1) decreasing, (2) no change, and (3) increasing. Fig. 8 shows the original accelerometer data for the mildaccident case and Fig. 9 shows the result of applying the Slope-codebook to the data. Similarly, Fig. 10 shows the originalaccelerometer data for the severe accident case and Fig. 11 shows the result of applying the Slope-codebook to the data.

The basic approach in applying HMMs consists of constructing a single HMM for each of the vehicle states. Severalwell-known algorithms for training the models of the HMM technique exist. Both Viterbi and Baum–Welch algorithms wereutilized for training [20]. The readings recorded from the apparatus were divided into two sets: a training set for training thealgorithm, and a testing set for testing how well the training was and for analyzing the algorithm’s performance. After gath-ering different readings from the test they are placed into a matrix that is given as input to the classifier algorithm duringtraining. The training algorithm determines the right function to be able to distinguish between the different types of sever-ity. Based on the features of each severity from data, the training algorithm tries to find the exact line that separates theseseverities. The algorithm continues searching until the error rate is less than or equal to provided threshold. Once the errorrate is less than or equal than specified, the Baum–Welch algorithm returns the function that represents the line that sep-arates the severities. The Baum–Welch algorithm determines the probability of occurrence of the test set and uses it as a

0 50 100 150 200 2500.9

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

Time (Number of Readings)

Am

plitu

de (G

For

ce)

Fig. 8. Original accelerometer data for the mild accident case.

Fig. 9. Result of applying the Slope-codebook to the mild accident data.

0 50 100 150 200 2501

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

2

Time (Number of Readings)

Am

plitu

de (G

For

ce)

Fig. 10. Original accelerometer data for the severe accident case.

Fig. 11. Result of applying the Slope-codebook to the severe accident data.

72 F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75

Table 1Confusion matrix for DTW.

Predicted Actual

No-accident Accident

No-accident 23 0Accident 2 75

Table 2Confusion matrix for HMM with K-means codebook.

Predicted Actual

No-Accident Accident

No-accident 18 0Accident 7 25

Table 3Confusion matrix for HMM with Slope-codebook.

Predicted Actual

Mild Severe

Mild 25 0Severe 1 24

F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75 73

learning mechanism for the model. Even though the complexity of the original Baum–Welch algorithm is O(n2), the algo-rithm is only run once on the in order to train the algorithm to be able to accurately specify which model the data belongsto, and is not run on the actual application on the Smartphone. After the training phase, the Viterbi forward–backward algo-rithm is run to find the most probable sequence of states or paths for a model given an observed sequence coded using thecodebook. The model that results in the highest probability is selected as the one to which the observed sequence belongs.There is one model for each of the no-accident, and severity states and the observed sequence is the sequence of statechanges as specified in the actual data being collected, filtered and converted using a code book as shown in Figs. 9 and 11.

4. Experimental results

In order to test the DTW algorithm, 25 test cases were developed for each of the states for a total of 100 test cases. Thealgorithm was executed on each of the test cases to make a prediction about the actual state. The results are shown in theform of a confusion matrix in Table 1. The confusion matrix shows the comparison of predicted vs. actual results. For exam-ple, in Table 1, out of 25 no-accident cases, 23 were predicted correctly by DTW and only 2 cases were false positives.Furthermore, all 75 accident cases were predicted correctly as accident cases.

The accident condition shown in Table 1 includes all three severity levels using DTW. As the table shows, the DTWmethod was able to achieve an overall performance of (23 + 75)/100 = 98% accuracy in distinguishing between a no-accidentand an accident state. The severity levels were combined into one category using DTW because it was not able to adequatelydifferentiate between the severity levels. This distinction was adequately established by using the HMM technique asdescribed below.

In order to test the K-means-codebook HMM approach, 25 test cases for the no-accident condition and 25 test cases forthe accident state were used. The results are shown in Table 2. As the table shows, this approach resulted in (18 + 25)/50 = 86% accuracy in distinguishing no-accident from the accident state.

Even though the results for HMM in detecting accident vs. no-accident were worse than DTW, the second codebookapproach using Slopes helped distinguish between the various levels of severity. As Table 3 shows, this approach obtainedan accuracy of (24 + 25)/50 = 98% in distinguishing between mild and severe accidents.

Consequently, a two-tiered approach as shown in Fig. 12 can be used. The K-means codebook and DTW are collectivelyused to detect whether there is an accident. In case of a disagreement, DTW is preferred because it has a lower false positiverate. If an accident is detected, Slope-based HMM is used to differentiate between mild and severe accidents.

The application of the two-tiered approach on a typical Samsung Galaxy Nexus Android smartphone, with Dual-core1.2 GHz processor and 1 GB RAM, resulted in an average response time of 295 ms. This response time is more than adequatefor detecting and reporting car accidents.

Fig. 12. Overview of the proposed two-tiered approach for detecting car accidents.

74 F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75

5. Conclusions

In this paper a hybrid approach using Dynamic Time Warping (DTW) and Hidden Markov Models (HMMs) has beenimplemented and tested to detect and report car accidents using smartphones. As opposed to the threshold-based methods,this pattern-based hybrid approach yields a low false positive rate of 2%. Even though the approach seems promising, itneeds to be tested in the field using automotive crash simulation and detection systems. One key advantage of this approachis that it only requires the user to download and run the application on their smartphone without any extra equipment orcost. This system can be used in any moving vehicle without the need for expensive car built-in accident detection systems.Future work includes porting the application to run on other smartphone platforms such as IOS and Windows.

Acknowledgments

The authors would like to thank Rona Kanbar, Sarah Al-Otaibi, and Farah Al-Haddad for their help in collecting data and inimplementation and testing activities.

References

[1] Kwapisz JR, Weiss GM, Moore SA. Activity recognition using cell phone accelerometers. ACM SIGKDD Explor Newsl 2011;12(2):74–82.[2] Fazeen M, Gozick B, Dantu R, Bhukhiya M, Gonzalez M. Safe driving using mobile phones. IEEE Trans Intell Transport Syst 2012;13(3):1462–8.[3] Sathyanarayana A, Nageswaren S, Ghasemzadeh H, Jafari R, Hansen JH. Body sensor networks for driver distraction identification. In: Proc IEEE

international conference on vehicular electronics and safety; 2008. p. 120–5.[4] Watthanawisuth N, Lomas T, Tuantranont A. Wireless black box using mems accelerometer and GPS tracking for accidental monitoring of vehicles. In:

Proc IEEE international conference on biomedical and health informatics; 2012. p. 847–50.[5] Mubashir M, Shao L, Seed L. A survey on fall detection: principles and approaches. Neurocomputing January 2013;100:144–52.[6] Tamura T, Yoshimura T, Sekine M, Uchida M, Tanaka O. A wearable airbag to prevent fall injuries. IEEE Trans Inform Technol Biomed 2009;13(6):910–4.[7] Shi G, Chan CS, Li WJ, Leung K-S, Zou Y, Jin Y. Mobile human airbag system for fall protection using MEMS sensors and embedded SVM classifier. IEEE

Sens J 2009;9(5):495–503.[8] Yang X, Dinh A, Chen L. A wearable real-time fall detector based on naive bayes classifier. In: Proc IEEE Canadian conference on electrical and computer

engineering, May 2010. p. 1–4.[9] Abbate S, Avvenuti M, Bonatesta F, Cola G, Corsini P, Vecchio A. A smartphone-based fall detection system. Perv Mob Comput 2012;8(6):883–99.

[10] Kerdegari H, Samsudin K, Ramli AR, Mokaram S. Evaluation of fall detection classification approaches. In: Proc IEEE international conference intelligentand advanced systems; 2012. p. 131–6.

[11] White J, Thompson C, Turner H, Dougherty B, Schmidt DC. Wreckwatch: automatic traffic accident detection and notification with smartphones. MobNetw Appl 2011;16(3):285–303.

[12] Dai J, Teng J, Bai X, Shen Z, Xuan D. Mobile phone based drunk driving detection. In: Proc IEEE international conference on pervasive computingtechnologies for healthcare, March 2010. p. 1–8.

[13] Zaldivar J, Calafate CT, Cano J-C, Manzoni P. Providing accident detection in vehicular networks through OBD-II devices and Android-basedsmartphones. In: Proc IEEE conference on local computer networks, October 2011. p. 813–9.

[14] Muscillo R, Conforto S, Schmid M, Caselli P, D’Alessio T. Classification of motor activities through derivative dynamic time warping applied onaccelerometer data. In: Proc IEEE international conference on engineering in medicine and biology society; 2007. p. 4930–3.

[15] Lemire D. Faster retrieval with a two-pass dynamic-time-warping lower bound. Pattern Recogn 2009;42(9):2169–80.[16] Li W, Li F, Liu J, Le J. Similarity search over data stream using LPC-DTW. In: Proc IEEE international conference on machine learning and cybernetics,

August 2007. p. 1631–4.[17] Liu J, Zhong L, Wickramasuriya J, Vasudevan V. Uwave: accelerometer-based personalized gesture recognition and its applications. Perv Mob Comput

2009;5(6):657–75.[18] Mäntyjärvi J, Kela J, Korpipää P, Kallio S. Enabling fast and effortless customisation in accelerometer based gesture interaction. In: Proc ACM

international conference on mobile and ubiquitous multimedia; 2004. p. 25–31.[19] Kanungo T, Mount DM, Netanyahu NS, Piatko CD, Silverman R, Wu AY. An efficient k-means clustering algorithm: analysis and implementation. IEEE

Trans Pattern Anal Mach Intell 2002;24(7):881–92.[20] Rabiner LR. A tutorial on hidden markov models and selected applications in speech recognition. In: Proc IEEE, vol. 77, no. 2, February 1989. p. 257–86.

F. Aloul et al. / Computers and Electrical Engineering 43 (2015) 66–75 75

Fadi Aloul received the B.S. degree in electrical engineering (summa cum laude) from Lawrence Technological University, Southfield, MI, and the M.S. andPh.D. degrees in computer science and engineering from the University of Michigan, Ann Arbor. He is currently a Professor of Computer Science andEngineering and the Director of the HP Institute at the American University of Sharjah, UAE.

Imran Zualkernan holds a B.S. (High Distinction) and a Ph.D. from the University of Minnesota in Minneapolis. His research is in advanced learningtechnologies and Internet of Things. He has published over 120 research papers in international conferences, workshops and journals. He has served as aCEO and CTO and has designed and deployed advanced commercial robotics and Internet-based systems.

Ruba Abu-Salma is a Ph.D. student in the Department of Computer Science at University College London (UCL). Ruba researches user-centred and secureend-to-end encrypted communications protocols. She earned an M.Sc. (Distinction) in Information Security from UCL in 2014, and a B.Sc. (Hons) inComputer Engineering from the American University of Sharjah in 2013.

Humaid Al-Ali received the B.S. degree in computer engineering from the American University of Sharjah, UAE and is currently pursuing Master degree inQuality Management in the University of Wollongong in Dubai. He is a member of the UAE mGovernment Center of Digital Innovation, overseeing the mLabof mobile applications quality assurance for UAE government entities.

May AlMarri received the B.S. degree in computer engineering from the American University of Sharjah, UAE. Her research interests include mobileapplication development and security.