Application of Machine Learning in Travel Behavior...
Transcript of Application of Machine Learning in Travel Behavior...
![Page 1: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/1.jpg)
Application of Machine Learning in Travel Behavior Analysis
Muhammad Awais Shafique
Assistant Professor
University of Central Punjab, Pakistan
17th Behavioral Modelling Summer School14-16 September, 2018
![Page 2: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/2.jpg)
What is Machine Learning?
“Field of study that gives computers the ability to learnwithout being explicitly programmed.”
Arthur Samuel (1959)
“A computer program is said to learn from experience E with respect to some task T and some performance measure P, if
its performance on T, as measured by P, improves with experience E.”
Tom Mitchell (1998)
2
![Page 3: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/3.jpg)
Human Learning
• X-ray• Ultrasound• Biopsy• MRI
• Clump Thickness• Uniformity of Cell Size• Uniformity of Cell Shape etc.
Diagnose TumorMalignant or Benign
3
![Page 4: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/4.jpg)
Machine Learning
“A computer program is said to learn from experience E with respect to some task T and some performance measure P, if
its performance on T, as measured by P, improves with experience E.”
• Task T: Determining whether Tumor is Malignant or Benign
• Experience E: Medical records of Tumor Diagnosis
• Performance Measure P: Number of correct Detections
4
![Page 5: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/5.jpg)
Types of Learning Algorithms
• Supervised learning• Teach the computer and then let it use the learning to do the
task (labelled data).
• Unsupervised learning• Let the computer determine structure and patterns in the
data (unlabeled data).
• Semi-supervised learning• Let the computer determine patterns and then teach the
computer to do the intended task (unlabeled and labelleddata).
• Reinforcement learning• Let the computer determine the ideal behavior in an
environment to maximize its performance.5
![Page 6: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/6.jpg)
Problems with Data Collection
6
Travel Information
• Starting time
• Ending time
• Route taken
• Mode used
• Accompanyingpersons
• Trip purpose
Traditional Travel Surveys• Face-to-face interviews• Paper questionnaires• Telephone surveys• Computer-assisted surveys
Shortcomings reported by Brog (1982), Richardson (1996), Zmud (1997), Hato (2010) etc.
• Time-consuming• Laborious• Dependence on memory• Non-reporting of short trips• Biased responses• Variable perception of time
Automatic Detection of Travel
Information
![Page 7: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/7.jpg)
Evolution of Collection Methods
Data collection method Advantages Disadvantages
Questionnaire (paper-based, internet-based, telephone surveys)Forrest and Pearson (2005)
• Economy• Uniformity• Standardization
• Biased responses• Inaccurate perception of time• Small percentage of returned
forms• Dependence on memory
Toll Tag ReadersFerman et al. (2005)
• Automatic collection of average travel time and speed
• Limited only to tagged vehicles.
• Limited only to tolled roads• Detailed O-D information
unavailable• Cost of sensors
BluetoothPuckett and Vickich (2010)
• Automatic collection of average travel time and speed
• Limited only to Bluetooth enabled devices.
• Limited only to roads• Detailed O-D information
unavailable• Cost of sensors• Stops are unnoticed
7
![Page 8: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/8.jpg)
Evolution of Collection Methods
Data collection method Advantages Disadvantages
Wireless Location TechnologyFontaine et al. (2007)
• Cell phones can be anonymously located when an action is taken (e.g. phone call or sms)
• Tracking cannot be done at regular intervals
• Biased sample due to limited number of participating mobile companies
• Stops are unnoticed
Smartphones (GPS + accelerometer)Reddy et al. (2010)
• GPS enabled cell phones tracked at regular intervals
• Detailed movement pattern recorded
• Precise acceleration data along three axes available at regular intervals
• Detailed O-D information available
• Privacy of the mobile user is an issue due to the data available at micro level
8
![Page 9: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/9.jpg)
Lets Start from the Beginning
• Equipped with multiple sensors• GPS
• Accelerometer
• Gyroscope
• Barometer etc.
9
![Page 10: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/10.jpg)
Accelerometer Data
User ID Trip ID Date TimeAcceleration
X axis Y axis Z axisgf001 1 12/25/2010 11:33:39 AM 0.96 5.32 8.43gf001 1 12/25/2010 11:33:44 AM 0.69 5.98 7.78gf001 1 12/25/2010 11:33:47 AM 0.73 6.21 7.16gf001 1 12/25/2010 11:33:49 AM 0.84 6.4 6.63gf001 1 12/25/2010 11:33:54 AM 0.46 6.47 7.05gf001 1 12/25/2010 11:33:59 AM 1.11 7.01 7.24gf001 1 12/25/2010 11:34:05 AM 1.15 7.32 6.78gf001 1 12/25/2010 11:34:10 AM 1.3 7.51 6.47gf001 1 12/25/2010 11:34:15 AM 1.26 7.66 5.86gf001 1 12/25/2010 11:34:20 AM 1.03 7.85 5.13gf001 1 12/25/2010 11:34:22 AM 0.73 7.89 5.67gf001 1 12/25/2010 11:34:25 AM 0.69 8.12 5.21
10
![Page 11: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/11.jpg)
Trip Data
User ID Trip IDDeparture
dateDeparture
timeArrival date Arrival time Mode
gf001 1 12/25/2010 11:33:00 AM 12/25/2010 12:02:00 PM Walk
gf001 2 12/25/2010 12:07:00 PM 12/25/2010 12:32:00 PM Walk
gf001 3 12/25/2010 4:03:00 PM 12/25/2010 4:12:00 PM Car
gf001 4 12/25/2010 9:56:00 PM 12/25/2010 10:02:00 PM Walk
gf001 5 12/25/2010 10:02:00 PM 12/25/2010 10:09:00 PM Car
11
![Page 12: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/12.jpg)
Data Cleaning
• Get rid of duplicate data
• Clean the outliers
• Keep only relevant data
• Scan data for abnormalities• 70 minutes long trip marked as “Walk”
12
![Page 13: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/13.jpg)
Feature Engineering
• Feature engineering is the process of using domainknowledge of the data to create features that make machinelearning algorithms work.
• If done correctly, it increases the predictive power ofmachine learning algorithms by creating features from rawdata that help facilitate the machine learning process.
13
![Page 14: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/14.jpg)
• Acceleration data can be recorded by using accelerometer.
• For a single data instance, it is difficult to identify themode.
14
0
5
10
15
20
25
0 500 1000 1500 2000 2500
Acc
eler
atio
n (
m/s
ec2)
Data Instances
Resultant AccelerationWalk Bicycle Car Bus Train Subway
Feature Extraction
![Page 15: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/15.jpg)
• But extracted features can assist in the identification.
15
9
9.5
10
10.5
11
11.5
0 500 1000 1500 2000 2500
Acc
ele
rati
on
(m
/sec
2)
Data Instances
Avg. Resultant Acceleration
Walk Bicycle Car Bus Train Subway
5
7
9
11
13
15
17
19
21
23
25
0 500 1000 1500 2000 2500
Acc
ele
rati
on
(m
/sec
2)
Data Instances
Max. Resultant Acceleration
Walk Bicycle Car Bus Train Subway
Feature Engineering
![Page 16: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/16.jpg)
• Additional features can also be extracted.
• No single feature can distinguish among all modes.
16
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 500 1000 1500 2000 2500
Stan
dar
d D
evia
tio
n (
m/s
ec2)
Data Instances
Standard Deviation
Walk Bicycle Car Bus Train Subway
-4
-3
-2
-1
0
1
2
3
4
5
0 500 1000 1500 2000 2500
Skew
nes
s
Data Instances
Skewness
Walk Bicycle Car Bus Train Subway
Feature Engineering
![Page 17: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/17.jpg)
Time Domain
• Resultant Acceleration (m/sec2)
• Maximum Resultant Acceleration (m/sec2)
• Average Resultant Acceleration (m/sec2)
• Standard Deviation (m/sec2)
• Skewness (measure of symmetry)
• Kurtosis (measure of flatness)
Frequency Domain
• Wavelet Transformations
• Fourier Transformations
Discrete Domain
• Symbolic String Representations17
Feature Engineering
![Page 18: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/18.jpg)
Training & Testing Data
• Training Data: Part of data used to train algorithm
• Testing Data: Part of data used to test an already trained algorithm
StudyPercentage of data
used for trainingNick et al. (2010) 90Tragopoulou et al. (2014) 80Lester et al. (2006), Nitsche et al. (2012) 75Nham et al. (2008) 70Abdulazim et al. (2013) 65Figo et al. (2010) 50
18
![Page 19: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/19.jpg)
Classification Algorithms
• Support Vector Machines (SVM)
• Decision Tree (DT)
• Random Forest (RF)
19
![Page 20: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/20.jpg)
Support Vector Machines
• Among the best “off-the-shelf” supervised learningalgorithms.
• Off-the-shelf: A method that can be applied directly to datawithout requiring a great deal of time-consuming datapreprocessing or careful tuning of the learning procedure.
• SVM is a two-class classifier which forms a separatinghyperplane.
• When a set of training data containing class labels issupplied to SVM, it outputs an optimal hyperplane whichthen classifies new examples.
20
![Page 21: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/21.jpg)
Support Vector Machines
21
0
10
20
30
40
50
60
70
0 200 400 600 800 1000 1200
Margin
![Page 22: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/22.jpg)
Support Vector Machines
• Map data into a feature space where they are linearly separable.
22From Presentations by Nello Cristianini and Jason Weston respectively
![Page 23: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/23.jpg)
Support Vector Machines
23Support Vector Machines Explained by Tristan Fletcher
![Page 24: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/24.jpg)
Decision Trees
• Decision trees repeatedly split the dataset in order to arriveat a final outcome.
• The split is made into branch-like segments and thesesegments progressively form an inverted tree, whichoriginates from the starting node called the root.
• The root is the only node in the tree which does not have anincoming segment.
• The tree terminates at the decision nodes, also known asleaves or terminals.
• All the other nodes present within the tree are calledinternals or test nodes.
24
![Page 25: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/25.jpg)
Decision Trees
• The variables or features associated with the data are usedto make each split.
• At each node, the variables are tested to determine themost suitable variable to make the split.
• This testing is repeated on reaching the next node andprogressively forms a tree.
• Each terminal node corresponds to a target class.
25
![Page 26: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/26.jpg)
Example
• Suppose that a person A, aged 33 years and having a Ph.D.in his field along with an experience of 2 years, wants toapply for a job at a company.
• He has the hiring history of the company and knows that theimportant points checked in the interview are qualification,age and experience (these are the variables or featuresassociated with the applicants).
• Based on the known data (training data) he develops adecision tree, in order to check whether he will be able toget the job or not.
26
![Page 27: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/27.jpg)
Example
27
![Page 28: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/28.jpg)
Random Forest
• A bootstrap sample of the dataset is used to independentlygrow individual trees, and the majority vote is taken toconclude the final prediction.
• In addition to using a randomly selected bootstrap sampleof the data for growing each tree in the forest, randomnesswas introduced in the splitting of nodes.
• In standard trees, each node in the classification orregression tree is split using the best split among allvariables.
• In random forests, each node is split using the best among asubset of the variables randomly selected at that node.
28
![Page 29: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/29.jpg)
Random Forest
• Suppose 𝑛 number of trees are grown. Each tree isgenerated by randomly selecting nearly 63% of the giventraining data.
• The sample data is therefore different for each tree. Theremaining 37% data, known as out of bag (OOB) data, isused to estimate the error rate.
• The trees are fully grown without any requirement ofpruning, which is one of the advantages of random forest.
29
![Page 30: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/30.jpg)
Random Forest
• At each node a subset of variables or features is selectedand the most suitable feature among them is used for thesplit.
• The size of subset is a variable which is generally taken as𝑘 where 𝑘 is the total number of features.
• Once the forest is grown by using the labelled trainingdataset, the test data is introduced for the prediction.
• The individual predictions by the trees are aggregated toconclude the final prediction result (i.e. majority vote forclassification and average for regression).
30
![Page 31: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/31.jpg)
31
![Page 32: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/32.jpg)
Concluding Remarks
• Machine Learning is invading Civil Engineering
• Applications are endless
• Travel Mode Detection
• Trip Purpose Detection
• Pile Bearing capacity Prediction
• Rainfall Runoff Prediction
32
![Page 33: Application of Machine Learning in Travel Behavior Analysisbin.t.u-tokyo.ac.jp/model18/lecture/Shafique.pdf · Application of Machine Learning in Travel Behavior Analysis Muhammad](https://reader035.fdocuments.us/reader035/viewer/2022070815/5f0ec31f7e708231d440d0b0/html5/thumbnails/33.jpg)
33
facebook.com/m.awaisshafique
Jp.linkedin.com/in/awaisshafique/
www.researchgate.net/profile/Muhammad_Shafique19