Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video...
-
Upload
kan-ouivirach -
Category
Education
-
view
371 -
download
1
description
Transcript of Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video...
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Clustering Human Behaviors with Dynamic Time Warpingand Hidden Markov Models for a Video Surveillance System
Kan Ouivirach and Matthew N. Dailey
Computer Science and Information ManagementAsian Institute of Technology
ECTI-CONMay 19-21, 2010
1 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Outline
1 Introduction
2 Human Behavior Pattern Clustering
3 Experimental Results
4 Conclusion
2 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction
Human behavior understanding is important for intelligent systems.
Difficult due to the wide range of activities possible in any givencontext
Figure: Reprinted from http://www.sourcesecurity.com/
A classic work by Yamato et al. who model tennis actions usinghidden Markov models (HMMs)
3 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction
Human behavior understanding is important for intelligent systems.
Difficult due to the wide range of activities possible in any givencontext
Figure: Reprinted from http://www.sourcesecurity.com/
A classic work by Yamato et al. who model tennis actions usinghidden Markov models (HMMs)
3 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction
Human behavior understanding is important for intelligent systems.
Difficult due to the wide range of activities possible in any givencontext
Figure: Reprinted from http://www.sourcesecurity.com/
A classic work by Yamato et al. who model tennis actions usinghidden Markov models (HMMs)
3 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
To help security personnel work reliably and efficiently,
filter out typical events;automatically present anomalous events to human operator.
Figure: Reprinted from http://sikafutu.com/
4 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
To help security personnel work reliably and efficiently,
filter out typical events;automatically present anomalous events to human operator.
Figure: Reprinted from http://sikafutu.com/
4 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
To help security personnel work reliably and efficiently,
filter out typical events;automatically present anomalous events to human operator.
Figure: Reprinted from http://sikafutu.com/
4 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
One limitation of most of the work
The number of “normal” behavior patterns need to be knownbeforehand.Nair and Clark (2002) use HMMs to model a common, predefinedactivity in a scene.
Unsupervised analysis and clustering of behaviors for a variety ofpurposes has started to draw attentions.
Li et al. (2006) cluster human gestures by constructing an affinitymatrix using dynamic time warping (DTW), and apply thenormalized-cut approach to cluster.
5 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
One limitation of most of the work
The number of “normal” behavior patterns need to be knownbeforehand.Nair and Clark (2002) use HMMs to model a common, predefinedactivity in a scene.
Unsupervised analysis and clustering of behaviors for a variety ofpurposes has started to draw attentions.
Li et al. (2006) cluster human gestures by constructing an affinitymatrix using dynamic time warping (DTW), and apply thenormalized-cut approach to cluster.
5 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
One limitation of most of the work
The number of “normal” behavior patterns need to be knownbeforehand.Nair and Clark (2002) use HMMs to model a common, predefinedactivity in a scene.
Unsupervised analysis and clustering of behaviors for a variety ofpurposes has started to draw attentions.
Li et al. (2006) cluster human gestures by constructing an affinitymatrix using dynamic time warping (DTW), and apply thenormalized-cut approach to cluster.
5 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
One limitation of most of the work
The number of “normal” behavior patterns need to be knownbeforehand.Nair and Clark (2002) use HMMs to model a common, predefinedactivity in a scene.
Unsupervised analysis and clustering of behaviors for a variety ofpurposes has started to draw attentions.
Li et al. (2006) cluster human gestures by constructing an affinitymatrix using dynamic time warping (DTW), and apply thenormalized-cut approach to cluster.
5 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
One limitation of most of the work
The number of “normal” behavior patterns need to be knownbeforehand.Nair and Clark (2002) use HMMs to model a common, predefinedactivity in a scene.
Unsupervised analysis and clustering of behaviors for a variety ofpurposes has started to draw attentions.
Li et al. (2006) cluster human gestures by constructing an affinitymatrix using dynamic time warping (DTW), and apply thenormalized-cut approach to cluster.
5 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
Some recent related works using HMMs to cluster behavior patterns
Swears et al. (2008) propose hierarchical HMM-based clustering tofind motion trajectories and velocities in a highway interchangescene.
6 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
Some recent related works using HMMs to cluster behavior patterns
Swears et al. (2008) propose hierarchical HMM-based clustering tofind motion trajectories and velocities in a highway interchangescene.
6 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Introduction (cont.)
New method for clustering human behaviors in the context of videosurveillance
Combination of clustering and HMMs to group human behaviors
Summary flow
1 After extracting sequences, we use DTW to measure the pairwisesimilarity between sequences.
2 Construct an agglomerative hierarchical clustering dendrogrambased on the DTW similarity measure.
3 Recursively, find the optimal set of behavior clusters using HMMs.
Oates et al. (2001) first proposed the idea of using the DTW withHMMs to cluster time series.
7 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Compared to the related works
Potential to improve upon the state of the art in intelligent videosurveillance applications by
Bootstrapping human behavior classification and anomaly detectionmodulesSupporting incremental HMM learning (performing statistical teststo select which cluster should be incrementally updated)
8 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Compared to the related works
Potential to improve upon the state of the art in intelligent videosurveillance applications by
Bootstrapping human behavior classification and anomaly detectionmodulesSupporting incremental HMM learning (performing statistical teststo select which cluster should be incrementally updated)
8 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Compared to the related works
Potential to improve upon the state of the art in intelligent videosurveillance applications by
Bootstrapping human behavior classification and anomaly detectionmodulesSupporting incremental HMM learning (performing statistical teststo select which cluster should be incrementally updated)
8 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Outline
1 Introduction
2 Human Behavior Pattern Clustering
3 Experimental Results
4 Conclusion
9 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview
We divide the proposed method into 2 phases.
1 Blob extraction
2 Behavior clustering
10 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Extraction
Video
Foreground
Extraction
Background
Modeling
Single�Blob
Tracking
List�of�blobs
Vector
Quantization
Blob�features
CCTV�camera
Discrete�symbolsequences
Backgroundmodel
Sequence
Aggregation
Observationsymbols
Figure: Block Diagram of Blob Extraction
11 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Extraction (cont.)
We represent a blob at time t by the feature vector
~ft =[xt yt st rt dxt dyt vt
],
where
(xt , yt) is the centroid of the blob.
st is the size of the blob in pixels.
rt is the aspect ratio of the blob’s bounding box.
(dxt , dyt) is the unit-normalized motion vector for the blobcompared to the previous frame.
vt is the blob’s speed compared to the previous frame.
12 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering
Similarity
Measurement
Discrete�symbol
sequences
Agglomerative
Hierarchical�Clustering
Distancematrix
Dendrogram
HMM-based
Hierarchical�Clustering
Set�of�HMMs
Figure: Block Diagram of Behavior Clustering
13 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering (cont.)
Is�the�HMMa�sufficient�modelof�the�sequences�
in�c?
Is�C�empty?
c����cluster�at�rootof�dendrogram
Train�a�HMMon�the�sequences
in�c
No
Yes
Yes
C���{��}
Add�the�trained�HMMto�model�list�M
Replace�c�in�Cwith�the�children
of�c�from�DTW�dendrogram
0
0
c����any�clusterin�C
Remove�c�from�C
No
c
Figure: Processing flow of the use of HMM clustering method
14 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering (cont.)
How the processing flow works
Root
15 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering (cont.)
How the processing flow works
Root
The HMM is sufficient?
15 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering (cont.)
How the processing flow works
Root
Not sufficient
15 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering (cont.)
How the processing flow works
Root
Child Child
15 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Behavior Clustering (cont.)
How the processing flow works
Root
Child Child
The HMM is sufficient?
The HMM is sufficient?
15 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
The HMM is not sufficient to model the sequences
When there are more than N sequences in a cluster whoseper-observation log-likelihood is less than a threshold.
16 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
The HMM is not sufficient to model the sequences
When there are more than N sequences in a cluster whoseper-observation log-likelihood is less than a threshold.
16 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
To determine the optimal rejection threshold
Use an approach similar to that of Oates et al. (2001).Generate random sequences from the HMM.Calculate µc and σc of the per-observation log-likelihood over theset of generated sequences.Let a threshold be pc = µc − zσc , where z is experimentally tuned.
17 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
To determine the optimal rejection threshold
Use an approach similar to that of Oates et al. (2001).Generate random sequences from the HMM.Calculate µc and σc of the per-observation log-likelihood over theset of generated sequences.Let a threshold be pc = µc − zσc , where z is experimentally tuned.
17 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
To determine the optimal rejection threshold
Use an approach similar to that of Oates et al. (2001).Generate random sequences from the HMM.Calculate µc and σc of the per-observation log-likelihood over theset of generated sequences.Let a threshold be pc = µc − zσc , where z is experimentally tuned.
17 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
To determine the optimal rejection threshold
Use an approach similar to that of Oates et al. (2001).Generate random sequences from the HMM.Calculate µc and σc of the per-observation log-likelihood over theset of generated sequences.Let a threshold be pc = µc − zσc , where z is experimentally tuned.
17 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Blob Clustering (cont.)
To determine the optimal rejection threshold
Use an approach similar to that of Oates et al. (2001).Generate random sequences from the HMM.Calculate µc and σc of the per-observation log-likelihood over theset of generated sequences.Let a threshold be pc = µc − zσc , where z is experimentally tuned.
17 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Outline
1 Introduction
2 Human Behavior Pattern Clustering
3 Experimental Results
4 Conclusion
18 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview
Recorded videos at a resolution of 320× 240 and 25 fps over 1 week.
Used a motion detection to save disk space.
Obtained videos corresponding to over 500 motion events, butselected the 298 videos containing only a single motion.
19 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Found that at least 4 common behaviors:
Walking into the building (Walk-in)Walking out of the building (Walk-out)Parking a bicycle (Cycle-in)Riding a bicycle out (Cycle-out)
Other less common activities:
Walking while telephoning, etc. (Other)
20 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Figure: Example of common human activities in our testbed scene. (a)Walking in. (b) Walking out. (c) Cycling in. (d) Cycling out.
21 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Our main hypothesis
Using DTW as a pre-process prior to HMM-based clustering shouldimprove the quality of the clusters in term of separating anomalousfrom typical behaviors.
Compared to
Using only HMMsSupervised classification with HMMs
22 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Our main hypothesis
Using DTW as a pre-process prior to HMM-based clustering shouldimprove the quality of the clusters in term of separating anomalousfrom typical behaviors.
Compared to
Using only HMMsSupervised classification with HMMs
22 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Our main hypothesis
Using DTW as a pre-process prior to HMM-based clustering shouldimprove the quality of the clusters in term of separating anomalousfrom typical behaviors.
Compared to
Using only HMMsSupervised classification with HMMs
22 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Our main hypothesis
Using DTW as a pre-process prior to HMM-based clustering shouldimprove the quality of the clusters in term of separating anomalousfrom typical behaviors.
Compared to
Using only HMMsSupervised classification with HMMs
22 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Performed 3 experiments.
Evaluated the results according to how well the induced categoriesseparate the anomalous sequences (hand-labeled with the category“Other”) from typical sequences (Walk-in, Walk-out, Cycle-in,Cycle-out).
1 Using our proposed method2 Using only HMMs3 Using HMMs with supervised learning
In all 3 experiments, we chose linear HMMs with 4 states based onour previous empirical experience.
23 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Performed 3 experiments.
Evaluated the results according to how well the induced categoriesseparate the anomalous sequences (hand-labeled with the category“Other”) from typical sequences (Walk-in, Walk-out, Cycle-in,Cycle-out).
1 Using our proposed method2 Using only HMMs3 Using HMMs with supervised learning
In all 3 experiments, we chose linear HMMs with 4 states based onour previous empirical experience.
23 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Performed 3 experiments.
Evaluated the results according to how well the induced categoriesseparate the anomalous sequences (hand-labeled with the category“Other”) from typical sequences (Walk-in, Walk-out, Cycle-in,Cycle-out).
1 Using our proposed method2 Using only HMMs3 Using HMMs with supervised learning
In all 3 experiments, we chose linear HMMs with 4 states based onour previous empirical experience.
23 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Performed 3 experiments.
Evaluated the results according to how well the induced categoriesseparate the anomalous sequences (hand-labeled with the category“Other”) from typical sequences (Walk-in, Walk-out, Cycle-in,Cycle-out).
1 Using our proposed method2 Using only HMMs3 Using HMMs with supervised learning
In all 3 experiments, we chose linear HMMs with 4 states based onour previous empirical experience.
23 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Performed 3 experiments.
Evaluated the results according to how well the induced categoriesseparate the anomalous sequences (hand-labeled with the category“Other”) from typical sequences (Walk-in, Walk-out, Cycle-in,Cycle-out).
1 Using our proposed method2 Using only HMMs3 Using HMMs with supervised learning
In all 3 experiments, we chose linear HMMs with 4 states based onour previous empirical experience.
23 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Performed 3 experiments.
Evaluated the results according to how well the induced categoriesseparate the anomalous sequences (hand-labeled with the category“Other”) from typical sequences (Walk-in, Walk-out, Cycle-in,Cycle-out).
1 Using our proposed method2 Using only HMMs3 Using HMMs with supervised learning
In all 3 experiments, we chose linear HMMs with 4 states based onour previous empirical experience.
23 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Overview (cont.)
Our configuration
the number of deviant patterns allowed in a cluster N = 10
z = 2.0 for a threshold pc = µc − zσc
24 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Results for Experiment I
Clustering results for Experiment I (DTW+HMMs).
Cluster # Walk-in Walk-out Cycle-in Cycle-out Other
1 96 0 18 0 0
2 0 54 0 5 0
3 0 3 0 8 0
4 0 2 0 0 0
5 0 1 0 2 0
...
14 0 0 0 0 4
15 0 0 0 0 4
16 0 0 0 0 2
17 0 0 0 0 2
One-seqclusters 4 17 34 21 4
25 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Results for Experiment II
Begin by training a single HMM on all sequences.
Assign every sequence with a per-observation log-likelihood above athreshold pc to a cluster.
Repeat the process by training a new HMM on the remainingsequences.
Clustering results for Experiment II (HMMs only).
Cluster # Walk-in Walk-out Cycle-in Cycle-out Other
1 15 77 49 43 16
2 80 0 11 2 0
3 5 0 0 0 0
26 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Results for Experiment III
Trained 4 HMMs on each of the four typical beahviors.
Maximize the F1 value to determine the best per-observationlog-likelihood threshold for each HMM.
For the best separation between the positive and negative testpatterns
Results for Experiment III (Supervised classification with HMMs).
Anomaly detection rate (%) False alarm rate (%)
50 24.6
27 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Outline
1 Introduction
2 Human Behavior Pattern Clustering
3 Experimental Results
4 Conclusion
28 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Conclusion
We have proposed and evaluated a new method for clusteringhuman behaviors.
Our method
provides an initial partitioning of a set of behavior sequences, thenautomatically identifies where to cut off the hierarchical clusteringdendrogram.could be used to bootstrap an anomaly detection module forintelligent video surveillance applications.shows a perfect separation between typical and anomalous behaviorson real-world surveillance data without any information about thelabels.
29 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Conclusion
We have proposed and evaluated a new method for clusteringhuman behaviors.
Our method
provides an initial partitioning of a set of behavior sequences, thenautomatically identifies where to cut off the hierarchical clusteringdendrogram.could be used to bootstrap an anomaly detection module forintelligent video surveillance applications.shows a perfect separation between typical and anomalous behaviorson real-world surveillance data without any information about thelabels.
29 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Conclusion
We have proposed and evaluated a new method for clusteringhuman behaviors.
Our method
provides an initial partitioning of a set of behavior sequences, thenautomatically identifies where to cut off the hierarchical clusteringdendrogram.could be used to bootstrap an anomaly detection module forintelligent video surveillance applications.shows a perfect separation between typical and anomalous behaviorson real-world surveillance data without any information about thelabels.
29 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Conclusion
We have proposed and evaluated a new method for clusteringhuman behaviors.
Our method
provides an initial partitioning of a set of behavior sequences, thenautomatically identifies where to cut off the hierarchical clusteringdendrogram.could be used to bootstrap an anomaly detection module forintelligent video surveillance applications.shows a perfect separation between typical and anomalous behaviorson real-world surveillance data without any information about thelabels.
29 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System
Introduction Human Behavior Pattern Clustering Experimental Results Conclusion
Conclusion
We have proposed and evaluated a new method for clusteringhuman behaviors.
Our method
provides an initial partitioning of a set of behavior sequences, thenautomatically identifies where to cut off the hierarchical clusteringdendrogram.could be used to bootstrap an anomaly detection module forintelligent video surveillance applications.shows a perfect separation between typical and anomalous behaviorson real-world surveillance data without any information about thelabels.
29 / 29
Clustering Human Behaviors with Dynamic Time Warping and Hidden Markov Models for a Video Surveillance System