Robust Video Surveillance for Fall Detection Based on Human Shape Deformation
description
Transcript of Robust Video Surveillance for Fall Detection Based on Human Shape Deformation
![Page 1: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/1.jpg)
Robust Video Surveillance for Fall Detection Based on Human Shape Deformation
Caroline Rougier, Jean Meunier, Alain St-Arnaud, and Jacqueline RousseauIEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 21, NO. 5, MAY 2011
![Page 2: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/2.jpg)
outline
Introduction Our System and data set Falls Characteristics
Shape deformation▪ mean matching cost▪ full Procrustes distance
Fall Detection Using GMM Experimental Results Conclusion
![Page 3: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/3.jpg)
Introduction (1/2) Establish new healthcare systems to
ensure the safety of elderly people at home. Falls are one of the major risks for old
people living alone. Fall detection wearable sensor:
Accelerometers or help buttonsProblem:-forget to wear-unconscious after the fall-recharged regularly
![Page 4: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/4.jpg)
Introduction (2/2) Computer vision systems has overcome
these problems. A camera provides a vast amount of
information on his/her environment▪ Monocular Systems▪ Bounding box[8]▪ Only placed sideways▪ Occluding objects
▪ Multi-Camera Systems▪ Auvinet et al.[17] reconstructed 3-D silhouette of an
elderly person ▪ Need to be calibrated▪ The video sequences need to be synchronize
![Page 5: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/5.jpg)
Our System and data set (1/2)
Uncalibrated multi-camera system
Low-cost IP cameras, 30 frames/s, 720 × 480 pixels
Wide angle to cover all the room
![Page 6: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/6.jpg)
Our System
and data set
Total of 75 different events , more than 12 min
![Page 7: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/7.jpg)
Falls Characteristics
1. Lack of significative movement2. A lying position3. A person lying on the ground4. Vertical speed5. An impact shock6. Body shape change
![Page 8: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/8.jpg)
Silhouette Edge Point Extraction The silhouette is extracted by a background
subtraction
N = 250 landmarks * Canny edge detector[25]
![Page 9: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/9.jpg)
Matching Using Shape Context (1/2) Shape context[20] is a way of
describing shapes.
Matching cost for pair (pi, qj):
, K=5*12 bins
![Page 10: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/10.jpg)
Matching Using Shape Context (2/2) Minimizing the total matching cost given a
permutation π (i)
Use the Hungarian algorithm[27] for bipartite matching Time complexity: O(n^3) Bad landmarks due to segmentation errors or partial occlusions ▪ Add dummy points (not easy to choose).▪ Match only the most reliable points in our implement (mini Cij = minj Cij)
mean matching cost: i j
bipartite graph
N∗: the total number of best matching points.
![Page 11: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/11.jpg)
Procrustes analysis
Procrustes analysis [21] has been widely used to compare shapes. Detect abnormal shape deformation for fall
detection▪ Step1 : image registration(one translation, no rotation,
no scaling)▪ Step2: Compute full Procrustes distance for compare.
centered landmarks Zc :
1
11
kl
Z
Zc
two centered vectors : v = (v1, · · · , vk)w = (w1, · · ·,wk).
full Procrustes distance :
![Page 12: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/12.jpg)
Fall feature
mean matching cost full Procrustes distance Consider 2 feature (F1, F2)
CfD
1) F1 representing the fall : F1 will high in case of fall
2) F2 representing the lack of significative movement after the fall : A period (t+1s to 5s) will low
![Page 13: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/13.jpg)
Fall Detection Using GMM Model normal activity data with a Gaussian Mixture
Model(GMM). GMM: weighted sum of Gaussian(normal) distributions
M : the number of components in the mixture P (j) : the mixing coefficients The jth Gaussian probability density function p (x | j)
▪ d is the dimensionality of the input space
expectation-maximization (EM)algorithm by maximizing the data likelihood
GMM Classifier : only tell normal or abnormal!
![Page 14: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/14.jpg)
Training and test the dataset Leave-One-Out Cross-Validation
1. Divided the dataset into N video sequences2. One sequence is removed3. Training using the N − 1 remaining sequences
(falls are deleted)4. This sequence is classified with the resulting
GMM.5. Repeat N times6. Count the number of errors, classification
error rate
![Page 15: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/15.jpg)
GMM Classifier Analysis
1. True Positives (TP): falls correctly detected;2. False Negatives (FN): falls not detected;3. False Positives (FP): normal activities detected
as a fall;4. True Negatives (TN): normal activities not
detected as a fall;5. Sensitivity: Se = TP/ (TP + FN);6. Specificity: Sp = TN/ (TN + FP);7. Accuracy: Ac = (TP+TN) / (TP+TN+FP+FN) ;8. Classification error rate: Er = (FN+FP) /
(TP+TN+FP+FN) .
![Page 16: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/16.jpg)
Experimental Results Shape matching : C++ using the OpenCV
library [33] Fall detection : MATLAB using the NETLAB
toolbox [32] to perform the GMM classification.
The original video sequences frame : 30 frames/s 5 frames/s was sufficient to detect a fall Intel Core 2 Duo processor (2.4 GHz) The computational time of the shape matching
step is about 200 ms
![Page 17: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/17.jpg)
Number of GMM Components
train a GMM with three components for our experiment.
![Page 18: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/18.jpg)
Classification Results Normalize training data.
Detection threshold depends on the sensitivity.
![Page 19: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/19.jpg)
Receiver operating characteristic (ROC) analysis
false positives
true positives
![Page 20: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/20.jpg)
Ensemble Classifier
Simply majority vote on all cameras (>= 3 vote) In fig. 9 : error rate 10%2.7%
![Page 21: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/21.jpg)
Comparative Study with Other 2-D Features (1/2)
![Page 22: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/22.jpg)
Comparative Study with Other 2-D Features (2/2)
![Page 23: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/23.jpg)
Occlusions and Other Difficulties
![Page 24: Robust Video Surveillance for Fall Detection Based on Human Shape Deformation](https://reader036.fdocuments.us/reader036/viewer/2022062410/56815ec1550346895dcd47c2/html5/thumbnails/24.jpg)
Conclusion
We presented a new GMM classification method to detect falls By analyzing human shape deformation
Robust to large occlusions and other segmentation difficulties