Human Action Recognition Based on Spacio-temporal features
-
Upload
nikhilus85 -
Category
Documents
-
view
1.221 -
download
8
description
Transcript of Human Action Recognition Based on Spacio-temporal features
![Page 1: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/1.jpg)
Human Action Recognition Based on Spatio-temporal Features
Nikhil Sawant and Dr. K.K. BiswasDept. of CSE, Indian Institute of Technology, Delhi
Third International Conference on Pattern Recognition and Machine Intelligence (PReMi’09)
![Page 2: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/2.jpg)
Human activity recognitionH
ighe
r res
oluti
on
Longer Time Scale
Courtesy : Y. Ke, Fathi and Mori, Bobick and Davis, Schuldt et al, Leibe et al, Vaswani et al.
Pose Estimation
Event Detection
Action Classification
Tracking
Activity Recognition
![Page 3: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/3.jpg)
Use Action recognition?
• Video surveillance
• Interactive environment
• Video classification & indexing
• Movie search
• Assisted Care
• Sports annotation
![Page 4: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/4.jpg)
Broad outline of our techniqueVideo with
human actions
Motions features
Shape features
Spatio-temporal features
Learning features though AdaBoos
Action Class 1 Action Class 2 Action Class 3 Action Class n
Motion analysis using Lucas –kanade
technique
Shape analysis using Viola-Jones feaures
Combining motion and shape features over finite time interval
……………......
![Page 5: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/5.jpg)
Target Localization
• Possible search space is xyt cube• Action needs to be localized in space and time• Target localization helps reducing search space• Background subtraction• ROI marked
Original Video Silhouette Original Video with ROI marked
![Page 6: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/6.jpg)
Motion estimation
• Make use of optical flows for motion estimation
• Optical flow is the pattern of relative motion between the object/object feature points and the viewer/camera
• We make use of Lucas – Kanade, two frame differential method, it comparatively yields robust and dense optical flows
![Page 7: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/7.jpg)
Noise Reduction
• Noise removal by averaging
• Optical flows with magnitude > C * Omean are ignored,
where C – constant [1.5 - 2], Omean - mean of optical flow within ROI
![Page 8: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/8.jpg)
Organizing optical flows
• Optical flows are aggregated near the motion
• Need for representing optical flow in meaningful way
• Fixed sized grid laid over the ROI
![Page 9: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/9.jpg)
• Magnitude and direction of Optical flows within each box bij is averaged and assigned to its centre cij
• All optical flows have same weight
Organizing optical flows (simple averaging)
![Page 10: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/10.jpg)
• Each optical flow given a weight
• More the distance from the centre cij less is the weight and vice-versa
Organizing optical flows (weighted averaging)
![Page 11: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/11.jpg)
• Optical flows are arranged in structured mannered
• Arranged optical flows are easier to analyze
Organizing optical flows
![Page 12: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/12.jpg)
Shape discriptor
• Shape gives information about the action
• Viola-Jones box features used to get shape features
• Shape information combined with motion information
![Page 13: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/13.jpg)
Spatio-temporal descriptor
TLEN
TSPAN
![Page 14: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/14.jpg)
Spatio-temporal descriptor
• Shape and motion features combined over the span of time to form spatio-temporal features
![Page 15: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/15.jpg)
Learning with Adaboost
• Adaboost is state of art learning algorithm
• Linear decision stumps are used as weak hypothesis
• Weak hypothesis combine to form a strong hypothesis
• Strong hypothesis is weighted sum of weak hypothesis
• Training and testing data is kept mutually exclusive
![Page 16: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/16.jpg)
Results
![Page 17: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/17.jpg)
Results (Weizman dataset)
![Page 18: Human Action Recognition Based on Spacio-temporal features](https://reader033.fdocuments.us/reader033/viewer/2022061208/54881b75b47959e20c8b5685/html5/thumbnails/18.jpg)
Thanks You