1 2012 IEEE International Conference on Multimedia and Expo.
-
Upload
heather-waters -
Category
Documents
-
view
222 -
download
0
description
Transcript of 1 2012 IEEE International Conference on Multimedia and Expo.
1
Video Based Real-world Remote Target Tracking On Smartphones
Qia WangAlex Lobzhanidze
Hyun I. Jang Wenjun Zeng
Yi ShangDept. of Computer Science, University of Missouri,
Columbia
2012 IEEE International Conference on Multimedia and Expo
2
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
3
Introduction Moving object tracking is an important problem in many
video applications, such as surveillance and traffic management.
Limitations
a) object tracking is limited to the image plane, not in the physical world [4].
b) it assumes static background [1].
c) it incurs high computational complexity.[6]
[1] R. Rosales and S. Sclaroff, “3D Traject Recovery for Tracking Multiple Objects and Trajectory Guided Recognition of Actions,”[4] S. Weng, C. Kuo and S. Tu, “Video object tracking using adaptive Kalman filter,”[6] M. Roh, T. Kim, J. Park and S. Lee, “Accurate object contour tracking based on boundary edge selection”
4
Introduction A new tracking algorithm and system for use on mobile
smartphones
Light-weight computing
due to the limited computation power of smartphones, complex algorithms requiring high time complexity and space complexity will not fit.
Reasonable accuracy
to estimate a remote target positionbased on video, an accurate tracking with the object boundaryappropriately identified is very important.
Interactive user interfaceInput from user’s interaction is a unique feature on smartphones and is very useful for video tracking.
5
Introduction With our system, a user holds the smartphone and takes
a short video clip of the remote moving object, then simply draws a rough bounding box enclosing the remote target on the first frame.
Our algorithm then derives a tight bounding box around the target object in all video frames.
Assuming the physical width and height of the remote target is known, we then borrow the idea from the POSTIT algorithm [13]: the remote target position could be calculated based on the four 3-D points of the target and their four correspondences on the image.
[13] D. F. Dementhon and L. S. Davis , “Model-based object pose in 25 lines of code,”
6
Introduction We then use the Extended Kalman Filtering (EKF) model
[12] to estimate a smooth trajectory and velocity of the moving target.
While the video is being taken, the smartphone’s GPS coordinates and digital compass readings are recorded so that the moving object’s trajectory could be drawn on the map.
7
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
8
System Overview
9
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
10
Target Object Tracking Our goal is to generate a tight bounding box around the
target for all frames in the video to facilitate subsequent remote target position estimation.
Optical Flow Tracking [18]
Patch Classification
[18] B. D. Lucas and T. Kanade, “An interactive image registration technique with an application to stereo vision”.
11
Target Object TrackingOptical Flow Tracking Object motion and background motion are easier to
estimate with the help of the optical flow features.
shows the moving object’s optical flow features.
Variance (noise)
12
Target Object TrackingPatch Classification To obtain a tight and accurate bounding box, we need to
find the object’s canny edges that belong to the object boundary. Canny edges are detected first and added as a mask on top of the cropped image as shown in Figure 3.
13
Target Object Tracking Then intensity-based segmentation is performed to get
different intensity clusters.background patches object patches
14
Target Object Tracking The Bayesian rule is then applied to determine which
category a patch belongs to.
For patch , its prior probability P(B) of belonging to the background and probability P(O) of belonging to the object should be equal if no other information is available.
15
Target Object Tracking
position of a pixel from patch L’s gray scale intensity value at frame t
motion compensated prediction residual errors
16
Target Object Tracking
threshold value
17
Target Object Tracking Patches located along the sides of the bounding box
have P(B) P(O)
Patches located around the geo-center of the bounding box have
P(O) P(B)
18
Target Object Tracking To obtain a tight bounding box, all we need is to detect
the canny edges belonging to the object. K-Nearest Neighbor algorithm For each canny edge pixel , find its neighborhood pixels
whose distance to is less than R. (R = 3) will be treated as a canny edge pixel belonging to the
object.
threshold value
19
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
20
Remote Target Position Estimation
21
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
22
Extended Kalman Filtering To estimate the moving object’s true moving trajectory.
�⃗�=(𝑎𝑥
𝑎𝑦)
�⃗�𝑡+1=(𝑝𝑜𝑠𝑥+𝑣𝑒𝑙𝑥∆𝑇+ 1
2𝑎𝑥∆𝑇 2
𝑝𝑜𝑠 𝑦+𝑣𝑒𝑙𝑦∆𝑇+ 12𝑎𝑦 ∆𝑇
2
𝑣𝑒𝑙𝑥+𝑎𝑥∆𝑇𝑣𝑒𝑙𝑦+𝑎𝑦∆𝑇
)
23
Kalman gain
Correct
State predictionCovariance prediction
Prediction
Correction
Extended Kalman Filtering
noise
measurement error𝐶=(1 0 0 0
0 1 0 0)
Observation model
Observation matrix
24
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
25
Implementation
26
Implementation
27
Implementation
a: Optical Flow, b:Canny Edge, c: Segmentation,
d:Patch Motion Estimation, e:Connected Component Analysis
28
Implementation
29
Implementation
30
Outline Introduction System Overview Target Object Tracking Remote Target Position Estimation Extended Kalman Filtering Implementation Conclusion
31
Conclusion A novel, easy-to-use, video-based remote target
tracking system on commodity smartphones is presented.
A new light-weight video object tracking algorithm is developed specifically for the smartphone platform, by taking into account smartphone’s computation limitation as well as taking advantage of its user-friendly interface.