TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van...
-
Upload
jolie-yearsley -
Category
Documents
-
view
216 -
download
0
Transcript of TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van...
![Page 1: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/1.jpg)
TRACKING THE INVISIBLE:LEARNING WHERE THE OBJECT MIGHT BE
Helmut Grabner1,
Jiri Matas2,
Luc Van Gool1,3,
Philippe Cattin4
1ETH-Zurich,
2Czech Technical University, Prague,
3K.U. Leuven,
4University of Basel
![Page 2: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/2.jpg)
Outline
1. Introduction 2. Learning the Motion Couplings 3. Practical Implementation 4. Experimental Results 5. Conclusion
![Page 3: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/3.jpg)
Outline
1. Introduction
![Page 4: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/4.jpg)
Introduction
Goal: Estimate the Position of an Object
![Page 5: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/5.jpg)
Introduction
Many temporary, but potential very strong links exists
between a tracked object and parts of the images.
We discover the dynamic elements – called SUPPORTERS – that predict the position of the target.
[L. Cerman, J. Matas, J., V. Hlavac, Sputnik Tracker,SCIA, 2009]
[M. Yang, Y. Wu, G. Hua. Context-Aware Visual Tracking PAMI, 2009]
![Page 6: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/6.jpg)
SUPPORTERS
1. Came with different strength. 2. Change over time.
SUPPORTERS help Tracking of objects which change
there appearance very quickly and occluded objects or object outside the image.
![Page 7: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/7.jpg)
Tracking Carl
Given the position of the object of interest in the frame.
Local image features from the whole image are extracted (yellow points).
Image features: our implementation uses Harris points [5] and describe them by a SIFT inspired descriptor [10].
![Page 8: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/8.jpg)
Local Image Features as Supporters
These image features are usually divided into object points and points belonging to the background.
![Page 9: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/9.jpg)
Local Image Features as Supporters
Object points lie on the object surface and thus always have a strong correlation to the object motion (green points)
![Page 10: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/10.jpg)
Local Image Features as Supporters
The object gets occluded or changed its appearance, the supporters can help infer its position.
![Page 11: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/11.jpg)
Discovering the Supporters
![Page 12: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/12.jpg)
Outline
2. Learning the Motion Couplings
![Page 13: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/13.jpg)
Problem Formulation
We want to learn a model of P(x|I), predicting the position x of an object in the image I.
![Page 14: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/14.jpg)
Implicit Shape Model - Features
After training from a large database of labeled images, the model can be used to detect objects from that object class in test images.
indicator functions P(f|I)
![Page 15: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/15.jpg)
Implicit Shape Model – Object Displacement
Each feature subsequently casts probabilistic votes for possible object positions, where the hypothesis score is obtained as a sum over all votes.
P(x|f):votes for the object position x.
![Page 16: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/16.jpg)
Implicit Shape Model
![Page 17: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/17.jpg)
Implicit Shape Model
![Page 18: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/18.jpg)
Implicit Shape Model
![Page 19: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/19.jpg)
Model of Carl – Object Detection
![Page 20: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/20.jpg)
Model of Supporters
![Page 21: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/21.jpg)
Voting
Where μ is the mean and Σ is the covariance matrix.
![Page 22: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/22.jpg)
Reliable Information & Motion Coupling
feature point x(i) = (x(i), y(i))
associated with a feature f(i)
and the position
x* = (x*, y*) of the target.
where parameter α ϵ [0, 1] determines the weighting of the past estimates.
The current image It and the object position Xt* is included using p(f|It), the indicator of feature f in the current image; and p(Xt*-Xt(i)|f), the current relative object position.
![Page 23: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/23.jpg)
Implementation
angle Ф distance r covariance matrix Σ
Store all detected feature points in a database DB. The entries (d ,r ,Ф ,Σ ) store the descriptor and the estimated voting vector, encoded by radius, angle and covariance matrix.
![Page 24: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/24.jpg)
Updates
Adjust the voting vector, the radius r and the relative angle Ф
Update the covariance matrix Σ
![Page 25: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/25.jpg)
Outline
3. Practical Implementation
![Page 26: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/26.jpg)
Algorithm : Determination of Position
![Page 27: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/27.jpg)
Outline
4. Experimental Results
![Page 28: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/28.jpg)
ETH-Cup: Supporters
![Page 29: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/29.jpg)
[11] S. Stalder, H. Grabner, and L. V. Gool. Beyond semisupervised tracking: Tracking should be as simple as detection,but not simpler than recognition. In Proc. IEEE WS onOn-line Learning for Computer Vision, 2009.
![Page 30: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/30.jpg)
Experimental Results
![Page 31: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/31.jpg)
ETH-Cup: Improving Object Tracking
Recall:
TP/(TP+FN) Precision:
TP/(TP+FP)
![Page 32: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/32.jpg)
Medical Application
(a) Manually labeled valve positions, (b-f) the found valve positions in 5 cardiac phases. As the confidence of the voting space was too low, in cardiac phase 16, the valve positions were interpolated (indicated by the dotted lines) from neighboring images.
![Page 33: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/33.jpg)
Outline
5. Conclusion
![Page 34: TRACKING THE INVISIBLE: LEARNING WHERE THE OBJECT MIGHT BE Helmut Grabner1, Jiri Matas2, Luc Van Gool1,3, Philippe Cattin4 1ETH-Zurich, 2Czech Technical.](https://reader035.fdocuments.us/reader035/viewer/2022062621/551bf9815503469e4f8b484f/html5/thumbnails/34.jpg)
Conclusion
SUPPORTERS help Tracking of objects which
1.Change there appearance very quickly. 2.Occluded objects or object outside the image.