robust feature point matching ohne.ppt [Kompatibilitätsmodus]
Transcript of robust feature point matching ohne.ppt [Kompatibilitätsmodus]
Robust Feature Point MatchingRobust Feature Point MatchingRobust Feature Point Matching Robust Feature Point Matching in General Multiin General Multi--Image SetupsImage Setups
WSCG 2011
A i S ll M i Ei M MAnita Sellent, Martin Eisemann, Marcus MagnorTU Braunschweig, Germany
GeneralGeneral Sets of ImagesSets of ImagesUnsynchronized => no epipolar geometry
GeneralGeneral Sets of ImagesSets of Images
Unordered => no trackingCommon field of view
2Robust Feature Point Matching Anita Sellent
Consistent Feature MatchingConsistent Feature MatchingConsistent Feature MatchingConsistent Feature MatchingSparse featuresFeatures correspond to 3D points => consistency
3Robust Feature Point Matching Anita Sellent
Three Image Optical FlowThree Image Optical FlowDense correspondences
Three Image Optical FlowThree Image Optical Flow
High accuracyBrightness constancySmall displacements
4Robust Feature Point Matching Anita Sellent
[Sellent et al.: Consistent Optical Flow for Stereo Video, ICIP 2010]
OutlineOutlineOutlineOutlineThree Image Feature Matchingg gCombination of Feature Matching and Optical Flow
5Robust Feature Point Matching Anita Sellent
OutlineOutlineOutlineOutlineThree Image Feature MatchingCombination of Feature Matching and Optical Flow
6Robust Feature Point Matching Anita Sellent
Three Image MatchingThree Image MatchingThree Image MatchingThree Image MatchingUnordered, unsynchronized imagesCommon field of viewAny features (SIFT, SURF, FAST, Harris,…)
7Robust Feature Point Matching Anita Sellent
Three Dimensional Assignment Three Dimensional Assignment P blP blProblemProblemSets of equal, finite cardinalityNP-hardApproximation algorithms exist[Crama, Spieksma:Approximation algorithms for the 3 dimensional assignment problem, 1992]
8Robust Feature Point Matching Anita Sellent
Step 1Step 1Step 1Step 1(Robust) nearest neighbor matchingSymmetry
3I
1I 2I
9Robust Feature Point Matching Anita Sellent
Step 2+3Step 2+3Step 2+3Step 2+3Combine distance function
S
),(),(),(ˆ323131 ffdffdffd +=
Symmetry3I
1I 2I
10Robust Feature Point Matching Anita Sellent
Step 4Step 4Step 4Step 4Restart from all directions
11Robust Feature Point Matching Anita Sellent
Step 5Step 5Step 5Step 5Only accept consistent solutions
12Robust Feature Point Matching Anita Sellent
Results: Scene Art with SIFT Results: Scene Art with SIFT FFFeaturesFeatures
Two Image Matching Three Image MatchingTwo Image Matching1444 Matches53 % wrong matches
Three Image Matching603 Matches11 % wrong matches
13Robust Feature Point Matching Anita Sellent
Removed 703 wrong matches at the cost of 138 correct matches
Results: Scene Laundry with Results: Scene Laundry with SURF FSURF FSURF FeaturesSURF Features
Two Image Matching Three Image MatchingTwo Image Matching675 Matches69 % wrong matches
Three Image Matching193 Matches29 % wrong matches
14Robust Feature Point Matching Anita Sellent
Removed 410 wrong matches at the cost of 72 correct matches
Results: Scene Stonemill with Results: Scene Stonemill with H i CH i CHarris CornersHarris Corners
Two Image Matching Three Image MatchingTwo Image Matching225 Matches50 % wrong matches
Three Image Matching133 Matches28 % wrong matches
15Robust Feature Point Matching Anita Sellent
Removed 75 wrong matches at the cost of 17 correct matches
Results: Real World Scenes Results: Real World Scenes i h SIFT Fi h SIFT Fwith SIFT Featureswith SIFT Features
Two Image Matching Three Image MatchingTwo Image Matching Three Image Matching
16Robust Feature Point Matching Anita Sellent
Three Image Feature MatchingThree Image Feature MatchingThree Image Feature MatchingThree Image Feature MatchingNo prior knowledge requiredMatch features consistently on three imagesTrade off number of matches for increased quality of the matchesSparse
17Robust Feature Point Matching Anita Sellent
OutlineOutlineOutlineOutlineThree Image Feature MatchingCombination of Feature Matching and Optical Flow
18Robust Feature Point Matching Anita Sellent
Combination of Feature Combination of Feature M hi d O i l FlM hi d O i l FlMatching and Optical FlowMatching and Optical Flow
Features Optical Flow◦ Sparse◦ Low accuracy
Optical Flow◦ Dense◦ High accuracyy
◦ Large motion◦ Invariant to color
g y◦ Small motion◦ Based on brightness
changes◦ …
gconstancy ◦ …
19Robust Feature Point Matching Anita Sellent
Feature Matches as Motion Feature Matches as Motion P iP iPriorPriorDefine motion prior only where )(xPmatches are foundAssign weight proportional to )(xμmatching costAdd weighted motion prior to optical flow d t tdata-term
20Robust Feature Point Matching Anita Sellent
Results: Results: A E d i EA E d i EAverage Endpoint ErrorAverage Endpoint Error
2 image OF 2 image OF &2 image features
3 image OF 3 image OF &3 image features
Stonemill 4 53 23 16 3 81 3 52Stonemill 4.53 23.16 3.81 3.52Waving 1.03 31.39 0.97 0.92Art 10.62 84.82 9.34 8.70Books 14.60 55.37 6.43 4.85
21Robust Feature Point Matching Anita Sellent
[Sellent et al.: Consistent Optical Flow for Stereo Video, ICIP 2010]
Flow Fields: Scene StonemillFlow Fields: Scene StonemillFlow Fields: Scene StonemillFlow Fields: Scene Stonemill
Ground-truth
2 image OF 2 image OF &2 image features
3 image OF &3 image features
3 image OF
22Robust Feature Point Matching Anita Sellent
Flow Fields: Scene WavingFlow Fields: Scene WavingFlow Fields: Scene WavingFlow Fields: Scene WavingGround-truth
2 image OF 2 image OF &2 image features
3 image OF &3 image features
3 image OF
23Robust Feature Point Matching Anita Sellent
SummarySummarySummarySummaryConsistency on three images can improve sparse and dense correspondences without further knowledge
N lib i◦ No calibration◦ No synchronization
M lti l di i◦ Multiple dimensionsFor high quality feature matches the accuracy of optical flow can be improved byaccuracy of optical flow can be improved by the inclusion of features
24Robust Feature Point Matching Anita Sellent
Future WorkFuture WorkFuture WorkFuture WorkWhat is the best number of images to be used?◦ Number of visible/ detected features◦ Quality of the matching
25Robust Feature Point Matching Anita Sellent
Thank You for Your Attention!Thank You for Your Attention!Thank You for Your Attention!Thank You for Your Attention!
http://graphics.tu-bs.de
This work has been funded by the German Science Foundation
26Robust Feature Point Matching Anita Sellent
This work has been funded by the German Science Foundation, DFG MA2555/4-1
Consistent Optical FlowConsistent Optical FlowConsistent Optical FlowConsistent Optical Flow
Brightness ConstancyBrightness Constancy)()( 2,121 wxIxI +≈
0ˆ22,12,1 ≈−ww
Smoothness Assumption
0ˆ 2,1
r≈∇w
Estimate dense flow between 3 images simultaneously
27Robust Feature Point Matching Anita Sellent
[Sellent et al.: Consistent Optical Flow for Stereo Video, 2010]
Estimate dense flow between 3 images simultaneously
Synchronized CamerasSynchronized CamerasSynchronized CamerasSynchronized Camerassp
ace
time
28Robust Feature Point Matching Anita Sellent
[Hartley, Zisserman: Multiple View Geometry in Computer Vision, 2003]
Unsynchronized CamerasUnsynchronized CamerasUnsynchronized CamerasUnsynchronized Camerassp
ace
time
[Veenman, Reinders, Backer: Establishing Motion Correspondence
29Robust Feature Point Matching Anita Sellent
Using Extended Temporal Scope, 2003]