An Integrated Pose and Correspondence Approach to Image Matching
description
Transcript of An Integrated Pose and Correspondence Approach to Image Matching
![Page 1: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/1.jpg)
An Integrated Pose and Correspondence Approach to Image
Matching
Anand Rangarajan
Image Processing and Analysis GroupDepartments of Electrical Engineering and Diagnostic RadiologyYale University
![Page 2: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/2.jpg)
Motivation I
• Human Brain Mapping:– Different subjects.
• Statistical analysis.
• Normal vs. abnormal.
– Different times.• Detect significant change, help diagnosis.
– Different modalities.• Combine complementary information.
![Page 3: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/3.jpg)
Motivation II
• Difficulty : – Variability in pose, size, shape and acquisition.
• Brain registration : – Common coordinate frame.– Data comparable.– Quantitative analysis.
![Page 4: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/4.jpg)
Results
Interactive 3D Sulcal Tracing
![Page 5: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/5.jpg)
Overview
• Extract features: – Sulcal traces represented as point sets.– Labeling, ordering information [optional].
• Jointly solve feature correspondence and spatial mapping.
![Page 6: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/6.jpg)
Overview II
• Part II: Information Analysis: – Measurements. – Learn from the data, construct statistical
models.• e.g., probabilistic atlas for structures / functions.
– Make inference for new data based on the learned models.
• e.g., automated sulcal labeling, segmentation, computer aided diagnosis.
![Page 7: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/7.jpg)
Outline
• Related work.
• The approach.– Point-based representation of sulci.– Robust point matching algorithm.
• Results and examples.
• Future work.
![Page 8: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/8.jpg)
Other Work in Brain Registration
• Voxel-based methods:– Volumetric Warping: Christensen et al., Gee et
al., Collins et al.
• Feature-based methods: – Landmarks: Bookstein.– Curves: Sandor and Leahy, Collins et al.– Surfaces: Thompson et al., Davatzikos et al. – Sulcal Graphs: Lohmann and von Cramon.
![Page 9: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/9.jpg)
Approach Rationale
• Voxel intensity matching does not ensure that corresponding sulci indeed match.
• Landmarks hard to define.
• Extraction, representation and matching of cortical curves / surfaces / graphs is difficult.
![Page 10: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/10.jpg)
Our Approach
Point-based Representation• Hundreds of points, statistically more
robust than just a few landmarks.
• Additional information can be used:– Major sulcal labels.
• Further analyses made easy:– Procrustes mean. – Eigen-analysis of the error covariance matrix.
![Page 11: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/11.jpg)
Our Approach
Robust Point Matching (RPM)
• Estimation : – Correspondence and spatial mapping.
• Softassign:– Soft correspondence.– Allows partial matching, noise.– Less sensitive to local minima.
• Handles outliers.
![Page 12: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/12.jpg)
Robust Point Matching
Alternating Optimization
• When correspondence M is known, standard least squares solution for spatial mapping A.
• When spatial mapping A is fixed, assignment solution for correspondence M.– Softassign - soft correspondence.– Deterministic Annealing - temperature T.
![Page 13: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/13.jpg)
Robust Point Matching Energy Function
![Page 14: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/14.jpg)
Robust Point Matching
Step I. Solve Spatial Mapping
• Given correspondence M, find the optimal spatial mapping A (affine):
• Standard least-squares solution.
• Gradually relaxed regularization on
![Page 15: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/15.jpg)
Robust Point Matching
Part II. Softassign
• Given spatial mapping A, solve the Linear Assignment Problem:
subject to
![Page 16: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/16.jpg)
Robust Point Matching
Step II. Softassign
Two-way constraints
M ij
M ij
M iji
Row Normalization
M ij
M ij
M ijj
Col. Normalization
Positivity
=exp( )QijM ij
•Step I: Mij = exp ( - Qij/T).
•Step II: Double Normalization. Sinkhorn’s Algorithm.
Outlier rejection using slack variables.
![Page 17: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/17.jpg)
Robust Point Matching Part II. Softassign
• Deterministic Annealing :– T as an extra parameter.– F = Eassign - TS =
• Gibbs Distribution :– Positivity ganranteed.– High T, insensitive to Q, uniform M .– Low T, sensitive to Q, binary M .
![Page 18: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/18.jpg)
Robust Point Matching Algorithm Summary
• Start: uniform M, high temperature T.
• Do until final temperature is reached.– Given M, solve for spatial mapping A.– Given A, use Softassign to update M.
• Decrease temperature.
![Page 19: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/19.jpg)
Experiment on Brain Sections
![Page 20: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/20.jpg)
Results of Method
![Page 21: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/21.jpg)
Results
Interactive 3D Sulcal Tracing
![Page 22: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/22.jpg)
Results
RPM Example
Two labeled sulcal point sets, initial position.
![Page 23: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/23.jpg)
RPM without label information
![Page 24: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/24.jpg)
Results
Visual Matching Comparison
![Page 25: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/25.jpg)
Results
Visual Matching Comparison
![Page 26: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/26.jpg)
Quantitative Comparison
![Page 27: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/27.jpg)
Quantitative Comparison
![Page 28: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/28.jpg)
Future Work
• Error measure on the entire volume.
• Fully non-rigid 3D spatial mapping.– Thin-plate spline and correspondence.
• Automated sulcal extraction, Zeng et al.
• Investigate partially labeled case.
• Automated labeling.
• Atlas construction.
![Page 29: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/29.jpg)
The End
![Page 30: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/30.jpg)
Thin-plate-spline Implementation
![Page 31: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/31.jpg)
Thin-plate-spline Implementation
![Page 32: An Integrated Pose and Correspondence Approach to Image Matching](https://reader035.fdocuments.us/reader035/viewer/2022070400/568134e5550346895d9c1bc4/html5/thumbnails/32.jpg)
Results
Visual Matching Comparison
TPS