18th August 2006International Conference on Pattern Recognition 2006

Epipolar Geometry from Epipolar Geometry from Two CorrespondencesTwo Correspondences

Michal Perďoch, Jiří Matas, Ondřej Chum

Center for Machine Perception

Czech Technical University, Praguehttp://cmp.felk.cvut.cz

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 2/16

Problem formulationProblem formulation

Problem: Wide-baseline stereo with severe viewpoint and scale change.

Message of this paper: Two correspondences of Local Affine Frames suffice to estimate epipolar geometry.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 3/16

Local Affine Frame Local Affine Frame correspondencescorrespondencesLocal Affine Frames (LAFs), Matas ICPR2002:

Affine-covariantly detected local coordinate systems. One matching LAF pair provides three point-to-point

correspondences.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 4/16

The correspondence The correspondence problemproblem

Wide-baseline stereo framework (Matas ICPR2002):

1. Detect regions. MSERs are

used here.

2. Build Local Affine Frames,

define Measurement Region.

3. Geometric and photometric

normalisation of MRs.

4. Establish correspondences

by nearest neighbour

search.

5. Verify correspondences,

estimate model using

RANSAC.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 5/16

How to find correct model of EG in presence of outliers? RANSAC - widely used robust estimator proposed by

Fishler and Bolles 1988. Hypothesise & verify search:

RANSAC AlgorithmRANSAC Algorithm

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 6/16

Average number of RANSAC samples k:

Example (= 0.01):

Chum DAGM2003, has shown that the theoretical number of samples can be achieved by introducing Local Optimisation step.

RANSAC AlgorithmRANSAC Algorithm

)log(1

log 0m

k

– probability of finding a better model

– proportion of inliers

– size of the sample

0

m

Proportion of inliers

m 50% 45% 40% 35% 30% 25% 20% 15%

2 17 21 27 36 49 72 113 203

3 35 49 70 106 169 293 574 1363

…

7 588 1231 2809 7156 21055 75449 359777 2695297

2 LAFs2 LAFs

3 LAFs3 LAFs

7 points7 points

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 7/16

LO-RANSAC AlgorithmLO-RANSAC Algorithm

Proposed by Chum DAGM2003, used with 3LAFs and seven point-to-point correspondences.

Input: set of data points T, confidence .Output: model M* of EG with the largest support S*.

Repeat Draw a random sample of minimal size m from data

points. Compute model parameters Mi and its support Si. If new maximum was detected (i.e., |Si| > |Sj| for (j < i):

Apply local optimisation. Store the best model with support S*.

Until the probability of finding model better than S* falls under 0.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 8/16

Six Point EG estimatorSix Point EG estimator

Stewénius CVPR2005 has shown that EG can be estimated from six points under following assumptions on the cameras: Unit aspect ratio, zero skew, principal point in the

middle of the image. Unknown but the same focal length.

We take Stewénius’ assumptions. Is it possible to estimate EG from two Local Affine Frame

correspondences?

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 9/16

Planar DegeneracyPlanar Degeneracy

Chum CVPR2005 has that planar degeneracy causes suboptimal EG estimates.

Six points on a plane do not provide enough constraints for estimation of the fundamental matrix.

Planar degeneracy test - take sample points and check if they lie on a plane.

When a plane is detected do plane and parallax search: Local optimisation that takes into account plane found. Sample another pair not consistent with the plane and compute

fundamental matrix

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 10/16

2LAF-LO-RANSAC Algorithm2LAF-LO-RANSAC Algorithm

Input: set of data points T, confidence , sample size m = 2.

Output: model M* of EG with largest support S*.

Repeat Draw a random sample of minimal size m from data points. Compute model parameters Mi and its support Si.

If new maximum has occurred (i.e. |Si| > |Sj| for (j < i): If a degenerated sample configuration is detected:

Perform plane-and-parallax search for EG,

otherwise Apply local optimisation.

Store the best model with support S*.

Until the probability of finding model better than S* falls under .

Performed at most log(k) times.

Performed at most log(k) times.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 11/16

ExperimentsExperiments:: Scenes Scenes

Corner= 0.26

The China Wall= 0.28

Wash=

0.23

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 12/16

Experiment 1: EfficiencyExperiment 1: Efficiency

What is the “efficiency” of six point estimator? Is the planar degeneracy test necessary?

Comparison of 2LAF, 3LAF with and without planar degeneracy test and PTS7 algorithm against reference EG.

Number of good EG models generated from 1000 all-inlier samples (quality is measured by number of inliers, good – at least 90%).

Method Corner China Wall Wash

7PTS 785 815 646

3LAF 641 341 353

3LAF ND 458 246 180

2LAF 553 274 247

2LAF ND 362 184 161

ND - without degeneracy test

ND - without degeneracy test

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 13/16

Experiment Experiment 22:: Performance Performance

Average number of inliers and samples measured in hundred runs of 7PTS, 3LAF, 2LAF-LO-RANSAC algorithms.

Small drop (two to four percent) in average number of inliers on difficult and noisy scenes for both 3LAF and 2LAF algorithm.

Note the significant speedup (measured by number of samples) of 2LAF algorithm.

MethodCorner China Wall Wash

#inl #sam #inl #sam #inl #sam

2LAF 160 44 102 47 130 69

3LAF 161 177 104 151 135 258

7PTS 162 42308 106 26396 136 96873

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 14/16

Experiment 3: Assumptions Experiment 3: Assumptions violationviolation The six point solver assumes equal focal length in both

images. The focal length ratio on the scene Corner is 1:3 and

off-plane rotation about 30 degrees. No performance drop was observed.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 15/16

Experiment 3: Assumptions Experiment 3: Assumptions violationviolation Robustness to change of pixel

aspect ratio was measured on scene China Wall (#inliers in 100 runs).

Width of pixel scaled: 0.5, 1.0, 2.0 times from top to bottom.

Perďoch M., Matas J., Chum O.: Epipolar Geometry from Two Correspondences, CTU Prague 16/16

ConclusionsConclusions

Contribution Novel wide-baseline stereo algorithm using two LAF

correspondences was proposed. 6 point (= 2 LAFs) EG estimator can be used in LO-RANSAC

framework with the planar degeneracy test. Performance was experimentally tested and compared to 7PTS

and 3LAF algorithms in wide-baseline stereo setup.

Future work Detect other degenerated configurations. Compare the number of verifications instead of the number of

samples.

Thank you for your attention.Thank you for your attention.