Reconnaissance d’objetset vision artificielle

Jean Ponce (ponce@di.ens.fr)http://www.di.ens.fr/~ponce

Equipe-projet WILLOWENS/INRIA/CNRS UMR 8548Laboratoire d’Informatique

Ecole Normale Supérieure, Paris

Outline

• Motivation: Making mosaics• Perspetive and weak perspective• Coordinates changes• Intrinsic and extrensic parameters• Affine registration• Projective registration

Feature-based alignment outline

Feature-based alignment outline

Extract features

Feature-based alignment outline

Extract featuresCompute putative matches

Feature-based alignment outline

Extract featuresCompute putative matchesLoop:

• Hypothesize transformation T (small group of putative matches that are related by T)

Feature-based alignment outline

Extract featuresCompute putative matchesLoop:

• Hypothesize transformation T (small group of putative matches that are related by T)

• Verify transformation (search for other matches consistent with T)

Feature-based alignment outline

Extract featuresCompute putative matchesLoop:

• Hypothesize transformation T (small group of putative matches that are related by T)

• Verify transformation (search for other matches consistent with T)

2D transformation models

Similarity(translation, scale, rotation)

Affine

Projective(homography)

Why these transformations ???

Pinhole perspective equation

zyfy

zxfx

''

''NOTE: z is always negative..

Affine models: Weak perspective projection

0

'where''

zfm

myymxx

is the magnification.

When the scene relief is small compared its distance from theCamera, m can be taken constant: weak perspective projection.

Affine models: Orthographic projection

yyxx

'' When the camera is at a

(roughly constant) distancefrom the scene, take m=1.

Analytical camera geometry

Coordinate Changes: Pure Translations

OBP = OBOA + OAP , BP = AP + BOA

Coordinate Changes: Pure Rotations

BABABA

BABABA

BABABABAR

kkkjkijkjjjiikijii

.........

TB

A

TB

A

TB

A

kji

ABA

BA

B kji

Coordinate Changes: Rotations about

the z Axis

1000cossin0sincos

RBA

A rotation matrix is characterized by the following properties:

• Its inverse is equal to its transpose, and

• its determinant is equal to 1.

Or equivalently:

• Its rows (or columns) form a right-handedorthonormal coordinate system.

Coordinate changes: pure rotations

PRP

zyx

zyx

OP

ABA

B

B

B

B

BBBA

A

A

AAA

kjikji

Coordinate Changes: Rigid Transformations

ABAB

AB OPRP

11111P

TOPRPORP A

BA

ABAB

AA

TA

BBA

B

0

Pinhole perspective equation

zyfy

zxfx

''

''NOTE: z is always negative..

The intrinsic parameters of a camera

Normalized imagecoordinates

Physical image coordinates

Units:k,l : pixel/mf : ma,b : pixel

The intrinsic parameters of a camera

Calibration matrix

The perspectiveprojection equation

The extrinsic parameters of a camera

Perspective projections induce projective transformations between planes

Weak-perspective projection

Paraperspective projection

Affine cameras

Orthographic projection

Parallel projection

More affine cameras

Weak-perspective projection model

r(p and P are in homogeneous coordinates)

p = A P + b (neither p nor P is in hom. coordinates)

p = M P (P is in homogeneous coordinates)

Affine projections induce affine transformations from planes onto their images.

Affine transformationsAn affine transformation maps a parallelogram ontoanother parallelogram

11001''

22221

11211

vu

baabaa

vu

Fitting an affine transformationAssume we know the correspondences, how do we get

the transformation?

2

1

43

21

tt

yx

mmmm

yx

i

i

i

i

i

i

ii

ii

yx

ttmmmm

yxyx

2

1

4

3

2

1

10000100

),( ii yx ),( ii yx

Fitting an affine transformation

Linear system with six unknownsEach match gives us two linearly independent

equations: need at least three to solve for the transformation parameters

i

i

ii

ii

yx

ttmmmm

yxyx

2

1

4

3

2

1

10000100

Beyond affine transformationsWhat is the transformation between two views of a

planar surface?

What is the transformation between images from two cameras that share the same center?

Perspective projections induce projective transformations between planes

Beyond affine transformationsHomography: plane projective transformation

(transformation taking a quad to another arbitrary quad)

Fitting a homographyRecall: homogenenous coordinates

Converting to homogenenousimage coordinates

Converting from homogenenousimage coordinates

Fitting a homographyRecall: homogenenous coordinates

Equation for homography:

Converting to homogenenousimage coordinates

Converting from homogenenousimage coordinates

11 333231

232221

131211

yx

hhhhhhhhh

yx

Fitting a homographyEquation for homography:

iT

T

T

ii xhhh

xHx

3

2

1

11 333231

232221

131211

i

i

i

i

yx

hhhhhhhhh

yx

0 ii xHx

iT

iiT

i

iT

iiT

iT

iT

i

ii

yxx

y

xhxhxhxhxhxh

xHx

12

31

23

00

00

3

2

1

hhh

xxxxxx

TTii

Tii

Tii

TTi

Tii

Ti

T

xyxy

3 equations, only 2 linearly independent

9 entries, 8 degrees of freedom(scale is arbitrary)

Direct linear transform

H has 8 degrees of freedom (9 parameters, but scale is arbitrary)

One match gives us two linearly independent equationsFour matches needed for a minimal solution (null space

of 8x9 matrix)More than four: homogeneous least squares

0

00

00

3

2

1111

111

hhh

xxxx

xxxx

Tnn

TTn

Tnn

Tn

T

TTT

TTT

xy

xy

0hA

Application: Panorama stitching

Images courtesy of A. Zisserman.