Projective Geometry and Geometric Invariance in

Computer Vision

Babak N. Araabi

Electrical and Computer Eng. Dept.

University of Tehran

Workshop on image and video processingMordad 13, 1382

2

Do parallel lines intersect?

Perhaps!

3

Fifth Euclid's postulate

For any line L and a point P not on L, there exists a unique line that is parallel to L (never meets L) and passes through P.

At first glance it would seem that the parallel postulate ought to be a theorem deducible from the other more basic postulates.

For centuries mathematicians tried to prove it. Eventually it was discovered that the parallel

postulate is logically independent of the other postulates, and you get a perfectly consistent system even if you assume that parallel postulate is false.

4

Alternative postulates

Non-Euclidean geometry; though not a typical one The projective axiom:

Any two lines intersect (in exactly one point). More intuitive approach:

Take each line of ordinary Euclidean geometry and add to it one extra object called a point at infinity. In addition, the collection of all the extra objects together is also called a line in projective plane (called the line at infinity).

5

Why Projective Geometry?

A more appropriate framework when dealing with projection related issues.

Perspective projection: Photophraphy Human vision

Perspective projection is a non-linear mapping with Euclidean coordinates, but a linear mapping with homogeneous coordinates

6

Overview

2D Projective Geometry 3D Projective GeometryApplication: Invariant matchingFrom 2D images to 3D space

7

Projective 2D Geometry

8

Homogeneous coordinates

0 cbyax Ta,b,c

0,0)()( kkcykbxka TT a,b,cka,b,c ~

Homogeneous representation of lines

equivalence class of vectors, any vector is representative

Set of all equivalence classes in R3(0,0,0)T forms P2

Homogeneous representation of points

0 cbyax Ta,b,cl Tyx,x on if and only if

0l 11 x,y,a,b,cx,y, T 0,1,,~1,, kyxkyx TT

The point x lies on the line l if and only if xTl=lTx=0

Homogeneous coordinates

Inhomogeneous coordinates Tyx,

T321 ,, xxx but only 2DOF

9

Points from lines and vice-versa

l'lx

Intersections of lines

The intersection of two lines and is l l'

Line joining two points

The line through two points and is x'xl x x'

Example

1x

1y

10

Ideal points and the line at infinity

T0,,l'l ab

Intersections of parallel lines

TTand ',,l' ,,l cbacba

Example

1x 2x

Ideal points T0,, 21 xx

Line at infinity T1,0,0l

l22 RP

tangent vector

normal direction

ab , ba,

Note that in P2 there is no distinction

between ideal points and others

11

A model for the projective plane

exactly one line through two points

exaclty one point at intersection of two lines

12

Duality

x l

0xl T0lx T

l'lx x'xl

Duality principle:

To any theorem of 2-dimensional projective geometry there corresponds a dual theorem, which may be derived by interchanging the role of points and lines in the original theorem.

For instance, the basic axiom that "for any two points, there is a unique line that intersects both those points", when turned around, becomes "for any two lines, there is a unique point that intersects (i.e., lies on) both those lines"

In projective geometry, points and lines are completely interchangeable!

13

Conics

Curve described by 2nd-degree equation in the plane

022 feydxcybxyax

0233231

2221

21 fxxexxdxcxxbxax

3

2

3

1 , xxyx

xx or homogenized

0xx CT

or in matrix form

fed

ecb

dba

2/2/

2/2/

2/2/

Cwith

fedcba :::::5DOF:

6 parameters

14

Five points define a conic

For each point the conic passes through

022 feydxcyybxax iiiiii

or

0,,,,, 22 cfyxyyxx iiiiii Tfedcba ,,,,,c

0

1

1

1

1

1

552555

25

442444

24

332333

23

222222

22

112111

21

c

yxyyxx

yxyyxx

yxyyxx

yxyyxx

yxyyxx

stacking constraints yields

15

Projective transformations

A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and

only if h(x1),h(x2),h(x3) do.

Definition:

A mapping h:P2P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx

Theorem:

Definition: Projective transformation

3

2

1

333231

232221

131211

3

2

1

'

'

'

x

x

x

hhh

hhh

hhh

x

x

x

xx' Hor

8DOF

projectivity=collineation=projective transformation=homography

16

Mapping between planes

central projection may be expressed by x’=Hx(application of theorem)

17

Removing projective distortion

333231

131211

3

1

'

''

hyhxh

hyhxh

x

xx

333231

232221

3

2

'

''

hyhxh

hyhxh

x

xy

131211333231' hyhxhhyhxhx 232221333231' hyhxhhyhxhy

select four points in a plane with know coordinates

(linear in hij)

(2 constraints/point, 8DOF 4 points needed)

Remark: no calibration at all necessary, better ways to compute

18

More examples

19

Transformation of lines and conics

Transformation for lines

ll' -TH

Transformation for conics-1-TCHHC '

xx' HFor a point transformation

20

A hierarchy of transformations

Projective linear group

Affine group (last row (0,0,1))

Euclidean group (upper left 2x2 orthogonal)

Oriented Euclidean group (upper left 2x2 det 1)

Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant

e.g. Euclidean transformations leave distances unchanged

21

Class I: Isometries

(iso=same, metric=measure)

1100

cossin

sincos

1

'

'

y

x

t

t

y

x

y

x

1

11

orientation preserving:orientation reversing:

x0

xx'

1

tT

RHE IRR T

special cases: pure rotation, pure translation

3DOF (1 rotation, 2 translation)

Invariants: length, angle, area

22

Class II: Similarities

(isometry + scale)

1100

cossin

sincos

1

'

'

y

x

tss

tss

y

x

y

x

x0

xx'

1

tT

RH

sS IRR T

also know as equi-form (shape preserving)

metric structure = structure up to similarity (in literature)

4DOF (1 scale, 1 rotation, 2 translation)

Invariants: ratios of length, angle, ratios of areas, parallel lines

23

Class III: Affine transformations

11001

'

'

2221

1211

y

x

taa

taa

y

x

y

x

x0

xx'

1

tT

AH A

non-isotropic scaling! (2DOF: scale ratio and orientation)

6DOF (2 scale, 2 rotation, 2 translation)

Invariants: parallel lines, ratios of parallel lengths,

ratios of areas

DRRRA

2

1

0

0

D

24

Class VI: Projective transformations

xv

xx'

vP T

tAH

Action non-homogeneous over the plane

8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)

Invariants: cross-ratio of four points on a line

ratio of ratio of area

T21,v vv

25

Action of affinities and projectivitieson line at infinity

2211

2

1

2

1

0v

xvxvx

xx

x

vAA

T

t

000 2

1

2

1

x

xx

x

vAA

T

t

Line at infinity becomes finite,

allows to observe vanishing points, horizon,

Line at infinity stays at infinity,

but points move along line

26

Decomposition of projective transformations

vv

sPAS TTTT v

t

v

0

10

0

10

t AIKRHHHH

Ttv RKA s

K 1det Kupper-triangular,decomposition unique (if chosen s>0)

0.10.20.1

0.2242.8707.2

0.1586.0707.1

H

121

010

001

100

020

015.0

100

0.245cos245sin2

0.145sin245cos2

H

Example:

27

Overview transformations

1002221

1211

y

x

taa

taa

1002221

1211

y

x

tsrsr

tsrsr

333231

232221

131211

hhh

hhh

hhh

1002221

1211

y

x

trr

trr

Projective8dof

Affine6dof

Similarity4dof

Euclidean3dof

Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio

Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞

Ratios of lengths, angles.The circular points I,J

lengths, areas.

28

Number of invariants?

The number of functional invariants is equal to, or greater than, the number of degrees of freedom of the configuration less the number of degrees of freedom of the transformation

e.g. configuration of 4 points in general position has 8 dof (2/pt) and so 4 similarity, 2 affinity and zero projective invariants

29

Projective geometry of 1D

x'x 22H

The cross ratio

Invariant under projective transformations

T21, xx

3DOF (2x2-1)

02 x

4231

43214321 x,xx,x

x,xx,xx,x,x,x Cross

22

11detx,x

ji

ji

ji xx

xx

30

Projective 3D Geometry

31

Projective 3D Geometry

Points, lines, planes and quadrics

Transformations

П∞, ω∞ and Ω ∞

32

3D points

TT

1 ,,,1,,,X4

3

4

2

4

1 ZYXX

X

X

X

X

X

in R3

04 X

TZYX ,,

in P3

XX' H (4x4-1=15 dof)

projective transformation

3D point

T4321 ,,,X XXXX

33

Planes

0ππππ 4321 ZYX

0ππππ 44332211 XXXX

0Xπ T

Dual: points ↔ planes, lines ↔ lines

3D plane

0X~

.n d T321 π,π,πn TZYX ,,X~

14 Xd4π

Euclidean representation

n/d

XX' Hππ' -TH

Transformation

34

Planes from points

0π

X

X

X

3

2

1

T

T

T

2341DX T123124134234 ,,,π DDDD

0det

4342414

3332313

2322212

1312111

XXXX

XXXX

XXXX

XXXX

0πX 0πX 0,πX π 321 TTT andfromSolve

(solve as right nullspace of )π

T

T

T

3

2

1

X

X

X

0XXX Xdet 321

Or implicitly from coplanarity condition

124313422341 DXDXDX 01234124313422341 DXDXDXDX 13422341 DXDX

35

Points from planes

0X

π

π

π

3

2

1

T

T

T

xX M 321 XXXM

0π MT

I

pM

Tdcba ,,,π T

a

d

a

c

a

bp ,,

0Xπ 0Xπ 0,Xπ X 321 TTT andfromSolve

(solve as right nullspace of )X

T

T

T

3

2

1

π

π

π

Representing a plane by its span

36

Lines

T

T

B

AW μBλA

T

T

Q

PW* μQλP

22** 0WWWW TT

0001

1000W

0010

0100W*

Example: X-axis

(4dof)

37

Points, lines and planes

TX

WM 0π M

Tπ

W*

M 0X M

W

X

*W

π

38

Hierarchy of transformations

vTv

tAProjective15dof

Affine12dof

Similarity7dof

Euclidean6dof

Intersection and tangency

Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞

The absolute conic Ω∞

Volume

10

tAT

10

tRT

s

10

tRT

39

The plane at infinity

π

1

0

0

0

1t

0ππ

A

AH

TT

A

The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity

1. canonical position2. contains directions 3. two planes are parallel line of intersection in π∞

4. line // line (or plane) point of intersection in π∞

T1,0,0,0π

T0,,,D 321 XXX

40

Application inInvariant Matching

41

How to define an invariant 2D identifier patch on an image?

Consider a multilateral identifier patch.

Lines are invariant under the perspective projection.

• 3 (4) reference points are required to obtain an affine (projective) invariant representation of the multilateral identifier patch.

• Weak-perspective/perspective camera models.

42

Affine invariant representation of a planar point Three reference points. Ratio of areas are invariant. X at intersection of

R1R2 and R3A.

(r1,r2) invariants defined by:

R3

R1R2 X

A

)()(

)()(

32232

321321

RARRXR

RXRRRR

areararea

areararea

43

Projective invariant representation of a planar point Four reference points. Ratio of ratio of areas

are invariant. X at intersection of

R1R4 and R3A.

(r1,r2) invariants defined by:

R3

R1R2

X

A

R4

)(

)(

)(

)(

)(

)(

)(

)(

42

322

42

32

42

321

421

321

RAR

RAR

RXR

RXR

RXR

RXR

RRR

RRR

area

arear

area

area

area

arear

area

area

44

Affine invariant parallelogram:Three reference points

Select A to make a parallelogram with three reference points R1, R2, and R3

R3

R2 R1

A

)()( 32121

321 ARRRRRR areaarea

45

Affine invariant parallelogram:

Construction of identifier patch (1)

B2

B1

R3

R2 R1

A

Select B1 on R2R3

)()( 32121

1211 ARRRRRB areararea 1.01 r

46

Affine invariant parallelogram:

Construction of identifier patch (2)

C2

C1

B2

B1

R3

R1

Select C1 on B1R1

)()( 123121

2311 RBRBRBC areararea 25.01 r

47

Affine invariant parallelogram:

Construction of identifier patch (3)

D2

D1

C2

C1

R3

R1

Select D1 on C1R1

)()( 123121

3121 RCRCRCD areararea 1.03 r

),,(

),,(

321

321

rrr

RRR

48

From 2D to 3D

49

Perspective projection

Perspective projection of 6 (feature) points from 3D world (dorsal fin) to 2D image:

pi=(xi,yi,zi) qi=(ui,vi) i=1,…,6

p1

p2p3

p4

p5

p6

Focal point

Image plane

q1

q2q3q4

q5q6

50

3D and 2D Projective Invariants

I1, I2, and I3 are 3D projective invariants (under PGL(4)).

i1, i2, i3, and i4 are 2D projective invariants (under PGL(3)).

There is no general case view-invariant. 2D invariants impose constraints (in the form of polynomial relations)

on 3D invariants.* PGL(n): Projective General Linear group of order n.

111

1

1

1

0

0

0

1

0

0

0

1

111111

??projectioneperspectiv

11

1

1

1

1

0

0

0

0

1

0

0

0

0

1

0

0

0

0

1

111111

4

3

2

1

5

5

4

4

3

3

2

2

1

1

0

0

3

2

1

5

5

5

4

4

4

3

3

3

2

2

2

1

1

1

0

0

0

33

44

i

i

i

i

v

u

v

u

v

u

v

u

v

u

v

u

I

I

I

z

y

x

z

y

x

z

y

x

z

y

x

z

y

x

z

y

x

T

T

51

Invariants of point sets:

Algebraic invariants

Transfer images to a reference frame.

A unique projective/affine transformation between any two sets of 4/3 corresponding points.

Coordinate of any other point is invariant in the reference frame.

00.2

0.40.6

0.81

0

0.5

10

0.2

0.4

0.6

0.8

1

P1 P2P4

P3

00.2

0.40.6

0.81

0

0.5

10

0.2

0.4

0.6

0.8

1

P4

P3

P2P1

00.2

0.40.6

0.8

0.4

0.6

0.8

10

0.2

0.4

0.6

0.8

1

P1 P3P2

P4

Reference frame

Globalinvariant

52

Invariant Relations

Constraints (here for 6 points) typically have the following form

This is a fundamental object-image relation for six arbitrary 3D points and their 2D perspective projections.

Therefore having at least 6 points in correspondence over a number of images, 3D invariants (Ik’s) can be estimated and invariant relations can be used as geometric invariant models.

4322

4222

3311

3111 11

iIIi

iIIi

iIIi

iIIi

0)())1()1(())1()1()(( 343323232414131221 iiIiiiiIIiiiiIiiII

n 6 points3n-15 Ik’s2n-8 ik’s

2n-11 relationsor

53

Where are we going?

54

Thanks for yourAttention!

Questions?