Projective Geometry and Camera model

Class 2

points, lines, planesconics and quadrics

transformationscamera model

Read tutorial chapter 2 and 3.1http://www.cs.unc.edu/~marc/tutorial/

Chapter 1, 2 and 5 in Hartley and Zisserman

Homogeneous coordinates

0=++ cbyax ( ) ( ) 0=1x,y,a,b,c T

( ) ( ) 0≠∀,~ ka,b,cka,b,c TT

Homogeneous representation of 2D points and lines

equivalence class of vectors, any vector is representative

Set of all equivalence classes in R3(0,0,0)T forms P2

( ) ( ) 0≠∀,1,,~1,, kyxkyx TT

The point x lies on the line l if and only if

Homogeneous coordinates

Inhomogeneous coordinates ( )Tyx,

( )T321 ,, xxx but only 2DOF

Note that scale is unimportant for incidence relation

0=xlT

Points from lines and vice-versa

l'×l=x

Intersections of lines

The intersection of two lines and is l l'

Line joining two points

The line through two points and is x'×x=lx x'

Example

1=x

1=y

1

)1-,1,0( y

x

1

)1-,0,1( y

x

Note:[ ] 'xx='x×x ×

with

0-

0-

-0

x

xy

xz

yz

Ideal points and the line at infinity

T0,,l'l ab

Intersections of parallel lines

( ) ( )TTand ',,=l' ,,=l cbacba

Example

1=x 2=x

Ideal points ( )T0,, 21 xx

Line at infinity ( )T1,0,0=l∞

∞22 l∪= RP

tangent vector

normal direction

-ab,( )ba,

Note that in P2 there is no distinction

between ideal points and others

• Line at infinity is a collection of all the points at infinity as x1/x2 varies, i.e. all Possible directions

3D points and planes

Homogeneous representation of 3D points and planes

The point X lies on the plane π if and only if

0=π+π+π+π 44332211 XXXX

0=XπT

The plane π goes through the point X if and only if

0=XπT

Planes from points

0π

X

X

X

3

2

1

T

T

T

0=πX 0=πX 0,=πX π 321TTT andfromSolve

(solve as right nullspace of )π

T

T

T

3

2

1

X

X

X

Points from planes

0X

π

π

π

3

2

1

T

T

T

x=X M 321 XXXM

0π MT

0=Xπ 0=Xπ 0,=Xπ X 321TTT andfromSolve

(solve as right nullspace of )X

T

T

T

3

2

1

π

π

π

Representing a plane by its span

• M is 4x3 matrix. Columns of M are null space of Tπ

Lines

T

T

B

AW μBλA

2×2** 0=WW=WWTT

0001

1000W

0010

0100W*

Example: X-axis

(4dof)

Representing a line by its span: two vectors A, B for two space points

T

T

Q

PW* μQλP

Dual representation: P and Q are planes; line is span of row space of

(Alternative: Plücker representation, details see e.g. H&Z)

• Line is either joint of two points or intersection of two planes

*W

Points, lines and planes

TX

WM 0π M

Tπ

W*

M 0X M

W

X

*W

π

• Plane defined by join of point X and line W is the null space of M

π

• Point X defined by the intersection of line W with plane is the null space of Mπ

Conics

Curve described by 2nd-degree equation in the plane

0=+++++ 22 feydxcybxyax

0=+++++ 233231

2221

21 fxxexxdxcxxbxax

3

2

3

1 , xxyx

xx or homogenized

0=xx CT

or in matrix form

fed

ecb

dba

2/2/

2/2/

2/2/

Cwith

{ }fedcba :::::5DOF:

Five points define a conic

For each point the conic passes through

0=+++++ 22 feydxcyybxax iiiiii

or

( ) 0=.1,,,,, 22 ciiiiii yxyyxx ( )Tfedcba ,,,,,=c

0

1

1

1

1

1

552555

25

442444

24

332333

23

222222

22

112111

21

c

yxyyxx

yxyyxx

yxyyxx

yxyyxx

yxyyxx

stacking constraints yields

Conic c is a null vector of the above matrix

Tangent lines to conics

The line l tangent to C at point x on C is given by l=Cx

lx

C

Dual conics

0=ll *CTA line tangent to the conic C satisfies

Dual conics = line conics = conic envelopes

1-* CC In general (C full rank):

Points lie on a point lines are tangent conic to the point conic C; conic C is the envelope of lines l

0CxxT

0* lClT

Degenerate conics

A conic is degenerate if matrix C is not of full rank

TT ml+lm=C

e.g. two lines (rank 2)

e.g. repeated line (rank 1)

Tll=C

l

l

m

Degenerate line conics: 2 points (rank 2), double point (rank1)

( ) CC ≠**Note that for degenerate conics

Quadrics and dual quadrics

(Q : 4x4 symmetric matrix)0=QXXT

• 9 d.o.f.• in general 9 points define quadric • det Q=0 ↔ degenerate quadric• tangent plane

Q

QX=π

0=πQπ *T

-1* Q=Q• relation to quadric (non-degenerate)

• Dual Quadric: defines equation on planes

2D projective transformations

A projectivity is an invertible mapping h from P2 to itself such that three points x1,x2,x3 lie on the same line if and

only if h(x1),h(x2),h(x3) do.

Definition:

A mapping h:P2P2 is a projectivity if and only if there exist a non-singular 3x3 matrix H such that for any point in P2 reprented by a vector x it is true that h(x)=Hx

Theorem:

Definition: Projective transformation

3

2

1

333231

232221

131211

3

2

1

'

'

'

x

x

x

hhh

hhh

hhh

x

x

x

x=x' Hor

8DOF

projectivity=collineation=projective transformation=homography

Transformation of 2D points, lines and conics

Transformation for lines

l=l' -TH

Transformation for conics-1-TCHHC ='

Transformation for dual conicsTHHCC ** ='

x=x' HFor a point transformation

Fixed points and lines

e=e λH (eigenvectors H =fixed points)

l=l λTH (eigenvectors H-T =fixed lines)

(1=2 pointwise fixed line)

Hierarchy of 2D transformations

1002221

1211

y

x

taa

taa

1002221

1211

y

x

tsrsr

tsrsr

333231

232221

131211

hhh

hhh

hhh

1002221

1211

y

x

trr

trr

Projective8dof

Affine6dof

Similarity4dof

Euclidean3dof

Concurrency, collinearity, order of contact (intersection, tangency, inflection, etc.), cross ratio

Parallellism, ratio of areas, ratio of lengths on parallel lines (e.g midpoints), linear combinations of vectors (centroids). The line at infinity l∞

Ratios of lengths, angles.The circular points I,J

lengths, areas.

invariantstransformed squares

Projective geometry of 1D

The cross ratio

Is invariant under projective transformations in P^1.

4231

43214321 x,xx,x

x,xx,xx,x,x,x Cross

22

11detx,x

ji

ji

ji xx

xx

Four sets of four collinear points; each set is related to the others by a line projectivitySince cross ratio is an invariant under projectivity, the cross ratio has the same value for all the sets shown

Concurrent Lines

Four concurrent lines l_i intersect the line l in the four points x_i; The cross ratio of these points is an invariant to the projective transformation of the plane

Coplanar points x_i are imaged onto a line by a projection with center C. The cross ratioof the image points x_i is invariant to the position of the image line l

A hierarchy of transformations

• Euclidean transformations (rotation and translation) leave distances unchanged

• Similarity: circle imaged as circle; square as square; parallel or perpendicular lines have same relative orientation

• Affine: circle becomes ellipse; orthogonal world lines not imaged as orthogonoal; But, parallel lines in the square remain parallel

• Projective: parallel world lines imaged as converging lines; tiles closer to camera larger image than those further away.

Similarity Affine projective

The line at infinity

00

l l 0 lt 1

1A

AH

A

TT

T T

The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity

Note: not fixed pointwise

Affine properties from images

projection rectification

APA

lll

HH

321

010

001

0≠,l 3321∞ llll T

Two step process:1.Find l the image of line at infinity in plane 22. Plug into H_pa

Image of line at infinity in plane

Affine rectificationv1 v2

l1

l2 l4

l3

l∞

21∞ vvl

211 llv

432 l×l=v

The circular points

0

1

I i

0

1

J i

I

0

1

0

1

100

cossin

sincos

II

iseitss

tssi

y

x

S

H

The circular points I, J are fixed points under the projective transformation H iff H is a similarity

Two points on l_inf which are fixed under any similarity transformation

Canonical coordinates of circular points

The circular points

“circular points”

0=++++ 233231

22

21 fxxexxdxxx 0=+ 2

221 xx

l∞

T

T

0,-,1J

0,,1I

i

i

( ) ( )TT 0,1,0+0,0,1=I i

Algebraically, encodes orthogonal directions

03 x

Two points on l_inf: Every circle intersects l_inf at circular points

Line at infinity

Circle:

Conic dual to the circular points

TT JIIJ*∞ C

000

010

001*∞C

TSS HCHC *

∞*∞

The dual conic is fixed conic under the projective transformation H iff H is a similarity

*∞C

Note: has 4DOF (3x3 ho mogeneous; symmetric, determinant is zero)

l∞ is the nullvector

*∞C

l∞

I

J

Angles

( )( )22

21

22

21

2211

++

+=cos

mmll

mlmlθ

( )T321 ,,=l lll ( )T

321 ,,=m mmmEuclidean:

Projective: mmll

mlcos

**

*

CC

CTT

T

0ml * CT (l and m orthogonal)

Once is identified in the projective plane, then Euclidean angles may be measured by equation (1)

(1)

*C

Transformation of 3D points, planes and quadrics

Transformation for lines

( )l=l' -TH

Transformation for conics

( )-1-TCHHC ='

Transformation for dual conics

( )THHCC ** ='

( )x=x' H

For a point transformation

X=X' H

π=π' -TH

-1-TQHH=Q'

THHQ=Q' **

(cfr. 2D equivalent)

Hierarchy of 3D transformations

vTv

tAProjective15dof

Affine12dof

Similarity7dof

Euclidean6dof

Intersection and tangency

Parallellism of planes,Volume ratios, centroids,The plane at infinity π∞

Angles, ratios of length

The absolute conic Ω∞

Volume

10

tAT

10

tRT

s

10

tRT

The plane at infinity0

00π π π

0- t 1

1

A

AH

A

TT

T T

The plane at infinity π is a fixed plane under a projective transformation H iff H is an affinity

1. canonical position2. contains directions 3. two planes are parallel line of intersection in π∞

4. line // line (or plane) point of intersection in π∞

( )T1,0,0,0=π∞

( )T0,,,=D 321 XXX

The absolute conic

The absolute conic Ω∞ is a fixed conic under the projective transformation H iff H is a similarity

04

23

22

21

X

XXX

The absolute conic Ω∞ is a (point) conic on π. In a metric frame:

T321321 ,,I,, XXXXXXor conic for directions:(with no real points)

1. Ω∞ is only fixed as a set2. Circle intersect Ω∞ in two circular points3. Spheres intersect π∞ in Ω∞

The absolute dual quadric

00

0I*∞ T

The absolute dual quadric Ω*∞ is a fixed conic under

the projective transformation H iff H is a similarity

1. 8 dof ( symmetric matrix, det is zero)2. plane at infinity π∞ is the nullvector of Ω∞

3. Angles:

( )( )2

*∞21

*∞1

2*∞1

πΩππΩπ

πΩπ=cos

TT

T

θ

Set of planes tangent to an ellipsoid, then squash to a pancake

Camera model

Relation between pixels and rays in space

?

Pinhole camera

Pinhole camera model

TT ZfYZfXZYX )/,/(),,(

101

0

0

1

Z

Y

X

f

f

Z

fY

fX

Z

Y

X

linear projection in homogeneous coordinates!

Pinhole camera model

101

0

0

Z

Y

X

f

f

Z

fY

fX

101

01

01

1Z

Y

X

f

f

Z

fY

fX

PXx

0|I)1,,(diagP ff

Principal point offset

Tyx

T pZfYpZfXZYX )+/,+/(),,(

principal pointT

yx pp ),(

101

0

0

1

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

Z

Y

X

y

x

x

x

Principal point offset

101

0

0

Z

Y

X

pf

pf

Z

ZpfY

ZpfX

y

x

x

x

camX0|IKx

1y

x

pf

pf

K calibration matrix

Camera rotation and translation

( )C~

-X~

R=X~

cam

X10

RC-R

1

10

C~

R-RXcam

Z

Y

X

camX0|IKx XC~

-|IKRx

t|RKP C~

R-t PX=x

~

CCD camera

1yy

xx

p

p

K

11y

x

y

x

pf

pf

m

m

K

General projective camera

1yx

xx

p

ps

K

1yx

xx

p

p

K

C~

|IKRP

non-singular

11 dof (5+3+3)

t|RKP

intrinsic camera parametersextrinsic camera parameters

Radial distortion

• Due to spherical lenses (cheap)• Model:

R

y

xyxKyxKyx ...))()(1(),( 44

222

1R

http://foto.hut.fi/opetus/260/luennot/11/atkinson_6-11_radial_distortion_zoom_lenses.jpgstraight lines are not straight anymore

Camera model

Relation between pixels and rays in space

?

Projector model

Relation between pixels and rays in space

(dual of camera)

(main geometric difference is vertical principal point offset

to reduce keystone effect)

?

Meydenbauer camera

vertical lens shiftto allow direct ortho-photographs

Affine cameras

Action of projective camera on points and lines

forward projection of line

( ) μb+a=μPB+PA=μB)+P(A=μX

back-projection of line

lP=Π T

PXlX TT ( )PX= x0;=xlT

PX=x

projection of point

Action of projective camera on conics and quadricsback-projection to cone

CPP=Q Tco 0=CPXPX=Cxx TTT

( )PX=x

projection of quadric

TPPQC ** 0lPPQlQ T*T*T

( )lP=Π T

Image of absolute conic

A simple calibration device

(i) compute H for each square (corners @ (0,0),(1,0),(0,1),(1,1))

(ii) compute the imaged circular points H(1,±i,0)T

(iii) fit a conic to 6 circular points(iv) compute K from w through cholesky factorization

(≈ Zhang’s calibration method)

Exercises: Camera calibration

Next class:Single View Metrology

Antonio Criminisi

A hierarchy of transformations• Group of invertible nxn matrices with real elements general linear group on n

dimensionsGL(n);• Projective linear group: matrices related by a scalar multiplier PL(n); three

subgroups:• Affine group (last row (0,0,1))• Euclidean group (upper left 2x2 orthogonal)• Oriented Euclidean group (upper left 2x2 det 1)

• Alternative, characterize transformation in terms of elements or quantities that are preserved or invariant

• e.g. Euclidean transformations (rotation and translation) leave distances unchanged

• Similarity: circle imaged as circle; square as square; parallel or perpendicular lines have same relative orientation

• Affine: circle becomes ellipse; orthogonal world lines not imaged as orthogonoal; But, parallel lines in the square remain parallel

• Projective: parallel world lines imaged as converging lines; tiles closer to camera larger image than those further away.

Similarity Affine projective