• Email list – let me know if you have NOT been getting mail• Assignment hand in procedure

– web page

– at least 3 panoramas:» (1) sample sequence, (2) w/Kaidan, (3) handheld

– hand procedure will be posted online

• Readings for Monday (via web site)• Seminar on Thursday, Jan 18

Ramin Zabih:

“Energy Minimization for Computer Vision via Graph Cuts “

• Correction to radial distortion term

Announcements

Today

Single View Modeling

Projective Geometry for Vision/Graphics

on Monday (1/22)• camera pose estimation and calibration• bundle adjustment• nonlinear optimization

3D Modeling from a Photograph

Shape from X

Shape from X• Shape from shading• Shape from texture• Shape from focus• Others…

Make strong assumptions – limited applicability

Lee & Kau 91 (from survey by Zhang et al., 1999)

Model-Based Techniques

Blanz & Vetter, Siggraph 1999

Model-based reconstruction• Acquire a prototype face, or database of prototypes• Find best fit of prototype(s) to new image

User Input + Projective Geometry

• Horry et al., “Tour Into the Picture”, SIGGRAPH 96

• Criminisi et al., “Single View Metrology”, ICCV 1999

• Shum et al., CVPR 98

• Images above by Jing Xiao, CMU

Projective Geometry for Vision & Graphics

Uses of projective geometry• Drawing• Measurements• Mathematics for projection• Undistorting images• Focus of expansion• Camera pose estimation, match move• Object recognition via invariants

Today: single-view projective geometry• Projective representation• Point-line duality• Vanishing points/lines• Homographies• The Cross-Ratio

Later: multi-view geometry

1 2 3 4

1

2

3

4

Measurements on Planes

Approach: unwarp then measure

Planar Projective Transformations

Perspective projection of a plane• lots of names for this:

– homography, texture-map, colineation, planar projective map

• Easily modeled using homogeneous coordinates

1***

***

***

'

'

y

x

s

sy

sx

H pp’

To apply a homography H• compute p’ = Hp• p’’ = p’/s normalize by dividing by third component

Image Rectification

To unwarp (rectify) an image• solve for H given p’’ and p• solve equations of the form: sp’’ = Hp

– linear in unknowns: s and coefficients of H

– need at least 4 points

(0,0,0)

The Projective Plane

Why do we need homogeneous coordinates?• represent points at infinity, homographies, perspective projection,

multi-view relationships

What is the geometric intuition?• a point in the image is a ray in projective space

(sx,sy,s)

• Each point (x,y) on the plane is represented by a ray (sx,sy,s)– all points on the ray are equivalent: (x, y, 1) (sx, sy, s)

image plane

(x,y,1)y

xz

Projective Lines

What is a line in projective space?

• A line is a plane of rays through origin

• all rays (x,y,z) satisfying: ax + by + cz = 0

z

y

x

cba0 :notationvectorin

• A line is also represented as a homogeneous 3-vector ll p

l

Point and Line Duality• A line l is a homogeneous 3-vector (a ray)• It is to every point (ray) p on the line: l p=0

p1p2

• What is the intersection of two lines l1 and l2 ?

• p is to l1 and l2 p = l1 l2

• Points and lines are dual in projective space• every property of points also applies to lines

l1

l2

p

• What is the line l spanned by rays p1 and p2 ?

• l is to p1 and p2 l = p1 p2

• l is the plane normal

Ideal points and lines

Ideal point (“point at infinity”)• p (x, y, 0) – parallel to image plane• It has infinite image coordinates

(sx,sy,0)y

xz image plane

Ideal line• l (a, b, 0) – parallel to image plane• Corresponds to a line in the image (finite coordinates)

(sx,sy,0)y

xz image plane

Homographies of Points and Lines

Computed by 3x3 matrix multiplication• To transform a point: p’ = Hp• To transform a line: lp=0 l’p’=0

– 0 = lp = lH-1Hp = lH-1p’ l’ = lH-1

– lines are transformed by premultiplication of H-1

3D Projective Geometry

These concepts generalize naturally to 3D• Homogeneous coordinates

– Projective 3D points have four coords: P = (X,Y,Z,W)

• Duality– A plane L is also represented by a 4-vector

– Points and planes are dual in 3D: L P=0

• Projective transformations– Represented by 4x4 matrices T: P’ = TP, L’ = L T-1

However• Can’t use cross-products in 4D. We need new tools

– Grassman-Cayley Algebra» generalization of cross product, allows interactions between

points, lines, and planes via “meet” and “join” operators

– Won’t get into this stuff today

3D to 2D: “Perspective” Projection

Matrix Projection: ΠPp

1****

****

****

Z

Y

X

s

sy

sx

It’s useful to decompose into T R project A

11

0100

0010

0001

100

0

0

31

1333

31

1333

x

xx

x

xxyy

xx

f

ts

ts

00

0 TIRΠ

projectionintrinsics orientation position

Projection Models

Orthographic

Weak Perspective

Affine

Perspective

Projective

1000

****

****

Π

1000yzyx

xzyx

tjjj

tiii

Π

tRΠ

****

****

****

Π

1000yzyx

xzyx

tjjj

tiii

fΠ

Properties of Projection

Preserves• Lines and conics• Incidence• Invariants (cross-ratio)

Does not preserve• Lengths• Angles• Parallelism

Vanishing Points

Vanishing point• projection of a point at infinity

image plane

cameracenter

ground plane

vanishing point

Vanishing Points (2D)

image plane

cameracenter

line on ground plane

vanishing point

Vanishing Points

Properties• Any two parallel lines have the same vanishing point• The ray from C through v point is parallel to the lines• An image may have more than one vanishing point

image plane

cameracenter

C

line on ground plane

vanishing point V

line on ground plane

Vanishing Lines

Multiple Vanishing Points• Any set of parallel lines on the plane define a vanishing point• The union of all of these vanishing points is the horizon line

– called vanishing line in the Criminisi paper• Note that different planes define different vanishing lines

v1 v2

Vanishing Lines

Multiple Vanishing Points• Any set of parallel lines on the plane define a vanishing point• The union of all of these vanishing points is the horizon line

– called vanishing line in the Criminisi paper• Note that different planes define different vanishing lines

Computing Vanishing Points

Properties• P is a point at infinity, v is its projection

• They depend only on line direction

• Parallel lines P0 + tD, P1 + tD intersect at X

V

DPP t 0

0/1

/

/

/

1Z

Y

X

ZZ

YY

XX

ZZ

YY

XX

t D

D

D

t

t

DtP

DtP

DtP

tDP

tDP

tDP

PP

ΠPv

P0

D

Computing Vanishing Lines

Properties• l is intersection of horizontal plane through C with image plane

• Compute l from two sets of parallel lines on ground plane

• All points at same height as C project to l

• Provides way of comparing height of objects in the scene

ground plane

lC

Fun With Vanishing Points

Perspective Cues

Perspective Cues

Perspective Cues

Comparing Heights

VanishingVanishingPointPoint

Measuring Height

1

2

3

4

55.4

2.8

3.3

Camera height

C

Measuring Height Without a Ruler

ground plane

Compute Y from image measurements• Need more than vanishing points to do this

Y

The Cross Ratio

A Projective Invariant• Something that does not change under projective

transformations (including perspective projection)

P1

P2

P3

P4

1423

2413

PPPP

PPPP

The Cross-Ratio of 4 Collinear Points

Can permute the point ordering• 4! = 24 different orders (but only 6 distinct values)

This is the fundamental invariant of projective geometry• likely that all other invariants derived from cross-ratio

vZ

vL

t

b

YZL

LZ

bvvt

vvbtImage Cross Ratio

Measuring Height

B (bottom of object)

T (top of object)

L (height of camera on the this line)

ground plane

YC

YY

LTBLT

LBT 1Scene Cross Ratio

• Eliminating and yields

• Can calculate given a known height in scenetvbl

tb

Y

TY

t

C

Measuring Height

reference plane TZX 10

TZYX 1

Algebraic Derivation• •

lvvΠb ZXT ZbXaZX 10

lvvvΠt ZYXT ZYbXaZYX 1

b

Y

So Far

Can measure• Positions within a plane• Height

– More generally—distance between any two parallel planes

These capabilities provide sufficient tools for single-view modeling

To do:• Compute camera projection matrix

Vanishing Points and Projection Matrix

4321 ππππΠ

lvvvΠ ZYX

Not So Fast! We only know v’s up to a scale factor

lvvvΠ ZYX ba

T00011 Ππ

Z3Y2 , similarly, vπvπ

origin worldof projection10004 TΠπ

lvv

vvπ thiscall choose toconvenient 4

YX

YX

• Can fully specify by providing 3 reference lengths

= vx (X vanishing point)

3D Modeling from a Photograph

3D Modeling from a Photograph

Conics

Conic (= quadratic curve)• Defined by matrix equation

• Can also be defined using the cross-ratio of lines• “Preserved” under projective transformations

– Homography of a circle is a…

– 3D version: quadric surfaces are preserved

• Other interesting properties (Sec. 23.6 in Mundy/Zisserman)– Tangency

– Bipolar lines

– Silhouettes

0

1***

***

***

1

y

x

yx