Lecture 1Lecture 122

Modules Employing Gradient Descent

Computing Optical Flow

Shape from Shading

2

Gibbs SamplerGibbs Sampler

ZP

TE

i

i

TEi i

ZP

1E/T is big

big is overflow

TE

ZP

1

11

TE

ZP

2

12

TE

ZP

3

13

1001 T

E

1202 T

E

1403 T

E

01 T

E

202 T

E

403 T

E

3

Gibbs SamplerGibbs Sampler

T

E

iPmax

Emax/T that will overflow

KTE

K

TE

11

1

KTE

K

TE

22

1

= BIGGEST DOUBLE

4

2 Modules that Employ Gradient Descent2 Modules that Employ Gradient Descent

1. Computing Optical Flow for Motion Using Gradient Based Approach

2. Shape from Shading

5

Optical FlowOptical Flow

Motion Field in Image Plane

udt

dx u

dt

dx

6

Optical FlowOptical Flow

2 Methods:

1. Featured Based - similar to stereo where you solve

- correspondence (matching) problem between 2 consecutive frames

2. Gradient of Intensity Based - No matching needed

- Works well when images have much texture

- Dense map of (u,v) at each pixel

7

Gradient of Intensity BasedGradient of Intensity Based

1 2 3

I(x,y,t1) I(x,y,t2) I(x,y,t3)

I(x,y,t)

16/sec

t

ALIASING

t 0t- Spatial Resolution (x,y) pixels per cm

- Temporal Resolution frames per second

8

AliasingAliasing

Problems noticeable when your sampling cannot truly estimate the underlying frequency

Have to sample double the frequency

9

Chain RuleChain Rule : I(x,y,t): I(x,y,t)

Assumption:“As an object moves, its intensity does not change”

t

I

dt

dy

y

I

dt

dx

x

I

dt

dI

),,(),,( 12 tyxItyxI

),,(),,( tyxIdttdyydxxI

0dt

dI

10

Specular RegionsSpecular Regions

Specular regions are noise for Computer Vision

2 2

11

Gradient of Intensity Gradient of Intensity BasedBased

t

I

dt

dy

y

I

dt

dx

x

I

dt

dI

0),(),(),(),(),( yxvyxIyxuyxIyxI yxt

Ix u Iy v It

12

Gradient of Intensity BasedGradient of Intensity Based

),(),1( 11 yxIyxIx

II ttx

t

IyxIyxII t

),(),( 12 0t

),()1,( 11 yxIyxIy

II tty

13

Gradient of Intensity BasedGradient of Intensity Based

2),()1,(12

),(),1(12

,

)( yxyxyxyxtyx

yx uuuuIvIuIE

2),()1,(12

),(),1(1 yxyxyxyx vvvv

Unknowns : u at each (x,y)v at each (x,y)

14

Gradient of Intensity BasedGradient of Intensity Based

Use Gradient Descent :

E(u,v)

u

E

dt

du

u

Euu tt

1

v

Ecv tt

1

Update Rule

Highly Textured

Knowns : Ix, Iy, It at (x,y)

15

Research TopicsResearch Topics

Find (u,v) through gradientMethod: Coarse-to-Fine

How to choose 1, 2 automatically

How to get the annealing schedule automaticallyT high Random Walk

T low Greedy

16

Shape from ShadingShape from Shading

Point Light at ∞

ping-pongviewer

viewer

Image Observed: f (viewer position, camera model, shape of object, material of object, light color, light model, light position)

17

Material of ObjectMaterial of Object

Color Shiny Transparency Texture Bumpy

18

Light ModelLight Model

Ambient – light (constant) at each point Spot

Omni – Neon – All Direction

Point Light - “Sun”

19

LightLight

R,G,B

I(x,y) = Ambient + Diffuse + Specular

= Iaka + kdIdcos + kss(cos)

Ia : Ambient LightId : Diffuse Light – Main Lightka : Ambient Constant “glow in dark”kd : Main Color Diffuse Constant

White is high , Black is lowks : Mirror Like, Specularity Constant

ks = 0 for ping pong = 0.5 for apple = 1 for billioud

20

Shininess FactorShininess Factor

= 20

= 1

Sharp Shiny Blurry Shiny

21

Shininess FactorShininess Factor

)(cosds Ik

: angle between V and R

: angle between L and N

cosq = L.N = |L||N|cos= cos

22

Shininess FactorShininess Factor

Diffuse = kd Id cos

cos decrease I

brighterdarker

0o 45o 85o

23

ShapeShape

Shape = Normal at a surface

(Nx, Ny, Nz) unit

24

NormalNormal

0 DCzByAx

0C

Dzy

C

Bx

C

A

C

Dy

C

Bx

C

Az

pC

A

x

z

)(

qC

B

y

z

)(

Equation of Plane

25

NormalNormal

Normal is different at every point

1),,(),,(

)1),,(),,((),(

yxqyxp

yxqyxpyxN

1),,(

222

CBA

CBAN

1

)1,,()1,,(

22

qp

qp

C

B

C

AN

26

Light DirectionLight Direction

L is the same at every point222 1

)1,,(

ba

baL

11

)1,,).(1,,().(

22222

qpba

qpbaIkNLIkI dd

dd

222 1),(),(

)1),(),((

yxqyxp

yxbqyxapkI

Contour of Constant Intensity

27

SFS: Data ConstraintSFS: Data Constraint

222 1

)1(

qp

bqapkI

kkbqkapqpI 222 1

01222 kkbqkapqpI Data Constraint

28

SFS: Energy FunctionSFS: Energy Function

Known : Ia, kd, (a,b,1), I(x,y) Unknown : p,q

2,

222 1 yx

kkbqkapqpIE

22 ),()1,(),(),1( yxpyxpyxpyxp ss

22 ),()1,(),(),1( yxqyxqyxqyxq ss

q

Eqq tt

1

p

Epp tt

1