Computer Vision

Spring 2006 15-385,-685

Instructor: S. Narasimhan

Wean 5403

T-R 3:00pm – 4:20pm

Lecture #12

Midterm – March 9

Syllabus – until and including Lightness and Retinex

Closed book, closed notes exam in class.

Time: 3:00pm – 4:20pm

Midterm review class next Tuesday (March 7)(Email me by March 6 specific questions)

If you have read the notes and readings, attended all classes, done assignments well, it should be a walk in the park

Mechanisms of Reflectionsource

surfacereflection

surface

incidentdirection

bodyreflection

• Body Reflection:

Diffuse ReflectionMatte AppearanceNon-Homogeneous MediumClay, paper, etc

• Surface Reflection:

Specular ReflectionGlossy AppearanceHighlightsDominant for Metals

Image Intensity = Body Reflection + Surface Reflection

Example Surfaces

Body Reflection:

Diffuse ReflectionMatte AppearanceNon-Homogeneous MediumClay, paper, etc

Surface Reflection:

Specular ReflectionGlossy AppearanceHighlightsDominant for Metals

Many materials exhibitboth Reflections:

Diffuse Reflection and Lambertian BRDF

viewingdirection

surfaceelement

normalincidentdirection

in

v

s

d

rriif ),;,(• Lambertian BRDF is simply a constant :

albedo

• Surface appears equally bright from ALL directions! (independent of )

• Surface Radiance :

v

• Commonly used in Vision and Graphics!

snIIL di

d .cos

source intensity

source intensity I

Diffuse Reflection and Lambertian BRDF

White-out: Snow and Overcast Skies

CAN’T perceive the shape of the snow covered terrain!

CAN perceive shape in regions lit by the street lamp!!

WHY?

Diffuse Reflection from Uniform Sky

• Assume Lambertian Surface with Albedo = 1 (no absorption)

• Assume Sky radiance is constant

• Substituting in above Equation:

Radiance of any patch is the same as Sky radiance !! (white-out condition)

2/

0

sincos),;,(),(),( iiiirriiiisrc

rrsurface ddfLL

skyii

src LL ),(

1

),;,( rriif

skyrr

surface LL ),(

Specular Reflection and Mirror BRDFsource intensity I

viewingdirectionsurface

element

normal

incidentdirection n

v

s

rspecular/mirror direction

),( ii ),( vv

),( rr

• Mirror BRDF is simply a double-delta function :

• Valid for very smooth surfaces.

• All incident light energy reflected in a SINGLE direction (only when = ).

• Surface Radiance : )()( vivisIL

v r

)()(),;,( vivisvviif specular albedo

Combing Specular and Diffuse: Dichromatic Reflection

Observed Image Color = a x Body Color + b x Specular Reflection Color

R

G

B

Klinker-Shafer-Kanade 1988

Color of Source(Specular reflection)

Color of Surface(Diffuse/Body Reflection)

Does not specify any specific model forDiffuse/specular reflection

Diffuse and Specular Reflection

diffuse specular diffuse+specular

Photometric Stereo

Lecture #12

Image Intensity and 3D Geometry

• Shading as a cue for shape reconstruction• What is the relation between intensity and shape?

– Reflectance Map

Surface Normal

Nsurface normal

y

z

x

Equation of plane 0 DCzByAx

0C

Dzy

C

Bx

C

Aor

Letp

C

A

x

z

qC

B

y

z

1,,1,, qpC

B

C

A

N

Surface normal

Gradient Space

y

z

x

1zq

p

1

s n

N

1

1,,22

qp

qp

N

Nn

1

1,,22

SS

SS

qp

qp

S

Ss

Normal vector

Source vector

i

11

1cos

2222

SS

SSi

qpqp

qqppsn

1z plane is called the Gradient Space (pq plane)

• Every point on it corresponds to a particular surface orientation

S

Reflectance Map

• Relates image irradiance I(x,y) to surface orientation (p,q) for given source direction and surface reflectance

• Lambertian case: yxI ,

s vni

: source brightness

: surface albedo (reflectance)

: constant (optical system)

k

c

Image irradiance:

sn kckcI i

cos

Let 1kc

then sn iI cos

• Lambertian case qpR

qpqp

qqppI

SS

ssi ,

11

1cos

2222

sn

Reflectance Map(Lambertian)

cone of constant i

Iso-brightness contour

Reflectance Map

• Lambertian case

0.1

3.0

0.0

9.08.0

7.0, qpR

p

q

90i 01 SS qqpp

SS qp ,

iso-brightnesscontour

Note: is maximum when qpR , SS qpqp ,,

Reflectance Map

• Glossy surfaces (Torrance-Sparrow reflectance model)

qpRGpkc

kcIr

si

d ,cos

cos

diffuse term specular term

p

q

5.0, qpR

SS qp ,

Diffuse peak

Specular peak

Reflectance Map

Shape from a Single Image?

• Given a single image of an object with known surface reflectance taken under a known light source, can we recover the shape of the object?

• Given R(p,q) ( (pS,qS) and surface reflectance) can we determine (p,q) uniquely for each image point?

NO

p

q

Solution

• Take more images– Photometric stereo

• Add more constraints– Shape-from-shading (next class)

Photometric Stereo

p

q

11 ,SSqp

22 ,SSqp

33 ,SSqp

• We can write this in matrix form:

Image irradiance:

1kc

11 snI1s

n

v

2s

22 snI3s

33 snI

n

s

s

s

T

T

T

I

I

I

3

2

2

2

1 1

Lambertian case:

sn

ikcI cos

Photometric Stereo

Solving the Equations

n

s

s

s

T

T

T

I

I

I

3

2

2

2

1 1

I S n~133313

ISn 1~ n~

n

n

nn

~

~

~

inverse

More than Three Light Sources

• Get better results by using more lights

n

s

s

TN

T

NI

I

11

• Least squares solution:

nSI ~nSSIS ~TT

ISSSn TT 1~

• Solve for as beforen, Moore-Penrose pseudo inverse

1331 NN

Color Images

• The case of RGB images– get three sets of equations, one per color channel:

– Simple solution: first solve for using one channel– Then substitute known into above equations to get

– Or combine three channels and solve for

SnI GG SnI BB

SnI RR

BGR ,,

n

n

SnIIII 222

BGR

n

Computing light source directions

• Trick: place a chrome sphere in the scene

– the location of the highlight tells you the source direction

• For a perfect mirror, light is reflected about N

Specular Reflection - Recap

otherwise0

if rvie

RR

• We see a highlight when • Then is given as follows:

n

vs

rv s

rnrns 2

rii

Computing the Light Source Direction

• Can compute N by studying this figure– Hints:

• use this equation:• can measure c, h, and r in the image

N

rN

C

H

c

h

Chrome sphere that has a highlight at position h in the image

image plane

sphere in 3D

Depth from Normals

• Get a similar equation for V2

– Each normal gives us two linear constraints on z

– compute z values by solving a matrix equation

V1

V2

N

Limitations

• Big problems– Doesn’t work for shiny things, semi-translucent things– Shadows, inter-reflections

• Smaller problems– Camera and lights have to be distant– Calibration requirements

• measure light source directions, intensities• camera response function

Trick for Handling Shadows

• Weight each equation by the pixel brightness:

• Gives weighted least-squares matrix equation:

• Solve for as before

n

s

s

TNN

T

N I

I

I

I

11

2

21

n,

iiii III sn

Original Images

Results - Shape

Shallow reconstruction (effect of interreflections)

Accurate reconstruction (after removing interreflections)

Results - Albedo

No Shading Information

Original Images

Results - Shape

Results - Albedo

Results

1. Estimate light source directions2. Compute surface normals3. Compute albedo values4. Estimate depth from surface normals5. Relight the object (with original texture and uniform albedo)

Next Class

• Shape from Shading

• Reading: Horn, Chapter 11.