RADIANCE - Washington University in St. Louisayan/courses/cse559a/PDFs/lec9_4UP.pdf · RADIANCE...

CSE 559A: Computer Vision

Fall 2019: T-R: 11:30-12:50pm @ Hillman 60

Instructor: Ayan Chakrabarti ([email protected]).Course Staff: Arushee Agrawal, Annie Lee, Jiahao Li, Xiaochen Zhou

Sep 24, 2019

http://www.cse.wustl.edu/~ayan/courses/cse559a/

1

GENERALGENERALPSET 1 Due 11:59pm today.git commit AND git push (and git pull a�er that to check)

Need to push only once you're doneTry to avoid force-adding all files in your directory

We've received lots of random files: .DS_store, synctex.gz, ...Late days applied automatically at the end of the semester.

PSET 2 out and ready to clone.

2

RADIANCERADIANCERecall

BRDF =

Ratio of outgoing radiance in one direction to incoming (infinitesimal) irradiance from another

Total Outgoing Radiance in a specific direction will integrate over contributions from incoming irradiancefrom all directions.

ρ( , , , )θi ϕi θo ϕo

( , ) = ∫ ρ( , , , ) ( , ) cos dLo θo ϕo θi ϕi θo ϕo Li θi ϕi θi ωi

3

RADIANCERADIANCE

6

RADIANCERADIANCE

8

RADIANCERADIANCE

9

RADIANCERADIANCE

10

RADIANCERADIANCEBi-directional Reflectance Distribution Function

Properties

Positivity:

Helmholtz Reciprocity:

Total Energy leaving surface is less than total energy arriving

ρ( , , , ) ≥ 0θi ϕi θo ϕo

ρ( , , , ) = ρ( , , , )θi ϕi θo ϕo θo ϕo θi ϕi

∫ ( , ) cos d ≥ ∫ [∫ ρ( , , , ) ( , ) cos d ] cos dLi θi ϕi θi ωi θi ϕi θo ϕo Li θi ϕi θi ωi θo ωo

11

RADIANCE, IRRADIANCE, BRDFSRADIANCE, IRRADIANCE, BRDFSAdditional Reference: Forsyth & Ponce: Chapters 4 & 5Less Detailed / Quick: Szeliski Sec 2.2

12

LIGHTSLIGHTS

15

LIGHTSLIGHTS

16

LIGHTSLIGHTS

20

LIGHTSLIGHTS

24

LIGHTSLIGHTS

25

LIGHTSLIGHTS

27

LIGHTSLIGHTS

30

LIGHTSLIGHTS

35

LIGHTSLIGHTS

38

PHOTOMETRIC STEREOPHOTOMETRIC STEREO

44


50


51


52


53


For each point, let the set of intensities be observed under lights .

Ignore color, assume is scalar (convert the images to grayscale / R+G+B).

Three observations of with different, linearly independent, will give us .Three linear equations in three variables.

Given , we can factor into and : is length of , and .

{ }Ii { }ℓi

Ii

= ρ ⟨ , ⟩ = ρ = (ρ ) = nIi n̂ ℓi ℓT

in̂ ℓT

in̂ ℓT

i

Ii ℓi n

n ρ n̂ ρ ‖n‖ = n/‖n‖n̂

55


But using only three images is unstable: there will be noise, etc. We solve in the "least squares" sense.

Given images under different lights, for each pixel:

where is a matrix, is a vector, and is a three-vector.

K K

n = ⇒ L n = I

⎡

⎣

⎢⎢⎢⎢⎢⎢

ℓT1

ℓT2

ℓT3

⋮

ℓT

K

⎤

⎦

⎥⎥⎥⎥⎥⎥

⎡

⎣

⎢⎢⎢⎢⎢⎢

I1

I2

I3

⋮

IK

⎤

⎦

⎥⎥⎥⎥⎥⎥

L K × 3 I K × 1 n

56


Take gradient, set to 0:

This is actually a equation. Solve using np.lingalg.solve to use Cholesky.

n = arg ‖L n − I = arg ( L) n − 2( I n + Iminn

‖2 minn

nT

LT

LT )T I

T

( L) n = ( I)LT

LT

3 × 3

57


61

PHOTOMETRIC STEREOPHOTOMETRIC STEREOSome Practical Issues:

Even though we assume light at infinity, works well in practice for just far away lights.

Calibrate light vector by looking at an image of some known shape and albedo (typically a matte sphere)

Technically only works for Lambertian objects. But o�en, can make objects Lambertian with polarizers.Also, estimated normals are typically well-defined for a 'valid' set of pixels in the image.You'll create / be given a "mask" of these valid pixels.

ℓ

62

NORMALS TO DEPTHNORMALS TO DEPTH

68


73

NORMALS TO DEPTHNORMALS TO DEPTHZ = arg ‖ − ∗ Z + ‖ − ∗ Zmin

Zgx fx ‖2 gy fy ‖2

74


We'll use as:

Z = arg ‖ − ∗ Z + ‖ − ∗ Z + λR(Z)minZ

gx fx ‖2 gy fy ‖2

R(Z)

R(Z) = (Z ∗ )[n for =∑n

fr ]2 fr

−1/9

−1/9

−1/9

−1/9

8/9

−1/9

−1/9

−1/9

−1/9

75


Version 1: Do it in the Fourier Domain (called Frankot-Chellappa)

Assume that in the masked out regions, .

is FT of depth map, is FT of , is (circular padded) FT of , and so on.

In general, should add some very small number (e.g., ) to denominator for stability.

In particular, what is the denominator for ?

Both numerator and denominator are 0, because normals tell us nothing about average depth / offset.

Explicitly set to 0.

Z = arg ‖ − ∗ Z + ‖ − ∗ Z + λR(Z)minZ

gx fx ‖2 gy fy ‖2

= = 0gx gy

(Z)[u, v] =[u, v] [u, v] + [u, v] [u, v]F̄x Gx F̄y Gy

| [u, v] + | [u, v] + λ| [u, v]Fx |2 Fy |2 Fr |2

(Z) Gx gx Fx fx

10−12

[u, v] = [0, 0]

(Z)[0, 0]

76


Version 2: Use conjugate gradient.

Allows us to use different weights for different pixels.

Set to 0 for masked out pixels.

Set to everywhere else.

Accounts for the fact that we got gradients by dividing by .

Smaller values will be noisier.

Z = arg w[n]( [n] − ( ∗ Z)[n] + w[n]( − ( ∗ Z)[n] + λR(Z)minZ ∑

n

gx fx )2 ∑n

gy fy )2

w[n]

w[n] ( [n]n̂ z )2

n̂ z

78


Begin with all zeroes guess for

Repeat (for say a fixed number of iterations)

Z = arg w[n]( [n] − ( ∗ Z)[n] + w[n]( − ( ∗ Z)[n] + λR(Z)minZ ∑

n

gx fx )2 ∑n

gy fy )2

Z = arg QZ − 2 b + cminZ

Z T Z T

Z0 Z

k = 0, ← b − Q , ←r0 Z0 p0 r0

←αkrTkrk

QpTk

pk

← +Zk+1 Zk αkpk

← − Qrk+1 rk αk pk

←βkrTk+1

rk+1

rTkrk

← +pk+1 rk+1 βkpk

k = k + 1

79


We need to figure out:

What is (should be same shape as image)

How do compute for a given (where is same shape as image)

Let be a diagonal matrix with values of . , and denote convolutions by , , and .

What are and ?

Z = arg w[n]( [n] − ( ∗ Z)[n] + w[n]( − ( ∗ Z)[n] + λ (( ∗ Z)[n]minZ ∑

n

gx fx )2 ∑n

gy fy )2 ∑n

fr )2


Z T Z T

b

Q p p p

W w[n] Fx Fy Fr fx fy fr

Z = argmin( − Z W( − Z) + ( − Z W( − Z) + λ( Z ( Z)Gx Fx )T Gx Fx Gy Fy )T Gy Fy Fr )T Fr

Q b

Q = W + W + λFTx Fx FT

y Fy FTr Fr

b = W + WFTx Gx FT

y Gy

80


What is (should be same shape as image)

How do compute for a given (where is same shape as image)

mean the flipped versions of . means element-wise product. ( can be same convolutions with zero padding)

Remember, is just . So compute , take element-wise product with ,and sum across all pixels.

Z = arg w[n]( [n] − ( ∗ Z)[n] + w[n]( − ( ∗ Z)[n] + (( ∗ Z)[n]minZ ∑

n

gx fx )2 ∑n

gy fy )2 ∑n

fr )2


Z T Z T

b

Q p p p

Q = W + W + λFTx Fx FT

y Fy FTr Fr

b = W + WFTx Gx FT

y Gy

Q p = ((p ∗ ) × w) ∗ + ((p ∗ ) × w) ∗ + λ((p ∗ ) ∗ )fx f̄ x fy f̄ y fr f̄ r

f̄ f ×

∗

b = ( × w) ∗ + ( × w) ∗gx f̄ x gy f̄ y

QppT ⟨p,Qp⟩ Qp p

81


82

RADIANCE - Washington University in St. Louisayan/courses/cse559a/PDFs/lec9_4UP.pdf · RADIANCE...

Documents

Transcript of RADIANCE - Washington University in St. Louisayan/courses/cse559a/PDFs/lec9_4UP.pdf · RADIANCE...