Symmetric Photography:Exploiting Data-sparseness in

Reflectance Fields

Gaurav Garg Eino-Ville Marc Levoy Hendrik P. A. Lensch

Symmetric Photography:Dealing with 8D Reflectance Fields

Relighting Ground Truth Example Capture

Overview

• Full 8D Reflectance field!– Changing View (4D) * Changing Light (4D)– Eg.: For Each 4D, 3x3 images at 100x100 res

results in 10^10 4D table

• How do we deal with data explosion?– Exploit Symmetry between light/view

• Helmholtz reciprocity

– Exploit Data Sparseness

Symmetric Photography

inout TLL Transport equation:

• T is symmetric (Helmholtz reciprocity)

• T is not sparse

• But sub-blocks of T are “data sparse”

4321

4

3

2

1

44434241

34333231

24232221

14131211

cccc

b

b

b

b

aaaa

aaaa

aaaa

aaaa

Visualizing Symmetry andData-sparsity

Outline

• Data Acquisition Setup

• Exploiting Symmetry and Data Sparsity in the Transport Matrix

• Results

Symmetric Setup

Acquisition

Outline

• Data Acquisition Setup

• Exploiting Symmetry and Data Sparsity in the Transport Matrix

• Results

Hierarchical Tensors –Parallel Acquisition

• If M=0, U1 and U2 are radiometrically isolated.

• If M!=0, but is known, we can subtract it out to isolate U1 and U2

• This allows us to illuminate projector pixels in U1 and U2 in parallel.

0M

M0

U20

0U1

U2M

MU1T TTsub

Hierarchical Tensors –Rank-1 Approximation

• “An image captured by the camera is the sum of the columns corresponding to the pixels lit by the projector. The image is also the sum of the corresponding rows”

• Use two projector patterns (Pr and Pc) s.t.

and

• The rank-1 approximation of M is

0M

M0

U20

0U1

U2M

MU1T TTsub

cMpc rpMr T

prM

Hierarchical Acquisition

1. Already have a rank-1 approximation– For root node, use flood lit image for first approximation

2. Divide node by 16 and move to next level– 4 projector blocks X 4 camera blocks

– Use 4 projector patterns and capture 4 images (8 images total)

3. Evaluate previous level’s rank-1 approx against these images– If good enough, finish

– If the size of the projector block is down to a pixel, finish

– Else, use these images to create 16 rank-1 approximations, and goto 1.) for each of them

Note – I have heavily glossed over the selection of projector patterns

Outline

• Data Acquisition Setup

• Exploiting Symmetry and Data Sparsity in the Transport Matrix

• Results

Changing Light

Changing View

Symmetric vs. Dual Photography

Artifacts Due to Non-Symmetry

Hierarchy Levels

Table