General Linear Cameras with Finite Aperture

Andrew Adams and Marc LevoyStanford University

Ray Space

Slices of Ray Space

• Pushbroom

• Cross Slit

• General Linear Cameras

Yu and McMillan ‘04

Román et al. ‘04

Projections of Ray Space

• Plenoptic Cameras

• Camera Arrays

• Regular Cameras

Ng et al. ‘04

Wilburn et al. ‘05Leica Apo-Summicron-M

What is this paper?

What is this paper?

• An intuitive reformulation of general linear cameras in terms of eigenvectors

What is this paper?

• An intuitive reformulation of general linear cameras in terms of eigenvectors

• An analogous description of focus

What is this paper?

• An intuitive reformulation of general linear cameras in terms of eigenvectors

• An analogous description of focus

• A theoretical framework for understanding and characterizing linear slices and integral projections of ray space

Slices of Ray Space

• Perspective View• Image(x, y) = L(x, y, 0, 0)

Slices of Ray Space

• Orthographic View• Image(x, y) = L(x, y, x, y)

Slices of Ray Space

• Image(x, y) = L(x, y, P(x, y))

• P determines perspective

• Let’s assume P is linear

Slices of Ray Space

P

Slices of Ray Space

Slices of Ray Space

• Rays meet when:

((1-z)P + zI) is low rank

• Substitute b = z/(z-1):

((1-z)P + zI) = (1-z)(P – bI)

• Rays meet when:

(P – bI) is low rank

Slices of Ray Space

• 0 < b1 = b2 < 1

Slices of Ray Space

• b1 = b2 < 0

Slices of Ray Space

• b1 = b2 = 1

Slices of Ray Space

• b1 = b2 > 1

Slices of Ray Space

• b1 != b2

Slices of Ray Space

• b1 != b2 = 1

Slices of Ray Space

• b1 = b2 != 1, deficient eigenspace

Slices of Ray Space

• b1 = b2 = 1, deficient eigenspace

Slices of Ray Space

• b1, b2 complex

Slices of Ray Space

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Equal Eigenvalues

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Equal Eigenvalues

Equal Eigenvalues,

2D Eigenspace

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Equal Eigenvalues

One slit at infinity

Equal Eigenvalues,

2D Eigenspace

Projections of Ray Space

y

xP

y

xL

y

x,Image

Projections of Ray Space

dudvv

u

y

xP

y

xL

y

x

,Image

Projections of Ray Space

dudvv

u

y

xP

v

uF

y

xL

y

x

,Image

Projections of Ray Space

• Rays Integrated at (x, y) = (0, 0):

F

Projections of Ray Space

• Rays meet when:

((1-z)I + zF) is low rank

• Substitute b = (z-1)/z:

((1-z)I + zF) = z(F – bI)

• Rays meet when:

(F – bI) is low rank

Projections of Ray Space

• 0 < b1 = b2 < 1

Projections of Ray Space

• 0 < b1 = b2 < 1

Projections of Ray Space

• b1 = b2 < 0

Projections of Ray Space

• b1 = b2 < 0

Projections of Ray Space

• b1 = b2 = 1

Projections of Ray Space

• b1 = b2 = 1

Projections of Ray Space

• b1 = b2 > 1

Projections of Ray Space

• b1 != b2

Projections of Ray Space

• b1 != b2

Projections of Ray Space

• b1 != b2

Projections of Ray Space

• b1 != b2 = 1

Projections of Ray Space

• b1 != b2 = 1

Projections of Ray Space

• b1 != b2 = 1

Projections of Ray Space

• b1 = b2 != 1, deficient eigenspace

Projections of Ray Space

• b1 = b2 != 1, deficient eigenspace

Projections of Ray Space

• b1 = b2 = 1, deficient eigenspace

Projections of Ray Space

• b1 = b2 = 1, deficient eigenspace

Projections of Ray Space

• b1, b2 complex

Slices of Ray Space

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Equal Eigenvalues

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Equal Eigenvalues

Equal Eigenvalues,

2D Eigenspace

Slices of Ray Space

Real Eigenvalues

Complex Conjugate Eigenvalues

Equal Eigenvalues

One focal slit at infinity

Equal Eigenvalues,

2D Eigenspace

Projections of Ray Space

• Let’s generalize:

Projections of Ray Space

• Let’s generalize:

dudv

v

uP

y

x

v

uF

y

xL

y

x,Image

Projections of Ray Space

• Let’s generalize:

dudv

IP

FIL

y

x

v

u

y

x

Image

Projections of Ray Space

• Let’s generalize:

dudvQL

y

x

v

u

y

x

Image

Projections of Ray Space

• Factor Q as:

A

M

IP

FI

APM

FAMQ

0

0

Projections of Ray Space

• Factor Q as:

• M warps lightfield in (x, y)– warps final image

A

M

IP

FI

APM

FAMQ

0

0

Projections of Ray Space

• Factor Q as:

• M warps lightfield in (x, y)– warps final image

• A warps lightfield in (u, v)– shapes domain of integration (bokeh, aperture size)

A

M

IP

FI

APM

FAMQ

0

0

Conclusion

Conclusion

• General Linear Cameras can be characterized by the eigenvalues of a 2x2 matrix.

Conclusion

• General Linear Cameras can be characterized by the eigenvalues of a 2x2 matrix.

• Focus can be described in the same fashion.

Conclusion

• General Linear Cameras can be characterized by the eigenvalues of a 2x2 matrix.

• Focus can be described in the same fashion.

• These matrices are a good way to analyze and specify linear integral projections of ray space.

Questions