To Do Advanced Computer Graphics -...

8
1 Advanced Computer Graphics CSE 190 [Winter 2016], Lecture 14 Ravi Ramamoorthi http://www.cs.ucsd.edu/~ravir To Do § Assignment 2 due Feb 19 § Any last minute issues or questions? § Next two lectures: Imaging, Texture Synthesis § Then resume rendering, animation Computational Imaging § Digital cameras now commonplace § Can we use computation for better images § Many novel capabilities relative to film § And new ways of processing images § Is this computer graphics, optics, or image proc? § All of the above; many rendering ideas apply § Application shift. Computer aided design to movies/games to photography (big market) § Brief lecture. Subject of whole conference ICCP § Industry: Light Field cameras, Google glass, Outline § Image formation, basic lens-based camera § Light Field camera § Coded aperture depth of field § Flutter shutter (coded aperture shutter) § Many many more old, new innovations How do we see the world? Lets design a camera G Idea 1: put a piece of film in front of an object G Do we get a reasonable image? Slide by Steve Seitz Pinhole camera Add a barrier to block off most of the rays G This reduces blurring G The opening known as the aperture G How does this transform the image? Slide by Steve Seitz

Transcript of To Do Advanced Computer Graphics -...

Page 1: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

1

Advanced Computer Graphics

CSE 190 [Winter 2016], Lecture 14

Ravi Ramamoorthi

http://www.cs.ucsd.edu/~ravir

To Do

§  Assignment 2 due Feb 19 §  Any last minute issues or questions?

§  Next two lectures: Imaging, Texture Synthesis §  Then resume rendering, animation

Computational Imaging

§  Digital cameras now commonplace

§  Can we use computation for better images § Many novel capabilities relative to film

§  And new ways of processing images

§  Is this computer graphics, optics, or image proc? §  All of the above; many rendering ideas apply §  Application shift. Computer aided design to

movies/games to photography (big market)

§  Brief lecture. Subject of whole conference ICCP §  Industry: Light Field cameras, Google glass, …

Outline

§  Image formation, basic lens-based camera

§  Light Field camera

§  Coded aperture depth of field

§  Flutter shutter (coded aperture shutter)

§  Many many more old, new innovations

How do we see the world?

Let’s design a camera   Idea 1: put a piece of film in front of an object   Do we get a reasonable image?

Slide by Steve Seitz

Pinhole camera

Add a barrier to block off most of the rays   This reduces blurring   The opening known as the aperture   How does this transform the image?

Slide by Steve Seitz

Page 2: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

2

Pinhole camera model

Pinhole model:   Captures pencil of rays – all rays through a single point   The point is called Center of Projection (COP)   The image is formed on the Image Plane   Effective focal length f is distance from COP to Image Plane

Slide by Steve Seitz Figures © Stephen E. Palmer, 2002

Dimensionality Reduction Machine (3D to 2D)

3D world 2D image

What have we lost?   Angles   Distances (lengths)

Funny things happen… Parallel lines aren’t…

Figure by David Forsyth

Lengths can’t be trusted...

Figure by David Forsyth

B’

C’

A’

…but humans adapt!

http://www.michaelbach.de/ot/sze_muelue/index.html

Müller-Lyer Illusion

We don’t make measurements in the image plane

Page 3: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

3

Camera Obscura

The first camera   Known to Aristotle   Depth of the room is the effective focal length

Camera Obscura, Gemma Frisius, 1558

From Pinhole to Lenses Computer graphics assumes pinhole model But making aperture narrow limits light Making aperture large causes blurriness Real cameras have lenses to collect more light,

and focus it on the image plane (Kolb et al. 95 simulates lens effects rendering)

Home-made pinhole camera

http://www.debevec.org/Pinhole/

Why so blurry?

Shrinking the aperture

Why not make the aperture as small as possible?   Less light gets through   Diffraction effects…

Less light gets through

Slide by Steve Seitz

The reason for lenses

Slide by Steve Seitz

Focus and Defocus

A lens focuses light onto the film   There is a specific distance at which objects are “in focus”

–  other points project to a “circle of confusion” in the image   Changing the shape/separation of lens changes this distance

“circle of confusion”

Slide by Steve Seitz

Page 4: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

4

Thin lenses

Thin lens equation:

  Any object point satisfying this equation is in focus   What is the shape of the focus region?   How can we change the focus region?   Thin lens applet: http://www.phy.ntnu.edu.tw/java/Lens/lens_e.html (by Fu-Kwun Hwang )

Slide by Steve Seitz

Depth of Field

http://www.cambridgeincolour.com/tutorials/depth-of-field.htm

Outline

§  Image formation, basic lens-based camera

§  Light Field camera

§  Coded aperture depth of field

§  Flutter shutter (coded aperture shutter)

§  Many many more old, new innovations

Light Field Inside a Camera

Lenslet-based Light Field camera

[Adelson and Wang, 1992, Ng et al. 2005 ]

Light Field Inside a Camera Stanford Plenoptic Camera [Ng et al 2005]

4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens

Contax medium format camera Kodak 16-megapixel sensor

Adaptive Optics microlens array 125µ square-sided microlenses

Page 5: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

5

Digital Refocusing

[Ng et al 2005]

Demo at: https://pictures.lytro.com/

Mask based Light Field Camera Mask Sensor

[Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]

1/f0

Mask Tile

Cosine Mask Used Captured 2D Photo

Encoding due to Mask

[Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]

2D FFT

Traditional Camera Photo

Heterodyne Camera Photo

Magnitude of 2D FFT

2D FFT

Magnitude of 2D FFT

Outline

§  Image formation, basic lens-based camera

§  Light Field camera

§  Coded aperture depth of field

§  Flutter shutter (coded aperture shutter)

§  Many many more old, new innovations

Page 6: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

6

Out of Focus Photo: Coded Aperture

Engineering the PSF when you cannot capture Lightfield

In Focus Photo

LED

2D Photo

Out of Focus Photo: Open Aperture Out of Focus Photo: Coded Aperture

Captured Blurred Photo

[Veeraraghavan, Raskar, Agrawal, Tumblin, Mohan, Siggraph 2007 ]

Refocused on Person

Increase DoF + large aperture

Page 7: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

7

Outline

§  Image formation, basic lens-based camera

§  Light Field camera

§  Coded aperture depth of field

§  Flutter shutter (coded aperture shutter)

§  Many many more old, new innovations

Traditional Camera

Shutter is OPEN

Our Camera

Flutter Shutter

Shutter is OPEN and CLOSED

Lab Setup

Blurring

==

Convolution

Traditional Camera: Box Filter

Sync Function

Page 8: To Do Advanced Computer Graphics - cseweb.ucsd.educseweb.ucsd.edu/~ravir/190/2016/lectures/190-lecture14.pdf · §Then resume rendering, animation Computational Imaging § Digital

8

Flutter Shutter: Coded Filter

Preserves High Frequencies!!!

Comparison

Inverse Filter Unstable

Inverse Filter stable

Input Image

Rectified Crop

Deblurred Result