Post on 05-Jul-2015
description
3D Shape and Indirect Appearance by Structured Light Transport
Matthew O’Toole John Mather Kiriakos N. Kutulakos
Department of Computer ScienceUniversity of Toronto
http://www.dgp.toronto.edu/~motoole/slt
1. indirect-only imaging
1. indirect-only imaging
2. indirect-invariant imaging
1. indirect-only imaging
2. indirect-invariant imaging 3. one-shot, multi-pattern imaging
contributions
using epipolar geometry to analyze light transport
contributions
using epipolar geometry to analyze light transport
new live video channels for computer vision• indirect-only and direct-only imaging
• indirect-invariant imaging of structured patterns
• one-shot, multi-pattern imaging
contributions
using epipolar geometry to analyze light transport
new live video channels for computer vision• indirect-only and direct-only imaging
• indirect-invariant imaging of structured patterns
• one-shot, multi-pattern imaging
structured light transport for 3D shape acquisition
contributions
basic light paths
projector
basic light paths
projectorcamera
basic light paths
camera projector
basic light paths
direct light
camera projector
basic light paths
indirect light(scattered)
camera projector
mirror
basic light paths
indirect light(specular)
camera projector
mirror
basic light paths
1. indirect-only imaging
camera projector
mirror
basic light paths
camera projector
mirror
epipolar constraint & light transport
direct paths satisfy epipolar constraints
camera projector
mirror
epipolar constraint & light transport
indirect paths almost never satisfy constraints
camera projector
mirror
epipolar constraint & light transport
indirect paths almost never satisfy constraints
projectorcamera
blocking epipolar paths with patterns & masks
mirror
camera projector
mirror
projection pattern
mask pattern
blocking epipolar paths with patterns & masks
camera projector
mirror
blocking epipolar paths with patterns & masks
projection pattern
mask pattern
camera projector
mirror
blocking epipolar paths with patterns & masks
complementary random epipolar patterns
camera projector
mirror
blocking epipolar paths with patterns & masks
complementary random epipolar patterns
camera projector
mirror
blocking epipolar paths with patterns & masks
complementary random epipolar patterns
camera projector
mirror
blocking epipolar paths with patterns & masks
complementary random epipolar patterns
camera projector
mirror
1. open electronic shutter2. for i = 1 to N
use random epipolar mask &project complementary pattern
3. close electronic shutter
blocking epipolar paths with patterns & masks
complementary random epipolar patterns
live indirect-only and direct-only video stream
conventional imaging
live indirect-only and direct-only video stream
conventional imaging indirect-only imaging
live indirect-only and direct-only video stream
conventional imaging indirect-only imaging
direct-only imaging
live indirect-only and direct-only video stream
conventional imaging indirect-only imaging
direct-only imaging
live indirect-only and direct-only video stream
conventional imaging indirect-only imaging
direct-only imaging
live indirect-only and direct-only video stream
conventional imaging indirect-only imaging
direct-only imaging
live indirect-only and direct-only video stream
conventional imaging indirect-only imaging
direct-only imaging
separating direct & indirect components [e.g. Achar et al. 2013]• multiple shots, motion processing
• only low-frequency indirect light (i.e. diffuse)
transport-robust shape acquisition [e.g. Gupta et al. 2012]• assumption: only high-freq patterns can be robust to indirect light
• no specular transport
structured light transport• live video, fully-general transport, no motion processing
• any projection pattern can be made robust to indirect light
• new forms of one-shot 3D imaging
related work
structured light transport: theory
structured light transport: theory
dominance of non-epipolar transport
transport matrixmask
illumin.
deriving mask and illumination patterns
• exact derivation of patterns is NP-hard [Zhong 2012]
• instead, use random codes, and approximate imaging in expectation
structured light transport: theory
dominance of non-epipolar transport
2. indirect-invariant imaging
live indirect-invariant video stream
conventional structured light
indirect-invariant structured light
mirror
shape acquisition by structured light
mirror
cameraprojector
shape acquisition by structured light
mirror
cameraprojector
shape acquisition by structured light
conventional structured light
reconstructed 3D shape
indirect-invariant structured light
reconstructed 3D shape (same algorithm)
conventional structured light
reconstructed 3D shape
indirect-invariant structured light
reconstructed 3D shape (same algorithm)
3. one-shot, multi-pattern, indirect-invariant imaging
one-shot, multi-pattern, indirect-invariant imaging
one-shot, multi-pattern, indirect-invariant imaging
reconstructed 3D shape
• a new general imaging technique for computer vision
• combines epipolar geometry and light transport analysis
• theoretical analysis & a physical device for live video generation
• SIGGRAPH 2014: extension to time-of-flight imaging
structured light transport
3D Shape and Indirect Appearance by Structured Light Transport
Matthew O’Toole John Mather Kiriakos N. Kutulakos
Department of Computer ScienceUniversity of Toronto
http://www.dgp.toronto.edu/~motoole/slt
Poster ID: O-4A-4