Object Stereo 1

Object Stereo

2014/3/25 Yichong Bai

Object Stereo 2

Problem

• Image pair Disparity map + Object level segmentation• “A combined algorithm for stereo matching and object

segmentation”

2014/3/25 Yichong Bai

Object Stereo 3

Overview

• Object level• Segmentation in object level• Model in object level

• Pixel level• Pixel-corresponding needed

• 3D connectivity term• Works with occlusion• Slow, 20m/p

2014/3/25 Yichong Bai

Object Stereo 4

Difference with ours

• Object level too much semantics• Initial disparity map needed• Too slow

2014/3/25 Yichong Bai

Object Stereo 5

Model

• Scene Representation• Coarse to fine• Object = object plane + parallax• Parallax, surface-based representation

• Object• Color model• Parallax model• Object plane

2014/3/25 Yichong Bai

Object Stereo 6

Model

• Problem Formulation• F: Pixel Planes ( Depth)• O: Pixel Object•

2014/3/25 Yichong Bai

Object Stereo 7

Model

• Terms• Photo Consistency

• Pixel dissimilarity• Occlusion• Corresponding pixels same depth plane and object•

•

2014/3/25 Yichong Bai

Object Stereo 8

Model

• Terms• Object Coherence

•

2014/3/25 Yichong Bai

Object Stereo 9

Model

• Terms• Depth Plane-Coherency

•

2014/3/25 Yichong Bai

Object Stereo 10

Model

• Terms• Object-Color

• GMM• the probability that a pixel lies inside the object according to its color value

2014/3/25 Yichong Bai

Object Stereo 11

Model

• Terms• Object-Parallax

• regularizes the disparities of an object with respect to its object plane• parallax is likely to be compact• non-parametric histogram•

• is histogram probability

2014/3/25 Yichong Bai

Object Stereo 12

Model

• Terms• Object-MDL

• Small no. of objects

2014/3/25 Yichong Bai

Object Stereo 13

Model

• Terms• 3D Connectivity Prior

• Connectivity = same object OR occluded• Approximation:

• We randomly sample a large set of pairs of points p and q, where p and q belong to different connected components of the object. We then draw a line between p and q and check if all pixels on the line satisfy condition (10).

2014/3/25 Yichong Bai

Object Stereo 14

Solve

• Energy Minimization• Fusion move• Proposal generator• Like generics algorithm

• Proposal Fuse Evaluate

2014/3/25 Yichong Bai

Object Stereo 15

Solve

• Proposals• Initialization

• We first compute a disparity map F using the fast stereo matcher• F is now derived by fitting a plane to each color segment using the initial

disparity map• To derive O, we take the color segmentation result and group segments that

have similar depths according to F• GMM: EM• 30 Initial proposals, with difference segmentations and parameters

2014/3/25 Yichong Bai

Object Stereo 16

Solve

• Iteration• Refit

• Fix F, refine O• Color model, Object plane, Parallax model

• Expansion• derived by setting all pixels of F to f and all pixels of O to o

2014/3/25 Yichong Bai

Object Stereo 17

Evaluation

• Middlebury benchmark• Works best on cone set (artificial planar scene)

2014/3/25 Yichong Bai