8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 1/25

OBJECT BASED 2D TO

3D CONVERSION

1College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 2/25

CONTENTS

TYPICAL 3DTV SYSTEM 3D GENERATIONTECHNIQUE

DISADVANTAGES & SOLUTIONS

2DTO 3D CONVERSION:EXISTINGMETHODS OF DEPTH EXTRACTION

NOVEL METHODS

WH

Y2D

TO 3D?

LIMITATIONS

FUTURE OF 3DTV

College Of Engineering,Chengannur 2

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 3/25

TYPICAL 3D TV SYSTEM

3College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 4/25

3D GENERATION TECHNIQUE

Stereo camera setting

Different perspectives for left and right eyes

Separate recording

4College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 5/25

DISADVANTAGES & SOLUTIONS

Disadvantages Highly skilled camera operators

Costly equipment

Solutions Infrared cameras

Depth-image-based rendering (DIBR)techniques

2D-to-3D conversion most effective

5College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 6/25

2D TO 3D CONVERSION: EXISTING

METHODS OF DEPTH EXTRACTION1.Depth From Geometric Constraints

2.Depth from Focus/Defocus(Blur) Analysis

3.Depth From Motion4.Other Approaches

6College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 7/25

1.Depth From Geometric

Constraints Depth extraction using geometric constraints

Geometric perspectives vanishing points

and lines Planar representation of background

Foreground geometrically modelled

Disadvantage Controlled environmentrequired

7College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 8/25

2.Depth from Focus / Defocus

(Blur) Analysis Depth determination by modeling the effect

of varying focal parameters on the image

Inverse filtering Drawbacks

Blurs unavailable in general case

Blurs due to external factors

o Lens aberration

o Atmospheric interference

o Blur due to motion

8College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 9/25

3.Depth From Motion

Based on vision geometry

Modified time difference method(MTD) to

detect object motion & estimate imagepresentation delay time

Disadvantages� Overall strategy is still heuristic

� Unable to deal with occlusions

� No method for recovery of layered motions

9College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 10/25

4.Other Approaches

Use of edge information to generate sparse

depth maps for DIBR

Depth determination using supervisedmachine learning

Color based image segmentation

10College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 11/25

NOVEL METHODS

1.Determining depth ordinal using optical-flow

based occlusion

2.Object segmentation3.Hybrid depth estimation scheme

11College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 12/25

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 13/25

Principles of Occlusion

reasoning Relies on optical flow to find the motion of 

each object and segment a scene to find the

occlusion Rules of Occlusion Reasoning

� Forward Reasoning: Object visible in nth frame andinvisible in (n+1)th frame

� Backward Reasoning : Object visible in (n+1)th frameand invisible in nth frame

College Of Engineering,Chengannur 13

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 14/25

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 15/25

Using optical based flow

occlusion

College Of Engineering,Chengannur 15

[(a) and (b)]Two consecutive fames from Flower Garden sequence.

(c)The optical flow of (a). (d) Layered representation of objects in (a). (e)The

occlusion map from (a) to (b). (f)The occlusion map from (b) to (a).

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 16/25

2.OBJECT SEGMENTATION

Two steps

Depth ordinal considered as initial

segmentation Extracting masks of each depth layer

� Seeded-Region growing method

16College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 17/25

Seeded region growing method

Let be binary mask of a depth layer in animage; be remaining mask in the imageexcluding

Refinement process� Skeletons of and extracted and denoted as So

and Sb and used for growing of foreground andbackground regions , respectively.

� Ro-foreground region ; Rb -background region� Initially Ro=So and Rb=Sb

� In each loop , all outer neighbouring boundary pixelsof Ro and Rb obtained as Co and Cb

College Of Engineering,Chengannur 17

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 18/25

Seeded region growing method

The minimum distance between Co and Cb to

Ro and Rb determined as do and db.

If do < db , Co is grown to Ro

If do> db , Cb is grown to Rb

With updated values of Ro and Rb repeat

above steps till no change in Ro and Rb ismade

College Of Engineering,Chengannur 18

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 19/25

3.HYBRID DEPTH ESTIMATION

SCHEME Recovery of Depth Maps

� Matching objects with existing libraries

� T

extural representation using local binary pattern(LBP)

19College Of Engineering,Chengannur

One example to show relationships among (left) disparity maps and(middle) texture maps via stereo pair using the left-image of (right)

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 20/25

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 21/25

ADVANTAGES & LIMITATIONS

ADVANTAGES Reduction in overall cost

Reuse of existing 2-D libraries

Cheapest and most efficient option suitablefor academic research labs

LIMITATIONS

Conversion process can be time-consuming

Degraded accuracy in sequences of high andmedium complexity

Occurrence of depth discontinuities

College Of Engineering,Chengannur 21

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 22/25

FUTURE OF 3D TV

3DTV without glasses

The potential of sets that contain tiny

cameras which track the viewers position Holographic technology

College Of Engineering,Chengannur 22

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 23/25

REFERENCE

Yue Feng; Jinchang Ren; Jianmin Jiang; ,

"Object-Based 2D-to-3DVideo Conversion for

Effective Stereoscopic Content Generation in3D-TVApplications," Broadcasting, IEEE 

Transactions on , vol.57, no.2, pp.500-509,

June 2011

College Of Engineering,Chengannur 23

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 24/25

24College Of Engineering,Chengannur

8/6/2019 Object Based 2d to 3d Conversion (3)

http://slidepdf.com/reader/full/object-based-2d-to-3d-conversion-3 25/25