1

SISTEM PENCITRAAN PENDETEKSI SISTEM PENCITRAAN PENDETEKSI POLARISASI CAHAYA MENGGUNAKAN POLARISASI CAHAYA MENGGUNAKAN

SISTEM VISI STEREOSISTEM VISI STEREO

Mohammad IqbalDisampaikan pada 24 Februari 2012

Di Kuliah Umum Pasca Sarjana Universitas Gunadarma

2

1 – 2 – 3 – 4 - 5

Itot

• Intensity

• Wave length – color

• Polarization

The fact of light in Nature

Introduction

3

1 – 2 – 3 – 4 - 5

Introduction

• Polarization is a property of waves that describes the orientation of their oscillations

• Light can be polarized by several processes :• Selective Absorption – Dichroism• Reflection• Scattering• Birefringent

What is Polarization of Light?

4

1 – 2 – 3 – 4 - 5

IntroductionPolarization of Light Model

Type of Polarized Light• Unpolarized light: random phase• Polarized light :

– Linear polarization– Circular polarization– Elliptical Polarization

5

1 – 2 – 3 – 4 - 5

IntroductionParameter of Polarization of Light

Itot

• Total Intensity (Itot)• Phase (δ) : phase shift between orthogonal transverse

• δ=0 linear polization• δ =±π/2 circular polarization.

• Angle of Polarization (ϕ): main direction of the electromagnetic vibration

• Degree of Polarization (ρ): proportion of the polarized light

• Perfectly polarized wave = DOP of 100%, • unpolarized wave = a DOP of 0%. • Partially polarized, = DOP somewhere in between 0 and 100%.

6

1 – 2 – 3 – 4 - 5

Introduction

• Useful for navigation tasks :– Self Localization – Orientation– Detection of water, food, prey or obstacle

• Bio-inspired examples :– terrestrial animal

• visual ability to analyze light pattern in the sky or in the reflected surfaces.

– marine animals • Camouflage or communication• Enhance the visibility of the scenes

• Most of animal have a stereo capabilities to aware scene around.

Why Polarization & Stereo ?

7

1 – 2 – 3 – 4 - 5

Introduction

The ChallengePolarization Imaging : The images are ‘darker’ than intensity images, need at least three different images.

Stereo vision : matching point problem, need clear images, different view, need more geometric approach.

HOW TO GET A WIN-WIN COMBINATION?

8

1 – 2 – 3 – 4 - 5

Introduction

• To build a prototype of stereo vision system with polarization sensitivity :–Can measure DoP and AoP for every

angle of incident light.–Can reconstruct 3D point of stereo

images

• To develop a simple and fast polarization imaging algorithm based-on stereo vision

–Simple and Efficient in setup and algorithm

–Easy to Use–Not expensive

The Objective

9

1 – 2 – 3 – 4 - 5

Combination Polarization & Stereo ImagingLearning from Previous Researches

Wolff• Increase the polarization

parameters estimation• No 3D information

Wolff et al (1990, 1994, 1995)

• Stereo video polarimetrysystem to visualize the polarization patterns in stereovision

• Displacement of the camera

Mizera et al 2001

1

2

Sarafraz 2009• Two images are taken

simultaneously with different polarization filter settings

• Only the degree of polarization is estimated from the ratio of the images difference.

• No 3D reconstruction

3

BeamSplitter

1 camera

Multi camera

10

1 – 2 – 3 – 4 - 5

Itot

Setup Design and CalibrationSetup Design and Calibration• Description of System• Polarization calibration• Geometric calibration

11

1 – 2 – 3 – 4 - 5

Description of SystemPrinciple of Design

Itot

• Stereovision => 2 cameras• Measurement of partially linearly polarized light =>

at least need 3 images• Automatic acquisition => Liquid crystal components

12

1 – 2 – 3 – 4 - 5

Description of SystemExample of Captured Polarized light Images

Itot

Right CameraLeft camera

45°0°90°0°

13

1 – 2 – 3 – 4 - 5

ItotMain Capabilities

Calibration

Stereo Evaluation

Description of SystemPolarization-Stereo Imaging System Schema

Image Acquisition

Calibration

Polarization Calibration

Stereo Geometric Calibration

Extract Extract Polarization Polarization InformationInformation

Remove Outlier

3D 3D ReconstructionReconstruction

Feature Detection

Rectifying Image

Stereo Matching

14

1 – 2 – 3 – 4 - 5

Imaging System Calibration1 - Polarization Calibration

Itot

•Why Calibration is Important ?–Misalignment of optical devices–Polarizers settings may be different–To provide accuracy result

• Principle–Provide incident light with known

polarization state–Estimate the offset of the LC

component

Rotating polarizer LC component

βα = 0°-180°

s s’ s"

Mpol MLC

s"= MLC(β) . Mpol(α) . s

15

1 – 2 – 3 – 4 - 5

Imaging System Calibration2 - Geometric Calibration

Itot

This geometric calibration step is providing :

‐ Intrinsic parameters‐Extrinsic parametersOf stereo cameras 

Using Bouguet’s Toolbox

16

1 – 2 – 3 – 4 - 5

Itot

Stereo Vision Evaluation• Rectifying Image• Feature Detector• Stereo Matching

17

1 – 2 – 3 – 4 - 5

Stereo Vision Evaluation1 – Rectifying Image

Itot

Epipolar geometry

Image Rectification

18

1 – 2 – 3 – 4 - 5

Stereo Vision Evaluation2 – Feature Detector

Itot

• Criteria to choose the feature detector :– to deal with the polarization effect

on images– Less complexity and not expensive

computation

• The features to be extracted can be grouped into three main classes [1], namely:– Low-level (e.g. colour, gradient,

motion)– Mid-level (e.g. edges, interest point

; corners, regions)– High-level (objects)

[1] Cavallaro and Maggio, 2011

Design scheme for compare the capability of Harris

Corner detector and SIFT feature detector algorithm in

Polarization effect

19

1 – 2 – 3 – 4 - 5

Stereo Vision Evaluation2 – Feature Detector (Cont.)

Itot

1. Harris Result

2. SIFT Result

20

1 – 2 – 3 – 4 - 5

Stereo Vision Evaluation2 – Feature Detector (Cont.)

ItotPrimary results show that SIFT feature detector is more

appropriated to work on polarized images than the Harris Corner detector.

Conclusion Feature Detector Experiment

Left Image Right Image

21

1 – 2 – 3 – 4 - 5

Stereo Vision Evaluation3 – Stereo Matching Score Algorithm

Itot

• Three different metrics have to be taken Choosing Local Matching Score for Polarized Images– Test 6 Local matching score algorithm : SAD, SSD, NSSD, NCC, Census,

Rank for Polarized Images

Rank metrics

Correlation based

Intensity differences

22

1 – 2 – 3 – 4 - 5

Stereo Vision Evaluation3 – Stereo Matching Score Algorithm (Cont.)

Mean of SAD Error Result for Six Matching Algorithm For Each Scenes Incident Light (0°,10°,20°,30°,45°) –

Lowest is the best

Conclusion

Normalized SSD algorithm gave better matching results when applied to polarized images, compared to other local matching algorithms

23

1 – 2 – 3 – 4 - 5

Itot

Main Capabilities• 3D reconstruction• Polarization estimation• Evaluation of System

Capabilities

24

1 – 2 – 3 – 4 - 5

3D Recontruction1 – Remove Outliers

RANSAC considers the features that do not fit the current geometric model as outliers and eliminates them in an iterative manner and the geometric model is estimated again on the basis of newly identified inliers.

Putative Match Inlying Matches

Fischler and Bolles, 1981

25

1 – 2 – 3 – 4 - 5

3D Reconstruction2 – Recovery 3D Coordinates

Triangulation need :1. The relative position and

orientation of the two cameras (intrinsic and extrinsic camera parameters) from geometric calibration result

2. Correspondence point From matching

point

26

1 – 2 – 3 – 4 - 5

3D Reconstruction2 – Recovery 3D Coordinates (Cont.)

First experiment : • extract all points in

the stereo images rather than using a feature detector algorithm.

27

1 – 2 – 3 – 4 - 5

3D Reconstruction2 – Recovery 3D Coordinates (Cont.)

Visual Result 3D Recontruction

Images for 3D Reconstruction : (I0left and I0right )

Feature Detector (SIFT)

Matching score(NSSD)

Remove Outliers (RANSAC)

Triangulation The reconstructed 3D points still need to be increased in quantity. 

Second experiment :• Using points from feature

detector algorithm.

28

1 – 2 – 3 – 4 - 5

Polarization Estimation1 – Our Method

The Computation is :• Based on the polarization calibration result. • Only in the point matches from stereo vision

part results. • Using Least mean Square of 4 Pol Intensity

to get S0, S1 and S2

right right right0 0 1 0 2 0

right right right45 0 1 45 2 45

1I ( S S cos 2 S sin 2 )21I ( S S cos 2 S sin 2 )2

α α

α α

⎧ = + +⎪⎪⎨⎪ = + +⎪⎩

left left left0 0 1 0 2 0

left left left90 0 1 90 2 90

1I ( S S cos 2 S sin 2 )21I ( S S cos 2 S sin 2 )2

α α

α α

⎧ = + +⎪⎪⎨⎪ = + +⎪⎩

We using four polarizer intensity images through a set ofpolarization filters in stereo system : I0left, I0right,  I45right

and I90left

29

1 – 2 – 3 – 4 - 5

Polarization Estimation2 – How to Vizualize Polarization (AoP and DoP)

(x,y) AOP /AOP /Angle ofPolarization

DOP /DOP /Degree of

Polarization

(x2,y2)

(x1,y1)

10°

20°

30°

30

1 – 2 – 3 – 4 - 5

Evaluation of System Capabilities1 – Homogenous Scene

0° and 90°

0° and 45°

For Stereo Matching using: left I0+I90 and right I45

Result :

Combination Polarization Intensity for Input Imaging System

A-First Test

B-Enchanced Result Test

31

1 – 2 – 3 – 4 - 5

Evaluation of System Capabilities2 – Heterogenous Scene

• In nature, the light reflected from real objects would have many variations in orientations.

The experiments is to show how our setup has the ability to capture the variations of incident light and extract the polarization information with the

proper orientation

Result :

32

1 – 2 – 3 – 4 - 5

Resume Polarization-Stereo System22 33

44

5B5B

3D Recontruction by Triangulation

Extract features by SIFTStereo matching by NSSDRANSAC to remove outliers

Source image : • Stereo Evaluation :

• Class data 4 or• Class data 1 : I0left and

I0right

• Polarization Extraction :Point matches at :• I0left and I0right

• I90left and I45

right

11

22

33

44

5A5A

Extract polarization information5B5B

5A5A

33

1 – 2 – 3 – 4 - 5

Conclusion and Future Work• Conclusion• Future Work

34

1 – 2 – 3 – 4 - 5

Conclusion

In this work, we have done to develop :• Polarization Stereoscopic Imaging system Prototype:

– Polarization calibration technique– Feature detection and Stereo Matching evaluation algorithm– Extract Polarization Information from matching point of stereo

System– Tested the system on various real scenes.

• Prototype design consists of: – two optical devices, based on liquid

crystal : polarizer oriented at 0°-90°, and 0°- 45°.

– two intensity cameras.

– Grayscale images with resolution 640x480.

35

1 – 2 – 3 – 4 - 5

Future Work

Image Acquisition

Calibration

Polarization Calibration

Stereo Geometric Calibration

Extract Polarization Information

Remove Outlier

Feature Detection

Rectifying Image

Stereo Matching

RoboticsApplication

Object normal estimation

Improvement Photometric Invariant

Feature Detector

3D Reconstruction

36

1 – 2 – 3 – 4 - 5

Terima KasihThank Youmerci

Itot