"Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," a Presentation from...

Copyright © 2017 Basler AG 1

Mark Hebbel, Head of New Business Development

May 2017

ToF Sensors: How Do I Choose Them

and How Do I Integrate Them?


Basler AG

Industrial

and

network

cameras

Area scan

and

line scan

cameras

CCD

and

CMOS

sensors

USB 3.0, GigE

Fast Ethernet,

FireWire,

Camera Link,

BCON

Interfaces

www.baslerweb.com

Basler is the largest unit volume

producer of digital cameras for

industrial applications in the world.


• What is ToF? (for beginners)

• When you integrate the sensor

• Remember the speed of light

• Synchronisation, lighting, shutter

• Which sensors are available?

• Let other people do the hard work: the ToF module

• Multiple module systems – things to remember

Agenda


What is ToF?


• ... to analyze volume, shape or 3D position of objects

• ... to detect parts and defects that are low-contrast but have a detectable

height difference

• ... to read imprinted code or text when contrast information is missing

Why 3D?

2D vision

doesn’t...

3D vision shows

height differences.


ToF Use Cases

Robot belt picking Box content verification Robot assembly

• Find the optimal

gripping position

• Verify presence and

position

• Verify position and

presence

• Measure co-planarity


ToF Use Cases

Food portioning Robot bin picking Obstacle recognition

• Give the total volume

• Find optimal cutting

positions

• Unload bins with

randomly oriented parts

• Find the best gripping

coordinates of the most

suitable part to pick

• Make 3D image of the

surroundings

• Bypass obstacles


ToF Advantages and Disadvantages

• Large field of view

• Low processing power

• Fast integration and ease of use

• No contrast requirements

• One component only

• No moving parts

• Compact

• High Speed

• Strong ambient light (e.g. Sunlight)

• Multipath and scattered light

• Depth accuracy: 1 cm

(Compared to other

3D techniques)


• Send out modulated waves of light

ToF Principles

I’m not going to talk about single pixel ToF (or 3 or 12 pixels). This is a vision summit.


ToF 101 Maths

𝑑 =𝑐

2∙ 𝑡𝑝 ∙

𝑆1𝑆0 + 𝑆1


• Influences the accuracy of the measurement and the kind of sensor and

electronics to be used

What Kind of Wave or Light Signal?


• Advantage

• Works with old LED technology

• Disadvantages

• Sensitive to daylight, small opening angle

• Requires a special pixel design resulting in larger pixels

ToF with Sine Waves


• Advantage

• More accuracy and

• With the right LED optics a large opening angle

• Smaller pixels

• Disadvantage

• Needs use of modern “expensive” LED technology (laser,

VCSEL diodes)

ToF with Square Wave


Integrating the Sensor


• Three types of sensor architecture

• Electronic fast shutters

• Electronics causes large pixel sizes. No amplification

• Demodulation devices

• Popular at the moment

• SPAD (single-photon-avalanched-diodes) based

• At the moment very low resolutions

Possible Sensor Architectures


• High Sensitivity

• Low noise

• High dynamic range

• High fill factor

• High QE in the IR

• High demodulation contrast

• High modulation frequency

• Low modulation power

Requirements for a Demodulation ToF Sensor

Common to all image sensors

Specific for ToF sensors


Available Raw Sensors

Frame Rate Power

Pmd/Infineon IRS1125C 352x288 Bare die 45 fps 300 mW

Panasonic MN34092BL 640x480 Packaged 30 fps -

TI OPT8241 320x240 COG (78) Up to 150 fps ?

Melexis MLX75023 320x240 BGA package 600 fps “raw” 400 mW

Samsung 5K32A 320x240 ? ? ?

Espros EPC660 320x240 Packaged 65 fps 0.75 W

All numbers are from the datasheets (where available) and not tested.


• ToF sensors are available in the range from $100 to $500

• Volumes of 10 k/year

• Maximum range? “up to 9 m” or “up to 240 m” depending on system

design

Soft Specifications


• Light is fast, very fast. (3x10^8 m/s or 1 m in 3.33x10^-9 s)

• Electrical signal is 95.1% of that

• Temperature stability (Cu 16.6 x10^-6 m/(m K) )

• 10K is 0.02% change of length

• PCB track length is important

• Lighting trigger

• Calibration is important

Integrating Electronics@Speed of Light


• Sheet of white paper

• 0.1 mm accurate positioning

• 22°C room temperature

• 20 minutes sensor + electronics warm-up time

• No ambient light

• 40x40 pixels in the center of the range map

• 25 images

Calibrating the Sensor


Accuracy of the Sensor Setup



Spatial noise



Temporal noise


Lighting: Multi-Path Effect


Lighting: Scattered Lighting Effect


Let Other People Do It – The ToF Module


• Processors are out, SoMs are in

• Let the others do the

• Light integration

• And calibration

• Get a ready integrated sensor and light in a module

ToF Modules


• Heptagon (TARO)

• Softkinetic

• Basler

Available ToF Modules


Multiple ToF Modules


• Interfering IR light signals from your own system

• Don’t point cameras at themselves

• Synchronise the cameras somehow (for instance PTP)

Watch Out With Multiple Modules


Trends in Sensor Development


• What to expect in the future

• Smaller pixels and higher resolution

• More combined 2D and 3D imaging

• Reduced power consumption for mobile applications

Trends in Sensor Development


• ToF has a high speed advantage and with combined sensors (normal +

depth) provides more info with one device

• ToF is “tricky” when a lot of ambient light is about or several ToF modules

are used in the same space

• Building a ToF camera is complex due to the calibration involved

• ToF technology is improving rapidly and expect big progress especially in

resolution and power consumption in the future

Summary


• Thank you for your attention

Questions and Answers


Resource

Stereo Vision Structured Light Laser

Scanner

Time of Flight

Range Medium Medium Short High

XY-Resolution Medium Medium Depends High

Depth Accuracy Medium Medium to high High Medium

Software Complexity High Medium High Low

Real Time Capability Low Medium Low High

Low Light Weak Good Good Good

Outdoor Good Weak Medium Currently Weak

Compactness Medium Medium Weak Good

Material Cost Medium Medium to High High Medium to low


• White Paper – Time-of-Flight Cameras

(http://www.baslerweb.com/en/products/cameras/3d-cameras/time-of-

flight-camera/white-paper-time-of-flight-cameras)

Resource

http://www.baslerweb.com/en/products/cameras/3d-cameras/time-of-flight-camera/white-paper-time-of-flight-cameras


• Sensors (datasheets as of 22.02.2017)

• EPC https://www.espros.com/index.php/portfolio_item/epc660/

• TI http://www.ti.com/product/OPT8241

• Pmd/infineon http://www.infineon.com/dgdl/Infineon-

REAL3+Image+Sensor+Family-PB-v01_00-

EN.PDF?fileId=5546d462518ffd850151a0afc2302a58

• Melexis https://www.melexis.com/en/product/MLX75023/Automotive-

QVGA-Time-of-Flight-Sensor

Resource

https://www.espros.com/index.php/portfolio_item/epc660/

http://www.ti.com/product/OPT8241

http://www.infineon.com/dgdl/Infineon-REAL3+Image+Sensor+Family-PB-v01_00-EN.PDF?fileId=5546d462518ffd850151a0afc2302a58

https://www.melexis.com/en/product/MLX75023/Automotive-QVGA-Time-of-Flight-Sensor

"Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," a Presentation from...

Technology

Transcript of "Time of Flight Sensors: How Do I Choose Them and How Do I Integrate Them?," a Presentation from...