TU Automotive Osram Presentation Final

www.osram-os.com

Infrared sensors for ADAS and beyond –

LIDAR / Infrared camera

Rajeev Thakur| 4th October 2016| Novi

Light is OSRAM

IR for ADAS | OS IR NA MK | R.Thakur

TU Automotive – ADAS & Autonomous | 10/04/2016

2

Content

Page

1. OSRAM Overview 03

2. Sensing challenges 06

3. LIDAR 12

4. Infrared Camera 19

5. Sensor Fusion 21

6. Collaboration & Competition in the self driving car business 22



3

Global Market Leader in LED & Laser

LIDAR – Infrared Lasers - AEB

Consumer

Industry

General Lighting

Laser front light

Xenon front light Laser front light

OLED rear light

Matrix LED light

Automotive Lighting

Source: OSRAM, excluding LAMPS1) at the end of the fiscal year2) countries where OSRAM had operations at the end of the fiscal year

Employees1) : 20,300

Worldwide Presence2) : >120 countries

Revenue1): 3,571.9 m €



4

Key Automotive Trends

Exterior , Interior & IR

Safety

Design

Visualization &

Connectivity

Comfort &

Safety

Key Automotive Trends

Exte

rior

Inte

rio

r

• µAFS

• High Luminance

LEDs

Dynamic

Lighting

Projection

HuDFull Digital Cluster

LED Applications New LED

Development

• Display Portfolio

• HuD Portfolio

BLU Displays

High ResolutionADB/Matrix

ProjectionUltra slim HL

LIDAR / ADB

Gesture

Wireless

Connectivity

Driver Monitoring

Night Vision



5

OSRAM Infrared & Laser Automotive Applications

Existing Applications / New Applications

Rain Light

Tunnel Sensors

Ambient light sensors for dimming

and illumination

• Dashboard

• Car radio

• Displays

Immobilizer

Steering wheel

angle sensor

Blue Lasers for

Headlamps

Driver monitoring

Gesture Recognition

IRED based Night vision

Blind spot detection

Lane departure warning

Family Entertainment

System

LIDAR sensing

AEB & ADAS

Laser HuD



6

Sensing Needs for Vehicle Environment

Continue

Brake

Steer to safety Prepare for crash

Steering is

best option Cannot avoid

crash

Braking is

best option

NO

YES

Current Sensing Range

Upper limit

(For Large Objects)

RADAR : 50 - 250m

Camera : 50 - 70m

LIDAR : 50 – 200m

Braking Distance / Minimum Sensing Range

(Assumptions : Dry Road, µ = 0.7, 1 sec reaction time)

@100mph (161kph/44.7m/s) : > 190 meters

@74mph (119kph,33 m/s) : > 112 meters

@45mph (72.4kph/20.1 m/s) : > 50 meters

@25mph (40kph/11.1 m/s) : > 20 meters

If the closing speed is less than ~ 45mph , current sensing technology can mitigate

collision to large objects under normal daylight dry conditions (distance < 70 meters)

Challenge 1 : Sensing Range

Is projected vehicle

trajectory safe for next

XX meters?

Calculate Time

to Crash



7


• Who decides ?

• Ignore objects at own risk ..

Challenge 2 : Angular Sensing Resolution

• Standard object list for detection does not exist (ignore / standardize with risk)

• LIDAR is capable of < 0.5° resolution at > 100 m (with small form factor)

• RADAR size for 0.5° resolution not practical (~ 0.5 m for 76 GHz RADAR)

• Camera range needs to improve & image quality in lowlight (or infrared)

1 Bosch Multi Purpose Camera (MPC2) , 1280 x 960 pixels, 50° HFOV, 28° VFOV 2 Velodyne VLP16 (0.1° – 0.4°) 3 RADAR equation

What objects should be detected to avoid collision ?

Typical Angular Resolution 1 Camera : 25 pixels / °2 LIDAR : 0.3°3 RADAR : 2.6° (76 GHz, 10 cm aperture)

1.5m 0.25m 0.4m 0.2m 0.1m 0.2m

tire

piecepotholedog Resolution Size (m)

1° @ 100 m = > 1.7 m

1° @ 200 m = > 3.4 m

0.1° @ 200 m = > 0.4 m



8


Challenge 3 : Field of View

Winding Roads

Need Wide FOV

Traffic Lights & Overhead Signs Need High FOV Up & Down Ramps Need High FOV

FOV – Field Of View



9


Challenge 4 : Computational challenges

• Time Needed = Sensing time + Reaction Time + Safety Margin

• Sensing Time per Sensor = (Points/Frame x # of Frames in Buffer x

compute time/point)

• Finer resolution => More data points => more time (or faster

computation)

• Redundancy / sensor fusion needed prior to reaction

• Reaction Time = (Human delay) + latency in steering or braking

system

• Safety Margin : To accommodate environment conditions (road /

temperature) , sensing and computational delays and tolerances

1 Can we unify monocular detectors for autonomous driving by using the pixel-wise semantic segmentation of CNNs? : Department of Electronics, UAH. Alcala de ´

Henares, Madrid, Spain ; IEEE Workshop in June 2016 on Intelligent Vehicles

1

Sensor / Processor / environment / algorithm .. affects

computational time and accuracy



10


Other Challenges

• Form factor – small and compatible to vehicle styling & materials

• Increasing noise from surrounding vehicle RADAR/LIDAR ..

• Dealing with satellite signal / GPS loss in real time

• Harsh environment – Snow/rain/dust/dirt/shock and vibrations

• Power / EMC / ESD / ..

• Service

• Cost

• ..

• Tremendous innovation currently in sensing field

• OSRAM working with multiple startups / Tier1 and OEM’s



11

RADAR / Camera / LIDAR Comparison

Sensor Typical

Range

Horizontal

FOV

Vertical

FOV

2020

Price RangeComments

24 GHz

RADAR60 m 1 56° 1 ~ ± 20° < $100

USA Bandwidth 100 -250 MHz 2

Robust for Rain/snow ;

People Detection / Angular Resolution

77 GHz

RADAR200 m 1 18° 1 ~ ± 5° < $100

USA Bandwidth 600 MHz 2

Robust for Rain/snow ;

People Detection / Angular Resolution

Front Mono

Camera50 m 1 36° 1 ~ ± 14 ° < $100

Versatile Sensor (Applications)

Limited depth perception ; affected by rain / fog

Needs illumination (Visible/IR)

LIDAR

(Flash)75 m 140° ~ ± 5° < $100

Concerns for Rain/Snow;

Good reflection off people w/ angular resolution

Range & S/N limited by eye safety

LIDAR

(Scanning)200 m 360° ~ ± 14° < $500

Concerns for Rain/Snow;

Typically higher price for angular resolution

Range & S/N limited by eye safety

1 : Vehicle-to-Vehicle Communications: readiness of V2V technology for application – DOT HS 812014 ; Table V-7

2 : Millimeter Wave Receiver concepts for 77 GHz automotive radar in silicon Germanium Technology – D.Kissenger (SpringerBrief’s 2012)

• False positives Nuisance to consumer Turns feature off (if possible)

• False negatives did not meet spec / expectations

• Optimum combination of sensors will be a learning process

• Sensor fusion …can be done at best on common subset in field of view



12

Flash LIDAR – Design Overview

start

stop

Δt

Laser

Photodiode

Array

Emitter

Lens

Receiving

Lens

Target

FOV

Working principle : Laser beam spread into field of view and received on photodiode array. Range

determined by eye safe laser power , resolution determined by number of photodiode pixels

Why use : Mature low cost sensor that can be integrated into headlamp / Tail lamp / behind windshield / ..

Range : ~ 30 - 60 meters @ 24 HFOV

Resolution : 3 deg or less

Wavelength : 905 nm has proven sufficient for short range

Laser : OSRAM lasers with peak power 75 – 120W , with & without drivers , bare die to SMT w/ < 5ns pulse

width (2019 SOP) , also with multiple emitters in one SMT package

Photodiode : OSRAM PD array concepts of various sizes planned for SOP 2018

Why not as popular as RADAR yet in NAFTA?: 2019 NCAP upgrade will incentivize market , more room

for creativity lower cost than RADAR ; market waiting for low cost scanning LIDAR …

R – Distance

C – speed of light

Δt – time between start - stop of pulse

HFOV – Horizontal Field Of View

SOP – Start of Production

PD – Photo Diode



13

LIDAR Head Lamp Integration – LeddarTech Concept

Leddartech Video link

Video automobile applications wo TOC.mp4



14

LIDAR Tail Lamp Integration – LeddarTech Concept



15

Phantom Intelligence : Guardian Flash LIDAR

The GuardianBY PHANTOM INTELLIGENCE

Fully customizable

2x8 Pixels (1x16 also available)

Field of View 9°x36° – Customizable up to 2°x120°

Range limited to 30 meters (for cost optimization)

Connectivity: USB, CAN, GPIO

Programmable alarms/triggers

Power Consumption less than 3 Watts

Laser Output of 70 Watts

Eye Safe (Class 1M)

Price: ~ 100$ in 10k units volume production

Engineering Samples December 2016

AWL Video link

YOU CAN AFFORD THE SAFEST JOURNEY !

PhantomCES2015.wmv



16

LIDAR – Low Cost Concept (Reference Design)

• 30m range

• ~1cm accuracy

• 16 pixel array

• 24°H x 6°V FOV : 2 x 8 array (3° x 3°per pixel)

• Arrangement of pixels and field of view can be customized in future products.

• Multiple targets in each pixel can be resolved

• Targets ~1m apart (range) can be separated

• Differentiating through performance, small size,

scalability, and low power consumption

• No moving (scanning) parts

• Sun blinding can affect no more than a single pixel

• Estimated BOM ~ $25 @ High Volume

• Functional sample Q1 2017

• Target SOP 2019

Distance (m) Area (m²)

1 0.003

2 0.011

5 0.069

10 0.274

20 1.097

30 2.469

Field Of View Per Pixel

FOV – Horizontal Field Of View

SOP – Start of Production

BOM – Bill of Material (For Hard Ware)



17

Scanning LIDAR Technologies

Mechanical - Velodyne

Principle: Matched pair of laser &

detectors rotating with a motor at 5

to 20 Hz

Range : 200 m (VLP 32A)

Resolution : 0.1 - 0.4° (VLP 16)

Vertical FOV: 28° (VLP 32A)

Price Target : < $500 ~ 2020

Pro : Proven technology

Con: Mechanical integration / price

Principle: Laser scanned with

OPA (& received on SPAD array )

Range : > 150 m

Resolution : 0.1°

FOV: 120° (HFOV & VFOV ; S3)

Price Target : <$100 ~ 2020

Pro : small size (1” x 1.5” , S3-Qi)

Con: OPA scanning is relatively

new technology

1 Velodyne.com 2 Quanergy.com 3 Innoluce.com

OPA - Quanergy MEMS – Innoluce

Principle: Laser scanned with 1D

MEMS Mirror (& received on APD

array )

Range : > 200 m

Resolution : < 0.5°

HFOV: 80°

VFOV: 16°

Price Target : <$100 ~ 2020

Pro : MEMS scanning is proven

Con: Working demo not shown

yet…

MEMS – Micro Electro Mechanical Systems

APD – Avalanche Photo Diode

OPA – Optical Phase Array

SPAD – Single Photon Avalanche Diodes

FOV – Field Of View



18

MEMS LIDAR – Innoluce – OSRAM : Concept Proof

Principle: 1 25W OSRAM laser scanned with 1 Innoluce MEMS Mirror and received on an APD array

Range : ~ 60 m

Resolution : 0.1° Horizontal and 0.2° Vertical

HFOV: 10°

VFOV: 3°

Next Steps :

• Show progressively improved reference design demos in next few months

• Targets : >200 m/car ; > 60m/Ped ; 80° HFOV ; 16° VFOV ; < 0.5° Resolution (High power multiple emitter lasers)

High resolution concept proof

MEMS LIDAR Video Link

Pedestrian_between_car_and_hedge_on_45_meters_full span.mp4



19

Infrared Camera - Interior

Mature applications transitioning to mainstream

• Why IR Camera : Works in day & night without visible illumination

• Moving to Mainstream : Driver monitoring (Drowsy/Distracted)

• Catching speed : Gesture recognition

• Mobile to Automotive : Iris recognition

• Technology frontiers: NIR sensitivity (15 – 35%), > 2Mp Global shutter

, increasing IRED o/p & efficiency

• Concern/Tradeoffs: Privacy Vs App. value , Redglow (850 940 nm)

• Future applications : Optimum airbag deployment, Mood lighting ..

Driver Monitoring

1 Deltaid.com

Gesture Control

Iris Recognition

1



20

Infrared Camera - Exterior

Forward Camera

Surround View

Camera

Rearview

Camera

Cars have to be autonomous at night also …Cameras need to work with IR also ..

• Why IR Exterior Camera : Need to see adjacent lanes at night w/o visible light

• What’s the problem : Visible cameras block IR for better image / use of color information

• Options : Use mechanical or SW filter to switch between IR & visible spectrums

• Challenges : Modify camera / Illumination / SW for wider FOV and range

• Things to watch out for : Laser beam headlamps (Dynamic range of oncoming camera) /

Use of matrix beam lighting (adaptive beams ..)

IR emitters in headlamp



21

Sensor Fusion – Fuse Information of high quality which overlap

• Objective of Sensor Fusion : Determine environment around vehicle trajectory

with enough resolution, confidence and speed - to navigate efficiently.

Object_list RADAR Camera LIDAR Sensor Fusion

Car@150mDon’t See it

(Noise)

Not_Classified@100

m & low light

Evaluate TTC &

brake if unresolved ?

@50m Person on bicycle

Not classified Don’t see it (Noise) Brake or ignore ?

Potholes & stuffWhat can be safely

ignored ?

• Object Identification & Classification in range & FOV of interest must be comparable.

• LIDAR + Camera fusion potentially better (Due to angular resolution)

• Camera improvements : Range (~ 70 m); speed (30 – 60 Hz) & Low light sensitivity



22

Collaboration & Competition - Self Driving Cars

• Why collaborate :

• Need NHTSA support Regulations / Testing / Infrastructure / ..

• Combine R&D resources & strengths

• Be / be with a technology leader to gain market share

• Why Compete?

• Branding / Technology Leadership (Intangible $ Value)

• ADAS technology has shown real market value ($1500 - $3000/car)

• Prepare for future market changes (Self driving cars occupy significant share)

• What more could/should be done ?

• Use Silicon Valley playbook more – open source development

• Example : Provide raw data from all sensors in a drive ; show me object

identification / classification & tracking .. (Buy the best solution..)

• Make sensor requirements and roadmap open

• Small startups have very creative solutions & fast development

• Why be more open ?

• 1 year after a new gadget is shown 3 more appear next year (Benefit/Cost)

• Will enable faster development of SDC technology for community & save lives !

www.osram-os.com

Thank you !

Contact :

[email protected]

TU Automotive Osram Presentation Final

Documents

Transcript of TU Automotive Osram Presentation Final