Title 44pt sentence case
Affiliations 24pt sentence case
20pt sentence case
© ARM 2017
Next-generation automotive image processing with ARM Mali-C71
Steve Steele
ARM Tech Forum Taipei
Director, Product Marketing
Imaging & Vision Group, ARM
July 4th 2017
© ARM 2017 2
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Pioneers in imaging and vision
© ARM 2017 4
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Automotive ADAS growth
▪ Camera volume rising very rapidly, around 20% CAGR
▪ By 2023 (expected average):
▪ Mid-range (high-volume) cars: three cameras
▪ Luxury sedans: <ten cameras
© ARM 2017 5
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Automotive needs: viewability and reliability
Viewability
▪ Dynamic range
▪ Low-light and low-noise performance
▪ Imagery close to human eyes and at times
beyond
Reliability
▪ Low latency
▪ Resistant to random and systematic faults
▪ Imagery feeds into computer vision
engine
Night vision
Pedestrian
protection system
Driver drowsiness
detection
© ARM 2017 6
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Automotive needs: viewability and reliability
© ARM 2017 7
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Automotive needs: viewability and reliability
© ARM 2017 8
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Advanced vision requires centralised processing
ISP per camera sensor
▪ Less flexible
▪ Higher BOM costs
Centralised sensor fusion SoC
▪ Centralised processing
▪ Flexible architecture
Sensor
fusion
SoC
© ARM 2017 9
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
And there are standards !!
▪ A number of functional safety standards exist:
▪ ISO 26262 – Road vehicles
▪ IEC 61508 – Electrical, electronic programmable
systems
▪ DO 254 – Electronics that fly e.g. in airplanes,
helicopters
▪ …
▪ …
© ARM 2017 10
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
▪ Relates to electrical and electronic systems used in automobiles
▪ Qualitatively assesses risk of hazardous operational situations
▪ Aims to avoid or control systematic failures and detect or control
random hardware failures, or mitigate their effects
ISO 26262-2011 road vehicles – functional safety
© ARM 2017 11
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
ISO 26262: goals
Provides an automotive safety lifecycle and supports tailoring the necessary activities during these lifecycle phases
Covers functional safety aspects of the entire development process
Provides an automotive-specific risk-based approach for determining risk classes (Automotive Safety Integrity Levels, ASILs)
Provides requirements for validation and confirmation measures to ensure a sufficient and acceptable level of safety is being achieved
© ARM 2017 12
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
High-performance ISP for automotive: ARM© Mali™-C71
Ultra-wide dynamic range▪ 24 stops
▪ Every detail captured
Simultaneous support for human display and computer vision▪ Virtual ISPs: <4 real-time camera inputs
▪ Pleasing and natural for human vision; predictable and reliable for computer vision
Every pixel reliable▪ Real-time safety for ADAS applications
▪ Enabling system-level ASIL D and SIL3 compliance
▪ 300+ fault detection circuits, built-in self-test (BIST), CRC on data paths, every pixel
tagged
Designed for safety
Ultra WDR
Multi-camera input
© ARM 2017 13
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Ultra-wide dynamic range
Revealed shadows
Enhanced mid-tones
Preserved highlights
0
2^24
2^8
0
0
2^8
Shadows details lost
Mid-tones not revealed
Highlights clipped
As perceived by the human eye
© ARM 2017 14
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Dynamic range management: iridix®
▪ More than just the world’s best local tone-
mapping engine…
• Precise model of human retina contrast
adaption
• Precisely simulates illumination by white light
• Basis of Nikon “D-Lighting” (2005)
• Shipped in over 2 billion devices
iridix enables cameras to see like the human eye
© ARM 2017 15
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
iridix® engine: pixel-by-pixel tone mapping
Contrast in highlights
enhanced
Contrast in midtones
preserved
Contrast in shadows
enhanced
iridix calculates a unique tone curve for each
pixel of each frame
Dyn
amic
ran
ge
© ARM 2017 16
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Management of dynamic range
© ARM 2017 17
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Management of dynamic range
© ARM 2017 18
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Supporting output for human and machine vision
Human display
Computer vision
© ARM 2017 19
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Enabling system level ASIL D compliance
▪ 300+ dedicated fault detection circuits
▪ Built in continuous self-test
▪ Fault interrupt controller
▪ Pixel consistency image plane tagging
reliability of every pixel
▪ Detection of sensor and hardware faults
▪ CRC checks to all data paths including
memories and configurations
Safety featuresDeveloped within robust requirements,
tracing and validation framework
▪ Safety manual
▪ Failure modes and effects analysis (FMEA)
▪ Development interface report
Providing support for system level:
▪ SIL 3 / IEC 61508
▪ ASIL D / ISO 26262
▪ With a standards-agnostic approach
Processes
Random
faults
Systematic
faults
© ARM 2017 20
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence caseDesigned to enable system-level certification
Safety Element out of Context (SEooC)
ISP has no direct control authority
System integrating ARM’s ISP needs to meet the requirements
• ISO26262 (ASIL D)
• IEC 61508
• SIL3
• Standard product
• Integrated into system (SoC) by our customers
• Specific use cases unknown to ARM
• Has to detect faults, perform self-tests
• Has a responsibility to provide timely, accurate and reliable information to the system
• Required for the system to be safe
• Road vehicles – functional safety
• This flows down to the components of the system
ARM© Mali™-C71 Image Signal Processor
© ARM 2017 21
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
A complete, high-performing solution
Control the ISP, sensor, auto-white
balance and auto-exposure
Roadmap to complete automotive
software designed for ASIL
compliance
Full set of tools provided
Ecosystem support for tuning and
bringing up specific use-cases and
sensors
1.2 Giga-pixels/sec of processing
performance
Supports up to 4 real-time camera
inputs and 16 camera streams with
a single pipeline
Performance Software Tuning
© ARM 2017 22
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Mali-C71 block diagram
Colour/data
plane outputs
Image sensor
sources
© ARM 2017 23
Title 40pt sentence case
Bullets 24pt sentence case
bullets 20pt sentence case
Mali-C71: the next step in automotive vision
First automotive ISP with ultra-wide dynamic range
Supports both human display and computer vision
Enables safer, more comfortable driving
The trademarks featured in this presentation are registered and/or unregistered trademarks of ARM Limited
(or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be
trademarks of their respective owners.
Copyright © 2017 ARM Limited
© ARM 2017
Thank you
謝謝 !
Top Related