A Scalable Heterogeneous Multicore Architecture for ADAS€¦ · Recognition (TSR) Night vision...

A Scalable Heterogeneous Multicore Architecture for ADAS

Presented at HOT CHIPS: A Symposium on High Performance Chips

Flint Center, Cupertino, CA

August 23-25, 2015

Zoran Nikolić*

Rama Venkatasubramanian*

Jason A.T. Jones*

Peter Labaziewicz*

*Embedded Processing Business Unit, Texas Instruments Inc. 1

Presentation Overview

• Agenda

– Highlight challenges of implementing Advanced Driver Assistance Systems

(ADAS) in embedded systems

– Discuss ADAS system options and compromises

– High level overview of TDAx SOC

– Mapping ADAS use cases to devices from TDAx SOC families

2

Worldwide Road Traffic Fatalities

3

Source World Health Organization

Leading Cause %

1. Ischemic heart disease 12.2

2. Cerebrovascular disease 9.7

3. Lower respiratory infections 7.0

4. Chronic obstructive pulmonary disease 5.1

5. Diarrheal diseases 3.6

6. HIV 3.5

7. Tuberculosis 2.5

8. Trachea, bronchus, lung cancers 2.3

9. Road traffic injuries 2.2

10. Prematurity and low birth weight 2.0

Leading Cause %

1. Ischemic heart disease 14.2

2. Cerebrovascular disease 12.1

3. Chronic obstructive pulmonary disease 8.0

4. Lower respiratory infections 3.8

5. Road traffic injuries 3.6

6. Trachea, bronchus, lung cancers 3.4

7. Diabetes mellitus 3.3

8. Hypertensive heart disease 2.1

9. Stomach cancer 1.9

10. HIV 1.8

Total 2004

Total 2030

~2 million

1.3 million

Eliminating Human Error Can Save Lives

• According to Tri-Level Study of the Causes of Traffic Accidents

published by NHTSA in 1979, "human errors and deficiencies" are a

definite or probable cause in 90-93% of the incidents examined.

• By eliminating human errors that cause traffic accidents ADAS can

save lives, reduce severity of injuries and reduce property damage

4 Source NHTSA

Video

180° FOV Video

180° FOV

ADAS Surround View - Applications Summary

Sensor Type

Application

Vision Infrared Long

Range Radar 76..81MHz

Short / Mid

Range Radar 24..26 / 76..81 GHz

Lidar

Adaptive Front Lighting (AFL), Traffic Sign

Recognition (TSR)

Night vision (NV)

Adaptive Cruise Control (ACC)

Lane Departure Warning (LDW)

Low-Speed ACC, Emergency Brake Assist

(EBA), Lane Keep Support (LKS)

Pedestrian detection X

Blind Spot Detection (BSD), Rear Collision

Warning (RCW), Cross Traffic Alert (CTA) X

Park Assist (PA) X

Camera monitor systems (CMS)

Driver Monitor

Video

0 to 80m

ADAS Front Camera - Applications Summary

Sensor Type

Application



Short / Mid


Lidar


Recognition (TSR) X

Night vision (NV)

Adaptive Cruise Control (ACC) X

Lane Departure Warning (LDW) X


(EBA), Lane Keep Support (LKS) X



Warning (RCW), Cross Traffic Alert (CTA)

Park Assist (PA)


Driver Monitor

Video

ADAS Driver Monitor

Sensor Type

Application



Short / Mid


Lidar


Recognition (TSR)

Night vision (NV)





Pedestrian detection



Park Assist (PA)


Driver Monitor X

Infrared

0.2 to 120m

ADAS Night Vision Applications Summary

Sensor Type

Application



Short / Mid


Lidar


Recognition (TSR)

Night vision (NV) X








Park Assist (PA)


Driver Monitor

LRR

1 to 280m SRR/MRR

0.2 to 160m

SRR

0.2 to 90m

ADAS Radar Applications Summary

Sensor Type

Application



Short / Mid


Lidar


Recognition (TSR)

Night vision (NV)

Adaptive Cruise Control (ACC) X X







Park Assist (PA) X


Driver Monitor

Lidar

ADAS Lidar Applications Summary

Sensor Type

Application



Short / Mid


Lidar


Recognition (TSR)

Night vision (NV)

Adaptive Cruise Control (ACC) X







Park Assist (PA) X


Driver Monitor

Video

ADAS CMS- Applications Summary

Sensor Type

Application



Short / Mid


Lidar


Recognition (TSR)

Night vision (NV)








Park Assist (PA)

Camera monitor systems (CMS) X

Driver Monitor

Mono Front Camera Block Diagram

12

DD

R FLA FLASH

QSPI,

GPMC

TDA2x CMOS

Sensor Video In

Vehicle

CAN bus

CAN0

SPI,

ADC

TP

S659039 o

r

TP

S65917

PM

IC

CAN1

I2C

16

bit/sample

GPIO

CAN

25/27MHz

RE

SE

T_O

UT

C

LK

_O

UT

CLK&RST

MON.

Braking MCU Steering MCU

MC

U

16

DD

R DD

R2/3

16

ADAS SOC BASED SYSTEM: D

DR

2/3

• Compute performance increase must come without compromising overall system cost.

• The system needs to be packaged in a miniature enclosure and must deliver maximum compute performance while dissipating minimum heat in order to operate at the extreme temperatures.

• The opposing requirements create a very challenging environment.

LM

53602

DC

/DC

Battery

ADAS Market Trends

13

Various Stages of ADAS Vision Applications

• Low level processing is characterized

by repetitive operations at pixel level

requiring high computational

requirements and memory bandwidth.

Low level processing is typically best

served by applying single instruction

on multiple data (SIMD).

14

Low Level

Processing

High Level

Processing

Medium Level

Processing

20-40%

20-30%

40-50%

• High-level processing is typically

responsible for decision making and

tracking and takes input from previous

processing stages.

• Mid level Processing has focus on certain objects or

regions of interest that meet particular classification

criteria (mid-level processing). Mid-level vision is

typically best served by using some combination of

SIMD and multiple instructions on multiple data (MIMD).

The TDA SoC Family Architecture

• One platform multiple HW and SW compatible products

15

TDA2Sx

TDA2Hx

TDA3x

Two ARM®

Cortex® -A15 Two

C66X

DSP One ARM®

Cortex® -A15

Two

DDR2/3

I/F

Two

C66X

DSP

One EVE

One

LP/DDR2/3 I/F

Two

EVEs

Four

EVEs

POWERVR™

SGX544-MP2 IVA-HD

ISP

TDAx Processing Pipeline Support through Optimized Heterogeneous Architecture

ARM/DSP for Control &

High-level vision stages

ARM Cortex A or M:

Scalable RISC

Data Fusion

Memory Coherency

DSP:

VLIW SIMD+MIMD

Data Fusion

EVE Vector Coprocessor:

High Bandwidth Pixel

Operations

SIMD Parallelism

Energy Efficiency

Hardware Acceleration:

High Bandwidth Pixel

Operations

Configurable HW

Acceleration

ENABLING ALGORITHMS Emergency braking

High beam assist

Lane keep assist

Adaptive cruise control

Traffic signal recognition


Collision warning

Complete HW & SW Scalability

High

Mid

Entry

Front Cam SV Fusion Rear Cam Radar

TDA2xA

ADAS

Processor

TDA3xA

ADAS

Processor

TDA2xV

ADAS

Processor

TDA3xV

ADAS

Processor

TDA2xF

ADAS

Processor

TDA3xF

ADAS

Processor

TDA3xR

ADAS

Processor

TDA3xV

ADAS

Processor

TDA2x scalable family (superset)

Package

– 23x23mm BGA (ABC) -

– 17x17mm BGA (AAS) - reduced feature set

Two Next Generation DSP Cores: C66x™ – Upto 750 MHz – 12GMpy/s (16bx16b->32b)

– Floating Point Extension / 24GFLOPs

Dual ARM Cortex™ A15 Cores – Upto 750MHz – 5250DMIPs

– NEON Vector Floating point

Two dual ARM Cortex™ M4 Cores – 200 MHz

Four Vision Accelerator Cores: EVE – Each core has an 16MAC per cycle computing engine

with up to 650 MHz (8bit or 16bit ) – 10.4GMACs per core

Video Codec Accelerator – IVA-HD core running at up to 532MHz

Graphics Engine – Dual SGX544 core delivering capability to render upto

166Mpoly/s / 5000MPixel/s / 31.9GFLOPs at 500Mhz

Internal Memory – DSPs: each w/ 32 KB L1D, 32 KB L1P, unified 288 KB L2

Cache

– ARM : 32 KB L1D, 32 KB L1P, combined 2 MB L2 Cache

– On Chip L3 RAM: 2.5MB with ECC

Peripherals Highlights (1.8/ 3.3V IOs)

– Up to three Video input Ports(total of 10 parallel video inputs)

– Display system Digital Video Output

– Two EMIFs: 2x 32bit wide DDR2/3/3L @ 532MHz, one with ECC

– GPMC: general purpose memory controller

– Support for NOR Flash

– PRU Subsystem

– PCIe, Gbit EMAC with AVB support

– 2x DCAN (High end CAN controller)

– 10x UART, 5x I2C, 4x McSPI, Quad SPI, McASP, 15x Timers, WDT, GPIO

2.5MB L3 RAM w/ECC

DDR2/3 32b w/ECC

DDR2/3 32b

WDT EDMA

System Services

Connectivity and IO

16 Timer

+ * - <<

C66x DSP + * - <<

C66x DSP

EV

E

ARM M4 ARM M4

High Speed Interconnect

Display Subsystem

Overlay

GFX Pipeline Video Pipeline

DVOUT HDMI

Video Front End 3 Video Input Ports

ARM A15

Video Codec Accelerator IVA HD 1080p Video

EVE

EVE

EVE

PCIe GMAC x2

NAND/NOR

DCAN

x2

UART x10

McASP JTAG QSPI GPMC

System Mailbox System x13

I2C x5

ARM A15 ARM M4 ARM M4

2x

SGX544

McSPIx4

USB

GC320

TDA3x scalable family (superset)

Two DSP Cores: C66x™ – Up to 750 MHz

– Floating Point Extension

Dual ARM Cortex™ M4 Cores – 200 MHz

Vision Accelerator Core: EVE – Core has an 16MAC per cycle computing engine with up

to 650 MHz (8bit or 16bit )

Image Signal Processer (ISP) – 200 MHz, CSI/HiSPI

Internal Memory – DSPs: each w/ 32 KB L1D, 32 KB L1P, unified 288 KB L2

– ARM : 32 KB L1D, 32 KB L1P, 64kB L2 Cache

– On Chip L3 RAM: up to 512kB with ECC

Peripherals Highlights (1.8/ 3.3V IOs)

– One Video input Port, with two 16 bit sub ports

– Display system Digital Video Output including SD-DAC

– One EMIF: 32bit wide DDR 2/3 or LPDDR 2 @ 400MHz with ECC

– GPMC: general purpose memory controller

– Support for NOR Flash

– 2x Gbit EMAC with AVB support

– 2x DCAN (High end CAN controller)

– 3x UART, 2x I2C, 4x McSPI, Quad SPI, McASP, 8x Timers, GPIO

Safety Support – 7 x Dual Clock Compare (DCC)

– Error Signaling Module (ESM)

– Run-Time BIST(TESOC)

– CRC

– 5xRTI

– Memory Protection Firewall (MPF)

Power (~1.0V Core, 1.8/ 3.3V IOs) – Target @ 125C Tj 1.5W at 500MHz 1x DSP & EVE, <1W at 250MHz 1x

DSP & EVE. Power will vary depending on use case.

Package – 15x15mm BGA, 0.65 mm ball pitch

– 12x12mm BGA PoP, 0.65 mm ball pitch (offered up to 350MHz DSP/EVE)

– Temperature Range: -40C to 125C Tj

512kB L3 RAM w/ECC

LPDDR2/DDR2/3 32b w/ECC

Connectivity and IO

28 nm

Parallel Video Input Port 2 Video Inputs

GMAC GPIO

DCAN

x2

SPI x4

UART

x3

McASP JTAG QSPI GPMC

System Mailbox Module x2 Spinlock Module x1

I2C x2

Imaging Sub System

CSI2/HiSPI Interface

DCC x 7

Safety

TESOC ESM

EDMA – 2TC

System Services

8 Timer


+ * - <<

C66x DSP

EVE ARM M4 ARM M4

CRC

MMU

5xRTI

PRCM COTROL Module

10bit 8ch. ADC

SDIO x1

Display Sub System + VENC

2x video, 1x GFX, 1 write-

back pipeline

PWMSS

+ * - <<

C66x DSP

MPF

Key Feature Deltas Between TDAxxx Superset Devices

Features TDA2x TDA3x

Pro

cesso

rs

Cortex ARM A15 1..2 with 2MB of shared L2

cache -

Dual Cortex ARM M4 2 1

C66x 1..2 1..2

EVE 1..4 1

Vid

eo

In

CSI2 Ports - One (1x 4 lane )

Image Signal Processing (ISP) - Yes. (With LDC and HDR)**

Parallel Video Inputs Up to 10 parallel video inputs

(6x12bit and 4x8bit)

Up to 4 video parallel inputs

(4x8bit)

Peri

ph

era

ls

Internal On Chip Memory (L3

Memory) 2.5MB 512KB

External Memory Interface 1 DDR3/3L 32 bit @533MHz

(DDR2 @400MHz) w/ ECC

1x 32 bit w/ECC

LPDDR2@400MHz OR

DDR3/3L @532 MHz

External Memory Interface 2 DDR3/3L 32 bit @533MHz

(DDR2 @400MHz) -

Video Codec Accelerator Yes (H.264, MJPEG etc.) -

Graphics 2x SGX544 -

Display Subsystem 3 Digital Video Outputs

(1GFX and 3 x VID Pipes)

1 Digital and 1 NTSC/PAL Out

(1GFX and 2 x VID Pipes)

Runtime Built in Self Test - Yes

10-bit ADC - Yes

I2C 5 2

Package 23mm and 17mm 12mm POP and 15mm

20 **) Hardware accelerator for Pixel Remapping/Lens Distortion Correction and HDR available in 15mm package of TDA3x

TDA3x POP Package Overview

• Package on Package (PoP)

• POP for automotive study concluded with high confidence of

meeting automotive requirements

• TI has 10+ years of practical POP experience and >100M units

shipped

• PoP Advantages

• Lower cost memory:

• Multiple memory suppliers mitigate supply risk

• Supply continuity and Customer control

• Customer Field Returns:

21

LPDDR2 / Flash

TDA3x

System Board

McSPI2

i2c

video Enet PHY RGMII

Debug Only

EMIF0

12

Timer

i2c

VIN1a/

HiSPI/

CSI2

Vehicle CAN bus

CAN PHY

MCU

CAN

TPS54340

DC/DC

Battery

LP3907

DC/DC

LP8731

DC/DC

VOUT1 DS90UB

913A

Bt656 or RGB888

Digital Output

TMS570LS

LPDDR2 +

QSPI in PoP

32

QSPI

TDA3x PoP

Optional

TDA3x Smart Rear View Camera – Digital Output

TPS54340

DC/DC

TDA3x LVDS Surround View (w/o External MCU)

23

FLASH QSPI

(or GPMC)

VOUT

CSI2

DS

90

UB

960

i2c

video

i2c

RGMII

Debug

Only

DS

90

UB

91

3

EMIF

RGB888

SD-DAC

DS

90

UB

92

5

To console

or cluster

24

Coax

Enet PHY

Battery

1Mpix

imaging

sensor

LP8731

Or LP8733

LP3907

or discrete

i2c

DS

PE

VE

CO

RE

5V

FS

i2c

video

DS

90

UB

91

3 1Mpix

imaging

sensor FS

i2c

video

DS

90

UB

91

3 1Mpix

imaging

sensor FS

i2c

video

DS

90

UB

91

3 1Mpix

imaging

sensor FS

TPS62170

1.8V

POWER OVER

COAX

TDA3x

TPS62261

TLV700xx

TPS62170

1.8V TPS62261

TLV700xx

TPS62170

1.8V TPS62261

TLV700xx

TPS62170

1.8V TPS62261

TLV700xx

DDR3L

32

TPS55340

TPS1H100

TPS51200

CAN PHY

Dual ARM

Cortex M4

DCAN

TDA2 Ethernet Surround View-High Level Block Diagram

24

GPMC

VOUT1 VIN1a

MDIO

RGMII

i2c

RGMII

EMIF0

UART

32MB

FLASH

32

64 MB

Vehicle CAN bus

MCU

CAN

SPI

LM53603

DC/DC

Battery

CAM#0

CAM#1

CAM#2

CAM#3

8

Analog Video In NTSC

Decoder

CAN PHY

Camera Supply 1

Camera Supply 2

Camera Supply 3

Camera Supply 4 TPS1Hxxx

Protected

HS Switch

TMS570LS

Po

lar

Sw

itch

TDA2x

TPS659039 or TPS65917

PMIC

DDR2/3L

DS90UB

925

RGB-444,

RGB-565,

RGB-666,

RGB-888

To console or cluster

Safety Evolution of TI SOCs

TDA1x TDA2x TDA3x

QM ASIL-A ASIL-B

Diagnostic Features

ISO26262 Compliant

Development Flow

System Integration Effort

512kB L3 RAM w/ECC

Connectivity and IO

28 nm

Parallel Video Input Port 2 Video Inputs

GPIO SPI x4

UART

x3

QSPI GPMC

System Mailbox Module x2 Spinlock Module x1

I2C x2

Imaging Sub System

CSI2/HiSPI Interface

Safety

TESOC ESM

System Services


+ * - <<

C66x DSP

EVE ARM M4 ARM M4

CRC

MMU

5xRTI

COTROL Module

SDIO x1

Display Sub System + VENC

2x video, 1x GFX, 1 write-

back pipeline

+ * - <<

C66x DSP

Overview of functional safety and diagnostic mechanisms for TDA3x

Overflow detection

mechanisms for video and

display ports

Interconnect safety features

Firewalls to ensure memory isolation

Error handling and abort interrupts

Timeout mechanisms

Statistics and perf monitor module

Bandwidth regulation

Ability to run s/w defined diagnostics

Additional Safety features

Clock monitoring

Error Signaling Module

Voltage monitoring

Interrupt routing to off-chip

CRC engines

On chip PBIST and LBIST

Temperature sensing

CLKOUT from PLL to off chip

Multiple clock, and power

domains to avoid common

cause fails

Fully ECC protected M4

subsystem

2 DSPs for redundant calculations

Parity on L1P $, SECDED on L2 $

Page based memory protection

Privilege modes and MMU to ensure

“freedom from interference”

EVE Engines -

Error detection in vector core & interrupt generation

Undefined instruction trap on scalar core

Memory parity errors detection as well as an MMU

Configuration lock mechanisms and CRC self test

S/W diagnostic

based voltage

monitoring for core

voltages

PWMSS

DCAN

x2

PRCM

LPDDR2/DDR2/3 32b w/ECC

10bit 8ch. ADC

JTAG McASP

GMAC

Window Watchdog functions

Message passing through

mailbox

Multiple timers for consistency

and redundancy

SECDED protection of Internal L3

RAM and External Memory

Interface EDMA – 2TC 8 Timer

DCC x 7

Power Dissipation Estimates for TDAx SoC – at Tj=105C

27

Tj=105C Frequency

[MHz]

Utilization

[%]

1xC66x 250 80

1xEVE 250 80

CortexM4 212 80

ISS+CSI2 200 80

TOTAL POWER 1.1 W

NOTES:

• All Power Estimates Assume Tj of 105 °C and Use of Smart Reflex (Adaptive Voltage Scaling).

• Assumed 80% Utilization for Compute Cores (C66x and EVE) is very conservative.

• Power figures shown represent a rough estimate based on a hypothetical use cases. For more accurate power

dissipation estimate a precise description of target use case is required (list of all used IPs along with their utilization and

clock frequencies, DDR type, DDR data width, DDR frequency and data-throughput).

Tj=105C Frequency

[MHz]

Utilization

[%]

2xC66x 500 80

1xEVE 500 80

CortexM4 212 80

VIP1 100

TOTAL POWER 1.3 W

Tj=105C

Frequency

[MHz]

Utilization

[%]

2xC66x 500 80

1xEVE 500 80

CortexM4 212 80

ISS+CSI2 200 80

TOTAL POWER 1.7 W

Tj=105C Frequency

[MHz]

Utilization

[%]

2xC66x 500 80

2xEVE 500 80

2xCortex A15 1176 80

2xICSS 200 100

GMAC_SW 125 100

TOTAL POWER 4.4W

Tj=105C Frequency

[MHz]

Utilization

[%]

2xC66x 500 80

2xEVE 500 80

1xCortex A15 500 80

CortexM4 212 80

VIP1 100

TOTAL POWER 2.3 W

ADAS Applications –SoC & SW

28

Scalable Performance Low Power

Safety

Front Camera

Integrate 2D/3D Graphics Scalable Analytics

Security

Surround View

Park Assist Rear Camera

Low Power Small Footprint

Scalable Analytics

Radar

Scalable performance MCU Integration

Safety

Core Applications

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware • AutoSAR MCAL

• Vision SDK • Framework • DSP/EVE Libs • Starterware • Diagnostics

Libs • AutoSAR MCAL • ISP Tuning Tool • ISP Example

Algos/Drivers

TDA2x TDA3x TDA3x

• Vision SDK • Framework • DSP/EVE Libs • Starterware • Diagnostics

Libs • AutoSAR MCAL • ISP Tuning Tool • ISP Example

Algos/Drivers

TDA2x TDA3x

TDA3x

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware

• Diagnostics

Libs • AutoSAR MCAL

• Linux SDK ->

InfoADAS

Package

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware • AutoSAR MCAL

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware • ISP Tuning Tool

• ISP Example Algos/Drivers

• Diagnostics

Libs • AutoSAR MCAL

Vision SDK

One SDK spanning all Processors & Applications

Enables scalable SW development

ADAS Applications –SoC & SW

29

TDA2x

Fusion

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware

• Diagnostics Libs

• AutoSAR MCAL

TDA3x

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware


• ISP Tuning Tool


• AutoSAR MCAL

TDA3x

• Vision SDK

• Framework

• DSP/EVE Libs

• Starterware


• ISP Tuning Tool


• AutoSAR MCAL

Sensor Fusion

Performance Safety

Security

Performance ISP Integration

Scalable Analytics

Mirror Replacement

Small Footprint ISP Integration Scalable

Analytics

Driver Monitoring

Emerging Applications

Vision SDK

One SDK spanning all Processors & Applications

Enables scalable SW development

ADAS Vision SDK: Two Flavors

BIOS

Drivers

(BSP)

FC

IPC 3.x

Starterware

SysBIOS

SDK Use Cases:

Front Camera, Rear Camera, Stereo

Camera, Radar, Fusion, 2D Surround View,

Mirror Replacement (CMS)

Encode/De

code: H264,

MJPEG

LINKS

Capture, Display,

VPE, AVB

Encode, Decode,

Algorithm

SOC: TDA2x, TDA3x

HW: EVM,

Demo

Algorithms

DSP, M4, EVE, [A15]

BIOS

Drivers

(BSP)

FC

IPC 3.x

SysBIOS

SDK Use Cases:

3D Surround View (EVM),

Autonomous Vehicle Project

Encode/D

ecode:

H264,

MJPEG

LINKS

Capture, Display,

VPE, AVB

Encode, Decode,

Algorithm

SOC:TDA2x,

HW: EVM

DSP, M4, EVE A15

IPC

3.x

Linux

Linux

Drivers

LINKS

Algorithm,

3D SV

Open GL

SGX

Vision SDK - BIOS Vision SDK - Linux

30

Demo

Algorithms

Starterware

Conclusions

• Continuous demand for performance increase in ADAS systems is in

conflict with miniaturization of the camera module, use of plastic

enclosures and cost reduction.

• It has been shown that given the diverse compute requirements for the

various algorithms in ADAS functions, a heterogeneous multicore

architecture like the TI TDA2x/3x SOC platform is required to provide a

scalable ADAS system solution.

• Further, functional safety and compliance to ISO26262 standard is an

orthogonal requirement that has to be satisfied for systems

implementing ADAS functions.

31

Thank You

www.ti.com/adas

32

A Scalable Heterogeneous Multicore Architecture for ADAS€¦ · Recognition (TSR) Night vision...

Documents

Transcript of A Scalable Heterogeneous Multicore Architecture for ADAS€¦ · Recognition (TSR) Night vision...