The Development and Applications of Multi-core Multimedia · PDF file ·...

The Development and Applications of

Multi-core Multimedia Platform

Jefferey Chiu Ph.D.DSP System Senior FAETexas Instruments

Outline

• Multi-core Multimedia Platform• TI Roadmap Update• DaVinci - DM6446• DaVinci HD - DM6467• DaVinci Low Cost - DM355• OMAP Platform

Multi-coreSoC


ARM DSP

GPU VICP

VPSS Peripherals

ARM: Cortex A8 or A9 up to 1GHzDSP: C64x+ up to 1 GHzGPU: SGX530 up to 10M polygons/secVICP: MPEG1/2/4, WMV, H.264

Encode/Decode up to dual 1080PVPSS: Video Process Front End

CCDC/LVDS/ISP/3A/ResizerVideo Process Back EndRGB888/HDMI/SD&HD-DAC

Peripherals: NOR/NAND/DDR1&2&3USB-OTG/S-ATA/G-EMAC/PCIeSD/MMC/UART/I2C/McASP/SPDIF/ etc

Signal Processing Layer (SPL)Codec

Engine

Application Layer (APL)

Kernel Space

Linux API w/EPSI

User Space

I/OBuffers

Video Audio

I2C

USB 2.0

EMAC

SPI

UART MMC/SD

2 W’dogs2 Timers

DSP/Link

File/ATA

Codec Engine Resource ServerGUISong

Volume

Bass

Treble

mySong.mp3

VIS

A A

PI

InstanceBIOS TSK

ImagingCodec

xDM

API

InstanceBIOS TSK

AudioCodec

xDM

API

InstanceBIOS TSK

VideoCodec

xDM

API

InstanceBIOS TSK

SpeechCodec

xDM

APIst

ub

skel

eton

s

DSP Link DSP/BIOS

DM

AN

, AC

PY

DS

KT

ME

M, T

SK

AP

I

I/OBuffers

Shared

Memory

ARM DSP

Local Instance

AudioCodec

xDM

API

Link

skel

eton

AV SyncMux/Demux

Network RTP/RTSP

Master thread

DaVinci Software Architecture Example

Outline


Fle

xibi

lity

for

Inno

vatio

n

DaVinci ™ Products Combine Flexibility and Completeness

Completeness for Efficiency

ASIC

ASSP

FPGA

DSP

CPU

First Complete Offering to Enable Digital Video Innovation

Processors: Digital video system-on-chips

DVEVMDigital Video

Evaluation Module

Tools: Validated software and hardware development

– TMS320DM6443 – Video decode– TMS320DM6446 – Video encode/decode

DM6446

DM6443

– Platform-optimized, multimedia codecs– Platform support package– Linux support package

Software: Open, optimized and production tested

– Industry-recognized APIs– Multimedia frameworks

Enable OEM product differentiation with flexible, complete solution

ProductInnovation

New DaVinci ™ ProductsDrastically Simplify Digital Video Innovation

Leverage integrated, production-tested, software and hardware components

Development Efficiency

Save months of development time

Lower system cost and leverage IP across multiple productsPerformance

ValueLower system cost up to 50%

Video Processing Subsystem


TMS320DM644x Processors

Video-Optimized TMS320C64x+™DSP @ 600MHz

– Up to 720p MPEG-4 SP Encode

– H.264 MP@L3, 30fps SD Decoding

– VC1/WMV9 Full D1 SD Decoding

– MPEG-2 MP@ML SD Decoding

– Up to 1080i MPEG-2 Decode

– Simultaneous H.264 BP CIF Coding

Peripherals

Back End

ARM Subsystem

DSP Subsystem

Video-Imaging Coprocessor (VICP)

Front End

CCD Controller

Video Interface

Resizer

Histogram/3APreview

On-Screen

Display

(OSD)10b DAC10b DAC10b DAC10b DAC

EDMA

ATA/

Compact

Flash

Async EMIF/NAND/

SmartMedia

MMC/

SD

WatchdogTimer PWM

PWMPWM

General-PurposeTimer

DDR2

Controller

(16b/32b)

USB

2.0

PHY

VLYNQEMACWithMDIO

Connectivity

Program/Data Storage

SPI UARTUARTUART

I2CAudioSerialPort

Serial Interfaces

System

DM6443

ARM926EJ-S 300 MHz

CPU

C64x+TM DSP 600 MHz

Core

Switched Central Resource (SCR)Dedicated video processing sub-system

– Back end - Integrated OSD, four video DACs, 24-bit digital RGB output

Video

Enc

(VENC)

/6

– Front end – Resizer, Image processing engine, 16-bit digital input

DaVinci ™ SoftwareOptimized for Simplified and Efficient Innovation

Multimedia Codecs

� H.264

� MPEG4

� H.263

� WMV9

� VC1

� MPEG2

� JPEG

� AAC+

� AAC

� WMA9

� WMA8

� MP3

� G.711

� G.728

� G.723.1

� G.729ab

Operating Systems & Device Drivers

�MontaVista Linux LSP�Open Source Linux

Multimedia Application Programming Interfaces (APIs)� Industry-recognized APIs�DaVinci APIs

�Codec abstraction� Interprocessor communication�Audio/Video frameworks

Middleware

Code C

omposer S

tudio™ID

EF

ramew

orks and IPC

Mgm

t.

Board/P

latform S

upport Package

MontaV

ista Linux

Open Programming Environment

Open Development

Multim

edia Codec Library

Industry-Recognized Multimedia

APIs

Audio

Video

Voice

Image

DaVinci TM

Digital Video SoCs

DM6446

DM6443

DM644x™

Development tools and support– Linux development tools – Reference guidelines for high-speed video

board design, DDR2 layout, etc.

DaVinci ™ Development Tools

Digital Video Evaluation Module (DV-EVM)

Hardware– Based on the DM6446– Additional including

hardware components:• NTSC/PAL video camera• 5.6-inch LCD screen • Speakers and

microphone• IR remote• Hard disk drive

(2.5-inch 40G)

Software– Codec evaluation and demos including:

H.264, MPEG-4, MPEG-2, AAC+, G.711– Multimedia APIs & frameworks– MontaVista 2.6.10 Linux support package

Connectivity– Connectivity capabilities:

USB 2.0, 10/100 EMAC – Multiple on-board memory

types: CompactFlash,ATA, SD, DDR

– Video input via NTSC/PAL– Video output via

NTSC/PAL & YPbPr/RGB– CD-quality audio input and

output– Daughter-card connections

to peripheral interfaces

DVEVMTMDXEVM6446Order entry open

Pricing: $2495

PLUTO 3.0 Single Channel, Micro Video Processing Platform

10/100MBitEthernet

AnalogVideo Out

ITU656

AnalogVideo In

RAW

US

B 2

.0

ITU656 VideoDecoder

(D/A)

EthernetPHY

VideoEncoder

(A/D)

CMOSCamera

I/F

ATA

GLUE Logic(FPGA IDEcontroller)

GLUE Logic(CPLD for I/F)

GLUE Logic(FPGA forMMC/SD

Controller)

USBController

HardDisk

GLUE Logic(CPLD for

GPIO addition)

Peripheral Hardware Interfaces

MMC/SDSocket

Value: DaVinci ™ ProcessorsReduced Overall BOM with System Integration

“TI’s new TMS320DM6446 processor integrates the correct combination of video peripherals and accelerators, resulting in a BOM savings of 30%. With a reduced BOM and less time spent on system integration, YMagic anticipates hitting the market earlier and at unprecedented price points.”

Digital Media

Processor

-Menashe HaskinCEO, YMagic

USBController

VideoEncoder

(D/A)

GLUE Logic(FPGA IDEcontroller)

GLUE Logic(CPLD for I/F)

GLUE Logic(FPGA forMMC/SD

Controller)

GLUE Logic(CPLD for

GPIO addition)

10/100MBitEthernet

AnalogVideo Out

ITU656

AnalogVideo In

RAW

US

B 2

.0

ITU656

EthernetPHY

VideoEncoder

(A/D)

CMOSCamera

I/F

ATA

HardDisk

Peripheral Hardware Interfaces

MMC/SDSocket

PLUTO 3.0 New GenerationSingle Channel, Micro Networked Video Security Processing Platform

Value: DaVinci ™ ProcessorsReduced Overall BOM with System Integration

“TI’s new TMS320DM6446 processor integrates the correct combination of video peripherals and accelerators, resulting in a BOM savings of 30%. With a reduced BOM and less time spent on system integration, YMagic anticipates hitting the market earlier and at unprecedented price points.”

Digital Media

Processor

DM6446

-Menashe HaskinCEO, YMagic

IP Video Phone

Video Security IP Set-Top

Box

AutomotiveInfotainment

High Definition

PortableMedia Player

Digital Camera

Low-Cost

VC1

AAC

WMAWMV

MPEG2JPEG

H.263

AAC+

G.729ab

G.723.1

G.728

G.711

First Complete Technology for Multiple Video Products

FutureInnovation

Outline


DM6467 Delivers Multi-format, HD Transcoding in a Single Chip

DM6467

Switched Central Resource

Program/DataStorage

SerialInterfaces System Control

HighDefinition

Video/Imaging

Co-Processors(HD-VICP)

C64x+TM

DSPCore

600 MHz

ARM926EJ-S

CPU 300 MHz

Samples Available

Today

� Video codec control

� Proprietary codecs

� Custom algorithms

� Audio processing

� Video content analysis

� Chromasampling

� Down scaling

� Menu overlay

Video Data Conversion

Engine

� Serial and parallel transport stream processing capture/display

� BT656 dual-channel capture/display

� BT1120 capture/ display

� Raw RGB capture

VideoPort

Interfaces

� Industry standard PCI bus

� Gigabit Ethernet

StandardConnectivity

� Up to 1080p 30 fps / 1080i 60 fps / 720p 60 fps� H.264 HP, MPEG-4, VC1, MPEG-2, encode/decode� Optimized for multi-format transcoding

Acceleration

Applications� Transcoding (HD-HD, HD-SD), HD-video conferencing, HD-IP

set-top boxes, video surveillance, video phones digital media adaptors and medical imaging

Features� Core

– ARM926EJ-S™ (MPU) at 300 MHz– TMS320C64x+™ DSP Core at 600 MHz

� Memory– ARM: 16K I-Cache, 8K D-Cache, 32K TCM RAM,

8K Boot ROM– DSP: 32K L1 I-Cache, 32K L1 D-Cache,

128K L2 Cache, 64K Boot ROM� HD Coprocessors

– Real-Time HD-HD Transcoding Up to 1080p• Multi-format (mf) HD to mf HD or mf SD• Up to 2× real time for HD-to-SD transcode• Real-time HD-HD transcoding for PVR

– Video Encode and Decode• HD 720p H.264 BP encode • HD 1080i/p H.264 HP@L4, decoding;

HD 1080i/p VC1/WMV9, decoding; HD 1080i/p MPEG-2 MP@HL, decoding; HD 1080i/p MPEG-4 ASP, decoding; DivX

• Simultaneous SD H.264 BP 30 fps encode and decode

� Peripheral Highlights– Video ports

• Two 8-bit BT.656 or one 16-bit BT 1120 capture• Two 8-bit BT.656 or one 16-bit BT 1120 display

• Samples Dec. 07; TMS Qual 2Q08• RTM December 3rdBenefits• Scalable video engine building on high-performance C64x+

media DSP, low-cost local controllers, and rich sui te of multi-format video accelerators

DM6467: HD Video Transcode, Encode & Decode Processing

ARM Subsystem

DSP Subsystem

ARM926EJ-S

CPU 300 MHz

C64x+TM

DSPCore

600 MHz

Switched Central Resource (SCR)

MEMC

IPDELFECD

HD VICP 0TCM RAM

CALC

MC LFECD

HD VICP 1

TCM RAM

CALC

V ideo Data Conversion Engine

Digital Video Interfaces

Capture2x BT.6561x BT.1120

Peripherals

EDMA

ATAAsyncEMIF/NAND

DDR2Controller(16b/32b)


I2C

Serial Interfaces

Connectivity

VLYNQ

G-EMAC WithMDIO

WDTimer

System

PWM

××××3

Timer××××2

PCI

UHPI

USB

2.0

PHY

UART

××××3

McASP1 ch

McASP4 ch

Display2x BT.6561x BT.1120

Stream I/O

ChromaSampler

HW MenuOverlay

DownScaler

DM6467 Drives Video Market Evolution

DigitalMedia Adaptor Video Broadcast

Transcoding

HD Video-conferencing

Video Surveillance DVRs, DVS,

Decoders

CommercialConsumer

Multi-Conferencing

Units

Media Gateways

Advanced Set-Top Boxes

IP Netcam

IP Video Phone

Medical Imaging

MPEG-2

Typical Set-top Box Today

H.264 BP

VGA

H.264 MP QVGA

WMV9 MP D1

JPEG, MJPEGMPEG-4

WMV9H.264

MPEG-2

DM6467

All MediaFormats

DM6467 Enabled Set-top Box Digital Media Adaptor for Tomorrow

Smaller FPGA

Smaller power supply

DDR2, higher performance

and lower cost

Reduced flash count

Smaller board area

DM6467 Delivers Multi -format, Video Transcoding at 1/10 the Cost

FPGA

Power Supply

DDR2

FLASH

All pricing @ 50 ku

With DM6467

PCI Bridge

…

PCI Bridge

FPGA

Power Supply

SDRAM

FLASH

SDRAM

FLASH

SDRAM

FLASH

Today

C6415T1 GHz

C6415T1 GHz

C6415T1 GHz

…DM6467

MCU

Integrated host processor

DM6467 Flexibility and Efficiency Provide Multi-channel Video Encoding

DM6467

DDR2

� H.264 BP 30fps D1 per DM642 with no secondary CIF channels

4-Channel D1DVR w/ DM6467

DM642

SDRAM

4-Channel D1 DVR TodayVideoSecurityDVR

� 4 H.264 MP 30fps channels� 4 CIF H.264 BP 30fps secondary channels� Room for analytics� Proprietary codecs

HDD

HDD

IP Network

SDRAM

Host Processor

(GPP)

FPGA

IP Network

Lower power

DDR2, higher performance

and lower cost

Smaller board area

DM642 DM642 DM642

SDRAM SDRAM SDRAM

Outline


Performance

• HD MPEG-4 Encode or Decode @ 720p, 30 fps• JPEG encode or decode at 50 MPixels/second• MPEG-4 and JPEG codecs included,

no royalties or license fees to TI

Features

� Core– ARM926EJ-S™ Core, 216 MHz or 270 MHz– MPEG-4 & JPEG Coprocessor (MJCP)– Video Processing Subsystem

� Memory– ARM: 16KB I-Cache; 8KB D-Cache;

8KB ROM; 32KB program/data

� Peripheral Highlights– USB 2.0 HS OTG device and mini-host w/ PHY– External memory interface (EMIF)

• Mobile DDR/DDR2

Package: 13 x 13 mm BGA, 329 pin, 0.65mm pitch

Availability• Commercial Temp (0 to 85C) – Production Now

• Extended Temp (-45 to 85C) – Production NowTMS320DM355ZCEA216 (only 216 Mhz)

TMS320DM355 Processors Available Now

ARM Subsystem

ARM926EJ-S CPU

DMA Data & Configuration Bus

Peripherals

NAND/ECCEMIF

MMC/SDIOx2

mDDR/DDR2EMIF


System

PWM

x4

Timerx6

Connectivity

SPI

x3

I2C

Serial Interfaces

UART

x3

ASP

x2

CCD Controller

Video Interface

Preview

Histogram/3A

Resizer

Enhanced

On-Screen

Display

10b DACVideo

Enc


Back End

Front End

USB 2.0HS OTG

EDMAWDT

High Definition Video

MJCP Co-Processor

MPEG-4

JPEG

DM355 Target Markets

Many More ……..

PVR (Time-shift, audio/video playback, etc.)

TV (HMI, Games, Web-browsing)

Medical (ECG Monitor, etc.)

Surveillance (IPNC, Multi-Ch DVR, CCTV, etc.)

Home (Video Door bell, Video Baby Monitor, etc.)

Portable Infotaintment (PND, Portable DTV, PMP, etc)

Digital Photo Frame, Digital Still Camera, Smart Projector

High Tech Toy, Robotics

Audio (Sat Radio, Inet Radio, VoIP, DECT)

Low Cost HD IP Network Camera

10/100MAC &

PHY

Audio ADCPCM1870

Temp SensorTMP100

Video EngineDM355

DDRMemory

ClockingCDCE949

CMOS Sensor

SD Card

TransientSuppressorTPD2E001

I/F PWRTPS2065

Core PowerTPS62050

Memory PowerTPS62200

I/O & SystemPower

TPS62050

5V

1.3V 1.8V 3.3V

Multi-Channel DVR

10/100MAC &

PHY

Video DecoderTVP5154

Audio ADCPCM1870

Authentication IC

BQ26100

Temp SensorTMP100

HDD

RS232TRS3221

Video EngineDM355

Memory

Level ShifterPGA9306

FPGAVideo Amp

ClockingCDCE949

1.3VCore Pwr

Power Management UnitTPS65053

1.8VMem Pwr

3.3VI/O Pwr

Control

Video Amp

Video Amp

Video AmpOPA361

SD Card

TransientSuppressorTPD2E001

I/F PWRTPS2065

Video AmpOPA361

Video Out

RTCMSP430

Video Door Bell - Capture

10/100MAC &

PHY

Temp SensorTMP100

Video EngineDM355

DDRMemory

ClockingCDCE949

CMOS Sensor

Video AmpOPA361

PoEControllerTPS23750

WLAN

Core PowerTPS62050


I/O & SystemPower

TPS62050

10V

1.3V 1.8V 3.3V

Audio CODECTLV320AIC33

Video Door Bell - Display

10/100MAC &

PHY

Video EngineDM355

DDRMemory

ClockingCDCE949

PoEControllerTPS23750

WLAN

Video AmpOPA361

Audio CODECTLV320AIC33

HDD

10V

Core PowerTPS62050


I/O & SystemPower

TPS62040

1.3V 1.8V 3.3V

DDR2 or mDDR

SDRAM

NAND/NORFlash

USB

DM355

SOC

Audio

USB 2.0 HS OTG Keypad / Keyboard

MLC

High Tech Toy / Robotics

MMC/SD

LCD Display

Audio

Codec

CMOS/CMOSSensor

DSCC2000

MotorsServos

SteppersDC, BLDC

CMOS Camera(Eyes)

Life Form’s/Robot’sFace

DDR2 or mDDR

SDRAM

NAND/NORFlash

USB

DM355

SOC

Audio

USB 2.0 HS OTG Keypad / Keyboard

MLC

CMOS Camera(Option)

TV HMI / Games

MMC/SD

DisplayLCD on Product or to TV

Audio

Codec

Avatar Image On Display

Outline


Drive for smarter devices reaches across product categories to improve effectiveness

Simultaneous:Advanced user interfaces

Enhanced productivityFull web browsing

MultimediaGreen

Connectivity

Applications Software

Compatibility

2D/3D Graphics

Compatibility

DSP Processing

& Multimedia Software

Compatibility

SharedPeripheral

Set

OMAP™ 35x Scalable Generation of Single-Chip Devices for Complete Portfolio

OMAP3503

OMAP™ ARMCortex-A8600 MHz

Peripherals$19.95 10 Ku

Sampling Now

OMAP3503


Peripherals

OMAP3515

OMAP™ GraphicsOpen GL ES

2.0

ARMCortex-A8600 MHz

Peripherals

Sampling Now

$19.95 10 Ku

OMAP3503


Peripherals

OMAP3515

OMAP™ GraphicsOpen GL ES

2.0

ARMCortex-A8600 MHz

Peripherals

OMAP3525

OMAP™ C64x+ DSP & video accelerator

ARMCortex-A8600 MHz

Peripherals

Sampling Now

$19.95 10 Ku

OMAP3503


Peripherals

OMAP3515

OMAP™ GraphicsOpenGL®ES 2.0

ARMCortex-A8600 MHz

Peripherals

OMAP3525

OMAP™ C64x+ DSP & video accelerator

ARMCortex-A8600 MHz

Peripherals

OMAP3530


C64x+ DSP & video accelerator

GraphicsOpenGL®ES 2.0

Peripherals

Sampling Now

Available 2H08

Pin-for-pin compatible

Hardware

OMAP35x EVM

Comprehensive software and tools

• Evaluates all four OMAP35x devices

TMDXEVM3503*$1495

• OMAP35x Linux BSP (MontaVista™ support coming soon)

• Windows® CE 6.0 BSP coming soon

OS

More information available at www.ti.com/omap35x

Digital Video Software Development Kit(DVSDK)

* Software upgradeable to OMAP3530

• Included/download and run on current OMAP35x EVM

• Speed development with proven DaVinci™technology multimedia framework & codecs

• API & software architecture compatible with DM644x & DM3x processors

• Reduce development cost with integrated codecs (waived fees with accepted terms)

• Demonstrate multimedia content

Save months of design time by eliminating the need to directly manage & develop codecs

Open source ecosystem options simplify development for embedded Linux innovators

Beagle Board Cogent Gumstix

• Low cost open source development board

• Based on OMAP3530 processor

• USB-powered • Creative application implementation by utilizing standard PC peripherals

• Based on OMAP3530 processor

• 600 MHz System on Module

• Small (2” X 2.6”) and powerful with all GPIO & peripherals available

• Multiple serial interfaces, 10/100 Ethernet and onboard 3.3V

• 128 MB memory

www.cogcomp.com

Digi-Key

www.gumstix.com

Logic

www.logicpd.com

• Zoom™ Medical OMAP35x Development Kit and companion System on Modules

$149

Beagle Board – Delivering laptop-like performance with USB and standard PC expansion

Laptop-like performance

USB & standard

expansionTI OMAP3530

� 600 MHz superscaler ARM®

Cortex™-A8� More than 1200

Dhrystone MIPS� Up to 10 million

polygons per sec graphics

� HD video capable C64x+™ DSP core

� 128MB LPDDR RAM

� 256MB NAND flash

Memory

� I2C, I2S, SPI, MMC/SD

� DVI-D

� JTAG

� S-Video

� SD/MMC+

� Stereo out

� Stereo in

� USB 2.0 HS OTG

� Alternate power

� RS-232 serial

3”

Comprehensive open source ecosystem options simplify development for embedded Linux innovators

Scalable processors provide best general purpose, multimedia & graphics processing in any combination All available now

TI announces growing software and development tool ecosystem for TI’s three new OMAP35x processors

Available now

OMAP3503

OMAP™OMAP3515

OMAP™OMAP3525

OMAP™OMAP3530

OMAP™

DaVinci™ technology enabled DVSDK for fast video application development

Community endorsement

Save months of development time

Thank you!

The Development and Applications of


Jefferey Chiu Ph.D.DSP System Senior FAETexas Instruments

The Development and Applications of Multi-core Multimedia · PDF file ·...

Documents

Transcript of The Development and Applications of Multi-core Multimedia · PDF file ·...