Introduction to Freescale's New RapidIO Interface and ... · Type 11 messaging • 4 Mailboxes 4...

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc.

FTF-NET-F0707

Introduction to Freescale's New RapidIO® Interface and Technology and Applicable Use Cases

June, 2010

Steve ColeSenior System Architect, Network Systems Division

TMFreescale, the Freescale logo, AltiVec, C-5, CodeTEST, CodeWarrior, ColdFire, C-Ware, mobileGT, PowerQUICC, StarCore, and Symphony are trademarks of Freescale Semiconductor, Inc., Reg. U.S. Pat. & Tm. Off. BeeKit, BeeStack, CoreNet, the Energy Efficient Solutions logo, Flexis, MXC, Platform in a Package, Processor Expert, QorIQ, QUICC Engine, SMARTMOS, TurboLink and VortiQa are trademarks of Freescale Semiconductor, Inc. All other product or service names are the property of their respective owners. © 2010 Freescale Semiconductor, Inc. 22

BTS Trends

►Wireless infrastructure a significant driver for innovation in• Multicore SoCs• System level interconnects

IP

Internet Protocol (IP) Use Cost

$$$ NRE

$$$ CAPEX

$$$ OPEX

Functionality

Latencymsec

Bandwidth

Mb/sQoS

Media

Applications

Voice

VideoBulk Transfer

SoCIntegration

Presenter

Presentation Notes

RapidIO has been largely driven in the market by wireless infrastructure needs for a peer-to-peer system level interconnect


Trends in the Networking and A&D Markets

►Internet Protocol (IP) everywhere ►VoIP adoption ►Triple-play convergence (voice, video & data) ►Ethernet expansion ►Wireless everything ►Power management and low-power processing ►Security elevation and secure processing nodes►Standard interconnects►High-performance in ever smaller form factors

► Increased processing performance ► Power management and low-power

processing ► Delivery of voice, data and video► Security and secure systems► Deploying high technology in all

phases of defense ► Standard interconnects► Emergence of systems integration of

off-the-shelf components in place of milspec manufacturing

► Industry consolidation

Networking Trends A&D Trends


Silicon Trends

► Multi-function device• Limited CPU integration with

integrated I/O

CPU

Shared Bus

DR

AM

I/O

Shared Bus

I/OA

ccel

CPU

DR

AM

I/O I/O

Accel

CPU

Accel

CPU CPU CPU

DR

AM

► Multicore SoC• Full CPU, accelerator,

I/O integration• Combined control and

data plane functions

CPU

Shared Bus

Bridge

DR

AM

I/O

Shared Bus

I/O

Accel

► Single-function devices• Connected via shared buses

DR

AM


Control Cards

Fabric / Backplane(s)

NetworkInterface

Card

RFModule

ChannelCard

To RNC orMME/SGWor ASN-GW

Channel Card (CC)

n

DSP

ASIC /FPGA

MPU SRIOSwitch

ASIC /FPGA

Time-SlotInterchange

(TSI)

MPU /Comm.

Processor

PHYLayer 2+

GigESwitch

RF interface (OBSAI, CPRI, etc)

and other inter-chipconnectivity

GigESwitch

Backplane

BackplaneGigE backhaul

ATM (IMA)or MLPPP

n x T1/E1or T3/E3backhaul

OC-3/STM-1backhaul ATM (IMA)

or POS

NetworkInterface Card (NIC)

Transport(backhaul)

RemoteRadio Head

fiber

Layer 2+

Broadband Wireless Base Station

DSP

ASIC /FPGA

DSP

DSP

DSP


Streams and More Streams

► Multicore SoCs• Convergence of control and dataplane

Driven by SoC integration trends• Many data streams between blocks

and silicon devices

Shar

ed B

us o

r Cro

ssba

r

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AMD

RAM

Accel

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AM

FPGA

CPUBridgeSh

ared

Bus

Shar

ed B

us

I/O

DRAM

I/O

I/O

ASICAccel

Shar

ed B

us

Shar

ed B

us

I/O

DRAM

I/O

I/OSh

ared

Bus

or C

ross

bar

► Discrete and multi-function silicon• Single data stream between devices


It’s the Logical Layer…

►Applications driving innovation at logical layer of interconnects• Off-load moving upward in the stack

Logical layer implemented in hardware– Ethernet example of when this isn’t the case

Power, performance and QoS are drivers

StackSW

HW

ClientSW

MAC

DMA

TCP/IP

Application

TCP/IP

MAC

DMA

UDP/IP

SAR/Err

Application

UDP

MAC

SAR/Err

Application

Layer 2

Link/PHY

DMA

Client

Trans

LogicalHW

AppSW

PCI Express® interface: Rd, Wr

RapidIO® interconnect: SWRITE, MSG, Streaming

Ethernet

DriverStackSW

Application API

DMA


But It’s Really About Channelization…

►Interconnects with native support for streams invoke a virtuous cycle• Opportunity to lower mux/demuxing in the system

Fewer proprietary headers required for stream differentiation• Reduces hardware and software overhead• Reduces power

TDM

VoiceDataControl 1Control 2

Ex: GSM/Edge … …

ExternalInterconnect

To Netw

ork


RapidIO and Channelization

►Flows• Four prioritized flows defined in specification

Three must be supported by all implementations

►Type 11 messaging• 4 Mailboxes

4 KByte PDU• 64 Mailboxes

256 Byte PDU

►Type 9 data streaming• 256 classes of service• 64K stream IDs

FlowID(Priority) TgtID SrcID Mailbox

Type 11

FlowID(Priority) Tgt ID SrcID CoS

Type 9StreamID


Streaming Support Grows Over Time

►Expect unraveling of stream muxing to occur gradually• Software won’t change overnight even if hardware is supported

Multi-stream hardware support must be there first– Requires standards to exist ahead of hardware

ExternalInterconnect

TDM

VoiceDataControl 1Control 2

Ex: GSM/Edge …

To Netw

ork


Today’s System Requirements

Convergence of control and dataplane devices driven by multicore economics

Stream Differentiation

Quality of Service

Robust Flow Control

Multiple Baud Rates and Widths

Interconnect Logical Streams Classes

Shar

ed B

us o

r Cro

ssba

r

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AMD

RAM

Accel

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AM

FPGA

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AMD

RAM

Accel

DR

AM

CPU

CPU

CPU

CPU Accel

I/O

I/O

AccelCPU

DR

AM

FPGA

Presenter

Presentation Notes

Stream differentiation implies the ability to identify specific traffic streams from the packet header without parsing. Quality-of-Service implies the ability to give individual traffic streams better service either in terms of latency or available bandwidth. Key to QoS is management of congestion within a network or at endpoints. Congestion can be of a short, medium or long-term nature. One flow-control methodology does not fit all. A number of schemes from link-level through to traffic policing and shaping are required. Channelization implies not only the ability to identify specific traffic “channels” but to also apply flow control to each channel or stream independantly. Modern interconnects need a variety of baud rate and width choices to allow efficient optimization in terms of bandwidth, power and cost.


RapidIO Value Proposition

►Built-in reliable deliveryLossy delivery defined in Rev 2.0

►Five speeds with variety of widths1.25G, 2.5G, 3.125G, 5.0G, 6.25G; 1x-16x

►Multiple transaction types with channelization►Multiple channelized flow control mechanisms

Link, source-based, end-to-end►Multiple use paradigms

Memory-mapped R/WMessages (doorbell, data)Datagrams

►Efficient and low CPU-overhead

http://www.rapidio.org/


RapidIO Is Unique Evolving Logical Layer Standard

►RapidIO interconnect is leading evolution in logical layer protocols and QoS• Eight prioritized flows• Robust flow control

Link levelCongestion to sourceEnd-to-end (traffic management)

• Type 11 messaging transaction type4 KB max user PDU16 streams max per flow

• Type 9 data streaming transaction type64 KB max user PDU256 classes-of-service (CoS) per flow65,536 streams/CoS per flowStandardized traffic management

– Stream setup/teardown– End-to-end flow control– Encapsulation

Endpoint

Line Card

Switch Switch Switch Endpoint

Endpoint

Endpoint

Endpoint

Endpoint

Line CardBack Plane

End-to-End Traffic Mgmt

Link-level Flow

Control

XOn-XOffCongestion

Control

Hdr Flow CoS StreamID PDU

Hdr Flow Ltr Mailbox PDU

Type 11 Packet

Type 9 Packet


DR

AM

Accel

Accel

DSP Core

DSP Core

DSP Core

DSP Core

DR

AM

sRIO

Accel

Accel

RMan for QorIQ: Taking RapidIO to a New Level►Multicore SoCs driving application and interconnect convergence

• Combine multiple streams over the same interconnect

►Industry-first performance & QoS features• Support for Rev 2.1 of RapidIO standard

Dual port @ x2/x1 or single @ x4/x2/x1Up to 5 Gbaud per lane, total of 16 Gbps

• Type 9 - data streaming transaction supportThousands of traffic streamsRealistic line-rate performance

• Robust quality-of-service featuresStream and priority-aware micro-architectureEnd-to-end flow control

Increased performanceRobust quality of serviceMany streamsLower system cost Simpler layout

14

CPUCore

CPUCore

CPUCore

CPUCore

Host CPUor ASIC

Data Plane Single Multi-stream Interconnect

sRIO

System Cost

Performance/Flexibility

sRIO sRIO

Switch

DRAM

Presenter

Presentation Notes

As multiple Cores are integrated, a diversity of application functions are also integrated on a single die. Trend motivates the need for external interfaces to carry multiple streams of traffic control, data, management, debug By combining control, data and management traffic over the same interconnect, significant cost savings are possible. RMan fully supports thousands of streams. It allows each stream to carry the natural streams present in the system (e.g. individual handset data/voice streams). Today, existing streams are manually muxed together for transport by a processor DSP Core causing loss of performance and increased latency. RMan natively supports individual streams end-to-end removing this overhead. RMan introduces the industry’s first implementation of Type 9 data streaming. It defines millions of streams with protocol support that allows line-rate performance. The internal architecture of RMan is fully stream and priority –aware to guarantee that high priority streams are never blocked by lower-priority ones. RMan supports also introduces Type 9 end-to-end flow control bringing new levels of congestion and latency jitter control to a RapidIO network.

http://www.rapidio.org/


QorIQ P3 Series P3041 Block Diagram

Quad e500mc Power Architecture®

►4 cores (up to 1.5GHz) with 128KB backside L2 cache

►1 MB shared L3 cache w/ECCMemory controller►32/64-bit DDR3/3L w/ECC up to

1.3 GHzEthernet►5 x 10/100/1000 Ethernet

controllers ►1 x 10 GE controllers

High speed interconnect►4 PCIe 2.0 controllers►2 SRIO 1.3 + 2.0 controllers►2 SATA 2.0►2 USB 2.0 w/PHY

CoreNet switch fabricTrusted architectureDatapath acceleration architecture►Security Engine (SEC)►Pattern Matching Engine (PME)►Enhanced RapidIO Messaging

(Rman)

PatternMatchEngine

2.0

1024 KBFrontsideL3 Cache

64-bitDDR3/3L

Memory Controller

Coherency FabricPAMU Peripheral

Access Mgmt Unit

eOpenPIC

Power Mgmt

2x USB 2.0 PHY

SD/MMC

Clocks/Reset

2x DUART

4x I2C

SPI

GPIO

PreBoot Loader

Security Monitor

Internal BootROM

CCSR

Power Architecture®

e500-mc Core

D-Cache I-Cache

128 KBBacksideL2 Cache 32 KB 32 KB

SEC4.0

Queue Mgr.

BufferMgr.

eLBC32b

Rapid IO

RMan

PAMU PAMU PAMU PAMU

Frame Manager

Parse, Classify,Distribute

Buffer

DMA x2

PCIe

18-Lane 5 GHz SerDes

PCIe

PCIe

PCIe

WatchpointCross

TriggerPerf

MonitorCoreNet

Trace

Aurora

1GE10GE

Real Time Debug

1GE

1GE

1GE

SATA

2.0

SATA

2.0

SRIO

SRIO

1GE

► Pin-compatible to P4080, P4040, P5020 & P5010► 45 nm SOI process

CoreNet™


One Channel Card Architecture Evolution

ASIC /FPGAL1 Sample Rate

Some L2 +L1 Symbol Rate

Control +L3/L2

2G

ASIC /FPGA

DSP

MPU

Switch

ASIC /FPGAL1 Sample Rate

Some L2 +L1 Symbol Rate

Control +L3/L2

ASIC /FPGA

Switch

MPU

3G-3.5G

Some L2 +L1 Symbol Rate +L2 Sample Rate

Control +L3/L2

Switch

MPU

DSP DSP

LTE, LTE-A

RapidIOor GigE

RapidIOor GigE

RapidIOor GigE

RapidIOor GigE

RapidIOor GigE

Presenter

Presentation Notes

One view of channel card evolution. Each vendor has their own take and twist.


QorIQ P5 Series P5020 Block Diagram

Frame Manager

Parse, Classify,Distribute

Buffer

DMA x2

PCIe

18-Lane 5 GHz SerDes

PCIe

PCIe

PCIe

CoreNet™

WatchpointCross

TriggerPerf

MonitorCoreNet

Trace

Aurora

PatternMatchEngine

2

1GE10GE

1024 KBFrontsideL3 Cache

64-bitDDR-3

Memory Controller

Coherency FabricPAMU

eOpenPIC

Power Mgmt

2x USB 2.0 PHY

SD/MMC

Clocks/Reset

DUART

4x I2C

SPI

GPIO

PreBoot Loader

Security Monitor

Internal BootROM

CCSR

Power Architecturee500mc-64 2GHz Core

D-Cache I-Cache

512 KBBacksideL2 Cache 32 KB 32 KB

Real Time Debug

2x e500mc-64 core, built on Power Architecture technology►2x 64-bit cores (up to 2+ GHz)

with 512 KB backside L2 cache►Dual 1 MB shared L3 cache

w/ECCMemory Controller

►Dual 32/64-bit DDR3/3L w/ECC up to 1.3 GHz

Ethernet►5 x 10/100/1000 Ethernet

controllers ►1 x 10 GE controller (XAUI)

High speed interconnect►4 PCI Express 2.0 controllers►2 Serial RapidIO 1.3 + 2.0

controllers►2 SATA 3 Gb/s►2 USB 2.0 with PHY

CoreNet switch fabricTrusted architectureDatapath acceleration architecture

►Security Engine (SEC)►Pattern Matching Engine (PME)►RAID 5/6 engine►Enhanced RapidIO Messaging

(Rman)

SEC4

Queue Mgr.

BufferMgr.

eLBC

1GE

1GE

1GE

PAMU PAMU PAMU

SATA

2.0

SATA

2.0

RAID 5/6

Engine

SRIO

SRIO

SRIOMgr.

1GE

► Pin-compatible to P4080, P4040 and P3041► 45 nm SOI process

Peripheral Access Mgmt Unit

PAMU


System DiagramRedundant SAN RAID Processor

StorageHBA

StorageHBA

SAS or FC Drives

Mirrored Ports

4 lanes SRIO

ProcessorWithRAID

Processor With RAID

HostHBA

4 or 8 lanesPCI-E

Fiber ChannelEthernet

HostHBA

Fiber ChannelEthernet

High-end redundant SAN processor1.) RAID processing2.) Movement of data3.) Extensibility4.) Virtualization of drives

4 lanesPCI-E

4 or 8 lanesPCI-E

4 lanesPCI-E


Next Generation RapidIO Messaging Unit

►P4080 and all prior devices use existing SRIO messaging unit• QorIQ P4080 includes some messaging performance enhancements

►All future devices with SRIO will use new SRIO messaging unit • Existing R/W capability retained in existing link controller• Implemented as “eMSG” for DSP, “RMan” for QorIQ• Improves Type 11 messaging performance

Lower CPU overheadFull descriptor-based queuing mechanism

– Data-Path Architecture (DPAA) enabled for QorIQ– Updates descriptor with classification info/byte count

More queues

• Adds support for Type 9 data streamingLine rate with low CPU overheadEnhanced QoS

• Adds selected Rev 2.0 featuresAdd 5.0 Gbaud operation2x lane supportExtended Type 9 traffic management

Control RMUMSC8156, QorIQ P4080

Control eMSG Control RMan

LinkController

Messaging Unit

LinkController

DPAAMessaging Unit

LinkController

EnhancedMessaging Unit

Next Gen StarCore™ P3041, P5020


P5020/304x RMan Datapath

CoreNet

Port 0 Port 1

OCN

DDR

SRIO

RMan QManBMan

SRIO

ATMU ATMU

DMADMA

Link Controllers

Logical Layer


RMan: Next Gen RapidIO Interconnect for Freescale Multicore Processors

►Virtualizes RapidIO interconnect data transfers for QorIQ communication platforms• Very low CPU overhead• Enables zero overhead direct core-to-core communication

►Supports all messaging-style transaction types• Type 11 messaging• Type 10 doorbells• Type 9 data streaming

►Many queues allow multiple inbound/outbound queues per core• Hardware queue management via QorIQ Datapath Architecture (DPAA)

►Enhanced QoS• End-to-end per stream flow control

Channelized Transport

Core to Remote Core Transport

PayloadType9

Core Core Core Core

10GEMAC

HW Accel

QorIQ

SRIO

… Core Core Core Core

10GEMAC

HW Accel

DSP

SRIO

…

PayloadMSG


Core Core Core Core

10G SRIO

Datapath

QorIQ

Core Core Core Core

10G SRIO

Datapath

QorIQ

Core Core Core

10G SRIO

Datapath

DSP or QorIQ

Core Core Core Core

10G SRIO

Datapath

QorIQ

RMan Enables New Zero-CPU Overhead Use Models

Ethernet Frame

MSG Ethernet Frame

Ethernet Frame Ethernet Frame

Channelized inbound Enet-to-remote CPU transport

Transparent bridging w/zero CPU intervention for multiple classified Enet streams

Scalable Multicore System

Ethernet Bridging

Type9 Ethernet Frame

Type9 Ethernet Frame

MSG Ethernet Frame

Core


Conclusion

►Wireless infrastructure key example of multicore, multi-stream embedded applications

►Wireless is driving innovation in logical layer of high-speed interconnects

►Many other markets need high-speed QoS-enabled interconnects, including storage and A&D

►Hardware offload at higher layers of a channelized interconnect protocol helps meet future QoS, performance and power goals

Introduction to Freescale's New RapidIO Interface and ... · Type 11 messaging • 4 Mailboxes 4...

Documents

Transcript of Introduction to Freescale's New RapidIO Interface and ... · Type 11 messaging • 4 Mailboxes 4...