March 2014 - Consultative Committee for Space Data Systems · Wireless Baseband Processor DSP...
33
RapidIO Overview March 2014 Barry Wood, Chair, RapidIO Technical Working Group [email protected] OR [email protected]
Transcript of March 2014 - Consultative Committee for Space Data Systems · Wireless Baseband Processor DSP...
RapidIO Overview March 2014
Barry Wood, Chair, RapidIO Technical Working Group
RapidIO Evolution
March 2014 2
Date Event Max Lane Speed Electricals
1997 Mercury Computer and Motorola (Freescale) begin to collaborate on a replacement for the Raceway interconnect
2000, March
RapidIO Trade Association formed
2000, June RapidIO 1.2 (Parallel PHY) 1 Gbps LVDS, 8/16 bit bus
2003, Oct RapidIO 1.3 (Serial PHY) Adopted as ISO standard (ISO/IEC DIS 18372)
3.125 Gbaud XAUI, 8b/10b
2008 RapidIO 2.0 (Faster) 6.25 Gbaud OIF CEI, 8b/10b
2013 RapidIO 3.0 (Faster) 10.3125 Gbaud 10G-kr, 10G-ab 64b/67b
2015(?) RapidIO 4.0 (Faster) 25 Gbaud 64b/67b?
• Over 6 million RapidIO switches shipped • Over 30 million 10-20 Gbps ports shipped > 10GbE
RapidIO Attributes
March 2014 3
Scalability Fault Tolerance Interoperable
Power Efficiency Low Latency
Any System Topology
Example System: ASML & Prodrive
March 2014 4
100+ KW Power Supply
Precision Optics
RapidIO Control
APPLICATION: Semiconductor photolithography machines 10,000 to 50,000 Hz control loop frequencies => nanometer precision https://www.youtube.com/watch?v=B9uDMNmajgw
Axxia Communications Processor DSP: several products
In TCI64xx family DSP, PowerQUICC & QorIQ multicore
XLS416 family Multicore Processor
FPGA: Arria and Stratix Family
FPGA: Virtex 4/5/6 families FPGA
Wireless Baseband Processor
DSP Oct22xx
PowerPC based processors 460GT
Switches, Bridges & IP CPS and Tsi Family
Network Processor Octeon 2 family Network Processor
WinPath3
Strong Ecosystem
March 2014 5 Used by all of the top 10 Wireless OEMs
RapidIO Specification Structure
March 2014 6
Physical Layer
(4) Parallel PHY (Deprecated)
(6) Serial
Transport Layer (3) Transport
Logical Layer (1) Logical I/O
(2) Message (9) Flow Control
(10) Data Streaming
(11) Multicast
(5) Global Shared Memory (8) Error
Management/Hot Swap
(12) VoQ Backpressure Annex II
Encapsulation
Annex I HW API System Bringup
(7) Interop
RapidIO Specification Mapped to Devices
March 2014 7
uProc Switch
DSP DSP
DSP DSP
Logical
Transport Switch
Physical A B Packet 0 CS CS CS
Packet 1 CS CS Packet 2 Packet 2 CS CS
Hardware Terminated Read/Write and Messaging
Packet Exchange Protocol
March 2014 8
Packet 2
Packet 3
Tx Rx
SOP Packet 1
Tx Rx
EOP
Tx Rx
EOP
SOP
EOP
Packet Transfer Receiver-based Flow Control
Error Recovery
RFR
Reliable, In Order, Packet Delivery 31-4095 Outstanding Packets
Control Symbols
March 2014 9
Control Symbol Identifier
Type 0 Parm 0 CRC CMD
Transmit Side “Type 1”
24, 48, or 64 bits
Parm 1
Receive Side “Type 0”
Type1
Packet Accepted Packet Retry Packet Not Accepted Status VC Status Link Response Timestamp
Start-of-packet Stomp End-of-packet Restart-from-retry Link Request Multicast Event No Operation/Ignore Timestamp Calibration
Packet Format Overview
March 2014 10
Logical Transport Physical
Logical Header FType
Data Payload Data Payload
TT Dest ID Src ID Physical Bits
Early CRC Final CRC
10 2 4 8,16,32 8,16,32
8 to 256 bytes
16 16
Control Symbol
32,64
Control Symbol 32,64
Packets cannot be altered by RapidIO switches – CRC is constant. RapidIO Standard defines 9 Virtual Channels (guaranteed bandwidth) Virtual Channel 0 supports 4 or 8 priorities.
Topology and Routing
March 2014 11
Mesh
Star
Switch
End Point
End Point
End Point
End Point
End Point
Switch
Switch
Switch
Switch
End Point
End Point
End Point
End Point
Switch
End Point
End Point
End Point
End Point
End Point
Switch
Switch
Tree • RapidIO is topology agnostic
• Packet DestinationID defines route • Routing is controlled through standard registers
•DestinationID routing can be multicast or unicast within each switch
Dual Star
End Point
End Point
End Point
End Point
End Point
Switch Switch
I/O System
March 2014 12
I/O System Memory Mapping
March 2014 13
CPU1
Packet
CPU2 RapidIO Memory
Map
Logical Transport Physical
Transaction FType Size
SrcTID Memory Address Data Payload Data Payload
TT Dest ID Src ID Physical Bits
Early CRC Final CRC
10 2 4 8,16 8,16 4 4
8 32, 48, 64
8 to 256 bytes
16 16
Control Symbol
32,64
Control Symbol 32,64
CPU2 Internal Memory
Map
Message Passing and Data Streaming
March 2014 14
Operation Transactions Used
Possible System Usage
Description Packet Format
Doorbell DOORBELL, RESPONSE
Event notification independent of source and target memory map implementation, efficient hardware implementation Logical Layer Retry
Part 2, Section 3.3.1
FType 10 Section 4.2.4
Data Message
MESSAGE, MESSAGE RESPONSE
Message transfer independent of source and target memory map implementation, limited number of connections/queues, efficient hardware implementation Logical Layer Retry
Part 2, Section 3.3.2
FType 11 Section 4.2.5
Data Streaming
Data Streaming Single Segment, Start/Continue /End Segments, Extended Header Flow Control
Message transfer independent of source and target memory map implementation. Connection oriented, 64K connections/queues, support for XON/XOFF, Rate and Credit based flow control. No Logical Layer Retry
Part 10, Section 3.2
FType 9 Part 10, Section 4.2
Type 10 Packet - Doorbell
March 2014 15
Response Packet
Request Packet
Logical Transport Physical
Rsvd FType TID TT Dest ID Src ID Physical Bits Final CRC 10 2 4 8 8 16
Info 16
Request Packet
TType FType TID TT Dest ID Src ID Physical Bits Final CRC 10 2 4 4 8 16
Response Packet
Status 4
“STATUS” is “Retry” when the receive queue is full.
FType 9 – Data Streaming
March 2014 16
Single Segment
Start Segment
End Segment
Start Segment
End Segment
Cont Segment
Cont Segment
Cont Segment
Cont Segment
Max 64KB transfer, Byte granularity
Data Streaming Flow Control
March 2014 17
Src 0
Src 1
Src 2
Server
Sources send Transmit Queue Status
to Server
Src 0
Src 1
Src 2
Server
Server Responds with Flow Control Messages
3 Flow Control Protocols Supported: - XON/XOFF - Rate Based - Credit Based
Sources communicate queue fill level 000/255 means “Empty” and 255/255 means “Full”
Flow Control can be applied to - A single Stream - A group of Streams - All Streams for a port
XON/XOFF Flow Control
March 2014 18
F E
Src 0
Src 1
Src 2
Server
Sources send Transmit Queue Status
to Server
Src 0
Src 1
Src 2
Server
Server Responds with Flow Control Messages
XON/XOFF - Service the fullest queue exclusively OR - Round robin among different queues OR - Use for “babbler” control
F E
F E XON
XOFF
XON
Rate Based Flow Control
March 2014 19
F E
Src 0
Src 1
Src 2
Server
Sources send Transmit Queue Status
to Server
Src 0
Src 1
Src 2
Server
Server Responds with Flow Control Messages
Rate Based - Increase rate by a fraction of current rate - Decrease rate by a fraction of current rate - XON - XOFF
F E
F E Inc 255/255
NoChange
Dec 64/255
Credit Based Flow Control
March 2014 20
F E
Src 0
Src 1
Src 2
Server
Sources send Transmit Queue Status
to Server
Src 0
Src 1
Src 2
Server
Server Responds with Flow Control Messages
Credit Based - Allocate credits to one of up to 16 buckets - XON - XOFF
F E
F E <All> 255 Cr
10 Credits left
10 Credits left
<Low Prio> XOFF
<All> 128 Cr
RapidIO System Discovery
March 2014 21
uProc Switch
DSP 0 DSP 1
DSP 3 DSP 2
1 2 3 4 5
System discovery uses a recursive algorithm to - Discover location and connectivity of all switches and endpoints - Allocate destination IDs to DSPs - Configure switch routing tables
Discovery can be executed from any endpoint
Standard system discovery supports two redundant hosts
RapidIO for Fault Tolerant Systems
March 2014 22
uProc Switch
Switch
Switch
DSP DSP
DSP DSP
Switch
DSP DSP
DSP DSP
System Host Data Processing
• Detection • Isolation • Notification • Diagnosis • Recovery
uProc
Redundant System Host
“Fail-stop” Error Management Support
RapidIO Form Factor & Connector Standards
March 2014 23
This Slide Intentionally Blank
New RapidIO Features for Space
March 2014 24
• It is possible to build Space systems with devices compliant to the RapidIO 1.x or 2.x specifications.
• RapidIO Trade Association’s NGSIS Task Group is defining a profile of required features for Space compliant devices
• Additional features for space operation are being defined.
March 2014 25
Bridge SpaceWire/Fibre RapidIO Packets
SpaceWire/Fibre Bridge Encapsulates RapidIO Packets
Endpoint Endpoint
SpaceWire/Fibre Endpoints process RapidIO packets • In software • In hardware
RapidIO Endpoints only know about RapidIO
RapidIO SpaceWire/Fibre Bridging
March 2014 26
RapidIO Packets Encapsulate SpaceWire/Fibre
Endpoint Endpoint
SpaceWire/SpaceFibre Endpoints need only know about SpaceWire/SpaceFibre
RapidIO Endpoints process SpaceWire/SpaceFibre packets • in software • in hardware
Bridge SpaceWire SpaceFibre
Message Packets (4K) Data Streaming (64K) Implementation Specific Read/Write (256 Bytes) Read/Write to Buffer
RapidIO SpaceWire/Fibre Bridging
RapidIO SpaceWire/Fibre Bridging
March 2014 27
Bridge SpaceWire/Fibre
RapidIO
• Bridge maps between SpaceWire and RapidIO routing values
• SpaceWire Routing => RapidIO DestID, priority, packet type, etc.
• RapidIO DestID => SpaceWire Routing Header, VC, priority, etc.
• Bridge could recognize whether SpaceFibre or RapidIO component had been plugged in based on presence/absence of K28.7
• Electrical specification harmonization requires discussion
Open Source Stack, Software Task Group
March 2014 28
Kernel
User Space
APPLICATION NOT IN SCOPE
OPEN SOURCE RTA STACK
OPEN SOURCE KERNEL
Hardware
Portable/ Legacy Application
Kernel Interface RapidIO Device Interfaces RIONET DMA Events Kernel Driver MPORT Drivers
Sockets R
DM
A Events
CLI Open source command line interpreter for fabric management RapidIO Standard APIs RapidIO standard interface definitions and behavior Fabric Functions Implementation of RapidIO Standard APIs Fabric Management Endpoint APIs Universal programming model for endpoint functions Switch APIs Universal programming model for switch functions. Used by Alcatel-Lucent systems today. Messaging/Sockets/RDMA /Events Implementation of RapidIO Standard API’s Data Path RapidIO Device Interfaces Standardized hardware functions.
Fabric Functions
Switch APIs
Endpoint APIs
Switch APIs
High Performance/ Embedded Application
RapidIO Standard APIs
C L I
Messaging
RapidIO Device Interfaces
Customization/Optimization
March 2014 29
APPLICATION NOT IN SCOPE
OPEN SOURCE RTA STACK
OPEN SOURCE KERNEL
Hardware
Portable/ Legacy Application
Kernel Interface
Fabric Functions
Switch APIs
Endpoint APIs
RIONET DMA Events Kernel Driver MPORT Drivers
RapidIO Standard APIs
CUSTOM COMPONENTS
Switch APIs
TOOLS, GUI, OPTIMIZATION
Extensions
Messaging
Sockets R
DM
A Events
MPORT Drivers
RapidIO Device Interfaces
TOOLS, GUI, OPTIMIZATION Value adders for debug/monitoring, system visualization, data interpretation, and topology specific functions MPORT Drivers Drivers optimized for and aware of executing hardware operations in user mode. Extensions Additional data path and/or fabric management services provided to applications
High Performance/ Embedded Application
C L I
Fabric Functions
Switch APIs
Endpoint APIs
Switch APIs
RapidIO Standard APIs Ex
tens
ions
Kernel
User Space
Next Steps
March 2014 30
• Map SOIS Service Specifications to RapidIO packets and/or software stack
• Join the RapidIO Trade Association to • influence Space Rev 2 feature definition • Specify RapidIO SpaceWire/Fibre bridging • Specify RapidIO CAN, 1533 bridging?
• Discuss further over beer?
Backup Charts March 2014
Barry Wood, Chair, RapidIO Technical Working Group
Maintenance Packet
March 2014 32
• Whenever a switch processing element that does not have as associated device ID receives a maintenance packet it examines the hop_count field. If the received hop_count is zero, the access is for that switch. If the hop_count is not zero, it is decremented and the packet is sent out of the switch according to the destinationID field.
• However, since maintenance response packets are always targeted at an end point, the hop_count field shall always be assigned a value of 0xFF by the source of the packets to prevent them from being inadvertently accepted by an intervening device.
Type 11 Packet - Message
March 2014 33
Logical Transport Physical
MsgLen FType Size TT Dest ID Src ID Physical Bits 10 2 4 8,16,32 8,16,32 4 4 4
Letter 2
Single-Segment Message Packet – 64 Mailboxes
TType FType TT Dest ID Src ID Physical Bits Final CRC 10 2 4 8,16,32 8,16,32 4 16
Single Segment Response Packet
Status 4
MBox XMBox 2
4 Letter
2 MBox XMBox
2
MsgLen FType Size TT Dest ID Src ID Physical Bits 10 2 4 8,16,32 8,16,32 4 4 4
Letter 2
Multi-Segment Message Packet – 4 Mailboxes
TType FType TT Dest ID Src ID Physical Bits Final CRC 10 2 4 8,16,32 8,16,32 4 16
Multi Segment Response Packet
Status 4
MBox MsgSeg 2
4 Letter
2 MBox MsgSeg
2
