DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200...

27
DDS Scalability: “One size fits all ??.. ” For OMG RTE-workshop, July 2007 Hans van ‘t Hag, OpenSplice™ DDS product Manager [email protected]

Transcript of DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200...

Page 1: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

DDS Scalability: “One size fits all ??.. ”For OMG RTE-workshop, July 2007

Hans van ‘t Hag, OpenSplice™ DDS product [email protected]

Page 2: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

1Agenda

INTO THE MOOD “… A paradigm shift …”

CRITICAL INFRASTRUCTURES “… The Data is the Network © …”

A MODEL “… For DDS applicability …”DDS APPLICABILITY “… One size fits all ?? ...”

DDS USECASE “… Work in Progress ...”

CONCLUSION “… Wait & See ...”

Page 3: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

INTO THE MOOD

Page 4: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

3

CH

J

G

K

F

B

E

I

A

L

N

U

T

V

R

M

P

Q

S

ZY X

W

D

A B C E F G H I J K L M

N O P Q R S T U V W X Y Z

D

DDS BACKBONE

Into the mood: ‘…Client/server vs. Pub-sub: A mind-shift…’

Page 5: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

CRITICAL INFRASTRUCTURES

Page 6: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

5© ARMS (Doug Schmidt): R&D Challenge

Page 7: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

6Net-centricity: The Data Is The Network ©

© DoD Chief Information Officer, Network and Informat ion Integration, Nov’05

Page 8: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

7DDS: a Fielded concept, suitable for mission-critic al systems

CHARACTERISTICS

Many different customers: fielded in over 15 Navies world-wide Many different ships/missions: tens of Ships classes (patrol boats to destroyers)Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/secReal-time and Fault-tolerant: Battle-damage resistant, deterministic, reliable

CHARACTERISTICS

Many different customers: fielded in over 15 Navies world-wide Many different ships/missions: tens of Ships classes (patrol boats to destroyers)Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/secReal-time and Fault-tolerant: Battle-damage resistant, deterministic, reliable

Page 9: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

8

SMART-L APAR

IFF LINK 11

KH/SCOUTNAV. RADAR

HELI APPR.RADAR

VESTA

VESTA

VEX/RCPTDLPP

APECS-3ESM / ECM

SRBOC

OFF LINER/S

SIRIUS TDS (2x) POS SYSTEMSHIP’S REF.

IRC

BTSOSD

BTSOSD

COMMS ROOMBRIDGESTAFF ROOM

LARGE SCREEN DISPLAY 2

PIOD /PTSU

PRINTERS

COTS & SIGMA TYPE PROCESSOR

COMBAT INFORMATION CENTER

LARGE SCREENDISPLAY 1

VIDEO INTERFACECABINET

VCR 1/2

DEPARTMENT OFF. / OFF. QUARTERSETC.

WORKSTATION 01 WORKSTATION M

SPARE

FL. DECK

TV CAMERA

TV CAMERA

15 15 15 15

INTEGRATEDMACHINERY

CONTROLSYSTEM COMMS NETWORK

CCC1 CCC2

ATAS

SPHERION+ XBT / XSV

+ SURF. TEMP. REC.HARPOON

TORPEDOWEAPONSYSTEM

MK32 MOD-9(MK46)

GUN 127 MM

30 MM GUNSOFF LINE

GIC-127

MIC-MK41

MIC-MK41

MK41 SM2LAUNCHER GOALKEEPER

CATV BUS

ATM NETWORK

Data-traffic: > 4.000 publications per second over the system-data busPrograms: 2.200 programs allocated over 150 processorsData flows: urgent & non-urgent data (latency), important & les s-important data (priority)

Data-traffic: > 4.000 publications per second over the system-data busPrograms: 2.200 programs allocated over 150 processorsData flows: urgent & non-urgent data (latency), important & les s-important data (priority)

An Example: Naval Frigate Combat System

Page 10: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

A MODEL FOR DDS-APPLICABILITY

Page 11: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

10SYSTEM SCOPES

Platform-centric systems

Mission Computer

Vehicle Management

Net-centric systems

Net-centric systems-of-systems

REQUIREMENTS

Page 12: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

11Information Grids in Net-Centric Operations (NCO)

� NCO: Autonomous processes, based upon ubiquitous information at required QoS� NCO: Autonomous processes, based upon ubiquitous information at required QoS

Planning Grid:CommonOperationalPicture

Command Support(CS)

Awareness Grid:CommonTactical Picture

Command & Control(CC)

Engagement Grid:Weapons Direction

Combat Execution (CE)

Page 13: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

12A MODEL FOR DDS APPLICABILITY

PlatformPlatform--centriccentric

““Execution gridExecution grid””

Net(work)-centric

“Awareness grid”

System-of-Systems

“Planning/support grid”

Scale

Realtime

Soft

Firm

Hard

Small Large Ultra-Large

Page 14: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

13MAIN CHARACTERISTICS

DETERMINISTICDETERMINISTIC--““ExecutionExecution--gridgrid””

EFFICIENTEFFICIENT--““AwarenessAwareness--gridgrid””

INTEROPERABLEINTEROPERABLE --““Planning/support gridPlanning/support grid””

Soft

Firm

Hard

Small Large Ultra-Large

Scale

Realtime

Page 15: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

14EXAMPLES

MilitaryMilitary: AAW fire control: AAW fire control

CivilCivil: Process control: Process control

MilitaryMilitary: Track: Track--ManagementManagement

CivilCivil: SCADA control systems: SCADA control systems

MilitaryMilitary: Mission planning: Mission planning

CivilCivil: Emergency Response: Emergency ResponseSoft

Firm

Hard

Small Large Ultra-Large

Scale

Realtime PlanningPlanning

AwarenessAwareness

ExecutionExecution

Page 16: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

15INFORMATION ARTEFACTS (Mil. Example)

Tracking informationTracking information

Timed ordersTimed orders

ObservationsObservations

Repository InformationRepository Information

Multimedia/HypertextMultimedia/Hypertext

DocumentsDocumentsSoft

Firm

Hard

Small Large Ultra-Large

Scale

Realtime

Engagement status

Mission Status

Engagement orders & Cues

Mission Objectives

PlanningPlanning

AwarenessAwareness

ExecutionExecution

Page 17: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

16TECHNOLOGIES

LANLAN

Soft

Firm

Hard

Small Large Ultra-Large

WANWAN

WEBWEB

Link/Radio

Internet, satellite

Scale

Realtime

�� STATIC LANSTATIC LAN

�� WS & Embedded HW/OSWS & Embedded HW/OS

�� TIMEDTIMED--IOIO

�� Hierarchical LANHierarchical LAN’’ss

�� Enterprise SMP machinesEnterprise SMP machines

�� AdAd--hoc networkshoc networks

�� COTS HW & SWCOTS HW & SW

�� Databases, GISDatabases, GIS

�� Classification/EncryptionClassification/Encryption

Page 18: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

17SYSTEM REQUIREMENTS

�� EndEnd--toto--end Latency & Jitterend Latency & Jitter

�� Safety & FaultSafety & Fault--tolerancetolerance

Soft

Firm

Hard

Small Large Ultra-Large

AwarenessAwareness

PlanningPlanning

Scale

Realtime

ExecutionExecution

�� Throughput performanceThroughput performance

�� Dependability & AvailabilityDependability & Availability

�� SecuritySecurity

�� COTS (DBMS/Web) integrationCOTS (DBMS/Web) integration

Tactical interoperability

COTS interoperability

Page 19: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

DDS-APPLICABILITY

Page 20: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

19DDS SUITABILITY: FUNCTIONALS (Specification)

�� RELIABILITYRELIABILITY�� TRANSPORT_PRIORITYTRANSPORT_PRIORITY�� OWNERSHIP/LIFELINESSOWNERSHIP/LIFELINESS

Soft

Firm

Hard

Small Large Ultra-Large

EFFICIENCY & AVAILABILITYEFFICIENCY & AVAILABILITY

INTEROPERABILITYINTEROPERABILITY

Scale

Realtime

DETERMINISMDETERMINISM

�� LATENCY_BUDGETLATENCY_BUDGET

�� FT DURABILITYFT DURABILITY

�� IDL/CORBAIDL/CORBA

�� XML ?? XML ??

�� SQL ??SQL ??

Partitions / Domains

IDL���� XML/JSON ??

Page 21: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

20DDS SUITABILITY: NON-FUNCTIONALS (Implementation)

�� Info priority based network schedulingInfo priority based network scheduling�� Safety & FaultSafety & Fault--tolerancetolerance�� Footprint & RTOS supportFootprint & RTOS support

Soft

Firm

Hard

Small Large Ultra-Large

Scale

Realtime

�� Latency budget based packingLatency budget based packing

�� Distributed/faultDistributed/fault--tolerant durabilitytolerant durability

�� WebWeb--services gateway ??services gateway ??

�� DBMS gateway ??DBMS gateway ??

Common Information Models

COTS interoperability

INTEROPERABILITYINTEROPERABILITY

DETERMINISMDETERMINISM

EFFICIENCY & AVAILABILITYEFFICIENCY & AVAILABILITY

Page 22: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

21DDS SUITABILITY: Standardization Challenges

�� PriorityPriority--bandingbanding

�� Traffic shapingTraffic shaping

Soft

Firm

Hard

Small Large Ultra-Large

Scale

Realtime

�� Shared subscriber cachesShared subscriber caches

�� UserUser--defined storagedefined storage--keyskeys

�� XML based API/topicsXML based API/topics

�� DBMS: topic/table mappingDBMS: topic/table mapping

Corba/RTJava integration

DBMS/Web-serviceintegration

EFFICIENCY/SCALABILITYEFFICIENCY/SCALABILITY

INTEROPERABILITYINTEROPERABILITY

RTRT--DETERMINISMDETERMINISM

Page 23: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

Use Case: OpenSplice

…. one size fits all … ???

Page 24: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

23USE-CASE: OpenSplice, Addressing the challenges

�� RT PriorityRT Priority--bandingbanding

�� Traffic shaping (size/burst)Traffic shaping (size/burst)

�� InfoInfo--priority prepriority pre--emptionemption

Soft

Firm

Hard

Small Large Ultra-Large

Scale

Realtime

�� DLRL data managementDLRL data management

�� Shared/readShared/read--only cachesonly caches

�� UserUser--defined/Dynamic keysdefined/Dynamic keys

�� XML: topicXML: topic--definition & APIdefinition & API

�� DBMS: topic/table mappingDBMS: topic/table mapping

- (Any) Corba integration: compilable C++ API)- (RT) Java: Pluggable JAVA API on C-core

- Soap-service: Tuner™- DBMS-service: 2-way

ININ--MEMORY (O)DBMS BASED COREMEMORY (O)DBMS BASED CORE

PLUGGABLE (GATEWAY) SERVICESPLUGGABLE (GATEWAY) SERVICES

RT NETWORK_SCHEDULERRT NETWORK_SCHEDULER

Page 25: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

24

Enterprise(Windows, Solaris, Linux, AIX)

Embedded(vxWorks, QNX, LynxOS)

Webservicegateway

NetworkScheduler

Fault-tolerantDurability

Minimum Ownership Content-Subscription

Persistence

DCPS

DLRL

DDS API’sC/C++/Java

DynamicXML-based API

Monitor &Control (Tuner)

Logging &Replay

Configuration

SECURITY

InformationModeling

ApplicationModeling

DeploymentModeling

OMG-DDSProfiles

Pluggable-Services

SupportedPlatforms

Middleware-Integration

High-levelServices

UML DDS-DSL

Remote-connect

EclipseFoundation &

Framework

Roundtrip-Engineering

Modeling-Plug-ins

Views &Editors

Development

Deployment

DBMSGatewaysoap

USE-CASE: Creating & Deploying DDS-based systems

COHABITATION

Corba/Java/Dbms

Page 26: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

25

Enterprise(Windows, Solaris, Linux, AIX)

Embedded(vxWorks, QNX, LynxOS)

Webservicegateway

NetworkScheduler

Fault-tolerantDurability

Minimum OwnershipContent-

SubscriptionPersistence

DCPS

DLRL

DDS API’sC/C++/Java

DynamicXML-based API

Monitor &Control (Tuner)

Logging &Replay

Configuration

SECURITY

InformationModeling

ApplicationModeling

DeploymentModeling

OMG-DDSProfiles

Pluggable-Services

SupportedPlatforms

Middleware-Integration

High-levelServices

UML DDS-DSL

OpenSplice Tuner &Remote-connect

EclipseFoundation &

Framework

Roundtrip-Engineering

Modeling-Plug-ins

Views &Editors

DBMSGatewaySoap

( Potential )DDS-related

Standardization

UML-profile for DDS(joint submission with SPARX systems)

Dynamic Topics(requirements: DDS-PSIG)

Dynamic DBMS gateway(available, but

IDL/table mapping not standardized)

Webservice gateway(available and used by OpenSplice Tuner™,

but no standard yet)

Security for DDS(node-to-node and end-to-end,

encryption available but no standard yet)

Logging & Replay(following C4I standardization)

Pluggable Discovery(for ultra-large/WAN systems,Several European/US studies)

CONCLUSION: ‘One concept’ fits all, we’re not there yet..

COHABITATION

Corba/Java/Dbms

Page 27: DDS Scalability: “One size fits all · Large-scale & mission-critical: >150 CPU’s, >2200 applications, >4.000 tracks/sec Real-time and Fault-tolerant: Battle-damage resistant,

….QUESTIONS….

?????

� THANK YOU !!

[email protected]