Douglas - cs.wustl.eduschmidt/PDF/ATM.pdfDouglas C. Schmidt High-p erfo rmance, Real-time ORBs...

High-performance,Real-time

CORBAORBsforATM

Networks

DouglasC.Schmidt

[email protected]

WashingtonUniversity,St.Louis

www.cs.wustl.edu/�

schmidt/TAO.html

Sponsors

NSF,DARPA,Bellcore,Boeing,CDI,

Kodak,Lucent,Motorola,OTI,SAIC,

SiemensSCR,SiemensMED,SiemensZT,Sprint

Douglas C. Schmidt High-performance, Real-time ORBs

Problem: Lack of Real-time Middleware

WIDE AREA

NETWORK

SATELLITESSATELLITESTRACKINGTRACKINGSTATIONSTATION

PEERSPEERS

STATUS INFO

COMMANDS BULK DATA

TRANSFER

LOCAL AREA NETWORK

GROUNDSTATION

PEERS

GATEWAY

� Many applications require

QoS guarantees

{ e.g., telecom, avionics,

WWW

� Building these applications

manually is hard

� Existing middleware doesn't

support QoS e�ectively

{ e.g., CORBA, DCOM, DCE

� Solutions must be integrated

Washington University, St. Louis 1


Candidate Solution: CORBA

DIIDII ORBORBINTERFACEINTERFACE

ORBORBCORECORE

operation()operation()

OBJECTOBJECT

ADAPTERADAPTER

IDLIDLSKELETONSKELETON

DSIDSI

in argsin args

out args + return valueout args + return value

CLIENTCLIENT

GIOPGIOP//IIOPIIOP

SERVANTSERVANT

STANDARD INTERFACESTANDARD INTERFACE STANDARD LANGUAGESTANDARD LANGUAGE

MAPPINGMAPPING

ORB-ORB-SPECIFIC INTERFACESPECIFIC INTERFACE STANDARD PROTOCOLSTANDARD PROTOCOL

IDLIDLSTUBSSTUBS

www.cs.wustl.edu/�schmidt/corba.html

� Goals of CORBA

{ Simplify distribution

by automating

� Object location

and activation

� Parameter

marshaling

� Demultiplexing

� Error handling

{ Provide foundation

for higher-level

services


DouglasC.Schmidt

High-performance,Real-timeORBs

MotivationforCORBA

�

Simpli�esapplicationinterworking

{CORBAprovideshigherlevelintegrationthantraditionaluntyped

TCPbytestreams

�

Providesafoundationforhigher-leveldistributedobjectcollaboration

{e.g.,WindowsOLEandtheOMGCommonObjectService

Speci�cation(COSS)

�

Bene�tsfordistributedprogrammingsimilartoOOlanguagesfor

non-distributedprogramming

{e.g.,encapsulation,interfaceinheritance,andobject-basedexception

handling


3


The ACE ORB (TAO)

NETWORKNETWORK

ORBORB QQOOSSINTERFACEINTERFACE

REALREAL--TIMETIMEORBORB CORECORE

operation()operation()

RIDLRIDLSTUBSSTUBS

REALREAL--TIMETIME

OBJECTOBJECT

ADAPTERADAPTER

RIDLRIDLSKELETONSKELETON

in argsin args

out args + return valueout args + return value

CLIENTCLIENT

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK ADAPTERSNETWORK ADAPTERS

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

RIOPRIOP

SERVANTSERVANT

OS KERNELOS KERNEL

HIGHHIGH--SPEEDSPEED

NETWORK ADAPTERSNETWORK ADAPTERS

REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM

www.cs.wustl.edu/�schmidt/TAO.html

� TAO Overview

{ A high-performance,

real-time ORB

� Telecom and

avionics focus

{ Leverages the ACE

framework

� Runs on RTOSs,

POSIX, and Win32

� Related work

{ QuO at BBN

{ ARMADA at U.

Mich.



The ADAPTIVE Communication Environment (ACE)

PROCESSESPROCESSES//THREADSTHREADS

DYNAMICDYNAMIC

LINKINGLINKING

MEMORYMEMORY

MAPPINGMAPPING

SELECTSELECT//IO COMPIO COMP

SYSTEMSYSTEM

VV IPCIPCSTREAMSTREAM

PIPESPIPES

NAMEDNAMED

PIPESPIPES

CAPIS

SOCKETSSOCKETS//TLITLI

COMMUNICATIONCOMMUNICATION

SUBSYSTEMSUBSYSTEM

VIRTUAL MEMORYVIRTUAL MEMORY

SUBSYSTEMSUBSYSTEM

GENERAL POSIX AND WIN32 SERVICES

PROCESSPROCESS//THREADTHREAD

SUBSYSTEMSUBSYSTEM

FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR

SELF-CONTAINED

DISTRIBUTED

SERVICE

COMPONENTS

NAMENAME

SERVERSERVER

TOKENTOKEN

SERVERSERVER

LOGGINGLOGGING

SERVERSERVER

GATEWAYGATEWAY

SERVERSERVER

SOCKSOCK__SAPSAP//TLITLI__SAPSAP

FIFOFIFO

SAPSAP

LOGLOG

MSGMSG

SERVICESERVICE

HANDLERHANDLER

TIMETIME

SERVERSERVER

C++WRAPPERS

SPIPESPIPE

SAPSAP

CORBACORBA

HANDLERHANDLER

SYSVSYSVWRAPPERSWRAPPERS

SHAREDSHARED

MALLOCMALLOC

THE ACE ORBTHE ACE ORB

((TAOTAO))

JAWS ADAPTIVEJAWS ADAPTIVE

WEB SERVERWEB SERVER

MIDDLEWARE

APPLICATIONS

REACTORREACTOR//PROACTORPROACTOR

PROCESSPROCESS//THREADTHREAD

MANAGERSMANAGERS

ADAPTIVE SERVICE EXECUTIVE ADAPTIVE SERVICE EXECUTIVE (ASX)(ASX)

SERVICESERVICE

CONFIGCONFIG--URATORURATOR

SYNCHSYNCH

WRAPPERSWRAPPERS

MEMMEM

MAPMAP

OS ADAPTATION LAYER

www.cs.wustl.edu/�schmidt/ACE.html

� ACE Overview

{ Concurrent OO

networking

framework

{ Ported to C++

and Java

{ Runs on RTOSs,

POSIX, and

Win32

� Related work

{ x-Kernel

{ SysV STREAMS


DouglasC.Schmidt


ACEStatistics

�

ACEcontain>

125,000linesofC++

{Over10person-yearsofe�ort

�

PortedtoUNIX,Win32,MVS,and

embeddedplatforms

{e.g.,VxWorks,LynxOS,pSoS

�

Largeusercommunity

{www.cs.wustl.edu/�

schmidt/ACE-

users.html

�

Currentlyusedbydozens

ofcompanies

{Bellcore,Boeing,

Ericsson,Kodak,

Lucent,Motorola,

SAIC,Siemens,

StorTek,etc.

�

Supportedcommercially

{www.riverace.com


6


TAO's Real-time ORB Endsystem ArchitectureRR

UU

NN

TT

II

MM

EE

S S

CC

HH

EE

DD

UU

LL

EE

RR

SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER

ATM PORT INTERFACECONTROLLER (APIC)

ZZ

EE

RR

OO

CC

OO

PP

YY

BB

UU

FF

FF

EE

RR

SS

II//OOSUBSYSTEMSUBSYSTEM

OBJECTOBJECTADAPTERADAPTER

SERVANTSERVANT DEMUXERDEMUXER

CLIENTSCLIENTSSTUBSSTUBS

SERVANTSSERVANTSSKELETONSSKELETONS

U

ORBORB CORECORE

REACTORREACTOR

(20 (20 HZHZ))

REACTORREACTOR

(10 (10 HZHZ))

REACTORREACTOR

(5 (5 HZHZ))

REACTORREACTOR

(1 (1 HZHZ))

QQooSSAPIAPI

� Solution Approach

{ Integrate RT dispatcher into

ORB endsystem

{ Support multiple request

scheduling strategies

� e.g., RMS, EDF, and MUF

{ Requests ordered across

thread priorities by OS

dispatcher

{ Requests ordered within

priorities based on data

dependencies and importance



Real-time Experiments over ATM

��

C 1C 0

Requests

C n

��

Client

...

...

2ATM Switch

Ultra 2 Ultra 2

I/O SUBSYSTEM

Server

Object Adapter

ORB Core

Servants RU

NT

IME

SCH

ED

UL

ER

www.cs.wustl.edu/�schmidt/RT-perf.ps.gz

� One high-priority client

� 1..n low-priority clients

� Server factory implements

thread-per-priority

{ Highest real-time

priority for high-priority

client

{ Lowest real-time

priority for low-priority

clients


DouglasC.Schmidt


ORBLatencyResultsoverATM

04812

16

20

24

28

32

36

40

44

48

52

15

10

15

20

25

30

35

40

45

50

Nu

mb

er o

f L

ow

Pri

ori

ty C

lien

ts

Latency per Two-way Request in Milliseconds

CO

RB

Aplu

s H

igh P

riority

MT

-OR

BIX

Hig

h P

riority

min

iCO

OL H

igh P

riority

CO

RB

Aplu

s Low

Priority

MT

-OR

BIX

Low

Priority

min

iCO

OL L

ow

Priority

TA

O H

igh P

riority

TA

O L

ow

Priority

�

Synopsisofresults

{COOL'slatencyislower

forsmall#ofclients

{TAO'slatencyislowest

forlarge#ofclients

{TAOavoidspriority

inversion

�

i.e.,highpriority

clientalwayshas

lowestlatency


9


ORB Jitter Results over ATM

1 5 10 15 20 25 30 35 40 45 50

TAO High Priority

TAO Low Priority

miniCOOL High Priority

miniCOOL Low Priority

MT-ORBIX High Priority

MT-ORBIX Low Priority

CORBAplus High Priority

CORBAplus Low Priority

0

50

100

150

200

250

300

Jitt

er in

mill

isec

on

ds

Number of Low Priority Clients

� De�nition

{ Variance from

average latency

� Synopsis of

results

{ TAO's jitter is

lowest and most

consistent

{ MT-Orbix's

jitter is highest

and more

variable


DouglasC.Schmidt


User-levelandKernel-levelLockingOverhead

41

94

199

175

100

231216

599

0

100

200

300

400

500

600

700

TA

Om

iniCO

OL

CO

RB

Aplus

MT

OR

BIX

OR

Bs T

ested

User Level Lock Operations per Request

client

server

00

44

0

13

8

14

0 2 4 6 8 10 12 14 16

TA

Om

iniCO

OL

CO

RB

Aplus

MT

OR

BIX

OR

Bs T

ested

Kernel Level Lock Operations per Request

client

server


11


Integrating TAO with a Real-time ATM I/O Subsystem

REACTORREACTOR

(20 (20 HZHZ))

REACTORREACTOR

(10 (10 HZHZ))

REACTORREACTOR

(5 (5 HZHZ))

REACTORREACTOR

(1 (1 HZHZ))

ORBORB CORECORE

APICAPIC ATMATM DRIVERDRIVER

II//OOSUBSYSTEMSUBSYSTEM

PERPER--VCVC

SOCKETSOCKET

QUEUESQUEUES

APICAPIC PACINGPACING

QUEUESQUEUES

ZZ

EE

RR

OO

CC

OO

PP

YY

BB

UU

FF

FF

EE

RR

SS

� Key Features

{ Vertical integration of QoS

through ORB, OS, and ATM

network

{ Real-time I/O enhancements to

Solaris kernel

{ Provides rate-based QoS

end-to-end

{ Leverages APIC features for cell

pacing and zero-copy bu�ering


DouglasC.Schmidt


ConcludingRemarks

�

Developersofdistributedapplicationsconfrontrecurringchallengesthat

arelargelyapplication-independent

{e.g.,serviceinitializationanddistribution,errorhandling, ow

control,eventdemultiplexing,concurrencycontrol,persistence,fault

tolerance

�

Successfuldevelopersresolvethesechallengesbyapplyingappropriate

designpatternstocreatecommunicationframeworksandcomponents

�

CORBAORBsareane�ectivewaytoachievereuseofdistributed

softwarecomponents

�

Thenext-generationofORBswillprovidemuchbettersupportfor

real-timeQoSoverATM


13

Douglas - cs.wustl.eduschmidt/PDF/ATM.pdfDouglas C. Schmidt High-p erfo rmance, Real-time ORBs...

Documents

Transcript of Douglas - cs.wustl.eduschmidt/PDF/ATM.pdfDouglas C. Schmidt High-p erfo rmance, Real-time ORBs...