Applying a Real-time CORBA ORB for Avionics …schmidt/PDF/TAO-usecases.pdfApplying a Real-time...
Transcript of Applying a Real-time CORBA ORB for Avionics …schmidt/PDF/TAO-usecases.pdfApplying a Real-time...
Applying a Real-time CORBA ORB forAvionics Mission Computing
Douglas C. [email protected]
Washington University, St. Louiswww.cs.wustl.edu/�schmidt/TAO4.ps.gz
SponsorsBoeing and CDI/GDIS
Douglas C. Schmidt High-performance, Real-time ORBs
Mission Computing Design Requirements and Forces
� Integrate real-time scheduling/dispatching in ORB and I/O subsystemfor Boeing military aircraft product families
– i.e., Harrier (AV/8b), F-15, and F/A-18
� Provide all applications with real-time capabilities
– Both method-oriented and event-oriented applications
� Meet deterministic and statistical QoS requirements
– i.e., minimize latency, context switching, priority inversion, andnon-determinism
Washington University, St. Louis 1
Douglas C. Schmidt High-performance, Real-time ORBs
Motivation for CORBA for Mission Computing
DIIDII ORBORBINTERFACEINTERFACE
ORBORBCORECORE
operation()operation()
OBJECTOBJECT
ADAPTERADAPTER
IDLIDLSKELETONSKELETON
DSIDSI
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
GIOPGIOP//IIOPIIOP
SERVANTSERVANT
STANDARD INTERFACESTANDARD INTERFACE STANDARD LANGUAGESTANDARD LANGUAGE
MAPPINGMAPPING
ORB-ORB-SPECIFIC INTERFACESPECIFIC INTERFACE STANDARD PROTOCOLSTANDARD PROTOCOL
IDLIDLSTUBSSTUBS
www.cs.wustl.edu/�schmidt/corba.html
� Benefits
– Simplify distributionby automating
� Object locationand activation
� Parametermarshaling
� Demultiplexing
� Error handling– Provide foundation
for higher-levelservices
Washington University, St. Louis 2
Douglas C. Schmidt High-performance, Real-time ORBs
The ACE ORB (TAO)
NETWORKNETWORK
ORBORB QQOOSSINTERFACEINTERFACE
REALREAL--TIMETIMEORBORB CORECORE
operation()operation()
RIDLRIDLSTUBSSTUBS
REALREAL--TIMETIME
OBJECTOBJECT
ADAPTERADAPTER
RIDLRIDLSKELETONSKELETON
in argsin args
out args + return valueout args + return value
CLIENTCLIENT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK ADAPTERSNETWORK ADAPTERS
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
RIOPRIOP
SERVANTSERVANT
OS KERNELOS KERNEL
HIGHHIGH--SPEEDSPEED
NETWORK ADAPTERSNETWORK ADAPTERS
REALREAL--TIME ITIME I//OOSUBSYSTEMSUBSYSTEM
www.cs.wustl.edu/�schmidt/TAO.html
� TAO Overview
– A real-time,high-performanceORB
– Leverages ACE
� Runs on POSIX,Win32, RTOSs
� Related work
– U. RI, Mitre– QuO at BBN– ARMADA at U.
Mich.
Washington University, St. Louis 3
Douglas C. Schmidt High-performance, Real-time ORBs
The ADAPTIVE Communication Environment (ACE)
PROCESSESPROCESSES//THREADSTHREADS
DYNAMICDYNAMIC
LINKINGLINKING
MEMORYMEMORY
MAPPINGMAPPING
SELECTSELECT//IO COMPIO COMP
SYSTEMSYSTEM
VV IPCIPCSTREAMSTREAM
PIPESPIPES
NAMEDNAMED
PIPESPIPES
CAPIS
SOCKETSSOCKETS//TLITLI
COMMUNICATIONCOMMUNICATION
SUBSYSTEMSUBSYSTEM
VIRTUAL MEMORYVIRTUAL MEMORY
SUBSYSTEMSUBSYSTEM
GENERAL POSIX AND WIN32 SERVICES
PROCESSPROCESS//THREADTHREAD
SUBSYSTEMSUBSYSTEM
FRAMEWORKS ACCEPTORACCEPTOR CONNECTORCONNECTOR
SELF-CONTAINED
DISTRIBUTED
SERVICE
COMPONENTS
NAMENAME
SERVERSERVER
TOKENTOKEN
SERVERSERVER
LOGGINGLOGGING
SERVERSERVER
GATEWAYGATEWAY
SERVERSERVER
SOCKSOCK__SAPSAP//TLITLI__SAPSAP
FIFOFIFO
SAPSAP
LOGLOG
MSGMSG
SERVICESERVICE
HANDLERHANDLER
TIMETIME
SERVERSERVER
C++WRAPPERS
SPIPESPIPE
SAPSAP
CORBACORBA
HANDLERHANDLER
SYSVSYSVWRAPPERSWRAPPERS
SHAREDSHARED
MALLOCMALLOC
THE ACE ORBTHE ACE ORB
((TAOTAO))
JAWS ADAPTIVEJAWS ADAPTIVE
WEB SERVERWEB SERVER
MIDDLEWARE
APPLICATIONS
REACTORREACTOR//PROACTORPROACTOR
PROCESSPROCESS//THREADTHREAD
MANAGERSMANAGERS
ADAPTIVE SERVICE EXECUTIVE ADAPTIVE SERVICE EXECUTIVE (ASX)(ASX)
SERVICESERVICE
CONFIGCONFIG--URATORURATOR
SYNCHSYNCH
WRAPPERSWRAPPERS
MEMMEM
MAPMAP
OS ADAPTATION LAYER
www.cs.wustl.edu/�schmidt/ACE.html
� ACE Overview– Concurrent OO
networkingframework
– Ported to C++and Java
– Runs on RTOSs,POSIX, andWin32
� Related work– x-Kernel– SysV STREAMS
Washington University, St. Louis 4
Douglas C. Schmidt High-performance, Real-time ORBs
ACE Statistics
� ACE contain > 135,000 lines of C++
– Over 15 person-years of effort
� Ported to UNIX, Win32, MVS, andembedded platforms
– e.g., VxWorks, LynxOS, pSoS
� Large user community
– www.cs.wustl.edu/�schmidt/ACE-users.html
� Currently used bydozens of companies
– Bellcore, Boeing,Ericsson, Kodak,Lockheed, Lucent,Motorola, SAIC,Siemens, StorTek,etc.
� Supported commercially
– www.riverace.com
Washington University, St. Louis 5
Douglas C. Schmidt High-performance, Real-time ORBs
Applying TAO to Avionics Mission Computing
OBJECTOBJECT REQUESTREQUEST BROKERBROKER
2:2:PUSH PUSH ((EVENTSEVENTS))
AirAirFrameFrame
SensorSensorproxyproxy
HUDHUD
NavNav
SensorSensorproxyproxy
SensorSensorproxyproxy
3:3:PUSH PUSH ((EVENTSEVENTS))
1:1: SENSORS SENSORS
GENERATEGENERATE
DATADATA
EVENTEVENT
CHANNELCHANNEL
www.cs.wustl.edu/�schmidt/oopsla.ps.gz
� Domain Challenges
– Periodic deterministic(and some statistical)real-time deadlines
– COTS infrastructure– Open systems
� Related work
– Deng, Liu, and J. Sun ’96– Gopalakrishnan and
Parulkar ’96– Wolfe et al. ’96
Washington University, St. Louis 6
Douglas C. Schmidt High-performance, Real-time ORBs
TAO’s Real-time ORB Endsystem Architecture
RR
UU
NN
TT
II
MM
EE
S S
CC
HH
EE
DD
UU
LL
EE
RR
SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER
ATM PORT INTERFACECONTROLLER (APIC)
ZZ
EE
RR
OO
CC
OO
PP
YY
BB
UU
FF
FF
EE
RR
SS
II//OOSUBSYSTEMSUBSYSTEM
OBJECTOBJECTADAPTERADAPTER
SERVANTSERVANT DEMUXERDEMUXER
CLIENTSCLIENTSSTUBSSTUBS
SERVANTSSERVANTSSKELETONSSKELETONS
U
ORBORB CORECORE
REACTORREACTOR
(20 (20 HZHZ))
REACTORREACTOR
(10 (10 HZHZ))
REACTORREACTOR
(5 (5 HZHZ))
REACTORREACTOR
(1 (1 HZHZ))
QQooSSAPIAPI
� Solution Approach
– Integrate RT dispatcher intoORB endsystem
– Support multiple requestscheduling strategies
� e.g., RMS, EDF, and MUF– Requests ordered across
thread priorities by OSdispatcher
– Requests ordered withinpriorities based on datadependencies andimportance
Washington University, St. Louis 7
Douglas C. Schmidt High-performance, Real-time ORBs
Server Request Reception Use-case
RR
UU
NN
TT
II
MM
EE
S S
CC
HH
EE
DD
UU
LL
EE
RR II//OO SUBSYSTEMSUBSYSTEM
ORBORB CORECORE
SERVANTSERVANT DEMUXERDEMUXER
OBJECTOBJECT ADAPTERADAPTER
REACTORREACTOR
(20 (20 HZHZ))
REACTORREACTOR
(10 (10 HZHZ))
REACTORREACTOR
(5 (5 HZHZ))
REACTORREACTOR
(1 (1 HZHZ))
SERVANTSSERVANTS
SKELETONSKELETON
SERVANTSSERVANTS
SKELETONSKELETONSERVANTSSERVANTS
SKELETONSKELETON
SOCKETSOCKET QUEUEQUEUE DEMUXERDEMUXER 1:1: I I//O SUBSYSTEMO SUBSYSTEM
RECEIVES INCOMING RECEIVES INCOMING
CLIENT REQUEST CLIENT REQUEST
2:2: RUN RUN--TIME SCHEDULERTIME SCHEDULER
DETERMINES PRIORITY DETERMINES PRIORITY
OF REQUEST OF REQUEST
3:3: REQUEST QUEUED REQUEST QUEUED
AND DEQUEUED AND DEQUEUED
ACCORDING TO ACCORDING TO
PRIORITY PRIORITY//RATERATE
4:4: REQUEST DEQUEUED REQUEST DEQUEUED
BY THREAD WITH BY THREAD WITH
SUITABLE SUITABLE OSOS PRIORITY PRIORITY
5:5: REQUEST DISPATCHED REQUEST DISPATCHED
TO SERVANT TO SERVANT
� Synopsis
– I/O subsystemuses portnumbers todemuxrequests toqueues andRT threadsper rate group
– A Reactordemuxes/dispatchesrequests foreach rategroup
Washington University, St. Louis 8
Douglas C. Schmidt High-performance, Real-time ORBs
Event Channel Reception Use-case
PROXYPROXY
SERVANTSERVANTPROXYPROXY
SERVANTSERVANTPROXYPROXY
SERVANTSERVANT
EVENTEVENT CHANNELCHANNEL
SERVANTSSERVANTSSERVANTSSERVANTSSERVANTSSERVANTS
RR
UU
NN
TT
II
MM
EE
S S
CC
HH
EE
DD
UU
LL
EE
RR
II//OO SUBSYSTEMSUBSYSTEM
ORBORB CORECORE
SERVANTSERVANT DEMUXERDEMUXER
OBJECTOBJECT ADAPTERADAPTER
REACTORREACTOR
(20 (20 HZHZ))
REACTORREACTOR
(10 (10 HZHZ))
REACTORREACTOR
(5 (5 HZHZ))
REACTORREACTOR
(1 (1 HZHZ))
QUEUEQUEUE DEMUXERDEMUXER
1:1: I I//O SUBSYSTEMO SUBSYSTEM
RECEIVES INCOMING RECEIVES INCOMING
CLIENT REQUEST CLIENT REQUEST
2:2: RUN RUN--TIME SCHEDULERTIME SCHEDULER
DETERMINES PRIORITY DETERMINES PRIORITY
OF REQUEST OF REQUEST
3:3: REQUEST QUEUED REQUEST QUEUED
AND DEQUEUED AND DEQUEUED
ACCORDING TO ACCORDING TO
PRIORITY PRIORITY//RATERATE
4:4: REQUEST DEQUEUED REQUEST DEQUEUED
BY THREAD WITH BY THREAD WITH
SUITABLE SUITABLE OSOS PRIORITY PRIORITY
5:5: REQUEST DISPATCHED REQUEST DISPATCHED
TO PROXY SERVANT TO PROXY SERVANT
6:6: EVENT CHANNEL EVENT CHANNEL
COMPUTES EVENT COMPUTES EVENT
IMPORTANCE AND IMPORTANCE AND
DEPENDENCIES AND DEPENDENCIES AND
DISPATCHES TO DISPATCHES TO
CONSUMER SERVANTS CONSUMER SERVANTS
� Synopsis
– Event Channelthreads handleevent importanceand dependencies
– I/O subsystemand ORB Corehandle priorities
Washington University, St. Louis 9
Douglas C. Schmidt High-performance, Real-time ORBs
ORB Latency and Priority Inversion Experiments
����
C 1C 0
Requests
C n
�� ��������������������������
Client
...
...
2ATM Switch
Ultra 2 Ultra 2
I/O SUBSYSTEM
Server
Object Adapter
ORB Core
Servants RU
NT
IME
SCH
ED
UL
ER
www.cs.wustl.edu/�schmidt/RT-perf.ps.gz
� Vary ORBs, hold OSconstant
� Methodology
– 1 high-priority client– 1..n low-priority clients– Server uses
thread-per-priority
� Highest real-timepriority forhigh-priority client
� Lowest real-timepriority forlow-priority clients
Washington University, St. Louis 10
Douglas C. Schmidt High-performance, Real-time ORBs
ORB Latency and Priority Inversion Results
0
4
8
12
16
20
24
28
32
36
40
44
48
52
1 5 10 15 20 25 30 35 40 45 50
Number of Low Priority Clients
Late
ncy p
er
Tw
o-w
ay R
eq
uest
in M
illiseco
nd
s
CORBAplus High Priority CORBAplus Low Priority
MT-ORBIX High Priority MT-ORBIX Low Priority
miniCOOL High Priority miniCOOL Low Priority
TAO High Priority TAO Low Priority
� Synopsis of results
– TAO’s latency is lowest– TAO avoids priority
inversion
� i.e., high-priority clientalways has lowestlatency
– Overhead stems fromconcurrency andconnection architecture
� e.g., synchronizationand context switching
Washington University, St. Louis 11
Douglas C. Schmidt High-performance, Real-time ORBs
ORB Jitter Results
1 5 10 15 20 25 30 35 40 45 50
TAO High Priority
TAO Low Priority
miniCOOL High Priority
miniCOOL Low Priority
MT-ORBIX High Priority
MT-ORBIX Low Priority
CORBAplus High Priority
CORBAplus Low Priority
0
50
100
150
200
250
300
Jitt
er in
mill
isec
on
ds
Number of Low Priority Clients
� Definition
– Variance fromaverage latency
� Synopsis ofresults
– TAO’s jitter islowest andmost consistent
– CORBAplus’jitter is highestand mostvariable
Washington University, St. Louis 12
Douglas C. Schmidt High-performance, Real-time ORBs
User-level and Kernel-level Locking Overhead
41
94
199
175
100
231216
599
0
100
200
300
400
500
600
700
TAO miniCOOL CORBAplus MT ORBIX
ORBs Tested
Use
r L
evel
Lo
ck O
per
atio
ns
per
Req
ues
t
client server
0 0
4 4
0
13
8
14
0
2
4
6
8
10
12
14
16
TAO miniCOOL CORBAplus MT ORBIX
ORBs Tested
Ker
nel
Lev
el L
ock
Op
erat
ion
s p
er R
equ
est
client server
TAO is carefully designed to minimize memory allocation and locking
Washington University, St. Louis 13
Douglas C. Schmidt High-performance, Real-time ORBs
Real-time OS/ORB Performance Experiments
[P ]0
[P ]1
SCH
ED
UL
ER
0
RU
NT
IME
[P ]
I/O SUBSYSTEM
Server
0 nS1
nC
������������������Pentium Pro
S S0C 1C
...
...
Object AdapterServants
ORB Core
Client
...
[P]
Requests
Priority
...
[P ]n
[P ]1
[P ]n
www.cs.wustl.edu/�schmidt/RT-OS.ps.gz
� Vary OS, hold ORBsconstant
� Methodology
– 1 high-priority client– 1..n low-priority clients– Server uses
thread-per-priority
� Highest real-timepriority forhigh-priority client
� Lowest real-timepriority forlow-priority clients
Washington University, St. Louis 14
Douglas C. Schmidt High-performance, Real-time ORBs
Real-time OS/ORB Performance Results
0
1
2
3
4
5
6
7
8
9
10
1 5 10 15 20 25 30 35 40 45 50
Number of Low Priority Clients
Late
ncy p
er
Tw
o-w
ay R
eq
uest
in M
illi
seco
nd
s
Solaris x86 High Priority Solaris x86 Low PriorityLynxOS High Priority LynxOS Low PriorityVxWorks High Priority VxWorks Low PriorityWindows NT High Priority Windows NT Low Priority
� Synopsis of results
– RTOS’s providelowest latency
– RTOS’s minimizepriority inversion
– ORB (TAO) provideslow latency and avoidspriority inversion
� i.e., high priorityclient always haslowest latency
Washington University, St. Louis 15
Douglas C. Schmidt High-performance, Real-time ORBs
Real-time OS/ORB Jitter Results
15
1015
2025
3035
4045
50
LynxOS High Priority
LynxOS Low Priority
VxWorks High Priority
VxWorks Low Priority
Windows NT High Priority
Windows NT Low PrioritySolaris x86 High Priority
Solaris x86 Low Priority
0
0.2
0.4
0.6
0.8
1
1.2
1.4
Jitt
er in
mill
isec
on
ds
Number of Low Priority Clients
� Definition
– Standarddeviation fromaverage latency
� Synopsis ofresults
– Some RTOS’sprovide lowjitter
– ORB (TAO)doesn’tintroduce jitter
Washington University, St. Louis 16
Douglas C. Schmidt High-performance, Real-time ORBs
Real-time OS/ORB CPU Utilization Experiments
I/O SUBSYSTEM
������
C
Requests
Client
Utilization
������������
Intel x86������������������
Object AdapterServants R
UN
TIM
E
SCH
ED
UL
ERORB Core
Server
SPARC
Ethernet Switch
www.cs.wustl.edu/�schmidt/RT-OS.ps.gz
� Vary ORBs, hold OSconstant
� Methodology
– 1 client thread– 2 server threads
� 1 thread servicesclient
� 1 thread factorsprime numbers
Washington University, St. Louis 17
Douglas C. Schmidt High-performance, Real-time ORBs
Real-time OS/ORB CPU Utilization Results
46
20 20
5254
80 80
48
0
10
20
30
40
50
60
70
80
90
Solaris x86 LynxOS VxWorks Windows NT
RTOSs Tested
Per
cen
tag
e o
f U
tiliz
atio
n o
f C
PU
CPU UtilizationIDLE Time � Synopsis of results
– RTOS’s providehighest effectiveutilization
– ORB (TAO)processing uses
�20% of the CPU
Washington University, St. Louis 18
Douglas C. Schmidt High-performance, Real-time ORBs
Concluding Remarks
� TAO is currently used at Boeing for avionics mission computing
– Initial flight dates are mid-summer 1998
� Extensive benchmarks demonsrate it is possible to meet stringentperformance goals with real-time CORBA
– e.g., for Boeing, target latency for CORBA oneway operations is150 �secs for 100 Mhz PowerPC running over MVME 177 boards
� Technology transfer to commercial vendors via OMG RT SIG andDARPA Quorom program
Washington University, St. Louis 19