ACM SIGPLAN Workshop on Optimization of Middleware and Distributed Systems (OM 2001) June 18, 2001...

ACM SIGPLAN Workshop on Optimization of Middleware and Distributed Systems (OM 2001)

June 18, 2001

Designing and Optimizing a Scalable CORBA Notification Service

Pradeep Gore, Ron Cytron {pradeep,cytron}@cs.wustl.edu

Department of Computer Science, Washington University,

Saint Louis

Doug Schmidt, Carlos O’Ryan {schmidt,coryan}@uci.edu

Electrical and Computer Engineering, University of

California, Irvine

Pradeep Gore Designing and Optimizing a Scalable CORBA Notification Service

Overview • CORBA Notification Service

– CORBA Event Service and drawbacks– Structured Events, Filtering and QoS properties.

• Design Challenges– Handling multiple event, supplier and consumer types uniformly.– Efficiently propagating different event types to different consumers.– Minimizing interference between channel participants.– Fairness in Event Processing– Optimizing the performance of Any's.– Footprint reduction.– Customizing for particular deployment environments.

• Performance Measurements– Event Channel Scalability (per consumer throughput).– Effect of different thread configurations on consumer throughput.– Filtering overhead.

• Concluding Remarks


CORBA invocation• Standard "CORBA"

method invocations result in synchronous execution of an operation provided by an object

• Both requestor (client) and provider (server) must be present

• Client blocks until operation returns

• Only supports uni-cast communication


CORBA Event Service

• For many applications, a more decoupled communication model between objects is required. (e.g. publish-subscribe)

• asynchronous communication with multiple suppliers and consumers.


Common Event Service Collaborations


Benefits of the OMG Event Service

• Anonymous consumers/suppliers

– Publish and subscribe model

• Group communication

– Supplier(s) to consumer(s)

• Decoupled communication

– Asynchronous delivery

• Abstraction for distribution

– Can help draw the lines of distribution in thesystem

• Abstraction for concurrency

– Can facilitate concurrent event handling


Notification Service• Extends Event Service• Filtering

– Consumers can specify arbitrarily complex filtering constraints in Extended TCL (Trader Constraint Language).

• Structured Events

– carry filtering and QoS parameters that influence the delivery of payload data to its destination.

• Sharing subscription information

– Subscriptions and publications are visible to participants.

• QoS properties

– Allows suppliers, consumers administrators of event channels to configure QoS properties such as reliability, priority, buffer ordering and timeliness on a per-channel, per-proxy or per-event basis.


Component Structure of the CORBA Notification Service


Structured Events • Defines a standard data structure into

which a wide variety of event messages can be stored.

• Fixed event header contains type information -– Domain (e.g.Telecom, Health

Care, Financial)– Type ( e.g. Communication

Alarm, VitalSigns, StockQuote)– Event Name (e.g. heartrate, ECG

reading)• Variable header has per event QoS

properties as (name, value) pairs.• Event Body has filterable body fields

that are used by filters, and the payload data.


Example use case of Notification Service


Design Challenges


Challenge 1: Handling Multiple Event, Supplier and Consumer types

uniformly • Context

– Events can have different representations - e.g. Any, Structured Sequence or custom

– Corresponding Consumers and Suppliers can have different representations e.g. StructuredConsumer

• Problem– Event Channel must propogate events in one form to

consumers that require it in another form.– Do not want to convert to a cannonical format or make many

copies.– For similar IDL interfaces, do not want to duplicate code in

implementation.


Solution1 : Adapter Pattern• Adapter converts the interface

of a class to another interface that a client expects.

• Adapter object implements a Target interface and delegates operations to the Adaptee.

• Notify_Event = target interface

• Any_Event, Structured_Event = Adapters

• CORBA::Any, CosNotification::StructuredEvent = Adaptee


Challenge 2: Efficiently propagating different event types to different consumers

• Context– The event channel can receive events that have

to be dispatched to consumers that accept another type of event. E,g, Any event to Structured consumer.

• Problem– Different event types have to be propagated to

different types of receiving consumers.


Solution 2: Visitor pattern• Visitor allows to

represent an operation to be performed on the elements of an object structure

• “Double Dispatching”– Channel invokes

dispatch_event method that in turn invokes push_event on the event.


Challenge 3: minimizing “interference” between channel participants

• Context– Event Channel must receive requests, apply filtering and

dispatch events to consumers.

– Some suppliers might not want to be blocked in a CORBA 2-way while event channel delivers the event to consumers.

– Filter evaluation might be a lengthy operation (e.g. A remote filter that needs to consult a database)

– A consumer might be arbitrarily slow hence causing the event channel to block in the dispatching 2-way

• Problem– A reasonable implementation should strive to minimize this

“interference” between channel participants.


Solution: 3, Active Object pattern

• Active object pattern decouples method execution from method invocation.

• Events and operations required on them are encapsulated as command objects.

• An enqueuing thread places events in the queue as per the buffering policy.

• Worker thread(s) dequeue each event and applies the operation.


Challenge 4: Ensuring fairness in event processing

• Context– At worst, the event processing overhead is a

maximum of 4 filters and the time to deliver to consumer.

– Though events are queued according to priority, a long winded event processing phase can starve other events in the queue from making any progress.

• Problem– Event processing should be broken up into different stages to

ensure fairness.


Solution 4: Command Object Pattern

• A command object encapsulates a request as an object, thereby allowing parameterization of different requests.

• Filter processing, subscription lookup and event dispatching are encapsulated as command objects.


Challenge 5: Optimizing the performance of Anys

• Context– A CORBA compliant Notification Service must be able to

process events containing Anys.

– An Any is an expensive data type that can have many levels of nesting. E.g. an Any that contains a sequence of Anys and so on.

– The ORB demarshalling engine makes copies of the entire data buffer used to represent the Any.

• Problem– Any intensive CORBA services such as Notification can

benefit from reduced data copies of this expensive type.


Solution 5: Reference Counting• A reference counted object can be shared by

multiple objects and destroyed when it is not in use.

• Client applications rather than the ORB can be make to release a ref. counted Any.

• Very well suited for Notification as the service itself does not modify the event.

• As the event can be logically multicast to multiple recipients, the ownership can be shared via a reference count.


Other design challenges• Footprint Reduction

– A use case might want a subset of the Notification Service.– The builder pattern can be used to make the service

“composable” by users.• Customizing event channels for particular deployment

environments.– A developer might want to extend the service

implementation.– Need a way to specify options and properties for a

configuration– The Component Configurator pattern decouples the

behavior of component services from the time the service is configured into an application.


Performance measurements•Experimental Testbed

• QuadCPU PC with 400MHz pentiumII processors•Linux 2.2.18, GCC 2.95.4•TAO 1.1.15, all optimizations enabled.


Event Channel ScalabilityPer Consumer Throughput

0

200

400

600

800

1000

12001 3 5 7 9 11 13 15 17 19

consumers

even

ts/s

ec

1 Dispatching Thread

2 DispatchingThreads



Reactive Dispatching

Expon. (ReactiveDispatching)

Expon. (4Dispatching Threads)


Effect of Thread configuration

Effect of Thread configuration on Throughput

0

100

200

300

400

500

600

Consumers

even

t/se

c

Series1 0.99 0.99 0.99 1.2 0.99 519

Slow Fast Slow Fast Slow Fast

Reactive 2 Threads, 1 shared queue

Thread and queue per proxy

• 2 consumers, 1 supplier

• “slow” consumer has a 1 sec. delay in push.

• Case 1: reactive• Case 2: 2

threads, 1 shared queue.

• Case 3: Thread an queue per proxy supplier.


Filtering overhead

• Filter attached to Consumer Admin, 2 dispatching threads.

• Filtering drops average throughput of 15% and picks up slightly by 2% with 2 more dispatching threads.


Concluding remarks• Notification Service allows decoupled,

anonymous communication between consumers and suppliers.– Filtering and QoS properties

• Suitable Design patterns and reusable framework components were utilized to implement TAO’s Notification Service.

• Critical path of event propagation was optimized.

• Configurable and extensible implementation.

ACM SIGPLAN Workshop on Optimization of Middleware and Distributed Systems (OM 2001) June 18, 2001...

Documents

Transcript of ACM SIGPLAN Workshop on Optimization of Middleware and Distributed Systems (OM 2001) June 18, 2001...