Messaging Systems and Technology
description
Transcript of Messaging Systems and Technology
Messaging Systems and Technology
Introduction
• Synchronous middleware doesn’t suit all applications
• Message Oriented Middleware (MOM) provides features like:
• asynchronous communications between processes• store-and-forward capabilities• transactional messaging
Basic Asynchronous Operations
• Send (dest, message)• Receive ( target, message)
receivesendqueue
MOM usage
• Applications often need:• deferred processing of some ‘slow’ transaction (eg
printing an invoice)• support efficient 1-to-many and many-to-many
communications• to send messages whether the server is available or not• to provide event notification to a dynanic user
community• loosely coupled client-server systems
Example - Transactions
//transaction 2transaction {
GetMessage (....)} onCommit { .... }onAbort {.....}
DBMS Queue
//transaction 1transaction {
UpdateDBMS(…..)PutMessage (....)
} onCommit { .... }onAbort {.....}
MOM Technologies
• Broadly two (somewhat intersecting) categories
• Message brokers (IBM, BEA, Microsoft, etc)• Broadcast-based publish-subscribe (TIBCO, IONA)
• Both categories differ greatly in their features and capabilities
Some Application Examples
• Stock price update notification• General workflow systems• Application integration• System management
Some Example Products
• TIBCO/Rendezvous• IONA’s OrbixTalk• IBM’s MQ Series (Publish-Subscribe)
TIB/Rendezvous• Pioneers of broadcast/multicast publish-
subscribe technology• Publishers and subscribers communicate
using subjects
PubSubject
SubSub
SubCreate/Publish
Register/Subscribe
Subjects
• Hierarchical names identify a subject of interest
• /CSIRO• /CSIRO/gossip• /CSIRO/gossip/ADSaT• /CSIRO/work/ADSaT
• Wildcards can be used• /CSIRO/**• /CSIRO/*/ADSaT
Multicast/Broadcast
• A published message is sent across the network only once
• All subscribers receive the same message• IP Multicast or broadcast is used on LANs• Important scalability issue as
bandwidth/processor usage is low
LAN Architecture
BroadcastNetwork
rvd
rvd
Subscribe
Publisher
Publisher
Subscribe
WAN Architecture
LocalBroadcastNetwork
rvd
Subscribe
Publisher
rvrdrvrd
LocalBroadcastNetwork
rvd
Publisher
Subscribe
Wide Area Network
Filtered,Point-to-Point
Quality-of-Service (QOS)
• Reliable• publisher informed after time-out period if message
cannot be delivered to a subscriber
• Certified• guaranteed delivery or both parties informed of
failure• to survive process failure messages can be logged to
disk until specified time-out period expires
Programming Options
• C/C++/Java• Self-describing message format• TIB/ObjectBus layers a standard CORBA-
compliant ORB upon the TIBCO protocols. • TIBIOP provides broadcast messages in a
CORBA environment.
Security
• Security is a problem in publish-subscribe multicast technology
• TIBCO Data Security (DS) product:• publisher controls secure key distribution to
subscribers using digital signatures• subscribers use key to encrypt messages on a subject• choice of encryption algorithm available
OrbixTalk
• Uses IP multicast in CORBA environment
OrbixTalkApplication
OrbixTalk Multicast Transport Protocol
OrbixTalk API
CORBAEvent Service
OrbixTalkApplication
Topics• Communication based on topics (read
subjects, wildcards, etc)• 3 QOS levels:
– otmcp: UDP IP multicast. Limits packet size to 1280 bytes and no guarantee of receipt.
– otrmp: Augments IP multicast to ensure message delivery to all subscribers until configurable time-out expires
– otsfp: OrbixTalk MessageStore daemon uses a store-and-forward protocol to provide guaranteed message delivery.
OrbixTalk Architecture
Sender
Sender
Sender
Sender
MulticastProtocols
Listener
Listener
Listener
Listener
OrbixTalkDaemon
MessageStore
Daemon
I P Address
otmcp
otrmp
Topic
otsfp
Topic
IPAddress
Architecture Issues
• Daemons can have hot backups to support failure of primary daemon
• Several MessageStore daemons can be used to facilitate load balancing:
• each OrbixTalk process only use one MessageStore
• Compaction utility must be scheduled to remove old messages
Security/WANs
• No specific security or WAN support• security can be programmed using Orbix Transformers
• WANs traffic needs appropriately configured routers, or an IIOP bridge between LANs to be written
MQSeries PubSub
• MQSeries is probably most widely deployed MOM product
• Basic technology provides a message queue architecture:
• communication via shared queues managed by Queue Managers
• persistent/non-persistent messages• transactional queue access with XA• broad platform support
MQSeries PubSub
BROKER A
BROKER B
Subscriber1
Subscriber2Publisher
1
Publisher2
Topic A
Topic B
Topic B
Topic B
Topic A
1 PubSub Broker per QM
Topics interchanged by Brokers if needed
LAN/WAN support
Architecture
• Brokers can be organized to communicate hierarchically
USA
West East
HQ
Asia
HK
Features
• Typical hierachical topic names, wildcards, etc
• Streams can be used as higher level topic partitioning scheme
• DEFAULT stream per broker• optional additional streams for specified topics• broker allocates a thread to handle each stream
Quality-of-Service
• Persistent• messages written to log file• survive broker/QM failure
• Non-persistent• messages lost if broker fails• fast
• Message priority supported in queues
Other features
• MQ can act as an XA transaction manager• Basic security provided, integrated with
native operating system• Security exits support integration with 3rd
party products• Other products, eg MQ Integrator
Performance Test
• Publishers and subscribers run on different machines
• Dual pentium 500Mhz NT 4.0 boxes, 0.5GB memory
• MQ QM/broker and Orbix daemons run on same machine as publishers
• Reliable, non-presistent protocols used
256 Byte Message
0100200300400500600700
10 20 30 40 50 No. Of Subscribers
Milliseconds
OrbixTalkTIBCOMQ
512 Byte Message
0
200
400
600
800
1000
10 20 30 40 50No. Of Subscribers
Milliseconds
OrbixTalkTIBCOMQSeries
Interpretation
• Multicast technologies are fast!• TIBCO rvd becomes bottleneck
• ‘Publisher too fast’ exceptions
• OrbixTalk generates large number of interrupts
• one per message per listener
• MQ scales well, but queue/broker architecture is inherently slower
Summary• MOM provides excellent solution to many
business problems• Range of technologies available, with very
different:• architectures/technologies• features• performance
• Important to understand issues and select right one for your business