An Introduction to the Message Queuing Technology & IBM WebSphere MQ
-
Upload
ravi-yogesh -
Category
Technology
-
view
793 -
download
1
Transcript of An Introduction to the Message Queuing Technology & IBM WebSphere MQ
An Introduction to the Message Queuing
Technology & IBM WebSphere MQ
Ravi YogeshWeb Performance Engineer, Wells Fargo
(https://www.linkedin.com/in/raviyogesh)
Agenda:
Basics of MQ Technology
(Persistence, Types, Styles)
Message Queuing in Banking
Different Message Queuing Implementations
In Focus : IBM WebSphere MQ
Message Queues in Action around Us
MQ Technology: Basics
• Message queuing is a middleware technology that greatly simplifies
communication between the nodes of a system and between the nodes that
connect systems together.
• This technology builds on two basic concepts:
Message : String of Bytes/Information that needs to be communicated from one
node to another.
Has two parts : Header (message ID, message type, priority, expiration time )
and Data (can be XML, binary data etc)
Queues : Data Structures used to store messages.
• They enable applications and services to communicate asynchronously,
without each having to be available at the same time.
• When both nodes are available, messages arrive at destination in near real
time.
• If either of the network or node is unavailable/busy, the queue holds the
messages until they can be received and processed, thus assuring
delivery.
Message Persistence:
• Persistent messages are logged (data synchronously written to disk at the
same time as sending) thus enabling recovery of message on failure.
• Semi Persistent Messaging : Tradeoff between robustness of delivery and
throughput- logging occurs asynchronously after message dispatch- faster but
riskier than persistent mode.
• Non persistent Messaging : No logging thus no recovery on failure !!
• Recovery , persistence and assured delivery of messages : handled by the
Queue Managers which are themselves backed up in high availability
environments.
Types of Queues:
Remote Queues: References/handles to queues residing on other machines.
Transmission Queues: Special Queues used internally to move messages across
a network.
Dynamic Queues: Local Queues created on demand.
Dead Letter Queues: Queues that store messages that can not be delivered
(destination queue full scenario).
Repository Queues: Queues holding clustering information.
Messaging Topology:
Point to Point : Useful for simple
scenarios and NOT for larger
deployments because of the large
number of connections required.
{n(n-1)/2}.
Hub and Spoke: Requires more
initial planning but much lesser
number of connections for the
same number of nodes. Single
point of failure exists !
Bus: Same number of connections
as Hub and Spoke with distributed
connectivity logic across the
backbone. Thus, no single point of
failure.
Messaging Styles:
• Fire & Forget: Sender does not need a confirmation from receiver.
• Request - Response: Replies need to be sent from receiver.
• Publish & Subscribe: Event driven style of messaging, backbone
dynamically determines where the messages have to be delivered, relives
applications and services of information burden.
Benefits of Message Queuing Technology in Banking:
• Letting the applications talk to the messaging backbone through a
simple common API saves a lot of complex and application specific
connectivity logic to be defined on each application.
• Assured delivery: Critical business information like a banking
transaction needs assured and only-once delivery.
• Time-independent (asynchronous) processing.
• Avoids redundancy and duplication of data.
• Can easily scale and adapt to business requirements change.
Message Queuing Implementations:
Proprietary software:• Microsoft Message Queuing (tied to specific operating system).
• IBM's WebSphere MQ.
Message Queuing as a Service: • StormMQ
• IronMQ
• Amazon SQS.
Open Source Software: • JBoss Messaging
• JORAM
• Apache ActiveMQ
• Sun Open Message Queue
• Apache Qpid
• RabbitMQ,
• Beanstalkd,
• Tarantool
• HTTPSQS
.
Queuing Solutions on the Cloud:
Amazon Simple Queue Service (Amazon SQS):
A distributed queue messaging service introduced by Amazon.com in April
2006.
Storm MQ: available as hosted, On-Premise or Cloud solution for
Machine-to-Machine Message Queuing using the standard Advanced
Message Queuing Protocol (AMQP).
Rabbit MQ : open source software that implements the AMQP & is written
in the Erlang programming language.
In Focus : IBM MQ WebSphere
WebSphere MQ moves more than 10 billion messages every day,
supporting more than US$1 quadrillion (US $1 000 000 000 000 000)
worth of business transactions !!
• launched by IBM in March 1992
• available on a large number of platforms (both IBM and non-IBM),
including z/OS (mainframe), OS/400 (IBM System i or
AS/400), Transaction Processing Facility, UNIX (AIX, HP-UX, Solaris),
HP NonStop, OpenVMS, Linux, OS 2200, and Microsoft Windows.
Message Handling in Websphere MQ:
• Each WebSphere MQ message can be up to 100 MB in size.
• Larger messages and files can be transported by segmenting them into
smaller chunks.
• Messaging traffic can be reduced when many very small
messages are being sent to the same destination by grouping these
together into larger WebSphere MQ messages.
• By default, WebSphere MQ uses persistent messaging.
• WebSphere MQ can preserve the order of messages, delivering them to
applications in the same order as they were dispatched. First-In-First-Out
(FIFO) is the default.
Clustering
• Enables to improve the performance of a messaging backbone, more
messaging engines to be added as needed.
Scalability
• WebSphere MQ takes advantage of multiprocessor and multicore
machines so that you can scale your messaging backbone by using
parallel processing to accelerate messaging.
Security
• Supports the industry standard Secure Sockets Layer (SSL) for strong
authentication of message channels before messages are exchanged to
help prevent malicious attacks to the backbone.
Message Queuing is Ubiquitous !
Twitter : Open Source Message Queue Service: Kestralhttps://blog.twitter.com/2009/building-open-source
Whatsapp and Facebook : Messaging Cluster based on XMPP
(Extensible Messaging and Presence Protocol) and Erlang Programming
Language.http://www.erlang-factory.com/upload/presentations/558/efsf2012-whatsapp-scaling.pdf
References:
• http://www.redbooks.ibm.com/redbooks/pdfs/sg247128.pdf
• ftp://ftp.software.ibm.com/software/integration/wmq/WS_MQ_Messaging_Backbone_for_SOA.pdf
• http://blog.codepath.com/2013/01/06/asynchronous-processing-in-web-applications-part-2-
developers-need-to-understand-message-queues/
• http://www.infoq.com/articles/AMQP-RabbitMQ