Message Queues : A Primer - International PHP Conference Fall 2012

download Message Queues : A Primer - International PHP Conference Fall 2012

of 77

  • date post

  • Category


  • view

  • download


Embed Size (px)


Message Queues - A Primer is an introduction to message queues and how they work with PHP.

Transcript of Message Queues : A Primer - International PHP Conference Fall 2012

  • Mike Willbanks | Barnes & NobleMessage Queues : A Primer
  • Housekeeping Talk Slides will be posted after the talk. Me Sr. Web Architect Manager at NOOK Developer Prior MNPHP Organizer Open Source Contributor Where you can find me: Twitter: mwillbanks G+: Mike Willbanks IRC (freenode): mwillbanks Blog: GitHub:
  • Agenda Message Queues? Protocols and Software Picking a Message Queue Best Practices
  • A DefinitionMessage queues and mailboxes are software-engineeringcomponents used for interprocess communication, or for inter-thread communication within the same process. They use a queuefor messaging the passing of control or of content.
  • What is messaging?Messaging describes the sending and receiving of data (in theform of messages) between systems. Messages are exchangedbetween programs or applications, similar to the way peoplecommunicate by email but with guarantees on delivery, speed,security and the absence of spam.
  • Producer Messages Consumer Messages Task MessagesGeneral AnatomyProducer creates a message and pushes it to the queue; theconsumer reads from the queue and processes the message.
  • Describing Message Queuesl Pub/Subl FIFO bufferl Push / Pulll A way to communicate between applications / systems.l A way to decouple components.l A way to offload work.
  • Why to use a Message Queuel Offload Heavy Workl Integration with Legacy Systemsl Asynchronous Processingl Parallel Processingl Process consistencyl Scalability
  • Unix Foundationsl Write programs that work together.l Do it in the background.
  • Why it mattersl Web systems need to be geared to run things asynchronously.l Distribution of loadl System integrity
  • Youve seen them before; they are used in most applications tohelp them scale.MESSAGE QUEUE EXAMPLES
  • When to make use of message queues in case youre wondering.GENERIC USE CASES
  • NotificationsEmail, SMS, Push Messaging.
  • Photo ProcessingThumbnails, Resizing, Watermarking, Converting
  • Video ProcessingResampling, Audio Overlay, Type Conversion
  • AnalyticsWeb Server Logs, Log Aggregation, PHP Errors, etc.
  • IntegrationsSave local first; push second.
  • AMQPAdvanced Message Queuing Protocol
  • Overview of AMQPl AMQP Working Group (Community and Vendor)l Platform agnostic protocol.l Completely open, interoperable and broadly applicable.l Many severs available and many client libraries.
  • How it WorksProducer, Exchange, Queue, Consumer
  • How it Worksl AMQP utilizes exchanges, queues and bindings.l An exchange are routers with routing tables.l A binding defines the routing rules.l A queue is where the messages wait for a consumer.
  • Exchanges
  • Understanding Exchangesl Fanout Exchange l No routing keys involved. Any message that is sent to the exchange is sent to all queues bound to that exchange.l Direct Exchange l Routing keys involved. A queue binds to the exchange to request messages that match a routing key exactly.l Topic Exchange l Routing keys involved. A queue binds to the exchange to request messages that match a routing key pattern.
  • Implementations Very popular and common message queue owned by VMware. Long standing project; apache foundation. Long standing project; ZeroMQ partner, no news since 2009.
  • Building a Queuel An exchange, queue and bindings must be defined first. Publishing can then commence after. l Create the queue l Create the exchange l Bind to the queue.
  • Persistence?l Default behavior is no persistence.l How important are the messages?l Just about all items have a level of persistence if you would like them to survive on reboot. l Mark exchanges, queues and messages as DURABLE.
  • PECL AMQPl Extension compatible with AMQP specification 0-9-1.l pecl install amqp
  • AMQP Client
  • AMQP Worker
  • STOMPSimple (or Streaming) Text Orientated Messaging Protocol
  • Overviewl Simple protocol l Behaviors follow very simple commands.l Most message queues can communicate over STOMP.
  • /queue/ msg Connect Send Disconnect /queue/ S msg T O Connect Subscribe Disconnect M P P H S P E R V Read E R AckHow It Worksl When you send in a message, you tell it which queue to go to.l When you subscribe you request a queue.
  • Sever Implementations One of the oldest message queues existing; a apache foundation project Next generation ActiveMQ Very popular and common message queue owned by VMware. Supported by Red Hat Middle Ware division, picking up steam.
  • PECL Stompl pecl install stompl That was easy J
  • STOMP Client
  • STOMP Worker
  • XMPPExtensible Messaging and Presence Protocol(Although not really a Message Queue)
  • Overviewl Best for real-time data.l Leveraging pub/sub can turn it into more of a generic message system.l Multiple libraries l JAXL - l Xmpp -
  • Publish to, id, message Subscribe id, from, to, message Sub1 Sub2 Sub3XEP-0060: Publish-Subscribel Specification for implementing Publish Subscribe models.l Extension to the original XMPP specification.
  • Publish
  • Message
  • Various others; including Gearman, ZeroMQ, etc.SPECIAL MESSAGE QUEUES
  • Overviewl There are job servers available that are more flexible or more specific. l Extreme Flexibility l Job Severs l Cloud Messaging