Introduction to ZeroMQ

eSpace TechTalks@modsaid

Outline

● Quick Definition

● Creators

● Why ØMQ?

● What the hell is it?

● Patterns

● Demos

● Should it be C?

● Discussion

Quick Definition

● ØMQ (aka ZeroMQ, 0MQ or ZMQ)

● a high-performance asynchronous messaging library

● aimed at use in scalable distributed or concurrent applications.

● It provides a message queue, but unlike message-oriented middleware, a ØMQ system can run without a dedicated message broker.

● The library is designed to have a familiar socket-style API.

Creators - Martin Sústrik

● Martin Sústrik (@sustrik)

● Expert in the field of messaging middle-ware

● Participated in the creation & implementation reference of AMQP standard.

● Founder of the ØMQ project.

● currently is working on integration of messaging technology with OS & Internet stack

Creators - Pieter Hintjens

● Belgian software developer, writer.

● Past president of the Foundation for a Free Information Infrastructure

● Author of Orielly ZeroMQ

● CEO or iMatix

Why ØMQ?

● Moore's Law means more moving pieces

● Cost of Connection is high

● Needed cheaper, fast and reliable connections

● Physics of Software Development

What the hell is it?

● Intelligent socket library for messaging

● Many kinds of connection patterns

● Multiplatform, multi-language (30+: ruby, C, C++, java, python, …. )

● Fast (8M msg/sec, 30usec latency)

● Small (20K lines of C++ code)

● Distributed

● Open source LGPL (large community)

What the hell is it? (2)

● Socket like API. But better than sockets

● Based on reactor pattern“Event handling pattern for handling service requests delivered concurrently to a service handler by one or more inputs. The service handler then demultiplexes the incoming requests and dispatches them synchronously to the associated request handlers.”

Patterns

● Request – Reply

● Publish-Subscribe

● Push-Pull (pipeline)

● Exclusive Pair

Request-Reply Pattern

Publish-Subscribe Pattern

Pipeline Pattern

Demo

● HelloWorld req-rep

● Weather pub-sub

● Tasks workers (pipeline)

Use Case

● Centralized Logging for rails instances

“Several application servers, consolidated logs”

ØMQ Transports

● Threads in one process (inproc://)

● Processes on one box (ipc://)

● Processes on one network (tcp://)

● Multicast group (pgm://)

ØMQ Routing

● Round-robin (REQ, PUSH, DEALER)

● Multicast (PUB)

● Fair-queuing (REP, SUB, PULL, DEALER)

● Explicit addressing (ROUTER)

● Unicast (PAIR)

ØMQ Benefits

● Start with simple / fast language (Python)

● Move to faster language where needed (C)

● Run on arbitrary platforms (Windows, Android)

● Scale to arbitrary sizes (2 cores, 16 cores...)

● No per-core or per-seat licensing

● Easy to experiment and learn

AMQP vs ØMQ

● Centralized (like SVN)

● Family of messaging protocols used through implementations like RabbitMQ, Apache Qpid..

● Main use cases: transient pubsub distribution and reliable request-reply

● Dozens of pieces doing perhaps a hundred thousand messages per second

● Distributed (like git)

● Messaging library, and tools

● Covers transient pubsub, unreliable request-reply, pipeline, and peer-to-peer.

● Hundreds or thousands of pieces perhaps doing hundreds of millions of messages per second

Why should it have been C?

● C++ was chosen for STL, coding style, destructors, virtual functions

● 5 years old admission: poor choice

● Exception Handling guarantee not failing, but not undefined behavior

● Meant to be infrastructure

● Preventing undefined behavior more important

● More efficient compiled code

● System program

● Recent translation to C, nanomsg (not released yet)

References

● http://zguide.zeromq.org/page:all

● http://en.wikipedia.org/wiki/%C3%98MQ

● http://www.imatix.com/

● http://www.zeromq.org/docs:welcome-from-amqp

● http://www.rabbitmq.com/blog/tag/zeromq/

● http://www.250bpm.com/blog:4 (why it should've been C)

● http://nanomsg.org/ (C endeavor)

● http://www.zeromq.org/docs:labs

● http://www.slideshare.net/pieterh/overview-of-zeromq