Hiram Chirino1

Hiram ChirinoConsulting Software EngineerRed HatAug 2013

Red Hat JBoss A-MQ is the Hotness

redhatHiram Chirino2

Presenter: Hiram Chirino Blog:  h%p://hiramchirino.com/blog/

Twitter:  @hiramchirino

GitHub:  h%ps://github.com/chirino

Engineer  at  Red  Hat

Apache  Member  and  Ac?veMQ  PMC  Chair

Apache  Commi%er  on:  Ac?veMQ,  Camel,  Karaf,  ServiceMix,  Felix,  and  Aries

Lead  of  STOMP  1.1  Specifica?onCo-­‐Founder  of  many  other  OS  projects

redhat

Hiram Chirino3

Messaging: The BasicsIt’s simple stuff!

redhatHiram Chirino4

Messaging Basics

• Uses Messaging oriented Middleware (MoM)

• Applications communicate by sending each other messages

• The MoM provides Applications a rich messaging API to control the QoS used to deliver messages

• At least once

• At most once

• Once and only Once

redhatHiram Chirino5

Destinations

App: A App: B

Consumer

Consumer

Consumer

QueueProducer

redhatHiram Chirino6

Point-to-Point: JMS Queues

Consumer

Consumer

Consumer

TopicProducer

redhatHiram Chirino7

Publish/Subscribe: JMS Topics

redhatHiram Chirino8

Selectors

redhatHiram Chirino9

Why use Messaging?

• Robustness  to  change• Time  Independence• Loca?on  Independence• Hide  Latency• Scalability• Event  driven• Simplicity• Configurable  Quality  of  

Service• PlaXorm  and  Language  

Integra?on• Fault  tolerant  

Hiram Chirino10

About JBoss A-MQLets dig in...

redhatHiram Chirino11

Based on Apache ActiveMQ

• Top Level Apache Software Foundation Project

• Wildly popular, high performance, reliable message broker

• Connects to nearly everything

• Native Java, C/C++, .Net,

• AMQP 1.0, MQTT 3.1, STOMP (1.0-1.2) and OpenWire

• STOMP enables Ruby, JS, Perl, Python, PHP, ActionScript …

• Embedded and standalone deployment options

• The key component of JBoss A-MQ!

redhatHiram Chirino

JBoss A-MQ

12

A small-footprint, high-performance, open source messaging platform

redhatHiram Chirino13

Fuse Fabric – Key Features

Fuse  IDE  

FMC

JBoss A-MQCluster

CamelEndpoints JBoss Fuse

Runtime Registry

• Supports  Hybrid  deployments

• Distributed  Configura?on

• Run?me  registry

• Centralized  Management

redhatHiram Chirino14

Deploying JBoss A-MQ

• Standalone  broker

• Highly  Available    message  broker  cluster

• Embedded  –  easy  to  use  an  en?re  broker  in  JUnit  tests  (no  need  to  Mock)

• In  Tomcat,  deployed  as  a  war

• In  JEE  Server  –  either  co-­‐locate  –  or  use  client  with  JCA• Ac?veMQ  distribu?ons  contain  a  rar  for  this  purpose

18

redhatHiram Chirino15

Smart Grids …

redhatHiram Chirino16

Next Generation Air Traffic Control …

NextGen Applications

FTI IP Backbone

En RouteControllers

TerminalControllers

Non-FAA Users(e.g., Airlines, DoDDHS, etc.) FAA

Command Center

SWIM Enterprise Infrastructure

redhatHiram Chirino17

And helping run the biggest science experiment – ever!

redhatHiram Chirino18

Btw - Why is Integration and Messaging Important ?

Head Office

STORE

24 houra

Enterprises Need to know Everything!

24 houra

24 houra

www

Hiram Chirino19

JBoss A-MQ Features every MoM should have

But some don’t…

redhatHiram Chirino20

Destination Wildcards

redhatHiram Chirino21

Composite Destinations

Producerqueue:BEER,WINE

Queue:BEER

Queue:WINE

Consumerqueue:BEER,WINE

redhatHiram Chirino22

Exclusive Consumers

redhatHiram Chirino23

Message Groups• Like  Exclusive  Consumers  –  but  in  parallel

• Guaranteed  ordering  of  related  messages  across  a  Queue

• But  –  load  balancing  of  messages  across  mul?ple  consumers

• All  messages  with  the  same  JMSXGroupID  go  to  the  same  consumer  

• How  you  group  messages  is  down  to  the  applica?on’s  producer

• To  explicitly  close  a  group,  set  the  JMSXGroupSeq  to  -­‐1

redhatHiram Chirino24

Message Groups

Hiram Chirino25

Protocols

redhatHiram Chirino26

Open Standards Supported

AMQP 1.0 MQTT 3.1 STOMP OpenWire

Wire

Pro

toco

lsTr

ansp

orts

Clie

nt A

PIs

TCP/SSL HTTP/HTTPS WS/WSS

JMS CMS NMS

redhatHiram Chirino27

OpenWire

• Advantages:

• Fast  –  op?mized  for  Ac?veMQ

• client  failover

• automa?c  reconnect

• Client  load  balancing

• Flow  control

• Many  advanced  features

Disadvantages:

• Not a recognized standard

• Only Java, C/C++/.Net

http://activemq.apache.org/openwire.html

redhatHiram Chirino28

MQTT http://mqtt.org

Advantages:

•M2M/”Internet  of  Things”  transport

•Proposed  as  an  OASIS  standard

•Extremely  light  weight

•Growing  support  from  vendors  and  OS  products• WebSphereMQ

• Ac?veMQ  +  Apollo

• Mosqui%o

• RabbitMQ

Disadvantages:

• 3.1 does not support Queues

• Advanced features not standard• Flow control• Failover etc.

redhatHiram Chirino29

AMQP – see www.amqp.org

Advantages:

•AMQP  1.0  OASIS  standard

•Proposed  as  an  OASIS  standard

•Commodi?zes  the  Broker

Disadvantages:

•One size doesn’t really fit all

•Currently no plans for IBM or Tibco to adopt it

redhatHiram Chirino30

STOMP – http://stomp.github.com

Advantages:•Easy  to  use  text  based  protocol•Can  use  telnet  as  a  client•Defacto  standard:• Ac?veMQ  +  Apollo• HornetQ• RabbitMQ• PocoMQ• StompServer• OpenMQ• Many  more  …

Disadvantages:

• Not as fast as binary formats

redhatHiram Chirino31

But there’s more: WebSockets STOMP  is  a  natural  wire  protocol  for  WebSockets

And  so  is  MQTT!

Hiram Chirino32

Networks of Brokers

redhatHiram Chirino33

• Link  Brokers  together

• Use  Store  and  Forward  

• Are  uni-­‐direc?onal  by  default

• All  Des?na?ons  are  global  

Network of Brokers

redhatHiram Chirino34

Head Office

Example of a Network of Brokers

STORE

STORE

outlet

shop

24 houra

STORE

shop

BrokerClusters

redhatHiram Chirino35

Apache ActiveMQ – Broker Topologies

Store and Forward

ActiveMQLocal broker

ActiveMQRemote broker

redhatHiram Chirino36

Apache ActiveMQ – Broker Topologies

Bi-directional network

ActiveMQLocal broker

ActiveMQRemote broker

redhatHiram Chirino37

Broker Networks (Store and Forward)

Hiram Chirino38

Embedded Camel

redhatHiram Chirino39

Embedded Camel

Client

Client

Advanced  rou?ng  in  the  Broker  –  reduce  latency  and  improve  performance

Route  between  different  brokers

redhatHiram Chirino40

<camelContext errorHandlerRef="errorHandler" xmlns="http://camel.apache.org/schema/spring">

<route> <from uri="activemq:queue:test.queue"/> <choice> <when> <xpath>$foo = 'bar'</xpath> <to uri="activemq:topic:topic.bar"/> </when> <when> <xpath>$foo = 'cheese'</xpath> <to uri="activemq:topic:topic.cheese"/> </when> <otherwise> <to uri="activemq:topic:topic.all"/> </otherwise> </choice> </route></camelContext>

Example Route

Hiram Chirino41

Message Storage

redhatHiram Chirino42

Message Delivery Mode

Messages  can  be  persisted  –  allowing  for  decoupled  applica?ons

For  Queues,  any  message  delivered  as  PERSISTENT  must  be  stored  on  long  term  storage  before  being  delivered  to  a  consumer

For  Topics,  a  message  delivered  as  PERSISTENT  will  only  be  stored    if  there  exists  a  durable  subscriber

NON-­‐PERSISTENT  messages  may  also  be  stored  on  disk  if  the  memory  limits  of  the  broker  have  been  reached

redhatHiram Chirino43

Message stores supported by JBoss A-MQ

■ SQL  (JDBC)

■ SQL  (JDBC)  with  journal

■ AMQ  Message  Store

■ Kaha

■ KahaDB

■ LevelDB

Slow but popular

Don’t use (deprecated)

The “current” default

The “best yet”

redhatHiram Chirino44

LevelDB Store vs KahaDB

• Fewer  index  entries  per  message  than  KahaDB

• Faster  recovery  when  a  broker  restarts

• LevelDB  index  out-­‐perform  Btree  index  at  sequen?al  access  .

• LevelDB  indexes  support  concurrent  read  access.

• Pauseless  data  log  file  garbage  collec?on  cycles.

• Fewer  IOPS  to  load  stored  messages.

• It  exposes  it's  status  via  JMX  for  monitoring

redhatHiram Chirino45

ActiveMQ: LevelDB Store

Hiram Chirino46

High Availability

redhatHiram Chirino47

• Java• C++• C#

Seamless Client Failover Support in

BrokerCluster

Client

failover://(tcp://host1:61616,tcp://host2:61616)

Polyglot Support

redhatHiram Chirino48

BrokerCluster

BrokerCluster

master

slave

master

slave

Combined HA and Networks

redhatHiram Chirino49

• JDBC  Master/Slave

• Shared  File  System  Master/Slave

• Replicated  LevelDB  Master/Slave

Master Slave Configurations

redhatHiram Chirino50

BrokerSlave

BrokerMaster

Client

BrokerSlave

Database or Shared FS

JDBC or Shared File SystemMaster Slave

Larry’s Removals

redhatHiram Chirino51

• Extreme  reliability  –  but  not  as  fast

• Recommended  if  already  using  an  enterprise  database

• No  restric?on  on  number  of  slaves

• Simple  configura?on

• Configurable  lockKeepAlivePeriod

JDBC Master Slave

redhatHiram Chirino52

• Recommended  if  you  have  a  SAN,  or  DRDB

• No  restric?on  on  number  of  slaves

• Simple  configura?on

• Ensure  file  locking  works  –  and  ?mes  out  –  NFSv4  good!  

Shared File System M/S

redhatHiram Chirino

Local File System Local File SystemLocal File System

53

BrokerSlave

BrokerMaster

BrokerSlave

ZooKeeperCluster

Replicated LevelDBMaster Slave

Client

redhatHiram Chirino

ZooKeeperCluster

Local File System Local File SystemLocal File System

54

BrokerSlave

BrokerMaster

Client

BrokerSlave

Larry’s Removals

Replicated LevelDBMaster Slave

redhatHiram Chirino55

• Will  be  available  in  JBoss  A-­‐MQ  6.1

• Requires  a  HA  ZooKeeper  Cluster

• No  Single  Point  of  Failure

• Dynamic  number  of  slaves

• Simple  configura?on

• Sync  or  Async  Replica?on

Replicated LevelDBMaster Slave

redhatHiram Chirino56

Demo Time!

redhatHiram Chirino57

Useful Resources:

h%p://ac?vemq.apache.org

h%p://camel.apache.org  

h%p://fuse.fusesource.org/fabric/docs/overview.html

h%p://fuse.fusesource.org/mq/docs/mq-­‐fabric.html

h%ps://github.com/fusesource/fuseide

h%p://hawt.io

Messaging and Integration

Developer tooling

Management

Configuration and provisioning

redhatHiram Chirino58

Questions

and Discussion