Spiral 1 DesignJoshua HorwitzMackenzie Sweeney

Architecture Components• Server Communication: AMQP

• Server-side Application: Python Tornado or NodeJS• RabbitMQ as message broker

• Client-Server Communication: SockJS & RESTful API

• Client-side Application: HTML, CSS, JS with jQuery

• Browser Communication: PubSubJS

Spiral 1 Proposal• Design/Prototype client pub/sub environment with PubSubJS• Design/Prototype Map abstraction using CMAPI specification• Either modify Mapstraction or implement our own• Maps to support for spiral 1:

• LeafletJS (at minimum)• Google Maps V3• OpenLayers

• Design/Prototype UI (for maps)• Design/Prototype Overlay Widget• Extremely useful to test interoperability with other widgets• Can incorporate this as an add-on to our widget

• Keep access control in mind, but don’t prioritize

Client Pub/Sub• Implement using global pub/sub library that meets criteria:• JS-native• Widely used• Actively maintained/developed

• Our recommendation is PubSubJS

// Basic pub/sub operationsPubSub.subscribe(‘map.overlay.create’, callback);PubSub.publish(‘map.overlay.create’, payload);

// Can also use jQuery syntax$.pubsub(‘subscribe’, ‘map.overlay.create’, callback);$.pubsub(‘publish’, ‘map.overlay.create’, payload)

Map Abstraction Layer• Goal: 0% code-rewriting, 100% interoperability• Possible starting point is Mapstraction (BSD License)• Fork and extend• Or create our own: take what’s useful and improve

• Implement CMAPI over all supported maps• At a minimum we should support• LeafletJS (uses OpenStreetMaps)• Google Maps V3• OpenLayers

• Should also support multiple data formats• KML

GeoJSON

How Does it Work?• Create a map template interface• For all maps, implement the same interface (think Java)• User selects an underlying map to use (LeafletJS, Google, etc)• Based on user choice, load in appropriate module• Object guaranteed to have defined template• Can subscribe to appropriate channels with known callbacks• Essentially, its polymorphism

• Benefits• No shim required: no unnecessary parsing• One handler, one time• Does not rule out subsequent reloads (swap underlying map)

User Interface• Much of the UI will be provided by the maps provided• Custom widgets (like overlay) should leverage:• HTML5• CSS3 (responsiveness for different clients)• JS/jQuery

• Priorities• Modern feel! • High degree of usability (learn from JC2CUI widget mistakes)• Persistent storage in browser

• Get fancy later• Could create our own Grid widget (industry folks hate JC2CUI’s)• Provide cool analytics on mapped data (Jake/David )

Server Pub/Sub (looking forward)

• Leverage RabbitMQ• Single exchange routes published messages to rooms• All messages for one room end up at one queue• Provides a pure FIFO structure to receive messages in order

• Producers (users):• publish directly to the exchange via SockJS and the RabbitMQ

Web-STOMP plugin• Consumer (web server):• handles the user subscriptions and sends the data to users via

SockJS

Consumer Model• 2-layer consumer model• Web server• Browser (user)

• Walkthrough1. Web server consumes messages from room queues2. Gets list of users in room from DB3. Broadcasts message to all users over SockJS connections4. Browsers (users) then receive messages5. Channel parsed from message; appropriate callback executed

Benefits of Consumer Model• Obvious benefits• Establishes central authority on room state• Update new users who join room• Recover from disconnects

• New features• Collect data on publishing patterns• Save existing session state for quick restore later

• This approach has been proven in the commercial world• It works!• No need to modify current CMAPI channel payloads!• HuffingtonPost Live pumps out over 20 million messages/week

AMQP

• Developed to solve the messaging interoperability problem in the financial industry• Used now by 100s of critical systems

• Open standard• Highly interoperable

• Use-cases• Transient pub/sub distribution• Reliable request-reply transactions

RabbitMQ

• Market leader MOM for AMQP• Most popular choice on EC2

• Bindings for multiple languages• Java, Ruby, Python, C, C#, Erlang, Node, Perl

• Multi-protocol Support• AMP 0-9-1, 0-9 and 0-8• AMQP 1.0• MQTT, XMPP, SMTP, HTTP, STOMP

RabbitMQ

• Built-in RESTful API & Web-STOMP plugin• The creators have web messaging in mind!

• Reliability – can persist data and recover from failures• Built-in clustering and transparent scaling• Self-healing capabilities and extensive tooling/configuration

• Built-in access control techniques

RabbitMQ Web-stomp• A simple bridge exposing STOMP protocol over emulated

HTML5 websockets• Makes it possible to use RabbitMQ from web browsers• Comes with RabbitMQ• Used in industry (see previous huffington post example)• They handle over 20 million messages/week with web-stomp

• EXTREMELY SIMPLE to use

Web-STOMP Code// Stomp.js boilerplatevar ws = new SockJS('http://' + window.location.hostname + ':15674/stomp');var client = Stomp.over(ws);

send = function(data) { client.send('/topic/bunny', {}, data);};

var on_connect = function(x) { id = client.subscribe('/topic/bunny', function(d) {

// do some stuff });};

var on_error = function() { console.log('error'); };

client.connect('guest', 'guest', on_connect, on_error, /');

So what is this SockJS?• A ws polyfill that provides a ws-like JS object in ANY browser• Automatic fallback if the channel can’t support websockets• SocketIO does not do this

• Also works in browsers behind ws-unfriendly proxies• If it doesn’t work, your network broke the Internet• Well documented scaling and load balancing techniques• Can utilize sticky sessions (JSESSIONID) or prefix-based LB

• Used in critical industry systems

Server Application• Must support SockJS• Our recommendation is Python/Tornado• 2nd option recommendation is NodeJS/Express

• The approach we are outlining has been proven in the commercial industry as a viable solution• http://www.erlang-factory.com/upload/presentations/791/

HuffingtonPost.pdf• Extremely easy to implement• simple prototype has been developed to proof feasibility