Event all the thingsModernizing apps with Node.js

C.V.● Descendent of hominini tribe● Founder at cloudconnect, Showyou, Alier● Engineering Director at Heroku

Node, Python, Objective-C, Ruby, Java, Tcl

ProblemTraditional Python architecture

Django DataSync

Postgres

polling

APIpolling

polling

front end back end

celery broker

Let’s go real-time!

Node.js!● Evented I/O● Fast● Scalable

Python?● Twisted● Tornado● gevent● Python3 asyncio● ugh!

Perfect for Node

… Not so much

Modernize!

Business Logic

+Node

Django

APIwebsocket /http

Redis

streaming

DataSync Postgres

notify

front end back end

Back end● Salesforce

Streaming API● Postgres

Listen/Notify Lingua Franca

● Redis● JSON

Backend events: PostgresCREATE FUNCTION table1_notify_trigger() RETURNS trigger AS $$ DECLARE BEGIN PERFORM pg_notify('channel1', ‘table1’); RETURN new; END; $$ LANGUAGE plpgsql; CREATE TRIGGER table1_insert_trigger AFTER INSERT ON table1 FOR EACH ROW EXECUTE PROCEDURE table1_notify_trigger();

pg = require(‘pg’);var redis = require(‘redis’).createClient();

pg.connect(db_url, function(err, client) { if (err) { console.log("Db connect error: " + err); } else { client.on('notification', function(msg) { console.log("DATABASE NOTIFY: ", msg.payload); redis.publish(‘db_event’, msg.payload); }); client.query("LISTEN channel1"); } });

Streaming API (CometD based)var nforce = require("nforce");var redis = require(‘redis’).createClient();

var org = nforce.createConnection(...);org.authenticate(...);

var pt_name = "hconnect_" + object_name;

// Create a connection to the Streaming APIvar str = org.stream({ topic: pt_name });

str.on('connect', function(){ console.log('connected to pushtopic');});

str.on('data', function(data) { console.log(‘Streaming API:’, data);

redis.publish(‘streaming_event’, data); });

Real-time UI

Django

http

Node HTTP Proxywebsocket

Redis

Django

http API

polling

API

Single page JS AppBefore

After

something changed!

node-http-proxy

https://github.com/nodejitsu/node-http-proxy

var djangoPort = process.env.DJANGO_PORT;

var proxy = httpProxy.createProxyServer({

target:'http://localhost:' + djangoPort

});

var router = express.Router();

router.use(function(req, res) {

proxy.web(req, res, function(e) {

console.log('ERROR');

console.dir(e);

res.status(500).send();

});

});

ProblemHow does Node know when

something has changed?

API proxy, no business logic

REST + Pub/SubCRUD:POST /api/mappings/<id>GET /api/mappings/<id>PUT /api/mappings/<id>DEL /api/mappings/<id>

pub/sub:SUBSCRIBE /api/mappings/<id>

Browser

Django

HTTP Proxy

Redis

socket.emit(‘subscribe’, ‘/api/mappings/1’)

socket.on(‘subscribe’, f(channel) { redis.subscribe(channel); // /api/mappings/1});

redis.on(‘message’, function(channel, body) { socket.emit(‘set’, {channel:channel, data:body}); });

m = Mapping.objects.get(1)...m.save()

def post_save(model): payload = serialize(model) channels = API.paths(model) for channel in channels: if has_subscribers(channel): redis.publish(channel, payload)

def has_subscribers(channel): return redis.execute_command( 'PUBSUB', 'NUMSUB', channel) > 0

Re-use API endpoints to drive pub/sub updates automatically

● API provides:○ CRUD endpoints○ PUB/SUB endpoints

● Node.js proxies CRUD, handles PUB/SUB directly via websocket○ Without any business logic

Result?

takeaways

Node speaks your protocolKeep your business logic

3-tier architectures are flexible

More layers = more complexity

Meet the team

David Gouldin@dgouldin

Howard Burrows

Marty Alchin@Gulopine

Jeremy West@pixeldonor

Marc Sibson@sibson

Winston@winsdog

Hunter Loftis@hunterloftis

Thank You

scottp@heroku.com @persingerscott

tinyurl.com/eventitall