@crichardson

NodeJS: the good parts? A skeptic’s view

Chris Richardson

Author of POJOs in ActionFounder of the original CloudFoundry.com

@crichardsonchris@chrisrichardson.net http://plainoldobjects.com

@crichardson

Presentation goal

How a grumpy, gray-haired server-side Java developer discovered an appreciation for NodeJS and JavaScript

@crichardson

VIEWER DISCRETION IS ADVISED

WARNING!

@crichardson

1982 1986

RPG 3 BCPLPascal

C

About Chris

1983

LispWorks

1980 1984 1985 1987 1988 19891981

Z806502Assembler

Basic

@crichardson

C++

EJB

1992 199619931990 1994 1995 1997 1998 19991991

@crichardson

CloudFoundry.com

2002 200620032000 2004 2005 2007 2008 20092001

@crichardson

?About Chris

2012 201620132010 2014 2015 2017 2018 20192011

@crichardson

Agenda

Overview of NodeJS

JavaScript: Warts and all

The Reactor pattern: an event-driven architecture

NodeJS: There is a module for that

Building a front-end server with NodeJS

@crichardson

What’s NodeJS?

Designed for DIRTy apps

@crichardson

Growing rapidly

Busy!

@crichardson

NodeJS Hello Worldapp.js

$ node app.js$ curl http://localhost:1337

http://nodejs.org/

Load a module

request handler

No complex configuration: simple!

@crichardson

NodeJS

JavaScript

Reactor pattern

Modules

@crichardson

NodeJS

JavaScript

Reactor pattern

Modules

@crichardson

Dynamic and weakly-typedDynamic:

Types are associated with values - not variables

Define new program elements at runtime

Weakly typed:

Leave out arguments to methods

Read non-existent object properties

Add new properties by simply setting them

@crichardson

JavaScript is object-oriented> var fred = {name: “Fred”, gender: “Male”};undefined> fred.name“Fred”> console.log("reading age=" + fred.age);reading age=undefinedundefined> fred.age = 99;99> fred{ name: 'Fred', gender: 'Male', age: 99 }> delete fred.agetrue> fred{ name: 'Fred', gender: 'Male' }

Unordered key-value pairs

Keys = properties

Add property

Delete property

@crichardson

overrides

JavaScript is a prototypal language

__proto__name “Chris”

__proto__sayHello function

... ...

inherited

Prototype

Person

Chris

“CER”nicknameobject specific

@crichardson

Prototypal code$ node> var person = { sayHello: function () { console.log("Hello " + this.name); }};[Function]> var chris = Object.create(person, {name: {value: "Chris"}});undefined> var sarah = Object.create(person, {name: {value: "Sarah"}});undefined> chris.sayHello();Hello Chrisundefined> sarah.sayHello();Hello Sarahundefined> chris.sayHello = function () { console.log("Hello mate: " + this.name); };[Function]> chris.sayHello();Hello mate: Chrisundefined

Not defined here

create using prototype properties

@crichardson

JavaScript is Functionalfunction makeGenerator(nextFunction) {

var value = 0;

return function() { var current = value; value = nextFunction(value); return current; };

}

var inc = makeGenerator(function (x) {return x + 1; });

> inc()0> inc()1

Pass function as an argument

Return a function closure

@crichardson

But JavaScript was created in a hurry

The ‘Java...’ name creates expectations that it can’t satisfy

Fake classes: Hides prototypes BUT still seems weird

global namespace

scope of vars is confusingMissing return statement = confusion

‘function’ is really verbose

‘this’ is dynamically scoped

Unexpected implicit conversions: 99 == “99”!

truthy and falsy values52-bit ints

Dynamic + weakly-typed (+ event-driven) code

+ misspelt property names

⇒

lots of time spent in the abyss

Essential: Use IDE integrated with JSLint/JSHint + tests

@crichardson

Prototypal languages have benefits BUT

Developers really like classes

JavaScript prototypes lack the powerful features from the Self language

e.g. Multiple (and dynamic) inheritance

http://www.cs.ucsb.edu/~urs/oocsb/self/papers/papers.html

@crichardson

Verbose function syntax> var numbers = [1,2,3,4,5]> numbers.filter(function (n) { return n % 2 == 0; } ).map(function (n) { return n * n; })[ 4, 16 ]>

scala> val numbers = 1..5scala> numbers filter { _ % 2 == 0} map { n => n * n }Vector(4, 16)

VersusPrelude> let numbers = [1,2,3,4,5]Prelude> map (\n -> n * n) (filter (\n -> mod n 2 == 0) numbers)[4,16]

Or

@crichardson

Verbose DSLsdescribe('SomeEntity', function () {

beforeEach(function () { ... some initialization ... });

it('should do something', function () { ... expect(someExpression).toBe(someValue); });});

class SomeScalaTest ...{

before { ... some initialization ... }

it should "do something" in { ... someExpression should be(someValue)}

Versus

Jasmine

Scalatest

@crichardson

JavaScript is the language of the web

“You have to use the programming language you have, not the one that you

might want”

@crichardson

It works but the result is lost opportunities

and impeded progress

@crichardson

Martin Fowler once said:

"...I'm one of those who despairs that a language with such deep flaws plays such an

important role in computation. Still the consequence of this is that we must take

javascript seriously as a first-class language and concentrate on how to limit the damage

its flaws cause. ...."

http://martinfowler.com/bliki/gotoAarhus2012.html

@crichardson

Use just the good parts

http://www.crockford.com/

Douglas Crockford

@crichardson

Use a language that compiles to JavaScript

TypeScript

Classes and interfaces (dynamic structural typing)

Typed parameters and fields

Dart

Class-based OO

Optional static typing

Bidirectional binding with DOM elements

Less backwards compatibility with JavaScript

Also has it’s own VM

@crichardson

CoffeeScript Hello Worldhttp = require('http')

class HttpRequestHandler constructor: (@message) ->

handle: (req, res) => res.writeHead(200, {'Content-Type': 'text/plain'}) res.end(@message + '\n')

handler = new HttpRequestHandler "Hi There from CoffeeScript"

server = http.createServer(handler.handle)

server.listen(1338, '127.0.0.1')

console.log('Server running at http://127.0.0.1:1338/')

Classes :-)

Bound method

Concise

@crichardson

No escaping JavaScript

@crichardson

NodeJS

JavaScript

Reactor pattern

Modules

@crichardson

About the Reactor pattern

Defined by Doug Schmidt in 1995

Pattern for writing scalable servers

Alternative to thread-per-connection model

Single threaded event loop dispatches events on handles (e.g. sockets, file descriptors) to event handlers

@crichardson

Reactor pattern structure

Event Handlerhandle_event(type)get_handle()

Initiation Dispatcherhandle_events() register_handler(h)

select(handlers)for each h in handlers h.handle_event(type)end loop

handleSynchronous Event

Demultiplexerselect()

owns

notifies

uses

handlers

Applicationregister_handler(h1)register_handler(h2)handle_events()

@crichardson

Benefits

Separation of concerns - event handlers separated from low-level mechanism

More efficient - no thread context switching

Simplified concurrency - single threaded = no possibility of concurrent access to shared state

@crichardson

DrawbacksNon-pre-emptive - handlers must not take a long time

Difficult to understand and debug:

Inverted flow of control

Can’t single step through code easily

Limited stack traces

No stack-based context, e.g. thread locals, exception handlers

How to enforce try {} finally {} behavior?

@crichardson

Application code

NodeJS app = layers of event handlers

NodeJS event loop

Basic networking/file-system/etc.

HTTP DB driver ...

Event listener

Callback function

One time events:async

operation completion

Recurring events from Event

Emitters

@crichardson

Async code = callback hell

Scenarios:

Sequential: A ⇒ B ⇒ C

Scatter/Gather: A and B ⇒ C

Code quickly becomes very messy

@crichardson

Messy callback codegetProductDetails = (productId, callback) -> productId = req.params.productId result = {productId: productId} makeCallbackFor = (key) -> (error, x) -> if error

callback(error) else result[key] = x if (result.productInfo and result.recommendations and result.reviews) callback(undefined, result)

getProductInfo(productId, makeCallbackFor('productInfo')) getRecommendations(productId, makeCallbackFor('recommendations')) getReviews(makeCallbackFor('reviews'))

The result of getProductDetails

Gather

Scatter

Update result

Propagate error

@crichardson

Simplifying code with Promises (a.k.a. Futures)

Functions return a promise - no callback parameter

A promise represents an eventual outcome

Use a library of functions for transforming and composing promises

Promises/A+ specification - http://promises-aplus.github.io/promises-spec

when.js (part of cujo.js by SpringSource) is a popular implementation

Crockford’s RQ library is another option

@crichardson

Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity

responses = [getProductInfo(productId), getRecommendations(productId),

getReviews(productId)]

all(responses).spread(makeProductDetails)all(responses) spread(makeProductDetails)

responses = [getProductInfo(productId), getRecommendations(productId),

getReviews(productId)]

@crichardson

Not bad but lacks Scala’s syntactic sugar

class ProductDetailsService .... {

def getProductDetails(productId: Long) = {

for (((productInfo, recommendations), reviews) <- getProductInfo(productId) zip getRecommendations(productId) zip getReviews(productId)) yield ProductDetails(productInfo, recommendations, reviews) }

}

getProductInfo(productId) zip getRecommendations(productId) zip getReviews(productId)

yield ProductDetails(productInfo, recommendations, reviews)

for (((productInfo, recommendations), reviews) <-

@crichardson

Long running computations

Long running computation ⇒ blocks event loop for other requests

Need to run outside of main event loop

Options:

Community: web workers threads

Built-in: NodeJS child processes

@crichardson

Using child processesvar child = require('child_process').fork('child.js');

function sayHelloToChild() { child.send({hello: "child"});}

setTimeout(sayHelloToChild, 1000);

child.on('message', function(m) { console.log('parent received:', m);});

function kill() { child.kill();}

setTimeout(kill, 2000);

process.on('message', function (m) { console.log("child received message=", m); process.send({ihateyou: "you ruined my life"})});

parent.js

child.js

Create child process

Send message to child

@crichardson

Modern multi-core machines vs. single-threaded runtime

Many components of many applications

Don’t need the scalability of the Reactor pattern

Request-level thread-based parallelism works fine

There are other concurrency options

Actors, Software transactional memory, ...

Go goroutines, Erlang processes, ...

Imposing a single-threaded complexity tax on the entire application is questionable

@crichardson

NodeJS

JavaScript

Reactor pattern

Modules

Core built-in modules

Basic networking

HTTP(S)

Filesystem

Events

Timers

...

@crichardson

Thousands of community developed modules

https://npmjs.org/

web frameworks, SQL/NoSQL database drivers, messaging, utilities...

@crichardson

What’s a module?

One or more JavaScript files

Optional native code:

Compiled during installation

JavaScript != systems programming language

Package.json - metadata including dependencies

exports.sayHello = function () { console.log(“Hello”);}

foo.js

@crichardson

Easy to install

$ npm install package-name --save

@crichardson

Easy to use

var http = require(“http”)var server = http.createServer...

Core module ORPath to file ORmodule in node_modules

Module’s exports

@crichardson

Developing with NodeJS modules

Core modules

Community modules

Your modules

Application code

@crichardson

There is a module for that...

Modules + glue code =

rapid/easy application development

AWESOME!...

@crichardson

... BUT

Variable quality

Multiple incomplete/competing modules, e.g. MySQL drivers without connection pooling!

Often abandoned

No notion of a Maven-style local repository/cache = repeated downloads

...

@crichardson

To summarize

NodeJS

JavaScript

Reactor patternModules

Flawed and misunderstood

Scalable yet costly and

often unnecessary

Rich but variable quality

@crichardson

How will future history view NodeJS?

C++EJB

?

@crichardson

Agenda

Overview of NodeJS

JavaScript: Warts and all

The Reactor pattern: an event-driven architecture

NodeJS: There is a module for that

Building a front-end server with NodeJS

@crichardson

So why care about NodeJS?

Easy to write scalable network services

Easy to push events to the browser

Easy to get (small) stuff done

It has a role to play in modern application architecture

@crichardson

Evolving from a monolithic architecture....

WAR

ReviewService

Product InfoService

RecommendationService

StoreFrontUI

OrderService

@crichardson

... to a micro-service architecture

Store front application

reviews application

recommendations application

product info application

ReviewService

Product InfoService

RecommendationService

StoreFrontUI

OrderService

orders application

@crichardson

Browser

WAR

StoreFrontUI

Model

View Controller

Presentation layer evolution....

HTML / HTTP

+ JavaScript

@crichardson

Browser Web application

RESTfulEndpointsModel

View Controller

...Presentation layer evolution

JSON-REST

HTML 5/JavaScriptIOS/Android clients

Event publisher

Events

Static content

@crichardson

Directly connecting the front-end to the backend

Model

View Controller Product Infoservice

RecommendationService

Reviewservice

REST

REST

AMQP

Model

View Controller

Browser/Native App

Traditional web application

Chatty API

Web unfriendly protocols

@crichardson

NodeJS as an API gatewayBrowser

Model

View Controller

HTML 5 - JavaScript

Product Infoservice

RecommendationService

Reviewservice

REST

REST

AMQP

APIGateway

Native App

Model

View Controller

Single entry point

Optimized Client specific APIs

Protocol translation

RESTproxy

Event publishing

NodeJS

@crichardson

Serving static content with the Express web framework

var express = require('express') , http = require('http') , app = express() , server = http.createServer(app) ;

app.configure(function(){ ... app.use(express.static(__dirname + '/public'));});

server.listen(8081);

From public sub directory

@crichardson

RESTful web services

@crichardson

Proxying to backend serverexpress = require('express')request = require('request')

app = express.createServer()

proxyToBackend = (baseUrl) -> (req, res) -> callback = (error, response, body) -> console.log("error=", error) originRequest = request(baseUrl + req.url, callback) req.pipe(originRequest) originRequest.pipe(res)

app.get('/productinfo/*', proxyToBackend('http://productinfo....'))

app.get('/recommendations/*', proxyToBackend(''http://recommendations...'))

app.get('/reviews/*', proxyToBackend('http://reviews...'))

Returns a request handler that proxies to baseUrl

@crichardson

Implementing coarse-grained mobile API

var express = require('express'), ...;

app.get('/productdetails/:productId', function (req, res) { getProductDetails(req.params. productId).then( function (productDetails) { res.json(productDetails); }});

@crichardson

Delivering events to the browser

@crichardson

Socket.io server-sidevar express = require('express') , http = require('http') , amqp = require(‘amqp’) ....;

server.listen(8081);...var amqpCon = amqp.createConnection(...);

io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(message.data.toString()); socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); });});

Handle socket.io

connection

Subscribe to AMQP queue

Republish as socket.io

event

https://github.com/cer/nodejs-clock

@crichardson

Socket.io - client side

var socket = io.connect(location.hostname);

function ClockModel() { self.ticker = ko.observable(1); socket.on('tick', function (data) { self.ticker(data); });};

ko.applyBindings(new ClockModel());

<html><body>

The event is <span data-bind="text: ticker"></span>

<script src="/socket.io/socket.io.js"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script>

</body></html>

clock.js

Connect to socket.io

Subscribe to tick event

Bind to model

Update model

@crichardson

NodeJS is also great for writing backend micro-services

“Network elements”

Simply ‘route, filter and transform packets’

Have minimal business logic

@crichardson

NodeJS-powered home security

Upload2S3 UploadQueueProcessor

SQS Queue DynamoDBS3

FTP ServerLog file

FTP ServerUpload directory

@crichardson

SummaryJavaScript is a very flawed language

The asynchronous model is often unnecessary; very constraining; and adds complexity

BUT despite those problems

Today, NodeJS is remarkably useful for building network-focussed components

@crichardson

Questions?

@crichardson chris@chrisrichardson.net

http://plainoldobjects.com