Jeremy Whitlock (@whitlockjc)

Gluecon 2014

Bringing Node.js to the JVM

Why bring Node.js to the JVM?Use Case: Node.js API for Managing API ProxiesUse Case: Writing MapReduce Jobs With JavaScript/Node.jsDiscuss why Java/Node.js interop makes sense

An introduction to the project enabling Java/Node.js interop

Let’s open source something

What this talk is about

Why do people love/hate Java?

Why do people love/hata JavaScript?

Which language is better?

(insert flame war starter here)...

We know people love/hate Java…

We know people love/hate JavaScript…

We know both languages have their pros/cons…

...

What this talk is not about

Use Cases

A while back, Apigee was looking for a simple, programmable approach for configuring and managing API proxies. We wanted to take an XML-based and provide an easier, programmable interface without losing what makes our API ecosystem what it is today.

The timing around this decision was right when Walmart, LinkedIn, eBay/PayPal and others were developing high performance, scalable REST API libraries/tools on Node.js.

We quickly realized how well-suited Node.js was for what we were trying to accomplish with these API proxies: highly concurrent, asynchronous scalable servers

Where Java interop becomes important in this scenario is that our backend is Java-based and we need some way to deploy/manage these Node.js servers from our existing infrastructure.

Node.js API for Managing API Proxies

On the data team, we were building a data processing system around MapReduce. When we talked to some of the PMs, we decided that we wanted a simpler way to do this...without Java.

While we wanted a simpler approach than Java, we did not want to reinvent the wheel and we wanted to make it easy for those with MapReduce experience to just jump in and be productive.

JavaScript was the language we decided on because it was built into the JVM from Java 6 onward and it was small/simple enough to learn.

While JavaScript was the language we chose to use, without Node.js we faced problems with packaging problems, code reuse, dependencies, etc.

We decided to align with the rest of the company and use Node.js for our data processing support.

Writing MapReduce Jobs With JavaScript/Node.js

The Problem

To summarize the problem: We want to use Node.js in a Java world

Like or not, many of us live in a Java world: Cassandra, Hadoop, Mule, Qpid, ...

For some Java-based tools, there are Node.js clients/ports to make interoperability a non-issue.

But for most of the aforementioned systems, like Hadoop, this is not an option. What do we do?

● Write a pure Node.js module? (Not always possible)● Not use Node.js? (Not ideal)● Not use the Java system (Hadoop, …)? (Not ideal)

The Problem

Node.js Is Bad Ass Rock Star Tech

The Solution

The best of both worlds is to figure out some way to provide interop

● This will allow code reuse between Java and Node.js● This will require the least amount of wheel reinventing● The time it takes to use both is invaluable

● Much less time than porting one tool/library to another language● Much less time than waiting on someone to port it for you

How do we do we provide this interoperability between Java and Node.js?

Thankfully, this is already solved for us and the solution we chose was aproject called Trireme: https://github.com/apigee/trireme

Trireme lets us embed Node.js in the JVM, something required for both of our use cases, and lets us share code between both languages.

The Solution

Node.js Trireme

What is Trireme?

Single-threaded event engine

Non-blocking TCP I/O

Non-blocking UDP datagrams

Timers

Non-Blocking File I/O

“Buffer” object

Module loading system

Utility modules

Third-party components

V8 JavaScript engine

OpenSSL

ZLib

Single-threaded event engine

Non-blocking TCP I/O

Non-blocking UDP datagrams

Timers

Non-Blocking File I/O

“Buffer” object

Module loading system

Utility modules

Third-party components

Rhino JavaScript engine

Java SE

Bouncy Castle (crypto, optional)

Trireme Architecture

One thread per Node.js application

Async I/O handled via NIO within that thread

Additional thread pool for blocking operations

File I/O (especially on Java 6)

DNS lookups

Replace native code from Node.js with Java alternatives

Internal modules such as “tcp_wrap,” etc.

Implement a few popular native modules with Java code

“iconv,” “node_xslt”, eventually others

Trireme lets you embed a Node.js runtime within the JVM. You have full control over its environment.

Trireme allows you to interact with Node.js modules and JavaScript objects from Java.

Trireme allows you to expose Java objects to JavaScript code either by calling JavaScript directly or by exposing Java-based Node.js modules

Trireme allows you to bundle/ship JavaScript Node.js modules (This seems trivial but imagine creating a vetted Node.js environment and shipping it with your runtime)

Don’t believe me? Let’s look at a project that uses Trireme to allow you to write MapReduce jobs using JavaScript/Node.js

What Does Trireme Bring to the Table

Let’s Open Source Something

Lembos provides you with a MapReduce runtime and runner that allows you to write your MapReduce jobs using Node.js.

Jobs run in the JVM with Hadoop driving the process, as if written in Java.

Lembos provides a runner, to be ran via hadoop jar, like all other MapReduce jobs, and a runtime that provides the bridge between the Java-based Hadoop world and the JavaScript-based Node.js world.

The name Lembos is a play on Trireme. (A Trireme is an ancient warship with “three banks of oars”. Since Trireme made this project possible, it seemed fitting to have a similar name. A Lembos is a lightweight warship typically used for piracy...not that I’m planning the same thing.)

Project Homepage: https://github.com/apigee/lembos

Introducing Lembos

Lembos Demonstration

● Trireme: https://github.com/apigee/trireme● Lembos: https://github.com/apigee/lembos

● Anatomy of a Node.js MapReduce Job● Lembos Architecture● Lembos CLI Reference● Lembos Node.js API Reference

● The Slides: http://www.slideshare.net/whitlockjc● Contacting me:

● Twitter: @whitlockjc● GitHub: whitlockjc

Resources

I Made It

Thank you

Gluecon 2014