Sunday, March 10, 13

From a monolithic Ruby on Rails app

to the JVM

phil calçado@pcalcado

http://philcalcado.com

JDC2013 - CairoSunday, March 10, 13

2011: Phil decides to move to Berlin

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

Growing a product is awesome

Sunday, March 10, 13

http://bit.ly/building-the-next-soundcloud

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

~ 180 million users / month

> 10 hours of audio uploaded every minute

Sunday, March 10, 13

But scaling is hard.

Sunday, March 10, 13

SoundCloud was born on Rails

It allowed us to iterate real fast and keep

code quality up(-ish).Sunday, March 10, 13

Controller

ActiveRecord Model

Database

Rails is simple.

Sunday, March 10, 13

Controller

ActiveRecord Model

Database

Rails is simple.

If you want to write a toy blog engine.

Sunday, March 10, 13

Controller

ActiveRecord Model

Database

In the real world, it gets messy.

Sunday, March 10, 13

Controller

ActiveRecord Model

Database

DelayedJobs

Memcached

EventMachine

sc-caching

sc-logging

lhm

statsy new relic

nokogiri

mysql master/slave adapter

haml

thin

amqp

yajl

In the real world, it gets messy.

Real messy.Sunday, March 10, 13

Scaling Rails is hard(er).

Sunday, March 10, 13

No concurrency? 1) Spawn A LOT of

processes.

Sunday, March 10, 13

No concurrency? 1) Spawn A LOT of

processes.2) Make everything a

background job

Sunday, March 10, 13

No Virtual Machine?Adapt your deployment to > 1 minute application

bootstrap time

Sunday, March 10, 13

No proper Garbage Collection?

Get used to restarting processes. Frequently.

Sunday, March 10, 13

Scaling Rails development is hard.

Sunday, March 10, 13

Idiomatic in Rails:

Sunday, March 10, 13

Idiomatic in Rails:

Just like Java circa 2004

Sunday, March 10, 13

Understanding the Rails mindset:

http://bit.ly/not-a-virtue

Sunday, March 10, 13

Understanding the Rails mindset:

i do not think it means what you think it

meanshttp://bit.ly/not-a-virtue

Sunday, March 10, 13

If you are injecting aggregation or composition you are Doing it Wrong(tm)

Sunday, March 10, 13

Idiomatic in Rails:

Sunday, March 10, 13

Understanding the Rails mindset:

http://bit.ly/trashable

Sunday, March 10, 13

Understanding the Rails mindset:

http://bit.ly/trashable

Just like aop, but even worse

Sunday, March 10, 13

Idiomatic in Rails:

Sunday, March 10, 13

Idiomatic in Rails:

no real complexity or loC reduction

Sunday, March 10, 13

Controller

ActiveRecord Model

Database

Rails is simple.

If you want to write a toy blog engine.

Sunday, March 10, 13

Effectively, controllers have a lot of logic

Controller

ActiveRecord Model

Database

Sunday, March 10, 13

Effectively, controllers have a lot of logic

But so do models!

Controller

ActiveRecord Model

Database

Sunday, March 10, 13

Where do we go from here?

Sunday, March 10, 13

We will not do Java.

Sunday, March 10, 13

We, the Java community, screwed up

Focused on the “Enterprise” while nimbler companies took over

Sunday, March 10, 13

We, the Java community, screwed up

Tried to solve core problems with more complicated tooling

Sunday, March 10, 13

We, the Java community, screwed up

JSF, Portlets, JavaFx, Oracle vs. Android: the joke is on us

Sunday, March 10, 13

But we will do the JVM.

Sunday, March 10, 13

Who cares about Oracle/JCP?

So many modern languages. None are “official”.

Sunday, March 10, 13

Who cares about Oracle/JCP?

It is crazy to use “Enterprise” middleware at scale.

Sunday, March 10, 13

Who cares about Oracle/JCP?

twitter, Tumblr, Foursquare, Netflix, Facebook, typesafe, etc. are the new powerhouses.

Sunday, March 10, 13

Options:1) Non-Rails Ruby2) Scala4) Clojure

Sunday, March 10, 13

Non-Rails Ruby

+ Simple+ Easy to hire engineers+ Lingua-franca inside SC- Most code not thread-safe- Lots of native code- Rails mindset everywhere

Sunday, March 10, 13

Scala

+ “twitter stack” (Finagle, docs)+ Good Java interop+ Static typing+ Good engineering practices starting to emerge- Binary compatibility- Can get complicated

Sunday, March 10, 13

Clojure

+ Easier to get started with+ Favours smaller apps + Good Java interop- Very early days for engineering good practices- 3rd party libs a bit off in quality

Sunday, March 10, 13

Integrating Clojure with Jetty

Sunday, March 10, 13

Async I/O with Scala/Finagle

Sunday, March 10, 13

Let’s try them all.

Sunday, March 10, 13

Enabling

Sunday, March 10, 13

1) Micro-services

- A couple of thousand LOC- Once you grasp the domain, can be rewritten in ~1 week- Reduces risk in trying new things

Sunday, March 10, 13

Legacy Application(Rails)

Database

internet

Sunday, March 10, 13

Stats

MessagingRecommenderSettingsNotifications

Sounds Users Stream

Sunday, March 10, 13

Stats

DB

Messaging

DB

Recommender

DB

Settings

DB

Notifications

DB

Sounds

DB

Users

DB

Stream

DB

Sunday, March 10, 13

Stats

MessagingRecommenderSettingsNotifications

Sounds Users Stream

API

internet

Sunday, March 10, 13

2) Easy to deploy

- Old Rails system has fully automated deployment- From change to production should take minutes- Integrated to CI- Reverting is easy

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

Sunday, March 10, 13

Current status

- ~6 new or extracted systems- 3 Clojure- 1 Scala- 2 JRuby

Sunday, March 10, 13

Current status

Rule of thumb: 1 JVM = 10 thins

Sunday, March 10, 13

main takeaways

Sunday, March 10, 13

Rails, PHP, etc. are a very good choice to start something

Sunday, March 10, 13

You need to know when to migrate away

Sunday, March 10, 13

Build infrastructure to reduce the risk of trying new things

Sunday, March 10, 13

The JVM is still the best multi-language VM out there

Sunday, March 10, 13

There is a lot of JVM-related stuff happening outside

the official channels

Sunday, March 10, 13

Even if you are doing Java, drop the extra weight

Sunday, March 10, 13

Java is the new C

Sunday, March 10, 13

phil calçado

http://philcalcado.com@pcalcado

www.soundcloud.com

Sunday, March 10, 13