Scala + Akka

+

ning/async-http-client

Or: How I stopped worrying about network

latency and drank the reactive cool-aid

Intro

$ whoami

RSS is dead, long live RSS!

● In constant decline over the years as

consumer-facing tech

● Still very popular between “content services”

● RSS/Atom feeds @ HSo Follow feeds on behalf of users

o Post to social networks based on content

o ~250K active feeds, ~400K messages daily

Scaling webfeeds using PHP

But latency isn’t the problem; blocking is.

Enter Scala + Akka + async HTTP client

Akka: a short detour for the uninitiated

“Toolkit and runtime for building highly

concurrent, distributed, and fault tolerant

applications on the JVM”

● Implementation of the Actor model

● Scala, Java

● Adheres strictly to the Reactive Manifesto:o Responsive, Resilient, Elastic, Message-driven

Akka (continued)

Source: http://www.scottlogic.com/blog/rdoyle/assets/ActorModel.png

Ning’s async-http-client

… in an actor

Actors in Scala webfeed service

So, what did this buy us?

Now

BeforeBefore ● 120 processes

● 6 X m3.xlarge

● 2 processes

● 2 X m3.xlarge

90 webfeed/sec 300 webfeed/sec

CPU utilization & load average - barely breaking a sweat!

Some lessons learned

● You can't ignore back pressureo Pull-based approach

o Akka Streams to the rescue?

● scala.xml.pull.XMLEventReader is not

production ready

● Be weary of the resulting load on the rest of

your infrastructure!

Some lessons learned (continued)

● Actors should only ever do one thing (and do

it well)o Easy to modify the actor topology (and it will

happen!)

o Easier to describe supervisor’s responsibilities

● Let it crash!

Thanks!

http://akka.io/

https://github.com/AsyncHttpClient/async-http-client

We’re hiring!

https://hootsuite.com/about/careers

yanik.berube@hootsuite.com

@yanik_berube