Developing distributed applications

with Akka and Akka Cluster

Presented by K.Tsykulenko

Introduction

Agenda• What is Akka?• Concurrency paradigms overview.• Actors and actor model.• Live demo #1.• Akka remoting and clustering.• Live demo #2.• CRDTs and Akka Distributed Data.• Live demo #3.• Summary.• Q&A.

What is Akka?

http://akka.io

Akka is a toolkit and runtime for building highly concurrent, distributed, and fault tolerant event-

driven applications on the JVM.

What is Akka?

http://akka.io

Concurrency paradigms

Concurrency paradigms• Shared state and locks• Software Transactional Memory (STM)• Message-Passing Concurrency (Actors)• Dataflow Concurrency • and more…

Software transactional memory

V 1 V 2

V 1 V 12

V 11

V 3

V 2 V 22

successfulwrite

transaction

retriedwrite

transaction

V 1

Time

Dataflow Concurrency

X 1

X12=f(X1,X2)

X123=f(X12,X3)

X 2

X 3

Actors

Actors• Originate in a 1973 paper by Carl Hewitt• Implemented in Erlang• Encapsulate state and behavior• Closer to the definition of OO than classes

Actors

Sender ActorRef

Actor

Dispatcher

hasMessage

Message

Message

Mailbox

has

enqueue dequeue

schedule

send

Actorsuser ! User(“John Doe")

class UserActor extends Actor { def receive = { case User(name) => sender ! s"Hi $name" }}

Actorsval greeting = user ? User(“John Doe")

class UserActor extends Actor { def receive = { case User(name) => sender ! s"Hi $name" }}

Supervision and hierarchy

worker 1

worker 2

worker 3

worker 4

supervisor 1

supervisor 2

user

Building a web crawler1. Fetch a page2. Parse the page to get links3. Check if max crawl depth has been reached

and if yes, finish4. Go to 1 for all parsed links

Building a web crawler

Parser

CrawlMaster

user

Fetcher

pass urls

pass page content

pass parsed urls

Building a web crawler

Routerpass urls

CrawlMaster

initial url

UrlHandler

create UrlHandler

Fetcher Fetcher

Router

Parser Parser

Building a web crawlerclass FetcherActor(val parser: ActorRef) extends Actor with ActorLogging { import context.dispatcher val pipeline: HttpRequest => Future[HttpResponse] = sendReceive

override def receive: Receive = { case Url(link, depth) => pipeline(Get(link)).map(…).pipeTo(parser) }}

Building a web crawlerclass ParserActor extends Actor with ActorLogging with HtmlParser { override def receive: Receive = { case UrlContents(Url(link, depth), resp, requester) => val links = parseHtml(resp) .map(l => if (l.matches("^[\\/#].*")) link + l else l) .filter(l => Try(new Url(l)).isSuccess links.foreach(requester ! Url(_, depth + 1)) }}

Building a web crawlerakka { actor.deployment { /parsers { router = round-robin-pool nr-of-instances = 5 } /fetchers { router = round-robin-pool nr-of-instances = 5 } }}

Live demo #1

Going remote• Everything works using asynchronous

message passing which is good for remoting• Akka-remoting allows working with remote

actors just as if they were in the same JVM• Still need to handle additional issues like

serialization and handling potential networking problems

Akka cluster – pool routing

akka.tcp://localhost:2551/user/crawler

Routees

akka.tcp://localhost:2552/user/crawler

Cluster PoolRouter

localhost:2550

5. Gossip: localhost:2551

is Up

2. Gossip: localhost:2552

is Up

6. Deployroutee

3. Deployroutee

localhost:2551

localhost:2552

1. Joins cluster

4. Joins cluster

/remote/…/(routee)

/remote/…/(routee)

1. Joins cluster

Akka cluster – group routing

akka.tcp://localhost:2551/user/crawler

Routees

akka.tcp://localhost:2552/user/crawler

Cluster GroupRouter

localhost:2550

5. Gossip: localhost:2551

is Up

2. Gossip: localhost:2552

is Up

6. Routesmessages

3. Routesmessages

/user/crawler

localhost:2551

4. Joins cluster

/user/crawler

localhost:2552

Simple crawler clusterClient VM

CrawlClient

Router

Worker VM

CrawlMaster(s)

Worker VM

CrawlMaster(s)

Going remotecluster { seed-nodes = [ "akka.tcp://CrawlerSystem@127.0.0.1:2551", "akka.tcp://CrawlerSystem@127.0.0.1:2552"] auto-down-unreachable-after = 10s role { client.min-nr-of-members = 1 backend.min-nr-of-members = 2 }}

Going remoteactor { deployment { /workerRouter { router = consistent-hashing-group nr-of-instances = 100 routees.paths = ["/user/master"] cluster { enabled = on allow-local-routees = on use-role = backend } } } provider = "akka.cluster.ClusterActorRefProvider“}

Live demo #2

Distributed state

Distributed state

But what if we need to?

CRDTs• Good performance and scalability, the cost is

eventual consistency• Two main classes: operation based and state

based

CmRDTs

CvRDTs

Live demo #3

Summary• Actor model provides a concurrency

paradigm that is easier to reason about than traditional Java concurrency

• Designing actor systems is a lot like OO• You can easily make your actor systems

distributed and have referential transparency… to an extent

• Akka has many useful modules, like Akka distributed data, which allows to manage distributed state

• Try to build your own application and see how it works for you

References• http://akka.io/• http://

www.allthingsdistributed.com/files/amazon-dynamo-sosp2007.pdf

• https://vimeo.com/43903960

Q&A

Thank you!Konstantin Tsykulenko

@Tsykulenko_K