Re-design with Elixir/OTP2016 - 2017ImgOut / On the fly thumbnail generator microservice using Elixir/OTP.

by Mustafa Turanhttps://github.com/mustafaturan/imgout

SummaryOn the fly thumbnail generator microservice, a web microservice:

- Basic Approach without Elixir/OTP- Anatomy of an Elixir Process (optional)- Fault Tolerance Systems with Supervision Tree Strategies (optional)- Approach 1 / with Supervision Tree + GenServer- Approach 2 / with Supervision Tree + GenServer + Partial Pooling- Approach 3 / with Supervision Tree + GenServer + Pooling- Metrics- Source Codes- Questions

Basic Approach without Elixir/OTP *

* Web Based Approach without OTP Benefits

Building BlocksThumbnail Microservice For All Languages:

Inputs Outputs

Image identifier (ID)Dimensions (width*, height*)

Thumbnail (Binary)Content type (String)

Tools and Requirements Sample Elixir Specific Tools

Web ServerImage conversion libraryCacheStorage or Image Source(remote/local)Metrics

CowboyExmagick(Graphmagick NIF package)Cachex, MemcacheClientHttpPoison (Web Client)Metrex (Metrics)

General Internal Process Flow

server cache(r)

storage

thumb

cache (w)

Defining Modular & Functional Needs defmodule ImgOut.CacheInterface do

@callback read(charlist) :: {:ok, binary} | charlist

@callback write(charlist, binary) :: {:ok, binary}

end

defmodule ImgOut.StorageInterface do

@callback read({:ok, binary})

@callback read({:error, integer, map})

@callback read(charlist)

end

defmodule ImgOut.ImageInterface do

@callback thumb({:error, integer, map}, any)

@callback thumb({:ok, binary}, map)

end

defmodule MicroserviceController do

def generate_thumb(id, %{} = dimensions) do

id

|> Cache.read

|> Storage.read

|> Image.thumb(dimensions)

end

end

defmodule AlternativeApproachController do

def generate_thumb(id, %{} = dimensions) do

id

|> Storage.read

|> Image.thumb(dimensions)

end

end

Implement CacheService using CacheInterfacedefmodule ImgOut.CacheInterface do

@callback read(charlist) :: {:ok, binary} | charlist

@callback write(charlist, binary) :: {:ok, binary}

end

defmodule ImgOut.CacheService do

@behaviour ImgOut.CacheInterface

def read(key) do

response = Memcache.Client.get(key)

case response.status do

:ok -> {:ok, response.value}

_ -> key

end

end

def write(key, val) do

Memcache.Client.set(key, val)

{:ok, val}

end

end

Sample Directory Structureapp

- lib- interfaces

- cache_interface.ex- ...

- services- cache_service.ex- …

- imgout.ex

General Internal Process Flow

server cache(r)

storage

thumb

cache (w)

How to Make Cache.write Async?

server cache(r)

storage

thumb

cache (w)

Task.start(..)

defmodule ImgOut.CacheService do

...

# way 1: to make async write to cache

def write(key, val) do

Task.start(fn -> Memcache.Client.set(key, val) end)

{:ok, val}

end

end

Anatomy of an Elixir ProcessWhat is an Erlang/Elixir Process?

An actor (Elixir/Erlang Process)

STATE

Mailbox

CALCULATION FUNCTIONS(MSG LISTENERS)

@mustafaturan

Erlang Virtual Machine

@mustafaturan

OSProcess (1)

Process (2)

Process (3)

….

Process (n)

Erlang VM -- pid 109 -- pid 206 -- pid 3114 -- ...

STATE

Mailbox

CALCULATION FUNCTIONS(MSG LISTENERS)

(pid 109)

Fault Tolerance SystemsErlang/Elixir Supervision Tree Strategies

Supervision Tree Strategies:one_for_one

S

WW W

S

WW W

S

WW W

if one worker dies respawn it

down signal restart child

@mustafaturan

Supervision Tree Strategies:one_for_all

@mustafaturan

S

WW W

S

WW W

S

WW W

if one worker diessupervisor

respawns all

down signal restart all

S

WW

supervisorkills rest of the workers

Supervision Tree Strategies:rest_for_one

@mustafaturan

S

W2

W3

W1

S

W2

W3

W1

S

W2

W3

W1

if one worker diessupervisor

respawns all killed

down signal restart all killed workers

supervisorkills rest of the workers

with start order (not W1)

Note: Assumed start order of workers are like W1, W2, W3 and W4

W4

W4

W4

S

W3

W1

W4W4

Supervision Tree Strategies:simple_one_for_oneSame as :one_for_one

- Needs to implement Supervision.Spec- You need to specify only one entry for a child

- Every child spawned by this strategy is same kind of process, can not be mixed.

With Named GenServer & Supervision Tree

* Creating Elixir Processes with GenServer** and Supervising** Process discovery with ‘name’ option

Building Our Supervision TreeA

(S)

Srv(S)

Im(S)

Str(S)

Ch(S)

Str(w)

Im(w)

Srv(w)

Ch(w)

defmodule ImgOut do

use Application

def start(_type, _args) do

import Supervisor.Spec, warn: false

children = [

supervisor(ImgOut.WebServerSupervisor, []),

supervisor(ImgOut.ImageSupervisor, []),

supervisor(ImgOut.StorageSupervisor, [])

]

opts = [strategy: :one_for_one, name:

ImgOut.Supervisor]

Supervisor.start_link(children, opts)

end

end

Sample Code For CacheSupervisor ...

@doc false

def init([]) do

children = [

worker(ImgOut.CacheWorker, [])

]

opts = [strategy: :one_for_one, name: __MODULE__]

supervise(children, opts)

end

end

defmodule ImgOut.CacheSupervisor do

use Supervisor

@doc false

def start_link,

do: Supervisor.start_link(__MODULE__, [], name:

__MODULE__)

...

Sample Code For CacheWorker...

def init(_opts),

do: {:ok, []}

## Private API

@doc false

def handle_call({:read, key}, _from, state),

do: {:reply, ImgOut.CacheService.read(key), state}

def handle_cast({:write, key, val}, state) do

ImgOut.CacheService.write(key, val)

{:noreply, state}

end

end

defmodule ImgOut.CacheWorker do

use GenServer

## Public API

def read(key),

do: GenServer.call(__MODULE__, {:read, key})

def write(key, val) do

GenServer.cast(__MODULE__, {:write, key, val})

{:ok, val}

end

def start_link,

do: GenServer.start_link(__MODULE__, [], name:

__MODULE__)

...

Sample Directory Structureapp

- lib- interfaces

- cache_interface.ex- ...

- services- cache_service.ex- ...

- supervisors- cache_supervisor.ex- ...

- workers- cache_worker.ex- ...

How is the Flow

server cache(r)

storage

thumb

cache (w)

Problem: Long Running Processes

serverstorage

+cache

thumbtimeouts

Thumbnail generation

- A calculation- Takes too much time to process

Solution: Spawning More Workers On Demand…

def start_link,

do: GenServer.start_link(__MODULE__, [], name: __MODULE__)

…

def thumb({:ok, img}, dimensions),

do: GenServer.call(__MODULE__, {:thumb, {:ok, img}, dimensions})

…

def handle_call({:thumb, {:ok, img}, dimensions}, _from, state) do

data = ImgOut.ImageService.thumb({:ok, img}, dimensions)

{:reply, data, state}

end

…

Cons:Spawn 1 Process Per GenServer

Pros:No need to store pid to call process funcs

Solution: Spawning More Workers On DemandPossible Solution to spawn more workers on GenServer:

- Process Registry- We can save pid on creation and deregister on exit- Too manual work- Reinventing wheels(special for this service)

- Gproc Package- Supports pooling- Has advanced techniques to register, deregister, monitor etc...

- Poolboy Package- Supports pooling- Easy

With Partial Pooled GenServer & Supervision Tree

* Creating Elixir Processes with GenServer** and Supervising** Process discovery with worker pool option (poolboy)

Poolboy Configuration For ImageWorker# prod.exs

config :imgout,

gm_pool_size: (System.get_env("GM_POOL_SIZE") || "25")

|> String.to_integer,

gm_pool_max_overflow:

(System.get_env("GM_POOL_MAX_OVERFLOW") || "0") |>

String.to_integer,

gm_timeout: (System.get_env("GM_TIMEOUT") || "5000") |>

String.to_integer

# dev.exs, test.exs

config :imgout,

gm_pool_size: 25,

gm_pool_max_overflow: 0,

gm_timeout: 5000

Sample Code For ImageSupervisor with Poolboy def init([]) do

worker_pool_options = [

name: {:local, :gm_worker_pool},

worker_module: ImgOut.ImageWorker,

size: @pool_size,

max_overflow: @pool_max_overflow

]

children = [

:poolboy.child_spec(:gm_worker_pool,

worker_pool_options, [])

]

opts = [strategy: :one_for_one, name: __MODULE__]

supervise(children, opts)

end

end

defmodule ImgOut.ImageSupervisor do

use Supervisor

@pool_size Application.get_env(:imgout, :gm_pool_size)

@pool_max_overflow Application.get_env(:imgout,

:gm_pool_max_overflow)

def start_link,

do: Supervisor.start_link(__MODULE__, [], name:

__MODULE__)

…

Sample Code For ImageWorker with Poolboy def start_link(_opts),

do: GenServer.start_link(__MODULE__, :ok, [])

@doc false

def init(_opts) do

{:ok, []}

end

def handle_call({:thumb, {:ok, img}, dimensions},

_from, state) do

data = ImgOut.ImageService.thumb({:ok, img},

dimensions)

{:reply, data, state}

end

end

defmodule ImgOut.ImageWorker do

use GenServer

@behaviour ImgOut.ImageInterface

@timeout Application.get_env(:imgout, :gm_timeout)

def thumb({:ok, img}, dimensions) do

:poolboy.transaction(:gm_worker_pool, fn(worker) ->

GenServer.call(worker, {:thumb, {:ok, img},

dimensions}, @timeout)

end)

end

def thumb({:error, status, reason}, _),

do: {:error, status, reason}

…

Solution: Spawned (n)Thumb Process

serverstorage

+cache

thumbno timeouts

Spawned Multiple ImageWorker(s) with Poolboy

- Process Registry handled by poolboy- We can change max, min spawned processes

thumb

thumb

thumb

Problem: Too many request to storage and cache worker

serverstorage

+cache

thumbno timeouts

Since we solved the timeout problem for Thumbnail processor

- Now storage and cache worker getting too many request- but not processing that fast with 1 instance!

thumb

thumb

thumb

timeouts

Solution: Spawning More Workers On DemandPossible Solution to spawn more workers on GenServer:

- Process Registry- We can save pid on creation and deregister on exit- Too manual work- Reinventing wheels(special for this service)

- Gproc Package- Supports pooling- Has advanced techniques to register, deregister, monitor etc...

- Poolboy Package- Supports pooling- Easy

With Fully Pooled GenServer & Supervision Tree

* Creating Elixir Processes with GenServer** and Supervising** Process discovery with worker pool option (poolboy)

Why Prefer Pooling Against Free Spawning?Memcache

- Connection limit- If you hit connection limit, you can’t get positive response

Storage

- Remote storage servers might have some limitation on requests- Req/per second- If you hit more, you will get error from remote

MetricsA very useful for microservices and system health checks,

determining bottlenecks

Tracking Heartbeats of Your Elixir ProcessesMetrex Package

- Creating new metric dynamically- Incrementing metric- TTL- Dumping metric data- Init and exit hooks

Results:

- 3260 req/min on Heroku Free Dyno- http://bit.ly/2bYRnpp

config :metrex,

counter: ["all"],

meters: ["cache", "storage", "thumb"],

ttl: 900

# Dynamically create a metric

Metrex.start_meter("pageviews")

# Increment a meter by 1

Metrex.Meter.increment("pageviews")

# Increment a meter by x(number)

Metrex.Meter.increment("pageviews", 5)

Tips & Source Code

LibrariesMetrex

- Has implementation for counter and meter patterns

Exmagick

- NIF package which means you can spawn graphicsmagick as Elixir Process

Poolboy

Cowboy

Plug

SourceGithub:

https://github.com/mustafaturan/imgout (Heroku Deploy Button available / Has 3 branches for all 3 approaches)

https://github.com/mustafaturan/metrex

QUESTIONS

THANK YOU