Scale like an ant, distribute the workload - DPC, Amsterdam, 2011

Distribute the workload

Helgi Þormar ÞorbjörnssonDutch PHP Conference, Amsterdam, 19th May 2011

Thursday, 19 May 2011

Who am I?


Helgi


VP of Engineering at Orchestra.io

Helgi



Developer at PEAR

Helgi



Developer at PEAR

From Iceland

Helgi



Developer at PEAR

From Iceland

@h on Twitter

Helgi


Why Distribute?


Why Distribute?

Budget


Why Distribute?

Budget

Efficiency


Why Distribute?

Budget

Efficiency

Perception


Budget


Budget

Spend wisely


Budget

Spend wisely

Commodity servers


Budget

Spend wisely

Commodity servers

Low overhead, high yield


Budget

Spend wisely

Commodity servers

Low overhead, high yield

Cloud Computing (EC2)


Efficiency

10 small servers > 1 big


Venue Security


1000 people can exit quicker through 10 small doors than 1 big


1000 people can exit quicker

through 10 small doors than 1 big


1000 people can exit quicker through 10 small doors than 1 big


Perception


Perception

Defer intensive processes


Perception


Give instant feedback


Perception


Give instant feedback

Users keep on browsing


“It all depends on how we look at things, and not how

they are in themselves.”

- Carl G. Jung


Chapter from Nature


Ant Colonies


Algorithms


Algorithms

Scheduling


Algorithms

Scheduling

Vehicle Routing


Algorithms

Scheduling

Vehicle Routing

Assignment


Algorithms

Scheduling

Vehicle Routing

Assignment

Sets


Algorithms

Scheduling

Vehicle Routing

Assignment

Sets

Other


How do ants fit?


How do ants fit?

Strength in numbers


How do ants fit?

Strength in numbers

Work together


How do ants fit?

Strength in numbers

Work together

Size benefits them


Teamwork

When faced with a problem they will solve the problem as one.


What if they were bigger?


Types of Ants


Types of Ants

Military


Types of Ants

Military

Maids


Types of Ants

Military

Maids

Tunnel diggers


Types of Ants

Military

Maids

Tunnel diggers

Food gatherers


How does this map to my application?


Colony = Application


Colony = Application Ants = Components


Colony = Application Ants = Components

Ants do many different types of work to keep their colony running


Architect for Distribution


Characteristics


Characteristics

Decoupling


Characteristics

Decoupling

Elasticity


Characteristics

Decoupling

Elasticity

High Availability


Characteristics

Decoupling

Elasticity

High Availability

Concurrency


Decoupling


Application

DB API

Cache FE


ApplicationDB API

Cache FE


ApplicationDB API

Cache FE

Cache


ApplicationDB API

Cache FE

Cache

API


ApplicationDB API

Cache FE

Cache

API

API


Elasticity


Cloud Computing


Load Balancing


HA Proxy

Nginx

My Favourite


Monitoring


When do I need more servers?


Needs to be around from the start!


Keep records


Spot trends


Different types


Different types

Hardware Performance


Different types


Software Performance


Different types



Availability


Different types



Availability

Resourcing


Applications


ApplicationsNew Relic



CloudKick



CloudKick

ScoutApp



CloudKick

ScoutApp

Nagios



CloudKick

ScoutApp

Nagios

Cacti



CloudKick

ScoutApp

Nagios

Cacti

Circonus


Automation


Want to sleep easy at night?


Want to sleep easy at night?

Go out partying without worrying about getting a phone call?


Plug into your monitoring


Bringing together Monitoring and Elastic behaviour into one

beautiful whole!


Add some intelligence to add / remove servers as needed based

on current information.


This is why good monitoring is essential or this wouldn’t be

possible


Just make sure it doesn’t turn into...


Skynet!!Thursday, 19 May 2011

High Availability


Get a highly available and resilient setup by following a few

of those recommendations


Remember, even Google has outages


Benefits


Benefits

Easy management


Benefits

Easy management

Ability to stop / start servers quickly


Benefits

Easy management


Responsibilities are separate


Benefits

Easy management



Quickly move to a new cluster


Benefits

Easy management



Quickly move to a new cluster

Reduced risk


What to avoid


Local Sessions


Store sessions in DB / Memcache

Solution


Local Memory


Networked Memcache

Solution


Local Files


Local Uploads


Writing to /tmp


Store on S3 or a networked FS

Solution


Serve up static files from CDNs

Solution


Servers can vanish at any given time


Internal APIs


Application

S3GFS FS

Internal Storage API


Application

MySQLMongo Cache

Internal DB API


SOA


Service Oriented Architecture


Sort of :-)


Eventually Consistent


CAP Therom


Consistency

Availability

Partition Tolerance


Consistency

All nodes see the same data at the same time


Availability

Node failures do not prevent survivors from continuing to

operate


Partition Tolerance

The system continues to operate despite arbitrary message loss


Consistency

Availability

Partition Tolerance


Queue Systems


Good for


Good forImage Processing



Distributed Logs



Distributed Logs

Data Mining



Distributed Logs

Data Mining

Mass Emails



Distributed Logs

Data Mining

Mass Emails

Intensive transformation



Distributed Logs

Data Mining

Mass Emails

Intensive transformation

Search


Common Tools


Common Tools

Gearman


Common Tools

Gearman

Hadoop


Common Tools

Gearman

Hadoop

Zero MQ (0MQ)


Common Tools

Gearman

Hadoop

Zero MQ (0MQ)

RabbitMQ


Common Tools

Gearman

Hadoop

Zero MQ (0MQ)

RabbitMQ

And many others!


Gearman


Your Client Code

Gearman Client API(C, PHP, Perl, MySQL UDF, ...)

Gearman Job Servergearmand

Gearman Worker API(C, PHP, Perl, Python, ...)

Your Worker Code

Your App Gearman


A Story!


Financial Software


3000+ Clients


Each one has 5 external data sources


Each data source is a web service


Ran every 6 hours every day


Cron

1

2

3

4

5

Job 1

Gearman

1

43

2

5

Web Services Processing


But! That wasn’t enough


Job kicked off on login


Another Story!


CloudSplit


Near Real Time Cloud Analytics


Clients install logging agent locally


syslogd


Public API


Multiple Persistent Gearman Servers


Internal DB API


Agent


Agent syslogd


Agent syslogd

API


Agent syslogd

API

Load Balanced


Agent syslogd

API

Gearman

Gearman

Load Balanced


Agent syslogd

API

Gearman

Gearman

Load Balanced

PersistentThursday, 19 May 2011

Agent syslogd

API

Gearman

Gearman

Worker

Worker

Worker

Load Balanced


Agent syslogd

API

Gearman

Gearman

Worker

Worker

Worker

Internal API

Load Balanced


Agent syslogd

API

Gearman

Gearman

Worker

Worker

Worker

Internal API

Load Balanced

Load Balanced


Agent syslogd

API

Gearman

Gearman

CouchDB

Worker

Worker

Worker

Internal API

Load Balanced

Load Balanced


CouchDB Setup


Write vs Read


Writes


Writes

Multi Master setup


Writes

Multi Master setup

Replicated


Writes

Multi Master setup

Replicated

Deals with writes only


Reads


Reads

Multi Master setup


Reads

Multi Master setup

Replicated from write cluster


Reads

Multi Master setup

Replicated from write cluster

Slaves handle website requests


Heavy Map/Reduce usage for data


Supervisord


phpadvent.org/2009/daemonize-your-php-by-sean-coates


http://phpadvent.org/2009/daemonize-your-php-by-sean-coates




Map/Reduce


Map


Map

Master gets a problem to solve


Map


Breaks into multiple sub-problems


Map



Distributed to multiple workers


Map




A worker can take the same steps


Map




A worker can take the same steps

Answer passed back to Master


Reduce


Reduce

Takes in answers from the map workers


Reduce


Combines together to get an answer


Reduce


Combines together to get an answer

There can be multiple reducers


process petabytes of data in few hours on commodity server farm


CouchDB


CouchDB

Highly Concurrent


CouchDB

Highly Concurrent

Schema free, document based


CouchDB

Highly Concurrent


RESTful API


CouchDB

Highly Concurrent


RESTful API

Map/Reduce Views


CouchDB

Highly Concurrent


RESTful API

Map/Reduce Views

Easy Replication


Hadoop


Hadoop is a framework for running applications on large clusters of commodity hardware.


The Hadoop framework transparently provides applications both reliability and data motion


Uses Map/Reduce concept to farm out work


Distributed FS to handled node failure automagically


Join 2 datasets together of a significant size


500 GB worth of log files with a large location dataset


ØMQ


ØMQ

Async Message System


ØMQ


Thin and lightweight


ØMQ



High Performance


ØMQ



High Performance

Simple


ØMQ



High Performance

Simple

Scalable


One socket can load balance to multiple end points


Multiple end points can be funnelled into a single socket


Handle deployments to multiple servers


Scale is an example of that


Mongrel2 is a web server that uses it in a similar way as fastcgi


Move around text (JSON) and Binary data for real time communication


Could have replaced syslogd and the external API in my previous example


Code time? :-)


Questions?

[email protected]: @h

Joind.in: http://joind.in/3212


mailto:[email protected]

mailto:[email protected]

http://joind.in/3212

http://joind.in/3212

Scale like an ant, distribute the workload - DPC, Amsterdam, 2011

Technology

Transcript of Scale like an ant, distribute the workload - DPC, Amsterdam, 2011