Planning LAMP infrastructure

Presented 2010-05-19 by David Strauss

Designing, Scoping, and Configuring Scalable LAMP Infrastructure

Wed 2010-06-09

About me

Wed 2010-06-09

About me‣ Founded Four Kitchens in 2006 while at UT Austin

Wed 2010-06-09

About me‣ Founded Four Kitchens in 2006 while at UT Austin‣ In 2008, launched Pressflow,

which now powers the largest Drupal sites

Wed 2010-06-09


which now powers the largest Drupal sites‣ Worked with some of the largest sites in the world:

Lifetime Digital, Mansueto Ventures, Wikipedia, The Internet Archive, and The Economist

Wed 2010-06-09




‣ Engineered the LAMP stack, deployment tools, and management tools for Yale University, multiple NBC-Universal properties, and Drupal.org

Wed 2010-06-09





‣ Engineered development workflows for Examiner.com

Wed 2010-06-09





‣ Engineered development workflows for Examiner.com‣ Contributor to Drupal, Bazaar, Ubuntu, BCFG2,

Varnish, and other open-source projects

Wed 2010-06-09

David Strauss

Some assumptions

Wed 2010-06-09

David Strauss

Some assumptions‣ You have more than one web server

Wed 2010-06-09

David Strauss


‣ You have root access

Wed 2010-06-09

David Strauss



‣ You deploy to Linux(though PHP on Windows is more sane than ever)

Wed 2010-06-09

David Strauss




‣ Database and web servers occupy separate boxes

Wed 2010-06-09

David Strauss




‣ Database and web servers occupy separate boxes

‣ Your application behaves more or lesslike Drupal, WordPress, or MediaWiki

Wed 2010-06-09

David Strauss

UnderstandingLoad Distribution

Wed 2010-06-09

David Strauss

Predicting peak trafficTraffic over the day can be highly irregular. To plan for peak loads, design as if all traffic were as heavy as the peak hour of load in a typical month — and then plan for some growth.

Wed 2010-06-09

David Strauss

Analyzing hit distribution

Wed 2010-06-09

David Strauss

100%


Wed 2010-06-09

David Strauss

100%


Static

Content

Wed 2010-06-09

David Strauss

30%

100%


Static

Content

Wed 2010-06-09

David Strauss

30%

100%


Dynamic

Pages

Static

Content

Wed 2010-06-09

David Strauss

30%

100%

70%


Dynamic

Pages

Static

Content

Wed 2010-06-09

David Strauss

30%

100%

70%


Authenticated

Dynamic

Pages

Static

Content

Wed 2010-06-09

David Strauss

30%

100%

70%

20%


Authenticated

Dynamic

Pages

Static

Content

Wed 2010-06-09

David Strauss

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content

Wed 2010-06-09

David Strauss

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content

Wed 2010-06-09

David Strauss

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Wed 2010-06-09

David Strauss

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Wed 2010-06-09

David Strauss

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Web

Crawler

Wed 2010-06-09

David Strauss

10%

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Web

Crawler

Wed 2010-06-09

David Strauss

10%

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Web

CrawlerNo Special Treatm

ent

Wed 2010-06-09

David Strauss

3%

10%

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Web


ent

Wed 2010-06-09

David Strauss

3%

10%

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Web


ent

“Pay Wall” Bypass

Wed 2010-06-09

David Strauss

3%

10%

40%

50%

30%

100%

70%

20%


Anony

mou

s

Authenticated

Dynamic

Pages

Static

Content Human

Web


ent

7%

“Pay Wall” Bypass

Wed 2010-06-09

David Strauss

Throughput vs. Delivery MethodsGreen

(Static)

Yellow(Dynamic, Cacheable)

Red(Dynamic)

Content Delivery Network

Reverse Proxy Cache

PHP + APC + memcached

PHP + APC

PHP (No APC)

●●●●●●●●●● ✖ ✖

●●●●●●●● ●●●●●●● ✖

●●●● ●●● ●●●

●●●● ●● ●●

●●●● ● ●

1

Delivered by Apache without PHP

1

1

1

More dots = More throughput Some actually can do this.2

2

10 req/s

5000 req/s

Wed 2010-06-09

David Strauss

Objective

Deliver hits using the fastest, most scalable

method available

Wed 2010-06-09

David Strauss

Layering: Less Traffic at Each Step

Wed 2010-06-09

David Strauss


Traffic

Wed 2010-06-09

David Strauss


CDN

Traffic

Wed 2010-06-09

David Strauss


CDN

Traffic

Your Datacenter

Wed 2010-06-09

David Strauss


CDN

Traffic

Your Datacenter

DNS Round Robin

Wed 2010-06-09

David Strauss


CDN

Load BalancerTraffic

Your Datacenter

DNS Round Robin

Wed 2010-06-09

David Strauss


CDN

Load Balancer

Reverse Proxy Cache

Traffic

Your Datacenter

DNS Round Robin

Wed 2010-06-09

David Strauss


CDN

Load Balancer

Reverse Proxy Cache

Application Server

Traffic

Your Datacenter

DNS Round Robin

Wed 2010-06-09

David Strauss


CDN

Load Balancer

Reverse Proxy Cache

Application Server

Database

Traffic

Your Datacenter

DNS Round Robin

Wed 2010-06-09

David Strauss

Offload from the master database

Your master database is the single greatest limitation on scalability.

Wed 2010-06-09

David Strauss



Application Server

MasterDatabase

Wed 2010-06-09

David Strauss


Memory Cache


Application Server

MasterDatabase

Wed 2010-06-09

David Strauss


Memory Cache

SlaveDatabase


Application Server

MasterDatabase

Wed 2010-06-09

David Strauss


Search

Memory Cache

SlaveDatabase


Application Server

MasterDatabase

Wed 2010-06-09

David Strauss

Tools to use

Wed 2010-06-09

David Strauss

Tools to use

‣ Apache Solr or Sphinx for search

‣ Solr can be fronted with Varnish or another proxy cache if queries are repetitive.

Wed 2010-06-09

David Strauss

Tools to use



‣ Varnish, nginx, Squid, or Traffic Serverfor reverse proxy caching

Wed 2010-06-09

David Strauss

Tools to use



‣ Varnish, nginx, Squid, or Traffic Serverfor reverse proxy caching

‣ Any third-party service for CDN

Wed 2010-06-09

David Strauss

Do the math‣ All non-CDN traffic travels through your load

balancers and reverse proxy caches. Even traffic passed through to application servers must run through the initial layers.

Wed 2010-06-09

David Strauss



InternalTraffic

Wed 2010-06-09

David Strauss



Load Balancer

InternalTraffic

Wed 2010-06-09

David Strauss



Load Balancer

Reverse Proxy Cache

InternalTraffic

Wed 2010-06-09

David Strauss



Load Balancer

Reverse Proxy Cache

Application Server

InternalTraffic

Wed 2010-06-09

David Strauss



Load Balancer

Reverse Proxy Cache

Application Server

InternalTraffic

What hit rate is each layer getting?How many servers share the load?

Wed 2010-06-09

David Strauss

Get a management/monitoring box

Wed 2010-06-09

David Strauss


Management

Wed 2010-06-09

David Strauss


ManagementApplication

Server

Wed 2010-06-09

David Strauss



Server

Reverse Proxy Cache

Wed 2010-06-09

David Strauss



Server

Reverse Proxy Cache

Database

Wed 2010-06-09

David Strauss



Server

Reverse Proxy Cache

Database

Load Balancer

Wed 2010-06-09

David Strauss



Server

Reverse Proxy Cache

Database

Load Balancer

(maybe even two and have them

specialize or be redundant)

Wed 2010-06-09

David Strauss

Planning + Scoping

Wed 2010-06-09

David Strauss

Infrastructure goals

Wed 2010-06-09

David Strauss


‣ Redundancy: tolerate failure

Wed 2010-06-09

David Strauss



‣ Scalability: engage more users

Wed 2010-06-09

David Strauss




‣ Performance: ensure each user’s experience is fast

Wed 2010-06-09

David Strauss




‣ Performance: ensure each user’s experience is fast

‣ Manageability: stay sane in the process

Wed 2010-06-09

David Strauss

Redundancy

Wed 2010-06-09

David Strauss

Redundancy‣ When one server fails, the website should

be able to recover without taking too long.

Wed 2010-06-09

David Strauss



‣ This requires at least N+1, putting a flooron system requirements even for small sites.

Wed 2010-06-09

David Strauss




‣ How long can your site be down?

Wed 2010-06-09

David Strauss





‣ Automatic versus manual failover

Wed 2010-06-09

David Strauss





‣ Automatic versus manual failover

‣ Warning: over-automation can reduce uptime

Wed 2010-06-09

David Strauss

Performance

Wed 2010-06-09

David Strauss

Performance

‣ Find the “sweet spot” for hardware. This is the best price/performance point.

Wed 2010-06-09

David Strauss

Performance


‣ Avoid overspending on any type of component

Wed 2010-06-09

David Strauss

Performance



‣ Yet, avoid creating bottlenecks

Wed 2010-06-09

David Strauss

Performance




‣ Swapping memory to disk is very dangerous

Wed 2010-06-09

David Strauss

Performance




‣ Swapping memory to disk is very dangerous

‣ Don’t skimp on RAM

Wed 2010-06-09

David Strauss

Relative importance

Processors/Cores Memory Disk Speed

Reverse Proxy Cache

Web Server

Database Server

Monitoring

●● ●●● ●●

●●●●● ●● ●

●●● ●●●● ●●●●

● ● ●

Wed 2010-06-09

David Strauss

All of your servers

Wed 2010-06-09

David Strauss

All of your servers

‣ 64-bit: no excuse to use anything less in 2010

Wed 2010-06-09

David Strauss

All of your servers


‣ RHEL/CentOS and Ubuntu have the broadest adoption for large-scale LAMP

Wed 2010-06-09

David Strauss

All of your servers



‣ But pick one, and stick with it for development, staging, and production

Wed 2010-06-09

David Strauss

All of your servers



‣ But pick one, and stick with it for development, staging, and production

‣ Some disk redundancy: rebuilding a serveris time-consuming unless you’re very automated

Wed 2010-06-09

David Strauss

Reverse proxy caches

Wed 2010-06-09

David Strauss

Reverse proxy caches‣ Varnish and nginx have modern architecture and

broad adoption‣ Sites often front Varnish with nginx

for gzip and/or SSL

Wed 2010-06-09

David Strauss



for gzip and/or SSL‣ Squid and Traffic Server are clunky

but reliable alternatives

Wed 2010-06-09

David Strauss





Save Your Money

CPU

Wed 2010-06-09

David Strauss





Save Your Money

CPU

1 GB base system+ 3 GB for caching+

Memory

Wed 2010-06-09

David Strauss





Save Your Money

CPU


MemorySlow

+ Small+ Redundant

Disk

+Wed 2010-06-09

David Strauss





Save Your Money

CPU


MemorySlow

+ Small+ Redundant

Disk

+= 5000 req/s

Wed 2010-06-09

David Strauss

Web servers

Wed 2010-06-09

David Strauss

Web servers‣ Apache 2.2 + mod_php + memcached

Wed 2010-06-09

David Strauss

Web servers‣ Apache 2.2 + mod_php + memcached‣ FastCGI is a bad idea‣ Memory improvements are redundant w/ Varnish‣ Higher latency + less efficient with APC opcode

Wed 2010-06-09

David Strauss


‣ Check the memory your app takes per process

Wed 2010-06-09

David Strauss


‣ Check the memory your app takes per process‣ Tune MaxClients to around 25 × cores

Wed 2010-06-09

David Strauss



Max out cores

(but prefer fastcores to density)

CPU

Wed 2010-06-09

David Strauss



Max out cores


CPU1 GB base system

+ 1 GB memcached+ 25 × cores × per-

process app memory+

Memory

Wed 2010-06-09

David Strauss



Max out cores


CPU1 GB base system


process app memory+

MemorySlow

+ Small+ Redundant

Disk

+Wed 2010-06-09

David Strauss



Max out cores


CPU1 GB base system


process app memory+

MemorySlow

+ Small+ Redundant

Disk

+= 100 req/s

Wed 2010-06-09

David Strauss

Database servers

Wed 2010-06-09

David Strauss

Database servers‣ Insist on MySQL 5.1+ and InnoDB

Wed 2010-06-09

David Strauss

Database servers‣ Insist on MySQL 5.1+ and InnoDB‣ Consider Percona builds and (eventually) MariaDB

Wed 2010-06-09

David Strauss

Database servers‣ Insist on MySQL 5.1+ and InnoDB‣ Consider Percona builds and (eventually) MariaDB‣ Every Apache process generally needs at least one

connection available, and leave some headroom

Wed 2010-06-09

David Strauss


connection available, and leave some headroom‣ Tune the InnoDB buffer pool to at least half of RAM

Wed 2010-06-09

David Strauss



No more than 8-12

cores

CPU

Wed 2010-06-09

David Strauss



No more than 8-12

cores

CPUAs much as you can

afford (even RAM not used by MySQL caches

disk content)+

Memory

Wed 2010-06-09

David Strauss



No more than 8-12

cores



disk content)+

MemoryFast

+ Large+ Redundant

Disk

+Wed 2010-06-09

David Strauss



No more than 8-12

cores



disk content)+

MemoryFast

+ Large+ Redundant

Disk

+= 3000 queries/s

Wed 2010-06-09

David Strauss

Management server

Wed 2010-06-09

David Strauss

Management server‣ Nagios: service outage monitoring

Wed 2010-06-09

David Strauss

Management server‣ Nagios: service outage monitoring‣ Cacti: trend monitoring

Wed 2010-06-09

David Strauss

Management server‣ Nagios: service outage monitoring‣ Cacti: trend monitoring‣ Hudson: builds, deployment, and automation

Wed 2010-06-09

David Strauss

Management server‣ Nagios: service outage monitoring‣ Cacti: trend monitoring‣ Hudson: builds, deployment, and automation‣ Yum/Apt repo: cluster package distribution

Wed 2010-06-09

David Strauss

Management server‣ Nagios: service outage monitoring‣ Cacti: trend monitoring‣ Hudson: builds, deployment, and automation‣ Yum/Apt repo: cluster package distribution‣ Puppet/BCFG2/Chef: configuration management

Wed 2010-06-09

David Strauss


Save Your Money

CPU

Wed 2010-06-09

David Strauss


Save Your Money

CPU

Save Your Money+Memory

Wed 2010-06-09

David Strauss


Save Your Money

CPU


Slow+ Large

+ Redundant

Disk

+Wed 2010-06-09

David Strauss


Save Your Money

CPU


Slow+ Large

+ Redundant

Disk

+= good enough

Wed 2010-06-09

David Strauss

Assembling the numbers

Wed 2010-06-09

David Strauss

Assembling the numbers‣ Start with an architecture providing redundancy.

‣ Two servers, each running the whole stack

Wed 2010-06-09

David Strauss



‣ Increase the number of proxy caches based on anonymous and search engine traffic.

Wed 2010-06-09

David Strauss




‣ Increase the number of web servers based on authenticated traffic.

Wed 2010-06-09

David Strauss




‣ Increase the number of web servers based on authenticated traffic.

‣ Databases are harder to predict, but large sites should run them on at least two separate boxes with replication.

Wed 2010-06-09

David Strauss

Extreme measuresfor performanceand scalability

Wed 2010-06-09

David Strauss

When caching and search offloading isn’t enough

Wed 2010-06-09

David Strauss

When caching and search offloading isn’t enough‣ Some sites have intense custom page needs

‣ High proportion of authenticated users

‣ Lots of targeted content for anonymous users

Wed 2010-06-09

David Strauss




‣ Too much data to process real-time on an RDBMS

Wed 2010-06-09

David Strauss




‣ Too much data to process real-time on an RDBMS

‣ Data is so volatile that maintaing standard caches outweighs the overhead of regeneration

Wed 2010-06-09

David Strauss

Non-relational/NoSQL tools

Wed 2010-06-09

David Strauss

Non-relational/NoSQL tools‣ Most web applications can run well

on less-than-ACID persistence engines

Wed 2010-06-09

David Strauss


on less-than-ACID persistence engines‣ In some cases, like MongoDB, easier to use than

SQL in addition to being higher performance

Wed 2010-06-09

David Strauss



SQL in addition to being higher performance‣ Interested? You’ve already missed the tutorial.

Wed 2010-06-09

David Strauss




‣ In other cases, like Cassandra, considerably harder to use than SQL but massively scalable

Wed 2010-06-09

David Strauss





‣ Current Erlang-based systems are neat but slow

Wed 2010-06-09

David Strauss





‣ Current Erlang-based systems are neat but slow‣ Many require a special PHP extension,

at least for ideal performance

Wed 2010-06-09

David Strauss

Offline processing

Wed 2010-06-09

David Strauss

Offline processing‣ Gearman

‣ Primarily asynchronous job manager

Wed 2010-06-09

David Strauss



‣ Hadoop

‣ MapReduce framework

Wed 2010-06-09

David Strauss



‣ Hadoop

‣ MapReduce framework

‣ Traditional message queues

‣ ActiveMQ + Stomp is easy from PHP

‣ Allows you to build your own job manager

Wed 2010-06-09

Edge-side includes

Wed 2010-06-09

Edge-side includes

ESI Processor(Varnish, Akamai, other)

Wed 2010-06-09

Edge-side includes<html><body>

<esi:include href=“http://drupal.org/block/views/3” /></body></html>


Wed 2010-06-09

http://drupal.org/block/views/3




<div>My block HTML.

</div>ESI Processor

(Varnish, Akamai, other)

Wed 2010-06-09





<div>My block HTML.

</div>

<html><body>

<div>My block HTML.

</div></body></html>


Wed 2010-06-09



Edge-side includes‣ Blocks of HTML are

integrated into the page at the edge layer.

<html><body>


<div>My block HTML.

</div>

<html><body>

<div>My block HTML.



Wed 2010-06-09





‣ Non-primary page content often occupies >50% of PHP execution time.

<html><body>


<div>My block HTML.

</div>

<html><body>

<div>My block HTML.



Wed 2010-06-09





‣ Non-primary page content often occupies >50% of PHP execution time.

‣ Decouples block and page cache lifetimes

<html><body>


<div>My block HTML.

</div>

<html><body>

<div>My block HTML.



Wed 2010-06-09



David Strauss

HipHop PHP

Wed 2010-06-09

David Strauss

HipHop PHP‣ Compiles PHP to a C++-based binary

‣ Integrated HTTP server

Wed 2010-06-09

David Strauss



‣ Supports a subset of PHP and extensions

Wed 2010-06-09

David Strauss




‣ Requires an organizational commitment to building, testing, and deploying on HipHop

Wed 2010-06-09

David Strauss




‣ Requires an organizational commitment to building, testing, and deploying on HipHop

‣ Scott MacVicar has a presentation on HipHop later today at 16:00.

Wed 2010-06-09

Credits

Cluster Problems

Wed 2010-06-09

David Strauss

Server failure

Wed 2010-06-09

David Strauss

Server failure‣ Load balancers can remove broken or overloaded

application reverse proxy caches.

Wed 2010-06-09

David Strauss


application reverse proxy caches.‣ Reverse proxy caches like Varnish can

automatically use only functional application servers.

Wed 2010-06-09

David Strauss




‣ Memcached clients automatically handle failure.

Wed 2010-06-09

David Strauss




‣ Memcached clients automatically handle failure.‣ Virtual service IP management tools like

heartbeat2 can manage which MySQL servers receive connections to automate failover.

Wed 2010-06-09

David Strauss




‣ Memcached clients automatically handle failure.‣ Virtual service IP management tools like

heartbeat2 can manage which MySQL servers receive connections to automate failover.

‣ Conclusion: Each layer intelligently monitors and uses the servers beneath it.

Wed 2010-06-09

David Strauss

Cluster coherency

Wed 2010-06-09

David Strauss

Cluster coherency‣ Systems that run properly on single boxes may

lose coherency when run on a networked cluster.

Wed 2010-06-09

David Strauss



‣ Some caches, like APC’s object cache, have no ability to handle network-level coherency. (APC’s opcode cache is safe to use on clusters, though.)

Wed 2010-06-09

David Strauss




‣ memcached, if misconfigured, can hash values inconsistently across the cluster, resulting in different servers using different memcached instances for the same keys.

Wed 2010-06-09

David Strauss




‣ memcached, if misconfigured, can hash values inconsistently across the cluster, resulting in different servers using different memcached instances for the same keys.

‣ Session coherency issues can be helped with load balancer affinity or storage in memcached

Wed 2010-06-09

David Strauss

Cache regeneration races

Wed 2010-06-09

David Strauss

Cache regeneration races‣ Downside to network cache coherency:

synched expiration

Wed 2010-06-09

David Strauss


synched expiration

‣ Requires a locking framework (like ZooKeeper)

Wed 2010-06-09

David Strauss


synched expiration


Old Cached Item

Wed 2010-06-09

David Strauss


synched expiration


Old Cached Item

Time

Wed 2010-06-09

David Strauss


synched expiration


Old Cached Item

TimeExpiration

Wed 2010-06-09

David Strauss


synched expiration


Old Cached Item

TimeExpiration

{All servers regenerating the item.

Wed 2010-06-09

David Strauss


synched expiration


Old Cached Item

TimeExpiration

New Cached Item{All servers regenerating the item.

Wed 2010-06-09

David Strauss

Broken replication

Wed 2010-06-09

David Strauss

Broken replication

‣ MySQL slave servers get out of synch, fall further behind

Wed 2010-06-09

David Strauss

Broken replication


‣ No (sane) method of automated recovery

Wed 2010-06-09

David Strauss

Broken replication



‣ Only solvable with good monitoring and recovery procedures

Wed 2010-06-09

David Strauss

Broken replication



‣ Only solvable with good monitoring and recovery procedures

‣ Can automate DB slave blacklisting from use,but requires cluster management tools

Wed 2010-06-09

All content in this presentation, except where noted otherwise, is Creative Commons Attribution-ShareAlike 3.0 licensed and copyright 2009 Four Kitchen Studios, LLC.

Wed 2010-06-09

http://creativecommons.org/licenses/by-sa/3.0/




David Strauss

DrupalCamp StockholmPresentation Ended Here

Wed 2010-06-09

Credits

Managing the Cluster

Wed 2010-06-09

Credits

The problem

Application Server

Application Server

Application Server

Application Server

Application Server

Software and Configuration

Objectives:Fast, atomic deployment and rollbackMinimize single points of failure and contentionRestart servicesIntegrate with version control systems

Wed 2010-06-09

Credits

Manual updates and deployment

Application Server

Application Server

Application Server

Application Server

Application Server

Human Human Human Human Human

Why not: slow deployment,non-atomic/difficult rollbacks

Wed 2010-06-09

Credits

Shared storageApplication Server

Application Server

Application Server

Application Server

Application Server

NFS

Why not: single point of contention and failure

Wed 2010-06-09

Credits

rsync

Application Server

Application Server

Application Server

Application Server

Application Server

Synchronizedwith rsync

Why not: non-atomic, does not manage services

Wed 2010-06-09

Credits

Capistrano

Application Server

Application Server

Application Server

Application Server

Application Server

Deployed withCapistrano

Capistrano provides near-atomic deployment,service restarts, automated rollback, test automation, and version control integration (tagged releases).

Wed 2010-06-09

Credits

Multistage deployment

Application Server

Application Server

Application Server

Application Server

Application Server


Development Integration


Staging


Deploymentscan be staged.

cap staging deploycap production deploy

Wed 2010-06-09

Credits

But your application isn’t the only thing to manage.

Wed 2010-06-09

Credits

Beneath the application

Application Server

Application Server

Application Server

Application Server

Application Server

cfengine and bcfg2 are popularcluster-level system configuration tools.

Reverse Proxy Cache

DatabaseCluster-level configuration

Cluster management applies to package management, updates, and software configuration.

Wed 2010-06-09

Credits

System configuration management‣ Deploys and updates packages, cluster-wide or

selectively.

‣ Manages arbitrary text configuration files

‣ Analyzes inconsistent configurations (and converges them)

‣ Manages device classes (app. servers, database servers, etc.)

‣ Allows confident configuration testing on a staging server.

Wed 2010-06-09

Credits

All on the management box

Management {

Development

Integration

Staging

Deployment Tools

Monitoring

Wed 2010-06-09

Credits

Monitoring

Wed 2010-06-09

David Strauss

Types of monitoringFailure Capacity/Load

Analyzing Downtime

Viewing Failover

Troubleshooting

Notification

Analyzing Trends

Predicting Load

Checking Results of Configuration

and Software Changes

Wed 2010-06-09

Credits

Everyone needs both.

Wed 2010-06-09

David Strauss

What to use

Failure/Uptime Capacity/Load

Nagios

Hyperic

Cacti

Munin

Wed 2010-06-09

David Strauss

Nagios‣ Highly recommended.

‣ Used by Four Kitchens and Tag1 Consulting for client work, Drupal.org, Wikipedia, etc.

‣ Easy to install on CentOS 5 using EPEL packages.

‣ Easy to install nrpe agents to monitor diverse services.

‣ Can notify administrators on failure.

‣ We use this on Drupal.org

Wed 2010-06-09

Credits

Cacti

‣ Highly annoying to set up.

‣ One instance generally collects all statistics.(No “agents” on the systems being monitored.)

‣ Provides flexible graphs that can be customized on demand.

Wed 2010-06-09

Credits

Munin‣ Fairly easy to set up.

‣ One instance generally collects all statistics.(No “agents” on the systems being monitored.)

‣ Provides static graphs that cannot be customized.

Wed 2010-06-09

David Strauss

PressflowMake Drupal sites scale by upgrading corewith a compatible, powerful replacement.

Wed 2010-06-09

David Strauss

Common large-site issues‣ Drupal core requires patching to effectively

support the advanced scalability techniques discussed here.

‣ Patches often conflict and have to be reapplied with each Drupal upgrade.

‣ The original patches are often unmaintained.

‣ Sites stagnate, running old, insecure versions of Drupal core because updating is too difficult.

Wed 2010-06-09

David Strauss

What is Pressflow?‣ Pressflow is a derivative of Drupal core that

integrates the most popular performance and scalability enhancements.

‣ Pressflow is completely compatible with existing Drupal 5 and 6 modules, both standard and custom.

‣ Pressflow installs as a drop-in replacement for standard Drupal.

‣ Pressflow is free as long as the matching version of Drupal is also supported by the community.

Wed 2010-06-09

David Strauss

What are the enhancements?‣ Reverse proxy support

‣ Database replication support

‣ Lower database and session management load

‣ More efficient queries

‣ Testing and optimization by Four Kitchenswith standard high-performance softwareand hardware configuration

‣ Industry-leading scalability supportby Four Kitchens and Tag1 Consulting

Wed 2010-06-09

David Strauss

Four Kitchens + Tag1

‣ Provide the development, support, scalability, and performance services behind Pressflow

‣ Comprise most members of the Drupal.org infrastructure team

‣ Have the most experience scaling Drupal sitesof all sizes and all types

Wed 2010-06-09

David Strauss

Ready to scale?‣ Learn more about Pressflow:

‣ Pick up pamphlets in the lobby

‣ Request Pressflow releases at fourkitchens.com

‣ Get the help you need to make it happen:

‣ Talk to me (David) or Todd here at DrupalCamp

‣ Email [email protected]

Wed 2010-06-09

http://www.fourkitchens.com

http://www.fourkitchens.com

mailto:[email protected]

mailto:[email protected]

Planning LAMP infrastructure

Technology

Transcript of Planning LAMP infrastructure