L. Rajlich Case Study Zynga - Scaling From 100 to 100 Million Users Social Developer Summit

Connecting the World Through GamesConnecting the World Through Games



Scaling From 0 to 100 Million Users

Luke RajlichJune 29, 2010

FarmVille Growth

Design Motiviations

The Platform

Tooling

Team

Agenda

04/08/23| © Zynga. All Rights Reserved.3

Rocketship Growth after launch

1M daily users in 4 days




Largest social game at launch: 5M daily users.

FarmVille Growth


Design Motivations

We make games • Highly transactional workload.

• Large ratio of writes to reads.

• Small amounts of data.

• Highly Variable load


Design Motivations

We move fast• Rapid growth

• Facilitate rapid development of new features

• Simplified systems cost estimation and ramp up


The Platform


LAMP stack

• PHP + APC• Heavy use of memcache• Schema-less design

Cloud

• EC2• Elastic web server clusters• Push-button Provisioning

Tooling

• Monitoring– Nagios, Munin, Collectd

– Custom instrumentation and dashboards

• Configuration– Puppet, yum

– Custom runtime configuration tooling

• Deployment– Hudson


Tooling Example

• Runtime Tool – Dynamic Server Configuration– Web Servers pull config from a central store

– Fast and reliable changes without deploying new code


The Team

• Sometimes overlooked part of scaling

• Cross Train

• Extensive documentation– Incident diary

– Operations Runbook


General Advice

• Asynchronous Processing

• Degrade services– Identify and implement the operational modes for backend

services

• Feature on/off switches

• Double check network timeouts– defaults tend to be way too long

• Instrument performance


connecting the world through gamesconnecting the world through games

L. Rajlich Case Study Zynga - Scaling From 100 to 100 Million Users Social Developer Summit

Technology

Transcript of L. Rajlich Case Study Zynga - Scaling From 100 to 100 Million Users Social Developer Summit