Redis to the Rescue (Buzzwords Berlin)
27
Redis to the Rescue? Berlin Buzzwords 2011-06-07
description
Games in social networks offer interesting scaling challenges, as the user base can grow explosively once they become popular. When wooga started work on the facebook game 'Happy Hospital' in early 2010, we set ourselves an ambitious goal: "Let's try to scale it to a million players per day -- with as few servers as possible." This constraint led us to select Redis as the main backend storage, running on dedicated (but cheap, consumer-grade) hardware. As an in-memory database, Redis offers an order-of-magnitude reduction in query latency, but also introduces new challenges in the area of durability. This case study will recount how our backend architecture has evolved over time, highlight the difficulties we ran into, describe how they shaped our final setup ... and how we came up with a new Redis persistence model (diskstore) on the way. Find out how many machines you *really* need for a high-traffic game backend -- even using a 'slow' language like Ruby.
Transcript of Redis to the Rescue (Buzzwords Berlin)
Redis to the Rescue?
Berlin Buzzwords2011-06-07
Who
• Tim Lossen / @tlossen
• backend developer at wooga
• Berlin, Germany
apps.facebook.com/happy-hospital
Goals
• 1 million players per day (“DAU = daily active users”)
Goals
• 1 million players per day (“DAU = daily active users”)
• small production cluster
- less operational overhead
- cheaper
Architecture
Data Model
• user = single Redis hash
- each entity stored in hash field (serialized to JSON)
• custom Ruby mapping layer (“Remodel”)
1220032045 u1 {“level”: 4,“xp”: 241}
u1_pets [“p7”, “p8”]
p7 {“pet_type”: “Cat”}
p8 {“pet_type”: “Dog”}
1234599660 u1 {“level”: 1,“xp”: 22}
u1_pets [“p3”]
... ...
1220032045 u1 {“level”: 4,“xp”: 241}
u1_pets [“p7”, “p8”]
p7 {“pet_type”: “Cat”}
p8 {“pet_type”: “Dog”}
1234599660 u1 {“level”: 1,“xp”: 22}
u1_pets [“p3”]
... ...
Setup A (July 2010)
Setup B (Nov 2010)
Setup B (Dec 2010)
Analysis
• Redis virtual memory not compatible with:
- persistence
- replication
Setup B’ (Dec 2010)
Setup C (Dec 2010)
Setup D (May 2011)
Status
• peak traffic:
- 3000 requests/s
- 35000 Redis operations/s
• average backend response time:
- 10 ms
Goals
• 1 million DAU
• small production cluster
Production Cluster
Text
(1x intel core i7 X980,6 cores, 24 gig ram)
Goals
• 1 million DAU
• small production cluster
Redis Highlights
• high sustained throughput / performance
• very stable
• active & helpful community
Redis Gotchas
• memory consumption
- fragmentation!
• durability
Q & A
redis.iogithub.com/tlossen/remodelwooga.com/jobs
