World of TanksLinux and Open Source Inside

Maksim Melnikau

I’m

I developer in Wargaming (Belarus, Minsk)I Order of WarI Order of War: ChallengeI World of Tanks

I Linux Mobile hobbyistI OpenmokoI systemdI telepathyI Gentoo

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

World of Tanks

I mmorpg

I fps about tanks

I 15x15 pvp

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

World of Tanks Today

I 800k concurrent users in peak

I 8M messages per second

I 500 servers for game and web

I 60M game portal visits per month

I 5 PB (petabytes) for game installs and updates per month

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Cheaters

I many players want to cheat

I cheaters make other players unhappy

I cheaters xenophobia

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Scaleability

I better product — more users

I more users — more servers

I more servers — bigger synchronization problem

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Single Datacenter Issues

I latency

I availability

I single point of failure

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Big Data

I more users — more data

I more data — bigger disks

I suddenly, new storage solution required

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Rapid Growth

I simple solution — faster time to market

I great success — simple solutions completely unusable

I rewriting everything on-the-fly

I business changes every day

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Nobody Will Help You

I no time to educate people

I no time to wait 3rd party support

I no time to write good proper solutions

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Full Control

I software

I data

I team

I hardware

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Linux and Open Source Software

I ready to use components

I good documentation

I customize software when required

I hire people with required skills

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

World of Tanks Architecture

I game client — thin client, player

I server — world simulation

I cluster — thousands of process working as one server

I step-game world, with very small steps

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Development

I regular Python

I GC disabled

I some parts rewritten on C++

I message-based RPC

I UDP-based reliable internal protocol

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Cluster

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Multi Cluster

I scaleability

I geo distributed

I availability

I independence

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Production

1. 500 servers

2. 8k cpu cores

3. 32 TB RAM

4. Linux

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

MySQL

I database size: 300 GB

I 384 GB RAM

I Percona 5.5 (buffer pool warming — 1GBps)

I 40k selects, 1k inserts, 1k updates per second

I 24 HDD ∗ 600 GB ∗ 0.5 = 6 TB

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Client

1. regular Python

2. HUD - ActionScript, Scaleform

3. 3D graphics - C++

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Web Tasks

I registrations

I news

I docs

I media

I payment form

I receiving payments

I update distribution

I account management

I account profile

I statistics

I ratings

I ...

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

LNAMPMR

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Other

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Keys to Success

I Linux on server

I relaying on Open Source

I fast and easy development

I having full control on everything

I don’t afraid of different software stacks

World of Tanks: Linux and Open Source Inside, Maksim Melnikau

Thank You. Questions

Maksim Melnikaumailto:m_melnikau@wargaming.net

https://plus.google.com/114669104565190507739/

https://twitter.com/max_posedon

http://wargaming.com