Switchboard: A Matchmaking System for Multiplayer Mobile Games

Justin Manweiler, Sharad Agarwal, Ming Zhang, Romit Roy Choudhury, Paramvir Bahl

ACM MobiSys 2011

Battling demons and vampires on your lunch break…

2

Breakthrough of Mobile Gaming

Windows Phone 7Top 10+ apps are games

John Carmack (Wolfenstein 3D, Doom, Quake)…“multiplayer in some form is where

the breakthrough, platform-defining things are going to happen in the

mobile space”

iPhone App Store350K applications

20% apps, 80% downloads47% Time on Mobile AppsSpent Gaming

3

Mobile Games: Now and Tomorrow

Increasing Interactivity

Single-playerMobile

(mobile today)

Multiplayer Turn-based

(mobile today)

Multiplayer Fast-action

(mobile soon)

4

Key Challenge

Game Type Latency Threshold

First-person, Racing ≈ 100 ms

Sports, Role-playing ≈ 500 ms

Real-time Strategy ≈ 1000 msChallenge: find groups of peers

than can play well together

Bandwidth is fine: 250 kbps to host 16-player Halo 3 game

Delay bounds are much tighter

5

The Matchmaking Problem

Match to satisfy total delay bounds

End-to-end Latency Threshold

Clients

Conn

ecti

on L

aten

cy

6

Instability in a Static Environment

9:36 9:50 10:04 10:19 10:33 10:48 11:02 11:16 11:31 11:45 12:00150

170

190

210

230

250

270

290

310

Time of Day (AM)

Med

ian

Late

ncy

(ms)

Due to instability,must consider latency distribution

7

End-to-end Latency over 3G

0 100 200 300 400 500 6000

0.2

0.4

0.6

0.8

1

AT&T to AT&T DirectAT&T to AT&T via BingAT&T to AT&T via DukeAT&T to AT&T via UW

RTT (in ms)

Empi

rical

CDF

First-person Shoot. Racing Real-time StrategySports

Peer-to-peer reduces latency and is cost-effective

8

The Matchmaking Problem

Link Performance P2P ScalabilityGrouping

Targeting 3G:play anywhere

Latency not Bandwidthinteractivity is key

Measurement / Predictionat game timescales

9

Requirements for 3G Matchmaking● Latency estimation has to be accurate

Or games will be unplayable / fail

● Grouping has to be fast Or impatient users will give up before a game is initiated

● Matchmaking has to be scalable For game servers For the cellular network For user mobile devices

10

State of the Art● Latency estimation

Pyxida, stable network coordinates; Ledlie et al. [NSDI 07] Vivaldi, distributed latency est.; Dabek et al. [SIGCOMM 04]

● Game matchmaking for wired networks Htrae, game matchmaking in wired networks;

Agarwal et al. [SIGCOMM 09]

● General 3G network performance 3GTest w/ 30K users; Huang et al. [MobiSys 2010] Interactions with applications; Liu et al. [MobiCom 08] Empirical 3G performance; Tan et al. [InfoCom 07] TCP/IP over 3G; Chan & Ramjee [MobiCom 02]

Latency estimation and matchmaking are established for wired networks

11

A “Black Box” for Game Developers

InternetIP network

RNC

SGSN

RNC

SGSN

GGSNGGSN

Link Performance (over time)

End-to-end Performance

“Black Box”

CrowdsourcedMeasurement

12

Latency Similarity by Time

Crowdsourcing 3G over Time

Time

13

Crowdsourcing 3G over Space

Latency Similarity by Distance

14

Can we crowdsource HSDPA 3G?● How does 3G performance vary over time?

How quickly do old measurements “expire”? How many measurements needed to characterize the

latency distribution? …

● How does 3G performance vary over space? Signal strength? Mobility speed? Phones under same cell tower? Same part of the cellular network? …

Details of parameter space left for the paper (our goal is not to identify the exact causes)

15

Methodology● Platform

Windows Mobile and Android phones HSDPA 3G on AT&T and T-Mobile

● Carefully deployed phones Continuous measurements Simultaneous, synchronized traces at multiple sites

● Several locations Princeville, Hawaii Redmond and Seattle, Washington Durham and Raleigh, North Carolina Los Angeles, California

16

Stability over Time (in a Static Environment)

120 140 160 180 200 220 2400

0.2

0.4

0.6

0.8

1Redmond, AT&T, 15m Intervals

RTT (Msec)

Empi

rical

CDF

Black line represents phone 1 (all other lines phone 2)

Similar latencies under the same tower

Performance drifts over longer time periods

Live characterization is necessary and is feasible

17

Stability over Space (at the same time)

0 20 40 60 80 100 120 140 160 180 2000

0.2

0.4

0.6

0.8

1

S-homeLatonaU VillageHerkimerNorthgate1st Ave

RTT difference at 90th percentile (ms)

Empi

rical

CDF

Similarity at the same cell tower

Divergence between nearby towers

Substantialvariation

18

Switchboard Cloud Serviceon MSFT Azure

Switchboard: crowdsourced matchmaking

Game

Phone Client

Network Testing Service

Latency Data

Latency Estimator

Measurement Controller

Grouping Agent

19

Scalability through Reuse…

● Across Time Stable distribution over 15-minute time intervals

● Across Space Phones can share probing tasks equitably for each tower

● Across Games Shared cloud service for any interactive game

20

Client Matchmaking Delay

0 100 200 300 400 500 600 7000

0.2

0.4

0.6

0.8

1

Total 1 client/secTotal 10 clients/s

Time until placed in group (s)

Empi

rical

CDF

10 client arrival/sec1 client arrival/sec

Switchboard clients benefit from deployment at scale

21

Conclusion

● Latency: key challenge for fast-action multiplayer

● 3G latency variability makes prediction hard

● Crowdsourcing enables scalable 3G latency estimation

● Switchboard: crowdsourced matchmaking for 3G

22

QUESTIONS?Thank you!

cs.duke.edu/~jgmjgm@cs.duke.edu

23

Stability over Time (in a Static Environment)

0 50 100 150 200 2500

5

10

15

20

25

30

35

95th90th50th

Interval Size (in minutes)

Mea

n Se

quen

tial D

iffer

ence

(ms)

At timescales longer or shorter than 15 minutes: successive interval pairs have less similarity

24

How Many Measurements Req.?

0 5 10 15 20 25 30 35 40 45 500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

5s15s45s60s90s

RTT difference at 90th percentile (in ms)

Empi

rical

CDF

Reasonably small number of measurements are required per 15-minute interval

25

End-to-End Performance

0 50 100 150 200 250 300 350 4000

0.1

0.2

0.3

0.4

0.5

0.6

0.70.8

0.9

1

Durham FRH to San Antonio FRH

Durham phone to FRH

San Antonio phone to FRH

phone to phone

RTT (ms)

Empi

rical

CDF

End-to-end performance predictable as the sum of edge and Internet latencies

26

ICMP Probes by Client Arrival Rate

0 10 20 30 40 50 600

0.2

0.4

0.6

0.8

1

1 client/s2 clients/s5 clients/s10 clients/s

ICMP Probes per Client

Empi

rical

CDF

More clients = less probing overhead for each

27

Scalability by Client Arrival Rate

0 15 30 45 600

10

20

30

40

50

60

70

80

1 client/s 2 clients/s5 clients/s 10 clients/s

Time (minutes)

Clie

nt-to

-ser

ver T

raffi

c (K

bps)

After the initial warming period,later clients reuse effort by earlier clients

28

Group Size by Client Arrival Rate

2 3 4 5 6 7 8 9 10 110

0.2

0.4

0.6

0.8

1

1 client/s

2 clients/s

5 clients/s

10 clients/s

Size of Client Group

Empi

rical

CDF

Availability of high-quality matches increases with utilization

29

Timescale Statistical Analysis

0.00001 0.0001 0.001 0.01 0.1 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1sig=90 α=0.1

240 minutes120 minutes60 minutes2 minutes15 minutes

KS Test P-Value (inter-interval instability where p < α), (log scale)

Empi

rical

CDF