Post on 14-Jul-2015
1
GDC 2015 – Amazon Developer Day
Connecting With Your Players Using AWS Analytics
Nate Wiger Principal, Gaming Solutions
@nateware nateware@amazon.com
AWS Gaming Solutions
Projected Mobile App Revenue
0 10000 20000 30000 40000 50000 60000 70000 80000 90000
2011 2012 2013 2014 2015 2016 2017
Ads IAP Paid
Source: Gartner
AWS Gaming Solutions
Analytics at Supercell
“You cannot predict success. You will only find out after releasing it.”
- Ilkka Paananen, CEO, Supercell
AWS Gaming Solutions
Analyze What?
Emotions • Enjoying game • Engaged • Like/dislike new content • Stuck on a level • Bored • Giving up
Behaviors • Hours played day/week • Number of sessions/day • Level progression • Friend invites/referrals • Response to mobile push • Money spent/week
AWS Gaming Solutions
Winning at Free to Play
• Phase 1: Collect Data • Phase 2: Analyze • Phase 3: Profit
AWS Gaming Solutions
Example: Level Progression (One Metric)
0
2
4
6
8
10
L1 L2 L3 L4 L5 L6 L7 L8 L9 L10
Tries / Level
# of Tries
AWS Gaming Solutions
Example: Level Progression (Two Metrics)
0 10 20 30 40 50 60
0
2
4
6
8
10
L1 L2 L3 L4 L5 L6 L7 L8 L9 L10
Tries / Level
% Highest Level # of Tries
AWS Gaming Solutions
Events & Metrics
• Event = Moment in Time – Login/quit – Game start/end – Level up – In-app purchase
• Metrics = What to Measure – KISS – Numbers – Booleans – Strings (Enums)
• Context = Always Include – User – Action – Session (context-dependent) – Timestamp in ISO8601
2014-‐03-‐16T16:28:26
AWS Gaming Solutions
Off The Shelf Analytics
• Easy To Integrate • Pre-Baked Reports • Limited Flexibility • Retention Windows • Data Ownership
AWS Gaming Solutions
Ok, A Real Business Plan
Ingest • S3 PUT • Analytics SDK • Kinesis
Store • S3 • DynamoDB • HDFS
Process • Redshift • EMR (Hadoop) • Spark
Analyze • Tableau • Pentaho • Jaspersoft
AWS Gaming Solutions
• Collect Events on Device • Periodically Store in S3 • Process Data into Redshift • Analyze with GUI Visualization Tool
Start Simple
2015-‐03-‐03,nateware,e4df,login 2015-‐03-‐03,nateware,e4df,gamestart 2015-‐03-‐03,nateware,e4df,gameend 2015-‐03-‐03,nateware,a88c,login 2015-‐03-‐03,nateware,a88c,friendlist 2015-‐03-‐03,nateware,a88c,gamestart
Profit!
AWS Gaming Solutions
Redshift at a Glance
10 GigE (HPC)
Ingestion Backup Restore
SQL Clients/BI Tools
128GB RAM
16TB disk
16 cores
Amazon S3/DynamoDB
JDBC/ODBC
128GB RAM
16TB disk
16 cores Compute Node
128GB RAM
16TB disk
16 cores Compute Node
128GB RAM
16TB disk
16 cores Compute Node
Leader Node
• Leader Node – SQL endpoint – Stores metadata – Coordinates query execution
• Compute Nodes – Columnar table storage – Load, backup, restore via Amazon S3 – Parallel load from Amazon DynamoDB
• Single node version available
AWS Gaming Solutions
Plumbing
① Create S3 bucket ("mygame-analytics-events") ② Request a security token for your mobile app:
http://docs.aws.amazon.com/STS/latest/UsingSTS/Welcome.html
③ Upload data from your users' devices ④ Run a scheduled copy to Redshift ⑤ Setup Tableau to access Redshift ⑥ Go to the Beach
AWS Gaming Solutions
Loading Redshift from S3
copy events from 's3://mygame-‐analytics-‐events' credentials 'aws_access_key_id=<access-‐key-‐id>; aws_secret_access_key=<secret-‐access-‐key>' delimiter=',';
Scheduled Redshift Load using Data Pipeline: http://aws.amazon.com/articles/1143507459230804 Search Slideshare for "BDT303" – re:Invent 2014
AWS Gaming Solutions
• Collect Server Logs • Periodically Send to S3 • Process into Redshift • External Analytics Data Too
More Data Sources
EC2
External Analytics
AWS Gaming Solutions
Logrotate to S3
/var/log/apache2/*.log { sharedscripts postrotate sudo /usr/sbin/apache2ctl graceful s3cmd sync /var/log/*.gz s3://mygame-‐logs/ endscript }
Blog Entry on Log Rotation: http://www.dowdandassociates.com/blog/content/howto-rotate-logs-to-s3/ ELB Access Logs: http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/access-log-collection.html
AWS Gaming Solutions
• Different File Formats • Device vs Apache vs CDN • Cleanup with EMR Job • Output to Clean Bucket • Load into Redshift
Dealing With Messy Data
EC2
AWS Gaming Solutions
Redshift vs Elastic MapReduce
Redshift • Columnar DB • Familiar SQL • Structured Data • Batch Load • Faster to Query • Long-term Storage
Elastic MapReduce • Hadoop • Custom Java / Python • Unstructured Data • Streaming Loop • Scales > PB's • Transient
AWS Gaming Solutions
• Game Servers Uses DynamoDB • Directly Export to Redshift • Or Stage to S3
Integrate With Your Game Database
EC2
DynamoDB
AWS Gaming Solutions
Amazon Mobile Analytics Engagement +
Monetization Active Users Sessions In-app Revenue Lifetime Value (LTV) Retention Post-install Retention Funnel Behavior Custom Events
AWS Gaming Solutions
Integrating Mobile Analytics is Simple Initialize the MobileAnalyticsManager.
For Custom Events activity reports, add events.
Handle OnResume() and OnPause()
private static MobileAnalyticsManager analytics;analytics = MobileAnalyticsManager.getOrCreateInstance( this.getApplicationContext(),"yourCompany.yourAppId",
Regions .US_EAST_1, config, cognitoProvider );
EventClient eventClient = analytics.getEventClient();MobileAnalyticsEvent visitEvent = eventClient.createEvent("level1Complete");eventClient.recordEvent(visitEvent);
analytics.getSessionClient().resumeSession();analytics.getSessionClient().pauseSession();analytics.getEventClient().submitEvents();
AWS Gaming Solutions
Key Business Metrics (with one line of code) 1. Monthly Active Users
(MAU) 2. Daily Active Users
(DAU) 3. New Users, 4. Daily Sessions, 5. Sticky Factor, 6. 1-Day Retention, 7. Avg. Revenue per DAU, 8. Daily Paying Users, 9. Avg. Paying DAU
AWS Gaming Solutions
# of Likes /Shares
Player Abort Rates per Level
# of Plays per Hour
In-app Item Popularity
Track Custom Events
AWS Gaming Solutions
Custom Events public void onLevelComplete(String levelName, String difficulty, double timeToComplete, int playerState) { // Create a Level Complete event with some attributes and metrics AnalyticsEvent levelCompleteEvent = analytics.getEventClient().createEvent("LevelComplete") .withAttribute("LevelName", levelName) .withAttribute("Difficulty", difficulty) .withAttribute("EndState", playerState) .withMetric("TimeToComplete", timeToComplete); //Record the Level Complete event analytics.getEventClient().recordEvent(levelCompleteEvent); }
AWS Gaming Solutions
Amazon Mobile Analytics
Fast Event data is processed in < 60 minutes Affordable Free 100 MM Events per Month $1 per MM Events beyond the free tier Private You own your data We do not use your data, or share with 3rd parties
AWS Gaming Solutions
• Collect Events w/ Mobile Analytics SDK • Auto-Export to S3 • Process Data into Redshift • Analyze with GUI Visualization Tool
Mobile Analytics Auto-Export to S3
AWS Gaming Solutions
• Collect Events w/ Mobile Analytics SDK • Auto-Export Directly to Redshift • Analyze with GUI Visualization Tool
Start Simple
AWS Gaming Solutions
Exporting to Amazon Redshift
172.16.0.0/20
Public Subnet 172.16.0.0/22
172.16.0.0/20 Local
0.0.0.0/0 IGW
Amazon Mobile Analytics
EC2
AWS Gaming Solutions
DAU and MAU in SQL
select date, count(*) over (partition by date_trunc('day', date) order by date) as dau, count(*) over (partition by date_trunc('month', date) order by date) as mau from user_sessions order by date;
AWS Gaming Solutions
Measure Retention: Repeated Plays
create view events_by_user_by_month as select user_id, date_trunc('month', event_date) as month_active, count(*) as total_events from events group by user_id, month_active;
AWS Gaming Solutions
First-Pass Retention – Individual Plays
0 5
10 15 20 25 30 35 40
# Play Sessions / Month
nateware Lazyd0g AK187 3strikes
AWS Gaming Solutions
Cohorts & Cambria
• Enables calculating relative metrics • Group users by a common attribute
– Month game installed – Demographics
• Run analysis by cohort – Join with metrics
• Use Redshift as it's SQL – Example of where SQL is a good fit
AWS Gaming Solutions
Creating Cohorts with Redshift
create view cohort_by_first_event_date as select user_id, date_trunc('month', min(event_date)) as first_month from events group by user_id;
http://snowplowanalytics.com/analytics/customer-analytics/cohort-analysis.html
AWS Gaming Solutions
Retention by Cohort – Join Events
0
5
10
15
20
25
Week 1 Week 2 Week 3 Week 5 Week 6 Week 7
# Sessions / Week
2013-11 2013-12 2014-01 2014-02 2014-03 2014-04
AWS Gaming Solutions
Moar Cohorts
• Define multiple cohorts – By activity, time, demographics – As many as you like
• Change cohort depending on analysis • Join same metrics with different cohorts
– Retention by date – Retention by demographic – Retention by average plays/month quartile
AWS Gaming Solutions
5-9 notifications retain 1.5x better than 10-14 notifications
Forza: Push Notifications vs Retention
AWS Gaming Solutions
Data Collection
• Mobile Devices
• Game Servers
• Ad Networks
• Size of event ~ 1 KB
• 500M+ events/day
• 500G+/day & growing
• JSON format
Source of Data Data Size & Growth
AWS Gaming Solutions
Redshift Schema
• Every game has its own database
• Each game event = table (e.g., battle_fight, iap)
• 40-50 tables per DB schema
• All game titles ~ 1000 tables in DW
AWS Gaming Solutions
{"player_id":"323726381807586881","player_level":169,"device":"iPhone 5","version":"iOS 7.1.2”,"platfrom":"ios","client_build":"440”, "db":”mw_dw_ios","table":"player_login", "uuid":"1414566719-rsl3hvhu7o","time_created":"2014-10-29 00:11:59”} {"fight_time":"2014-10-29 00:11:59","attacker_id":"413142642356365377”, "attacker_clan_size":500,"attacker_level":270, "db":"mw_dw_ios","table":"battle_fight", "uuid":"1414566719-p0oogk0bep","time_created":"2014-10-29 00:11:59"}
• PHP/Python Web Services • Client side analytics SDK • JSON data format
Data Stream
AWS Gaming Solutions
Analytics Store Architecture
S3
S3
Consumer
Game DB
Game Servers
Kinesis
Amazon Redshift
Elastic MapReduce
DSV
JSON
AWS Gaming Solutions
Writing to a Kinesis Stream POST / HTTP/1.1 Host: kinesis.<region>.<domain> x-‐amz-‐Date: <Date> Authorization: AWS4-‐HMAC-‐SHA256 Credential=<Credential>, SignedHeaders=content-‐type;date;host;user-‐agent;x-‐amz-‐date;x-‐amz-‐target;x-‐amzn-‐requestid, Signature=<Signature> User-‐Agent: <UserAgentString> Content-‐Type: application/x-‐amz-‐json-‐1.1 Content-‐Length: <PayloadSizeBytes> Connection: Keep-‐Alive X-‐Amz-‐Target: Kinesis_20131202.PutRecord { "StreamName": "exampleStreamName", "Data": "XzxkYXRhPl8x", "PartitionKey": "partitionKey" }
AWS Gaming Solutions
Amazon S3
Spark SQL JDBC Server
SQL
Dashboard
Redshift
Consumer
EMR
Consumer
Game DB
Game Servers
Kinesis
DynamoDB
DSV
JSON
AWS Gaming Solutions
Saving Money… EC2 Spot Instances You bid your own price for instances
c3.large
$0.105/hour
$0.0161/hour*
m3.2xlarge
$0.560/hour
$0.0641/hour*
On-Demand
Spot
On-Demand
Spot
* Price in US-East @ Feb 26, 2014 – 9:40AM PST
85% off
88% off
AWS Gaming Solutions
Auto Scaling: Let AWS Optimize For You
as-create-launch-config spotlc-5cents --image-id ami-e565ba8c --instance-type m1.small --spot-price “0.05” . . . as-create-auto-scaling-group spotasg --launch-configuration spotlc-5cents --availability-zones “us-east-1a,us-east-1b” --max-size 16 --min-size 1 --desiredcapacity 3
AWS Gaming Solutions
EMR Cluster With Spot
aws emr create-cluster --name "Multiple task instance group cluster" --ami-version 3.2.3 \--ec2-attributes AvailabilityZone=eu-west-1b \--instance-groups InstanceGroupType=MASTER,Name="Master",InstanceCount=1,InstanceType=m3.xlarge \InstanceGroupType=CORE,Name="Core",InstanceCount=4,InstanceType=m3.xlarge \InstanceGroupType=TASK,Name="Task1",InstanceCount=25,InstanceType=m3.xlarge,BidPrice=0.28 \InstanceGroupType=TASK,Name="Task2",InstanceCount=10,InstanceType=r3.xlarge,BidPrice=0.28
AWS Gaming Solutions
• S3 as Data Lake • Redshift for Analysis • Collect Everything • Iterate on Reports
Bring It All Back
EC2
External Analytics
AWS Gaming Solutions
Amazon S3
Spark SQL JDBC Server
SQL
Dashboard
Redshift
Consumer
EMR
Consumer
Game DB
Game Servers
Kinesis
DynamoDB
DSV
JSON