Powering a Player-First Culture with Massive Gameplay Data
A Sneak Peek into Data and Electronic Arts
Navid Aghdaie, PhDSr. Director of Data Science & EngineeringSep 2015
About Me
UCLAComputer Science PhD
Distributed/Fault-Tolerant Systems
Comparison Shopping Startup
Ask.comSearch Engine Core
Web/News Search Components
VP Data Systems
Electronic ArtsDigital Platform, Data
Science & Engineering
New Large Scale Data Platform
Unlock Value of EA’s Rich Gameplay Data
Outline
3
• EA and Games Why Data Matters
• Large Scale Data Platform Design and Architecture for Gamer & GamePlay Data
• Data in Action Examples of Data Usage
EA Overview
• Rich history of games, founded 1982Current Strategic Goals:
• Digital Transformation• Player First Culture
• Dozens of games, multiple platforms: console, pc, mobile• Sports: FIFA, Madden, NHL, NBA• DICE: Battlefield, StarWars Battlefront• Bioware: Dragon Age, Mass Effect• Maxis: The Sims Franchise (Sims4), SimCity• Need for Speed, Bejeweled, Plants vs. Zombies, Simpsons Tapped Out, etc…
• 10s M players/day, across the world4
Data Usage at EA (Gameplay Data)
Game Design and Development• Game updates, new features, new games
Live Services• Game operations• Gameplay optimization• Fraud
Marketing• Player acquisition, re-engagement• Cross Promotions• Advertisement
Customer Service• Player Facing Issues with Game
Executive Decisions5
Advert Push NoteEmail Personalizedfeatures
In-game Banner
Acquisition
CE
Customer Experience
Example Player Journey through EA Ecosystem
Digital Platform: Data Science & Engineering
7
Core Tech Principles
Leverage Open-Source• Join the community and ride its progress – requires investment in talent
Embrace the Benefits of the Cloud• Downward price trend• Lowers risk of volume/game success mispredictions• Build and spend only as needed• Avoid vendor lock-in
Build with Scalability, Extensibility, Reliability from the Start• One platform for all EA games• Standards with flexibility to support variations of use
Invest in “Crown Jewel IP” Data Components• Data Science, Algorithms, Data Layer Tools• Smart build vs buy decisions
8
Data Sources
Access & Applications
Storage & Processing
Reporting & BI Tools
Game Analytics
Subscription API
GameServers
Marketing, Ads, …
External Sources
And More…
Access Layer
Player 360
Segmentation Manager
Engagement Manager
ExperimentationApplications
Lightning (Streaming Ingestion & Processing) Tide (Batch Ingestion)
Capture &Ingestion
AccessAPI
Live Viewer
Bug Sentry
River(Capture layer)
Shark(Processing)
Ocean(Hadoop storage)
Pearl (RDBMS)
Black Pearl (RDBMS)
Pond (Hive)
Data Platform Architecture
Surf(Data Science)
PlatformServices
Data Capture & Ingestion
Data Sources
• Client Telemetry (mobile, console, pc)
• Server Telemetry
• EA Internal Services• e.g. online e-comemerce, micro txn, virtual goods purchase/trade, etc
• 1st Party (e.g. sales data from xbox, playstation, android, ios)
• 3rd Party (e.g. acquisition marketing, ads)
• EA web sites traffic
Challenges:• Definition and Enforcement of taxonomy standards• Silos and Duplication
10
Streaming and Lambda Architecture
Tech Stack• Kafka
• distributed pub/sub messaging• Storm
• stream event processing
11
Storage & Processing Engine
Storage: multi-tier approach
• HDFS
• Cloud Storage
• Archive/Backup
Tradeoff: cost vs performance
Processing Engine
• Apache Hadoop Stack: Hive, Oozie
12
Data Access & Applications
• Reporting & Dashboards
• Adhoc Analytics• Hive (HQL)
• RDBMS (SQL)
• APIs, Data Subscription
• Closed-Loop Data Driven Online Applications• Personalization/Targeting Systems• Recommendation Engines
13
Data in Action: Examples
14
Dynamic Player Experience
Real-time recommendation engine
• Modify game configuration to optimize for targeted metrics
• Example:
Maximize retention by manipulating game difficulty according to user state
15
16
Initial Configurations Dramatically Affect Win-RatesLevel: Deep Sea Creature
• Initial seed affects the starting board configuration
• # of orange, green, and purple pegs
• Potential locations of the pegs
• Win ratio ranges from 10-50% depending on the seed
• Effective knob for us to create a better experience
17
Game Client
Recent Gameplay
Historical Profile
Predicted Churn Risk
(0% – 100%)
Mapping to Chosen Difficulty
Recommended Levers to Pull
Targeting Recommendation
Churn Risk
How Dynamic Experience Works
Managing Player Relationships
18
Provide the right value
Data Science
What to show them?
Optimization
How to reach them?
Engagement
Who to target?
Segmentation
A set of tools to curate the player journey through differentiating and improving the
player engagement
EA Games
A self-serve tool which enables granular targeting of EA players.
Segmentation
Manage and deliver targeted messages to players in-game, out of game, across the EA network
Engagement
Identify the best placement to engage, track, and test messages to our players
Optimization
Optimize the Player First experience using Data Science
Data Science
Player Relationship Management – Application Components
• Player Profile
• Segmentation via Indexing of key attributes, leverage Lucene• Examples: demographics, game ownership, play time, etc
• within seconds
• Run-time Decisioning Engine
• Communication Channels• Email, PushNote, in-game msg
• Campaign management
• Recommendations, optimizations
19
Anomaly Detection and Reacting to Issues
20
Top Related