Post on 13-Jan-2015
description
Alan Avidan, PhD, MBAExec. Director & Chief BeesDev
Optimization Technologiesfor Lifting KPIs in Games
2012© Bees and Pollen
• Analytics• A/B Testing• A Priori Segmentation • Clustering Segmentation
Optimization Technologies We’ll Cover
2012© Bees and Pollen
Game Elements/Events
3
Payment Page TutorialPromotions
Landing page
Full tutorial
Level 2
2012© Bees and Pollen
Payment Page
4
Which one produces more revenue?
Low Range High Range
2012© Bees and Pollen
Promotions
5
Which one performs better?
Percent Discount Absolute Discount
2012© Bees and Pollen
Tutorial/Game Flow
6
Option 1 Option 2Landing page
Full tutorial
Level 2
Landing page
Short tutorial
Level 2
Option 3Landing page
No tutorial
Level 1
2012© Bees and Pollen
• Payment Page • Game flow/tutorial • Promotions and offers• Sharing Messages • Virtual Goods Selections • Payouts (Casino Games) • Themes/Arts/Creatives• Landing Pages• Emails (Subject, Body, Submission)• Special Deals Closing Time
More Elements Can be Optimized
2012© Bees and Pollen
Decide
Measure
DisplayAnalyze
Change
Analytics
2012© Bees and Pollen
A/B Testing
Define options
Split traffic Measure results Deploy winner Result
Low range
high range
high range
2012© Bees and Pollen
Make sure that the test is statistically significant - run it for long enough, and with enough traffic to make it count
I have learned how dramatically, and ridiculously wrong my most basic assumptions were
It's empirically proven that you should let the data tell you what works or not and you should constantly be testing
That the devil is in the detail - a minor change can generate a significant result
Experts Weigh In: A/B Testing
Q: What are the most unexpected things people have learned from A/B tests?
2012© Bees and Pollen
Upside• Simple; understandable
Can achieve good results
Downside:• One-size-fits-all• Results deteriorate over time
A/B Testing – Bottom line
2012© Bees and Pollen
Define segments Define Options and rule base
Result
A Priori Segmentation
Low range
high range
2012© Bees and Pollen
A Priori Segmentation Upside• Can be effective if segmentation was meaningful
Downside• Segments are predefined and remain unchanged
during the analysis• Different elements might require different segments• Hard to scale in terms of data-set and number of
elements• Hard to fine-tune
2012© Bees and Pollen
Clustering Segmentation Define options
A/B test options
Segment users based on result
Deploy winner
Low range
High range
2012© Bees and Pollen
Clustering Segmentation
Upside:• Highest Lift • Discover correlations never knew existed
Downside:• Requires storage of terabytes of data• Need really smart people• Effort = High
2012© Bees and Pollen
Real-time and automated predictive algorithmic technology that serves each user the page options they are most likely to convert on
Clustering SegmentationPredictive Best-Fit
2012© Bees and Pollen
Advanced algorithms find correlations between user data DNA and conversions
Predictive Best-Fit
UserUser Social and Behavioral Data
User DNA Generation
Predictive Best-Fit Algorithms
2012© Bees and Pollen
Backup slides
19
2012© Bees and Pollen
Geo-Demographic attributes: age, gender, education, country
Facebook attributes: Friends, Likes, Interests, Posts, Events
Behavioral attributes: level, spending, score, progress, custom
Session attributes: time of day, day, duration
Proprietary attributes: novice, high-bidder, risk-averse
3rd Party attributes: income level, education
Attribute Sources
2012© Bees and Pollen
• Automated end-to-end solution• Machine self-learning• Real-time• No game history required• Numerous data sources• Dashboard – Easy to swap-in options• Deep new insights (identify discriminators)
Predictive Best-Fit All the gain without the pain