Build 2016 - P510 - A/B Testing for UWP Apps: Experiment for Success
-
Upload
windows-developer -
Category
Technology
-
view
213 -
download
1
Transcript of Build 2016 - P510 - A/B Testing for UWP Apps: Experiment for Success
#Build2016
A/B Testing for UWP Apps:Experiment for SuccessJason WatsonSr Program Manager, Windows Dev Center
A/B Testing ScenariosTest variations of your app to discover optimizations that boost conversions and engagement:In-app offersAd placementsUser flowSign-up experienceGame level configuration
Make data-driven decisions about app changes
ExampleCan I boost conversions by changing in-app offer text?Would “Disable Ads Now!” encourage more clicks?My conversion rate is 9% today....
11% conversion rate
45% conversion rate
4514 users (Variation A)
4512 users (Variation B)
Winning Variation
11% conversion rate
45% conversion rate
4514 users (Variation A)
4512 users (Variation B)
Winning Variation
45% conversion rate
9026 users (Variation B)
Winning Variation
How does it work?Update code in your app once to accommodate future experimentsAdd reference to the Microsoft Store Engagement and Monetization SDKBuild app logic around A/B testable settingsCreate logging events for user views and conversions
Use Windows Dev Center to run experimentsRun experiments to identify winning app variationsSwitch your audience to winning app variations
Demo #1: Update App Code for ExperimentationJason Watson
Demo #2: Act on A/B Test ResultsJason Watson
AdvantagesExperiment with your audience using one code base
Identify app variations that boost conversions
Switch your audience to a variation in two clicks
Best PracticesMaximize your test sample sizesUse split distributions for variation assignmentsLimit your experiment to two variations, including the control group
Run experiments for at least 2 weeksDo peek at data after 1 day, but don’t peek afterwards Allow for time to collect statistically significant dataData from specific time of day can skew results (ex. Saturday mornings)
• Click on Experimentation in Windows Dev Center to get started.
• Download the Microsoft Store Engagement and Monetization SDK: http://aka.ms/store-em-sdk
• Re-visit Build on Channel 9.• Continue your education at
Microsoft Virtual Academy online.
Call to Action
© 2016 Microsoft Corporation. All rights reserved.