Increasing the ROI of Your Beta Tests
41
INCREASE THE ROI OF YOUR BETAS Luke Freiler: CEO and Founder of Centercode
-
Upload
centercode -
Category
Technology
-
view
526 -
download
0
Transcript of Increasing the ROI of Your Beta Tests
I N C R E A S E T H E
ROI OF YOUR BETAS
Luke Freiler: CEO and Founder of Centercode
‹#›
AGENDA 1. BETA TESTING BASICS
2. EXAMPLE BETA ROI
3. THE TYPICAL ROI OF BETA
4. INCREASING THE ROI OF BETA
5. BEST PRACTICES FOR A HIGHER ROI
6. NEXT STEPS + Q&A
Founder & CEO. We offer software, managed services, pre-qualified testers, and educational resources to help companies improve beta testing ROI.
Automotive
Web
Robotics
Hardware
Accelerator Smart Home Wearables
Enterprise Software
Startup
Gaming B2B
Telecom
Mobile
OU
R C
LIE
NT
S
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
‹#›
A test of an unreleased product by real customers in real environments.
LEVEL SET
WHAT EXACTLY IS BETA TESTING?
‹#›
MANY NAMES Customer Validation (CV) Customer Acceptance Testing (CAT) User Acceptance Testing (UAT) Friendly User Testing (FUT) Field Trials Pre-Release Early Release Enhanced Testing User Testing Customer Testing Target Market Testing In the Wild Testing Beta Testing (we like this one)
WHY IT’S CRITICAL
A beta test validates the total performance of the product in the hands of real customers.
HOW IT WORKS 1. Recruit a pool of candidates from your target market
2. Select qualified candidates as participants
3. Prep and NDA participants
4. Distribute product
5. Solicit feedback
6. Evaluate, distribute, and implement the results
HOW IT’S CHALLENGING 1. Dedicating qualified resources
2. Recruiting appropriate beta testers
3. Maintaining user participation
4. Collecting relevant feedback
5. Evaluating and distributing feedback
WHY IT’S SO PAINFUL
1. It’s probably not your full-time job
2. It typically lacks a well-defined process
3. It frequently has no clear owner and/or contributors
4. You likely lack appropriate tools to help
5. The timing is horrible
6. It involves real customers using a buggy product
7. You’re likely worried about controlling information
8. If you succeed, you’ve got a ton of data to deal with
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
IMPACT OF BETA TESTING
Sales and Costs
Estimated Sales 200k units
Unit Sale Price $100
Unit Cost $50
Beta Unit Cost $200
Incentive Costs $50/user
Product and Market
Release Type New
Product Type Hardware
Target Market Consumer
Competition High Complexity High
Quality Confidence High
Beta Management / Resources Tools Dedicated
Experience Part-Time Wage $40/h
Desired Testers 30
Participation Rate 30% Duration 4 Weeks
EXAMPLE ROI ELEMENTS & ASSUMPTIONS
Key Facts No Beta Basic Beta
Beta Management Hours 0 73
Total Beta Testers 0 100
Participation Rate (%) 0% 30%
Defects Found 0 41
Defects Fixed 0 25
Total Units Sold 190,000 200,000
Customers Requiring Support (%) 38% 36%
Product Return Rate 19% 19%
EXAMPLE BETA RESULTS
Financial Impact No Beta ($) Basic Beta ($)
Total Beta Management Cost - (5,491)
Total Logistics & Production Cost - (25,000)
Total Defect Cost (585,000) (562,250)
Total Support Cost (806,614) (787,510)
Total Return Cost (1,087,242) (1,092,025)
Total Warranty Cost (665,000) (633,333)
Gross Sales Profit 9,500,000 10,000,000
Net Gain - 538,246
TOTAL ROI 946%
FINANCIAL IMPACT
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
BUG REPORTS
‹#›
FEATURE REQUESTS
‹#›
LATE STAGE MARKET RESEARCH
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
BETA MIRRORS YOU
BETA TOUCHES THE ENTIRE PRODUCT
BETA REPRESENTS THE CUSTOMER
A
B
‹#›
BETA IS AN ARBITRATOR
BETA IDENTIFIES PRIORITY
BETA IS FOR PREPPING SUPPORT
BETA COVERS THE ENTIRE CUSTOMER
EXPERIENCE
‹#›
BETA IS A TOOLBOX
BETA IS FOR BUILDING RELATIONSHIPS
BETA IS KNOWLEDGE
BETA IS A SAFETY NET
BETA ASSESSES LAUNCH READINESS
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
1 FOCUS ON GOALS
IDENTIFY YOUR CONSTRAINTS
DETERMINE YOUR OBJECTIVES
MAP YOUR GOALS TO A SCHEDULE
DEFINE YOUR TARGET MARKET
2 PICK GREAT TESTERS
QUALIFY YOUR TESTERS
AVOID ARM’S REACH
SEEK OUT ENTHUSIASM
LIMIT YOUR POOL
3 BE BETA READY
READY YOUR SYSTEMS
READY YOUR TEAM
READY YOUR PRODUCT
READY YOUR TESTERS
4 REDUCE FRICTION
MESSAGE CLEARLY
OFFER SIMPLE FEEDBACK CHANNELS
PROVIDE SUPPORT
BE RESPONSIVE
t
5 EMBRACE THE RESULTS
FREQUENCY * SEVERITY = IMPACT
SHARE, BUT AVOID A FIREHOSE
INTEGRATE IF POSSIBLE
CONSIDER FEEDBACK IN CONTEXT
‹#›
BETA TESTING BASICS
EXAMPLE BETA ROI
THE TYPICAL ROI OF BETA
INCREASING THE ROI OF BETA
BEST PRACTICES FOR A HIGHER ROI
NEXT STEPS + Q&A
CALCULATING YOUR OWN BETA ROI
FRE
E
WWW.CENTERCODE.COM/ROI WWW.CENTERCODE.COM/MYROI
DIY GUIDED
Thank you for attending. Please join us for our next webinar on pricing, April 25th.
Download this webinar and sign up for more at
pragmaticmarketing.com/pragmaticlive
