Getting started with server-side testing
Dennis Pavlina and Thomas Davis
WiderFunnel Strategist and WiderFunnel Web Developer
● The presentation deck and recording will be available to you after the webinar.
● Feel free to ask questions during the presentation, using the Question panel.
Housekeeping
Dennis Pavlina, WiderFunnel Strategist
● Oversees experimentation strategy for some of WiderFunnel’s most notable full stack clients
● Guest lectures at local design and tech institution, RED Academy
Your Presenters
Thomas Davis, WiderFunnel Developer
● The go-to guy for server-side experiment development
● Instructor and mentor at local coding bootcamp, Lighthouse Labs
● Why adopt server-side testing?
● Laying the foundation for server-side testing
● Technical best practices
● Fostering server-side testing long-term
What We’ll Cover
Why Server-Side?
What is Server-Side Testing?
What is Server-Side Testing?
The Possibilities of Server-Side
The Possibilities of Server-Side
The Possibilities of Server-Side
● Improved user experience
● Ability to open up testing to back-end changes
● Feature testing
● Faster implementation of winning variations
Server-Side Experiment Example: Algorithms
Displaying “Other Products
You Might Be Interested In”
as opposed to “Frequently
Purchased With” products
Server-Side Experiment Example: Onboarding flows
Server-Side Experiment Example: Trial durations
Identifying whether more
users sign up when given a
7 day trial versus 14 days or
30 days.
Evaluating Opportunities for Server-Side
Accessibility Depth Goals Channels
WYSIWYG vs. Dev team Styling vs. Functionality Engagement vs. LTV Web vs. Multi-Channel
Laying the Foundation for Server-Side Testing
Define your success metrics: Identify new goals
Server-Side
● Rate of visitors who complete all onboarding steps● Rate of visitors who convert to paid signup within 24 hours● Rate of visitors who follow 5 or more people during onboarding
Client-Side
● Signups● Blog subscriptions● Visits to key pages
Define your success metrics: Place value on feature testing
Define your success metrics: Set the right timeline expectations
Full Stack Best Practices
● Eliminate the difficulty of integration
● Principle of “separation of concerns”
● Easily testable
● Doesn’t intrude on feature builds
It’s all about Middleware
Activate the experiment
Monkey Patch the res.send method
Serve the correct variation
Original Route method remains untouched
● Moves targeting to the Optimizely tool / Optimization specialist
● Can target anything from pages to device types
● Experiments become more dynamic (Not hardcoded)
Targeting through Attributes
Find the current page, return as an object
Send attributes and return the correct variation
Working with SPA’s
● Use getVariation() to get users into variation without tracking them
● Route to activate user later / Front End JS SDK
https://www.myawesomesite.com/activate?test=wf_test&user=xxxxxxxxxxx
● Create top level A/B testing components if using React or Vue
● Setup all global events first (i.e. page visits, revenue tracking)
● May need a ‘AJAXable’ route for UI component goals /
Alternatively use the client-side JS SDK
● Make naming conventions specific and namespaced
products-index/add-to-cart-cta
Custom Event Set up
● Have a roadmap for the installation process
● Use cookies or webhooks when dealing with page caching reverse
proxies
● Random string vs. database user ID to identify the Optimizely user
● Test Driven Development (TDD)
Tips and “Got ya’s”
Get Optimizely X Full Stack
Get the Resources You Need
Gather and educate the right talent
Coordinate the effort via dedicated team leaders or an agency partner
Identify the Best First Experiments
Hypotheses Potential Importance Ease PIE Score
Hypothesis #1
Web analytics
Heuristic analysis
Voice of customer
Cost
Traffic volume
ROI
Technical
“Political”
8 7 7 7.3
Hypothesis 2Displaying a free trial countdown timer on the dashboard will increase the rate of users who convert to a paid account.
Identify the Best First Experiments
Hypothesis 1Removing all optional steps from the onboarding flow will increase the rate of users who convert to a paid account.
Hypothesis 3Automatically filling in location-specific information in the paid signup form fields will increase the rate of users who convert to a free account.
Identify the Best First Experiments
Hypotheses Potential Importance Ease PIE Score
Remove optional steps
Countdown timer
Pre-filled form fields
8 9 9 8.7
6 8 8 7.3
6 8 5 6.3
The Long-Term Vision
Combining Client-Side & Server-Side
Work to establish a “We should test that” culture:
● Dedicated brainstorming sessions● Regular lunch and learns● Company-wide results/insights updates
Experimentation Culture
Q&A
Thank you!
Top Related