The Optimizely Experience Keynote by Matt Althauser - Optimizely Experience London 2014
Renee Thompson, TechTarget, Optimizely Experience
-
Upload
optimizely -
Category
Documents
-
view
112 -
download
0
description
Transcript of Renee Thompson, TechTarget, Optimizely Experience
TESTING TIPS AND LESSONS LEARNED
Renée Thompson, Senior Director, Analytics and Conversion Optimization, TechTarget
August 6, 2014
TECHTARGET: WHO WE ARE
• Online global B2B media company with >150 technology-focused websites (SearchSecurity.com, SearchCIO.com, etc.)
• We deliver targeted technology content to IT and business users
• We provide technology marketers media programs via custom advertising, branding and lead generation solutions
OUR TESTING BACKGROUND
• TechTarget has been actively focused on A/B and multivariate testing for over 6 years
• We formerly used Test & Target
• In the past year we switched to Optimizely
• “Always Be Testing” is our mantra
OUR TESTING TEAM• Small team – myself and one dedicated member (Brian DeLoach)
• I generally manage communications, test plan documentation, success and best practice documentation, etc.
• Brian manages designs, test setup, test coding and conducts user research
• We are looking to hire someone else to work under Brian
• We work closely with our team at large on ideas, brainstorming
TIPS
6
TIP 1: DON’T STOP AT ONE TEST WAVE
• Multiple iterations can compound success
• Results can inform additional ideas/iterations
Example: Registration Testing
Test 1: 30% lift in conversions
Test 2: 60% lift in conversions
Test 1: Control Experience
Test 1: Winning Experience
• Updated copy• Elimination of side rail• Modified button call to action • Larger form labels• Slight form field re-ordering• Removal of ‘required’ explanation
30% lift in conversions
Test 2: Dynamic Display
60% lift in conversions
TIP 2: USE QUALITATIVE DATA TOO
• Quantitative only tells part of the story (what, not why)
• Combine with quantitative data to run more successful tests
• Test your “hunches”
Example: TechTarget Redesign
UserTesting.com
Qualaroo
Verify
• KPI metrics comparison before/after showed positive results for most engagement metrics (pages per visit, clicks, mobile engagement, etc.)
• We conducted a series of qualitative tests using UserTesting.com, Qualaroo and Verify
• Most qualitative data jived with quantitative/analytics data
• On Definition pages (a major source of organic traffic), users were very favorable about new “Continue Reading” section, which jived with click data we were seeing
HOWEVER…
• Users like the modern design/look but are frustrated by the lack of navigation
• After analyzing click data on home page, we noticed many users were clicking the drop-down next to the logo
• Most users expected this to be a search box or a navigation menu
• As a result, we are planning more qualitative tests • Do users understand the content labeling methodology?
• And we are planning the following tests:• Navigation addition
• “Continue Reading” optimization and content
TIP 3: VIEW MANY DIFFERENT METRICS
• Each test has a main success metric
• Important to also look at other health metrics in context with the test
• Is a positive result worth a negative trend somewhere else?
• Metrics to watch will depend on your business
Engagement,
Navigation UsageAd CTR
Affiliate Clicks
Site Engagement
TIP 4: INTERNAL PR IS CRUCIAL
• Does the business at large know about the great things you’re doing?
• Are you having trouble getting resources?
• Are other people/groups doing “testing” on their own?
Socialize your testing program!
• Talk to other groups about your process, tests you’ve run, results you’ve seen
• Invite them to share ideas or bring problems to you where you might be able to help
• Develop case studies / one-pagers highlighting your success; document best practices to be shared
• Serves as a proof point for your team
• Ensures everyone knows what you’re doing
• Positions you/your team as expert!
• We hold monthly update meetings with a key group of stakeholders, and quarterly updates with a larger group (including executives)
• We publish success sheets and best practices to share with various groups
HAPPY EXPERIMENTING!
27