Combining Methods: Web Analytics and User Testing
30
Combining Methods Web Analytics and User Testing Martijn Klompenhouwer & Adam Cox UPA - May 27 th , 2010
-
Upload
user-intelligence -
Category
Design
-
view
113 -
download
1
description
Combining Methods: Web Analytics and User Testing. Presentation delivered at the UPA2010 in Munich
Transcript of Combining Methods: Web Analytics and User Testing
Combining MethodsWeb Analytics and User TestingMartijn Klompenhouwer & Adam CoxUPA - May 27th, 2010
Combining MethodsWeb Analytics and User TestingMartijn Klompenhouwer & Adam CoxUPA - May 27th, 2010
Who are we..?Web Analytics User Research
Why you should
combine methods
Practical examples Why you should
try it too
About User Intelligence
Martijn Klompenhouwer
UX researcher
11 years UX experience
8 years at User Intelligence
Adam Cox
Web Analyst
7 years UX / Analyst experience
3 years at User Intelligence
User Intelligence
It’s our goal to design interactive products that provide great User Experiences, based on knowledge of both the users and the business.
Services we offer:• Research • Design• Optimize
www.userintelligence.com
How to conduct Global User Research written by partners of the UXalliance and other experts
Available since November 2009 published by Morgan Kaufman
Handbook of Global User Research
About Web Analytics
"Web Analytics is the measurement, collection, analysis and reporting of Internet data for the purposes of understanding and optimizing Web usage.“
- The Official WAA Definition of Web Analytics
Web Analytics Definition
Measurement
Data Collection
Analysis
Reporting
What does the definition mean?
Where are your users
coming from?What are they doing?
Where and when are
they leaving?
Some examples of Web Analytic measurements
Referrals Visitors, visits and page views
Popular pages
Bounce rates Funnel analysis Path Analysis
Issues with Web Analytics
Data often perceived as boring statistics
Implementation of WA tool not always correct
Reports: just the facts, no insights, no actions
Web Analytics is quantitative in natureTo get insights, interpretation of the numbers is needed
About User Research
Field observations
Some examples of User Research methods
Personas Diary studies Card sorting
Expert Review InterviewsFocus groups User Testing
Issues with User Research methods
Data usually from small numbers
Most methods take a snapshot in time
Difficult to capture some behavior
Setting sometimes artificial (e.g. lab test)
User Research is qualitative in natureYou get the ‘why’, but not the ‘big numbers’…
So, why should you combine these two methods?
User Research findings can help interpret web data
Web data can help focus the User Research
More certainty of findings (based on two sources)
The methods complement each other Quantitative vs. Qualitative
Let’s clarify with some examples…
Example 1: Recruiting the right participants
Visitor AnalysisAnalyzing current traffic on website gives clues about audience
Target audienceWho to invite to participate in a test?
Better insightsMore knowledge on users helped in defining recruitment profiles
Example 2: Test scenarios
User journeys Homepage was not the main entry point
Writing test scriptCreating realistic test scenarios: how is site used now..?
Google scenarioTesting outcomes of a common scenario
Example 3: Explaining abandonment rates
Funnel analysisSales funnel analysis revealed in which steps users left the flow
Target test tasksKnowledge enabled us to concentrate usability test on those steps
More focusFinding reasons for abandonment and looking for solutions
Example 4: Use of Advanced features
Not measuredUse of feature was known, but no data on the use of settings
Popular settings?Which options are used and are there any patterns or issues?
Collect dataDecided not to use too much test time: wait for proper measurements
Example 5: Validating findings
Impact analysisQuantified issue using data of thousands of visitors
Big issue?Only 2 out of 10 test participants had this problem
End of discussionDiscussion changed from: “Is that an issue?” to: “Let’s solve it!”
Some additional examples…
Effectiveness of segmentation
Verifying user
feedback
Unintended user-flows
Interpreting bounce
rate data
Why you should try it too
Combining methods works!
You can tell a story backed up with data
One report: Clear actions and no conflicting recommendations
Useful in different stages of projects (Research, Design, Optimize)
The combination works both ways! Web Analyst and User Researcher benefit from each other
Web Analytics can be used with many User Research methods
Just remember…
Fruitful results can already be gained from basic analysis
You don’t need to be a Web Analytics expert, but it helps ;-)
Web Analytics tools do not magically provide the insights...
Measure the impact of changes (Optimize)
The UX team should make more use of Web Analytics!
It shouldn’t just belong to the IT or Marketing departments
What we want you to take away…
Find out what web data is available
Use it!..... Just try it! No more excuses ;-)
Web data will help you in different stages of a project
If important data is missing, try to get it measured in the future
Integrate Web Analytics into your process and methods
user intelligence
Amsterdam office
www.userintelligence.com
Thank you!
Martijn [email protected]
Adam [email protected]
