DrupalCon Austin: UX Bootcamp workshop
-
Upload
perfetti-media -
Category
Design
-
view
2.307 -
download
0
Transcript of DrupalCon Austin: UX Bootcamp workshop
UX Bootcamp
Christine Perfetti, Acquia @cperfetti
The LogisticsThe schedule
Ask questions as they occur to youDon’t surprise us on the evaluation form
Please keep your laptop turned off unless we’re doing an exercise
You’ll be working in groups of 3Preferably with someone you don’t know
Traditional usability tests
A usability test involves putting your users in front of your product and observing
what they do
Exercise: A usability test of Disney.com
Exploration: Usability TestBreak into teams of 3
One person is the userThink aloud as you work
Two people are the observersObserve silently and take notes
After we perform the task, we’ll ask you to switch roles and do another task
As the user...
We’re evaluating the design, not you
The answer is somewhere on the siteIf you experience problems, it’s not your fault
You’re helping the observers learn more about the design problems
You can stop the task at any time
As the observer...
Take notes as the user worksWas the user successful?
How did the user go about performing the task?
Did they go to the search engine, what links did they click on?
What obstacles did the user encounter?
What confusion did they experience?
Note things that worked well
When you complete a task, say, “Got it!”
Disney.comScenario:Your 5-year-old niece has always wanted to go to Disney World. You have decided to take her there for a special vacation.
Task Assignment:You are looking for a hotel within the park. You want it to be the cheapest hotel with access to the monorail. Which Disney World hotel would you choose to stay in?
When you’re satisfied that you’ve found the answer to this question, say, “Got it.”
Discussion
How many users found the answer?
What helped users succeed?
What obstacles prevented them from finding what they wanted?
What do you expect will happen when you test your product?
Exercise: A usability test of Acquia.com
Acquia.comScenario:Acquia’s CEO, Tom Erickson, was interviewed last year about the power of open source Drupal and what’s driving the success of Acquia.
Task Assignment:Find information on the Acquia web site about Tom’s view on the power of Drupal.
When you’re satisfied that you’ve found the answer to this question, say, “Got it.”
Exercise: A usability test of Amazon.com
Amazon.comScenario:Your job has been causing you a lot of stress lately, and you're looking for some relief from the tension that's been building in your back and shoulders.
Task Assignment:Find a product, for under forty dollars, that will help relieve the tension in your back and shoulders.
When you’re satisfied that you’ve found the answer to this question, say, “Got it.”
Signs of content and design failure
Back button
Pogosticking
Going to Search
FAQs, Help, and Site Map
Definition of UsabilityBasic definition of Usability
Can users accomplish their goals with your product?
Shades of gray: How effectively can users accomplish their goals?
To make a product usable, we need to know, What are the company’s goals for the product?
Who are the product’s users?
What are the users’ goals? Do they have different goals?
What we’ll coverHow to complete all of the steps for a usability study
Plan a study
Create tasks
Moderate a session
Analyze the data
Communicate the findings
Quick-and-dirty techniques
Study planning
The study planProvides a systematic approach to the testing process
Serves as a blueprint for the test
Facilitates communication and buy-in from stakeholders
Addresses concerns or issues about the product and data to uncover
Outlines project deliverables and timelines
Topics to addressStudy’s goal
Research questions and issues to explore
User characteristics
Study method
Schedule
Task list
Data to collect
Reporting strategy
Hold a planning meetingAt the beginning of the project, we hold a planning meeting with all product team members and stakeholders to discuss:
An overview of the testing process
Project scope
Issues (risks and concerns) with the design
Decide on number of users to study
Testing schedule
No surprises! Avoid the game of telephone
Who should attendProduct and design team members
Marketing, engineering leads, and executives
Any person who is involved in the design
Issues, risks, and concernsWhat do you want to learn?
Most studies aren’t purely exploratory
The issues and research questions will drive the tasks and participants you choose
Work with the design team and stakeholders to outline the research questions
Vague statement: Will this work for users?
Creating your tasks
Designing tasksTasks allow us to assess a product’s usability
The tasks create a realistic context to observe the problems users encounter
Interest affects behaviorUsers search more vigorously
They are more likely to read the content
They are willing to wait for downloads
Traditional tasksVerb-based tasks
Scavenger hunt tasks
Verb-based tasksAsk users to do something specific
Assess the product’s functionality
Most common for evaluating software, hardware, and web applications
ExamplesRespond to the email you just received from Edmund Boston
Write a note to your mother
Copy the text of this page to another document
Send the message from Edmund to your friend, Lisa
Pros and consPositives:
Test functionalityMeasure how different users perform the same task
Negatives:Is this something users really do?Not taking the larger context of when/why users perform these tasks into account
Scavenger hunt tasksChallenge users to find something specific
Test users’ understanding of information
Most common form of task used for information-rich web sites
ExamplesYou were at a party last week. The discussion turned to recipes for authentic Italian pasta dishes. Go to the Food Network site and find an Italian recipe for pasta.
Of the bookcases on the IKEA site, find the ideal one for your living room.
Pros and consPositives:
Test how well users find and understand informationMeasure how easy/difficult it is for different users to find the same information
Negatives:Is this something users really do?Not taking the larger context of when/why users perform these tasks into account
Creating verb-based tasksList out the things the product or design allows users to do
Identify critical verbs from the list
May need a logical order
Create tasks for each verb
Success is when the user completes the task
What are Gmail’s verbs?
Verb-based tasksSign up for a new account
Send mail to [email protected]
Attach a Word file to the message
Add Lisa’s email to your address book
Open and read Lisa’s reply
Send a meeting invitation to Lisa
Creating scavenger hunt tasks
Surf site or product
Find topic that could be of interest to users
Create scenarios and questions to go with the topic
Avoid “giveaway” words
Success is when the topic is found
Scavenger hunt tasks are verb-based tasks with the verb, “find”
Finding IRS forms
Scavenger hunt tasksFind the latest version of the 1099 form
Will you be eligible to file your taxes online?
What changes are planned for taxes in 2011?
ExerciseBrainstorm at least four tasks for the product or site you would like to evaluate
They can be a combination of verb-based or scavenger hunt tasks
Write one task per page
Create a short scenario to set the context for each task
You will use these tasks to test your product
Testing exerciseBreak into groups of 3.
Spend 10 minutes testing your product
You can also pick another site to test
After 10 minutes, switch roles
Running a usability study
Moderating a sessionHow to reduce user anxiety
How to maximize the information you get from each test
How to get users unstuck
Overview of a sessionPre-test briefing (users and observers)
Introduce users and observers
Explain testing protocol to users
Conduct test
Mini Q&A session after each task
General Q&A session at the end
Debrief observers
How We Test
Lab configuration Conference room
Moderator PrioritiesFlight attendant
Responsible for safety and comfort of passengers
Sportscaster
Doing the play-by-play
Probing issues
Maximizing information to observers
Carolyn Snyder, “Paper Prototyping” (2003)
Moderator PrioritiesScientist
Planning the study activities
Gathering and prioritizing data
Writing the reports
Carolyn Snyder, “Paper Prototyping” (2003)
Pre-test briefingWe are testing the product, not you
You can stop or take a break at any time
It’s fine if you don’t complete a task
Problems aren’t your fault
What we are learning is valuable
You’re allowed to give up
Maximizing InformationVerbally reinforce the action
Observers may not see everything
Respond to user vocalizations
Hmmm, Aha, Oops
Respond to user silence
Take advantage of download time and pauses
Summarize the users’ recent actions
Ask what they will do next
Questions to AskIs that what you expected?
What are you thinking?
What are you trying to accomplish?
What does XX mean to you?
Can you say more about that?
Getting Users UnstuckFacilitators must balance two responsibilities: gather as much information as possible and assist the user
Help the user when:
They show signs of frustration or discomfort
The observers aren’t learning anything new
Helping the userMove from general to specific
Use questions rather than answers whenever possible
Can you explain what you're trying to do?
What do you think the next step is?
Do you see anything on the screen that might help you?
What do you think that button does?
Provide encouragementProvide reassurance to a struggling user
Note any user questions you don't answer
Respond immediately to any user statements about their lack of ability
At the end of a taskProvide positive feedback on user's performance ("You're doing great!")
Answer user's questions (when possible)
Ask if user needs a break, a drink
After a particularly stressful task
Take a break
Talk to user; assess how they're feeling
End of session Q&AWhat are two things you liked about the product?!
Helps turn the test session into a positive experience
What are two things you would like to see improved about the product?
Helps prioritize usability problems or unresolved issues
Quick and Dirty Techniques
Quick techniques5 second test
First click test
Comprehension test
5 Second Page Tests
Quick and dirty technique for measuring content pages
Takes less than 10 minutes to run
Measures if content pages quickly convey their purpose
Sharing your pictures online
You want to post pictures online from your last vacation
You are concerned that the upload process will be difficult
How confident are you that you can upload photos quickly and easily?
Sharing your pictures online
You want to post pictures online from your last vacation
You are concerned that the upload process will be difficult
How confident are you that you can upload photos quickly and easily?
Sharing your pictures online
You want to post pictures online from your last vacation
You are concerned that the upload process will be difficult
How confident are you that you can upload photos quickly and easily?
Not for the Home Page
First Click Test
Useful method to assess where users first click on your site’s home or entry page
Provide users with a specific task to complete when they arrive at the site
By observing where users first click, it’s a clear indicator whether they’ll eventually be successful
Comprehension TestPages containing complex content
Such as policies or procedures
How your product works
User comprehension is imperative to their success
Questions determine if users understand content
Exercise: Practice Your Moderation Skills
Analyzing the data
Analyzing the dataGather all observations
Identify problems
Prioritize problems based on importance and frequency
Iterate the design
Data analysis for exploratory tests
Identify patterns of usability problems
Group related observations together
Prioritize the usability problems based on importance
How widespread is the problem?
How critical is the problem to fix?
The KJ MethodNamed after Kawakita Jiro
Consensus method for grouping and prioritizing usability findings
Quickly determines most important observations
Captures everyone’s perspective and observations
Step 1: Organize the group
Call together the group
Only invite members of team who observed a test session
KJ takes one hour
Step 2: Decide on focusEvery KJ has a focus question that drives the prioritization exercise:
What are the biggest usability problems we observed in the test?
What needs to be fixed in the product to improve the user experience?
Step 3: List problems observed in test
Step 4: Put observations on wall
Step 5: Group the notes
Step 6: Name each group
Step 7: Choose and rank the groups
Step 8: Voting
Step 9: Order and discuss groups
We order the groups based on number of votes, highest numbers at the top
Participants combine groups and discuss identified priorities
Advantages of methodWriting down observations lets all team members contribute
Prevents too much influence from select team members
Identifies top priorities
Exercise Conduct a KJ
? Airline check-in and flight experienceWhat are the biggest problems that need to be fixed to improve the experience?
Communicate the results
The importance of communicating results
Must convey important findings to the people responsible for making design changes
Teams often underestimate the amount of communication necessary
Plan a communication strategy before testing
Success factorsCommunicate results quickly
Debrief often
Report to stakeholders and project team when they need it
Don’t report too much information
Determine best delivery method considering the needs of your different audiences
No surprises!
Methods for communicating results
Informal
Emails or internet dashboard with short memos of findings and recommendations
Working meetings with project teams
Integration with existing problem/bug reporting strategy
Formal
Presentations and workshops
Highlight video
Usability reports
Email or internet dashboard
Effective communication tool for sharing test results immediately
After each day of tests, send a short email to design team with major observations from the day
Working meetingWhen quick changes are required after a round of tests, schedule a one-hour meeting to share and prioritize the test results
Review the usability test observations
Determine next steps and short-term fixes
Integrate with bug trackingMost organizations already have a method for collecting problem or bug reports
Merge usability problems with other bugs
Prevents having two tracking systems
Helps determine relative importance of a problem
Presentation and workshops
1-2 hour presentation with design team (and sometimes engineers, marketing, key stakeholders)
Share most important usability findings based on prioritization of problems
Effective communication technique
People often ignore written communications
Active involvement with design team and stakeholders
Highlight videos20-30 minute video capturing the most important findings from the usability test
They can be helpful to convince stakeholders of problems
Time consuming to produce and many teams don’t watch them
A formal reportA formal report may be necessary if:
The usability study is being conducted by an external consultant
The results won’t be implemented immediately
The company culture demands it
Reporting tipsDescribe findings in terms of user behaviors and expressed thoughts
Include positive findings in addition to the negative findings
Include screen (prototype) snapshots; the report has to stand alone
Write clearly and concisely
short
to the point
bulleted lists
Focus of reportFocus on:
Important findings
Data to support each finding
Recommendations for each findings
Relate your findings back to the focus of the study
What did you learn about each of your issues?
What else did you learn?
What is working well?
What needs to change?