Educational Research 102: Selecting the Best Study Design for your Research Question
User Research 102
-
Upload
audrey-crane -
Category
Design
-
view
112 -
download
1
description
Transcript of User Research 102
Research Workshop
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.
Confidential – do not copy or distribute
Agenda
1. Research Landscape2. Focus: Interviews, Usability Research3. Best Practices4. Examples (good, bad, and so-so)
5.Exercises6. Wrap-Up
Confidential – do not copy or distribute
Workshop Goals
This half-day workshop will cover:•key types of research•best practices•actual practice
By the end of this workshop, you’ll have an awareness of and experience with research and best practices that will improve the quality and dependability of your findings.
Confidential – do not copy or distribute
Research Landscape
Confidential – do not copy or distribute
JJG’s Elements
text
text
text
text
texttext
Confidential – do not copy or distribute
Research Methods by Data Source vs. Approach vs. Context of Product Use
text
text
text
text
texttext
* updated from Christian Rohrer
Confidential – do not copy or distribute
Research Methods by Data Source vs. Approach vs. Context of Product Use
text
text
text
text
texttext
Confidential – do not copy or distribute
Workshop Focus
Confidential – do not copy or distribute
Research Methods by Data Source vs. Approach vs. Context of Product Use
text
text
text
text
texttext
Usability Studies
Interviews
Confidential – do not copy or distribute
Interviews
strategy
Confidential – do not copy or distribute
DescriptionGoal
Location
Context
Protocol
…to understand underlying motivations, attitudes, and behaviors (attitudinal).
strategy
... often in person, may also be via WebEx or phone.
... does not test a specific product.
Interviews
... describes areas of inquiries and sample questions, but the session is often wide-ranging.
Confidential – do not copy or distribute
DescriptionNeed
Have
Can
...insight into customers or potential customers to discover new ideas or find inspiration for new ways to help or delight them.
strategy
...a hypothesis about a customer or product opportunity, time to conduct and synthesize research, access to 10 or so customers, with ~60-90 minutes, preferably in their environment, and a budget for stipends....comfortably conduct a relaxed open-ended research session, and can video record the session.
Interviews
Confidential – do not copy or distribute
Limitations
1. Predictions often don’t work.2. Details aren’t always insights.3. Biases and assumptions show and
color our insights.4. Insights aren’t guaranteed.5. Synthesizing is critical!6. …
strategy
Interviews
Confidential – do not copy or distribute
After the Session
Post-session synthesis focuses on identifying opportunities based on attitudes / problems. These activities include things like:1.design thinking2.language analysis3.experience maps4.affinity diagrams
strategy
Interviews
Confidential – do not copy or distribute
Usability Research
structure /
skeleton / surface
Confidential – do not copy or distribute
DescriptionGoal
Location
Context
Protocol
…to understand how they might use a product, and to uncover problems and opportunities (behavioral).
…may be in person, via WebEx or phone.
…tests a specific product (yours or a competitors).
Usability Research
...may start with a few attitudinal questions, but usually is longer and more closely followed.
structure /
skeleton / surface
Confidential – do not copy or distribute
DescriptionNeed
Have
Can
...to get a sense of how potential customers will react to a specific product offering or feature, or fine-tune an interface.
structure /
skeleton / surface
...a prototype, competitor, or working code, a protocol, time do conduct research, access to 5 or so customers.
...comfortably conduct a relaxed usability research session.
Usability Research
Confidential – do not copy or distribute
Limitations
1. Predictions often don’t work.2. Observer effect.3. Limitations of the artifact.4. False tasks give false results.5. …
Usability Research structure /
skeleton / surface
Confidential – do not copy or distribute
After the Session
Post-research synthesis focuses on specific changes to a UI based on observed behavior, including:1.affinity diagramming2.visual queues with screen shots and photos and stickies3.analysis of detailed notes
Usability Research
Confidential – do not copy or distribute
Best Practices
Confidential – do not copy or distribute
Moderating (first steps)
1. listen, be quiet!2. be neutral and avoid value words
– use incomplete sentences
– parrot to probe and avoid answering ?s
– avoid asking for predictions
– use their language
– don’t assume their process
Best Practices
Confidential – do not copy or distribute
Moderating (mastering that…)4. be calm5. observations ≠ recommendations6. note hypotheses up front — is success
proving them right or wrong?
Best Practices
Confidential – do not copy or distribute
Moderating (bonus points)7. behavior first, then attitudes and
motivation, not the other way around– BUT if you’ve asked lots of questions about
behavior and still don’t have a good sense of attitude, it’s ok to ask a general question
Best Practices
Confidential – do not copy or distribute
Note Taking
1. In person, sit behind and slightly to one side of the moderator. Remotely, sit off screen or in another room.
2. Be polite, attentive, and moderately interested. (Do not react too much.)
3. Remember, observations before ideas.4. Save your questions for the end.5. Note your top ~5 takeaways
immediately.6. Don’t take your own notes.
Best Practices structure /
skeleton / surface
Confidential – do not copy or distribute
Examples
Confidential – do not copy or distribute
Recording 1structure /
skeleton / surface
Confidential – do not copy or distribute
Recording 1 Findings
1. He thinks my idea is great!
structure /
skeleton / surface
Confidential – do not copy or distribute
Recording 2structure /
skeleton / surface
Confidential – do not copy or distribute
Recording 2 Findings
• Looking at Toyotas• Doing research online using OEM sites
and Google• Uses web daily, not into Facebook
structure /
skeleton / surface
Confidential – do not copy or distribute
Recording 3structure /
skeleton / surface
Confidential – do not copy or distribute
Recording 3 Findings
• Looking at Toyotas• Shows concerns about safety, privacy• Doing test drives, renting cars, and
research online with OEM sites and Google
• Uses web three times a week, uses Facebook (mainly Farmville) every time he logs in
structure /
skeleton / surface
Confidential – do not copy or distribute
Exercise
Confidential – do not copy or distribute
Exercise
1. Groups of three2. One computer recording3. Using our protocol, rotate through:
• participant• moderator• note-taker
4. For each moderator:• 10 minutes conducting session• 5 minutes for feedback
Usability Research structure /
skeleton / surface
Confidential – do not copy or distribute
Giving Feedback
1. Try, “I like…” / “I wish…” / “What if…” (IL/IW/WI) or, “How might we…?”
2. State the specific observation and explain the impact.
3. Be direct, respectful, and sincere (don’t say, “You need to…”.)
4. If you have something nice to say, please say it.
5. Pause and wait for reactions.
Best Practices structure /
skeleton / surface
Confidential – do not copy or distribute
Receiving Feedback
1. Be introspective — what do you think? Note your own positive and constructive feedback. (Hint: if you can’t identify any, this might not be your thing.)
2. It is personal, but even if you start doing this, it will be a small part of your craft. Don’t take it too heavily.
3. This isn’t for everyone — being this neutral is hard.
Best Practices structure /
skeleton / surface
Confidential – do not copy or distribute
Exercise Complete
Confidential – do not copy or distribute
Nota Bene
• For all of these, garbage in, garbage out.
• Agreements must be documented somewhere.
• These are some tools, not THE tools.
Confidential – do not copy or distribute
More Reading1. Don’t Make Me Think, Steve Krug2. The User is Always Right, Steve Mulder and Ziv Yaar3. The Elements of User Experience, Jesse James Garrett4. “From Use to User Interface”, Jeff Patton
http://www.slideshare.net/abcd82/from-use-to-user-interface-presentation5. “Personas for Product Management”, Marty Cagan http://www.svpg.com/personas-for-product-management/ 6. “Summary of Usability Inspection Methods”, Jakob Nielsen
http://www.useit.com/papers/heuristic/inspection_summary.html 7. “When to Use Which User Experience Research Methods”, Christian Rohrer
http://www.useit.com/alertbox/user-research-methods.html8. “Usability Demo Test” by Steve Krug
http://www.peachpit.com/promotions/promotion.aspx?promo=1376029. User Interface Engineering - Articles about User Behavior
http://www.uie.com/browse/user_behavior/10. “Three Questions You Shouldn't Ask During User Research” by Jared Spool
http://www.uie.com/articles/three_questions_not_to_ask/11. Observing the User Experience by Mike Kuniavsky, pages 117-12712. “Meet the Respondents: Understanding User Personalities” (Part 1 & 2) by Mike Fisher - UX Consultant
http://www.uxbooth.com/blog/meet-the-respondents-understanding-user-personalities-part-1/http://www.uxbooth.com/blog/meet-the-respondents-understanding-user-personalities-part-2/
13. “My Best Advice for Conducting User Interviews” by Whitney Hess - UX Consultanthttp://whitneyhess.com/blog/2010/07/07/my-best-advice-for-conducting-user-interviews/
14. Understanding Your Users by Catherine Courage & Kathy Baxter, pages 275-29315. From Self-Prediction to Self-Defeat: Behavioral Forecasting, Self-Fulfilling Prophecies, and the Effect of
Competitive Expectations, by Kristina A. Diekmann, Ann E. Tenbrunsel and Adam D. Galinsky
Confidential – do not copy or distribute
Good Luck!
Please feel free to contact me if you have questions or need help:
This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA.