C(ollab) RITE: How to run impactful iterative studies in a fast paced environment

C(ollab) RITE How to run impactful iterative studies in a fast paced environment

Helena RoeberJhilmil Jain

Google

#uxpa2013#crite

@jhilmiljain@helenaroeber

#androidux

C-RITE Impact

• User research insights are adopted quickly• Product team has clear sense of progress • Researchers and designers earn

understanding and respect for their work• Advocates for the user centered design

process are born

#uxpa2013 #crite #androidux

C-RITE• Iterative UX research method suitable for agile

environments

• From research to new design solution in 1 day

• 7 research sessions in 2 days

• Collaborative design workshop each day

• Prototype changes between day 1 and 2

• Can be repeated weekly

• Stakeholders collaborate in all UX phases


User Research Stages

Research AnalysisReporting

&Prioritization

Design PrototypingPlanning

& Recruiting


ThePlayers

The Players and Their Roles

TheResearcher


Traditional Researcher Role

Research Analysis Design PrototypingReporting

&Prioritization

Planning & Recruiting

Researcher

TheDesigner


Reporting &

Prioritization


Engineers

Traditional Designer Role

Engineer


&Prioritization


Designer Designer

Researcher

TheProduct

Manager



&Prioritization


& Recruiting

Traditional PM Role

Reporting &

Prioritization

Planning & Recruiting Research Analysis Design Prototyping

Reporting &

Prioritization


Designer Designer

Researcher

PM PM

TheEngineer



&Prioritization


& Recruiting

Traditional Engineering Role

Reporting &

Prioritization

Planning & Recruiting Research Analysis Design Prototyping

Reporting &

Prioritization


Designer Designer

Researcher

PM PM

Engineer

Collaboration in C-RITE


&Prioritization


Designer

PM

Engineer

Researcher

Time Line - Traditional Usability


&Prioritization


& Recruiting

1-2 weeks 1-2 weeks 1/2 - 1week 1/2 -1week

3 - 6 weeks


Time Line - Traditional Usability


&Prioritization


& Recruiting

1- 2 weeks 1- 2 weeks 1/2 - 1week 1/2 -1week

4 - 8 weeks


1/2 - 1 week1/2 -1 week

Time Line - RITE


&Prioritization


& Recruiting

1 - 2 weeks 1 - 2 weeks

2 - 4 weeks


Time Line - C-RITE


&Prioritization


& Recruiting

1-2 weeks 2.5 days

1.5 - 2.5 weeks


Time Line - Cyclic C-RITE


&Prioritization


& Recruiting

1 week 2.5 days

1.5 weeks


Research Analysis Reporting &Prioritization Design Prototyping


Example: TV Voice Search



1st TV Voice Search Design



C-RITE Research Timing

10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm

Day 1 Day 2




C-RITE Collaborative Analysis

10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm

Day 1 Day 2




Observation Board Example



User Goals Example

Observation ExampleGoal example

P1P1 P2P2Bob Lin Ali Nur Bob Lin Ali Nur

User knows what to say 2 1 2 2

The system understood user’s intent 5 3 1 1

The system doesn’t slow down user w unnecessary confirmations 4 4 2 2

User can easily tell where speech will work 2 2 2 2

User comfortable using their voice 4 4 4 4

Rating on a scale from 1- 5 1 = poor 5 = good

Research Analysis Reporting &Prioritization Design Implementation


Collaborative Observation


• Re-call to count observations, analysis step reduced

• Disagreements about observations are resolved immediately

• Research protocol can be adjusted based on stakeholder feedback

• Researcher can explain what observations mean

• Small sample size discussion is obsolete



C-RITE Analysis

10:00 am 11:00 am 1:00 pm 2:00 pm10:00 am 11:00 am 1:00 pm

Day 1 Day 2

2:00 pm 3:00 pm




From Observation to Insight




Prioritizing Insights

Goal example

Insights



C-RITE Design Workshop

3:00 pm

Day 1

10:00 am 11:00 am 1:00 pm 2:00 pm 10:00 am 11:00 am 1:00 pm 2:00 pm

Day 2

6:30 pm5:30 pm




Design Workshop



Design Workshop Actions




Action Item Owners & Due Dates





• Turn panic into confidence





• Turn panic into confidence• Designers demonstrate problem solving skills





• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints





• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints• Researchers guide insight interpretation





• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints• Researchers guide insight interpretation• PM gives business and scheduling input





• Turn panic into confidence• Designers demonstrate problem solving skills• Engineers solve technical constraints• Researchers guide insight interpretation• PM gives business and scheduling input• Everyone participates in creative process



Before & After



C-RITE Reporting

Day 1

Send action items to team

Day 2

Send action items to team Short report

Day 3




C-RITE Prototyping

3:00 pm10:00 11:00 1:00 2:00

Day 2

6:30 pm

Day 1

10:00 11:00 1:00 2:00 5:30 pm




C-RITE Cycles

Week1 Week 2 Week 3




C-RITE Recruiting

• Create a panel of users that meet general recruitment criteria

• Panel members commit to availability at short notice on research days

• Each week, schedule panel members• Recruit for specific criteria when needed




C-RITE Room Design

• Comfortable seating for all to observe sessions

• Whiteboard to record observations, and drawing during design workshop

• Sunlight and windows is good for creativity




C-RITE Preparations

• Set expectations with all stakeholders beforehand about time commitment, collaboration and workshop format

• Decision makers need to participate in C-RITE studies and workshops




C-RITE Summary


• Iterative UX research method suitable for agile environments

• From research to new design solution in 1 day

• 7 research sessions in 2 days

• Collaborative design workshop each day

• Prototype changes between day 1 and 2

• Can be applied weekly

• Stakeholders collaborate in all UX phases

ResearchObservation & Analysis

Reporting &Prioritization Design Prototyping


Thank you


[email protected] @jhilmiljain

@[email protected]

mailto:[email protected]




Appendix

Agile Methods: RITE

• Changes are made as soon as a problem is identified, sometimes after 1 session

• 1-2 sessions a day. Breaks between sessions long enough to make changes

• 1-2 weeks of sessions

• Requires stakeholder attendance and participation

Medlock, Wixon, Fulton, Terrano and Romero, UPA, 2002




Agile Methods: KRUG + RITE

• 4 participants over 2 days, at least 1 hour between sessions to make changes

• Researcher sends out observations and recommendations 30 min after last session

• Team debrief of observations and action items on day 2, 1 hour after last session

• No collaborative design workshop

McGinn, Chen, Journal of Usability Studies, 2013



Agile Methods: PULSE

• 3 participants, 1 day every week

• Analysis and write-up of results by researcher within 2 days

• No collaborative analysis and design workshop

Konno, Agile 2012



Variations We’ve Tried

• C-RITE Remote• If time zones allow, stakeholders participate through ‘Hangouts’

• Sessions are recorded and watched with time delay

• Next day debriefs and design workshops for incompatible time zones

• C-RITE Diary Studies• In-lab diary kick-offs and debriefs are run in C-RITE format

• Researcher analyzes and summarizes diary data for debrief and design workshop

C(ollab) RITE: How to run impactful iterative studies in a fast paced environment

Technology

Transcript of C(ollab) RITE: How to run impactful iterative studies in a fast paced environment