OBSERVATION & USABILITY

Chapter 5

Sharla Carpenter, Wynter Myers, Regina Reynolds, Deedra Totten, Stephanie Vap, Fran Varella.

Finding a Topic: ObservationTYPES FOCUS OF STUDIES

Participant Observation

Natural Environment

Reactive Observation

Controlled Setting

Unobstrusive Observation

Naturalistic Observation

Unobstrusive methods measure reference transactions

Analyze patron or librarian behaviors in the library

Examine patrons’ use of resources or tools

Finding a Topic: Usability

TYPES:

Exploratory: Conducted in early stages

Assessment: Conducted halfway

Verification: Conducted in the final stages

Comparison: Not dependent on stage

Observation Questions

Focus on how individuals do something, how well, how frequently and in what ways

Explore behavior and what people do in specific situations

Help gain better understanding of population

Discover user practices and actions – Help extend common knowledge base

Usability Questions

Focused on practical ways to fix problems; designing or redesigning products

Features problem statements that are testable

Moves the broader question to smaller, task focused questions

Usability questions lead to further problem solving questions

Defining the population within an observation and usability research study

Observation styles –

participant: observation in a natural environment over an extended time frame

reactive: participants within a very controlled study environment with a short duration time frame

unobtrusive: observation of participants in natural environment without their awareness

Participant sampling formats - event sampling : participants are observed in a

given activity and the how, when, where is monitored.

time sampling : participants are observed in given activity for a predetermined duration of time.

Usability - sample size: population controls that will vary

upon the observational study required. typical user style: creating baseline profiles with

the research study in mind as to guide researcher in the proper recruitment and selection of the participant pool.

Participant skill level: the wider the variety of skill levels within participant group, the more accurate and reliable data gathered

Demographic information: accurate information on participants involved will validate results reporting

Selecting A Research Design

Observation/Usability StudyBasic Elements

○ Develop problem statements or objectives○ Using representative sample users that don’t need

to be random○ Using testing tasks that represent the actual

workings of the product○ Controlled observation & questioning of

participants○ Collecting both quantitative and qualitative data to

measure product performance and user performance

Selecting A Research Design (Cont.) Tests employed

Usefulness○ Time on Task

Ease of use○ Accuracy

Ease of learning○ Recall Skills

Satisfaction level○ Emotional Response

to Tasks

Things to RememberKeep user in mindKeep tests simple but

elegantUsability is about

improving performance through design

Refine design and then retest

Gathering Datain Observation Studies

Record participants’ behaviors Coding Plan

clearly defined Codebooks

○ Written descriptions of events to be observed/recorded○ Should describe any rules and provide any instructions to

researchers

○ Incorporate any preexisting, external coding systems○ Researchers can develop their own coding and

analysis schemes Rely on the coding during the analysis phase

Gathering Datain Usability Studies

Rigid in data collection procedures Data collection procedures

Intake demographic survey Series of tasks Debriefing session Can also include: think-aloud protocols, screen capture

programs, worksheets, etc. Types of Tests

Card Sort Method Focus Groups Prototypes Individual Interview Usability tasks

INTERPRETING THE EVIDENCE

Observation

1. Frequencies

2. Percentages

3. Mean event

duration

Usability

4. Specify a level of success

5. Identify user error/confusion

6. Pinpoint & prioritize problems

Telling the Story

Observation & Usability

Traditional style:

1. Describe problem/posed question(s)

2. Literature review

3. Methods

4. Report findings

5. Make recommendations

Telling the Story

Observation & Usability

Visualizing findings

Limitations & problems

Usability Exclusive *-- Test instruments

Key Points

Focus on your end user

Observation is part of the methodology

Usability is a broad view that results

from mixed methods

Observation focuses on behavior,

usability is product-driven