Observation and Usability Studies

Ericka BrunsonDorothy HirschMollee MarchJoanna McCloudDenise Tiller

Focus on human behavior & actions

Part of mixed-methods approach

Naturalistic: how people really behave

What they have in common

Test validity of other findings

Study specific processes or behaviors

Access behaviors of those unable to communicate

Observation studies can

Three types of studies

Participant Observation

Reactive Observation

Unobtrusive Observation

Study reference transactions

Analyze patron/librarian behavior

Study patron use of resources and tools

Library uses

a thorough study over time

conducted in a natural environment

when the observer becomes part of the group being studied

rarely used by libraries because of expense in time and money

the basis of most cultural anthropological/ethnology group studies

Participant observation is

Only studies behaviors, not cognition

Prone to research bias/misconceptions

Can be expensive, both in time and money

Ethical considerations

Limitations

How information is obtained

What is done with the information

Do no harm: emotionally mentally socially economically

Ethical issues

Are you watching me right now…or am I watching you?

Reactive observation

A “real world” observational research method that studies individuals and groups of people in a natural, controlled setting

Participants are aware that they are being studied.

What is reactive observation?

Behavior reflects aspects of social desirability

Observations can be manipulated to fit the research design

Reactive observation positives

Individuals are reacting to observation

External validity

Reactive observation negatives

Continuous monitoring

Time allocation

Common methods of reactive observation

Involves studying the behavior of participants and recording as much as possible.

Works best in organizational settings such as:the workplaceeducational instituteswhen monitoring nutrition (exercise, food

intake, etc.)

Continuous monitoring

The “Hawthorne Effect” states that “workers react to the attention they are getting from the researchers and in turn their productivity increases.”

The “Hawthorne Effect”

Done at random

Researcher will choose the time and place to record activities before the researcher’s presence is known

Time allocation

Descriptive

Inferential

Evaluative

Variables to consider in evaluating observations

Subjects are unaware that they are being observed and researchers do not intrude on the study.

Unobtrusive measures

Behavior trace studies

Disguised field observations

Types of measures

Natural behavior of subjects

Reduces bias

Unobtrusive positives

Researcher has little control over type of data collected

Not always a usable method

Subjects unaware of data being gathered

Other ethical issues

Unobtrusive negatives

Tests system performance with typical users

Many library applications

Quick, easy, inexpensive

Usability studies - a bright idea

Exploratory – early stage using screen shots and paper visuals

Assessment – mid-point focusing on the actual product

Verification – final stage to ensure product meets standards

Comparison – used in any stage to compare design options

Four types of usability studies

Usefulness

Ease of learning

Ease of use

User satisfaction

Usability studies test

Card sort

Prototypes

Usability tasks

Focus groups

Individual interviews

Methods used to collect data

Type of study Exploratory

Purpose Test JoCo Library Children’s Web site

Methods used Prototypes, usability tasks, focus group

Participants Eleven children ages 6 to 11

Examples of library usability studies

Johnson County Public LibrariesJoCoKids Usability Study

April 25, 2009

Type of study Assessment

Purpose Test new web site

Methods used Prototypes, usability tasks, focus group

Participants Five mothers with children under age 6

Johnson County Public Libraries6 by 6 Usability StudySeptember 1-2, 2009

Step 1Let participants explore the prototype web site and record comments

Step 2Usability tasks

6 by 6 usability study

Type of study Verification

Purpose Verify usability of new paper application

Methods used Usability task

Participants 10 patrons

Johnson County Public LibrariesNew Paper Application Card Usability

StudyJune 16, 2009

Anschutz Library Renovation Project

2009-2010

Dec Emailed survey to faculty, staff and students

Mar-Sept Redesigned the main library floor and created the Learning Studio

Oct “Snapshot Day”

Beck, S.E. & Manuel,K. (2008). Observation and usability. In Practical research methods for librarians and information professionals. New York, N.Y.: Neal-Schuman Publishers, Inc.

Brown, L. & Trochim, W.M. (2006). Research methods knowledge base. Retrieved from Cornell University website: http://researchmethods.net.

Brown, L. (n.d.). Observational ield Research [Online tutorial]. Retrieved from Cornell University Library website: http://www.socialresearchmethods.net/tutorial/Brown/lauratp.htm.

Hendry, J. (1999). Other people's worlds: An introduction to cultural and social anthroplogy. New York, New York: Washington Square University Press.

Kumar, K. (1996). Using direct observation techniques [ CDIE Tips Series,4]. Retrieved from USAID Evaluation Publications website: http://pdf.usaid.gov/pdf_docs/PNABY208.pdf.

Lee, R.M. (2000). Introduction to unobtrusive methods. In Unobtrusive methods in social research, pp. 1-16.

Norlin, E. (2000). Reference evaluation: a three strep approach – surveys, unobtrusive observations, and focus groups. College and Research Libraries, 61(6),pp. 546-553.

Paul, B.D. (1953). Interview technigues and field relationships. In Kroeber, A.L. (1953). Anthropology Today: an encyclopedic inventory. Chicago, Illinois: University of Chicago Press.

Radin, P. (1966). The method and theory of ethnology: An essay in criticism[2nd ed.] New york and London, England: Basic Books. Reece, R.D. & Siegal, H. A. (1986). Studying people: A primer in the ethics of social research. Macon, Georgia: Mercer University Press.

Romanczyk, R.G. ,Kent, R.N., Diament, C., & O'Leary, D. (1973). Measuring the reliability of observational data: a reactive process. Journal of Applied Behavior Analysis, 6(1), 175-184.

Trochim, W.M. The research methods knowledge base, [2nd ed.]. Retrieved from: http://www.socialresearchmethods.net/kb/. Last revised: 10/20/2010.

Usability studies at Johnson County Libraries (n.d.) Retrieved from: http://jocolibrary.org/templates/JCL_InfoPage.aspx?id=3063&epslanguages=EN.

References