Growing pains: Launching our first longitudinal study

Growing painsLeanna Gingras - @leegoesplacesInternet User Experience 2011

Hi.

IUE October 2011 @leegoesplaces


Grab ‘n Go Gus

Savvy Sally

“that’s not it…that’s not it either…”

“this looks good enough.”


1. Our research need

http://www.flickr.com/photos/lwr/2446626352/sizes/l/in/photostream/

Grab ‘n Go Gus

“this looks good enough.”


Selecting our tools


“A longitudinal study is a correlational research study that involves repeated observations of the same variables over long periods of time.”

--Wikipedia

http://dux.typepad.com/dux/2009/05/usability-over-time-longitudinal-research-studies.html“I participated in the "Best Practices in Longitudinal Research" workshop at CHI 2009 with a small group of academics and practitioners from five countries. … Here's a brief sampling of a few topics from the day's discussions.”

User experience over time: An initial framework“This paper presents an in-depth, five-week ethnographic study that followed 6 individuals during an actual purchase of the Apple iPhone". We found prolonged use to be motivated by different qualities than the ones that provided positive initial experiences.”

(and many abstracts and workshops)


UX resources are sparse

http://dux.typepad.com/dux/2009/05/usability-over-time-longitudinal-research-studies.html



http://portal.acm.org/citation.cfm?id=1518701.1518814&coll=DL&dl=GUIDE&CFID=1032851&CFTOKEN=56759960

http://portal.acm.org/citation.cfm?id=1518701.1518814&coll=DL&dl=GUIDE&CFID=1032851&CFTOKEN=56759960

To learn how to retain participants: Minimizing respondent attrition in longitudinal research: Practical implications from a cohort study of adolescent drinking

More on retaining participants: Retaning Participants in Longitudinal Community Research: A Comprehensive Protocol

Examples of how much to pay: Psychology Department at the University of Vermont


Psychology resources galore

http://www.sciencedirect.com/science/article/pii/S0140197103000113



http://jab.sagepub.com.proxy.lib.umich.edu/content/32/3/262.short



http://www.uvm.edu/~psych/?Page=research_studies_list.html&SM=studentopportunitiessubmenu.html

2. Planning

http://www.flickr.com/photos/dacotahsgirl/299198494/sizes/o/in/photostream/ IUE October 2011 @leegoesplaces

“The goal of this study is to learn how a novice researcher’s research patterns and strategies change over time.”


Decide the research question



Decide who to study

“All of it.”


Decide what data to collect

“We will be collecting data from usability tests, interviews, Twitter entries, diary entries, and focus group activities.”


Decide what data to collect

1 - Initial interview & usability test1 - Contact the recruits for longitudinal study1 - Orientation, 1 – Follow-up email 3 - check-in emails1 - Mid-semester interview based on tweets1 - End-semester interview1 - Focus group


Plan touchpoints and activities


Plan compensation strategy


Plan retention strategy

http://www.flickr.com/photos/goincase/2720494170/sizes/o/in/photostream/

3. Execution



The actual study

1 usability test10-30 tweetsAt least 3 diary entries3 in-person interviewsFocus group


Screener usability tests

Agenda:

• Explain study expectations and compensation

• Plan dates for touchpoints throughout semester


Kickoff meeting


Twitter guidelines


Diary due dates


Diary entries

Standard set of questions:“Tell me about your toughest moment.”

Observations from Twitter/diary to build interview from:“Does she understand where to find scholarly stuff?”

Exercise:“Let’s revisit one of your earlier Tweets. Walk me through finding resources on Faust. Is there anything you’d do differently?”


Interviews

“so how did your research change over the semester?”


End of semester focus group

“so how did your research change over the semester?”


End of semester workshop


Building activities


Articulating activities

YAY! We’re done, right?

http://www.flickr.com/photos/markfive/4121088992/ IUE October 2011 @leegoesplaces


4. Analysis

• Does she understand where to get academic content?

• What keywords did she try?

http://www.island94.org/2009/09/boston-subway-in-vector-format-svg/

Analyze as we go


Asked specific questions of the data


What do they learn? What tools and hacks are useful to them and why? What do they hate and why?


“What do they hate and why?”


Archived everything


What’d we find out?


Finding: Emotions run high


Finding: Wikipedia is a broad resource


Finding: Resources drive research


Yeah, but what about the deliverables?


Cross-unit brainstorming


Persona-building

Project steering


Recycle, recycle, recycle



5. Lessons learned


Study management takes time

Retain ‘em


Nag, nag, nag


Make it easy for them


Adapt.


1. Define research need2. Planning 3. Execution4. Analysis and findings5. Lessons learned


Questions?

http://www.flickr.com/photos/picken/5694078937/sizes/o/in/photostream/ IUE October 2011 @leegoesplaces

Leanna [email protected]

Thanks

Growing pains: Launching our first longitudinal study

Business

Transcript of Growing pains: Launching our first longitudinal study