Diaries KSE966/986 Seminar - Fall 2012/Spring 2013 March 29, 2013 Uichin Lee.
-
Upload
primrose-shields -
Category
Documents
-
view
213 -
download
0
Transcript of Diaries KSE966/986 Seminar - Fall 2012/Spring 2013 March 29, 2013 Uichin Lee.
Diaries
KSE966/986 Seminar - Fall 2012/Spring 2013 March 29, 2013Uichin Lee
ContentsWhy use diaries in HCI?Challenges with diariesParticipants for a diary studyTypes of Diaries (feedback,
elicitation)Data collection When to record an entry?Analysis of diaries
What is a diary?A diary is a document created by an
individual who maintains regular recordings about events in their life, at the time that those events occur
Diaries are useful for recording information that is fluid and changes over time◦User mood, feelings, such as frustration, anger,
and pain, (mobile) information needs (any types of multiple events occurred within a day)
If recall after-the-fact will be hard, inaccurate, or subject to biases, diaries can be a useful data collection method
What is a diary?Informal diaries:
◦Blogs, status updates using Facebook, or “tweets” on Twitter
◦Entries are made as they occur, in real-time (although there are no stated research purposes)
Diaries have been adopted, from sociology and history, for use in human-computer interaction research
What is a diary?Example: time diary
◦A time diary is when there is a focus on how time is used, and entries need to be made on a regular basis (with a specified dimension of time)
◦Time diaries are useful, because people often remember time inaccurately
Diary is a useful approach as much of research in HCI focuses on understanding how we interact software and hardware artifacts in our daily lives
Why use diaries in HCI?Diaries fill the gap between observation in
naturalistic settings and fixed laboratory settings, and surveys
Users may have different reactions when being observed, and observers may not always understand what is going on
If interested in collecting data that is fluid and changes over time (rather than factual data), surveys can lead to biased data due to biases in recall
Multi-method research is often the best approach◦ as all methods have strengths and weaknesses
Why use diaries in HCI? Diaries allow for more collecting more detailed research
data than surveys Survey has predefined questions w/ little flexibility given
to the respondents ◦ Mostly appropriate if we ask users to recall info that does not
change over time Diaries are good at understanding not only what users
are doing, but why they are doing it Dairies are useful for collecting:
◦ Fluid data: occurring only at a specific time, and changing, e.g., mood, feelings, perception, time, or response
◦ Responses: needing a very short time period between the occurrence of the event and the recording of the event
Time is an important dimension, because asking users to recall after-the-fact how much time they spent or wasted will lead to inaccurate answers
Why use diaries in HCI? Useful if data cannot be accurately collected
as we do in experimental research (measuring human performance), or observational means (e.g., automatic data collection)
Survey/diary: ask users about themselves◦ How did they perceive a certain experience with
the computer or device?◦ How did they feel?◦ How did they respond?◦ How much time did it take them?◦ How did it impact on their mood?◦ When did they use it?
Why use diaries in HCI? Diaries can investigate the use of
technology that exists in multiple stages◦Tech that does not exist but could
(where researchers investigate communication or info usage patterns, separate from the tech)
◦Tech that exists but needs to be improved (how people use existing technology)
◦Prototypes of new tech that need to be evaluated
Why use diaries in HCI?Diaries are good for recording user-
defined incidents◦When users intended to perform a task,
but decided not to do so◦When users feel that they have learned
something newDiaries are also good at researching
situations where users move around and don’t stay in one place◦Mobile phones, GPS devices, hand-held
tech
Challenges with diariesUsers sometimes are not introspective
and are not even aware of the specifics of what they are doing and therefore may have trouble recording it in a diary entry
Users may not follow through and record (via paper or electronic) a sufficient number of entries
Time recording may still be less accurate for time diaries than for controlled laboratory setting or automated data collection
Challenges with diariesGenerally harder to recruit users
for a diary study than for something less intrusive, like a survey
Since data are both qualitative and quantitative, data analysis may take a long time
Hard to strike a balance between a frequent-enough series of diary entries, and infringement on daily activities (user participation may then trail off)
Participants for a diary studyLike any other type of research, prior
planning and testing are a must to ensure a valid outcome
First step is to determine who will take part in the study (e.g., demographic, education, computer experience)
Try to get a representative group of participants, but it’s more important to have users who can provide useful insight
Potential diarists must understand the purpose, be motivated and have competence in using any required technology for diary entries
Participants for a diary studyDiary study:
◦Must be structured so that it yields useful data, without imposing an unreasonable burden on users e.g., employment, health, relationships with
others◦Should not negatively impact on
employment, health, or relationshipsParticipants:
◦Should be paid for taking part in the diary study
◦Need to be informed of their rights, including to remain anonymous
Types of DiariesFeedback diary: the data recorded in the
diary is itself the purpose of the research (i.e., meaningful to the researcher)◦Users make entries when a certain event or
threshold occurs, or on a stated time basis
Elicitation diary: the users record only basic information about important events occurring in their day (i.e., meaningful to the user; also the researcher)◦These data points are used as prompts for
expansion at a later time
Types of Diaries: Feedback diaryHow often a diary entry is made?
◦ What event, time, or threshold triggers?Examples:
When you feel frustrated with an interface When you completed a certain task
Users can also make a recording at a set time every day (say 9PM), or during a specific time period (say 12-6PM)
Feedback diaries can be structured or unstructured◦ Likert scales, checkboxes, time recording (e.g., every 15
minutes)◦ General reflection (“How are you feeling right now about
your computer?; How did you respond? How did you think the tech could be improved?”)
Types of Diaries: Elicitation diaryData points recorded in an
elicitation diary are quick and simple◦Short snippets of text◦Short audio records◦Short video clips or pictures
Should not interrupt what users were doing
Then, later (in an interview, on a web site, or other format), users expand on their entries, describing in more detail
Elicitation diary (examples)Development of a new handheld document
scanner (Brown et al. 2000)◦ 22 users were asked to take a photo any time they
felt that there was a document they would like to capture
◦ Over 7 days, they made 381 entries◦ Pictures were then used during a series of semi-
structured interviews to prompt users Mobile information seeking (Sohn et al., 2008)
◦ Users were asked to send in a short text message, identifying when they had an info need
◦ This short text entry is then used to remind the users of answering a series of questions on a web site (e.g., Where were you? What were you doing?)
Types of Diaries: Hybrid approachExample: student’s use of
transportation (Carter and Mankoff, 2005)◦For two weeks, users were asked to
call a specific number every time they made a transit decision (location is automatically captured using GPS)
◦During a later interview, the users were presented with the recordings and were prompted to expand their thoughts on specific decisions
Cf) Experience Sampling Capture participants’ in-situ responses to hypothetic
requests◦ Precisely controlled timing◦ Tracking compliance◦ Characteristics of rating behavior (w/ additional data collection)◦ Reduction of human error (in data management)
Example: understanding location disclosure (Consolvo et al., CHI 2005)◦ Phase1: identify a buddy list ◦ Phase 2: participants receive 10 randomly timed
questionnaires everyday for two weeks from 9AM-9PM on weekdays; 10AM-10PM on weekends Questionnaire: Context (Where are you? What are you doing? With
whom are you?) + What the user wants to disclose about location (based on the hypothetical request from a social relation on the buddy list)
◦ Phase 3: participants reflected their experiences in Phase 1&2
Data collectionHow will the diaries be recorded?
◦Paper?◦Electronic?◦Voice?◦Smart phones?
Often now, technology is being used to record diary entries
Crucial to choose the media that will be most natural for the diarists in their everyday life
Two weeks is often an appropriate length of time for a diary study
When to record an entry?Participants should be given clear
guidance on when to perform an entry in the diary◦What activities are of interest?◦What events, incidents, or feelings should
result in a diary entry?◦How often should diary entries be made?◦How detailed should the entries be?
Make sure NOT to pay participants based on the number of diary entries◦Any payment should be for regular
participation but should not be linked directly to # entries
When to record an entry?If diary reports are turned in during
the study period, monitor the incoming reports (e.g., who’s not reporting or not providing useful data)
Reminders and feedback can be sent during the period of the diary study◦e.g., 5 text messages were sent a day to
remind users to send in diary entries (Sohn et al. 2005)
Analysis of diariesTransfer all records to an easy-to-
analyze electronic formatDo statistical analysis on
quantitative dataPrepare and examine qualitative
data, potentially do a content analysis
With qualitative data, you can contact the participants after the fact to see if your interpretations are correct
SummaryWhy use diaries in HCI?Challenges with diariesParticipants for a diary studyTypes of Diaries (feedback,
elicitation)Data collection When to record an entry?Analysis of diaries