©2010 John Wiley and Sons Chapter 11 Research Methods in Human-Computer Interaction Chapter 11-...
-
Upload
jasmine-dixon -
Category
Documents
-
view
223 -
download
3
Transcript of ©2010 John Wiley and Sons Chapter 11 Research Methods in Human-Computer Interaction Chapter 11-...
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Research Methods inHuman-Computer Interaction
Chapter 11-Analyzing
Qualitative Data
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Overview
• Introduction• Stages of qualitative analysis• Grounded theory• Content analysis• Analyzing text content• Analyzing multimedia content
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Stages of qualitative analysis
• Identify components of the substance• Study properties and dimensions of each
component• Understand and make inference about the
substance
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Grounded theory
• An inductive research method
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Procedures of grounded theory
• open coding• development of concepts• grouping concepts into categories• formation of a theory
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Grounded theory
• Advantages– a systematic approach to analyzing qualitative,
mostly text-based, data,– generating theory out of qualitative data that can
be backed up by ample evidence of the coding– Interplay between data collection and analysis
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Grounded theory
• Disadvantages– Researcher can be overwhelmed by the details of
the data– The theory generated is hard to evaluate– Findings may be subject to bias
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Content analysis
• A more specific view: a systematic, replicable technique for compressing many words of text into fewer content categories based on explicit rules of coding
• A broader view: any technique for making inferences by objectively and systematically identifying specified characteristics of messages
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Content categories
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Preparing for content analysis
• Define the data set• Define the population• Clean up the data• Understand the context of the data
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Analyzing text data
• A priori coding– Identify coding categories– Coding– Reliability check
• Emergent coding– Multiple coders identify coding categories based on subset of data– Consolidate category list– Code a subset of data– Reliability check– Repeat the process until satisfactory result is met– Code the rest of the data
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Identify coding categories
• Theoretical framework• Researcher-denoted concepts• In-vivo codes• Building a code structure (nomenclature)
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Coding the text
• Look for key items
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Coding the text
• Ask questions about the data• Making comparisons
– Between different coding category– Between different participant group– Between existing data and previous literature
• Using computer software
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Ensure high quality analysis
• Validity– constructing a multi-faceted argument in favor of
your interpretation of the data– Constructing a database– Data source triangulation– Interpretation should account for as much as
possible of the data– Alternative interpretations may also help
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Ensure high quality analysis
• Reliability check– Stability
• also called intra-coder reliability• examines whether the same coder rates the data in the
same way throughout the coding process
– Reproducibility• also called inter-coder reliability or investigator
triangulation • examines whether different coders code the same data
in a consistent way
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Reliability measures
• Percent agreement
• Cohen’s Kappa:
K = (Pa − Pc)/(1 − Pc)
% Agreement =
Number of cases coded the same way
Total number of cases
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Reliability check
• Agreement matrix
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Reliability check
• Interpretation of Cohen’s Kappa
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Subjective vs. objective coder• Subjective coders
– Knowledge and experience can help interpret the data
– Less training required– May cause inflated reliability
• Objective coders– Less likely to cause inflated reliability– Lack of knowledge affect the ability to understand
the data– More training required
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
Analyzing multimedia content• The supporting techniques and methods are less mature
compared to text data analysis• Approaches:
– Manual analysis: • highly labor intensive and time consuming• More accurate
– Completely automated analysis: • Faster, less amount of work• Highly inaccurate
– Partially automated approach• Combines the advantages of the manual process and the
completely automated process
©2010 John Wiley and Sons www.wileyeurope.com/college/lazar Chapter 11
End-of-chapter
• Summary• Discussion questions• Research design exercise