Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

Innovative Media and Tools Track7th Annual International Symposium Emerging Technologies for Online Learning

(sponsored by Sloan-C and MERLOT)

Shalin Hai-JewApril 2014

2

Conference description

NVivo 10 is a widely used qualitative and mixed methods data analysis research tool used in many domain fields. This presentation will showcase some of the capabilities of this software by reviewing how to start a project in NVivo, how to manage complex data, how to begin coding raw information, and some of the methods for making meaning “in vivo” or “within the living” with various analyses and data visualizations. This presentation will include a demonstration of NCapture, a feature for data extraction from some social media platform APIs. For context, this session assumes some basic prior training in qualitative research, such as ethnographies, surveys, interviews, qualitative meta-analyses, portraiture, grounded theory, action research, Delphi studies, and / or others. The goals of this presentation are to introduce NVivo 10 as a data management and analysis tool for qualitative

and mixed methods research introduce the concept of multimodal data (from various types of

multimedia sources) show data ingestion into the system establish how data may be coded for research value show how data structures may be created from the coded data and then

analyzed demonstrate the extraction of a social media platform to open up another

data channel for research


3

Overview

Pre-story Types of qualitative and mixed

methods research General timeline of the research

process Quality standards Ingestion of data

Importation of data sets File types: multimedia

NVivo 10 (an overview) Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

4

Overview (cont.)

Data coding Folder and node data structures Codebooks Manual coding Two types of auto-coding Flexible coding structures Coding for matrix analysis Human-machine analysis of data


5

Overview (cont.)

Data coding (cont.)

Visual models Text search query ▪ Text search query and resulting word tree

Manual research journaling Automated ‘event log’ journaling

NCapture Essential mechanics My wants re: the software Contact and conclusion Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and

Analyze Data

6

Pre-story

Multi-year site license at Kansas State University Sociology, anthropology, social work, education,

communications studies, mass media, business, veterinary sciences, food sciences, family studies, human ecology, and others; university research office; multi-campus support ▪ Installed in student labs in multiple departments

Available to faculty, staff, administrators, and graduate students through protected website (with software download and a university-wide license key)

Y1 webinar trainings Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

7

Types of qualitative and mixed methods research

Grounded theory Heuristics / methods of

discovery Hermeneutics Life history Participatory action

research Phenomenology Ethnography Standard causal analysis

(SCA) Narratology; serious

narration; storytelling; historical narration

Social experimentation Formalization; analysis of

“stylized facts” Literary critique Market research Interviews Focus groups Surveys Delphi study; modified e-

Delphi study Actor-network analysis;

electronic social network analysis; social network analysis, and others


8

Types of qualitative and mixed methods research (cont.)

Mixed methods / multimethodology: qualitative research methods with quantitative research elements


9

General contrasts in approaches QUALITATIVE METHODS Relativism (with

attendant theories, tools, and research practices); post-structuralism and post-modernism

Non-measurability of phenomena

Narration Culturalism Emergentism Contextualization; social

constraints

QUANTITATIVE METHODS Positivism (with

attendant theories, tools, and research practices)

Measurability of phenomena

Analysis Behavioralism Individualism Non-contextualization;

independent agentry; open choices


10

General contrasts in approaches (cont.)

QUALITATIVE METHODS Situated knowledge Subjectivity Constructionism Experiential and

detailed Low n, case-based in vivo: “within the

living” (studying empirically in-world) (in Latin)

QUANTITATIVE METHODS Transcendent or

universal knowledge Objectivity Realism Statistical and numerical “Statistical significance”

/ relatively high n (and even “big data”)

in vitro: “within the glass” (experimental model) (in Latin)


11

General contrasts in approaches (cont.)

QUALITATIVE METHODS Rigor, saturation,

immersion, triangulation

Criticality of identification (and methodological mitigation) of researcher biases

Centrality of language Unclear end point

QUANTITATIVE METHODS Rejection of the null

hypothesis Criticality of nullifying

researcher biases through strict methodologies

Centrality of numbers Generally defined end

point


12

Mixed methods / multimethodology

Single approach designs, with augmentation of either qualitative or quantitative strategies to enhance research richness and outcomes (with a predominant qualitative or quantitative approach)

Mixed approach designs, to enhance construct validity for multiple analytical interests


13

General timeline of the research process

14

General timeline of the research process (cont.)

General steps: 1. Initial secondary and primary research2. Immersive research and documentation3. Coding and annotation4. Data analysis5. Write-up6. Dissemination

Flexible semi-recursive process


15

Quality standards

Various quality standards at every step (depending also on ontology and research methods) Protection of human participants (informed consent;

protection from harm; voluntary participation throughout; anonymity and confidentiality; beneficence)

Data saturation (comprehensive literature review) Data triangulation (multi-sourcing of data) Clear definitions of terms Address of researcher bias Inter-rater reliability; rater norming to standards Transparency in research methods


16

Quality standards (cont.)

Internal logic and non-contradiction; plausibility Falsifiability of hypotheses and assertions Logical analysis Bounded and delimited assertions Sufficient detail and depth Consistent and appropriate unit(s) of analyses Explanatory power Clear language Intellectual elegance and originality


17

Quality standards (cont.)

Puzzle solving Problem-solving Informative for decision-making Practical applications (decision-making;

policies) Data protection against leakage and

compromise


18

Ingestion of data

PRIMARY

Interviews Surveys (.xls, electronic

platform files) Focus groups Cases Observations Oral histories Text, photos, videos,

and audio files Social media platform

contents (NCapture)

SECONDARY

Literature reviews YouTube videos (links

only) Web pages Social media contents Grey literature


19

Importation of data sets

AUTOMATED (data import wizard, third-party tools like Survey Monkey) From electronic survey sites and tools ▪ Cells whited out for open-ended questions; grayed-out

cells for fixed response (attribution details) From social media platforms and sites (NCapture

in IE and Chrome and extracted into NVivo) From content providers From Excel (codebooks); SPSS; SAS, etc. From bibliographic data managers / citation tools


20

Importation of data sets (cont.)

MANUAL (drag-and-drop) From online libraries and repositories From Excel


21

File types: multimedia

Text files (.docx, .doc, .rtf, .txt, .pdf, and others)

Slideshows (.pptx, .ppt) Datasets (.xls, .rtf, .txt, .doc, and others) Websites captured as .pdfs (using NCapture

or other tools) Video files (and link-outs to video files on

content-sharing platforms) Audio files (and link-outs to audio files) Image files (virtually all digital image files)

and others… Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

22

File types: multimedia (cont.)

Requires manual transcription of audio and video files

Requires annotation Annotation of audio Annotation of video stills and video

segments


23

NVivo 10An overview


24

The software tool

Formerly NUDIST (non-numerical unstructured data indexing, searching, and theorizing)

Runs on PCs and parallels / virtual machines on Macs; professional Mac version forthcoming June 2014; mobile app access in development

Includes add-ins that enable extraction of social media platform data

Created by QSR International Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

http://www.qsrinternational.com/

25

Help supports

Tool as theory and research methodology “agnostic” and flexible May be a longitudinal many-year project or a short-term

one May be individual researcher-based or team-based

(without requirement for co-location, even with non-server based licensure because of data integration options)

Live support webinars with responsive presenters (for both broad audiences and organizations with site licenses, at additional cost)

Pre-recorded help webinars Textual help supports well documented


26

The work space


27

Project management

Project files saved as .nvp (QSR NVivo Projects) files Back up to server, cloud

service, external hard drive, or other device beyond main work device ▪ Coding is time-sensitive and

irrecoverable and impossible to recreate if there is data loss

May hover over file names at bottom left of opening screen to see the whole path of the files


28

Data coding


29

Folder and node data structures: a priori, emergent, or mixed

A PRIORI DATA STRUCTURE Pre-defined with

general constructs, data categories, and ideas (based on deductive logic and prior research)

A priori and emergent data structure

EMERGENT DATA STRUCTURE

Initially undefined structures with information extracted from the research documentation and researcher journal and other sources (based on inductive logic and interpretation of collected information); aka freecoding Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and

Analyze Data

MIXED DATA STRUCTURE

30

Points to consider in folder structuring and coding

Units of analysis Data manageability Relevance Ways to flexibly query the data using

the software Semantics and terminology

Should document and describe all nodes for coding clarity (particularly if there are teams working together on one project); codebook definitions, too


31

Node structure (example)

32

Codebooks

Definitions of codes and their meanings (often with examples) May be static (and predefined, partially defined

based on hypothesis and secondary research) May be evolving (and emergent and recorded in

codebooks along the way…but initially unstructured)

Digital or on paper May be used with one coder or a group

Inter-rater reliability (Kappa coefficient in coding comparisons between coders) and norming


33

Manual coding

Categories or classifications as nodes Textual / auditory / visual (still) / video

information highlighted and marked as belonging to a certain node Highlighting and dragging text (of text

or .pdf OCR-text) to node or right clicking and coding to node

Highlighting an area (of image or sound or video) and dragging to node or right clicking and coding to node


34

Two types of “auto-coding”

“STYLES” IN WORD

Ingestion and simultaneous autocoding of MS Word files with questions coded as headings and answers as paragraph styles (styles used to code a mass of data)

ALGORITHM-BASED

Machine-based emulation of qualitative / mixed methods researcher highlighting of contents into certain nodes

Machine-coded information clearly identified as such (with coding easily reversible)


35

Flexible coding structures Nodes used for

data structuring (and data visualizations) data clustering for analysis (for patterns and

researcher sense-making and understanding) further data querying pattern identification

Folders and nodes may be re-named, moved, annotated, regrouped, and changed at any time

Coding stripes to indicate density of coding and identify patterns

Can always uncode, revise, move nodes around, merge nodes, and even delete them


36

Coding for matrix analysis

May export to Microsoft Excel for larger range of data visualizations


Variable

Variable

Variable

Variable

Individual or Group (based on attributes)

”

Variable

Variable

Variable

Variable

Variable

Variable

37

Coding for matrix analysis (cont.)

Matrix analysis enabling ability to slice and dice data for findings and insights Within and between-group analysis Enabling comparisons of variables from

categories as rows and columns Enabling simple cell counts for

comparisons, anomaly detection, others (if relevant)


38

Coding for matrix analysis (cont.)

Code… Participants / interview subjects as own nodes Participants with demographic data (in

attributes and categorizing / classifying data fields such as age group, sex, and others)

Participants with close-ended query responses as attributes (such as Likert scale responses, Y/N responses)

Questions as own nodes (variables) Matrix analyses results saved in new

node(s) Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

39

Text analysis of Twitter microblogging extraction

Text frequency count (with a built-in and modifiable stop-words list) Word clouds (word size and

placement centrality = frequency)

Treemap (size = frequency) Cluster analysis (branch = co-

appearance)

40

Human-machine analysis of the data

Human observations and readings / interpretations of the research

Text searches Text frequency searches Machine-enhanced data queries Data visualizations and structures


41

Visual models

42

Visual models (cont.)

Node hierarchies in any direction (parent, child, etc.)

Types of layouts Circular Directed Hierarchical Orthogonal

Revealing of relationships between entities

More effective with sparse contents Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

43

Text search query

Words used in a group with “and” to indicate co-location in the same document (not in the same sentence or near-proximity)

Data management Find a document or identify an author Identify information for use in a quote

Use “summary” view to look at types of sources


44

Text search query and resulting word tree


45

Text search query and resulting word tree (cont.)

Enables intuitions and leads; provides a gist of the data

May click over a part of the branch to get its full phrasing

May request broader captures (than the narrow 5 words on either side of the focus term), termed “spread coding”

May put cursor over phrases in a tree to see the source (which is linked and may be accessed for further exploration)

Words in red are ones in initial text search query


46

Manual research journaling

Via built-in memo and Memo Links Via unique folder or node (and

variety of multimedia files)

47

Automated “event log” journaling

Apply real-time event logging to project


48

Annotation of documents

May highlight text for annotation (highlighted in blue and viewable from various approaches) Analyze -> Annotations -> New

Annotations Limit annotations because too much

blue highlighting will be ignored If too many annotations are needed,

that may indicate the need for a coding node instead Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and

Analyze Data

49

Tips re: support memos, nodes, and folders

Per Dr. Cynthia Jacobs (of QSR International) Keep a memo for justifying methods for the coding Set up a node for “I am struggling with how to code

this” issues for later reference Maintain a node for “good quotes” Keep a “to do” node list Do not delete nodes which you don’t plan to use,

but save them in a folder for possible future reference if needed

Use a “left off” node to remember where the coding was stopped, so you can find your place later


50

NCaptureAn add-on to Google Chrome or Internet Explorer browsers


51

Web-based data extractions

Websites (with or without ads)…as .pdfs

Twitter microblogs Facebook text and images LinkedIn text and images YouTube links

May require log-ins on the various platforms to access the social media platform APIs (application programming interfaces)

A sociometric online ethnographySetting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

52

Essential mechanics


53

Where NVivo comes in


Collection Coding Analysis Output

54

Manual equivalencies

Collection of data in folder and file structures

Manual coding and analysis of data electronically (or on paper) (cognitively taxing no matter what form) Wordle (for word clouds), Freemind (spatial

outlining), and others Notetaking in MS Word Manual modeling in MS Visio

Outputs for dissemination in MS Office Suite and other tools


55

My wants re: the software


56

Some wants

Wizarding and templating: Clearer setup for different types of qualitative and mixed methods research

Quant analysis features: A wider range of basic quant and statistical analysis packages for more of an all-in-one feel

A friendlier learning curve: Simpler semantics; more aggressive and interventionist help supports

A built-in source bibliographic data manager / citation tool (or an integrated freeware one): Not just EndNote, which is a commercial tool


57

Some wants (cont.)

A public repository for NVivo data sets (to be released with formal publications) A repository of research journals linked

to NVivo-managed data sets and archived research

Agreements to enable the release of copyrighted materials in NVivo-based data sets (for this limited purpose)


58

Contact and conclusion

Dr. Shalin Hai-Jew, Instructional Designer Information Technology Assistance Center

(iTAC) 212 Hale Library Kansas State University Manhattan, KS 785-532-5262 [email protected]

The presenter has no tie with QSR International.


mailto:[email protected]

Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data

Technology

Transcript of Setting Up a Qualitative or Mixed Methods Research Project in NVivo 10 to Code and Analyze Data