THE Q-SUITE: Q-BANK AND Q-NOTES

Justin Mezetin and Meredith Massey

NCHS Questionnaire Design Research Laboratory

FedCASIC 2014: Survey Evaluation Methods

March 19, 2014

Outline

Introduction Analytical Framework Q-Notes: Qualitative Analysis software Q-Bank: Question Evaluation library and

resource Next Steps

Introduction

What is cognitive interviewing/question evaluation?

Benefits of Cognitive Interviewing

Validity studies showing all phenomena the question is capturing

Analytical Framework

Background Challenges

Time consuming to conduct a thorough analysis

Larger sample sizes (N > 20)Multiple projects with over 100-150 interviews

CollaborationMultiple Interview LocationsResearchers with varying levels of

experienceTesting in Multiple Languages

Q-NOTES: ANALYSIS

SOFTWARE

Level One: Conducting Interviews Interview guide Consistency Pre-selected

Themes Multiple Languages Raw data storage

Interview Guide

Question Translation

Level Two: Summarizing Interview Notes Synthesizing raw

data Tagged videos Researcher

collaboration Viewing data for a

single respondent

Summarizing Interview Notes

Level Three: Comparing Across Respondents Developing thematic

schema Project level themes Question level

themes Flexibility Consistency across

coders Cross interview

comparison

Viewing one question across all respondents

Level Four: Comparing Across Groups Development of

Advanced Schema Sub-group analysis Comparative

analysis

Subgroup Analysis

Subgroup Analysis Results

Comparative Analysis

Level Five: Drawing Conclusions

Report template Summary tables Thematic schema

Report Template

Q-BANK: DISSEMINATING

FINDINGS

Q-Bank Users

Data Analysts Questionnaire Designers Survey Methodologists Survey Managers

Q-Bank Benefits

Deeper understanding of what survey variables are capturing

Share/view findings from studies conducted at other organizations

Resource to learn more and dialog about Question Evaluation

Mechanism to compare different methods and findings

Searching Q-Bank

Q-Bank Search Results

Question Selection

Question Details

Q-Suite Features

Collaboration Consistency Transparency

Q-Notes: Qualitative Analysis Software Q-Bank:

Library of Evaluation studiesResource on Question Evaluation

Q-Suite Next Steps

Q-NotesRun a public beta test Summer 2014*Export feature to submit into Q-Bank

Q-BankContinue increasing content Inclusion of all methods in Q-BankPilot test with NHIS to link key variables to

Q-Bank testing reports

Questions or Feedback?

Contact Us:QBank@cdc.gov

Visit:www.cdc.gov/QBankwww.cdc.gov/QNotes