Chapter 18

Primary Studies:

Developing a Questionnaire

Overview

A questionnaire is a tool for systematically gathering information from study participants.

Questionnaires can be designed for self-reporting or as scripts for interviews.

FIGURE 18- 1 Questionnaire Design Plan

Questionnaire Content

It is often helpful to start with a list of the main categories of questions to be asked, and then to add

detail about the specific topics to be covered.

Questionnaire Content

• The questionnaire must include questions confirming that participants meet the eligibility criteria for the study.

• The questionnaire must also be able to accurately place participants into key categories, such confirming that all cases in a case-control study meet the case definition.

• A survey should be neither too short nor too long.

Types of Questions

After determining the broad categories of questions and the specific topics to be addressed in each section, each question topic should be assigned a specific type of question. •Close-ended questions allow a limited number of possible answers•Open-ended questions allow participants to give free-response answers

Open-Ended Questions

Open-ended questions allow participants to:•explain their selections and qualify their responses•give multiple answers•provide responses not anticipated by the researcher

However:•they take longer to ask and answer•they may result in irrelevant answers•recoding for statistical analysis is often difficult

Close-Ended Questions

Close-ended questions come in a variety of formats, including:•Date and time variables •Numeric variables•Categorical variables•Paired-comparison variables•Rank-order variables

Close-Ended Questions

Categorical variables come in a variety of formats:•Dichotomous variables have only two response options (like yes/no)•Ordinal variables are ranked based on an inherent order•Nominal variables are have no built-in order

FIGURE 18-3

Examples of Types

of Questions

Anonymity

• Many questions can be asked in more than one valid way, and the researcher must decide which question type is most appropriate and will best protect participants’ anonymity.

• If a name, an address, a birthdate, or other information could link a participant to the study, then there must be a solid plan in place for protecting the privacy of participants and the confidentiality of the information they share.

Types of Responses

Once the types of questions have been selected, a decision must be made about the kinds of responses that are appropriate for the question.

Types of Responses

• For numeric responses, the question should state exactly how specific the answers should be.

• For categorical questions, consider all possible responses for each question, and include as many as needed (including, perhaps, “other” and “I do not know”).

• For ranked questions, decide how many entries to include on the scale and whether there will be a neutral option.

FIGURE 18-4 Examples of Responses for Ranked Questions

Wording of Questions

After drafting the questionnaire, check each question for clarity: •Does each question ask what it is intended to ask? •Is the language of each question clear and neutral? •Will members of the study population understand the language? •Is the question sensitive to potential cultural issues related to language?

Wording of Questions

Check to be sure that the responses are carefully worded:•Is the choice of response clear? •For scaled questions, is the rank order clear? (For example, is it clear that 1 is “strongly disagree” and 5 is “strongly agree”?)•For questions with unranked categories, is the order of possible responses alphabetical or otherwise neutral?

FIGURE 18-5 Problems to Avoid

Order of Questions

• Many questionnaires start with easy or at least general questions before moving to more difficult or sensitive questions.

• The questions should be in an order that flow naturally from one topic to another, and similar questions should be grouped.

• Think carefully about how previous questions could taint the answers to later ones.

Layout and Formatting

The layout of the survey instrument will vary depending on the mechanism of data collection used. •A self-report survey, either paper-based or computer-based, may require instructions about how to indicate answers.•For an oral survey, the interview script requires an opening statement, transitions between sections of the survey, closing sentences, spaces for recording responses to questions, and other instructions.

FIGURE 18-6 Example of a Self-Reported Questionnaire

FIGURE 18-7 Example of a Telephone Interview Script

Validation

A valid questionnaire measures what it was intended to measure in the population being assessed. •One way to seek validity is to include survey questions or modules that are identical to the ones used in previous research projects, but access to survey questions is often not possible in the health sciences. •Pilot testing of the new questionnaire is essential for the development of a valid and useful tool.

Commercial Research Tools

A number of widely used and validated tests, primarily from the social sciences, are available to researchers as commercial products.•Some of these tools are free of charge, but most require payment for use. •Some tests provide the instrument for free but require researchers to pay to have the results scored and validated against previous users of the survey instrument.

Translation

Translation of the survey instrument into one or more additional languages may be necessary.•Check to be sure that the translated version expresses the same meaning as the original survey. •Accuracy may require the rephrasing of whole sentences, not just direct word-for-word translations.

Pilot Testing

A pilot test (pretest) of the questionnaire is helpful for checking content, clarity, layout, timing, and other factors. •Volunteers from the target population (but not the sample population) should be asked to complete the preliminary survey and provide feedback on it.•The survey instrument should be revised based on these observations. •Several rounds of pilot testing may be required.