RESEARCH CONTROL

Research Control

Control that the investigator puts on the research situation and the subjects

Control extraneous variables to determine the true relationship between the IV and DV under investigation

Extraneous Variables

A variable that confounds the relationship between the IV and DV and that needs to be controlled in the research design or through statistical procedures

The most important question to ask is:

What variables may affect the dependent variable besides the independent variable?

Example

What may affect readmission to the acute care facility BESIDES adequacy of home care? Look at the literature Own clinical knowledge

Extraneous Variables

External- factors stemming from the research situation

Intrinsic- factors that are inherent to the subjects in the research study

Controlling the Research Situation (External Factors) What can the researcher do to control

the research situation itself? Minimize situational contaminants To make conditions under which the data

are collected as similar as possible for every participant in the study

Controlling the Research Situation (External Factors) Constancy of condition

Environmental context within which the study is to be conducted

Consider the setting type and emotional and behavioral aspects of the setting

Avoid contamination of treatment (control group and experimental group sharing with each other)

Controlling the Research Situation (External Factors) Timing

Time of day or year Strive for constancy of time for all the

subjects May or may not be critical to the variables

Controlling the Research Situation (External Factors) Communications

Information about the purpose, how data collected, etc

Prepare in advance Same message to all participants with little

ad libbing Given by the same person(s) whenever

possible

Controlling the Research Situation (External Factors) Research Protocols

Written down in great detail Done by as few as possible trained

research assistants Adhered to specifications

Controlling Subject Characteristics (Intrinsic Factors) Randomization

Most effective way to control Secure comparable groups Equalize the groups with respect to the

extraneous variables

Controls all possible sources of extraneous variation without any conscious decision by the researcher about which variables need to be controlled

Expect that the experimental group and control group are comparable

Controlling Subject Characteristics (Intrinsic Factors)

Repeated Measures or Crossover Design Same subjects serve as both control group

and experimental group Problem: carryover effect Ordering of the treatment is important

Use randomized ordering

Controlling Subject Characteristics (Intrinsic Factors) Homogeneity

Use of subjects who are homogeneous with respect to variables that are considered extraneous

Achieved through determining what possible variable influence the DV besides the IV

Determine eligibility and ineligibility criteria

Controlling Subject Characteristics (Intrinsic Factors) Blocking

Controlling extraneous variable by including them in the design as a IV

Random assignment Table 11.1 page 291 Mostly used for experimental designs but

can be used for non experimental

Matching Statistical Control

Analysis of Covariance

Problems

Except for randomization, the researcher must know in advance what the extraneous variable are

Can practically only deal with 1-3 extraneous variables at a time (except analysis of covariance)

Validity

Does the design do the best possible job of providing trustworthy answers to the research question? Statistical Conclusion Validity Internal Validity Construct Validity External Validity

Validity

Statistical Conclusion Validity PowerPrecision

Answers the question: What is the strength of evidence that a relationship exists between the two variables.

Statistical Conclusion Validity Power -- ability of the design to detect

true relationships among the variables Sample Size Definition of the IV

Statistical Conclusion Validity Precision

Accurate measuring tools Controls over extraneous variables Statistical methods

Validity

Internal Validity Extent to which it is possible to make an

inference that the IV is truly influencing the DV and that the relationship is not spurious

Answers the question If a relationship exists, what is the strength of the evidence that IV of interest rather than extraneous factors caused the outcome?

Threats to Internal Validity

The nature of the true experiment lends itself to a high degree of internal validity because of manipulation and randomization

All other design the researcher must contend with competing explanations for the obtained results (internal validity)

Threats to Internal Validity

History - occurrence of external events that take place concurrently with the IV that can affect the dependent variable of interest In a true experiment can assume that it

affects both the control group and the experimental group similarly

Threats to Internal Validity

Selection- biases resulting from preexisting differences between the group Group differences on the DV are the result

of the groups not being equivalent in the first place rather than the effect of the IV

One of the most common, problematic threats to internal validity

Threats to Internal Validity

Maturation Processes occurring within the subjects

during the course of the study as a result of the passage of time rather than as a result of treatment or independent variable.

Threats to Internal Validity

Testing effect- the effect of the pretest on the participant’s performance on a post test The mere collecting of information can

change the subject’s attitude, behavior, etc

Threats to Internal Validity

Instrumentation- changes in the researcher’s measuring instruments between an initial point of data collection and a subsequent point Keep the pre and post test the same Consider changes in the people collecting

the data, subject’s attitude toward the instruments on post test

Threats to Internal Validity

Mortality- differential attrition from the groups being compared Becomes a consideration in the initial

design of the study If the attrition is random (those dropping

out and those staying are similar bias is low

More than 20% attrition is a concern

Validity

External Validity achieved when the results can be confidently generalized to situations outside the specific research setting

Answers the question: If the relationship is plausibly causal, what is the strength of the evidence that the relationship is generalized across people, settings, and time?

External Validity

Refers to the generalizability of the research findings to other settings or samples Aim of research is discover enduring

relationships To what populations, environments, and

conditions can the results of this study be applied?

External Validity

Sampling design must be adequate “Strictly speaking the findings of a study

can only be generalized to the population of subjects from which a study sample has been selected at random.”

Threats to External Validity

Environmental or research situations that affect the study’ representativeness

Hawthorne Effect Novelty Effect Interaction of History and Treatment Effect Experimenter Effects Measurement Effects

Hawthorne Effect Subjects may behave in a certain manner

simply because they know they are being studied

Thus you cannot generalize to a more natural setting

Novelty Effect New treatment might cause excitement of

skepticism and thus alter behavior

Interaction of history and treatment effect Impact of the treatment AND some other

event external to the study

Experimenter effect Subject’s reaction to the researcher

Measurement Effect Collection of data and attention from

having all the data collected