2. Research Design-LDR 280.PPT

8/10/2019 2. Research Design-LDR 280.PPT

1/25

1

RESEARCH DESIGN


2/25

2

What--What was studied? What about--What aspects of

the subject were studied? What for--What is/was the

significance of the study?

What did prior lit./research say?

What was done--How was thestudy conducted?

What was found? So what? What now?

1. Introduction,

Research Problems/

Objectives, &Justification

2. Literature Review

3. Methodology(Research sample, datacollection, measurement,data analysis)

4. Results & Discussion

5. Implications

6. Conclusions and

Recommendations forFuture Research

PROCESS

OF DESIGNING AND CONDUCTING ARESEARCH PROJECT:


3/25

3

RESEARCH DESIGN

RESEARCH DESIGNrefers to the plan, structure, andstrategy of research--the blueprint that will guide the

research process.

Developing ResearchHypotheses

Intriguing Observation,

Intellectual Curiosity

Defining Research

Problem & Objectives

Testing Hypo.:

Data Analysis &Interpretation

Sampling Design

Refinement of theory

(Inductive Reasoning)

Data Coding,

And

Editing

Developing Operational

Definitions for

Research Variables

Building the Theoretical

Framework and the

Research Model

Data Collection

More Careful Studying

of the Phenomenon

THE PROCESS OF

EMPIRICAL RESEARCH


4/25

4

RESEARCH DESIGN

CONCLUSION VALIDITYrefers to the extent ofresearchers ability to draw accurate conclusions from theresearch. That is, the degree of a studys:

a) Internal Validitycorrectness of conclusions regarding therelationships among vari

ables examinedWhether the research findings accurately reflect how the research

variables are really connected to each other.

b) External ValidityGeneralizability of the findings to the

intended/appropriate

population/settingWhether appropriate subjects were selected for conducting the study

RESEARCH DESIGN: The blueprint/roadmap that will guide theresearch.

The test for the quality of a studys research design is the

studys conclusion validity.


5/25

5

RESEARCH DESIGN

Variance of the INDEPENDENT & DEPENDENTvariables (Systematic Variance)

Variability of potential NUISANCE/EXTRANEOUS/

CONFOUNDING variables (Confounding Variance)

Variance attributable to ERROR IN MEASUREMENT

(Error Variance).

How?

How do you achieve internal and external validity (i.e.,conclusion validity)?

By effectively controlling 3 types of variances:


6/25

6

Effective Research Design

MAXimize Systematic Variance

MINimize Error Variance

CONtrol Variance of Nuisance/Extraneous/Exogenous/Confounding variables

Guiding principlefor effective control ofvariances (and, thus, effective research

design) is:The MAXMINCONPrinciple


7/25

7


IN EXPERIMENTS?

(where the researcher actually manipulates the independent

variable and measures its impact on the dependent variable): Proper manipulation of experimental conditions

to ensure high variability in indep. var.

IN NON-EXPERIMENTAL STUDIES?

(where independent and dependent variables are measuredsimultaneously and the relationship between them areexamined):

Appropriate subject selection (selecting subjectsthat are sufficiently different with respect to thestudys main var.)--avoid Range Restriction

MAXimizing Systematic Variance:

Widening the range of values of research variables.


8/25

8


Sources of error variance: Poorly designed measurement instruments

(instrumentation error)

Error emanating from study subjects (e.g.,response error)

Contextual factors that reduce a sound/accuratemeasurement instruments capacity to measureaccurately.

How to Minimize Error Variance? Increase validity and reliability of

measurement instruments. Measure variables under as ideal

conditions as possible.

MINimizing Error Variance (measurement error):Minimizing the part of variability in scores that is

caused by error in measurement.


9/25

1. Historical data on pollutionand longevity

2. Relationship between likelihood of

hearing problems and high blood pressure

3. Recent stat. show in-vitrokids are 5 times more likely to develop eye tumors

(Culprit: in-vitro fathers older age)

4. Significantly more armed store robberies during the cold winter days. 9


May or may not be of primary interest to the researcher,

But, can produce undesirable variation in the study'sdependent variable, and cause misleading or weird results

Thus, if not controlled, can contaminate/distort the truerelationship(s) between the independent and dependentvariable(s) of interest

i.e., confounding var. can result in a spurious-- as opposed tosubstantive--correlation between IV and DV. Example?

Hearing Blood

Problem Pressure

CONtrolling Variance of Confounding/Nuisance Variables:

FIRST, what areNuisance/ConfoundingVariables?

Age


10/25

10


Conducting the experiment in a controlled environment (e.g.,laboratory), where we can hold values of potential confoundingvariables constant.

Subject selection (e.g., matching subjects in experiments)

Random assignment of subjects (variations of confounding variablesare evenly distributed between the experimental and control groups)

In Survey Research:

Sample selection (e.g., including only subjects with appropriate

characteristicsusing male college graduates as subjects will controlfor potential confounding effects of gender and education)

Statistical Control--anticipating, measuring, and statisticallycontrolling for confounding variables effects(i.e., hold themstatistically constant, or statistically removing their effects).

HOW TO CONTROL FOR CONFOUNDING/NUISANCE VARIABLES?

In Experimental Settings (e.g., Fertilizer Amount Rate of Plant Growth):Some Potential Confounding Variables?


11/25


12/25

12

BASIC DESIGNS

Experimental Designs:

True Experimental Studies

Pre-experimental Studies

Quasi-Experimental Studies

Non-Experimental Designs:

Expost Facto/Correlational Studies

SPECIFIC TYPES OF RESEARCH DESIGN

BASIC RESEARCH DESIGNS:


13/25

13

EXPERIMENTAL DESIGNS

RESULT: Significant Improvement from O1 to O2(i.e., sig. O2 - O1 difference)

QUESTION: Did X (the drug) cause theimprovement?

One of the simplest experimental designs is the ONE GROUP PRETEST-POSTTEST DESIGN--EXAMPLE?

One way to examine Efficacy of a Drug:

O1 X O2

Measure DRUG Measure

Patients Condition Experimental Patients Condition

(Pretest) Condition/ (Posttest)

intervention


14/25

14


Have only shownX is a SUFFICIENTconditionfor the changeY(i.e., presence of X isassociated with a change in Y).

But, is X also a NECESSARYcondition forY?

How do you verify the latter? By showing that the change would not have

happened in the absence of Xusing aCONTROL GROUP.

David Humewould have been tempted to say YES.

He was a positivist and wanted to infer causality based

on high correlations between events.

But such an inference could be seriously flawed.

Why?

David Hume, 18th

Century Scottish

Philosopher


15/25

15


CONTROL GROUPsimulates absence of X Origin of using Control Groups (A tale from ancient Egypt)

Pretest Post-Test Control Group Design--Suppose random

assignment (R) was used to control confounding variables:

R Exp. Group O1E X O2ER Ctrl Group O1C O2C

RESULT: O2E > O1E & O2C Not> O1CQUESTION: Did X cause the improvement in Exp.

Group?


16/25

16


Need proper form of controle.g., Placebo.

R Exp. Group O1E X O2ER Ctrl Group O1C Placebo O2C

QUESTION: Can we now conclude X caused the improvementin Exp. Group?

NOT NECESSARILY! Why not?Power of suggestibility (The Hawthorne Effect)

CONCLUSION?

Maybe, but be aware of the Experimenter Effect(it tends toprejudice the results especially in medical research).

SOLUTION: Double Blind Experiments(neither the subjects

nor the experimenter knows who is getting the placebo/drug).


17/25

17


Experimental studies need to control for potentialconfounding factorsthat may threaten internal validityof the experiment:

Hawthorne Effect is only one potential confounding factorin experimental studies.

Other such factors are:

History? Biasing events that occur between pretest and post-test

Maturation? Physical/biological/psychological changes in the subjects

Testing? Exposure to pretest influences scores on post-test

Instrumentation? Flaws in measurement instrument/procedure


18/25

18


Experimental studies need to control for potentialconfounding factorsthat may threaten internal validity

of the experiment (Continued):

Selection? Subjects in experimental & control groups different from the start

Statistical Regression (regression toward the mean)? Subjects selected based on extreme pretest values

Discovered by Francis Galtonin 1877

Experimental Mortality?

Differential drop-out of subjects from experimental and controlgroups during the study

Etc.

Experimental designs mostly used in natural and physicalsciences.

Generally, higher internal validity, lower externalvalidity


19/25

19


20/25

20


21/25

21

CORRELATIONAL DESIGNS

The design of choice in social sciences since the phenomenon

under study is usually not reproducible in a laboratory setting

Researcher has little or no control over studys indep., dep.and the numerous potential confounding variables,

Often the researcher concomitantly measures all the studyvariables (e.g., independent, dependant, etc.),

Then examines the following types of relationships:

correlations among variables or

differences among groups,

Inability to controlfor effects of confounding variables makescausal inferences regarding relationships among variablesmore difficult and, thus:

Generally, higher external validity, lower internal validity

NON-EXPERIMENTAL/CORRELATIONAL DESIGNS


22/25

22

CORRELATIONAL DESIGNS

NOT NECESSARILY! EXAMPLES: Water Fluoridation and AIDS(San Francisco Chronicle, Sep. 6, 1984)

Armed store robberies and cold weather

Longevity and Pollution

In-vitro birth and likelihood of developing eyetumors

Hearing problem and blood pressure

What can a significant correlation mean then?

Non-experimental designs rely on correlational evidence.

QUESTION: Does a significant correlation between two

variables in a non-experimental study necessarily represent a

causal relationship between those variables?


23/25

23

CORRELATIONAL STUDIES

a. Both variables are effects of a common cause (or bothcorrelated with a third variable), i.e., spurious correlation(e.g., air pollution and life expectancy, hearing problem &

blood pressure, countrys annual ice cream sales andfrequency of hospital admissions for heat stroke)

b. Both var. alternative indicatorsof same concept(e.g., Church attend. & Freq. of Praying--religiosity).

c. Both parts of a common "system" or "complex;" tend tocome as a package(e.g., martini drinking and opera attendance--life style)

d. Fortuitous--Coincidental correlation, no logical relationship

(e.g., Outcome of super bowl games and movement of stockmarket)

AT LEAST FOUR OTHER POSSIBLE INTERPRETATIONS/REASONS

FOR CORRELATIONS BETWEEN TWO VARIABLES:


24/25

24

CORRELATIONAL STUDIES

Covariation Rule(X and Y must becorrelated)--Necessary but not sufficient condition.

Temporal Precedence Rule(If X is the cause, Yshould not occur until after X).

Internal Validity Rule(Alternative plausibleexplanations of Y and X-Y relationships should beruled out (i.e., eliminate other possible causes).

In practice, this means exercising caution byidentifying potential confounding variables andcontrolling for their effects).

WHEN IS IT SAFER TO INFER CAUSAL

LINKAGES FROM STRONG CORRELATIONS?John Stuart MillsRules for Inferring Causal Links:

John Stuart Mill

1806-1873


25/25

2. Research Design-LDR 280.PPT

Documents

Transcript of 2. Research Design-LDR 280.PPT