Final Study GuideFinal Study GuideResearch DesignResearch Design

Experimental Research Experimental Research

Experimental ResearchExperimental Research

• Researchers manipulate independent variable - 2 levels

• And measure the other (dependent variable)

• Give treatment to participants and observe if it causes changes in behavior

• Compare experimental group (w/ treatment) with a control group (no treatment)

• Can say IV caused change in the DV

Independent VariableIndependent Variable

• The variable whose impact you want to know

• ‘Stimulus’ ‘Input’ Variable

• The variable you manipulate in experimental research

Dependent VariableDependent Variable

• The variable whose changes you want to know

• You measure it

• ‘Outcome’ ‘Response’ variable

• Random Selection– A way to choose your sample of study– Any member of population has equal chance of

being selected

• Random Assignment– A way to assign participants in sample to the

various treatment conditions (groups will receive different level of IV)

– Any member of your sample has equal chance of being assigned in any treatment group

Internal ValidityInternal Validity

• Ability of your research design to adequately test your hypothesis

• Showing that variation in I.V. CAUSED the variation in the D.V. in experiment

• In correlational study,

• Showing that changes in value of criterion variable relate solely to changes in value of predictor variable

ConfoundingConfounding

• Whenever 2 or more variables combine in a way that their effects cannot be separated = confounding.

• Thus, the teaching method study as designed lacks internal validity.

• You don’t know if the change in the DV is from the IV or from confounding variable

Quasi-experimental researchQuasi-experimental research

• Naturally occurring conditions

• (IV change)

• No control over variables influencing behavior (confounding variables)– Another variable that changed along with the

variable of interest may have caused the observed effect

– (NO random assignment)

Non-Experimental Non-Experimental ResearchResearch

Non-experimentalNon-experimental Correlational research Correlational research

• Determine whether 2 or more variables are associated,

• If so, to establish direction and strength of relationships

• Observe variables as they are, – can’t manipulate them

Research designResearch design

Manipulate IV Random Assignment

• Experimental (Causal) x x• Quasi-experimental x• Non-experimental /

– Correlational • Predictive• Descriptive

• Causal - (Experimental)

• one variable directly or indirectly influences another.

• Correlational - (Non-experimental)

• Changes in one variable accompany changes in another. – A relationship exists. Don’t know if either

variable actually influences the other.

TERMSTERMS

Population• Universe/entire set of people you want to

draw conclusions about

Sample• Subset of the population• People actually in your study

Sampling error• Differences between sample & population

SamplingSampling

• Drawing a subgroup from a population (vs. Census)

Probability vs. Non-probabilityProbability vs. Non-probability

• Simple random• Systematic random• Stratified random• Cluster

• Convenience• Snowball • Quota• Purposive

Probability Sampling Non-probability Sampling

Population info Available

Population info Not available

Representativenss Representativenss & Generalizability& Generalizability

• Representativeness = Resemblance to the population characteristics

• Generalizability = An ability to generalize the results of your study to the whole population

• High representativeness = High generalizability

• Probability sampling allows higher representativeness than non-probability

External ValidityExternal Validity

• Degree that results can be extended beyond the limited research setting– Generalizable

– Based on sample ( rats, college students, whites, males, lab setting)

Non-Probability SamplingNon-Probability Sampling

Convenience SamplingConvenience Sampling

• Get available people in the population

• Low representativeness / generalizability

Quota SamplingQuota Sampling

• Predetermine the proportion of groups in the sample (e.g., male 50%, female 50%)

Conceptualization & Conceptualization & OperationalizationOperationalization

Idea

Conceptualization

Operationalization

Clarificatio

n

OperationalizationOperationalization

• From complex variable to series of simpler variables

• Redefining a variable in terms of steps to measure

• Conceptual definition Operational definition

• What the researcher must do to MEASURE it

Types of Measurement Types of Measurement ValidityValidity

• Face validity

• Content validity

• Predictive

• Concurrent

• Convergent

• Discriminant

Judgmental Empirical (Criterion-

related)

Observed score = True score + Eerror

Observed = measured score, result

True = “true”, actual, exact state

Error = measurement error

““O = T + E” ruleO = T + E” rule

Reliability of a Reliability of a MeasureMeasure

Degree to which a measure (score, observation) is affected by error

• A reliable measure has little or no error

Types of ReliabilityTypes of Reliability

• Interobserver (interrater) reliability

• Test-Retest reliability

• Parallel-forms reliability

• Split- half

Inter-rater AgreementInter-rater Agreement• Consistency between measurements by

two or more observers

• Different observers watch the same sample of behavior

• Compute proportion of time both observers recorded the same behavior as happening

# agreements

# agreements + # disagreements (# of observations)

• Training needed for observers

Increasing reliabilityIncreasing reliability

• Increase number of items on your questionnaire (no 1 or 2 item measures)

• Write clear, well-written items on survey

• Standardize administration procedures– Treat all participants alike– Timing, procedures, instructions alike

• Score survey carefully -- avoid errors

Valid and ReliableValid and Reliable• A good measurement• Measures what it should measure in a

consistent way

Reliable but InvalidReliable but Invalid

• Your measurement is consistent, but not measuring what it is supposed to measure

Research Report StructureResearch Report Structure Abstract Introduction Method Results Discussion Reference