Lecture NotesNota_6_Validiti Dan Reliabiliti [Autosaved]

8/10/2019 Lecture NotesNota_6_Validiti Dan Reliabiliti [Autosaved]

1/40

Validity

Reliability

Control


2/40

2

Establishing Cause and Effect

Temporal Precedence

Cause has to happen beforethe effect

Covariation of Cause and Effect Show there is a relationship between X and Y

No Plausible Alternative Explanations

Any other cause that can bring about the effect


3/40

3

Establishing Cause and Effect


4/40

4

Validity and Reliability

Measure-Related Concerns

Construct Validity

Are you measuring what you think youre measuring?

Reliability

Is your measure consistent in its evaluation of thesame individuals?


5/40

5

Types of Reliability

Inter-rater reliability

Assesses the degree to which different

raters/observers give consistentestimates of the same phenomenon

% agreement

Correlation between observers scores


6/40

6


Test-retest

Correlation between two observations (i.e.,

set of scores) on the same test administeredto the same (or similar) sample on twodifferent occasions

Assumes no change in construct beingmeasured

Time between observations is crucial


7/407


Parallel-forms

Correlation between two observations (i.e.,set of scores) on parallel forms of a testadministered to the same sample

Requires generation of many items thatmeasure the same construct

Two forms of test can be usedindependently

Assumes randomly divided halves are

equivalent


8/408


Internal consistency Single test administered to a sample on one

occasionAssesses the consistency of the results for

different items for the same construct withinthe measure

Average inter-item correlation

Average item-total correlation

Split-half reliability

Cronbachs Alpha


9/409

Relationship between reliability and validity


10/4010

Validity and Reliability

Method-Related Concerns

Internal Validity

Is the independent variable the only possibleexplanation of the results shown?

External Validity

How far can the study generalize?


11/4011

Validity

Internal Validity

Is the independent variable the only possible

explanation of the results shown? Protecting internal validity allows us to eliminate

potential alternative explanations for theoutcome of an experiment

Extraneous variables

Influence the DV without your intent orconsent!


12/4012

Experimental Control

Extraneous Variables

Nuisance variables

Make the effects of the IV more difficult todetermine by affecting variation within groups

Participant characteristics

Unintended influences of experimental situation


13/40

13



Confounding variables

Vary systematically with the IV Influence the difference between groups

The differences observed between groups could beattributed to the IV orthe confound


14/40

14



Experimenters

Physiological differences Participants

Demand characteristics

Good participant effect

Response biases


15/40

15

Internal Validity

Exercise controlto protect internal validitybeforethe experiment is conducted

Random Assignment Elimination

Constancy

Counterbalancing Single- and Double-blind Experiments

Altering Response Sets


16/40

16

Basic Control Techniques

Randomization

Individuals have an equal chance of being

assigned to any group in the experimentAllows us to assume groups are roughly equivalent

before administering the IV


17/40

17


Elimination

Extraneous variables are completely removed

from the experiment


18/40

18


Constancy

An extraneous variable is reduced to a single

value experienced by all participants Standard control technique

Hold environment, temperature, lighting constant


19/40

19


Balancing and Counterbalancing

Balancing

Achieves group equality by distributing extraneousvariables to all groups

Counterbalancing

Controls order effects by presenting treatments in a

different sequence Order or sequence effects

Carryover effects

Differential carryover


20/40

20


Within-Subject Counterbalancing

Within-Group Counterbalancing

Each treatment must be presented to each participantan equal number of times

Each treatment must occur an equal number of times ateach testing session

Each treatment must precede and follow each of theother treatments an equal number of times


21/40

21

The Cola Wars!

Participant 1 Participant 2

Participant 3


Participant 6

P CP C

P C

C PC P

C P


22/40

22

The Cola Wars!


Participant 3


Participant 6

P C DPP DP C

C P DP

C DP PDP P C

DP C P


23/40

23


Single- and Double-Blind Experiments

The experimenter (single-blind) or both the

experimenter and participants (double-blind)are unaware of the participants condition


24/40


25/40


26/40

26

Threats to Internal Validity

History

Significant events occur between measurements


27/40

27


Maturation

Participants change over time in the experiment


28/40

28


Testing (Pretest Sensitization)

Repeated tests cause changes in DV;

practice effects


29/40

29


Instrumentation

Changes in measurement due to changes in the

measurement device (human or machine)


30/40

30


Statistical regression

Extreme scores regress toward the mean

Difficult to maintain extreme scores overrepeated measures


31/40


32/40

32


Attrition or Mortality

Experimental dropouts


33/40

33


Diffusion of treatment

Participants in one group communicate

information from the IV to the other group


34/40

34


Interactions with Selection

Maturation

History Instrumentation


35/40

35

How Important is Internal Validity?

It is themost importantproperty of anyexperiment

An experiment without internal validity cannothave external validity

With no internal validity, you can haveno confidence in your results


36/40

36

External Validity

How far can you generalize your resultsbeyond your experiment?

3 types: Population

Environmental

Temporal


37/40

37

Threats to External Validity

Testing/Treatment Interaction

Pretest may make reaction to the IV different

from those not tested Selection/Treatment Interaction

Effect is found only for a specific group of

participants


38/40

38

Threats to External Validity

Reactive Arrangements

A la Hawthorne effects

Demand characteristics Participant Characteristics

Specific animal species

Experimental participant populations: College students

White male Americans


39/40

39


40/40

Is External Validity Necessary?

May want to find out if something canhappen

We may be predicting from the real world to the

lab If we can show something happens in the labs

unnatural setting, we may have more confidencein the phenomenon

We may study something without a real-worldanalogy

Lecture NotesNota_6_Validiti Dan Reliabiliti [Autosaved]

Documents

Transcript of Lecture NotesNota_6_Validiti Dan Reliabiliti [Autosaved]