Introduction to ValidityIntroduction to ValidityIntroduction to ValidityIntroduction to Validity

What is Validity?What is Validity?What is Validity?What is Validity?

The best available approximation to the The best available approximation to the truth or falsity of a given inference, truth or falsity of a given inference, proposition, or conclusionproposition, or conclusion

A set of standards by which research A set of standards by which research can be judgedcan be judged

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

cause-effect constructcause-effect construct

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

What you What you thinkthink

cause-effect constructcause-effect construct

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

ProgramProgram

What you What you thinkthink

In In thisthis study study

Cause-Effect ConstructCause-Effect Construct

OperationalizeOperationalize

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

ProgramProgram ObservationsObservations

What you What you thinkthink

In In thisthis study study

Cause-effect ConstructCause-effect Construct

OperationalizeOperationalize OperationalizeOperationalize

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

ProgramProgram ObservationsObservations

What you What you dodo What you What you seesee

What you What you thinkthink

In In thisthis study study

Cause-effect ConstructCause-effect Construct

OperationalizeOperationalize OperationalizeOperationalize

The Causal ContextThe Causal ContextThe Causal ContextThe Causal Context

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

ProgramProgram ObservationsObservations

What you What you dodo What you What you seesee

What you What you thinkthink

What you What you testtestIn In thisthis study study

Cause-effect ConstructCause-effect Construct

Program-outcome RelationshipProgram-outcome Relationship

OperationalizeOperationalize OperationalizeOperationalize

Conclusion ValidityConclusion ValidityConclusion ValidityConclusion Validity

What you did and what you saw?What you did and what you saw? Your program and your observations?Your program and your observations?

ObservationObservationObservationObservation

ProgramProgram ObservationsObservations

What you What you dodo What you What you seesee

Is there a Is there a relationshiprelationship between... between...

In In thisthis study study

Program-outcome RelationshipProgram-outcome Relationship

Internal ValidityInternal ValidityInternal ValidityInternal Validity

• What you did and what you saw?What you did and what you saw?• Your program and your observations?Your program and your observations?

ObservationObservationObservationObservation

ProgramProgram ObservationsObservations

What you What you dodo What you What you seesee

Is the relationship Is the relationship causalcausal between... between...

AlternativeAlternativecausecause AlternativeAlternative

causecause

AlternativeAlternativecausecause

AlternativeAlternativecausecause

In In thisthis study study

Program-outcome RelationshipProgram-outcome Relationship

Construct ValidityConstruct Validity

TheoryTheoryTheoryTheory

ObservationObservationObservationObservation

CauseCauseconstructconstruct

EffectEffectconstructconstruct

ProgramProgram ObservationsObservations

What you What you dodo What you What you seesee

What you What you thinkthink

Cause-effect ConstructCause-effect Construct

Program-outcome RelationshipProgram-outcome Relationship

Can we generalize to the Can we generalize to the constructsconstructs??

External ValidityExternal ValidityExternal ValidityExternal Validity

TheoryTheoryTheoryTheory

CauseCauseconstructconstruct

EffectEffectconstructconstruct

What you What you thinkthink

Cause-effect ConstructCause-effect Construct

ObservationObservationObservationObservation

ProgramProgram ObservationsObservations

What You What You DoDo What You What You SeeSee

program-outcome relationshipprogram-outcome relationship

ObservationObservationObservationObservation

ProgramProgram ObservationsObservations

What You What You DoDo What you What you seesee

program-outcome relationshipprogram-outcome relationship

ObservationObservationObservationObservation

ProgramProgram ObservationsObservations

What You What You DoDo What You What You SeeSee

program-outcome relationshipprogram-outcome relationship

ObservationObservationObservationObservation

ProgramProgram ObservationsObservations

What You What You DoDo What You What You SeeSee

program-outcome relationshipprogram-outcome relationship

Can we generalize to other persons, places, times?Can we generalize to other persons, places, times?

The Validity Questions Are The Validity Questions Are CumulativeCumulative......The Validity Questions Are The Validity Questions Are CumulativeCumulative......

The Validity Questions Are The Validity Questions Are CumulativeCumulative......The Validity Questions Are The Validity Questions Are CumulativeCumulative......

In this studyIn this studyIn this studyIn this study Is there a relationship between the cause and effect?

The Validity Questions Are The Validity Questions Are CumulativeCumulative......The Validity Questions Are The Validity Questions Are CumulativeCumulative......

ConclusionConclusionConclusionConclusionIs there a relationship between the cause and effect?

Is the relationship causal?In this studyIn this studyIn this studyIn this study

The Validity Questions Are The Validity Questions Are CumulativeCumulative......The Validity Questions Are The Validity Questions Are CumulativeCumulative......

In theoryIn theoryIn theoryIn theory

Is there a relationship between the cause and effect?

Is the relationship causal?

Can we generalize to the constructs?

ConclusionConclusionConclusionConclusion

InternalInternalInternalInternal

The Validity Questions Are The Validity Questions Are CumulativeCumulative......The Validity Questions Are The Validity Questions Are CumulativeCumulative......

ConstrucConstructtConstrucConstructt

Is there a relationship between the cause and effect?

Is the relationship causal?

Can we generalize to the constructs?

Can we generalize to other persons, places, times?

In theoryIn theoryIn theoryIn theory

ConclusionConclusionConclusionConclusion

InternalInternalInternalInternal

The Validity Questions are The Validity Questions are cumulativecumulative......The Validity Questions are The Validity Questions are cumulativecumulative......

Is there a relationship between the cause and effect?

Is the relationship causal?

Can we generalize to the constructs?

Can we generalize to other persons, places, times?

ExternalExternalExternalExternal

ValidityValidityValidityValidity

ConclusionConclusionConclusionConclusion

InternalInternalInternalInternal

ConstructConstructConstructConstruct

Threats to ValidityThreats to ValidityThreats to ValidityThreats to Validity

You Want to Make an Inference...You Want to Make an Inference...

There There isis a relationship between a relationship between the cause and effect.the cause and effect.

The relationship The relationship isis causal. causal. You You cancan generalize to the generalize to the

constructs.constructs. You You cancan generalize to other generalize to other

persons, places, and times.persons, places, and times.

Threats to ValidityThreats to ValidityThreats to ValidityThreats to Validity

There is a relationship, but you don’t There is a relationship, but you don’t seesee it. it.

There is no relationship, but you There is no relationship, but you dodo see one.see one.

How could you be How could you be wrongwrong in the inference? in the inference?How could you be How could you be wrongwrong in the inference? in the inference?

Conclusion ValidityConclusion Validity

Threats to ValidityThreats to ValidityThreats to ValidityThreats to Validity

There is a causal relationship, but you There is a causal relationship, but you don’t don’t seesee it. it.

There is no causal relationship, but There is no causal relationship, but you you dodo see one. see one.

How could you be How could you be wrongwrong in the inference? in the inference?How could you be How could you be wrongwrong in the inference? in the inference?

Internal ValidityInternal Validity

Threats to ValidityThreats to ValidityThreats to ValidityThreats to Validity

You can generalize to constructs, but You can generalize to constructs, but you you conclude you can’t.conclude you can’t.

You can’t generalize to constructs, but You can’t generalize to constructs, but you you conclude you can.conclude you can.

How could you be How could you be wrongwrong in the inference? in the inference?How could you be How could you be wrongwrong in the inference? in the inference?

Construct ValidityConstruct Validity

Threats to ValidityThreats to ValidityThreats to ValidityThreats to Validity

You can generalize to other contexts, You can generalize to other contexts, but you but you conclude you can’tconclude you can’t

You can’t generalize to contexts, but You can’t generalize to contexts, but you you conclude you canconclude you can

How could you be How could you be wrongwrong in the inference? in the inference?How could you be How could you be wrongwrong in the inference? in the inference?

External ValidityExternal Validity

Threats to ValidityThreats to ValidityThreats to ValidityThreats to Validity

How you can be wrong in making your How you can be wrong in making your inferenceinference

Specific factors that can bias or distort Specific factors that can bias or distort your conclusionsyour conclusions

A list of common threats to the quality of A list of common threats to the quality of your studyyour study

A “checklist” you can use in planning A “checklist” you can use in planning your studyyour study