Understanding Variability
Unraveling the Mystery of the Data’s Message
Becoming a “Data Whisperer”
Overview of Research Issues and Goals
I. Idea – hypothesisConceptualization –
Nominal Definitions – Operational Definitions –
Measurement ProceduresII. Design (framework/procedure) –
Strategy -- Implementation –
Data Collection -- Measures
III. Statistics – Exploratory –
Descriptive – Inferential –
DecisionsIV. Conclusions in context –
Interpretation -- Meaning --
Generalization
Search for Cause – Effect connections
Information needed to establish a cause is a cause(J. S. Mill)
Cause and Effect are related – they covary
Cause comes before the Effect in time
All alternative Causes are eliminated
Cause and Effect Connections can be
Necessary and Sufficient – single clear cause
Necessary, not Sufficient – combinations of or interactions of variables needed to get effect
Sufficient, not Necessary – alternative independent causes exist
Goal of Research – to understand variability,
how do variables vary (descriptive), when do they vary (covariation),
and why do variables vary (causes)
Assume that variables cannot vary without cause
Research Strategies –
depend upon research goals/purposes/constraints
Descriptive Research what variables exist and how do variables vary
structure of a variable – qualitative or quantitativedoes the variable exist separately from othersone-dimensional or multidimensionalpossible values and range of values
variability within a sample/population
Covariation Research (when do variables vary - context)
when variables vary, with what do they covary
“response variable - response variable” covariation
assess strength and direction of relationship
“predictor variable – response variable” covariation
use one variable to ‘explain’ or ‘predict’ variability in the other variable
limitations on ‘causal’ interpretations
Search for Cause – Effect connections
Information needed to establish a cause is a cause
Cause and Effect are related – they covary
Cause comes before the Effect in timeLogic may support direction -
All alternative Causes are eliminated
Group Difference Research - focus on typical behavior
Quasi-experimental research –
covariation related to ‘existing’ differences (grouping)
“manipulation” by selection
Experimental research –
creating groups to control variation of some variables
manipulation by control
Data Collection IssuesMeasurement scale –
what type of information is providedqualitative and/or quantitative
operations that can be performed on measures
Source of measuresdirect
physiological measuresbehavioral observations
indirectself reportsinferred
secondarycollected by others
Research Setting
Controlled conditions ------- Uncontrolled conditions
reducing the noise to improve ability to hear the signal
allowing noise to see if signal still has impact
Evaluating Research Strategies
Internal Validity – can outcome be interpreted clearly
enhanced by control over all variables and precision of activities
External Validity – can outcome be generalized
enhanced by inclusion of representative variables and activities
Research involves solving a mystery (Must decide on Guilt or Innocence – and hope that decision matches the unknown ‘truth’)
understanding why things happen – search for TRUTH!
How strong a case can you make,
but NEVER 100% confident
TRUTH
DECISION Innocent Guilty
Guilt wrong right
Innocence right wrong
Research involves solving a mystery (Must decide on Guilt or Innocence – and hope that decision matches the unknown ‘truth’)
understanding why things happen – search for TRUTH! (You are like CSI and Prosecutor)
Research Strategy – collect the evidence…the data upon which your case will be based
Statistics – organize, summarize, and evaluate the strength of the evidence
Decide on your ‘interpretation’ of the evidence(Decide on your version of the “Truth” – guilt or innocence)
Persuade others of the validity of your interpretation
(Other Researchers are like Defense attorney – need only raise doubts!)
validity of your researchappropriateness of evaluation
The ‘Case”
a body is found in the library – “who” and/or “what” caused the death?
Law & Order: Psychological Research Unit
Descriptive Research
Covariation Research
Causal EvidenceMeaningful level of Explanation?
Main Characters – all the variables that might be involved
Dependent Variable(s) – the ‘response’ variablealways present, the basis for the data
used to answer the question
Independent Variable(s) – explanatory/predictor variablethe suspected cause or variable related to the cause
Extraneous Variables – all variables not cast in either of the roles above
nuisance or confound – possible impacts (evil) (Orient Express)
best dealt with in the context of the research strategy – design and procedures
(Constant, Control, Random)
alternatively, can be dealt with indirectly, using statistical adjustments
Qualities of the Main Characters – any variables which will be ‘measured’
Actually are Qualities of Operational Definitions Used
reliability – precision and consistency
validity – appropriateness and accuracy
RecapResearch involves solving a mystery – understanding why things happen
Research Strategy – collect the evidence…the data upon which your case will be based
Statistics – organize, summarize, and evaluate the strength of the evidence
Decide on your ‘interpretation’ of the evidencethe logic of statistical decision-making
Persuade others of the validity of your interpretation
validity of your researchappropriateness of evaluation
Research involves solving a mystery – understanding why things happen
Research Strategy – collect the evidence…the data upon which your case will be base
Issues:correct variables, measured well (operational
definitions)design is appropriate for questions
descriptive, covariation, causalabsence of contamination (internal validity)can generalize beyond specifics (external
validity)
Research involves solving a mystery – understanding why things happen
Research Strategy – collect the evidence…the data upon which your case will be based
Statistics – organize, summarize, and evaluate the strength of the evidence
Issues:organize/summarize data (correct descriptive
statistics)isolate signal from noise (correct inferential
statistics)evaluate the signal (understand what the
message is)
Research involves solving a mystery – understanding why things happen
Research Strategy – collect the evidence…the data upon which your case will be based
Statistics – organize, summarize, and evaluate the strength of the evidence
Decide on your ‘interpretation’ of the evidence Issues:
what do the results meanwhat decision will you make (guilt or
innocence)how confident are you (probability you are
wrong)
Research involves solving a mystery – understanding why things happen
Research Strategy – collect the evidence…the data upon which your case will be based
Statistics – organize, summarize, and evaluate the strength of the evidence
Decide on your ‘interpretation’ of the evidence
Persuade others of the validity of your interpretation
Issues:validity of your researchvalidity of your analyses of the datavalidity of your interpretation of the analyses
Mystery Solved
Top Related