Research Methods: Research Methods: Midterm ReviewMidterm Review

Dr. DodgeDr. Dodge

February 28, 2006February 28, 2006

Introduction to ResearchIntroduction to Research

What is Scientific Research?What is Scientific Research?

““Scientific research is systematic, Scientific research is systematic, controlled, empirical, and critical controlled, empirical, and critical investigation of natural phenomena investigation of natural phenomena guided by theory and hypotheses guided by theory and hypotheses about the presumed relations about the presumed relations among such phenomena.” among such phenomena.”

- KerlingerKerlinger

The Scientific Method and the The Scientific Method and the Empirical ApproachEmpirical Approach

The fundamental characteristic of the The fundamental characteristic of the scientific methodscientific method is is empiricismempiricism (knowledge based on observations). (knowledge based on observations).

Most importantly, the scientific Most importantly, the scientific method embodies a number of rules method embodies a number of rules for for collecting, evaluating, and collecting, evaluating, and reporting datareporting data (observations put into (observations put into measurable form).measurable form).

Goals of Scientific ResearchGoals of Scientific Research

1.1. To Describe BehaviorTo Describe Behavior2.2. To Predict BehaviorTo Predict Behavior3.3. To Determine the Causes of To Determine the Causes of

BehaviorBehavior4.4. To Understand or Explain BehaviorTo Understand or Explain Behavior- In order to accomplish these goals, In order to accomplish these goals,

scientific research can be scientific research can be conducted in ways that are both conducted in ways that are both basicbasic and and appliedapplied

Basic vs. Applied ResearchBasic vs. Applied Research

Neither type of research is superior Neither type of research is superior to the otherto the other

The progress of science depends on The progress of science depends on the synergy between the two the synergy between the two

Both may cause the other to be Both may cause the other to be modifiedmodified

Both use the scientific method of Both use the scientific method of researchresearch

Evidence-Based PracticeEvidence-Based Practice

A Hierarchy of Levels of A Hierarchy of Levels of Best EvidenceBest Evidence

Level 1 evidenceLevel 1 evidence: strong evidence from at : strong evidence from at least one systematic review of multiple least one systematic review of multiple well-designed randomized controlled trials.well-designed randomized controlled trials.

Level 2 evidenceLevel 2 evidence: strong evidence from at : strong evidence from at least one properly designed randomized least one properly designed randomized controlled trial of appropriate size.controlled trial of appropriate size.

Level 3 evidenceLevel 3 evidence: evidence from well-: evidence from well-designed trials without randomization, designed trials without randomization, single group pre-post, cohort, time series, single group pre-post, cohort, time series, or matched case-controlled studies.or matched case-controlled studies.

A Hierarchy of Levels of A Hierarchy of Levels of Best EvidenceBest Evidence

Level 4 evidenceLevel 4 evidence: evidence from well-: evidence from well-designed non-experimental studies from designed non-experimental studies from more than one center or research group.more than one center or research group.

Level 5 evidenceLevel 5 evidence: opinions of respected : opinions of respected authorities, based on clinical evidence, authorities, based on clinical evidence, descriptive studies, or reports of expert descriptive studies, or reports of expert committeescommittees

Which level(s) should we strive for in our Which level(s) should we strive for in our research?research?

Meta-AnalysisMeta-AnalysisMeta-analysisMeta-analysis: method used to review : method used to review

research literature based on statistical research literature based on statistical integration and analysis of research integration and analysis of research findingsfindings

In meta-analysis, the DV is the In meta-analysis, the DV is the effect sizeeffect size (i.e., the outcomes or results of each study (i.e., the outcomes or results of each study selected for review transformed into a selected for review transformed into a common metric across studies) and the IV common metric across studies) and the IV are are study characteristicsstudy characteristics (i.e., participants, (i.e., participants, interventions, and outcome measures). interventions, and outcome measures).

Meta-AnalysisMeta-AnalysisBenefits of meta-analysisBenefits of meta-analysis: :

•Synthesize the results from many studies Synthesize the results from many studies succinctly and intuitively for nonscientific succinctly and intuitively for nonscientific communities, communities,

•Illustrate the amount and relative impact Illustrate the amount and relative impact of different programs on different criteria of different programs on different criteria for policy decision-making purposes, and for policy decision-making purposes, and

•Identify the most effective programs and Identify the most effective programs and highlight gaps or limitations in the highlight gaps or limitations in the literature to suggest directions for future literature to suggest directions for future researchresearch

Research EthicsResearch Ethics

Areas of Focus in Research Ethics

Harm Informed Consent Confidentiality Deception Reporting Results and Plagiarism

The Scientific MethodThe Scientific Method

ScienceScience

Testing ideas empirically according Testing ideas empirically according to a specific set of procedures that is to a specific set of procedures that is open to public inspectionopen to public inspection

Based on objectively observed Based on objectively observed evidenceevidence

Without personal beliefs, Without personal beliefs, perceptions, biases, attitudes or perceptions, biases, attitudes or emotionemotion

The Scientific MethodThe Scientific Method

The Scientific MethodThe Scientific Method

Step 1: ObservationStep 1: Observation Step 2: Form a hypothesisStep 2: Form a hypothesis Step 3: Use hypothesis to generate a Step 3: Use hypothesis to generate a

testable predictiontestable prediction Step 4: Design the studyStep 4: Design the study Step 5: Conduct the studyStep 5: Conduct the study Step 6: Perform hypothesis testingStep 6: Perform hypothesis testing

APA Style *APA Style *

* Refer to Dr. Pruett’s PowerPoint!* Refer to Dr. Pruett’s PowerPoint!

Quantitative and Quantitative and Qualitative ApproachesQualitative Approaches

Observing BehaviorObserving Behavior

Behavior can be Behavior can be observed or observed or measuredmeasured..

Because behavior varies so much, Because behavior varies so much, scientists need to have scientists need to have numerous numerous ways to measure and observe ways to measure and observe behaviorbehavior in both experimental and in both experimental and non-experimental settings.non-experimental settings.

Research PerspectivesResearch Perspectives Two major theoretical perspectivesTwo major theoretical perspectives

underlie much of social science research:underlie much of social science research:• a. the a. the positivist positivist tradition, which “seeks tradition, which “seeks

facts or causes of social phenomena facts or causes of social phenomena apart from the subjective states of apart from the subjective states of individuals”individuals”

• b. the b. the phenomenological phenomenological perspective, perspective, which “is committed to understanding which “is committed to understanding social phenomena from the actor’s own social phenomena from the actor’s own perspective”perspective”

Research PerspectivesResearch Perspectives

Quantitative ResearchQuantitative Research• generally stems from a positivist generally stems from a positivist

traditiontradition Qualitative ResearchQualitative Research• generally follows the generally follows the

phenomenological traditionphenomenological tradition Both, in theory, are validBoth, in theory, are valid!!

Quantitative ResearchQuantitative Research

Quantitative researchQuantitative research: Data are : Data are collected and observations are collected and observations are reported numerically.reported numerically.

Questionnaires, tests, and other Questionnaires, tests, and other measures to record frequency of measures to record frequency of behavior, occurrence of behavior behavior, occurrence of behavior and/or duration of behaviorand/or duration of behavior

After numerical data are collected, After numerical data are collected, they are analyzed statistically.they are analyzed statistically.

Qualitative ResearchQualitative Research

Qualitative ResearchQualitative Research: Data are : Data are collected from observations and collected from observations and interviewsinterviews

Data often expressed in Data often expressed in non-numerical non-numerical termsterms using language and images – using language and images – analyzed systematicallyanalyzed systematically

Summaries of discussions, interviews, Summaries of discussions, interviews, and video- or audio-taping of behaviorsand video- or audio-taping of behaviors

Quantitative vs. Qualitative Quantitative vs. Qualitative DesignDesign

Quantitative Quantitative Research DesignsResearch Designs::

ExperimentalExperimental

Quasi-ExperimentalQuasi-Experimental

Ex post factoEx post facto

CorrelationalCorrelational

DescriptiveDescriptive

Time Series/Single Time Series/Single Subject Subject

Qualitative Qualitative

Research DesignsResearch Designs::

EthnographicEthnographic

Case studyCase study

PhenomenologicalPhenomenological

Historical Historical

PhilosophicalPhilosophical

Grounded Grounded TheoreticalTheoretical

Quantitative vs. Qualitative Quantitative vs. Qualitative SummarySummary

Neither is superior to the otherNeither is superior to the other Choosing one or the other is Choosing one or the other is

based on what is needed in your based on what is needed in your studystudy

High quality research projects High quality research projects may may incorporate aspects of both incorporate aspects of both approachesapproaches..

Introduction to Introduction to Qualitative Research *Qualitative Research *

* Refer to Dr. Simpson’s PowerPoint!* Refer to Dr. Simpson’s PowerPoint!

Introduction to StatisticsIntroduction to Statistics

Statistics and Research Statistics and Research DesignDesign

StatisticsStatistics: Theory and method of : Theory and method of analyzing quantitative data from samples analyzing quantitative data from samples of observations in order to help make of observations in order to help make decisions about hypothesized relations.decisions about hypothesized relations.

Statistics are merely tools used in research Statistics are merely tools used in research design!design!

Research DesignResearch Design: Plan and structure of : Plan and structure of the investigation to answer the research the investigation to answer the research questions (or hypotheses)questions (or hypotheses)

StatisticsStatistics

There are two types of statisticsThere are two types of statistics• Descriptive StatisticsDescriptive Statistics: involves : involves

tabulating, depicting, and describing tabulating, depicting, and describing datadata

• Inferential StatisticsInferential Statistics: predicts or : predicts or estimates characteristics of a population estimates characteristics of a population from a knowledge of the characteristics from a knowledge of the characteristics of only a sample of the populationof only a sample of the population

Statistics & ParametersStatistics & Parameters

• ParameterParameter:: a value, usually unknown a value, usually unknown (estimated), used to represent a certain (estimated), used to represent a certain population characteristicpopulation characteristic

• StatisticStatistic: : a quantity that is calculated a quantity that is calculated from a sample of data. Used to give from a sample of data. Used to give information about unknown values in the information about unknown values in the corresponding population.corresponding population.

Descriptive StatisticsDescriptive Statistics

• Nominal scalesNominal scalesNo numerical or quantitative No numerical or quantitative

properties. A way to classify properties. A way to classify groups or categories.groups or categories.

Gender: Male and FemaleGender: Male and Female• Ordinal scalesOrdinal scales

Used to rank and order the levels Used to rank and order the levels of the variable being studied. No of the variable being studied. No particular value is placed between particular value is placed between the numbers in the rating scale.the numbers in the rating scale.

Restaurant Ratings: 4 Stars, 3 Restaurant Ratings: 4 Stars, 3 Stars, 2 Stars, and 1 StarStars, 2 Stars, and 1 Star

Descriptive StatisticsDescriptive Statistics

• Interval scalesInterval scales Difference between the numbers on the Difference between the numbers on the

scale is meaningful and intervals are scale is meaningful and intervals are equal in size. NO absolute zero. Allows equal in size. NO absolute zero. Allows for comparisons for comparisons

Temperatures on a thermometerTemperatures on a thermometer• Ratio scalesRatio scales

Scales that do have an absolute zero Scales that do have an absolute zero point than indicated the absence of the point than indicated the absence of the variable being studied. Can form ratios.variable being studied. Can form ratios.

Weight: 250 pounds is ½ of 500 poundsWeight: 250 pounds is ½ of 500 pounds

Descriptive StatisticsDescriptive Statistics

Frequency DistributionsFrequency Distributions• Constructed by summarizing data in Constructed by summarizing data in

tables according to the number or tables according to the number or frequency of observations in each frequency of observations in each category, score, or score intervalcategory, score, or score interval

• Data can be concisely summarized in Data can be concisely summarized in bar graphs, histograms, or frequency bar graphs, histograms, or frequency polygonspolygons

Descriptive StatisticsDescriptive Statistics Measures of Central TendencyMeasures of Central Tendency

• ModeMode Most frequently occurring scoreMost frequently occurring score

• MedianMedian Score that divides a group of scores in Score that divides a group of scores in

half.half.• MeanMean

Preferred whenever possible and is the Preferred whenever possible and is the only measure of central tendency that is only measure of central tendency that is used in advanced statistical calculationsused in advanced statistical calculations

An average of all scores. Add up scores An average of all scores. Add up scores and divide by total number of scores.and divide by total number of scores.

Descriptive StatisticsDescriptive Statistics Measures of VariabilityMeasures of Variability

• RangeRange Calculated by subtracting the lowest score from Calculated by subtracting the lowest score from

the highest score. the highest score. Used only for Ordinal, Interval, and Ratio scalesUsed only for Ordinal, Interval, and Ratio scales

• VarianceVariance The extent to which individual scores in a The extent to which individual scores in a

distribution of scores differ from one anotherdistribution of scores differ from one another• Standard DeviationStandard Deviation

The square root of the varianceThe square root of the variance Most widely used measure to describe the Most widely used measure to describe the

variation of a set of observations in a variation of a set of observations in a distribution. distribution.

Descriptive StatisticsDescriptive Statistics CorrelationCorrelation

• Statistical summary of the degree or Statistical summary of the degree or magnitude and direction of the magnitude and direction of the relationship or association between two relationship or association between two variablesvariables

• Correlations can be positive or negativeCorrelations can be positive or negative Linear RegressionLinear Regression

• Make predictions on a new sample of Make predictions on a new sample of observations from the findings on a observations from the findings on a previous sampleprevious sample

Inferential Statistics: Inferential Statistics: SamplingSampling

Degree to which those surveyed are Degree to which those surveyed are representative of a specific populationrepresentative of a specific population

Sampling frameSampling frame: set of people who have : set of people who have the chance to respond to the surveythe chance to respond to the survey

External validityExternal validity: degree to which the : degree to which the sample frame corresponds to the sample frame corresponds to the population to which the researcher wants population to which the researcher wants to apply the resultsto apply the results

Inferential Statistics: Inferential Statistics: SamplingSampling

Two basic typesTwo basic types: probability and non-: probability and non-probabilityprobability

Probability samplingProbability sampling: including : including random samplingrandom sampling

Non-probability samplingNon-probability sampling: including : including convenience samplingconvenience sampling

Response RatesResponse Rates

Most government-sponsored surveys require Most government-sponsored surveys require response rates of 75%response rates of 75%

A response rate of 70% is very good, 60% is A response rate of 70% is very good, 60% is good, and 50% is adequategood, and 50% is adequate

Post-cards, follow-up letters, and telephone Post-cards, follow-up letters, and telephone calls are used to increase the response ratescalls are used to increase the response rates

Results of non-response bias can be Results of non-response bias can be examined by comparing those who respond examined by comparing those who respond early with those who respond after follow upearly with those who respond after follow up

Questions?Questions?