Post on 20-Dec-2015
Work Session 2Methodological Approaches
Office of Research, Evaluation and Policy StudiesXimena D. Burgin, Ed.D.
December 2nd, 2010
Intent of a Study
Purpose statement: Overall directioni.e.,The purpose of this grant is to enhance and expand existing
partnerships. The Partnership and Community Readiness section of the application should demonstrate that a school-community partnership has been in existence for 6 or more months prior to submission of the application (henceforth referred to as preexisting school-community partnership). The preexisting school community partnership should involve at least two of the four required SS/HS partners that have worked together on issues of school safety, drug and violence prevention, and/or healthy childhood development issues for 6 or more months prior to submission of the application.
Work Plan Requirements…
The SS/HS Initiative draws on the best practices of education, justice, social services, and mental health systems to provide integrated and comprehensive resources for prevention programs and prosocial services for youth. This comprehensive plan include six elements to address problems of school violence and alcohol and other drug abuse:♦ Element 1: Safe school environment.♦ Element 2: Alcohol and other drugs and violence prevention
and early intervention programs.♦ Element 3: School and community mental health preventive
and treatment interventionservices.♦ Element 4: Early childhood psychosocial and emotional
development programs.♦ Element 5: Supporting and connecting schools and
communities.♦ Element 6: Safe school policies.
Grant Measurements…
GPRA1 – SS/HS grant sites will experience a decrease in the number of violent incidents at schools during the 3-year grant period.
GPRA2 – SS/HS grant sites will experience a decrease in substance use during the 3-year grant period.
GPRA4 – SS/HS grant sites will increase mental health services to students and families during the 3-year grant period.
Grant Outcomes…GPRA1 – SS/HS grant sites will experience a decrease in the number of violent incidents
at schools during the 3-year grant period.
Outcomes: Objective 1. In each of three years cumulatively, decrease by 60% the percentage of violent incidence occurrence for each of the categories: threatening behavior, unprovoked attack on student, fighting, fighting with or threatening teacher, possession or use of dangerous weapon, arson, and endangering safety of others.
GPRA2 – SS/HS grant sites will experience a decrease in substance use during the 3-year grant period.
Outcomes: Objective 2. In each of three years cumulatively, decrease by 60% the percentage of substance abuse occurrence for each of the categories: alcohol, tobacco, and controlled substances (possession or use).
GPRA4 – SS/HS grant sites will increase mental health services to students and families
during the 3-year grant period.
Outcome: Objective 4. In each of three years cumulatively, increase by 50% the percentage of students and families accessing and receiving mental health services, as measured by frequency of service requested or inquired about, follow-through in accessing such service, and persistence in pursuing such service to closure.
Is the study feasible?
Tradeoffs between rigor and practicality. You may have to: Control the implementation of your program more
carefully. Ask program participants lots of questions. Evaluate research rigor, desire and practicality.
Is the study feasible?…cont.
Practical considerations for a feasible research project: How long the research will take to accomplish. Ethical constraints Needed cooperation Costs.
GPRAs &Outcomes
(Research Questions)
Quantitative Research Questions
Quantitative research is mainly concerned with the degree in which phenomena possess certain properties, states and characters, and the similarities, differences and causal relations that exist within and between.
Qualitative Research Questions
Qualitative research is concerned with the properties, the state and the character (i.e., the nature, of phenomena). Emphasis on processes and meanings.
Grant Outcomes…
GPRA1 – SS/HS grant sites will experience a decrease in the number of violent incidents at schools during the 3-year grant period.
Outcomes: Objective 1. In each of three years cumulatively, decrease by 60% the percentage of violent incidence occurrence for each of the categories: threatening behavior, unprovoked attack on student, fighting, fighting with or threatening teacher, possession or use of dangerous weapon, arson, and endangering safety of others.
Grant Outcomes…
GPRA2 – SS/HS grant sites will experience a decrease in substance use during the 3-year grant period.
Outcomes: Objective 2. In each of three years cumulatively, decrease by 60% the percentage of substance abuse occurrence for each of the categories: alcohol, tobacco, and controlled substances (possession or use).
Grant Outcomes…
GPRA4 – SS/HS grant sites will increase mental health services to students and families during the 3-year grant period.
Outcome: Objective 4. In each of three years cumulatively, increase by 50% the percentage of students and families accessing and receiving mental health services, as measured by frequency of service requested or inquired about, follow-through in accessing such service, and persistence in pursuing such service to closure.
Qualitative Analysis
Qualitative analysis seeks to capture the richness of people's experience in their own terms. Understanding and meaning emerge from in-depth analysis of detailed descriptions and verbatim quotations.
QUANTITATIVE (Deductive)
• Ho, Ha• RQ• Variables• Predict• Instruments before
study starts• Difference over time.
Differentiating Approaches
QUALITATIVE(Inductive)
• Ho, Ha• RQ• Single concept (central phenomena)• Build themes• Open-ended questions.
Stance-emerge during the study
• Deep understanding.
Research Methodology
Experimental design Correlational design Survey design Grounded theory design Ethnographic design Narrative design Mixed design Action research design
Experimental design
To establish possible cause-effect between IV and DV. Between-group designs
True experiments Quasi-experiments Factorial analysis
Within-group or Individual designs Time series experiments (interrupted, equivalent) Repeated measures experiments Single-subject experiments
Correlational design
To predict scores and explain the relationship among variables.
Raw Data: Self-esteem GPA score1 42 3.82 10 1.43 15 2.54 22 3.1
Resource: http://www.uky.edu/Classes/PSY/313-001/research%20method%20ch2s.ppt
Survey Design
To describe attitudes, opinions, behaviors, or characteristics.
o Cross-sectional survey designs
Resource:http://images.freshmeat.net/editorials/r_intro/images/line-graph-1.jpg
Survey Design…
O Longitudinal survey
Designs
Resource: http://www.nature.com/eye/journal/v21/n6/images/6702322f1.jpg
Survey Design…
o Trend studies
Resource: http://coastalchange.ucsd.edu/images/wet_dry_2.gif
Survey Design…
o Cohort studies
Resource: http://www.ncsall.net/fileadmin/resources/research/br11-graph2.gif
Survey Design…
o Panel studies
Resource: http://www.nsf.gov/news/special_reports/survey/images/dynamic_logo_sm.jpg
Grounded Theory Design
Systematic design Open coding Axial coding Selective coding
Grounded Theory Design…
Emerging design Exists at the abstract level. Grounded in data Meet four central criteria:
fit work relevance modifiability
Grounded Theory Design…
Constructivist design Position between a positivist view and postmodern
view. Interest on views, values, beliefs, feelings,
assumptions, and ideologies of individuals. Explain the experience of the phenomenon or
process rather than facts and acts.
Ethnographic Design
Realist ethnography Confessional ethnography Life history Autoethnography Microethnography Ethnography case study
Ethnographic Design…
Critical ethnography Feminist ethnography Postmodern ethnography Ethnography novels
Narrative Research Design
Autobiographies Biographies Life writing/histories/stories Personal accounts/narratives/documents Narrative interviews So on
Mixed Methods Design
Triangulation Mixed Methods Designs Embedded Mixed Methods Designs
Resource: Hanson et al, 2005
Mixed Methods Design…
Explanatory Mixed Methods Designs Exploratory Mixed Methods Designs
Resource: Hanson et al, 2005
Mixed Methods Design…
Resource: Burke & Onwuegbuzie, 2004
Mixed Methods Design…
Resource: Hanson et al, 2005
Action Research Design
Practical Participatory
Sampling
Random sampling methods: Simple random sampling
Sampling…
Stratified random sampling Cluster random sampling
Resource: http://www.socialresearchmethods.net/kb/Assets/images/sampstrt.gif
Sampling…
Two-stage random sampling: combine Cluster random sampling & Simple random sampling
Resource: http://www.clas.ufl.edu/jur/200411/images/emerson_2.jpg
Sampling
Nonrandom sampling methods: Systematic sampling
Resource: http://www.disector.com/stereology-analysis-graphics/sampling1.gif
Sampling…
Convenient sampling
Resource: http://www.nedarc.org/nedarc/collectingData/participantSelection/probabilitySampling/images/simpleSample.gif
Sampling…
Purposive sampling
Resource:http://www.david.curtis.care4free.net/esfig3.gif
Sample Size
It depends on the type of study. However, the larger the sample the better.
Quasi-Experimental Design
REQUEST FOR APPLICATIONS Evaluation of State and Local Education Programs and Policies CFDA Number: 84.305E
C. Implementing Rigorous Evaluations “The methodological requirements for evaluations under
this program…other experimental and quasi-experimental designs that substantially minimize selection bias or allow it to be modeled may be employed” (pg.7).a. Random Assignment “Studies using random assignment to intervention and
comparison conditions have the strongest internal validity for causal conclusions and thus are preferred whenever they are feasible” (pg.7).
Threats to Validity: Internal
Related to participants: History Maturation Regression Selection Mortality Interaction with selection
Related to treatments: Diffusion of treatments Compensatory
equalization Compensatory rivalry Resentful
demoralization
Related to procedures: Testing Instrumentation
Threats to Validity: External
External: It may affect generalizability: Interaction of selection
and treatment Interaction of setting
and treatment Interaction of history
and treatment
Threats to Internal Validity in Types of Experimental Designs True Experiment Quasi-Experimental
To Participants: Controlled Potential Threat
History Controlled Potential Threat
Maturation Controlled Potential Threat
Regression Controlled Potential Threat
Selection Controlled Potential Threat
Mortality Controlled Potential Threat
Interaction of selection and maturation, history, and instrumentation.
Controlled Potential Threat
Threats to Internal Validity in Types of Experimental Designs
True Experiment Quasi-Experimental
To Procedures:
TestingPotential Threat if pre and posttest
used
Potential Threat if pre and posttest used
Instrumentation
Potential Threat if instrument or observational
procedures change
Potential Threat if instrument or observational
procedures change
Types of Between-Group Designs Quasi-Experimental Designs
Select Control Group Pretest No treatment Posttest
Select Experimental Group Pretest Experimental Treatment Posttest
Pre- and Posttest DesignTime
Select Control Group No treatment Posttest
Select Experimental Group Experimental Treatment Posttest
Post-Only DesignTime