The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 !...
Transcript of The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 !...
![Page 1: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/1.jpg)
Presented by: Kim Gibbons, Ph.D. Associate Director for Innovation and Outreach
(CAREI)
CLM Fall Conference November 20th, 2015
The Importance of Using Research and Data to Design, Implement, and Evaluate Quality Core Instruction
![Page 2: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/2.jpg)
AGENDA
1. Share some examples of how research can inform practice.
2. Discuss the importance of evaluating core instruction using data.
3. Talk about CAREI and our mission to collaborate in the areas of research, evaluation, and assessment.
![Page 3: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/3.jpg)
THREE CULTURES THAT NEED TO CHANGE
We need to move from: Excuse to accountability Compliance to performance Uniformity to differentiality based on talent and need
![Page 4: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/4.jpg)
CHANGE
True or False: People resist change.
People resist change when they experience a loss.
![Page 5: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/5.jpg)
PIECEMEALNESS
“It is not the pace of change that is the culprit, it is the peacemealness and
fragmentation that wears us down.” Fullan, 2003
![Page 6: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/6.jpg)
QUESTIONS
How important is research to you as administrator? How do you stay current on research? What are the barriers to using research to inform practice?
![Page 7: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/7.jpg)
REASSESSING OUR DIRECTION IS NOT ALWAYS EASY!
How to do it:
1. When adopting new policies, programs, or curriculum, examine the research base and pick things that have already been proven to work with large effect sizes.
2. Take inventory of current policies, programs, & practices and compare to current research.
3. Continually evaluate policies, programs, & practices using data to drive decisions.
![Page 8: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/8.jpg)
![Page 9: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/9.jpg)
0.4 0.15
0.0
-‐0.2
Teacher
Developmental
Reverse
1.2
Neg
ative
Medium
d = 1.09
0.7
1.0
Response to Intervention (MTSS)
Desired Effects
![Page 10: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/10.jpg)
VARIABLES IMPACTING STUDENT ACHIEVEMENT: THE BIGGIES!
Student self report of grades 1.44
Formative Evaluation .90
Teacher Clarity .75
Reciprocal Teaching .74
Feedback .73
Teacher-Student Relationship .72
Spaced vs. mass practice .71
Metacognitive strategies .69
Repeated Reading .67
Vocabulary Programs .67
![Page 11: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/11.jpg)
S e =
0.4 0.15
0.0
-‐0.2
Teacher
Developmental
Reverse
1.2
Neg
ative
Medium
Feedback d = 0.90
0.7
1.0
Formative evaluation
Teaching
Desired Effects
![Page 12: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/12.jpg)
![Page 13: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/13.jpg)
ASK THESE QUESTIONS:
Does our district have reliable and valid systems in place to monitor the progress of students receiving supplemental and intensive intervention? Do teachers in our district regularly collect formative assessment data to guide their decisions in the classroom with all students?
![Page 14: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/14.jpg)
S e =
0.4 0.15
0.0
-‐0.2
Teacher
Developmental
Reverse
1.2
Neg
ative
Medium
Feedback d = 0.73
0.7
1.0
Feedback
Teaching
Desired Effects
![Page 15: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/15.jpg)
FEEDBACK (.73)
Feedback is most powerful when it is from student to teacher!
Teachers need to seek and be open to feedback about: What students know What they understand Where they make errors When they have misconceptions When they are not engaged
Feedback is the breakfast of champions! -‐ Ken Blanchard
![Page 16: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/16.jpg)
ASK THESE QUESTIONS:
1. How do teachers in your district know: What students know? What they understand? Where they make errors? When they have misconceptions? When they are not engaged?
2. What type of professional development and coaching exists in this area?
![Page 17: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/17.jpg)
S e =
0.4 0.15
0.0
-‐0.2
Teacher
Developmental
Reverse
1.2
Neg
ative
Medium
© John Hattie Visible Learning
Feedback d = .72
0.7
1.0
Relationships
Teacher
Desired Effects
![Page 18: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/18.jpg)
TEACHER STUDENT RELATIONSHIPS
Factors that are important:
ü Demonstrate that you care for the learning of each student as a person
ü High levels of empathy (see their perspective and communicate back)
ü Warmth
ü Encouragement of higher-‐order thinking
ü Adapting to differences
ü Genuineness
ü Respect of self and others
![Page 19: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/19.jpg)
ASK THESE QUESTIONS:
ü How do teachers in your district connect with students who have challenging behaviors?
ü How do teachers in your district connect with students who lack motivation?
ü How do teachers in your district connect with students who are very shy and withdrawn?
ü How do teachers in your district connect with students from different ethnic backgrounds?
![Page 20: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/20.jpg)
THE POLITICS OF DISTRACTION (HATTIE)
• Too much discussion focused on between school differences (34%) when the biggest issue is within school differences (64%)!
• What is the cause of the variance?
• Teachers: Variability in impact on student learning
• We all know this, but it is absent in any discussions about policy, teachers, and schools!
![Page 21: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/21.jpg)
INSTEAD WE USE………
The Politics of Distraction!
![Page 22: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/22.jpg)
DISTRACTION #1: APPEASE THE PARENTS
q School Choice q Class Size What changes in teaching as result of smaller class size?
![Page 23: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/23.jpg)
DISTRACTION #2: FIX THE INFRASTRUCTURE
Do we need:
• More effective curriculum?
• More rigorous standards?
• More tests?
• More alternatively shaped buildings?
![Page 24: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/24.jpg)
DISTRACTION #3: FIX THE STUDENTS
If only we had better and more well-‐prepared students!
ü Label students who don’t fit the norm.
ü Hold students back. ü Teach to learning styles.
![Page 25: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/25.jpg)
DISTRACTION #4: FIX THE SCHOOLS!
If only schools had more money and more autonomy they would be better schools! ü Charter schools ü Local decision-‐making ü New leaders ü Invest more money
![Page 26: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/26.jpg)
DISTRACTION #5: FIX THE TEACHERS
If only teachers had better initial training, were paid for performance, and had better technology!
ü Change teacher education – while there may be some truth to this, the greatest source of teacher learning is not from their teacher education program but from their first year of teaching, and then from their second. After the second year, teacher education programs matters very little.
ü Performance Pay
ü Technology ü More adults in schools
![Page 27: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/27.jpg)
SUMMING IT ALL UP!
ü We love talking about distractors that don’t matter!
ü Some of our most politically popular “fixes” have the lowest impact on student achievement!
![Page 28: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/28.jpg)
![Page 29: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/29.jpg)
EVALUATING CORE INSTRUCTION
![Page 30: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/30.jpg)
WHY SHOULD WE EVALUATE CORE INSTRUCTION?
![Page 31: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/31.jpg)
EVALUATING CORE INSTRUCTION
1. Problem Identification Is the core program sufficient?
2. Problem Analysis If the core program is not
sufficient, why isn’t it?
3. Plan Development How will the needs identified in
the core be addressed?
4. Plan Implementation How will the effectiveness and efficiency of
the core be monitored over time?
5. Plan Evaluation Have improvement to the
core been effective?
![Page 32: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/32.jpg)
TURN AND TALK
How do you define a sufficient core instructional program? “A sufficient core instructional program is one in which…”
![Page 33: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/33.jpg)
DEFINING A STRONG CORE
All materials & instruction used to provide the main classroom instruction in a particular content area Often more than a single textbook
Whatever it takes to get most students meeting grade level standards Will differ from district to district, school to school, cohort to cohort
![Page 34: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/34.jpg)
GOAL FOR A STRONG CORE
To create a core instructional program that results in about 80 percent of students meeting grade level expectations without additional support
At least 95% of students who begin the year at grade level expectations will end the year (begin the next year) at grade level expectations Utilizing evidence based materials and instructional techniques
Utilizing personnel and time resources creatively and wisely
![Page 35: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/35.jpg)
A SUFFICIENT CORE PROGRAM
At least 80% of students will meet grade level standards given access to core instruction alone These students did not need supplemental support to meet grade level standards Under this goal scenario, more than 80% of students are proficient: some reached proficiency through the support of core + supplemental or intensive instruction!
![Page 36: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/36.jpg)
![Page 37: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/37.jpg)
![Page 38: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/38.jpg)
ASSESSING CORE PROGRAM SUFFICIENCY
MCA and other District Assessments: Total student population Students who only received core instruction Disaggregated subgroups Individual grade levels Subject areas
![Page 39: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/39.jpg)
KEY QUESTIONS
What percentage of students in each grade level met our target expectations in fall, winter, and spring in Reading and Math?
![Page 40: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/40.jpg)
KEY QUESTIONS
How do percent above target scores for current year in each grade level compare to those of previous (3?) years?
![Page 41: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/41.jpg)
KEY QUESTIONS
What percentage of students in our grade level met our target expectations in spring without receiving supplemental interventions?
![Page 42: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/42.jpg)
KEY QUESTIONS
What percentage of students who began the year at or above target also ended the year at or above target?
Fall 76% above target Spring 75% above target
136 = 91% stayed proficient 150
![Page 43: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/43.jpg)
HOW DO YOU KNOW:
What percentage of students who began the year at or above target also ended the year at or above target?
![Page 44: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/44.jpg)
![Page 45: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/45.jpg)
EVALUATION OF IMPLEMENTATION: THE BIG FIVE!
Assessments Data-‐Based Decision Making
Multilevel Instruction
Infrastructure & Support
Fidelity & Evaluation
![Page 46: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/46.jpg)
Tools • Reliable, valid, predictive validity • Staff can articulate
Screening • All students, implementation accuracy, more than once per year
Data Points • Screening data plus two other data sources • Convergence
PM Tools • Alternate forms with equal difficulty, specification of minimum expected growth, benchmarks, reliable & valid
PM Process
• Schedules • Implementation accuracy
Assessments
![Page 47: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/47.jpg)
DATA-‐BASED DECISION MAKING
Process
• Decisions about participation in intervention levels is data-‐driven, involves teams, operationalized with clear decision rules
Data System
• System in place to access data, timely, graphical display, process for setting and evaluating goals.
Responsiveness
• Decisions made based on reliable and valid data reflecting slope or goal attainments
• Implemented accurately
![Page 48: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/48.jpg)
MULTILEVEL INSTRUCTION
Research-‐Based Curriculum
Articulations of Teaching and Learning
Differentiated Instruction
Standards Based
Exceeding Benchmark
Evidence-‐Based
Compliments Universal
Instructional Characteristics
Supplemental to Universal
Intensity matched to Need
Instructional Characterizes
Relationship to Universal
Universal Tier 1 Supplemental
Tier 2 Intensive Tier 3
![Page 49: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/49.jpg)
Fidelity & Evaluation
Curriculum & Instruction
Assessments
Action plan Evaluation of Plan
Review of Student Data
Implementation Data
Fidelity
Evaluation
![Page 50: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/50.jpg)
Adherence
Exposure
Quality of Delivery
Program Specificity
Student Engagement
FIVE ELEMENTS OF FIDELITY
50
(Dane & Schneider, 1998; Gresham et al., 1993; O’Donnell, 2008)
Adherence: How well do we stick to the plan/ curriculum/assessment?
Exposure/Duration: How often does a student receive an intervention? How long does an intervention last?
Quality of Delivery: How well is the intervention, assessment, or instruction delivered? Do you use good teaching practices?
Program specificity: How well is the intervention defined and different from other interventions?
Student Engagement: How engaged and involved are the students in this intervention or activity?
![Page 51: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/51.jpg)
FIDELITY AT THE PRIMARY PREVENTION LEVEL
Staff implements program components or core components of the curriculum with fidelity Might not be scripted but is comprehensive given the age group and content area
High-‐quality instruction Includes differentiation
Staff may use a mix of whole group, small group, dyadic, and independent practice
§ Staff implement curriculum based on established delivery timeframe For example: Provide 90–120 minutes of reading instruction five days a week
Students are engaged throughout lessons 51
![Page 52: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/52.jpg)
Take a moment to discuss with your colleagues…
How do we evaluate our core instruction? What data do we use, how often, where stored? What are our areas of strength and weakness? How will we work to address these?
Your Turn
![Page 53: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/53.jpg)
CENTER FOR APPLIED RESEARCH AND EDUCATIONAL IMPROVEMENT
(CAREI) How can we help?
![Page 54: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/54.jpg)
CONVERSATIONS AND SURVEYS What we know:
ü Most educators agree that quality data can improve decision making!
ü Schools are data rich but information poor. An overarching theme was that our school staff and leaders need to understand how to use data for various types of decisions related to student outcomes!
Needs Around Data Usage:
ü Professional development around data literacy
ü Point person with expertise
ü Customized Data Reports
ü Timely Data
ü Help with evaluating existing assessments (almost half indicated this was a need)
ü Time to Collaborate
![Page 55: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/55.jpg)
CAPACITY BUILDING AROUND DATA USE
ü A recent survey of MASA and MASE members indicted that only 7% of respondents believe they have “very good” capacity for data use.
ü Staff expertise is an issue. ü Needs exist in the areas of:
Providing training on data interpretation How to use data to make decisions Developing surveys and questionnaires Data for resource allocation decisions Selecting diagnostic assessments Infrastructure for progress monitoring Analyze data from screening Selecting screening assessments
![Page 56: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/56.jpg)
SUMMARY OF CONVERSATIONS AND SURVEYS
1. District level respondents believe that quality data can improve their decision-‐making.
2. There is a large need for increased data literacy at all levels (teachers and administrators).
3. Districts currently lack capacity in the area of data based decision making for a variety of reasons. Only 33% of districts indicated that they have staff with advanced training in evaluation and assessment.
4. Statewide MTSS implementation survey indicates many needs Ø Differentiated instruction (27% full implementation) Ø Multiple measures used to evaluate effectiveness of instruction (21%) and for
decision making (23%) Ø Evaluation of interventions (29%) Ø Accessible and timely data for decision making (40%) Ø Valid and reliable assessments at the unit and lesson levels (30%) Ø Use of valid and reliable progress monitoring measures (39%) Ø Using progress monitoring data to determine if interventions are effective (33%)
![Page 57: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/57.jpg)
CAREI WANTS TO HELP!
v CAREI is well-‐positioned to address the statewide gaps in data based decision making!
v We can assist with data literacy training!
v We have staff with high levels of expertise in areas of research, evaluation, and assessment!
v We can increase services statewide through collaboration!
![Page 58: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/58.jpg)
EXAMPLES OF QUESTIONS WE CAN ANSWER:
1. What is the current level of implementation of a district framework or initiative (e.g., MTSS, PBIS,PLC’s Anti-‐bullying Curriculum)? What things are working well? Barriers to success?
2. What are the effects of school wide or district wide decisions (block scheduling, after-‐school programs, curriculum adoption, etc.) on student achievement?
3. What is the effect of a flipped instruction model on student achievement? Does this framework produce better outcomes for certain students?
4. What impact do 1:1 technology initiatives have on student achievement? Student engagement?
5. Is a particular assessment valid for the intended purpose? 6. What is the process to develop reliable and valid common formative
assessments?
![Page 59: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/59.jpg)
TECHNICAL ASSISTANCE WE CAN PROVIDE:
1. How to develop surveys to assess …… (student engagement, staff satisfaction, parental engagement)
2. Customized data reports targeting district variables of interest.
3. What types of assessments would be needed to answer specific district questions.
4. And much more! If you ask, we can help figure it out!
![Page 60: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/60.jpg)
CAREI ASSEMBLY ü Participation in the District Assembly four times per year with
remote access to meetings via video conferencing.
ü Participation in the CAREI collaborative grant program, with the potential of your district receiving up to $3,500 in funds for a teacher/leader in your district to collaborate with a College of Education faculty member on a co-‐designed research project.
ü A half-‐day of research consultation available to your district at no cost (e.g., meeting with school leaders, feedback on a survey design, a brief compilation of research literature on a topic, etc.) with CAREI staff.
ü Linkage to other resources in the University of Minnesota.
ü At-‐cost assistance in larger projects, such as large-‐scale survey design and data analysis, in-‐district or grant-‐funded evaluation projects, and so on.
![Page 61: The$Importance$of$Using$Research$and$Data$to$ … · 2015-11-23 · Formative Evaluation .90 ! Teacher Clarity .75 ! Reciprocal Teaching .74 ! Feedback .73 ! Teacher-Student Relationship](https://reader033.fdocuments.us/reader033/viewer/2022043011/5fa70ba4bc234f2bca644acd/html5/thumbnails/61.jpg)
FOR MORE INFORMATION
Please contact Kim Gibbons! [email protected] 651-‐303-‐4141
CAREI Website http://www.cehd.umn.edu/carei/