CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND...
-
Upload
leslie-ford -
Category
Documents
-
view
214 -
download
0
Transcript of CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND...
![Page 1: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/1.jpg)
CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS
FOR MEASURING TEACHER EFFECTIVENESSAND STUDENT ACHIEVEMENT
Brenda Burrell, Ph.D. , Paul M. Bole , Ed.D. Janice Janz, Ph.D, & Richelle Voelker, M.Ed.
University of New Orleans (UNO)
Kristin A. Gansle, Ph.D.Louisiana State University (LSU)
![Page 2: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/2.jpg)
CONTRIBUTE TO THE NATIONAL DISCUSSION OF THE ACCOUNTABILITY FOR STUDENT LEARNING THROUGH EFFECTIVE TEACHER PREPARATIONS
Purpose
![Page 3: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/3.jpg)
Assessment • Demand for accountability
• Opportunity to examine the relationship between teaching and learning
• Evidence of strengths, needs for improvement, and impact
“The processes
of teaching
and learning
stimulate
one another.”
(Confucius)
![Page 4: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/4.jpg)
Agenda
Louisiana Department of Education LA Teacher Preparation Program (TPP) AssessmentValue-Added Teacher Evaluation ModelData and Results
UNO Collaborative Redesign ProjectCollaborative ProcessesResults: In-ProgressSubsequent Plans
Conversations with ColleaguesDiscussion QuestionsOpen Discussion
![Page 5: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/5.jpg)
LOUISIANA METRICS OF TEACHER PREPARATION PROGRAM EFFECTIVENESS
INCORPORATING STUDENT ACHIEVEMENT
Kristin A. Gansle , Ph.D.Louisiana State University (LSU)
![Page 6: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/6.jpg)
LA Teacher Preparation Program (TPP) Assessment 2000-2001 & Following
• How to measure our program completers’ success in terms of the achievement of their students?
• Especially important given statewide redesign of academic programs including SpEd by TPPs
• BoR (Jeanne Burns), Blue Ribbon Commission, LSU (George Noell, Kristin Gansle)
• Have been using Value-Added metrics to evaluate TPPs since 2003
![Page 7: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/7.jpg)
What Do We Do?• Predict achievement of individual students based on
prior achievement, demographics, and attendance (using State achievement tests: HLM, mean approx 300, sd approx 50)
• Assess actual student achievement• Link the difference between predicted achievement and
actual student achievement to their new teachers and the TPPs that taught them
• Act on results
![Page 8: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/8.jpg)
Predictors for LDOE Value-AddedTeacher Evaluation Model (Current Model)
• Prior Student Achievement• Attendance• Gifted Classification• Free/Reduced Lunch Status• Section 504 Status• Disability Status• Discipline Records
• Retained students are included in the analysis
![Page 9: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/9.jpg)
Value-Added Model
• State Achievement Tests (Math, Science, Social Studies, Reading, & Language Arts - Grades 4-9)
• New Teachers: • 1st and 2nd year teachers with regular certificates• Teaching within area of certification• Full academic year with their students
• Experienced Teachers:• 3rd or subsequent year teachers with regular certificates • Teaching within area of certification
• Each year: all districts, 250,000 students, 7000+ teachers, 1300+ schools
![Page 10: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/10.jpg)
Data We Provide to TPPs• (October, 2006) By content areas (Reading, Mathematics, Science,
Social Studies, English/Language Arts)
• (March, 2011) By certification/grade spans within content areas (Grades 1-5, Grades 4-8, Grades 6-12, & Special Education)
• (Fall, 2011) • By student performance subsets within content areas (Low,
Middle, and High)• By special education status (yes/no) for student groups• By free lunch status (free lunch/pay own lunch)• By LEP status
![Page 11: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/11.jpg)
Data We Provide to TPPs(continued)
• (Fall 2011) Deidentified individual teacher data for programs• Value-Added Score for each teacher• Mean content standards scores from standardized testing
program (LEAP, iLEAP) for ELA, Math, Reading, Science, Social Studies
• What does that look like?
![Page 12: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/12.jpg)
Overall Results
overall
content Nmean effect sem
ELA 75 1.0 0.7
Mathematics 103 5.1 0.9
Reading 48 -2.2 0.9Science 88 2.7 1.0
Social Studies 66 0.7 1.7
![Page 13: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/13.jpg)
FOR ADDITIONAL INFORMATION
Kristin A. Gansle
Jeanne M. Burns
George H. Noell
http://regents.la.gov/value-added-teacher-preparation-program-assessment-model/
![Page 14: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/14.jpg)
Certification Endorsements
Elementary Grades Middle School Secondary Grades Special Education
content n
mean effect
sem n
mean effect
sem n
mean effect
sem n
mean effect
sem
ELA 42 -0.8 1.1 30 3.1 1.6 41 -1.0 1.1 12 1.2 2.8
Mathematics47 1.4 1.3 69 4.2 1.2 28 3.8 1.3 12 1.0 1.8
Reading 40 -0.6 0.9 8 3.2 2.9 9 3.0 2.6 8 3.7 2.9
Science 38 -1.6 1.1 40 2.5 1.6 20 -0.4 1.3 5 -2.9 1.9
Social Studies35 -2.7 1.7 10 2.0 4.2 19 4.1 3.2 5 -5.1 4.2
![Page 15: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/15.jpg)
Achievement Bands Lowest 25% Middle 50% Highest 25%
content n
mean effect
sem n
mean effect
sem n
mean effect
semELA 25 -4.7 1.7 34 1.0 1.0 21 -2.2 1.2
Mathematics24 -5.2 1.9 31 2.1 1.2 21 -0.6 1.5
Reading 17 -1.8 1.6 24 0.3 0.9 17 -0.2 0.9Science 27 -2.4 1.0 29 0.0 1.1 19 -0.9 1.2
Social Studies30 -0.9 1.5 37 0.0 1.4 25 -0.1 2.1
![Page 16: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/16.jpg)
Free Lunch/Not SpEd/no SpEd
Free Lunch Paid Lunch Special Education No Special Education Services
n
mean effect
sem n
mean effect
sem n
mean effect
sem n
mean effect
sem
ELA 50 -0.5 0.8 47 1.3 0.7 40 0.3 1.4 44 -1.3 0.9
Mathematics40 -1.4 1.0 31 2.0 1.1 20 -1.2 1.1 45 -1.9 1.1
Reading 34 -1.9 0.7 43 1.5 0.8 17 -0.9 1.5 38 -2.1 0.9
Science 38 -0.2 1.0 38 0.8 0.8 23 1.0 1.0 32 0.3 1.0
Social Studies45 -1.4 1.1 40 0.7 1.0 29 -0.8 1.5 40 -1.5 1.1
![Page 17: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/17.jpg)
Deidentified Data in Excel File
TPPs Can Use as They Wish
![Page 18: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/18.jpg)
UNO: PROGRAM DATA
Brenda Burrell, Ph.D.
Paul Bole, Ed.D.
Janice Janz, Ph.D.
& Richelle Voelker, M.Ed.
University of New Orleans (UNO)
![Page 19: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/19.jpg)
Collaborative Processes
•Analyzing and synthesizing professional standards and literature
• Interviews, surveys, and discussions with multiple and diverse stakeholders
• Individual and committee design and development work sessions
![Page 20: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/20.jpg)
Results: Work In-Progress
•Response to Louisiana TPP Assessment•Constructive Principles •Concordance of Professional Standards •Revised Conceptual Framework•Answers to Key Questions: Perspectives
![Page 21: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/21.jpg)
Response to LA TPP AssessmentConcerns/Opportunities
• School administrators may not know what to look for when observing special education instruction
Potential Strategies
• Provide professional development opportunities for school leadership personnel
• Develop guidelines aligned with research based special education practices
![Page 22: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/22.jpg)
Response to LA TPP AssessmentConcerns/Opportunities
• It may be difficult to determine how to assign effectiveness scores to special education teachers working in inclusive settings
• Teachers in high-need schools may not be rated similarly as those in more advantaged schools
Potential Strategies
• Research • Disseminate findings• Make recommendations • Facilitate systemic
changes
![Page 23: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/23.jpg)
Response to LA TPP AssessmentConcerns/Opportunities
• It may be difficult for special education teachers to score “highly effective” on some elements of the Danielson framework
• Some teachers may develop Student Learning Targets (SLTs) that are problematic
Potential Strategies
• Advocate for revised descriptions in rubric to align with special education practices and provide corresponding examples to assist principals when observing effective instructional practices
• Review samples of SLTs developed by special education teachers to determine specific issues and develop corresponding training materials
![Page 24: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/24.jpg)
Constructive Principles•Collaboratively Designed• Informed by Multiple and Diverse Resources•Linked to Professional Standards•Useful for Multiple Assessment Points•Yields Data to Inform/Improve Teacher Education
![Page 25: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/25.jpg)
Concordance of Professional Standards
• Danielson Framework for Teaching
• Council for Exceptional Children (CEC) • International Reading Association (IRA)• National Council of Teachers of English (NCTE)• National Science Teachers Association (NSTA)• National Council for the Social Studies (NCSS)• National Council of Teachers of Mathematics (NCTM)
• Louisiana Components of Effective Teaching • UNO Roles and Responsibilities of Teachers
![Page 26: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/26.jpg)
Revised Conceptual FrameworkRoles & Responsibilities of Effective Teachers
•Manage Classroom Contexts/Environments
•Design Curriculum and Instruction
•Deliver Instruction and Assess Learning
•Participate in Professional Responsibilities (e.g., Advocacy, Collaboration, Using Data to Improve Practice)
![Page 27: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/27.jpg)
Answers to Key Questions: PerspectivesQuestions
• How should IHE faculty assess teachers’ use of evidence-based practices and teacher effectiveness
• How should IHE faculty assess program completers’ impact on student learning and student behavioral progress?
Sample Responses• Video Sample of Teaching
• Surveys/Interviews
• Informal Observations
• Progress Monitoring
• Student Work Samples
• Student/Teacher Portfolios
![Page 28: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/28.jpg)
Subsequent Plans• Develop our own teacher evaluation rubric
• Use same teacher evaluation rubric across concluding semesters of preparation and during initial years of teaching after program completion (induction); using evaluation to provide support
• Conduct field testing and research
• Make recommendations to the Louisiana DOE regarding rubric currently used for State teacher assessment and/or training provided to evaluators
![Page 29: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/29.jpg)
Conversations with Colleagues
Question: Set 1
• What data related to teacher effectiveness and student achievement should and can be collected by faculty in teacher education programs?
• What data will represent authentic and meaningful measures of academic and behavioral performance and growth of students with high incidence disabilities and is also available to university faculty?
![Page 30: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/30.jpg)
Conversations with Colleagues
Question: Set 2• Are administrators and other teacher evaluators sufficiently informed to assess teachers with students with special needs in an inclusive setting or other settings?
• What professional development opportunities would be most helpful and how could or should they be delivered?
![Page 31: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/31.jpg)
Conversations with Colleagues
Question: Set 3• How can IHEs assess positive teacher impact on students who do not demonstrate academic or behavior progress?
• Should teacher performance be aligned exclusively with teacher effectiveness, as indicated by student performance?
![Page 32: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/32.jpg)
Conversations with Colleagues
Open Discussion
![Page 33: CONSTRUCTING AUTHENTIC AND MEANINGFUL ASSESSMENT SYSTEMS FOR MEASURING TEACHER EFFECTIVENESS AND STUDENT ACHIEVEMENT Brenda Burrell, Ph.D., Paul M. Bole,](https://reader036.fdocuments.us/reader036/viewer/2022062516/56649e595503460f94b531bd/html5/thumbnails/33.jpg)
Contact InformationUniversity of New Orleans (UNO)Department of Special Educationand Habilitative Services
Brenda Burrell Janice Janz
[email protected] [email protected]
Paul Bole Richelle Voelker