Integrating Student Standardized and Interim Assessments

with National Board's Teacher Development Process

The Eleventh MARCES/MSDE EventOctober 20, 2011

• Overview of how National Board assesses teacher practice and provides standardized feedback based on the Five Core Propositions

• A case study of integrating both student standardized and interim assessments with teacher professional development

• Bringing together the results of teacher and student assessments to enhance teacher practice and student learning

• Considerations for taking these lessons to scale

Session Goals

A Way Forward• The U.S. educational accountability movement has not

yet delivered desired results

– U.S. performance on the OECD’s PISA*

• 14th in reading

• 17th in science

• 25th in math

• We need to support teaching practice – the tools and methods of our learning communities

(Organization for Economic Cooperation and Development’s Programme for International Student Assessment)

• Over the last 20 years, the nation’s focus on measuring educational results has grown and intensified

• Historically, two separate conversations

• Student outcomes

• Teacher evaluation

• NBPTS has always believed that the success of teachers in promoting student learning should be a defining measure of an accomplished teachers

• Teacher demonstration of student learning is integral to becoming an NBCT

Balanced Focus

Policy - Outcome

• Student Achievement• Teacher Evaluation

Practice - Input

• Student Learning• Teacher Growth

The Challenge

How do we leverage accomplished teacher practice to address the societal need of

graduating well-educated students?

Provide states and school districts whole school transformation tools and processes that are:

– Successful– Systematic– Scalable– Sustainable

NBPTS’ Approach

Take One! Whole School Transformation

• Validate a process to improve teaching effectiveness and student learning in low-performing schools

• Assess NB’s Take One! professional development feeder pattern approach in strengthening teacher effectiveness and student outcomes

• Determine feasibility of taking to scale

• Compare two feeder school systems, using one as a control

• Through December 2013

• Initial year, 2010-11, funded by the Gates Foundation

To better understand appropriate measures of teacher practice linked to student performance, NBPTS convened a task force of educational experts in assessment, reform and measuring teacher quality.

Lloyd Bond, University of North Carolina, Greensboro

Peggy Carr, National Center for Education Statistics

Linda Darling-Hammond, Stanford University

Doug Harris, University of Wisconsin, Madison

Frederick Hess, American Enterprise Institute

Robert Linn (chair), University of Colorado, Boulder

Lee Shulman, Carnegie Foundation

Task Force Members

Greater clarity will net better outcomes for college and workforce preparedness

• Student achievement is the status of subject-matter knowledge, understandings and skills at one point in time and is commonly measured by standardized tests.

• Student learning is the growth in subject-matter knowledge, understandings and skills over time. Learning is gauged by comparing student mastery at successive points in time.

Clarity in Language

• Be grounded in student learning, not student achievement– “The only defensible way to determine teacher effectiveness is to focus on

the gains that students have realized over the period during which the teacher provided instruction.”

• Employ measures of student learning explicitly aligned with the elements of curriculum for which the teachers are responsible– “Tests may need to be differentiated to address the needs of the groups of

students being taught, including students with disabilities or language-acquisition needs.”

Recommendations for Teacher Evaluation

As a measure of teacher effectiveness, they must meet the following minimum conditions

• Curriculum-related scale with equivalent unit of measure along a considerable continuum of achievement

• Information on validity of tests for assessing special populations

• A data system that tracks students and links to teachers

• Alignment between tests and state curricula

Criteria for Large-Scale Standardized Tests

What is formally tested in core subjects only

Knowledge and learning that can be measured

All classroom learning

From Learning to MeasuringWhat is Tested Does Count,

But Much of What Counts Is Not Measured

Why Jefferson County School District??

National: NBPTS and Gates

Local:Jefferson County

Schools (Superintendent

Hammonds)

Factors That Make The Difference

• A Structured Approach• Buy-in that Creates a Shift in School Culture• Clarity of Roles• Feeder Systems

Take One Question

Alabama Quality Teaching Standards, Teacher Evaluation,

Professional Learning Plan

Response to Instruction

(RTI)

Continuous Improvement

Plan(CIP)

NBPTSStandards

ForEntry 2

2. What are the relevant characteristics of this class that influenced your instructional strategies for this learning sequence: ethnic, cultural, and linguistic diversity; the range of abilities of the students; the personality of the class?

1.5 Provides instructional accommodations , modifications, and adaptations to meet the needs of each individual learner.

4.7 Understands the characteristics of exceptionality in learning.

Tier IProgress monitoring all studentsIdentification of needs

Part I of the CIP I. Knowledge of Learners II. Knowledge of the Field of Literacy: Reading–Language Arts III. Equity, Fairness, and Diversity IV. Learning Environment V. Instructional Resources VI. Instructional Decision Making VII. Assessment VIII. Integration IX. Reading XIV. Teacher as Learner

A Structured Approach: Alignment of Teacher Tasks

Buy-in to Change School Culture• Critical Mass of Teachers

– Works collaboratively with peers around a common purpose– With colleagues, addresses issues across the feeder system – Focused dialogue and peer support

• Teacher Leader– Augments principals with instructional leadership– Facilitate collaborations, structured discussions, feedback and support

• Principals– Demonstrated leadership through changed interactions with teachers – Structure of environment that supports focused collaboration– Serves as key instructional leader

• School District– Superintendent involvement & presence– Deployment of human capital– Alignment of resources supportive of implementation

• Train-the-trainer model

– Project Director and Coordinator provide leadership and support to principals who will meet once monthly to receive training on guiding their faculty.

• All-participant group meetings– Host four meetings to build professional learning communities among the

feeder school system meetings will focus on, for example, reflective thinking.

• Small group meetings– Twice monthly, principal and facilitator will meet with faculty to guide the

process, lead peer reviews, and provide teacher support.

Project Approach

Feeder System• Permits a systems approach, creating more feedback loops

during the entire K-12 experience

• Facilitates a more long-term connection to students, enabling teachers to preview & follow-up about students

• Strengthens collaboration among teachers at different schools to build connections, strengthen practice & resolve issues

• Enables early identification of student issues, while successful intervention is meaningful

• Enables principal cohort & learning community focused on instruction & classroom improvements

• Eight low-performing schools

• 63% of schools receive Title I funding

• 58% of feeder pattern’s students receiving free/reduced lunch

• High school has failed to meet AYP for the last 5 years

• 210 teachers participating – 70% of all teachers

Feeder System Characteristics

Freq. Respondent Group

Assessment

Broader Learning

CommunityPrincipal Certified

StaffClassified

Staff Parent Student

National Board Take One! Yearly ü ü ü

Engagement Survey, Pre and Post Yearly ü ü ü ü ü ü

Standardized Tests• Alabama Reading and Math

Test/SAT-10 (Grades 3-8)

• EXPLORE and PLAN (Grades 8-11)

Yearly ü

Add’l Student Growth Measures (e.g., attendance rates, promotion/retention rates, graduation /dropout rates)

Yearly ü

Measures

StudentWork

Small Groups Video

Entry 2

Whole GroupVideo

Entry 3

DocumentedAccomplishments

Exercise 1

Exercise 2

Exercise 3

Exercise 4

Exercise 5

Exercise 6

Portfolio Assessment Center

Take One! One Entry of NBPTS’ Certification Process

Teaching Skill

Port. #1

Port. #2

Port. #3

Port. #4

Content Knowledge

AC #1AC #2

AC #3

AC #4

AC #5

AC #6

ε

ε

ε

ε

ε

ε

ε

ε

ε

ε

NBPTS’ Certification Assessment 2-Factor Confirmatory Factor Analysis

Feedback/Comment area Overall (N=165)A. More evidence of knowledge of child development, knowledge of

students, and insight into learning38.8%

B. Evidence of knowledge of the subject as you appropriately design and deliver your classroom lessons

47.3%

C. Evidence that demonstrates your ability to employ effective instructional strategies to facilitate individual participation

63.0%

D. Evidence that you develop a relevant learning sequence 38.8%

E. Evidence of your ability to foster a purposeful learning environment in which all students are encouraged to participate

50.9%

F. Evidence of your ability to set worthwhile goals for learning. 53.9%

G. Evidence that you use a variety of appropriate resources to enhance student learning.

64.2%

H. Evidence that you employ appropriate assessments and provide constructive feedback.

66.7%

I. Evidence that you are able to describe your own practice and reflect on instructional decisions

6.1%

Summary of Standardized Feedback – Year 1

Feedback/Comment area ELEM (N=19)

ELEM (N=21)

ELEM (N=6)

ELEM (N=12)

ELEM (N=24)

ELEM (N=13)

MIDDLE (N=29)

HS (N=41)

Overall (N=165)

A. More evidence of knowledge of child development, knowledge of students, and insight into learning

26.3% 33.3% 0% 58.3% 41.7% 30.8% 41.4% 46.3% 38.8%

B. Evidence of knowledge of the subject as you appropriately design and deliver your classroom lessons

52.6% 42.9% 83.3% 41.7% 58.3% 53.8% 51.7% 31.7% 47.3%

C. Evidence that demonstrates your ability to employ effective instructional strategies to facilitate individual participation

31.6% 66.7% 50.0% 50.0% 41.7% 84.6% 72.4% 80.5% 63.0%

D. Evidence that you develop a relevant learning sequence 15.8% 33.3% 33.3% 50.0% 29.2% 38.5% 58.6% 41.5% 38.8%

E. Evidence of your ability to foster a purposeful learning environment in which all students are encouraged to participate

26.3% 57.1% 33.3% 75.0% 45.8% 30.8% 51.7% 63.4% 50.9%

F. Evidence of your ability to set worthwhile goals for learning. 52.6% 66.7% 33.3% 66.7% 54.2% 46.2% 48.3% 53.7% 53.9%

G. evidence that you use a variety of appropriate resources to enhance student learning.

84.2% 57.1% 33.3% 50.0% 54.2% 69.2% 69.0% 68.3% 64.2%

H. evidence that you employ appropriate assessments and provide constructive feedback.

68.4% 52.4% 50.0% 58.3% 70.8% 69.2% 86.2% 61.0% 66.7%

I. evidence that you are able to describe your own practice and reflect on instructional decisions

0% 14.3% 0% 25.0% 8.3% 0% 0% 4.9% 6.1%

Learning Community Engagement and Leadership SurveyFormative and Summative Administration

EngagementPerceptions About the Learning Community– Principal– Students– Parents– Certified Staff– Classified Staff– Community Members

Leadership 360Perceptions About the Principal’s Behaviors– Principal– Supervisor– Peers– Certified Staff– Classified Staff

Leadership BehaviorsCluster Behaviors

Thinking Gathering DataAnalyzing Information and IdeasThinking FlexiblyReflective Practice

Developing Valuing Others and Their PerspectivesCollaborationDeveloping Human Capital

Inspiring Cultivating Ownership and SupportBuilding ConfidenceCommunication

Achieving Planning, Implementing and EmpoweringGoal Setting and MonitoringStakeholder Centered

Mapped to the Standards in Behavioral Language

Respondent Group

Engagement Survey Dimension

Broader Learning

CommunityPrincipal Certified

StaffClassified

Staff Parent Student x2

1. Level of Academic Challenge ü ü ü ü ü

2. Student-Staff Interaction ü ü ü ü ü

3. Active and Collaborative Learning ü ü ü ü ü

4. Active and Collaborative Teaching ü ü ü    

5. Enhancing Education Experiences ü ü ü ü ü

6. Student Commitment Levels ü ü ü ü ü

7. The School Environment ü ü ü ü ü

8. Parent-Learning Community Interaction ü ü ü ü  

9. Work and Community Engagement ü ü ü  

10. Organizational Citizenship ü ü ü  

11. Work Withdrawal ü ü ü  

12. Turnover Intentions ü ü ü  

13. Broader Learning Community Engagement ü

Cross-Rater Items

Questions to Respondent Group

“Teachers present materials at a level our students can understand.”

“I present materials at a level our students can understand.”

“My teachers present materials at a level I can understand.”“My teachers give lessons at a level I can understand.”

“Teachers present materials at a level my child can understand.”

Pre-post Reporting

Benefits of Student Measures

• Longitudinal results• Guide student development• Positions college as a viable opportunity

Alabama Reading and Math TestThe 4th & 5th grade teachers are new

Grade 4

Grade 4

Grade 5

Grade 5

Grade 5

Grade 5

Math 2011 (post) 614 619 633 638 660 630

Math 2010 (baseline) 587 624 589 622 627 613

Math Gain +27 -5 +44 +16 +33 +17Reading 2011 (post) 611 613 624 629 653 591

Reading 2010 (baseline) 593 596 619 600 641 570

Reading Gain +18 +17 +5 +29 +12 +21

How Are We Doing?Some Interim Assessments

What is formally test in core subjects only

Knowledge and learning that can be measured

All classroom learning

Interim Integrated Classroom Evaluation

Knowing Your

Students

data

attendance

background

Engagement/efficacy

Know Students and Subject AreaThe Architecture of Accomplished Teaching

Step 1: Know Students and Subject Area

Areas to know about your students• Where they are intellectually,

socially & emotionally• Whether they have special

learning needs• How they feel most comfortable

learning• What they know & do not know• What they can & cannot do• What they are interested in • How any issues or event taking

place in their lives may affect learning

Resources • Personal observation of students• Interest inventories & surveys• Student work products• Report cards• Records from previous schools• Standardized tests scores• IEPs• Student extracurricular activities• Results from diagnostics testing• Previous teachers• Guidance counselors• Family members• Students themselves

Student Engagemen

t

Deep Questionin

g

Classroom Practice

Peer Video

Review

Teacher Monitoring

Interim Integrated Classroom EvaluationPre Interim Post

Evaluate Student LearningThe Architecture of Accomplished Teaching

Step 4: Evaluate Student LearningFollowing instruction, evaluate student learning to see if goals were met.

Analyzing Your Video• Extent of classroom involvement• Extent of student engagement in lesson• Evidence of student learning• Student success in meeting learning

goals• Types of questions you asked• Opportunities for students to ask

questions• Adjustment in lesson so learning goals

achieved by every student

Interim Integrated Classroom Evaluation

• Evaluate practice and learning against rubric

Review

Reflect on Teaching PracticeThe Architecture of Accomplished Teaching

5th

Step 5: Reflect on Teaching Practice• What would I do

differently? • What are my next steps?

Reflection• Link reflection to knowledge of your

students• Link reflection back to goals of lesson• Cite specific strengths and weaknesses• Describe appropriateness of your

instructional choices• Provide clear, consistent & convincing

evidence & examples of student learning

• Discuss how you continue with your instruction

• Discuss any modifications to advance student learning

Take One! Submission Rate Comparison

General Population

50%

Take One! Whole School Transformation

80%

2010-11 2011-12

N Total N % N Total N %

NBC N/A N/A N/A 42 277 15%

Take One! 215 310 69% 219 277 79%

NBCP N/A N/A N/A 2 8 25%

Principal Take One! N/A N/A N/A 4 5 80%

TOTAL 215 310 69% 267 290 92%

Participation Increase

Status Preliminary reactions are positive• Teachers - talking, planning, and communicating with each other• Teachers - excited to have the opportunity to work with others in their

field from across the feeder pattern, an atypical experience• Principals - connecting with teachers in new ways through Take One!• Principals - communicating about student support• Superintendent – encouraging his central office staff, principals,

teachers and other staff to build on the success• Everyone -- Identifying strengths and weaknesses across the schools in

the feeder pattern• Everyone - seeing the students as “their students,” a collective

responsibility

Where Are We Going?• Refined year one and building year two• Developing a structured process with a supporting suite

of tools that:– Defines the elements for success– Road-tested and readily available– Is affordable to implement & sustain– Facilitates continued school growth – multi-year– Is research-based

• Getting the word out– Department of Education– Alabama publications

Year One• Teacher selects favorite

lesson to teach

• External experts support teachers after school

• After school cohort meetings

Year Two• Teacher evaluates data and

selects lesson to support area of student need

• Identify and train school-based teacher leaders to build capacity

• In-school cohort meetings

Evaluation and Modification of Program

Measurement is only part of the story

Moving from measuring teacher effectiveness

to…

Building teacher capacity

Move from 30% teachers based on student achievement to 100% teachers based on student growth

Provides the ‘how to’ for developing a sustainable a teacher evaluation and support system that delivers learning for all students

For more informationon NBPTS, visit

nbpts.org