Mathematics Performance TasksApplying a Program Logic Model to a

Professional Development Series

California Educational Research AssociationDecember 6, 2013

Sally J. Bennett-Schmidt, Ed.D.Assessment Director

San Diego County Office of Education

Overview of the Session

•Context of the Professional Learning Experience•Summary of the Professional Development Series•Application of the Program Logic Model▫Vision & Current Situation▫Resources & Activities▫Outputs▫Outcomes: Short-, Intermediate-, Long-Term

• Learnings and Next Steps

Context of the Project

•San Diego County Office of Education▫Support 42 school districts + county-run programs▫Curriculum, instruction, and assessment staff working

collaboratively to support districts in implementation of the common core state standards

•San Diego County School Districts▫Along a continuum in implementation of CCSS▫Substantial needs in mathematics, both content

knowledge and instructional approaches

The Professional Development Series:Mathematics Performance Tasks & Formative Assessment

•Purpose: Support district efforts in the use of performance tasks to maximize student achievement

•Key Activities▫Examine assessment in the light of the new Common Core

State Standards and work being explored by the Smarter Balanced Assessment Consortium

▫Experience performance tasks along with scoring protocols▫Explore the formative use of performance task results

including student re-engagement▫ Incorporate formative assessment practices in districts’

ongoing work

The Professional Development Series

•Days 1-2: August 2012▫Audience: District teams of 5-7 members (district and

site administrators, mathematics teacher leaders)▫Facilitated by David Foster from the Silicon Valley

Mathematics Initiative▫Focus:

Understand the new assessment context under the common core standards

Experience several performance tasks with scoring and analysis protocols

Begin discussion of the formative use of performance tasks and student re-engagement

The Professional Development Series

•Day 3: September – October 2012▫Audience: Expanded district teams with additional

classroom teachers▫Facilitated by SDCOE Mathematics & Assessment

Staff▫Focus:

Select an extended response mathematics task to use with students

Anticipate student successes and challenges Plan for embedding the task in instruction

The Professional Development Series

•Day 4: December 2012 – January 2013▫Audience: Expanded district teams with additional

classroom teachers▫Facilitated by SDCOE Mathematics & Assessment

Staff▫Focus:

Score student work from the performance task Analyze student performance for mathematical

understanding and misunderstanding Plan a re-engagement lesson

The Program Logic Model

•Context for using a Logic Model▫Learning and Leadership Services Division began

exploring tools to help us better determine the impact of our work on district systems, instructional leadership, teacher practice, and student learning.

▫Logic model introduced as a tool to help us frame our services within an outcome orientation For planning For program evaluation

▫Logic model completed via a series of guiding questions

The Program Logic Model

1. What is our vision and our current situation?2. Who do we serve?3. What end result do we desire – how will the “rules

of the game” change?4. What new behaviors will we see?5. What will change right away?6. What do we do?7. What will we count?8. What will we need to make it happen?9. How will we know we’re on the right track?

The Program Logic ModelSituation / Vision: Question 1Target Audience: Question 2

Assumptions, Environmental / External Factors

Resources

Question 8:What will

we need to make it

happen?

OutcomesStrategies Activities

Question 6:What do we do?

Outputs

Question 7:What will we count?

Short-Term

Question 5: What will

change right away?

Intermediate

Question 4: What new behaviors

will we see?

Long-Term

Question 3: How will the rules of the

game change?

Indicators

Question 9: How will we know we’re on track?

Vision

•Mathematics instruction will reflect the common core instructional shifts, and will embed high-quality tasks and formative assessment strategies to gauge students’ conceptual understanding and re-engage students in mathematical thinking.

Current Situation

•Current classroom instructional practices generally reflect training aligned to curricular materials and prior state content standards.

•Teachers have had limited professional learning and experience with embedding high-quality, open-ended mathematics tasks and formative assessment into instructional practice.

•Student achievement in mathematics has improved, but continues to show weaker performance in middle and high school grade levels and courses.

Target Audience

•Mathematics teacher leaders•Site administrators•District curriculum / instruction / assessment

administrators

Resources

•SDCOE Mathematics and Assessment staff (managers and support staff)

•Time for collaborative planning and delivery of professional development

•Funding to contract for SVMI member resources•Funding for costs associated with professional

development sessions

Activities

•Four-part mathematics performance task and formative assessment professional development series

•Contract for SDCOE and small districts in the county to access SVMI resources

Output and Indicators•Number of professional development sessions▫1 four-part series; 9 total sessions (Day 3 offered four times;

Day 4 offered three times)•Number of participating teachers and administrators▫135 participants from 14 districts and 4 independent charters

•Number of teachers that select and administer a task from SVMI MARS or other resource▫88% of participating classroom teachers administered a task

•Number of teachers that score and analyze student work▫76% brought back student work to score and analyze

Short-term Outcomes and Indicators

• Increased awareness of how mathematics performance tasks and formative assessment can improve instruction and student learning▫94% reported better understanding of assessment in

relation to CCSS▫78% reported better understanding of analyzing student

work to identify students’ mathematical understanding

Short-term Outcomes and Indicators

• Increased awareness of where to find mathematics performance tasks, rubrics, and scored student papers▫78% reported better understanding of how to find and

access identified websites to select performance tasks• Increased awareness of how to select and administer a

performance task, and score student work using a rubric with training & standardizing papers▫89% reported the PD helped them plan to administer a

performance task with their students▫86% reported the PD helped them score and analyze

their students’ work

Intermediate Outcomes & Indicators

• Teachers try out performance tasks in classrooms, and score and analyze student work▫88% tried out a performance task▫76% scored and analyzed student work

• Teachers use student work for professional conversation▫100% of teachers who brought back work engaged in

professional conversation on Day 4▫Proposed use of teacher learning logs to document continuing

practice• Teachers share student work and instructional practices

through PLCs▫Proposed use of PLC logs and observations at a sample of sites

Long-term Outcomes & Indicators•Teachers and students engage in rigorous, coherent,

and focused mathematics learning experiences•Classroom practice reflects high-quality mathematics

instruction and assessment•PLCs are focused on student learning and instruction•Student achievement in mathematics improves▫Teacher learning logs and classroom observations▫PLC logs and observations▫District / Site and Smarter Balanced assessment results

Learnings & Next Steps

•What we learned about supporting teacher use of mathematics tasks and formative assessment strategies▫More time was needed on re-engagement portion of

mathematics performance task & formative assessment work

▫Teachers need continued support to embed tasks and formative assessment strategies into ongoing instructional practice

▫School site structures need to support teacher collaboration around selecting tasks, analyzing student work, and re-engaging students in mathematics learning

Learnings & Next Steps

•What we learned about collecting “evidence of impact”▫We were able to collect evidence of short-term

outcomes in terms of increased awareness▫We were able to collect limited evidence of

intermediate outcomes in terms of changing practice▫We need a structured plan and resources (human,

time) to collect evidence of intermediate and longer-term outcomes

Learnings & Next Steps

•What we learned about the program logic model▫The approach helped us better define our service(s)

and our desired outcomes▫The tool helped us differentiate short, intermediate,

and long-term outcomes and identify appropriate evidence to collect to assess impact

▫The process helped us understand that, to assess long-term outcomes / impact, we need a multi-year project and ongoing data collection methods

Contact Information

•Sally J. Bennett-Schmidt, Assessment Director▫San Diego County Office of Education▫858-292-3688▫sbennettschmidt@sdcoe.net