Use and Impact of the Use and Impact of the Student Progress Student Progress

Monitoring SystemMonitoring System

Mississippi Department of EducationOffice of Instructional Programs and Services

Office of Student AssessmentMASS January 10, 2006

2

Assessment that is explicitly designed to Assessment that is explicitly designed to promote learning is the single most promote learning is the single most powerful tool we have for raising standards powerful tool we have for raising standards and empowering life long learning.and empowering life long learning.

– – Assessment ReformAssessment Reform

Group (1999)Group (1999)

3

Student Progress Monitoring System (SPMS) is Student Progress Monitoring System (SPMS) is designed to offer a web-based approach to designed to offer a web-based approach to help districts track which students are help districts track which students are mastering key objectives and which are not.mastering key objectives and which are not.

The ultimate goal of SPMS is that teachers The ultimate goal of SPMS is that teachers use SPMS assessments to focus on reviewing, use SPMS assessments to focus on reviewing, revising, and improving daily instruction revising, and improving daily instruction through rapid assessment and feedback.through rapid assessment and feedback.

Through the daily/weekly review of student Through the daily/weekly review of student progress, districts will have a better way to progress, districts will have a better way to track their expectations for meeting annual track their expectations for meeting annual accountability requirements and for preparing accountability requirements and for preparing students for national standardized tests.students for national standardized tests.

Overview

4

District UseDistrict Use Participation is a district and school decision. Participation is a district and school decision.

Districts registration for SPMS may include all Districts registration for SPMS may include all schools or only targeted schools.schools or only targeted schools.

For the 2005-06 year, the state is funding the For the 2005-06 year, the state is funding the entire per pupil cost, and the MDE intends to entire per pupil cost, and the MDE intends to fully fund SPMS for the 2006-07 school year.fully fund SPMS for the 2006-07 school year.

Implementation depends heavily on local Implementation depends heavily on local control; therefore, districts can structure the control; therefore, districts can structure the use of the system.use of the system.

Some districts have chosen to use SPMS more Some districts have chosen to use SPMS more for “benchmark” testing, while others are using for “benchmark” testing, while others are using the system for formative assessment data as the system for formative assessment data as well.well.

5

The item bank currently contains math, The item bank currently contains math, reading, and language arts items aligned to reading, and language arts items aligned to the “old” Mississippi Curriculum Frameworks.the “old” Mississippi Curriculum Frameworks.

Coverage includes items for grades 1-12 in Coverage includes items for grades 1-12 in these areas.these areas.

Coverage for science and social studies Coverage for science and social studies includes items for grades 3-12.includes items for grades 3-12.

SPMS also supports writing at grades 4, 7, SPMS also supports writing at grades 4, 7, and 10 (English II) with automated scoring.and 10 (English II) with automated scoring.

Tests can be built by teacher, school, or Tests can be built by teacher, school, or district. district.

Application

6

Individual and summary reports enable Individual and summary reports enable teachers and administrators to track, monitor, teachers and administrators to track, monitor, and disaggregate data by various demographic and disaggregate data by various demographic categories at the classroom, school, and district categories at the classroom, school, and district level.level.

Student portfolios enable teachers and students Student portfolios enable teachers and students to track progress in real time.to track progress in real time.

SPMS provides system-generated parent letters.SPMS provides system-generated parent letters.

A new resource unit will be unveiled at the A new resource unit will be unveiled at the district level at the end of January and will allow district level at the end of January and will allow users to use lesson plans, models, and activities users to use lesson plans, models, and activities already in the system and to add personal already in the system and to add personal resources for all SPMS users.resources for all SPMS users.

Application

7

Reporting features:Reporting features:

State test results are automatically State test results are automatically loaded into the system.loaded into the system.

Summary performance reports provide Summary performance reports provide results for curriculum competency and results for curriculum competency and objective.objective.

Item analysis and roster reports show Item analysis and roster reports show performance for each test and for each performance for each test and for each student.student.

Application

8

Status ComparisonStatus Comparison

2004-20052004-2005 Districts: 63Districts: 63 Schools: 244Schools: 244 Students: Students:

133,000+133,000+

2005-20062005-2006 Districts: 116Districts: 116 Schools: 647Schools: 647 Students: 398,413Students: 398,413

9

2005-2006 State Users2005-2006 State Users

To date in 2005-2006, 247 schools across To date in 2005-2006, 247 schools across 78 districts are recording test results in 78 districts are recording test results in the Mississippi Student Progress the Mississippi Student Progress Monitoring System. Monitoring System.

Over 70,400 test sessions have been Over 70,400 test sessions have been

administered to students in elementary administered to students in elementary through high schools in the subject areas through high schools in the subject areas of reading, math, writing, science, and of reading, math, writing, science, and social studies. social studies.

10

2005-2006 Users2005-2006 Users

SPMS testing began in August 2005, with SPMS testing began in August 2005, with November and December being the heaviest November and December being the heaviest testing months in the school year thus far. testing months in the school year thus far.

Testing continues as a brisk pace in 2006, Testing continues as a brisk pace in 2006,

with over 5,000 tests already recorded in with over 5,000 tests already recorded in January.January.

11

2005-2006 Test Sessions by Month2005-2006 Test Sessions by MonthSPMS Testing Activity Report

0

5000

10000

15000

20000

25000

30000

35000

Aug

Sept

Oct

Nov

Dec

Jan

Feb

Marc

h

April

May

June

July

Month

Vo

lum

e

2005/06

2004/05

12

2005-2006 Test Sessions by Month2005-2006 Test Sessions by MonthSPMS Testing Activity Report

0

5000

10000

15000

20000

25000

30000

35000

Aug

Sept

Oct

Nov

Dec

Jan

Feb

Marc

h

April

May

June

July

Month

Vo

lum

e

2005/06

2004/05

13

Relationship Between SPMS Usage Relationship Between SPMS Usage and State Test Scoresand State Test Scores

Little is known about the nature of the tests Little is known about the nature of the tests administered, the importance of the SPMS administered, the importance of the SPMS tests, and how the data from SPMS were used tests, and how the data from SPMS were used to drive instruction.to drive instruction.

Investigations into the relationship between Investigations into the relationship between usage of SPMS and performance on state usage of SPMS and performance on state assessments show that the usage across the assessments show that the usage across the state in the 2004-05 year may not have been state in the 2004-05 year may not have been organized, integrated, or strategic enough in organized, integrated, or strategic enough in order to show major trends across the board. order to show major trends across the board.

Case studies provide the most insight into the Case studies provide the most insight into the effectiveness of the SPMS.effectiveness of the SPMS.

14

Case Study: Forrest CountyCase Study: Forrest CountyGrade Math Reading Writing Social Science Science

1 231 61 0 0 0 2 361 232 0 0 17 3 999 715 0 10 15 4 441 257 0 0 19 5 234 269 0 0 18 6 464 369 0 62 47 7 199 470 0 248 80 8 114 300 25 974 58 9 6 4 0 0 6

10 9 17 0 15 19 11 5 5 0 72 6 12 0 2 0 2 4

unknown 124 26 0 0 0 Total 3187 2727 25 1383 289

15

Forrest County Grade 3 MCT MathForrest County Grade 3 MCT Math

Gains from 2003-04 to 2004-05 for Grade 3 MCT Math mean scale score

State gain: 2.4 scale score

points 499.8 to 502.2

Forrest County gain: 10.9 scale score

points 494.6 to 505.5

Gains for MCT Math from 2003-04 Grade 2 to 2004-05 Grade 3 % Proficient and Above

State gain: 0.2% 91.7% to 91.9%

Forrest County gain: 6.0% 86.7% to 92.7%

16

17

Forrest County Grade 3 MCT MathForrest County Grade 3 MCT Math Black-White Gap from 2003-04 to 2004-05 for Grade 3 MCT Math, % Proficient and Above

Forrest Co. State

2003-04

2004-05

2003-04

2004-05

Black 71 82 87 88

White 96 96 96 96

Gap 25 14 9 8

18

Forrest County Grade 3 MCT MathForrest County Grade 3 MCT Math Economically Disadvantaged Gap from 2003-04 to 2004-05

for Grade 3 MCT Math, % Proficient and Above

Forrest Co. State

2003-04

2004-05

2003-04

2004-05

ED 81 88 89 89

Non-ED 95 96 96 96

Gap 14 8 7 7

19

Case Study: Kemper CountyCase Study: Kemper CountyGrade Math Reading Writing Social Science Science

1 0 0 0 0 0 2 10 7 0 0 0 3 0 0 0 0 0 4 97 12 0 0 0 5 0 3 0 0 0 6 0 113 20 0 0 7 264 771 0 0 5 8 423 1851 0 0 0 9 0 78 0 0 0

10 0 2 0 0 0 11 0 0 0 0 0 12 0 0 0 0 0

unknown 0 0 0 0 0 Total 794 2837 20 0 5

20

Kemper County Grade 8 MCT Kemper County Grade 8 MCT ReadingReading

Gains from 2003-04 to 2004-05 for Grade 8 MCT Reading mean scale score

State decrease: 7.0 scale score

points 563.6 to 556.6

Kemper County gain:

9.4 scale score points

544.5 to 553.9

Gains from 2003-04 to 2004-05 for Grade 8 MCT Reading % Proficient and Above

State decrease: -4.8% 61.6% to 56.8%

Kemper County gain:

14.2% 41.1% to 55.3%

21

22

Economically Disadvantaged Gap from 2003-04 to 2004-05 for Grade 8 MCT Reading, Percent Proficient and Above

Kemper Co. State

2003-04

2004-05

2003-04

2004-05

ED 37 54 47 44

Non-ED 52 60 77 72

Gap 15 6 30 28

Kemper County Grade 8 MCT Kemper County Grade 8 MCT ReadingReading

23

Kemper County and SPMS 2004-05Kemper County and SPMS 2004-05

Vicki Parker is a retired Mississippi Vicki Parker is a retired Mississippi educator and certified Vantage trainer.educator and certified Vantage trainer.

Vicki worked with Kemper County during Vicki worked with Kemper County during the 2004-2005 school year.the 2004-2005 school year.

Vicki will also be a trainer for the SPMS Vicki will also be a trainer for the SPMS Curriculum, Assessment, and Instruction Curriculum, Assessment, and Instruction (CAI) modules.(CAI) modules.

24

MS In-State Vantage ContactMS In-State Vantage Contact

David Timmons, who taught in the Bay-David Timmons, who taught in the Bay-Waveland School District, is a certified Waveland School District, is a certified Vantage Trainer.Vantage Trainer.

He is now serving as the MS Contact He is now serving as the MS Contact Person for SPMS. Person for SPMS.

His e-mail is dtimmons@vantage.com.His e-mail is dtimmons@vantage.com.

25

New for 2006New for 2006 3-in-1 bubble sheets with significant cost 3-in-1 bubble sheets with significant cost

reduction reduction Version 2.0 for district usersVersion 2.0 for district users MCT practice testsMCT practice tests Science practice testsScience practice tests MS-teacher reviewed items for social MS-teacher reviewed items for social

studies and sciencestudies and science SPMS Curriculum, Assessment, and SPMS Curriculum, Assessment, and

Instruction (CAI) modulesInstruction (CAI) modules

26

Bubble Sheets for Off-line Testing

27

3-in-1 Bubble Sheet3-in-1 Bubble Sheet

28

Version 2.0Version 2.0

Version 2.0 will be ready for the 2006-Version 2.0 will be ready for the 2006-2007 school year.2007 school year.

Vantage is making 2.0 available for users Vantage is making 2.0 available for users at the district level only at the end of at the district level only at the end of January for “trial use”; 2.0 will not contain January for “trial use”; 2.0 will not contain “live” files.“live” files.

Version 2.0 will have a new resource site Version 2.0 will have a new resource site where districts users will be able to enter where districts users will be able to enter lesson plans or activities to the resource lesson plans or activities to the resource list already identified within the system.list already identified within the system.

29

Version 2.0Version 2.0

Version 2.0 will have new functions to Version 2.0 will have new functions to allow users to nominate items to be allow users to nominate items to be included in the SPMS database for all included in the SPMS database for all users.users.

Version 2.0 will have functionality to allow Version 2.0 will have functionality to allow external test data (such as DIBELS) to be external test data (such as DIBELS) to be entered into the system.entered into the system.

30

MCT and Science Practice TestsMCT and Science Practice Tests Release set for late January for first practice Release set for late January for first practice

test for grades 3 and 7, all three content areastest for grades 3 and 7, all three content areas Second release set for February for first Second release set for February for first

practice test for all grades, all three content practice test for all grades, all three content areasareas

Spring release for practice tests two and three Spring release for practice tests two and three for all grades and content areasfor all grades and content areas

Release in spring for practice science tests for Release in spring for practice science tests for grade 5, grade 8, and Biology Igrade 5, grade 8, and Biology I (These are not the Riverside hard copy (These are not the Riverside hard copy

tests.)tests.)

31

MS Teacher Reviewed ItemsMS Teacher Reviewed Items

New science and social studies itemsNew science and social studies items All items reviewed by Mississippi teacher All items reviewed by Mississippi teacher

committeescommittees Items to be identified to distinguish them Items to be identified to distinguish them

from original SPMS itemsfrom original SPMS items Mississippi courses new to SPMS Mississippi courses new to SPMS

Science: Biology II, Physical Science, Science: Biology II, Physical Science, Anatomy and PhysiologyAnatomy and Physiology

Social Studies: MS Studies, US Social Studies: MS Studies, US Government, and EconomicsGovernment, and Economics

32

New Math and Language Arts ItemsNew Math and Language Arts Items

Addition of math and language arts items Addition of math and language arts items to SPMS was delayed so that new items to SPMS was delayed so that new items will match the revised frameworks.will match the revised frameworks.

Teachers will meet in late spring and/or Teachers will meet in late spring and/or early summer to review items.early summer to review items.

Items will be added to SPMS by January Items will be added to SPMS by January 2007.2007.

New MCT2 will not be administered until New MCT2 will not be administered until spring 2008.spring 2008.

33

SPMS CAI ModulesSPMS CAI Modules

34

SPMS CAI modulesSPMS CAI modules Districts selected for Districts selected for

Series I or II were notified Series I or II were notified January 6.January 6.

Ten teams were selected Ten teams were selected for Series I and for Series for Series I and for Series II based upon a rubric II based upon a rubric evaluation of the evaluation of the district’s application.district’s application.

Letters regarding Series Letters regarding Series III and IV will be mailed III and IV will be mailed to superintendents and to superintendents and SPMS contacts next SPMS contacts next week.week.

Districts that applied Districts that applied but were not selected but were not selected for Series I or II will be for Series I or II will be able to submit a able to submit a “Reapplication” to be “Reapplication” to be considered for the next considered for the next series.series. Blank boxesBlank boxes Non-usersNon-users

Series V and VI sites Series V and VI sites have not yet been have not yet been selected, but one will be selected, but one will be in south Mississippi, in south Mississippi, close to the coast.close to the coast.

35

CAI ModulesCAI Modules

Each series consists of three days of training, Each series consists of three days of training, each day separated by about two weeks.each day separated by about two weeks.

Districts are asked to send three-member Districts are asked to send three-member teams. We suggest a central office person teams. We suggest a central office person directly involved the district’s plan for student directly involved the district’s plan for student improvement, a school administrator, and a improvement, a school administrator, and a lead teacher/SPMS-user teacher. The same lead teacher/SPMS-user teacher. The same team should attend all sessions.team should attend all sessions.

These series are These series are notnot initial training sessions initial training sessions for users. All team members who attend for users. All team members who attend these sessions should be active SPMS users.these sessions should be active SPMS users.

36

CAI Curriculum ObjectivesCAI Curriculum Objectives

1) 1) Gain a clear understanding of the Gain a clear understanding of the alignment between SPMS and the alignment between SPMS and the Mississippi Curriculum Frameworks Mississippi Curriculum Frameworks

2) Develop a scope and sequence guide to 2) Develop a scope and sequence guide to assist with the integration of SPMS for assist with the integration of SPMS for instructional purposesinstructional purposes

3) Develop a plan to use SPMS to ensure 3) Develop a plan to use SPMS to ensure student growth in mastering Mississippi student growth in mastering Mississippi Curriculum Framework objectives as well Curriculum Framework objectives as well as local district curriculum goalsas local district curriculum goals

37

CAI Assessment ObjectivesCAI Assessment Objectives

1) Create appropriate SPMS assessments for 1) Create appropriate SPMS assessments for targeted objectives targeted objectives

2) Understand SPMS reports available 2) Understand SPMS reports available

3) Utilize SPMS assessment data to inform 3) Utilize SPMS assessment data to inform instructioninstruction

4) Establish district priority goals for 4) Establish district priority goals for increasing student achievementincreasing student achievement

5) Design a district/school strategic plan and 5) Design a district/school strategic plan and program of action with measurable program of action with measurable outcomes and an evaluative componentoutcomes and an evaluative component

38

CAI Instruction ObjectivesCAI Instruction Objectives1) Use SPMS assessment results to 1) Use SPMS assessment results to

review/critique instructional practices for review/critique instructional practices for targeted objectives targeted objectives

2) Explore SPMS reports to refocus instruction on 2) Explore SPMS reports to refocus instruction on targeted curriculum, including pre-requisite targeted curriculum, including pre-requisite skillsskills

3) Connect SPMS reports to the use of research-3) Connect SPMS reports to the use of research-based instructional strategiesbased instructional strategies

4) Examine SPMS reports to create learning 4) Examine SPMS reports to create learning opportunities and revise instructional strategies opportunities and revise instructional strategies based on individual or group needsbased on individual or group needs

5) Access resources through SPMS to assist in 5) Access resources through SPMS to assist in providing quality instructionproviding quality instruction

39

Under ConsiderationUnder Consideration

For the 2006-2007 school year, the MDE For the 2006-2007 school year, the MDE and Vantage are working to have data and Vantage are working to have data available when school begins.available when school begins.

The idea is to “promote” all SPMS students The idea is to “promote” all SPMS students at the end of the 2005-06 school year to at the end of the 2005-06 school year to the next grade and/or next school.the next grade and/or next school.

This process would mean districts might This process would mean districts might have to “clean up” the data, but the have to “clean up” the data, but the process would allow districts access as process would allow districts access as soon as the 2006-07 school year begins.soon as the 2006-07 school year begins.

40

SPMS Advisory CommitteeSPMS Advisory Committee

The SPMS Advisory Committee meets The SPMS Advisory Committee meets quarterly during the school year.quarterly during the school year.

The committee makes recommendations The committee makes recommendations and considers new functions.and considers new functions.

Currently, there are 21 members on this Currently, there are 21 members on this committee.committee.

Michael Hood, assistant superintendent Michael Hood, assistant superintendent from Montgomery County, is a “super” from Montgomery County, is a “super” user of SPMS and a member of the user of SPMS and a member of the Advisory Committee. Advisory Committee.