Developing a Monitoring and

Pre-Scoring Plan for the Virginia Grade Level

Alternative (VGLA)Adapted from the Virginia Department of Education

Division of Student Assessment and School ImprovementAugust 2010

Purpose of Presentation

• To assist schools in developing plans to monitor and pre-score Collections of Evidence (COE) for the Virginia Grade Level Alternative (VGLA)

Developing Monitoring and

Pre-Scoring Plans

• What are monitoring and pre-scoring?

• Why should administrators develop monitoring and pre-scoring plans at their school?

• What components should a monitoring plan include?

• What components should a pre-scoring plan include?

What are monitoring and pre-scoring?

Monitoring

A periodic and systematic review of a Collection of Evidence, while it is in the development process, with opportunities for feedback and intervention

What Monitoring is Not

• Simply “checking-in” with a teacher

• Flipping through a Collection of Evidence

• Reviewing a Collection of Evidence without providing feedback

Pre-Scoring

A detailed review of Collections of Evidence to address technical errors (i.e., grading, SEI tags), prior to scoring, with opportunities for feedback and corrections

What Pre-Scoring is Not

• Assigning a rubric score to evidence

• Altering student evidence or student responses

• Judging the instructional quality of the evidence

Why should administrators develop

monitoring and pre-scoring plans at

their school?

Monitoring

• Keeps staff apprised of the status of each Collection of Evidence

• Helps to ensure that teachers are providing instruction based on the Standards of Learning (SOLs)

• Ensures that instruction is on target with the Division’s pacing guide

Monitoring

• Helps ensure that technical issues (i.e., grading, SEI tags) are appropriately addressed

• Provides the opportunity to address issues with ample time for correction and/or intervention prior to scoring at the Division level

Pre-Scoring

• Provides a final opportunity before scoring to correct technical errors and omissions that could result in the loss of credit to students

• Results in the creation of better Collections of Evidence that accurately represent the student’s true knowledge and/or proficiency

What components should a monitoring

plan include?

Critical Planning Information

• Schools should develop a Master Participation List for each alternate/alternative assessment program that includes the...– Name of student– Grade level of student– School – Content area being assessed by the

alternate/alternative assessment– Teacher(s) responsible for instruction in

the content area assessed

Monitoring Plan Decisions

– Who will monitor?– How will monitors be trained?– When will monitoring occur?– Who will receive feedback?– How will feedback be given?– What options are available if

corrections are needed?

Remember!

• Monitors will provide training and/or technical guidance to teachers submitting Collections of Evidence.

• Monitors and/or teachers are not allowed to score their own Collections of Evidence or ones they have monitored during the course of the school year.

Who should monitor?• Staff with content knowledge and

experience with alternate/alternative assessments– Special education or testing specialists– Instructional specialists– LEP specialists– School Test Coordinators– Title I mathematics and reading specialists– Experienced teachers– Building Administrators

• Note: The Testing Office recommends Principals and/or VGLA Administrators be responsible for monitoring Collections of Evidence at his/her school.

When should monitoring occur?

• Weekly?• Monthly? • At the end of each grading period?• At the same time as benchmark

testing?• Other times?

Who receives feedback?

• Teachers

• Principals

•Alternate/Alternative Assessment Leaders

•Central Office Staff?

• Trainers/Technical Assistance?

How will feedback be provided?

• Post-It notes on Collections of Evidence

• Locally created feedback forms given to teachers, principals, and/or others

• Meetings with teachers, principals, and/or others about monitoring results

• Other?

What options are available if corrections are needed?

• Individualized consultation and support• Peer or small group training sessions• Professional development• Access to state and local SOL and/or

ASOL assessment resources• Other

A Sample Monitoring Plan

• School-based review teams are created in the school.

• Division Director of Testing, Special Education Director and Content Supervisors will train school-based review teams using implementation manuals, VDOE power points, division pacing guides, and other resources.

• Teachers submit Collections of Evidence to the school-based review teams at the end of each review cycle.

A Sample Monitoring Plan

• School-based review teams can report their findings on each Collection of Evidence using a locally created form. A copy of the form can be reviewed by both teachers and principals.

• Central Office may dispatch specialists to schools in need of additional support and training upon request.

A Sample Monitoring Plan

1Select and trainschool-based review teams

3If necessary, specialists may be dispatched to schools based on

the reports

2School-based review teams

check Collections of Evidence every review cycle

and report to school leadership/administration.

Sample School-Based Review Team

9-Week Monitoring Form – Grade 4 Reading

StudentName

100 - 90 % of SOL Evidenced

89 - 70 % of SOL

Evidenced

69- 50 % of SOL

Evidenced

Less than 50% of SOL Evidenced

Ted Smith

Kay Blue

Joe Davis

Ann Jones

A Sample Teacher Review Form

VGLA 9-Week Review SheetSchool-Based TeamsTeacher: ____________________Student: ___________________________ Grade Level:______Content Area:_________________________________________REVIEW for: ____ 1st Nine Weeks ____ 3rd Nine Weeks ____ 2nd Nine Weeks ____ 4th Nine Weeks Reviewed By:____________________________ Date:________ Collection Status: Address the following questions: (1) Is there evidence for all of the

standards for the nine weeks according to the Pacing Chart? (2) Does the work submitted align with the standards satisfactorily? (2) Does the evidence demonstrate student mastery? (3) Has the student work been graded accurately? (4) Other?

Recommendation(s): Be as specific as possible

Follow-up review needed for implementation of recommendations: ___Yes ____No

Follow –up Review Date: ________________

Pre-Scoring Decisions

– Who will pre-score?– How will pre-scorers be trained?– When will pre-scoring occur in relation

to submission date?– Who will receive feedback?– How will feedback be given?– What options are available if

corrections are needed?

Who will Pre-Score?

• Persons selected for pre-scoring cannot serve as scorers for their students’ Collections of Evidence.

• Pre-scorers should be knowledgeable about all scoring rules and the SOL and/or ASOL content.

• Pre-scorers should be detail oriented.

Remember!

• VGLA Administrators should provide training and/or technical guidance to teachers submitting Collections of Evidence.

• Pre-scorers are not allowed to score their own Collections of Evidence.

How will Pre-Scorers be trained?

Training should include a list of specific issues to look for in each Collection of Evidence– Grading?– Correctly completed SEI tags?– Detailed anecdotal records (Not

allowed)– Captioned photographs (Not

allowed)

Other helpful questions to ask

while Pre-Scoring

• Is the evidence organized according to the scoring worksheet?

• Are all required state and local forms included and completed?

When should Pre-Scoring Occur

Select a time to Pre-Score: • Give ample time for Collections of

Evidence to be complete or near completion.

• Give an opportunity to return Collections of Evidence to teachers for corrections well in advance of the due date– Collection of Evidence are scheduled for

pick up on April 25, 2013

Who should receive feedback?

• Teacher(s) submitting Collections of Evidence

• Building level principals

• Others?

How will feedback be given?

• Attach notes to Collections of Evidence

• Create forms to give to teachers and/or others

• Meet with teachers and/or others• Other ideas?

• PWCS Testing Office recommends frequent and direct communication with teachers!

What options are available if corrections are needed?

• Return Collections of Evidence to teachers for corrections.

• Assign staff to assist teacher in corrections such as grading evidence, organizing evidence, completing SEI tags.

• Some options may simply be limited or non-existent due to time constraints and volume. Plan accordingly so as to not disadvantage the student!

Sample Pre-scoring Plan

• Pre-scoring will be conducted by teachers and building level staff in two stages – Note: Central Office staff may be

consulted, if required.

• Scoring rules should be reviewed with all persons involved in pre-scoring.

• Checklists are provided in the VGLA Implementation Manual 2013-14.

Sample Pre-Scoring Plan

Stage 1 – Final Teacher Review

– Conducted early to mid April

– Submitting teacher reviews COEs using Teacher Checklists and in accordance with scoring rules outlined in the VGLA Implementation Manual 2013-14

Stage 2 – Final School Review

– Conducted late April – Building

Administrator and/or VGLA Administrator reviews Collections of Evidence using Administrator Checklist and in accordance with scoring rules outlined in the VGLA Implementation Manual 2013-14.

• NOTE: Collections of Evidence are scheduled to be picked up from schools on April 25th.

Monitoring & Pre-Scoring Results

• Helps reduce and eliminate Collections of Evidence with:

– Missing Evidence

– Missing or incorrectly specified SEI tags

– Unacceptable evidence (i.e., in violation of scoring rules)

– Inaccurate or Ungraded Evidence

Monitoring & Pre-Scoring Results

• Eliminates Surprises– Collections of Evidence not done

for students educated outside of the Division

– Incomplete Collections of Evidence in the Division

– Collections of Evidence not driven by IEP, 504 or LEP Plans

Questions

Please visit:

• Testing Office page on the PWCS Intranet Portal

• VDOE Alternate/Alternative Assessment Page at:

http://www.doe.virginia.gov/testing/alternative_assessments/vgla_va_grade_level_alt/index.shtml

• Additional questions may be directed to the PWCS Testing Office by calling 703.791.8308.