Test Security June 26, 2014 >

Test Security

June 26, 2014

www.education.state.pa.us >


2009 Data Forensics Report

July 2009

PA Department of Education Director of Bureau of Assessment and Accountability resigned

Had worked with contractor to produce a Data Forensics report

Report arrived just as Director was leavingNewly appointed Director received the report but found it difficult to useSimply didn’t have the time or staff to access every grade (3-8 and 11) in every school (3,182) in two content areas (reading and math)

2


2009 Data Forensics Report

July 2011

Philadelphia Notebook inquires if Pennsylvania Department of Education (PDE) had ever conducted a Data Forensics ReportPDE turns over the report to the Notebook Notebook discovers a summary at the end of the Technical Report

Publishes names of schools that were “flagged”PDE forms a Data Forensics team

3


PDE Data Forensics Team

Investigations

Asked every Superintendent/CAO to investigate the cause for being flagged

Unusually high numbers of wrong to right erasures Unusually high percentage increase in students

Advanced/Proficient Lots of negative responses from

Superintendents/CAOs Changes in number of subgroup populations that put

the subgroups below the minimum NSome flagged schools were so egregious they were turned over to the Office of the Inspector General for investigation

4



Investigations

PDE didn’t want to rush to judgment Investigations still ongoing

PDE requested Data Forensics reports for 2010 and 2011 Reports have not been released because of ongoing

investigationsIn 2012, PDE requested only erasure analyses

Only concrete evidence of possible violations

5



Erasure Analyses

Schools were flagged if At least 10% of the student population in a grade had

five (5) or more wrong to right erasures in a particular content area State average was around one wrong to right

erasure per studentIf a school met the 10% criteria, PDE asked the school for a list of the Test Administrators matched to the students If a Test Administrator had 20% or more of his/her students with wrong to right erasures, the Superintendent/CAO was asked to investigate and report the results of the investigation to PDE

6



Preventive Measures

Change in training Old training model

PDE trained District Assessment Coordinators District Assessment Coordinators trained School

Assessment Coordinators School Assessment Coordinators trained Test

Administrators New training model

All of the above PLUS All Test Administrators have to complete an online

Test Administrators’ training

7



Other changesScrambled itemsCell phone and electronic devices policySuggested that children’s teacher not administer the assessment to his/her children

If not possible, have another person in the room (Proctor)

Excessive login reportSigned certificates

District Assessment Coordinator Test Security Certificate School Assessment Coordinator Test Security Certificate Principal Test Security Certificate Test Administrator Security Certificate Proctor Test Security Certificate General Test Security Certificate Student Code of Conduct

8



Additional Test Security Measures

Materials inventoryMaterials must be kept in a locked locationTest distribution and return procedurePDE MonitoringAny suspected violations must be reported to PDE immediatelyMaterials Accountability FormMissing materials report

9

Test Security June 26, 2014 >

Documents

Transcript of Test Security June 26, 2014 >