Caveon Webinar Series - Unlocking the Mystery of the Validity Triangle 11-2014
Caveon Webinar Series - Conducting Test Security Investigations in School Districts and LEAsppt...
26
www.caveon.com 1 Caveon Webinar Series Conducting Test Security Investigations in Districts and LEAs May 25, 2016
-
Upload
caveon-test-security -
Category
Education
-
view
218 -
download
0
Transcript of Caveon Webinar Series - Conducting Test Security Investigations in School Districts and LEAsppt...
www.caveon.com1
Caveon Webinar SeriesConducting Test Security
Investigations in Districts and LEAs
May 25, 2016
www.caveon.com2
Today’s Presenters
Steve Addicott Marc Weinstein
www.caveon.com3
Agenda for Today• What’s going on in K-12 testing• The impact of a security incident• Preparing for an incident, before one
occurs• Building your Security Incident
Response Plan • Conducting an internal investigation• Summary• Q&A
www.caveon.com4
What’s Happening Out There?• Recent, real-life situations some
districts are facing:o “Disappearance” of Austin High Students
Sparked Questionso 50 Students Face Grade Tampering
Allegationso Two Colorado Springs Teachers May Have
Shared Test Information with Studentso U. of Iowa Investigates Possible Cheating
in Online Courseso 3 Face Expulsion in Coast Guard
Academy Cheating Probe
www.caveon.com5
What is at stake?
Consider what you are protecting
Validity of assessments Accountability determinations Educational resource decisions Professional evaluations Exam content Funding Public trust and confidence Public Welfare
www.caveon.com6
SECURITY = TEST SCORE VALIDITY
Test Security = Exam Integrity
Exam Integrity = Test Score Validity
www.caveon.com7
Test Security is an Ongoing ProcessInvestigations Serve an Important Function In Every Step of the Process
Protect
Detect
Respond
Improve
www.caveon.com8
Before an Incident Occurs• Know your test security processes and
procedures• All programs have vulnerabilities. Know
yours.• Ensure all testing staff is thoroughly
trained• Identify all stakeholders in the testing
process• Create your Security Incident Response
Plan
www.caveon.com9
What is an SIRP?Security Incident Response Plan• Dictates what to do when a testing
irregularity occurs• Creates a consistent methodology for
treating infractions and breaches• Establishes protocols for penalties
imposed when incidents occuro Decision-makingo Escalation Patho Communications
www.caveon.com10
Security Incident Response Plan
Key Components of an SIRP• Agreements in place• Policies in place• Practices in place for analysis and
monitoring• Incident response matrix• Investigative strategies• Communications plan
www.caveon.com11
Key Functions
• Management• Budgeting• Analysis – Statistical analysis; monitoring
of the internet; etc.• Training– Staff training and awareness• Investigations– First line investigation
processes
www.caveon.com12
Seven Step Incident Investigation Process
www.caveon.com13
Incident Investigation Process
• Identify and document triggering incidento Tip lineo Proctor reportso Statistical analyseso Mediao Web monitoring
www.caveon.com14
Incident Investigation Process
• Determine scope of potential harmo What is the impacto Who is involvedo Effect on test scores?
www.caveon.com15
Incident Investigation Process• Complete initial incident report as
required by applicable policies and procedureso Leveraging SIRPo Disseminate it appropriatelyo Escalation and communication
www.caveon.com16
Incident Investigation Process
• Identify evidence availableo Varies by the nature of the incidento May include myriad physical and electronic
documents and correspondences
www.caveon.com17
Incident Investigation Process
• Initial report/allegation• Documentary evidence• Forensic data analysis• Comparison data• Physical evidence• Electronic evidence• Witness statements
www.caveon.com18
Incident Investigation Process
• Instruct administrators, educators, students, vendors and others to preserve relevant evidence
• Know document/data retention policies• Collect all relevant evidence• Preserve chain of custody for all
documents and electronic evidence• Review/analyze all evidence• Conduct interviews after reviewing all
relevant evidence
www.caveon.com19
Incident Investigation Process
• Gather and preserve evidenceo Leverage case management software
solutionso Role based accesso Collaboration
www.caveon.com20
Incident Investigation Process• Interviews and Witness statements
o Nature of incident will help identify witnesses to interview Person who reported triggering incident Test participants
School Assessment CoordinatorStudentsTest AdministratorProctor
Administrators Educators
www.caveon.com21
Incident Investigation Process
• Analyze evidenceo Decision-making criteria o Outputs and reporting
www.caveon.com22
Incident Investigation Process
• Report findings in accordance with applicable policies and procedureso Consider the audience: courts, appeal
boards, board of education, licensure committee, etc.
o Objective, never subjective
www.caveon.com23
Resources
• Handbook of Test Security• TILSA Guidebooks• Caveon Webinar Series• NCME Whitepapers• Operational Best Practices• Caveon District Investigations Training
Workshop• www.cespcert.org
www.caveon.com24
Key Points Summarized
• SEAs & LEAs are operating in a crucible
• Test security incidents may impact the validity of test scores
• An ounce of prevention is worth a pound of cure
• Importance of both formal, approved SIRP and investigation process
www.caveon.com25
Q&A
www.caveon.com26
Thank You! Follow Caveon on twitter @caveonCheck out our blog www.caveon.com/blogLinkedIn Group “Caveon Test Security”
Steve [email protected]
Marc [email protected]
