Online Testing: Online Testing: Transitioning from Paper Transitioning from Paper

to Computer‐Based to Computer‐Based TestingTesting

Florida Association of MIS

June 14, 2011

Bo Elzie, Project Manager,Computer-Based TestingDepartment of Education

BackgroundBackground

Florida has had had optional Computer-Based Assessments since 2006

Spring 2011 Assessments: (Required) FCAT Mathematics Retakes FCAT Reading Retakes (optional on

computer) FCAT Grade 10 Mathematics Algebra 1 End-of-Course Assessment Biology 1 End-of-Course Field Test (selected schools) Geometry End-of-Course Field Test

(selected schools)

HistoryHistory

In 2002, the Governor’s Blue Ribbon Panel on Accommodations recommended that the FCAT accommodations available for students with disabilities be expanded to include computer-based accommodations.

The Advisory Committee on Computer-Based Assessment was established in 2003 to advise the Department on issues related to computer-based assessment.

HistoryHistoryThe Department issued a request for

information in 2005 to learn more about the innovations currently being used and developed in support of computer-based testing. This resulted in numerous presentations to a review committee, including district representatives. The review committee recommended that, by 2010, all non-performance task grade-level tests be administered solely on computers.

Commissioner John Winn encouraged Superintendents to invest more heavily in hardware and other infrastructure to allow expanded use of computers in delivery of statewide assessments (memorandum dated December 11, 2006).

HistoryHistoryDuring the summer of 2009, the Advisory

Committee on Computer-Based Assessment met to provide feedback on the plan for phase in of CBT.

  In March 2010, 13 districts voluntarily

participated in the computer-based administration of FCAT Retake Reading and Mathematics.

In May 2010, selected schools in 51 districts participated in the computer-based administration of the Algebra 1 End-of-Course Field Test.

HistoryHistory A DOE Communications Team was formed in the summer of

2010 to address the CBT readiness process for 2010-11. This group of assessment, MIS, technology, policy, and charter school staff meet weekly to develop strategies and support for successful preparation and administration of computer-based tests.

  A CBT focus meeting was held with district staff on August 30,

2010 to elicit district input on the certification process and the web-based tool.

The Department created a Superintendent - nominated 15 member Computer-Based-Testing (CBT) Advisory Committee in September, 2010, which meets monthly via conference call, comprised of district assessment, MIS, and technology staff, to provide input on all readiness activities for computer-based testing.

  The Commissioner’s Leadership Policy Advisory Committee

(LPAC), comprised of a cohort of Superintendents, also provided oversight and guidance regarding Computer-Based testing in 2010.

How did we ensure that our How did we ensure that our schools were ready for this schools were ready for this transition?transition?

Department implemented a year-long Computer-Based Assessments Certification Process

Three Phases to this ProcessPhase 1: Initial Certification through

a web- based toolPhase 2: Infrastructure TrialPhase 3: Final online Certification

CBT Minimum Specifications (TestNav CBT Minimum Specifications (TestNav 6.9)6.9)

2010–2011 FCAT and EOC Computer-Based TestingSystem Minimum Specifications

General Specifications (Windows/PC and Apple/Macintosh)

Each computer must meet or exceed the following General Specifications in addition to the applicable Operating System Specifications below:

500 MB available disk space 1024x768 screen resolution Keyboard and mouse (or other pointing device) High-speed internet connection

Operating System Specifications

Windows/PC

Windows XP (Service Pack 3)

Pentium III 733 MHz processor

512 MB RAM

Windows Vista (Service Pack 1)

Pentium III 1 GHz processor

1 GB RAM

Windows 7 Pentium III 1 GHz

processor 1 GB RAM

Apple/Macintosh

Mac OS X 10.4 G3, G4, G5 500

MHz processor 512 MB RAM

Mac OS X 10.5 Intel, G4, G5 867

MHz processor 512 MB RAM

Mac OS X 10.6 Intel processor 1 GB RAM

Updated 10/26/2010

Phase 1:Phase 1: All 1,760 schools completed the web-based Computer-

Based Assessments Certification Tool in September and were asked to estimate:

How many computers?How many testing sessions?How many students to be tested?

Approximately 200 schools in 44 districts had some degree of deficiency based on estimates and were required to complete an action plan (23 districts had no schools with deficiencies)

Superintendent certified that action plans would be supported and carried out

DOE teams offered 48 hour-long conference call sessions to districts in order to to address action plans and answer any questions – 16 districts participated

Algebra 1 testing window was extended 5 days, so most of the 200 schools’ capacity issues were resolved for that assessment

Phase 2:Phase 2:

Infrastructure Trial – December 1-3

All 1,760 schools were asked to participate in the trial

Using the same number of computers that will be needed in the spring

Conducting the trial at their peak time (usually AM before 11:30)

104,000 tests administered December Schools completed a survey indicating any

problems they noted during the trial No pervasive problems noted by DOE or the

Contractor

Infrastructure TrialInfrastructure TrialLessons LearnedLessons Learned Screen resolution must be set to 1024x768 on

student workstations. Students must have full permissions to the

TestNav/Temp and TestNav/Log folder. A primary save location must be set and students

have read/write access to the save location. Screen savers, power management, and pop-up

updates must have been disabled on student workstations.

Proctor Caching software must have been installed and must be checked to ensure it is functioning correctly.

Test content must be pre-cached prior to student testing.

The test session must be started in PearsonAccess.

Infrastructure TrialInfrastructure TrialPositive CommentsPositive Comments

We appreciated the opportunity to test this out well in advance of the exam.

The Infrastructure trial ran very smoothly.

The Trial went very well. Good planning tool. I think it made the schools feel better about moving to CBT EOCs.

We were very happy with the way the trial went.

It really did run very smoothly. It is exciting to be a part of computer based technology.

Overall went well at our site. Feel better about the real tests in April.

The students were excited about taking an online test and seemed very pleased with the format and the online tools.

I'm glad to have had the opportunity to participate. It relieved any anxiety which I may have had about proctoring a new format.

It makes us more confident that the EOC exam will be relatively problem free.

Phase 3:Phase 3:

Final Certification of Readiness

Schools completed Final Computer-Based Assessments Certification Tool that was submitted 3 weeks before each spring assessment

The Certification Tool opened January 26, 2011 Actual numbers were described:

How many computers?How many days of testing?How many students to be tested?Checklist issues?

DOE immediately worked with districts to address any school deficiencies in readiness

Computer Based Assessment Computer Based Assessment Coordination of Federal and State Coordination of Federal and State FundsFunds

Computer Based Computer Based Assessment Assessment

Program Educator Equipment/ Hardware

Student Equipment/ Hardware

Professional Development

Title I-A x x xTitle II-A x xTitle II-B* (Competitive) xTitle II-D* (Competitive) x x xIDEA x xSchool Improvement Grants (1003 a and g)

x x

RTTT Funding x x x

The following computer-based assessments were successfully administered this spring:

28,399 FCAT Mathematics Retakes (March-April)

5,290 Reading Retakes (March-April) 186,104 Grade 10 FCAT Mathematics (April) 28,614 Biology 1 Field Tests (May 2-June 3) 35,399 Geometry Field Tests (May 2-June 3) 213,784 Algebra 1 EOC Assessments (May 9-

May 27)

Approximately 497,590 computer-based assessments were successfully administered this spring. All participating middle and high schools demonstrated 100% readiness through the online certification tool. Additional computer-based assessments will be added each year, and the online certification process for 2011-12 is being designed.

2011-12 CBT Certification Tool is 2011-12 CBT Certification Tool is being Developedbeing Developed

Four Assessment Windows◦ Fall FCAT Retakes◦ Spring FCAT 2.0◦ Fall Algebra 1 EOC◦ Spring EOCs

New Legislative Requirement: PERT: Postsecondary Education Readiness Test for all 11th graders in November 2011

Four weeks to complete

Due to DOE via the tool at least two weeks prior to testing

The Winds of Change…..PARCCThe Winds of Change…..PARCC

Partnership for the Assessment of Readiness for College and Careers◦September 2010 – PARCC awarded RTTT

funds to develop a computer-based K-12 Assessment System aligned to the Common Core State Standards in English/Language Arts and Mathematics by 2013-14

◦Florida is Fiscal agent and one of 11 governing states that will lead a 26 state partnership

◦PARCC will provide a common metric among the 26 states – focused on a global marketplace

Looking Ahead…Looking Ahead…

Citrix Type Environments

Thin Client Web-Based Solutions

Utilizing students’ own Equipment in a WIFI Environment

Stay Tuned…Technology Changes at the Speed of Light!

Questions?