Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Accountability and Effectiveness: Maintaining Security at Test Centers

Cynthia Miller Murphy, RN, MSN, CAE

Executive Director

Oncology Nursing Certification Corp.

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

How do we, as test sponsors, verify that our

examinations are administered in a secure

manner?

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Continuous Quality ImprovementOngoing Process

Before examination

During examination

After examination

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Before Examination

Standardized training of all proctors and supervisors

Standardized policies and procedures for test administration

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Before Examination

Assure that all candidates meet eligibility criteria

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Before Examination:

Assure that candidates are well-informed about what security

measures to expect

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Inform Candidates About Security

Through:

Candidate bulletin/handbook

Website

Direct communications:Authorization to test letters

Admission letters

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Inform Candidates About:

Identification requirements

Sign in/out of test center

Personal belongings

Use of study aids

Use of scratch paper

Breaks

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Inform Candidates About:

Direct and indirect observation/monitoring

Signing confidentiality/security statements

Grounds for dismissal/cancellation of results

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

During the Examination

Scheduled and Unscheduled Observation

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Paper/pencil TestSite Observers

Certified professionals voluntarily observe the test administration at local test sites

Selection of sites – convenience, random, specific

Select approximately 20% of local test sites

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Paper/pencil TestSite Observer Evaluation

Complete a specific questionnaire

Provides immediate feedbackProctor behavior

Incidents

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Computer-based TestingObservers

Test candidate observers who are Board or Test Development members

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Computer-based Test Observers

Register for and take the examination

Observe all security measures

Report back to test sponsor

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

After Examination

Data from candidatessurveys

complaints

Reports from testing companyirregularity

incidents

summaries

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Candidate Surveys

Ongoing – every administrationTest Center Survey - immediate“After Test” Quality Improvement Survey – administered 2-3 weeks after test

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Test Center SurveyPaper/pencil Testing

Security and quality of the testing experience from receipt of admission letter to completion of the test

Additional page for specific test question comments

Results for individual test sites and summary

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Test Center Survey “CBT Exit Survey”

Five minutes included in “seat time”

Security measures

Overall satisfaction

Results by individual sites

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Test Center Survey

Data on the compliance of supervisors and proctors at local test centers

Data on the adequacy of local test site facilities

Specific feedback to testing company

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

After Examination

Irregularity Reports

Incident Reports

Candidate Complaints

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

After Test Quality Improvement Survey

Email link to random selection of 20% of test candidatesComprehensive customer service review from first contact with test sponsor through test administration, including enforcement of security measuresWeb-based survey

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

After Test Quality Improvement Survey

Responses are anonymous

Obtain quantitative and qualitative data

Summarize by test sites and overall

Presented at CLEAR’s 23rd Annual ConferenceToronto, Ontario September, 2003

Accountability and Effectiveness

Review all data

Provide specific feedback to testing company

Require specific improvement plans