Caveon Webinar - International Testing 11-15-2011

Slide 1

Steve AddicottVice President, Caveon Test Security

Tom GibbsSenior Analyst, SHL Group

Aimee Hobby Rhodes, JDDirector of Exam Security, CFA Institute

International Testing: Crossing Global Borders While Protecting Your Own

AgendaIntrosSecurity Threats to International Test ProgramsSteve Addicott, CaveonEmployment Testing: The Stakes are High, and Theyre Global, TooTom Gibbs, SHL GroupAssessing Country RisksAimee Hobby Rhodes, CFA InstituteQ&A

Steve AddicottVice President, Caveon

About Caveon Test SecurityStarting 9th yearOffering the Three PsProvenPracticalProtectionCertification/Licensure & Education ClientelePre-eminenceTesting globallyWeve seen a few things

International Test ThreatsProxy Test TakingBraindumps & Rogue Review Courses Nefarious Test CentersSee aboveUnusual practices

Nefarious Test CentersNOT vendor-owned centersExample of Unusual Test TakingTest Center Hours 6 am to 6pm130 tests taken at odd hours, ie 2amCandidates unable to schedule tests when Test Center is closedWHAT IS GOING ON?And, Why are these candidates scoring much higher than others?

Braindumps and Rogue Review CoursesTest Prep Courses advertising & selling live test itemsPrevalence of stolen-answer key usage may exceed 20%How are items stolen?Infiltrate a top vendors premium security, orBribe a remote test center?Evidence points to unauthorized decryption of exam after downloading to test site servers

Proxies and GunmenProxy Test Takers:Make Big MoneyAnalysis uncovered a large proxy test taking service operating in at least six testing sites: USA, Bangladesh, Turkey, UAE, and an unknown country.500 test instances @ $1000 per instance in 6 monthsOwn and operate test centersAre branching out

A Gunman for Hire9 Caveon Test Security

September 10, 20xx

Dear sir:

We can take XTest exam service for you now in china, please fill out the attached file and send to me. We'll finish the exam within one week.

The price now for XTest exam service is USD 1000, you can send half payment before exam, half payment after exam.

Best regards,Mr. Zhao

Gunman10 Caveon Test Security

September 12, 20xx

Hello:

We only accept WESTERN UNION payment, you can send half before exam, half after exam. If you do not have a passport, you can send me a copy of your driver's ID, you can scan it and send to me.

Please fill out the attached file and send to me when you want to start the XTest exam.

Cheers.Mr. Zhao


October 16, 20xx

My answer for you question: Testing ID,when we register XTest for you, the test system will automatically give you the ID.

2. After exam, XTest will send you the XTest certification directly from USA.

3. When we finish XTest for you, you can check exam result by XTest official site,which can verify your exam result.

For the payment, you can send half before exam, half after we finish XTest for you, XTest will send you certification within 4-8 weeks. When you want to start the exam, please let me know.

Cheers.Mr.Zhao


October 27, 20xx

Hello:

Our X test centre is being checked until now, I'm afraid we should wait for a little more time to start the XTest. Do you need YTest or ZTest exam service? I can prepare for you next week. Please let me know your decision, thanks.

Best regards.Mr. Zhao


November 10, 2007

The payment is OK, thanks.Now I send you XTest info: [Link to Site] Test ID* or XTest Certification ID: XTest11327691Password: 123456

***Please keep your personal information updated always. You'd better login the XTest site each week. You can modify the password now.

If you or your friend need YTest, ZTest exam in the future, you can contact me, there are XTest certification status in the XTest site, please pay attention to it.

Contact me if you have any question.

Best regards,Mr.Zhao

Tom GibbsSenior Analyst, SHL

Overview of presentationThe user modelIt is 24/7/365 and its onlineWhy the stakes are highThe war for talent and the current economic conditionsWhy the stakes are globalThe war for talent is everywhereAre we managing the cheats and the pirates?The value of web patrols and a clear security frameworkSummary and further informationWe are publishing more about our security processes

The User Model

24/7/365The user model is increasingly not proctored and not delivered via group testingDelivered online with a variety of security featuresLinear On-the Fly Testing (LOFT) and Computer Adaptive Testing (CAT)Systems for managing item exposure and answer key exposureSupported by a security framework including data forensics, web patrols and incident response procedures (see next slide)Initial version has been operating since 2002Verify has been operating since 2006Now available in 26 languages

Test security monitoringSuspected security breachReporting (through clients/business/other)Web PatrolsItem Bank compromised?YesNo

ContinuemonitoringSecurity Breach ProcessCollect EvidenceReport to LegalCommunications with suspected offender Data ForensicsRun analysis on data specific to test, date range and client of suspected breach Replace compromised items

Further Information...

Burke, E. (2009). Preserving the integrity of online testing. Industrial and Organizational Psychology, 2, 35-38.

This is a special edition of I/O Psychology on this topicand can be accessed via www3.interscience.wiley.com/journal/118902544/home



The stakes in the employment market have increased

Quotes from a well known graduate recruitment portalBeing the honest idiot in a sea full off successful liars will get you nowhereI put sweat, blood and tears into improving my CV and am absolutely sickened that lazy, lying cheating graduates are getting jobs they have done nothing to deserveHave to say that you shouldn't lie about what you can get caught out on but anything else is fair game. Real life is about communication, persuasion and negotiationIt's important that you are able to describe in detail your fake job just in case you are asked about it

The war for talent is local and globalGeography is becoming less of aconstraint

An applicant could be applying for a job located anywhere from anywhere

So, how do we know if we are managing the cheats and the pirates?

Web patrols are telling us that there is a threat but we can detect it and manage it

Web patrols are also telling us that the applicants know there is security and that there are consequences

Internet discussion from April 12th 2010ttsstt With all the psychometric testing for a number of jobs i have applied for i am wondering how many people cheat and get help from family/friends while doing the test. From a few people i've spoken too I get the sense that it is MORE than common Thanks guysunderthepump: I didn't for two reasons, well two reasons on top of being honest :p 1) They make you do a supervised one at ACs anyway 2) Time limit. You don't exactly have time to sit there and come to a consensus and different people work at different paces. Personally having someone else sitting there would annoy me more than it would helpbrendan58: what underthepump said pretty much. they make you sit a supervised test in the AC and it would be quiet a shame to get rejected based on fluctuating test results just need to have the confidence to complete them yourselfnebasil: There is no point in cheating, how do you know the person doing the test for you will do better than you? I'd rather trust myself than someone elseHuynhzie: lol cheat now, ur going to get screwed in there. there was a PWC guy who went to his AC, he absolutely got owned at the verification test there and he was pointed and told off by the HR. HR was like "how can u perform absolutely so low", SHL is to test ur honesty as well, not just brilliance

Cheating and piracy is a fact

It is global!

It requires constant vigilance

It requires innovation and commitment

Coming soon Bartram, D., and Burke, E. (In press). Industrial/Organizational Testing Case Studies. In Wollack, J. A. and Fremer, J. F. (Eds.). Handbook of Test Security. New York, NY: Routledge.

This also includes a review of faking good on self-report (typical performance) measures such as personality questionnairesLievens, F., and Burke, E. (2011). Dealing with the threats inherent in unproctored Internet testing of cognitive ability: Results from a large-scale operational test program. Journal of Occupational and Organizational Psychology (due for publication this year)

First scientific journal article describing features designed to promote the security of online high stakes testing

Thank You

Aimee RhodesDirector of Examination SecurityCFA Institute

Due Diligence for International Testing CFA InstituteCFA Institute awards the CFA Charter to candidates who successfully complete 3 levels of testing and meet other education and work-related criteria.

In 2010, approximately 200,000 candidates registered for CFA Exams administered in 89 countries across the globe. Exams are administered twice a year.

All three levels of the CFA Exam are paper and pencil.

CFA Institute also gives the CIPM designation and exam, which is CBT.

Country Risk AssessmentTwo Different Assessments:Focus on Legal, Regulatory & TaxFocus on ability to protect rights and intellectual property (including test items) in a country. CFA Institute does this prior to opening a test center in a new country.

Legal, Regulatory & TaxWhat governmental agencies, if any, must we interact with in order to administer our tests in the way our program is designed?

What Taxes do we owe based on our program?

- keep in mind other activities in the country- inquiries on a no name basis

Assessment to Open a New Test CenterPractice established in 2009

Multi-part assessment

Goal is to examine risks & exposures unique to each country

Demographics and Political LandscapeHi, were new here!Understand the country to which you are goingSocial and Economic history informs how you may interact with regulators, staff, and candidatesPropensity for unrest and how it generally unfolds.

Protection of Intellectual PropertyWhy is our logo on that guys test?To which IP conventions is the country a signatory?What protections are in place on a national basis?How do you register your trademark? Is it a first to file system? Automated or manual?Are service marks recognized?Is there institutional capacity to address IP issues? Judicial resources and corruption should be considered.

Corruption & CrimeAre you gonna lose your shirt and how will you get it back when you do?Transparency International Corruption Index also consider if the areas of corruption are related to your business.Anti-corruption legislation and enforcement is it robust?Violent street crime in area of your test does it endanger candidates and staff?Do the police have the ability and desire to be helpful?Is this a location in which you would want to involve the police?

Ability to InvestigateWhat can you do when you lose your stuff?Is it safe for your home office staff to travel to the location for investigation?Is a visa required? How long to get it?If home office staff cannot go what is your plan? Outside security vendor? Do you have one identified?

First-Hand Experience of other OrganizationsHey, did you guys lose your shirt?Reach out to other organizations that test in the location you are evaluating.Requires building relationships with other organizations.This can be invaluable. You compare what your research reveals to real experiences.

CustomsHey, our stuff doesnt seem to be getting to our testing staff!Understand how to work with the customs officials.U.S. Department of Commerce can advise and may intervene if necessary cultivate this relationship.International Trade Office (part of Dept. of Commerce).Talk to your delivery agent what is their experience in that country?Know your timelines.

Current Test CentersDo we really need a test center here?Proximity of nearest centers in other countries?Ease of travel for candidates to adjacent countries understand that not everyone can go everywhere.Does expense of travel keep potential examinees out of your program?

Testing Partners and LocationsSo you decided to take the plunge with whom are you swimming and in which city?Is your testing partner in the country? If not, who is?Which city (or locations) are best for our test?

Questions?

[email protected]

[email protected]

[email protected]!

Caveon Webinar - International Testing 11-15-2011

Documents

Transcript of Caveon Webinar - International Testing 11-15-2011