Gdonm

Testing International Volume 41, June 2019 Editor: Nicky Hayes

International Test Commission

PRESIDENT Kurt Geisinger Buros Center for Testing / University of Nebraska, USA PRESIDENT-ELECT Aletta Odendaal Stellenbosch University, South Africa SECRETARY-GENERAL Paula Elosua, Universidad del Pais Vasco, San Sebastian, Spain TREASURER April Zenisky, University of Massachusetts Amherst, USA COUNCIL MEMBERS Elected Members Anna Brown, University of Kent, United Kingdom Neal Schmitt, Michigan State University, USA Stephen G. Sireci, University of Massachusetts Amherst, USA Solange Wechsler, Pontificia Universidad Catolica de Campinas, Sao Paulo, Brazil Co-Opted Members Dragos Iliescu, SNSPA Bucharest, Romania Kadriye Ercikan, Educational Testing Service, USA Peter Macqueen, Compass Consulting, Australia Wayne Camra ACT Inc. Observers Samuel Greiff, Université du Luxembourg, Luxembourg Gonggy Yan, Beijing Normal University REPRESENTATIVES IUPsyS Representative Ann Watts, IUPsyS IAAP Representative Jacques Gregoire, Université Catholique de Louvain, Belgium EDITORS International Journal of Testing Stephen Stark, University of South Florida, USA Testing International Nicky Hayes, United Kingdom

Testing International is a publication of the

International Test Commission

Contents

ITC President’s Message 2 Editor’s Message 4 A tribute: Fons van de Vijver 5 Conference News 7 Testing News 9 Food for Thought By Norman Buckley 11 Publication and Communications Committee 13 International Journal of Testing update 14 Membership, Involvement & Marketing Committee 15 Book Review: 17 Around the World: Evaluation & Assessment in Mexico 18

Kurt Geisinger

President, International Test

Commission

Kurt Geisinger Buros Centre for Testing / University of Nebraska

IamwritingshortlyaftermembersoftheITCCouncilmetforitsannualmeetinginVilnius,Lithuania.Allreportsarethatthemeetingwasextremelysuccessful.LetmesharewhywemetinLithuania;itwillprovidesomeinformationtoyouaboutthecurrentrunningoftheassociation.Mygraduateassistant(fromBelize)attheUniversityofNebraska-LincolnandIcomputedsomeestimatedairlineexpensesfortheentireCouncilanddeterminedthatthelowestsumofairfaresthatwecouldachievewastomeetinnorthernEurope.ThenIsurveyedsomeindividualpsychologistswhomIknowinnorthernEuropetodetermineaninterestingplacethatwecouldvisitthatwouldalsokeepourpriceslow.Iwastoldthatoneofthemostcost-effectiveplaceswouldbeintheBalticStates,perhapsLithuania.TheLithuanianPsychologicalAssociationhaslongbeenamemberofITC,agoodreasonforustoconsiderLithuania.IcontactedtheLithuanian/Vilniustraveloffice,gotalistingofhotelsthatwouldmeetourneedsandselectedoneintheOldCity.MycolleaguestellmeitwasabeautifulcityandtheyespeciallyappreciatedatraditionalLithuaniandinneronenight.NowletmetellyouabouttheleadershipteamworkfortheITCorganization.InpreparationfortheVilniusmeeting,itwastrulyheartwarmingtoseetheworkofourSecretary-

GeneralPaulaElosua,ourPresident-electAlettaOdendaal,ourinveterateofficemanagerandgeneral-jane-of-alltradesAnandavanTonder,andournewtreasurer,AprilZenisky.IwouldliketothinkthatIhelpedtoo,butitwasjustfascinatingtoseehighlyfocusedpeopleworkingwithsomuchteamwork,sointer-dependently,andeffectively.Theydidandcontinuetodosoforthebestoftheorganization.Then,mycrisishit.ThedaybeforeIwastoleaveforEuropeIwasinaterribleautomobileaccidentandtransportedtothehospitalinanambulance.IfoughtwiththedoctorstoletmecontinuetotheVilniusmeetinguntiltheytoldmeIhad8brokenribsandabrokensternumandthatflyingtoEuropewouldlikelyleadtocollapsedlungs.Atthatpoint,IcontactedAlettaandletherknowasPresident-electshewouldneedtoberunningthemeeting.AnditshouldsurprisenoonethatshedidoutstandinglyandthattheITCCouncilmeetingwaseffective,whoknows,perhapsmoreeffectivewithoutme!Thankstoallinvolved!TheCouncillearnedthatthe2020meetinginLuxembourgwasmovingforwardwonderfully;theyhavesetuptheScientificCommittee,andmademanygreatdecisionsregardingthefuturesuccessoftheconference.WehaveprimarilySamuelGreiffandLindievanderWesthuizentothankforthisplanning.TheCouncilalsolookedatsomepossibilitiesforfuturemeetings,lookedintooptionsrelatedtothelocalincorporationoftheInternationalTestCommission,waystoencouragesponsorstosupportourconferences,reviewedtheITCbudget,andsoon.Ihavenotyetseentheminutesofthemeeting,butbyallaccountsitwaseffective.Ialsohaveanapologytomakebaseduponmypreviouspresidentialcolumn.ImentionedinthatcolumnthatProfessorsDavidBartramandBarbaraByrnereceivedITCfellowstatusatthe2018Montrealconferenceandwouldbegivingkeynotepresentationsatthe2020Luxembourg

ITC President’s Message

2

conference.AcorrectedsetoffactsincludesthatbothofthemreceivedfellowstatusinITCatthewonderful2012ITCconferenceinAmsterdam,thattheyindependentlyreceivedtheTomOaklandAwardatthe2018conference,andthattheywouldbeeachbeprovidingTomOaklandLecturesatthe2020Luxembourgconference.ThatmaybereasonenoughtoplanyourattendanceattheLuxembourgconference!Pleaseacceptmyapology,DaveandBarbara!WealllookforwardtoyourtalksinLuxembourg!Ihaveonelastidea.ITCasanorganizationhaslargelygrownandspreadbywordofmouthandbyprofessionalstellingothersaboutoursuperprofessional,andfriendlymeetings.Wealsohaveawell-developedmentorshipprogram.PleaseshareinformationaboutourupcomingmeetinginLuxembourgwithyourgraduatestudentsandjuniorcolleagues.IthinkthatITCisanespeciallyvaluableconferenceforsuchjuniorcolleaguesandcolleaguesfromcountriesthatdonotyethavewell-developedassessmentsystems.IhavebeentothesettingattheUniversityofLuxembourgpersonallyanditisanexcitingplacetovisit.TheITCconferenceisagreatplacetoshareone’sresearch,tonetwork,andtodevelopinternationalpartnershipsandassociations.Weallknowthattheworldisshrinkingandconnectionsaroundtheworldabounding.LetITCbeacatalysttotestingresearchandpractice,andbyimplication,worldpeace.Ifpoliticianscannotbringaboutworldpeace,maybewecan!Pleasehaveagreatsummer!KurtGeisingerAfterwordIamnowdoingmuchbetter.Afterbeinginwhatwasessentiallyahead-oncrashonApril23,peopleatthescenetoldmeIhadtogothehospital.Iwalkedtotheambulance,climbedin,andwalkedintothehospitalonmyown,atwhichtimetheymademerideinawheelchair.ItturnedoutthatIhadsixbrokenribsonmyrightside,twoontheleft,aswellasabroken

sternumandthreebreakstomypelvis.Theybelievethatallthebreakswereduetomywearingaseatbelt.IhadbeentellingthedoctorsintheemergencyroomthatIneededtoflytoEuropethenextdayfortheITCCouncilmeeting.AfteraCTscantheytoldmeIcouldnotfly;havingthebreakstomyribs,theybelievedthatinanairplanemylungswouldlikelycollapseandIwoulddie.Fairlyconvincing.Iultimatelyspent5daysinintensivecare,then6daysinrehabilitationinthehospital.InitiallytheyweremostlyconcernedthatIhadinternalbleedingnearmyheartduetothebrokensternum.Afterthreedaysthatquasi-crisispassedwithoutsurgery.WhenIcamehome,Istayedhomeforalmosttwoweeks.Ihavenowbeenbackatworkforovertwoweeksandammakingprogress,walkingabitmorewithacaneeachday.Iexpecttobebackto100%sometimeinJuly.Moralofthestory:wearyourseatbelteverytime.WecannotaffordtoloseITCmembersinvoluntarily!Iconsidermyselfverylucky.Iwouldprobablyhavediedwithoutit.

The International Journal of Testing

The International Journal of Testing (IJT) is dedicated to the advancement of theory, research, and practice in the

areas of testing and assessment in psychology, education, counseling, organizational behavior, human

resource management, and related disciplines.

IJT publishes original articles addressing theoretical issues, methodological approaches, and empirical

research, as well as integrative and interdisciplinary reviews of testing-related topics and reports of current testing practices. All papers are peer-reviewed and are of interest to an international audience. The journal is

published quarterly, with each issue containing 4-5 articles.

If you have a paper you would like to be considered for publication in IJT, or if you are interested in reviewing

papers for publication, please contact the editor, Stephen Stark, by email: (sestark@usf.edu).

3

WelcometothelatestissueofTestingInternational.Webeginonasadnote,withatributetoFonsvandeVijverandthesterlingcontributionshehasmadetopsychologicaltestingingeneral,andtotheITCinparticular.Fons'untimelydeathwhileexercisingcameasashocktoallwhoknewhim,andhewillbesadlymissed.

Onalesssombrenote,ourconferenceorganisersLindievonderWesthuizenandProfessorSamuelGreiffhaveprovideduswithapreviewofnextyear'sconferenceprogramme,andI'msureyou,likeme,willbelookingforwardtovisitingtheGrandDuchyofLuxembourginJuly2020.OtherTestingNewsincludesareportonanEnglishtranslationoftheCOTANreviewsystemandandupdateonhowtherevisionoftheGuidelinesforTechnology-BasedAssessmentisprogressing.Wefollowthatwithsomefoodforthought,asNormanBuckleydeconstructsasignificanttestingshibbolethandchallengesusalltore-thinksomecherishedbeliefs.Youmayhaveyourownopinionsaboutsomeissueintesting.Ifso,TestingInternationalwouldlovetohearfromyou!Sendyourcontribution(nomatterhowsmall)tomeatnewsletter@intestcom.org.NealSchmitttellsusaboutnewdevelopmentswithITCpublicationswhileStephenStarkgivesusthelatestnewsfromtheInternationalJournalofTestingandPeterMacqueenupdatesusonmembershipandinvolvementissues.Peterhasalsoincludedhisever-helpfulsummaryofforthcomingconferencesIfindthatlistreallyuseful-it'salwaysgoodtoknowwhat'sgoingonaroundtheworld,andPetergoestoalotoftroubletocompileit.Soifyou'reawareofaforthcomingconferenceorcongresswithanyrelevancetotestingcomingupinyourregion,dolethimknowaboutit!Youcancontacthimat:p.macqueen@compassconsulting.com.au

FollowingPeter'scontributionwehaveabookreview,andthenaninterestingoverviewofevaluationandassessmentinHigherEducationinMexico.OurthankstoAgustinTristan-Lopezforthiscontribution,whichIhopeitwillbethefirstofmany:countryoverviewslikethisaredefinitelyhelpfulforanyoneworkingataninternationallevel,andequallydefinitelyinterestingfortherestofus!Soifyoufeellikegivingusapictureoftestinginyourcountry,pleasedoit!Sothereitis:thissummer'sTestingInternational.Ihopeyouenjoythisissue-butremember,it'sonlyourmemberswhomakeitinteresting!Youcansendyourcontribution,oranythingelseyou'dliketoseeincluded,tomeatnewsletter@intestcom.org.HappyReading!Nicky Hayes Editor, Testing International

Greetings from the Editor

Call for Articles and Announcements:

Testing International (TI)

Deadline for the December 2019 issue: November 11, 2019

TI is the newsletter of the International Test Commission, and disseminates information about national / international assessment

projects and initiatives, new test developments, recently published books /

articles, upcoming conferences and workshops, and topical issues in the field of testing and assessment to the international

community.

Please contact Dr. Nicky Hayes with your ideas, proposals, announcements,

and brief papers

newsletter@intestcom.org

4

Fons van de Vijver, PhD 1952-2019

ItwaswithagreatdealofsadnessthatmembersoftheITCCouncilwererecentlyinformedoftheveryuntimelyandsuddendeathofProfessorFonsvandeVijver,oneofthelongstandingleadersoftheInternationalTestCommission(ITC),inadditiontoahostofotherprofessionalorganizations.GiventhemultiplicityofimportantcontributionsDr.vandeVijvermadeoverthepast25yearstotheITC,webelievethatallITCmemberswillverymuchenjoyknowinghisoverallacademiccredentialsandaccomplishmentsingeneral,andhiscuttingedgecontributionstotheITC,inparticular.WebeginwithasummaryofhisacademiccredentialsandfollowwithhismorespecificworkfortheITC. Alphonsius(Fons)JosephusRachelwasbornonOctober4,1952inKoewacht,TheNetherlandsandwas66atthetimeofhispassing.FonsstudiedpsychologyatTilburgUniversityintheNetherlands,wherehealsoearnedaPh.D.,andstayedthereasafacultymemberforthemajorportionofhiscareer.Heheldachairincross-culturalpsychologyaswellasanextraordinarychairatNorth-WestUniversity,SouthAfrica,andtheUniversityofQueensland,Australia.Overtheyears,FonsheldavarietyofimportantleadershippositionsatTilburgUniversity,authoredorco-authoredover400publications-mainlyinthedomainofcross-culturalpsychology,servedrecently(2016-2018)aspresidentoftheInternationalAssociationofCross-CulturalPsychology,andwasaformerpresidentofDivision2(AssessmentandEvaluation)oftheInternationalAssociationofAppliedPsychologyaswellasaformer

PresidentoftheEuropeanAssociationofPsychologicalAssessment.Themaintopicsinhisresearchinvolvedbiasandequivalence,psychologicalacculturationandmulticulturalism,cognitivesimilaritiesanddifferencesinthecognitivedomain,responsestyles,translationsandadaptations.Hisworkonanalyticapproachesandconceptualizationofacculturationandmulticulturalismhasledtothedevelopmentofanindigenouspersonalityscalerelevantto11SouthAfricanlanguages.AtTilburgUniversity,hesupervisedapproximately35PhDstudiesand6post-doctoralstudents.Amongthebookshewroteincludethefollowing:Methodsanddataanalysisforcross-culturalresearch(bothindividuallyandwithKwokLeung),Cross-culturalsurveymethods(withJanetHarknessandothers),andCross-culturalresearchmethodsinpsychology(withDavidMatsumoto).Indeed,hehasbeenoneofthemostfrequentlycitedcross-culturalpsychologistsinEurope,ifnotworldwide.Fonshaspresentedkeynotesandinvitedlecturesatavastnumberofappropriateconferencesandofferedworkshopsworldwide,oftenincludingofcourse,theITC.Hewasamemberofthefollowingprofessionalorganizations:InternationalAssociationforCross-CulturalPsychology,InternationalAssociationofAppliedPsychology,EuropeanAssociationofPsychologicalAssessment,InternationalAcademyforInterculturalResearch,andtheITC.HeistheformereditoroftheJournalofCross-CulturalPsychologyandtheEuropeanJournalofPersonalityAssessment,andhasoftenservedasareviewerandeditorialboardmemberondozensofjournals.Ofcourse,hewasconsideredespeciallyappropriatetoreadpapersdealingwiththeadaptationoftestsandmeasuresandmethodologicalissuesinsodoing.Finally,Fonswasthe2013recipientoftheAmericanPsychologicalAssociation’sAwardforDistinguishedContributionstotheInternationalAdvancementofPsychology(forhiscontributionstointernationalcooperationandtotheadvancementofknowledgeofpsychology)andthe2014recipientoftheInternationalAssociationofAppliedPsychologyFellowsAward(forcontributionstoappliedpsychology).

5

WeturnnowtoFons’criticallyimportantcontributionstotheITC,wherehisfirstpositiononCouncilwasaseditorofwhatwasoriginallycalledtheITCBulletin,whichatthattimeexistedasaseparatesectionintheEuropeanJournalofPsychologicalAssessment.Fonsheldthispositionfrom1994until1999atwhichtimetheITCpresident,ThomasOakland,signedacontractwithLawrenceErlbaumAssociatesfortheITCtopublishitsownjournalentitledtheInternationalJournalofTesting.Duringthistimeperiodalso,FonsworkedcloselywithRonHambletonindevelopingadetailedsetofguidelinesforuseintheadaptionofeducationalandpsychologicaltestsforuseinotherlanguagesand/orculturalcontexts.Unquestionably,Fons’cross-culturalandmethodologicalexpertisehasbeencriticallyneededandthusgreatlyappreciatedforallsubsequentupdatesoftheseadaptationguidelines.Indeed,theITCwillbeforevergratefulforthededicatedcontributionsofFonsvandeVijveroveraspanofseveralyearsinworkingwithRonHambletonandotherITCmembersintheirdevelopmentandupdatingoftheITCGuidelinesforTranslatingandAdaptingTests.Manyofusbelievethatwewillmisshiscontributionstremendously,butwillmisshisfriendshipmore.KurtF.GeisingerandBarbaraM.Byrne

The ITC International Handbook of Testing and Assessment

Editors: Frederick T. L. Leong, Dave Bartram, Fanny M. Cheung, Kurt F. Geisinger & Dragos Iliescu.

Oxford University Press ISBN 9780199356942.

The ITC International Handbook of Testing and Assessment addresses the many challenges facing the cross-cultural applications of psychological and educational testing in this era of globalization. It represents and showcases the concerted efforts of the ITC in tackling the wide range of problems and complexities in international psychological testing; and provides information and resources to help guide professionals and graduate students regarding testing and assessment from an international and global perspective.

6

The12thConferenceoftheInternationalTestCommission:Luxembourg2020ByLindievanderWesthuizen(SecretaryoftheLocalOrganisingCommittee)&ProfSamuelGreiff(ChairoftheLocalOrganisingCommittee)Moien!Bonjour!GutenTag!Goodday!Wearedelightedtoinviteyoutothe12thConferenceoftheInternationalTestCommissionwhichwillbeheldintheGrandDuchyofLuxembourgfromthe14thtothe17thofJuly,2020.The2020ITCConferencepromisestobeanexceptionalprofessionalandscientificexperienceinauniqueculturalenvironmentrenownedforitsvibrancyandcosmopolitanflair.ExploringthethemeDiversityandequityinaglobalized,digitalworld:Opportunitiesandchallengesforassessment,the12thITCconferencewillpresentthelatestdevelopmentsandinnovationsinassessment.Thethemehighlightsculturaldiversityresultingfromglobalisationandtherapidadvancementsinthefieldoftechnology-basedassessment.Itisfurtherdividedintofivesub-themes:(1)Testdevelopmentandvalidationininternationalandmulticulturalenvironments;(2)Innovationsandadvancesinpsychometrictheory,modelling,andtechnologies;(3)Currentissuesofpolicy,diversity,andequity;(4)Bestpracticesintestingandassessment;and(5)Testsecurityandprivacyinaglobalizeddigitalworld.Animpressiveline-upofkeynotespeakershasbeensecuredthatisguaranteedtoexciteandengageyourmind:

ProfSachaEpskamp,oneofthepioneersinnetworkmodelling,willpresentastate-of-theartlectureontheCurrentStateandFutureofNetworkPsychometrics.

ProfJonathanTemplin,arenownedleaderinthefieldofdiagnosticclassificationmodels,willpresentastate-oftheartlectureonBuildingaDiagnosticModel-BasedFormativeAssessmentSystemforPersonalizingEducationalProgress.

ProfAnitaHubley,recognizedinternationallyforherexpertiseintestdevelopment,validity,and

psychologicalandhealthassessment,willshareherexpertiseonbestpracticesintestingandassessmentwithhertalkonContributionsofResponseProcessestoTestValidationandDevelopment.

Giventheimpactofrecentdataprotectionandsecuritydevelopmentsontesting,ProfJohnFremerwilldiscussChallengesConfrontedandLessonsLearned:ProtectingTestContentandPersonalInformationfromTestSecurityThreatsinInternationalTestingPrograms.

DrSaraRuto’stalkonMeasuringLearningforAllChildren:TheCitizenLedAssessmentApproachwilladdresstheincreasingimportanceofandchallengesassociatedwithinclusivityineducationaltestinginthedevelopingworld.

Playingdevil’sadvocate,ProfJohnO’Gorman,willconsidertheimplicationsofneuroscienceforindustrialorganizationalpsychologyinhistalkTheFutureforOrganisationalNeuroscienceinSelectionandAssessment,orIsThereOne?

Representingthegrowingfieldoflanguagetesting,ProfLianzhenHewilluseChina’sStandardsofEnglishframeworkasacasestudyandwillsharelessonslearnedforlanguagelearning,teaching,andassessment.

Lastly,ProfAlettaOdendaal,IndustrialPsychologistandITCPresident-Elect,willpresentadevelopingcountryperspectiveonchallengesandcriticalissuesinresponsestylesandtestuse.

Thestimulatingscientificprogrammewillbecomplementedbyapackedsocialprogramme,whichincludestheopeningceremony,thegaladinner,sightseeingtours,andnewscientificspeednetworkingandspeedmentoringsessions.Wehavealsorevitalisedthepostersessionformattoincreasepresenterandaudienceengagement.Besuretokeepaneyeoutforthesecondconferenceannouncementandthecallforpapersthatwillbereleasedonthe15thofJuly2019.Formoreinformationandupdatesontheconference,youcanvisittheconferencewebsiteathttps://www.itc-conference.com.

WelookforwardtoseeingyouinLuxembourginJuly2020!

Conference News

7

8

EnglishversionoftheCOTAN

reviewsystem TheCOTANreviewsystemforevaluatingtestquality(2010)providescriteriaforratingpsychologicalandeducationaltests,scalesandquestionnairesusedinthefieldsofworkandorganisationalpsychology,education,clinicalpsychology,developmentalpsychologyandothercontexts.TestreviewsarepublishedbytheCOTAN(onlyinDutchviawww.cotandocumentatie.nl)inordertoimprovetestsandtestingintheNetherlands.Thesereviewscanbeusedbypsychologistsandotherprofessionalsasanaidinmakingaresponsiblechoiceforatest.Recently,theDutchAssociationofPsychologists(NIP)publishedtheEnglishtranslationoftheCOTANreviewsystemforevaluatingtestquality(2010).ThisreviewsystemisnotonlyusedbythereviewersoftheDutchCommitteeonTestsandTesting(COTAN)andDutchtestpublishers,butisalsofrequentlyconsultedatDutchuniversitiesinthecontextofpsychologyeducation.Toillustrate:severalPsychologycoursesintheNetherlandsrequirestudentstosystematicallyassessthequalityofatestanditsmanualbasedontheCOTANreviewsystemforevaluatingtests.However,intheNetherlandsmoreandmorepsychologycoursesaretaughtinEnglish,withDutch-languagestudentsincreasinglyenrollingintheEnglish-languagespecializationsoftheseprograms.Inaddition,foreignresearchersandpsychologistsincreasinglyaskedtheCOTANforanEnglishtranslationofitstestreviewsystem.UntilnowonlyaDutchversionoftheCOTANreviewsystemhasbeenavailable.TheCOTANandtheNIPthereforeconsidereditimportanttopublishanEnglishtranslationoftheCOTANtestreviewsystem.

Thepdfdocumentcanbefoundonthispage:https://www.psynip.nl/en/dutch-association-psychologists/about-nip/psychological-testing-cotan/KarinVermeulenStafmedewerkerCOTAN

Testing News

9

UpdateontheGuidelinesforTechnology-BasedAssessment

TheInternationalTestCommissionhaspartneredwiththeAssociationofTestPublishers(ATP)tojointlydevelopGuidelinesforTechnology-BasedAssessment.TheformerITCguidelinesoncomputer-basedandinternet-deliveredtestingwerereleasedin2005.ATP’sGuidelinesforComputer-BasedTestingpredatethatdocumentbythreeyears.Thepurposeofthenewguidelinesistoprovideinformationaboutthekey

factorsandissuesthatshouldbeconsideredwhendesigninganddeliveringtestsusingdigitalplatforms,andtoprovideguidancetotestdevelopers,testadministrators,andtestusers,onhowtobestensurefairandvalidassessmentinadigitalenvironment.

ThedevelopmentofthesejointguidelinesisbeingledbyStephenSirecifortheITCandJohnWeinerforATP.TheyserveastheCo-ChairsoftheSteeringCommittee,whichispopulatedwithmembersasfollows:ITC:KadriyeErcikan(USA);DragosIliescu(Romania);AprilZenisky(USA)ATP:AlexTong(China);AlinavonDavier(USA);LindaWaters(USA)Overthepastyear,theSteeringCommitteehasheldmonthlyconferencecalls,duringwhichthefocusofthecommittee’sworkhasprimarilyfocusedontwoareas.OnecriticaltaskforthecommitteewastoconceptualizethecollaborativeprocesstobeimplementedtoproducetheseGuidelineswithrespecttotheroleoftheSteeringCommitteeaswellasvariouscontributors.Intentionallyso,theprocesswasdesignedtofullyincorporatemanyindividualswhowerepurposefullyidentifiedandrecruitedtoensurethatadiversityofperspectivesarerepresentedatmultiplestagesinthedevelopmentoftheseGuidelines,toensurethatthefinalproductwouldberepresentativeofglobaltechnology-basedtestingapplications.

Theguidelinesthemselvesaretobedevelopedbyindividualcontentauthorsincoordinationwithseveraladvisorygroupsthatarebeingformedtoreflecta)practiceareas(Education,Certification/Licensure,Workplace,Clinical,Security,Technology)andb)regions(Europe,Asia,India,MiddleEast,Australia,Africa,SouthAmerica).Asguidelinesandrelatedcontentiswritten,itwillbereviewedbytheadvisorygroups,aswellasadhocandlegalreviewers,andoncethefulldocumentisdrafted,itwillbeopenedforaperiodofpubliccomment.Thesecond,butequallycriticaltask,hasbeentooutlinethestructureofthenewGuidelinesandtodevelopintroductorytextreflectingthepurpose,rationale,scope,andintendedaudiences.TheSteeringCommitteemembersdiscussedthepositiveaspectsaswellaslimitationsofthepriorsetsofGuidelinestoreflectcurrentandemergingpracticesintheaimofpromotingbestpracticesintestdevelopment,administration,andscoringtofacilitatefairandvalidmeasurementofthepsychologicalandeducationalcharacteristicstargetedbycontemporaryassessments.Recently,theCo-Chairspresentedasessionatthe2019ATPmeetinginFebruary,entitled“Technology-BasedAssessmentGuidelines:PresentandFuture”.Inaddition,ameetingofthesteeringcommitteeandseveralotherpracticeareaandregionalleadswasheldwiththosepresentatATP(withotherscallingin)totouchbaseontheprocessanddiscusstheworkcompletedtodate.SubmittedbyAprilL.Zenisky,onbehalfoftheSteeringCommittee.

101

More Testing News

NormanBuckleygivesushisviewsonawidelyheld

articleoffaith.

Deconstructingtheshibboleth:FeedbackShibboleth: A custom, principle, or belief

distinguishing a particular class or group of people, especially a long-standing one

regarded as outmoded or no longer important.

Myfirstjobasababypsychologistwaswithafirmof“Industrial”PsychologistsinAustralia.WeusedthetermIndustrialratherthanOrganisationalbackthen.Yes,I’mthatold.Thejobconsistedoftakingasetofcognitive,attitudinal,preferenceandpersonalityresultsandwritingareportthatpulledthemalltogether.Ifyougotitright,itproducedapictureoftheperson.Ifyougotitwrong,itwasstillagoodpicture–justnotofthatperson.I’dliketosayitwasinterestingandIsupposeIcan.Iwroteover3000oftheseinthreeyears.Butwritingwasn’ttheissue.Itwasstandardpracticethateachpersonreceiveddetailedfeedbackontheresults.Notafullcoachingsessionsincethefeedbackwasratheronewaybutitmeantthatthepersonkneweverythinginthereportandwhereitcamefrom.Nosecrets.Thesewerethedayswhentestresultswerereallynothingmorethanasetofnumbers.Youneededtobetrained(accredited)tointerpretthemandthatwastheroleofthepsychologist.Hardtobelieveasitis,computerswerearoundbutnotveryfriendly.Butweknewthattheywouldmakeadifference.Wewerecertainthatreportscouldbecomputergenerated.Wewerealsoconvincedthatthey’dneverreplacethepsychologist.Whenthefirstworkablecomputer-generatedreportsemerged,itlookedasifwewereright.TheywereOKbutwecouldalwaysdobetter.IanFraserwhenhewasatwhatwasthenSHLcommentedthat“acomputergeneratedOPQreportisnevergoingtobeasgoodasthatproducedbyagood,trainedpsychologist.But,"hesaid,"itwaswaybetterthantheaveragereportthatwasbeingwrittenintherealworld!”

becomemoresituationspecificanduserfriendly.Butitseemsthatwestillbelieveintheimportanceoffeedback.Andthatitshouldbegivenbytrainedpeople.Butfeedbackhastwoelements.Firsttherearetheresultsofwhatevertest,profileorinventorywasused.Theclientneedstoknowthissothatitislessofablackbox.ArespondentcanlookataconstructlikeExtraversion(EnergyinFacet5)andseethat,comparedtootherDanes,ChineseorArgentines,thescoreisquitehigh.Andwithaccurateunderstandingofwhatanextravertislike,he/sheshouldbeabletosay“Yep,soundsaboutright”.Thisisthefirstelementoffeedback.Agreementastotheoutput.Butthere’sthenextstage.Myson’sEnglishtutorwaschargedwithteachinghimhowtowriteessays.Nothisstrongpointwhenhewas15.Heradvicewastoimagineaparrotonhisshoulderandwheneverhewrotesomething,hisparrotwouldsay“sowhat?”.Sowhenthebuddingessayistwroteafewstrugglingwordshewouldhear“sowhat?”andthinkabouttheimplicationofwhathewassaying.Sameforfeedback.Stagetwooftheprocessis“sowhat?”SoI’mextraverted.Sowhat?Feedbackwithoutthe“sowhat”islittlemorethananegotripforthepractitioner(lookwhatIfoundout)orabitoffunfortheclient.Counsellorshavefiguredthisout.Theyfocusonthe“sowhat”.Infact,giventheenthusiasmofcounsellorsfortoolsliketheMBTI,youcouldarguethatthey’renotthatinterestedinstage1–theaccuracyoftheresults.ManyyearsagoIknewaBritishpsychologistwhoreckonedhegotgreatinformationfromaSentenceCompletionTest.Isuspecthewasusingittoopena“sowhat”discussion.Nowautomatedreportshavegotmuchbetter.Theyprovidereallygood,oftengraphicalresultssothepersoncanseewheretheysitinthepsychometricuniverse.Andfrequentlyreportsarecontextualizedtoopenthe“sowhat”discussions.Whatdoesthismeanformeinconflict?Whatdoesitsayaboutmyinteractioninateam?HowdoIcomparetoJeanoverthere?Wherearewesimilar?Wheredifferent?Whatdoesitmean?SoIthinkwecanhavesomeconfidencethatcurrentgenerationreportingcanprovidegoodfeedbackofcontentandincreasinglyuseful“sowhats”.Butwhataboutthefeedbackprocess?Doesithavetobegivenbyapsychologist?Doweneedtobetrained?

11

Food for Thought

ToolspecificvslevelspecifictraininghasbeenadebatewithintheBPS.Andwhataboutthemethod?Isthemediumthemessage.RoyChildsatTeamFocusaskedthisquestioninashortpapertitled“Togiveornottogive”.Hehadasampleofpeoplewhohadreceivedfeedbackindifferentways.Someface-to-face.Somebyvideo.Whichwasbetter?Theshortanswerhegotwasitdidn’tseemtomattermuch.Abitembarrassingforthehighlyskilled,empathicpractitionersbutthedeliveryvehicleswerebroadlyequallyeffectiveintheviewoftherespondents.ButmostpractitionersIknowwouldpreferfeedbacktobeface-to-faceandthatcanbedifficult.Whataboutoilrigworkers?OrafactorymanagerinBangladesh?Ioncegavea90minutetelephonefeedbacktotheheadofoneoftheUSsecurityservices.It’sonewiththreelettersbutIcan’ttellyouwhichone.They’relistening,youknow.Iwasdreadingit,(hadmytinfoilhaton)butitwentverywell(Ithink).Ithelpedthathewasvery,verysmartanddeducted,inductedandinferredatwill.IwasbroughtuptobelievethatweshouldalwaysofferfeedbacktotherespondentandIhaveneverheardanybodycounterthis.Somehavesaidtheycan’taffordtoordon’treallycareenoughbuttheyagreesomebodyshouldgivefeedback.Ideallythisisface-to-facebuttelephoneorvideofeedbackisOK.Butdon’tjustdumpthereportonthem.MycolleagueRebekahWilliamsreckonsthatifyoudothat,whenyoudositwiththemorphonethem,youendupspendinghalfthetimeunwindingthemisunderstandingsfromtheirinitialreadingofthereport.Somythought,andIthinkit’squitecommon,istopresentthereportduringthefeedback.Ifit’sapaperreport,youhavecontrolofthepage.Ifit’svideoortelephone,youcandoitpagebypagewhereagainyouhavecontrol.Buthangonaminute.Therearetwootherquestions.First,whoshouldgivethefeedback.Formanyyearsthiswaslimitedtoqualifiedpsychologists.LargelyduetotheworkofUKcommercialtestpublishersthishasbeenoverturnedinmanycountries.It’snowacompetency-basedmodel.Butweneedtobeawareofaresurgenceofthisidea.InBrazil,ItalyandsomeotherplacesthereispressureforonlyPhDpsychologiststobeallowedtofeedbackpsychometrictestdata.AndersWahlberg–chairmanoftheSwedishPsychologistsAssociationrecentlytoldmethereisoccasionalpressurethere.Butit’snotgoing

tohappen.Thathorsehasbolted.Ifofficialbodiestrytoimpose,businesswilljustfindawayaroundit.Ihaveseenpsychometricsreclassifiedas“behaviouralprofiles”.Nothing“psychometric”atall.Nothingtoseehere.Havingsaidthis,theimprimaturoftheBPSandNorskVeritasisstillhighlyvaluedespeciallyinScandinavia.Secondrelatestomanagementofthedata.Somecountrieshaveanopiniononhowdataismanaged,andithasasignificantimpact.TheNIP(NederlandsInstituutvanPsychologen),forexample,saystherespondenthastherighttoseetheresultsofanytestfirstanditisuptothemastowhetheranybodyelseseesit.Thisisverydifferent.Wehaveclientswhostatecontractuallythatthey“own”thedataandcandecidewhattodo.Buttheyprobablydon’t.AsfarasIknowthishasn’tactuallybeenchallengedbutundertheslowawakeningofcompaniesandindividualstotheimpactoftheGDPR(GeneralDataProtectionRegulation),itmayrequireachangeofthinking.Therightsof“datasubjects”(inGDPRlingo)areverytightlydefinedincludingwhohasit,whousesit,whatitisusedforandwhereandforhowlongitisstored.GDPRdoesnotdifferentiatebetweencharactertraits,education,politicalopinionsormedicalhistory.It’sallpersonaldata.Samerulesapply.Wealreadyhaveclientsrequestingthattestresultsshouldbesentautomaticallyfirsttotherespondentwhothendecideswhethertheywouldlikepersonalfeedback.Sowithregardtofeedbackitmaybemovingfrom“goodpractice”tolaw.Andthatwillputpressurebackontotestpublisherstomakesurethattheiroutputsaredesignedtobeseenfirstbytheclientandonlythenbysomebodyelseoftherespondent’schoosing.Sowhatstartedasaclearagreementtotheneedforqualifiedpeopletopresentfeedbackinacontrolledandprecisemannerviathechanneloftheirchoosingmayhavetochange.Itmaynolongermatterwhattheythink.Theworldhasmovedon.Thatshibbolethhasbeendeconstructed.Note:TheGeneralDataProtectionRegulation2016/679isaregulationinEUlawondataprotectionandprivacyforallindividualswithintheEuropeanUnionandtheEuropeanEconomicArea.ItalsoaddressestheexportofpersonaldataoutsidetheEUandEEAareas.

12

Publication and

Communications Committee Report

Neal Schmitt, Chair

ThePublicationCommittee'sactivitiesincludetheInternationalJournalofTesting,TestingInternational,andthebookseries.ThefirstthreevolumesintheITCbookseriespublishedbyCambridgeUniversityarenowavailable.ThefirstbookauthoredbyDragosIliescuistitled“Adaptingtestsinlinguisticandculturalcontexts”.JohnScott,DaveBartramandDouglasReynold’sbooktitled“Nextgenerationtechnology:Enhancedassessment:ChallengesandOpportunities”isalsoavailable.

ThevolumebyWilliamSchmidtandhiscolleaguestitled“Measuringopportunity:Insightsfrominternationallarge-scaleassessment”wasjustpublished.

CraigWellscontinuesworkingon“Assessingmeasurementinvarianceforappliedresearchers”

MariaElenaOliveriandCathyWendlerareeditingavolumetitled“Highereducationadmissionpractices:Aninternationalperspective”whichwillsoonbeavailable.

WehavealsocontractedwithSumayaLahertodoavolumeonthehistoryofpsychology.Sheismakingexcellentprogressandhasidentifiedsectioneditorsandmanychapterauthors.

ThecontracttodotheseriesendedonMarch1,2019andwehavesignedanotherthree-yearcontracttodofivebooks.WelookforwardtocontinuingworkingwithCambridgeaswedevelopaseriesofbooksthatdiscusstheroleandimpactofassessmentinaglobalcontext.

Continuedviabilityoftheseriesdoesdependongettingnewauthors.ThefollowinglistincludessomebooksIthoughtmightbegoodadditionstotheseries(somehavebeendiscussedbefore).Iwouldlikesomereactiontothislistandsuggestionsregardingpotentialauthors.Additionaltopicsthatyouthink

wouldbeagoodmatchfortheseriesarealsowelcome.

• Securityissuesworldwide.• Personalitytesting(Ithinkavolumeoneticandemicapproachestopersonalitytestingisclearlywithinthepurviewoftheseriesandshouldbeagoodselleraswell.Ihaveaskedseveralpeopletodoavolumeonpersonalityandnoonehasagreedtotakeitonforvariousreasons.• PrinciplesofTestingandAssessment.Ithinkwecoulduseanintroductorytextbookontestingforundergraduatesorthoseinneedofanintroductiontopsychometrics.• Licensingandcertificationtests.Thiscouldbeacompendiumofhowtestsareusedbyregulatoryauthoritiesindifferentcountriestocertifyoneiscompetentinaprofession.Ihavetalkedwithseveralpotentialauthorswithinterestbutunabletodotheworkatthistime.• Globalissuesindiversityandtesting.InatleastintheUS,Europe,andAustralia,alongstandingconcernhasbeenthefairandunbiaseduseoftestswithunderrepresentedminorities.Thegroupsaboutwhichthereisconcern,themannerinwhichthetestingcommunityaddressestheseconcerns,andthemannerinwhichgovernmentsdealwiththeseissuesvarieswidelyacrosstheglobe.Abookthatdetailstheseconcernsandhowtheyareaddressedwouldbeusefultoanyoneworkinginaninternationalcontext.

ThePublicationsCommitteehasalsoconsideredaproposalfromLarryRudnerthatITCconsiderpublishingPracticalAssessmentandResearchEvaluationwhichLarryhasdevelopedandeditedsinceitsinceptionoverthepasttwodecades.Thejournalhasanexcellentreputationandappearstohavehadasignificantimpactonassessmentpractice.TheITCCouncildiscussedthisproposalatlengthbutdecidedtodeclinefortworeasons.Onewasfinancial;wehaveverylimitedfundswithwhichtosupportanotherjournal.Thesecondwasourreluctancetotakeonthetimecommitmentthatwouldbenecessarytocontinuethisjournal’ssuccess.WewishLarrytheverybestasheseeksapartnertoassumeresponsibilityforthispublication.

ITC Committee Reports

13

InternationalJournalofTesting

NewsandUpdates

TheInternationalJournalofTesting(IJT)isdedicatedtotheadvancementoftheory,research,andpracticeintheareasoftestingandassessmentinpsychology,education,counseling,organizationalbehavior,humanresourcemanagement,andrelateddisciplines.IJTpublishesoriginalarticlesaddressingtheoreticalissues,methodologicalapproaches,andempiricalresearch,aswellasintegrativeandinterdisciplinaryreviewsoftesting-relatedtopicsandreportsofcurrenttestingpractices.IJTispublishedquarterlywitheachissuecontaining4-5articles.Wearedelightedtoannouncetheimpendingpublicationofourspecialissueon“ChallengesandOpportunitiesintheDesignofNextGenerationAssessmentsof21stCenturySkills,”guesteditedbyDrs.RobertMislevyandMariaElenaOliveri.Wewerefortunatetoreceivemanuscriptsfromhighlydistinguishedauthorsinpsychometricsandtechnology-basedtesting,sobeonthelookoutforitsreleaseverysoon!Thissummer,wewillalsoissueacallforpapersbroadlyrelatedtogamifiedassessmentandnewdevelopmentsinscoringoutsidetherealmoftraditionalpsychometrics(e.g.,machinelearning,artificialintelligence).Thisisarapidlygrowingareaofinterestineducationalandworkplacecontexts,soweplantohighlightsomeofthetechnologyanddrawattentiontotheneedforresearchonreliabilityandvaliditypertheAPA/AERA/NCMETestingStandardsandITCGuidelines.WewillissuethiscallbyearlyJulywithapproximately90daystosubmitapaper,soifyouareinterestedindoingso,pleaseperiodicallycheckforjournalnewsontheIJTandITCwebsites,aswellasonLinkedIn.

Inadditiontothisspecialissue,weremaininterestedinpublishingsome“targetarticles”andcommentariesonvarioustopics.Theideaistosubmitapaperonanissueofbroadinterest,whichhasempiricalunderpinnings,buttakesapositionthatislikelytogenerateproandconresponses.Ifyouhaveatopicideaorapaperthatyoufeelmightbesuitable,pleasecontactmebyemail(sestark@usf.edu).Finally,wecontinuetolookfornewreviewersandstreamlineourmanuscriptreviewprocess.PleasecontactmeifyouarewillingtoreviewforIJT,orifyouwouldliketonominatesomeone(includingyourself)forourBoardofConsultingEditors.Tobeselectedfortheeditorialboard,anomineemusthaveanaccomplishedrecordofpublicationandexpertiseinmeasurement/testingmethodologyandpractice.Toexpandthejournal’soutreachandinternationalimpacts,weencouragenominationsofindividualsfromunderrepresentedgroups,cultures,andregionsbeyondNorthAmerica.StephenStarkUniversityofSouthFloridaEditorofInternationalJournalofTesting

14

Membership, Involvement and Marketing (MIM) Committee Peter Macqueen, Chair

TherecentITCCouncilMeetinginVilnius,Lithuaniawasavaluableopportunitytodiscussarangeofimportantmatters,includingthoserelevanttotheMIMCommittee.

1. Membership and Fee Payment: Thankyouverymuchtoourmembers(Full,Affiliate,Individual)whohavepaidtheir2019dues,andperhapspriordues.Severalofyouhaverespondedtoourcallforsuch,andweappreciateit.ThisisparticularlytrueasweleadintoournextconferenceinLuxembourg(July,2020)andtheneedtodrawuponourfundstoensurethesuccessinstagingthisevent.IwouldliketothankAnanda,CommitteeMembers,andCouncilMembersfortheirassistance.Westillneedtofollowuponafeworganisations,andAnandaandIwillprovideafriendlyreminderofoutstandingduesoverthenextfewmonths. 2. MIM Committee Membership: WeareintheprocessofrefreshingmembershipofthisCommittee.Inparticular,wearekeentoaddapersonwithsocialmediaskills.Thereisnodoubtingthatprofessionalbodies,increasingly,needtoaddressthisimportantchannelinattemptstoengagewithmembers,andpotentialmembers.IfyouknowofanyITCmember(orpotentialmember)whowouldliketoassisttheITCfromthisperspective,pleasecontactme(atp.macqueen@compassconsulting.com.au)andtheITCSecretary-General,PaulaElosuaatsecretary@intestcom.org.Pleasenotethatthisisavoluntaryrole.3.UpcomingTestingEvents:Aspartofouractivityinfosteringorsupportingrelevantprofessionaleventsaroundtheglobe,itwouldbegreatfortheITCtobemoreawareofsuchsothatthesemaybelisted(ifnotpromoted)ontheITCwebsite,orwithinTestingInternational,thisnewsletter.

Please send your suggestions to: secretary@intestcom.org

AnandavanTonder(OfficeManager)orPaulaElosua(Secretary-General)canthendirectyouremailforaction.

Herearesomeoftheofferingsofwhichweareaware:AFRICA TheAfricanJournalofPsychological

Assessment(AJOPA)hasnowlaunched.PublishedviaAfricanOnlineScientificInformationSystems(Pty)Ltd(AOSIS),thisonlinejournaliseditedbyAssociateProfessorSumayaLaher.

https://ajopa.org (Articlesinitiallyonlinethenprintedinanend-of-

yearpublication.)

AssociationforEducationalAssessmentinAfrica(AEAA)

5–9August2019,Abuja,Nigeria https://www.aeaafrica.org/programmeASIA 22ndJapanLanguageTestingAssociation

(JLTA)Conference 11–12September2019,Niigata,Japan www.jlta2016.sakura.ne.jp 5thAnnualIndia-ATP(I-ATP)Conference 22November2019,NewDelhi,India (SeethemainATPwebsite)

EUROPE 15thEuropeanConferenceon

PsychologicalAssessment(EAPA) 7–10July2019,Brussels,Belgium https://ecpa15.com EuropeanAssociationofTestPublishers

(E-ATP)Conference 25–27September2019,MadridSpain http://www.eatpconference.eu.com/ 61stInternationalMilitaryTesting

Association(IMTA)Conference 7–11October2019,Tallinn,Estonia http://www.imta.info/conference/conference_home.aspx

15

*** 12thITCConference*** 14–17July2020,Luxembourg https://www.itc-conference.com. Seethereportonpage7ofthisnewsletter

NORTHAMERICA InternationalAssociationofComputerized

AdaptiveTesting(IACAT) 10–13June2019,Minneapolis,MNhttp://www.iacat.org/2019-iacat-conference-minneapolis-usa NationalConferenceonStudentAssessment

(NCSA) 24–26June2019,Orlando,FLhttps://ccsso.org/events/national-conference-student-assessment-ncsa ConferenceonTestSecurity 16–18October2019,Miami,FL https://conferenceontestsecurity.org/ NationalCouncilonMeasurementin

Education(NCME)AnnualMeeting 16–20April2020,SanFrancisco,CA https://www.ncme.org/home AssociationofTestPublishers(ATP)

Conference 29March–1April2020,SanDiego,CA http://www.innovationsintesting.org/atp2018/future-

conferences.aspxOCEANIA Therearenotestspecificeventsscheduledalthough

broaderconferenceswilloffersessionsrelatedtotestingandassessment.

AustralianPsychologicalSociety(APS):IOP

Conference July2019,Adelaide,Australia[IwillbeChairingaProfessionalPracticeForum:Innovationsand

DevelopmentsinPsychologicalTestingandAssessment] NewZealandPsychologicalSociety(NZPS) August2019,Rotorua,NewZealand

SOUTHAMERICA 9thCongressofTheBrazilianInstituteof

PsychologicalEvaluation(IBAP) 25–28June2019,Salvador,Brazil www.ibapnet.org.br/congresso2019(InPortuguese) 37th Congress of Interamerican Society of

Psychology (SIP) 15 – 19 July 2019, La Habana, Cuba http://www.cipcuba2019.com/2019July (Spanish and English websites) [38th Congress: 2021 in Paraguay]

NB: SomeoftheeventsIhavelistedmayhavepassed

priortopublicationofthisissueofTI.However,wesuggestyoutakenoteoftheconferencewebsitesforpossibleupdatesonfurtherconferencesin2020or2021.

PeterMacqueenp.macqueen@compassconsulting.com.auBrisbane,Australia

16

Book Review Adapting Tests in Linguistic and Cultural Situations Dragos Iliescu Cambridge University Press 2018 ISBN 9781107110120 AspartoftheITCseriesofbooks,CambridgeUniversityPresspublished"AdaptingTestsinLinguisticandCulturalSituations"byDragosIliescu,theITCPast-President,in2018.Itischallengingtowriteabookabouttestadaptation,andthatforatleastthreereasons.First,testadaptationdoesnotfollowaclearlyprescribedprocess.Thepossiblevariationsintheprocessareverylarge,anddependnotonlyonthefocaltest,butalsoonanumberofcontextualvariables(e.g.,thesourceandtargetculture,theobjectivesoftheresearcher,etc.).Ultimately,theydependonthechoicesmadebytheresearcherduringeveryphaseintheadaptationprocess.Second,testadaptationhasnoclearlyprescribedoutcomes.Althoughalargenumberofprojectsareidentifiedas“testadaptations”,thedifferencesbetweenthemaresometimelargerthanthesimilarities.Third,testadaptationliteratureisnotaclearbodyofknowledge.Theliteratureisverymuchsplitbetweenreportsofcross-culturalresearch(bothsmall-scaleandlarge-scale),reportsofactualtestadaptations,testmanuals,technicaldetailsthatareusuallyrelatedtosomeformoranotherofstatisticalprocedure,andagreatbodyofadviceliterature.Despitethedifficultiesintrinsicinthistopic,theauthorhasrisentothechallengeandhasprovidedanimportantcontributiontothisarea.Thebook,thefirstofitskind,providesanin-depthtreatiseandguideonwhyandhowtoadaptatesttoanewcultureinsuchawayastopreserveitspsychometricvalue.Overmorethan700pages,theauthorexplorestestadaptation,testlocalization,andtestindigenization,animportantscientificandprofessionalactivitythatisnowubiquitousinthesocialandbehaviouralsciencesaspartofworkintoday'sglobalizedworld.AdaptingTestsinLinguisticandCulturalSituationsdiscussestheprinciplesandhistoricalrootsoftest

adaptation.Itacknowledgestheinterdisciplinaryvocationofthisactivity,statingthatitisunclearwhetherit“belongs”-atleastfromahistoricalpointofview-moretoanygivensciencethantoanother.Atthesametime,itunderlinesthatpsychology,andspecificallycross-culturalpsychology,hasprobablythestrongestclaimonthedomainoftestadaptation,withothersciencesandbranchesalsoofferingstrongcontributions,suchaseducationalsciences(e.g.educationaltesting),sociology(e.g.surveytranslation),medicine(e.g.epidemiologicalstudies)andothers.Thebookhasthreeparts.Thefirstpartcoverssomepreliminaryissuesintestadaptation.Itdiscussesterminology,andexplainsthevariousformsinwhichadaptationscanbefound;delvesintoissuesrelatedtocopyrightandtheprotectionofintellectualproperty;andpresentsanumberofdocumentswhichconstitutethestandards,guidelines,andrecommendationsofthedomain.Thesecondpartfocusesonequivalence-arguablythemostimportantconceptinthisprocess.Itexplainswhatequivalenceis,describesitsvariousforms,discussesindetailhowtotestforthevariousformsofequivalenceandtodetectbias,andhowtodealwithbiasonceithasbeendetected.Thethirdpartexaminesthecriticalphasesintheadaptationprocess.Itdiscussesvarioustranslationdesigns,describespretestingandnorming,andlooksathowtocompletetheprocessbyfinallypublishingthetestmaterials.Thebookconcludeswithalargeandveryappliedchapterwithpracticalexamplesoftestadaptations.Whatmakesthisbookaparticularlystrongcontributionisitspracticalstance.Itprovidesastep-by-stepapproachtotestadaptation,presentingitemphaticallyasamélangebetweenscienceandpractice.Thevolumeisdrivenbythefirst-handpracticalexperienceoftheauthorinalargenumberoftestadaptationprojectsinvariouscultures,andissupportedbytheconsistentscientificbodyofknowledgeaccumulatedonthetopicoverthelastseveraldecades.Itwillproveinvaluableforanyoneinvolvedinthisaspectofpsychologicaltesting.

17

AgustinTristan-LopezdiscussesevaluationinMexico.EvaluationandAssessmentinHigherEducationinMexicoAgustinTristan-Lopez,PhD.Director.InstitutodeEvaluacióneIngenieríaAvanzada,Mexico&ITCmember.IntroductionDuringthelastfourdecades,therehasbeenfluctuatinginterestinevaluationtopicsinMexico.Nowadays,theunfavourableappreciationismainlyduetoahistoryofinformal,subjective,non-systematicevaluationperformedbyteachersfrombasiceducationtohighereducation,wheretheindiscriminateuseofexams(andactuallywiththeabuseofrubricsandportfolios)isusuallyreducedtotheassignmentofgradesandnottoinformorfeedbacktostudentsorteachers.Theeffectthatgradeshaveontheallocationofprizesandpunishmentsdistortstheinterestinevaluationinmanyschoolenvironments,contributingtoapoorappreciationoflarge-scaleteststhataresupposedtoinformandsupportacademicandprofessionaldevelopment.Theapplicationofnationwidelarge-scaletestsinitiatesinthe1960s,withexamsproducedbytheSecretariatofPublicEducation(SEP),thesocalled“KnowledgeOlympics”andthefinal(orextraordinary)teststopassacourse(Martinez-Rizo,2001).SubsequentlytheSEPproducedlearningtests(oncepersemester)withscarcetechnicalfoundations,administeredtostudentsamples.Atthattimepeoplebelievedthatitaffectedminorityordisadvantagedpopulations,assuming–withoutdemonstration–aculturalbiasagainstindigenouspopulationsandstudentsoflowsocioeconomicstatus.TheseSEPtestswereofficiallyusednotonlytoevaluatestudents,butalsofortherankingofteachers,institutions,statesubsystemsandthenationalsystem.Inanycasethoseexamshadtheimplicitpurposeofimprovingplanningandfavouringtheaccountabilityofeducationstakeholders.Thediversityofusesofthetestsinvolvedseveraldefectsofinterpretation,errorsofappreciationandannoyanceinsomeacademicandsocialgroups.ForHigherEducationLevel,large-scaleevaluationsweredevelopedfrom1960.Inparticular,theNationalAutonomousUniversityofMexico(UNAM)begantoapplyadmissiontestswithmorethanonehundredmultiple-choiceitems(MCI),gradedbysoftwareprovidedbytheopticalmarkreader.

Theseinitialtestsdidnotguaranteetheequalizationoftestversionsinthesamepromotionoroverseveralyears,nordidtheyhavecalibrationanalysis.InterestinevaluationinHigherEducationincreasedinthe90swhenuniversitiesandexternalagenciesstartedtoprovidetestservices.Itispossibletoclassifythetestsaccordingtothemomentandpurposeoftheevaluation:a) Selectiontests.Designedasnormreferencedtestsforthe

admissiontotheinstitution(producedbyuniversitiesandexternalagencies).

b) Diagnostictests.Designedtoidentifythestrengthsandweaknessesofstudents,inordertoprovidefeedbackintheircompetenciesforsuccessintheuniversitycareer.Thesetestsweredesignedbyexternalagenciesandappliedatdifferenttimes(onadmissionorattheendofthefirsttwoyearsofthecareer).Unfortunately,theyarestillnotwidelyused.

c) Degreetests.Generallyproducedbyanexternalagencybasedonageneralspecificationforalluniversitiesinthecountry.ThesetestsaredefinedbytechnicalcommitteesfromprofessionalboardsandareonlyanoptionattheendoftheCollegeorUniversitycourse,becauseincertainuniversities,studentscouldgettheprofessionaldiplomabyfinishingallthecourseswithahighmeanofgradesorwithashortthesis.

d) Certificationtests.Theseexamsareprofessionalacademiccertificationoftheperformanceofprofessionalswhohavemorethan3to5yearsofexperiencesincefinishingUniversity.ThesetestsareofferedbyProfessionalBoardsinagivenarea(forinstanceAccountancy,Medicine,Nursing,andCivilEngineering).TheProfessionalBoardmusthaveacontractwithanexternalevaluationbodyandbeauthorizedbytheGeneralDirectorateofProfessionsoftheSecretariatofPublicEducation.

Alltheseexamsareindependentofthosepreparedbyindividualteachersorinstitutions.Generally,classexamsandquizzes,aswellasso-calledauthenticassessmenttools,areinformaltoolsshowingdeficienciesonvalidity,objectivityandreliability.Institutionalexams(calleddepartmentalexams)aremorestructuredandarepreparedbyagroupofteachersaccordingtotheplannedprogressattheendofaunitorarea,buttheyarenotalwaysadministeredorscoredunderstandardizedconditions.Inconsequencetheyofferonlyalimitedadvantageforqualitycontroloflearningandteaching.Asmallnumberofinstitutionshavecalibrateditembanks,andalessernumberofthoseanalysetheteststocalibratetheitemsordeterminethemeasurementerror.Itisacommonpracticetogradestudentsbycombiningtheresultsofexamsandportfolioswithhomework,participation

18

Around the World

inclassandavarietyofotherelements,leadingtosubjectiveevaluationsandlow-qualityassessmentprocesses.Currentlytherearesomeuniversities(mainlyprivate)thathavecomputerplatformstoadministerquizzes,quicktestsorexamsattheendofatopicorunit.Theyhavetoolstostoresomeevidenceanddocumentstointegrateaportfolioandforadministrativepurposes.Unfortunately,theseplatformsingeneralarenotfocusedonprovidingfeedbacktostudentsandteachers.EvaluatingAgenciesInMexicotherearesomeagenciesorinstitutionsthatproducetestsforsaleasaservice,inschemessimilartotheACTortheETSoftheUnitedStates,CITOintheNetherlandsortheNFER in the UK.MexicanevaluationagenciesarehighlyprofessionalandpossesstechnicalcapabilitiestoprovidequalityproductstoevaluateHigherEducation.Asanewformofattractingprojects,someuniversityinstitutionsofferevaluationservicesforspecificpurposes.Forinstance,since1990theAutonomousUniversityofBajaCaliforniahasofferedtheKnowledgeandBasicSkillsTest(EXCOHBAbyitsnameinSpanish).TheUNAM(AutonomousNationalUniversityofMexico),theUniversidadIberoamericanaUniversityandtheUniversidadVeracruzanaoffersometestsforprofessionalcertificationsince2013.Withintheevaluationagencies,theNationalCentreofEvaluationforHigherEducation(CENEVALbyitsnameinSpanish),basedinMexicoCitysince1994,offerssomeproducts(http://www.ceneval.edu.mx/),suchas:(a)admissionteststohighschool,universityandpostgraduatecourses(calledEXANII,IIandIII),(b)graduationtestsforsomeprofessions(calledEGEL)and(c)certificationtestsastheExternalEvaluationBodyforProfessionalBoards(suchasaccountants,actuariesordentists).Thesecriterionreferencedtests(CENEVAL,2017)defineaprofessionalprofilebasedonacademicknowledge,withthetestscontainingbetween120and200multiplechoiceitemsorganizedintwosessions(from3to4hourseach).Intheteacher’sevaluation,thecertificationprocessincludesareviewofavideoperformanceinclass.ThewebpageofeachevaluationprojectprovidesthecandidatewithaguideinPDFformat.Ingeneral,thisagencyfollowssomeofthestandardsfromAERA-APA-NCME(1999).TheInstituteofEvaluationandAdvancedEngineering(IEIAbyitsnameinSpanish)isanongovernmentalandnon-profitcivilassociationbasedinSanLuisPotosísince1989(www.ieia.com.mx).TheIEIAoffersseveralproducts:(a)diagnosticassessmentsofcompetenciesforadmissionofstudentsatundergraduateorgraduatelevels(calledDICOIN),(b)formativeevaluationsystemforhighereducationteachers(ECPEMS)(Tristan&Ylizaliturri,2012),(c)certificationtestsasExternalEvaluatingBodyforProfessionalBoardsintheareasofhealth(Nursing,GeneralMedicine,Pharmaceutical

Chemistry)andengineering(Civil,Mechanical,Electrical,Petroleum,Appraisals)and(d)softwarefortestscoring,itembankingandon-linetesting.TheassessmentprojectsoftheIEIAarecriterionreferenced,basedonthedefinitionofaprofile:academicsuccessexpectationsforthediagnostictestsorcompetenciesforprofessionalcertification.Eachprojectincludesavarietyofinstruments:objectivetestswithmultiplechoiceandforcedresponseitems,openresponsesectionsforclinicalcases(inhealth)orpracticalcases(inengineering),simulators,analysisofperformanceusingasequenceinavideo;lexicalanalysisandautomaticscoringoftexts.Eachprojectincludesaformativecomponentonaweb-basedplatform,wherethecandidateandtheinstitutioncanfindfeedbackreportsandavarietyofresourcessuchaswhitepapers,linkstoarticles,videosandactivities.ThisagencyfollowsthestandardsfromAERA-APA-NCME(2014),theICTguidelinesrecommendationsforfairtesting,testscoringandreportandtheIEIAproprietarystandards(Tristan&Vidal,2006).EvaluationexperiencesinaccordancetoareasofknowledgeThemostinterestingareasofdevelopmentinevaluationpertaintoMedicineandEngineering.HealthSciencesDuetothelargenumberofstudentsinMedicine,since1973theAreaofMedicalEducationoftheFacultyofMedicine(UNAM)hasproducedanobjectiveprofessionaltest.Atthebeginning,itwasarigidmodelof280multiplechoiceitems,wherethefirst210wereindependent,withtheremaining70itemsorganizedin14testletsorclinicalcases,consistingof5itemseach.Thismodelsubsequentlychangedtoencompassmoreclinicalcasetestlets,onthesamebasisof5itemsforeachcase.Formanyyearsresponseswereregisteredusingpre-punchedcardsandscoredusingasoftwareprogrammedinFORTRANorALGOLbyaresponsibleindividualinthesameFaculty.ThissoftwarechangedtoothertoolsforPCinthe90s.Thetestblueprintandspecificationswerealignedtothecurriculumandmorerecentlytosomeprofessionalcompetencies.Astudyguideisofferedtothestudent,whichhasaninterestingevolution,inthatatfirst,theitembankfromwhicheachtestwasderivedwasfullydisclosed,effectivelyrequiringthestudenttomemorizeseveralthousandquestions.Thiserroneouspracticeofpreparingforthetestwasgraduallymodifiedandisnolongerused.GraduationtestsarecurrentlydividedintoatheoreticalphasewithmultiplechoiceitemsandapracticalphaseincludingObjectiveStructuredClinicalExaminations(OSCE).Otheruniversitiesinthecountryareusingasimilarmodel.Professionalcertificationtestsareappliedwhenapersonhasmorethan5yearsofexperienceaftercompletingthemedicalschool.Thisnationwidecertificationhasbeenproducedfor

19

theMexicanBoardofCertificationinMedicinebytheIEIAsince2010.Nursingcertificationisproducedbythreeagencies(CENEVAL,IEIAandUniversidadIberoamericana)forthreedifferentProfessionalBoards.ThatfromtheMexicanBoardofCertificationinNursingistheoldest,havingbeenestablishedin2007.Theexamcontainsfrom220to280multiplechoiceitems,aportfolio,andaclinicalcasetodevelopinanopenresponseanswersheet.EngineeringThesecondprofessionwithasolidtrajectoryinevaluationisengineering.Since1976,evaluationshavebeenprepredbytheFacultyofEngineering(UNAM).Itemphasisestheevaluationofmathematicsforsixareasduringthefirstfoursemesters,includingdiagnostictestsforadmission,asmentionedintheUNESCO’sreportonMathematicalEvaluation(Kilpatrick,1979).TheevaluationmodeloriginallyincludedthedesignofascoringsoftwareinFORTRANIVabletohandlemultiplechoice,true-false,ranking,columnrelationship,andbriefresponseopenquestions.Initiallyitusedpre-punchedresponsecards,butthenmovedtoanopenformatneedingateamoftypists.Aspecificprogramwasproducedtoscoreandcalibratetheitems,providingstatisticalanalysisofthetestandthestudents.Feedbackreportswereprovidedtostudents,teachers,academiccoordinatorsandfacultydirectors.Aninterestinginnovationwasthereportgoingtotheschoolsoforiginofthestudentswhohadenteredtheuniversity.Theitembanksystemofferedinnovativeserviceswithquicktests(quizzes)andinformativereportstotheteachers,toimprovetheirclassplanningandtheirownevaluations.ThoseoriginalsystemshavebeenupdatedfortheWindowsenvironmentandarenowcommerciallyavailableontheWebbyanexternalagency.OtherDomainsInPsychology,thefocusinpsychometricsismainlyontheuseofcommercialtestssuchasMMPIorWechsler,forclinicalapplicationsandemploymentselection.AgraduatetestisproducedbyCENEVAL,butnootherprofessionalnationwidecertificationtestingisactuallyproduced.Since1960,severalMexicaninstitutionshavehiredtheSATtotheCollegeBoardofPuertoRico.ThisproducedsomeinitialproblemsofdecontextualizedlanguageforMexico,whichhavebeenimprovingovertime.Evidencepointstoevaluationinhighereducationbeingaheterogeneouspanorama.Ononeside,thereareinternationallyknownimportantagencies,producingprojects,softwareandpsychometricmodelsusingclassicaltesttheory,RaschandIRT,withasignificantemphasisonassessmentandcomprehensivereports.However,inthepastfewyears,politicalandsocialrequirementsarechangingthefocusofevaluation.Forinstance,thecurrentholisticeducationalmodelisnot

interestedinmeasuringmodels,masteringpsychometrictechniquesorprovidingtechnicalfeedbacktothestudents,professionalsandinstitutions.Universitiesaimtosellevaluationservicesforprofessionalcertification,despiteitnotbeingpartoftheircorecompetencies.Thegovernmentandunionsarereluctanttoparticipateinhighstakestesting,arguing,withoutevidence,thattheyareunfairfortheirlabourcontracts.ThenewtrendsinevaluationarerespondingtocomplexandsubjectivepoliticalandsocialdecisionsthataffectthefutureconditionofallMexicans.OthercountriesinLatinAmerica,AfricaandevenEuropeareusingsomeoftheapproachesanddevelopmentsproducedinMexico.Inaddition,atuniversitylevelingeneralandintheevaluationfieldinparticular,Mexicohasworldwiderecognizedevaluationagenciesandspecialistswithaveryhighlevelofexpertiseatthetopofthedevelopmentofmeasurementmodelsandsoftwaredesign.ReferencesAERA-APA-NCME(1999/2014).Standardsforeducationaland

psychologicaltesting.Washington,AmericanEducationalResearchAssociation,AmericanPsychologicalAssociation&NationalCouncilonMeasurementinEducation.Washington,D.C.Availablein:https://www.apa.org/science/programs/testing/standards

CENEVAL(2017).AnuarioCeneval.México,CENEVAL.Availablein:http://www.ceneval.edu.mx/documents/20182/49855/Anuario+Ceneval+2017_2/5d95c427-bd58-45df-b052-a332d626cf5c

Martínez-Rizo,F.(2001).Evaluacióneducativaypruebasestandarizadas.Elementosparaenriquecereldebate.RevistadelaEducaciónSuperior.ANUIES.Vol.120.México.Availablein:http://publicaciones.anuies.mx/revista/120

Kilpatrick,J.(1979).Methodsandresultsofevaluationwithrespecttomathematicseducation.In"Newtrendsinmathematicsteaching",Vol.4,pp.162-179.InternationalCommissiononMathematicalInstruction.UNESCO.Paris.Availablein:https://unesdoc.unesco.org/ark:/48223/pf0000124827

Tristán,L.A.eYlizaliturri,S.M.A.(2012).CertificationofTeachingCompetenciesattheHigh-SchoolLevelinMexico:AnEvaluationModelforaSocialChallenge.TheInternationalJournalofEducationalandPsychologicalAssessment.9(2)p.139-164.Availablein:https://sites.google.com/site/tijepa2012/articles/vol-9-2

20