Tools for Successful Selection Employment Testing: General.

Tools for Successful Tools for Successful SelectionSelection

Employment Testing: GeneralEmployment Testing: General

Types of Employment TestsTypes of Employment Tests

Ability TestsAbility Tests• Mental (Cognitive) AbilityMental (Cognitive) Ability• Psychomotor AbilityPsychomotor Ability• Physical AbilityPhysical Ability• Sensory AbilitySensory Ability

Achievement and Skill TestsAchievement and Skill Tests• Job KnowledgeJob Knowledge• Performance/Work SamplePerformance/Work Sample

Types of Employment TestsTypes of Employment Tests

Personality/Interest InventoriesPersonality/Interest Inventories Other Types of TestsOther Types of Tests

• Integrity (honesty)Integrity (honesty)• DrugDrug

Selecting Employment TestsSelecting Employment Tests Which KSAs are relevant to testing?Which KSAs are relevant to testing? Search sources of testsSearch sources of tests

• GeneralGeneral Tests: A Comprehensive ReferenceTests: A Comprehensive Reference...... Web-based information sources

• ReviewsReviews Mental Measurement Yearbook

• Useful Useful article on using the MMYon using the MMY British Psychological Society Test CritiquesTest Critiques

http://leeds-faculty.colorado.edu/Rosse/Courses/4020/Emptests.htm

http://libnet.colorado.edu/survey/survey.cfm?url=http://search.ebscohost.com/login.asp?profile=web&defaultdb=loh

http://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=63890862&site=ehost-live

http://www.psychtesting.org.uk/test-registration-and-test-reviews/test-reviews.cfm

Evaluating Test ManualsEvaluating Test Manuals

Evidence of reliabilityEvidence of reliability Relevant evidence of validityRelevant evidence of validity

• Use of appropriate criteriaUse of appropriate criteria• Norms, based on appropriate samplesNorms, based on appropriate samples

Evidence of test fairnessEvidence of test fairness PracticalityPracticality


Cognitive Ability (Intelligence) Cognitive Ability (Intelligence)

TestsTests

What is “Intelligence”?What is “Intelligence”?

And is it relevant And is it relevant to job to job performance??performance??

““Successful” IntelligenceSuccessful” Intelligence

Analytical intelligenceAnalytical intelligence• ““Academic” intelligenceAcademic” intelligence

Practical IntelligencePractical Intelligence• ““Common sense” as well as social or Common sense” as well as social or

emotional intelligence; “street smarts”emotional intelligence; “street smarts” Creative IntelligenceCreative Intelligence

Ability to achieve success in life and work; requires a balance among:

Analytical IntelligenceAnalytical Intelligence

Traditional aspects of intelligence Traditional aspects of intelligence measured by standard testsmeasured by standard tests• Verbal and numerical ability, memory, Verbal and numerical ability, memory,

processing speedprocessing speed Predicts job performance in nearly all jobsPredicts job performance in nearly all jobs

• More important as jobs become more complexMore important as jobs become more complex• Probably due to:Probably due to:

Broader knowledge baseBroader knowledge base Ability to learn on the job more quicklyAbility to learn on the job more quickly Ability to problem-solveAbility to problem-solve

Practical IntelligencePractical Intelligence Ability to deal with everyday life challengesAbility to deal with everyday life challenges

• E.g., computing lowest price/unit when grocery E.g., computing lowest price/unit when grocery shopping, or interpreting a bus scheduleshopping, or interpreting a bus schedule

Social/emotional intelligenceSocial/emotional intelligence• Ability to understand people and relationshipsAbility to understand people and relationships• Impression management skillsImpression management skills• Ability to persuade othersAbility to persuade others

Adaptability to different social situationsAdaptability to different social situations Shown to add to predictive power of Shown to add to predictive power of

analytical intelligence for sales and execsanalytical intelligence for sales and execs May be culture-specificMay be culture-specific

Creative IntelligenceCreative Intelligence

Relies on a base of knowledge + Relies on a base of knowledge + ability to apply it to new situations or ability to apply it to new situations or problems problems • Ability to generate novel high-quality Ability to generate novel high-quality

ideasideas• Ability to recognize opportunities or Ability to recognize opportunities or

define challenges in new waysdefine challenges in new ways• Ability to adjust to new situationsAbility to adjust to new situations

Measuring IntelligenceMeasuring Intelligence

Wide variety of measures of analytical Wide variety of measures of analytical intelligence/general mental abilityintelligence/general mental ability• Wonderlic Personnel Test is most popularWonderlic Personnel Test is most popular

Sternberg Multidimensional Abilities Test Sternberg Multidimensional Abilities Test measures all 3 intelligencesmeasures all 3 intelligences

Sternberg and Wagner’s test of Practical Sternberg and Wagner’s test of Practical Managerial Intelligence: Managerial Intelligence: • ability to write effective memos ability to write effective memos • ability to motivate people ability to motivate people • knowledge of when to delegate knowledge of when to delegate • ability to “read” people ability to “read” people

Lingering Controversies about Lingering Controversies about IQIQ

Is intelligence testing fair?Is intelligence testing fair? Is intelligence—even when measured Is intelligence—even when measured

more broadly—sufficient?more broadly—sufficient? Do you want “bright” or “well Do you want “bright” or “well

trained” employees?trained” employees?

Fairness and Intelligence Fairness and Intelligence Testing: A DilemmaTesting: A Dilemma

CA predicts performance in a wide CA predicts performance in a wide variety of jobsvariety of jobs• Measures knowledgeMeasures knowledge• Measures ability to learn, reason, and Measures ability to learn, reason, and

solve new problemssolve new problems• Few alternative predictors with Few alternative predictors with

comparable utility for entry-level jobscomparable utility for entry-level jobs But are they “fair” to all groups?But are they “fair” to all groups?

True Validities of Predictors For Entry-Level Jobs

Ability Test Battery .53 Interview (structured) .44 Job Tryout .44 Biographical Inventory .37 Reference Checks .26 Personality Inventories ~.23 Experience Ratings .18 Interview (unstructured) .14 Academic Achievement .11 Interest Inventories .10

For Promotions Work Sample .54 Ability Test Battery .53 Peer Ratings .49 Experience Ratings .49 Job Knowledge Test .48 Assessment Center .43

1. Differences between groups’ average test 1. Differences between groups’ average test scoresscores• ~1 SD difference for African-Americans~1 SD difference for African-Americans• ~.75 SD for Latinos~.75 SD for Latinos

But what do these differences mean?But what do these differences mean?

Definitions of Test “Fairness”Definitions of Test “Fairness”

• Groups differ in ability?• Test discriminates against one group?


2. 2. Differential ValidityDifferential Validity: Group : Group differences in differences in validityvalidity of test of test


2. 2. Differential ValidityDifferential Validity: Group : Group differences in validity of testdifferences in validity of test

White

Black

Test Scores

JobPerf


3. Test Bias3. Test Bias: Group differences in : Group differences in predicted job performance with same predicted job performance with same cutoff scorecutoff score


3. Test Bias3. Test Bias: Group differences in : Group differences in predicted job performance with same predicted job performance with same cutoff scorecutoff score

Black

Test Scores

JobPerf

White

Typical CA Test SituationTypical CA Test Situation

Adverse ImpactAdverse Impact No differential validityNo differential validity No test biasNo test bias

White

Black

Test Scores

JobPerf

Minimizing Adverse ImpactMinimizing Adverse Impact

Recruit high quality applicantsRecruit high quality applicants Use tests only when appropriateUse tests only when appropriate Use tests as one part of processUse tests as one part of process Use to eliminate unqualified (rather Use to eliminate unqualified (rather

than to differentiate among most than to differentiate among most qualified)qualified)

Be sure test is valid (job-related)!!Be sure test is valid (job-related)!!

Why Intelligent People FailWhy Intelligent People Fail

Lack of motivation Lack of motivation Lack of impulse control Lack of impulse control Lack of perseverance Lack of perseverance Fear of failure Fear of failure Procrastination Procrastination Inability to delay gratification Inability to delay gratification Too little/too much self-confidence Too little/too much self-confidence

Source: Sternberg Source: Sternberg (1986)(1986)

Intelligence or Training?Intelligence or Training?

Is being smart Is being smart a pre-requisitea pre-requisite for learning? for learning? Is being smart Is being smart sufficientsufficient for learning? for learning?


Personality TestingPersonality Testing

InvasivenessInvasiveness

Employer right to know vs. right to Employer right to know vs. right to privacyprivacy

Over-emphasis on individual itemsOver-emphasis on individual items Reducing invasiveness:Reducing invasiveness:

• Use only for job-related traitsUse only for job-related traits• Use employment (not clinical) measuresUse employment (not clinical) measures• Use as part of overall systemUse as part of overall system• Explain purpose as matching not Explain purpose as matching not

screeningscreening

Extreme Items from MMPI

I have never indulged in any unusual sex practices.

I have never been in trouble because of my sex behavior.

My soul sometimes leaves my body.

Evil spirits possess me at times.

I sometimes feel like I’m going to pieces.

Often I feel as if there were a tight band around my head.

I have had no difficulty starting or holding my bowel movements.

Sample Items Modeled After Those on NEO-PI

I am generally not a worrier.

I would rather cooperate than compete.

I don’t mind bragging about what I’ve accomplished.

I’m pretty set in my ways of doing things.

I try to keep all my thoughts realistic and to avoid flights of fancy.

I have no sympathy for “street people.”

I prefer jobs that allow me to work alone without being interrupted by others.

I am a person who always gets the job done.

Is Personality Job-Relevant?Is Personality Job-Relevant?

The “Big Five” Personality TraitsThe “Big Five” Personality Traits• ConscientiousnessConscientiousness• Emotional StabilityEmotional Stability• ExtraversionExtraversion• AgreeablenessAgreeableness• <Openness to Experience><Openness to Experience>

Others may apply in particular jobs Others may apply in particular jobs • e.g., customer service, safety, attention e.g., customer service, safety, attention

to detailto detail

Do Personality Tests Predict?Do Personality Tests Predict?

Is the situation Is the situation weakweak or or powerfulpowerful?? Is the measure valid?Is the measure valid? Validities typically in low .20s (when Validities typically in low .20s (when

based on job analysis)based on job analysis)

Faking & Personality TestsFaking & Personality Tests

Applicants often distort their answers Applicants often distort their answers to look goodto look good• Some applicants more than othersSome applicants more than others• This can affect who is hiredThis can affect who is hired

Reducing faking:Reducing faking:• Use faking (lie) scales Use faking (lie) scales • Reduce motivation to fakeReduce motivation to fake• Use less transparent questionsUse less transparent questions

Social Desirability Scale Items

I sometimes drive faster than the speed limit.

My table manners are not quite as good at home as when I’m out in company.

I gossip a little at times.

Sometimes I vote for people I know very little about.

I do not like everyone I know.

I sometimes tell lies if I have to.

Fairness of Personality TestsFairness of Personality Tests

Little evidence of racial Little evidence of racial discriminationdiscrimination

Men and women differ on many Men and women differ on many scalesscales• but no evidence of “discrimination”but no evidence of “discrimination”

Watch out for adverse impact due to:Watch out for adverse impact due to:• religious beliefsreligious beliefs• disabilitydisability

Ways to Assess PersonalityWays to Assess Personality

Paper & Pencil InventoriesPaper & Pencil Inventories Individual (clinical) assessmentIndividual (clinical) assessment During interviewsDuring interviews


Correlation Between Self- and Interviewer-Ratingsr

Extraversion 0.27Conscientiousness 0.22Openness to Experience 0.11Agreeableness -0.29Emotional Stability -0.30


Paper & Pencil InventoriesPaper & Pencil Inventories Individual (clinical) assessmentIndividual (clinical) assessment During interviewsDuring interviews Reports from othersReports from others Behavioral observationBehavioral observation


Assessing Deviance:Assessing Deviance:

Drug TestingDrug Testing

Integrity (Honesty) TestingIntegrity (Honesty) Testing

What is “Job Performance”?What is “Job Performance”?

In-Role Behavior

Extra-Role

Behavior

Counter-Productive Behavior

Risk Factors

General RationaleGeneral Rationale

Employee deviance is a costly Employee deviance is a costly problemproblem• $6 to 200 billion annual cost $6 to 200 billion annual cost • 33-75% of employees 33-75% of employees

It’s said that 30-50% of business It’s said that 30-50% of business failures are due to theftfailures are due to theft

Drug/alcohol abuse results in $197B Drug/alcohol abuse results in $197B in productivity costsin productivity costs

How Do You Screen out How Do You Screen out Deviant Employees?Deviant Employees?

Some are based on past behaviorSome are based on past behavior• analysis of drug metabolites in urine; analysis of drug metabolites in urine;

past theft; reference and background past theft; reference and background checkschecks

Others focus on likely future behaviorOthers focus on likely future behavior• attitudes toward theft, drug use; attitudes toward theft, drug use;

personalitypersonality Only a few focus on current behaviorOnly a few focus on current behavior

• impairment testingimpairment testing

Physiological Drug TestsPhysiological Drug Tests

UrinalysisUrinalysis• Quick screeningQuick screening• Confirmatory: Gas Confirmatory: Gas

Chromatography/Mass Spectrometry Chromatography/Mass Spectrometry (GC/MS)(GC/MS)

Hair analysisHair analysis Blood analysisBlood analysis Breath analysisBreath analysis

What’s Measured (& for how What’s Measured (& for how long?)long?)

The “NIDA 5”:The “NIDA 5”:• CannabisCannabis• CocaineCocaine• AmphetaminesAmphetamines• PCPPCP• OpiatesOpiates

What’s Measured (& for how What’s Measured (& for how long?)long?)

The “NIDA 5”:The “NIDA 5”:• CannabisCannabis 2 – 90 days2 – 90 days• CocaineCocaine 1 – 2 days1 – 2 days• AmphetaminesAmphetamines .5 – 2 days.5 – 2 days• PCPPCP 30 – 90 days30 – 90 days• OpiatesOpiates 2 – 4 days2 – 4 days

Other Forms of Drug TestsOther Forms of Drug Tests

Paper-and-pencil integrity testsPaper-and-pencil integrity tests Impairment testingImpairment testing

• based on person’s own “baseline” based on person’s own “baseline” performance on a computer taskperformance on a computer task

Types of Drug Testing Types of Drug Testing ProgramsPrograms

Pre-employmentPre-employment ““For cause” testingFor cause” testing

• mandatory post-accidentmandatory post-accident• based on supervisor’s observationsbased on supervisor’s observations

Fitness for duty Fitness for duty • daily (impairment testing)daily (impairment testing)• post-rehabilitationpost-rehabilitation

RandomRandom

Types of Honesty/Integrity TestsTypes of Honesty/Integrity Tests

““Clear Purpose” testsClear Purpose” tests ““Disguised Purpose” testsDisguised Purpose” tests

• Attitudes about devianceAttitudes about deviance• PersonalityPersonality

Polygraph/Voice Stress AnalyzerPolygraph/Voice Stress Analyzer

Clear Purpose TestClear Purpose Test - Honesty/Morality Items- Honesty/Morality Items

If you received an extra $1.00 in If you received an extra $1.00 in change when you purchased change when you purchased groceries, would you keep it? groceries, would you keep it?

If you saw a co-worker stealing, would If you saw a co-worker stealing, would you turn her in? you turn her in?

How many people have you known How many people have you known who have stolen valuables at work? who have stolen valuables at work?

Most people cheat on their income Most people cheat on their income tax. tax.

Clear Purpose Test:Clear Purpose Test: - Drug-Related Items - Drug-Related Items

I have never been a heavy drinker. I have never been a heavy drinker. On occasion I have used On occasion I have used

alcohol/drugs excessively alcohol/drugs excessively If I thought I could get away with it, If I thought I could get away with it,

I'd probably try cocaineI'd probably try cocaine Have you ever missed work because Have you ever missed work because

you were too high/drunk?you were too high/drunk?

Disguised Purpose TestDisguised Purpose Test - - Drug Attitude ItemsDrug Attitude Items

A drug addict who steals to support A drug addict who steals to support his habit should get the same his habit should get the same penalty as a person who steals for penalty as a person who steals for profit. profit.

Employers expect a certain amount Employers expect a certain amount of drug and alcohol use among their of drug and alcohol use among their workers. workers.

How many people have you known How many people have you known who drink or do drugs while working? who drink or do drugs while working?

Disguised Purpose Test:Disguised Purpose Test: - - Personality ItemsPersonality Items

I enjoy making people feel better.I enjoy making people feel better. I don't care if others like the things I I don't care if others like the things I

do.do. I shouldn't do many of the things I do.I shouldn't do many of the things I do. I have never been in trouble with the I have never been in trouble with the

law.law. Sometimes I enjoy going against the Sometimes I enjoy going against the

rules.rules.

Drug & Integrity Testing IssuesDrug & Integrity Testing Issues

LegalityLegality Accuracy/Job relevanceAccuracy/Job relevance Faking (integrity tests)Faking (integrity tests) Consequences of failingConsequences of failing Negative reactions (invasion of Negative reactions (invasion of

privacy/fairness)privacy/fairness)

Integrity Testing: A CautionIntegrity Testing: A Caution Integrity tests assume cause of Integrity tests assume cause of

deviance is bad employeesdeviance is bad employees But the situation may also be a factorBut the situation may also be a factor

• role models, group norms may “push” role models, group norms may “push” people towards deviance people towards deviance

• dissatisfaction may “pull” towards dissatisfaction may “pull” towards deviancedeviance

Integrity tests don’t replace good Integrity tests don’t replace good managementmanagement


Employment Testing: Employment Testing:

Performance Tests & Work Performance Tests & Work SamplesSamples

Signs vs SamplesSigns vs Samples

SignsSigns: indirect indicators of : indirect indicators of performance potentialperformance potential

SamplesSamples: direct indicators of : direct indicators of performance potentialperformance potential

Sign-Sample ContinuumSign-Sample Continuum

Probationary employmentProbationary employment ApprenticeshipsApprenticeships InternshipsInternships Mock-upsMock-ups Assessment centersAssessment centers Role-playing interviewsRole-playing interviews

Paper/pencil testsPaper/pencil tests

Sample

Sign

Pros & ConsPros & Cons

High validityHigh validity Low adverse impactLow adverse impact High face validityHigh face validity

Advantages

• More time consuming• Direct costs

Disadvantages

Step 1: Define Performance Step 1: Define Performance

Which performance dimensions?Which performance dimensions?• Refer to Performance Attributes MatrixRefer to Performance Attributes Matrix• Don’t focus only on easily simulated Don’t focus only on easily simulated

dimensionsdimensions

Performance Attributes MatrixPerformance Attributes Matrix

Attribute

App.

Blank Interview

Reference

Checks

Work

Sample

Knowledge

Test

Personality

Inventory

Ability to workaccurately

X X X X

Ability to workwell with customers

X X X X

Work-specificknowledge

X X X

Computer-operating skill

X X

Conscientiousness X X X

Step 1: Define Performance Step 1: Define Performance

Which performance dimensions?Which performance dimensions?• Refer to Performance Attributes MatrixRefer to Performance Attributes Matrix• Don’t focus only on easily simulated Don’t focus only on easily simulated

dimensionsdimensions Determine reasonable standards for Determine reasonable standards for

the test/samplethe test/sample What type of validation would you What type of validation would you

use?use?

Step 2: Creating SampleStep 2: Creating Sample

Key principlesKey principles• SpecificitySpecificity• Comprehensiveness (vs Narrowness)Comprehensiveness (vs Narrowness)• FidelityFidelity• PracticalityPracticality

Use Performance Attributes Matrix to:Use Performance Attributes Matrix to:• Ensure all KSAs are included in planEnsure all KSAs are included in plan• Weight all KSAs appropriatelyWeight all KSAs appropriately

Example: Example: Marketing SpecialistMarketing Specialist

Performing market analysisPerforming market analysis• Have candidates perform a market Have candidates perform a market

analysis for a particular product/marketanalysis for a particular product/market Developing appropriate marketing Developing appropriate marketing

materialsmaterials• Critique current materialsCritique current materials

Developing a marketing planDeveloping a marketing plan• Develop part of a planDevelop part of a plan

Tools for Successful Selection Employment Testing: General.

Documents

Transcript of Tools for Successful Selection Employment Testing: General.