Tools for Successful Tools for Successful SelectionSelectionEmployment Testing: General

EMPLOYMENT TESTS LECTURE.. ROWENA R. MANANSALA1

Types of Employment Types of Employment TestsTests1. Ability Tests

◦Mental (Cognitive) Ability◦Psychomotor Ability◦Physical Ability◦Sensory Ability

2. Achievement and Skill Tests◦Job Knowledge◦Performance/Work Sample

Selecting Employment Selecting Employment TestsTests

Which KSAs are relevant to testing?Search sources of tests

◦General Tests: A Comprehensive Reference... Web-based information sources

◦Reviews Mental Measurement Yearbook

Useful article on using the MMY British Psychological Society Test Critiques

Evaluating Test ManualsEvaluating Test ManualsEvidence of reliabilityRelevant evidence of validity

◦Use of appropriate criteria◦Norms, based on appropriate

samplesEvidence of test fairnessPracticality

Tools for Successful Tools for Successful SelectionSelectionCognitive Ability (Intelligence) Tests

What is “Intelligence”?What is “Intelligence”?

And is it relevant to job performance??

““Successful” IntelligenceSuccessful” Intelligence

Analytical intelligence◦“Academic” intelligence

Practical Intelligence◦“Common sense” as well as social or

emotional intelligence; “street smarts”Creative Intelligence

Ability to achieve success in life and work; requires a balance among:

Analytical IntelligenceAnalytical Intelligence

Traditional aspects of intelligence measured by standard tests◦ Verbal and numerical ability, memory,

processing speedPredicts job performance in nearly all jobs

◦ More important as jobs become more complex◦ Probably due to:

Broader knowledge base Ability to learn on the job more quickly Ability to problem-solve

Practical IntelligencePractical IntelligenceAbility to deal with everyday life challenges

◦ E.g., computing lowest price/unit when grocery shopping, or interpreting a bus schedule

Social/emotional intelligence◦ Ability to understand people and relationships◦ Impression management skills◦ Ability to persuade others

Adaptability to different social situationsShown to add to predictive power of

analytical intelligence for sales and execsMay be culture-specific

Creative IntelligenceCreative Intelligence

Relies on a base of knowledge + ability to apply it to new situations or problems ◦Ability to generate novel high-quality

ideas◦Ability to recognize opportunities or

define challenges in new ways◦Ability to adjust to new situations

Measuring IntelligenceMeasuring Intelligence

Wide variety of measures of analytical intelligence/general mental ability◦ Wonderlic Personnel Test is most popular

Sternberg Multidimensional Abilities Test measures all 3 intelligences

Sternberg and Wagner’s test of Practical Managerial Intelligence: ◦ ability to write effective memos ◦ ability to motivate people ◦ knowledge of when to delegate ◦ ability to “read” people

Lingering Controversies Lingering Controversies about IQabout IQ

Is intelligence testing fair?Is intelligence—even when

measured more broadly—sufficient?

Do you want “bright” or “well trained” employees?

Fairness and Intelligence Fairness and Intelligence Testing: A DilemmaTesting: A DilemmaCA predicts performance in a

wide variety of jobs◦Measures knowledge◦Measures ability to learn, reason,

and solve new problems◦Few alternative predictors with

comparable utility for entry-level jobs

But are they “fair” to all groups?

True Validities of Predictors For Entry-Level Jobs

Ability Test Battery .53 Interview (structured) .44 Job Tryout .44 Biographical Inventory .37 Reference Checks .26 Personality Inventories ~.23 Experience Ratings .18 Interview (unstructured) .14 Academic Achievement .11 Interest Inventories .10

For Promotions Work Sample .54 Ability Test Battery .53 Peer Ratings .49 Experience Ratings .49 Job Knowledge Test .48 Assessment Center .43

Definitions of Test Definitions of Test “Fairness”“Fairness”

1. Differences between groups’ average test scores◦ ~1 SD difference for African-Americans◦ ~.75 SD for Latinos

But what do these differences mean?

• Groups differ in ability?• Test discriminates against one group?

Definitions of Test Definitions of Test “Fairness”“Fairness”2. Differential Validity: Group

differences in validity of test

Definitions of Test Definitions of Test “Fairness”“Fairness”2. Differential Validity: Group

differences in validity of test

White

Black

Test Scores

JobPerf

Definitions of Test Definitions of Test “Fairness”“Fairness”3. Test Bias:

Group differences in predicted job performance with same cutoff score

Definitions of Test Definitions of Test “Fairness”“Fairness”3. Test Bias: Group differences in

predicted job performance with same cutoff score

Black

Test Scores

JobPerf

White

Definitions of Test Definitions of Test “Fairness”“Fairness”3. Test Bias: Group differences in

predicted job performance with same cutoff score

Black

Test Scores

JobPerf

White

Typical CA Test SituationTypical CA Test SituationAdverse ImpactNo differential validityNo test bias

White

Black

Test Scores

JobPerf

Typical CA Test SituationTypical CA Test SituationAdverse ImpactNo differential validityNo test bias

White

Black

Test Scores

JobPerf

Minimizing Adverse Minimizing Adverse ImpactImpactRecruit high quality applicantsUse tests only when appropriateUse tests as one part of processUse to eliminate unqualified

(rather than to differentiate among most qualified)

Be sure test is valid (job-related)!!

Lingering Controversies Lingering Controversies about IQabout IQ

Is intelligence testing fair?Is intelligence—even when

measured more broadly—sufficient?

Do you want “bright” or “well trained” employees?

Why Intelligent People FailWhy Intelligent People FailLack of motivation Lack of impulse control Lack of perseverance Fear of failure Procrastination Inability to delay gratification Too little/too much self-confidence

Source: Sternberg (1986)

Lingering Controversies Lingering Controversies about IQabout IQ

Is intelligence testing fair?Is intelligence—even when

measured more broadly—sufficient?

Do you want “bright” or “well trained” employees?

Intelligence or Training?Intelligence or Training?

Is being smart a pre-requisite for learning?Is being smart sufficient for learning?

Tools for Successful Tools for Successful SelectionSelectionPersonality Testing

InvasivenessInvasivenessEmployer right to know vs. right to

privacyOver-emphasis on individual itemsReducing invasiveness:

◦Use only for job-related traits◦Use employment (not clinical) measures◦Use as part of overall system◦Explain purpose as matching not

screening

Extreme Items from MMPI

I have never indulged in any unusual sex practices.

I have never been in trouble because of my sex behavior.

My soul sometimes leaves my body.

Evil spirits possess me at times.

I sometimes feel like I’m going to pieces.

Often I feel as if there were a tight band around my head.

I have had no difficulty starting or holding my bowel movements.

Sample Items Modeled After Those on NEO-PI

I am generally not a worrier.

I would rather cooperate than compete.

I don’t mind bragging about what I’ve accomplished.

I’m pretty set in my ways of doing things.

I try to keep all my thoughts realistic and to avoid flights of fancy.

I have no sympathy for “street people.”

I prefer jobs that allow me to work alone without being interrupted by others.

I am a person who always gets the job done.

Is Personality Job-Is Personality Job-Relevant?Relevant?The “Big Five” Personality Traits

◦Conscientiousness◦Emotional Stability◦Extraversion◦Agreeableness◦<Openness to Experience>

Others may apply in particular jobs ◦e.g., customer service, safety,

attention to detail

Do Personality Tests Do Personality Tests Predict?Predict?Is the situation weak or powerful?Is the measure valid?Validities typically in low .20s

(when based on job analysis)

Faking & Personality TestsFaking & Personality TestsApplicants often distort their

answers to look good◦Some applicants more than others◦This can affect who is hired

Reducing faking:◦Use faking (lie) scales ◦Reduce motivation to fake◦Use less transparent questions

Social Desirability Scale Items

I sometimes drive faster than the speed limit.

My table manners are not quite as good at home as when I’m out in company.

I gossip a little at times.

Sometimes I vote for people I know very little about.

I do not like everyone I know.

I sometimes tell lies if I have to.

Fairness of Personality Fairness of Personality TestsTestsLittle evidence of racial

discriminationMen and women differ on many

scales◦but no evidence of “discrimination”

Watch out for adverse impact due to:◦religious beliefs◦disability

Tools for Successful Tools for Successful SelectionSelectionAssessing Deviance:Drug TestingIntegrity (Honesty) Testing

General RationaleGeneral RationaleEmployee deviance is a costly

problem◦$6 to 200 billion annual cost ◦33-75% of employees

Screening out Deviant Screening out Deviant EmployeesEmployeesSome are based on past behavior

◦analysis of drug metabolites in urine; past theft; reference and background checks

Others focus on likely future behavior◦attitudes toward theft, drug use;

personalityOnly a few focus on current behavior

◦impairment testing

Physiological Drug TestsPhysiological Drug TestsUrinalysis

◦Quick screening◦Confirmatory: Gas

Chromatography/Mass Spectrometry (GC/MS)

Hair analysisBlood analysisBreath analysis

What’s Measured (& for how What’s Measured (& for how long?)long?)

The “NIDA 5”:◦Cannabis◦Cocaine◦Amphetamines◦PCP◦Opiates

Other Forms of Drug TestsOther Forms of Drug TestsPaper-and-pencil integrity testsImpairment testing

◦based on person’s own “baseline” performance on a computer task

Types of Drug Testing Types of Drug Testing ProgramsProgramsPre-employment“For cause” testing

◦mandatory post-accident◦based on supervisor’s observations

Fitness for duty ◦daily (impairment testing)◦post-rehabilitation

Random

Types of Honesty/Integrity Types of Honesty/Integrity TestsTests“Clear Purpose” tests“Disguised Purpose” tests

◦Attitudes about deviance◦Personality

Polygraph/Voice Stress Analyzer

Clear Purpose TestClear Purpose Test - Honesty/Morality Items- Honesty/Morality Items

If you received an extra $1.00 in change when you purchased groceries, would you keep it?

If you saw a co-worker stealing, would you turn her in?

How many people have you known who have stolen valuables at work?

Most people cheat on their income tax.

Clear Purpose Test:Clear Purpose Test: - Drug-Related Items - Drug-Related ItemsI have never been a heavy

drinker. On occasion I have used

alcohol/drugs excessively If I thought I could get away with

it, I'd probably try cocaineHave you ever missed work

because you were too high/drunk?

Types of Honesty/Integrity Types of Honesty/Integrity TestsTests“Clear Purpose” tests“Disguised Purpose” tests

◦Attitudes about deviance◦Personality

Polygraph/Voice Stress Analyzer

Disguised Purpose TestDisguised Purpose Test - - Drug Attitude ItemsDrug Attitude ItemsA drug addict who steals to

support his habit should get the same penalty as a person who steals for profit.

Employers expect a certain amount of drug and alcohol use among their workers.

How many people have you known who drink or do drugs while working?

Disguised Purpose Test:Disguised Purpose Test: - - Personality ItemsPersonality ItemsI enjoy making people feel better.I don't care if others like the

things I do.I shouldn't do many of the things

I do.I have never been in trouble with

the law.Sometimes I enjoy going against

the rules.

Drug & Integrity Testing Drug & Integrity Testing IssuesIssuesLegalityAccuracy/Job relevanceFaking (integrity tests)Consequences of failingNegative reactions (invasion of

privacy/fairness)

Integrity Testing: A CautionIntegrity Testing: A CautionIntegrity tests assume cause of

deviance is bad employeesBut the situation may also be a

factor◦role models, group norms may “push”

people towards deviance ◦dissatisfaction may “pull” towards

devianceIntegrity tests don’t replace good

management

Tools for Successful Tools for Successful SelectionSelectionEmployment Testing: Performance Tests & Work Samples

Signs vs SamplesSigns vs SamplesSigns: indirect indicators of

performance potentialSamples: direct indicators of

performance potential

Sign-Sample ContinuumSign-Sample ContinuumProbationary employmentApprenticeshipsInternshipsMock-upsAssessment centersRole-playing interviews

Paper/pencil tests

Sample

Sign

Pros & ConsPros & Cons

High validityLow adverse impactHigh face validity

Advantages

• More time consuming• Direct costs

Disadvantages

Step 1: Define Step 1: Define Performance Performance Which performance dimensions?

◦Refer to Performance Attributes Matrix

◦Don’t focus only on easily simulated dimensions

Performance Attributes Performance Attributes MatrixMatrix

Attribute

App.

Blank Interview

Reference

Checks

Work

Sample

Knowledge

Test

Personality

Inventory

Ability to workaccurately

X X X X

Ability to workwell with customers

X X X X

Work-specificknowledge

X X X

Computer-operating skill

X X

Conscientiousness X X X

Step 1: Define Step 1: Define Performance Performance Which performance dimensions?

◦Refer to Performance Attributes Matrix

◦Don’t focus only on easily simulated dimensions

Determine reasonable standards for the test/sample

What type of validation would you use?

Step 2: Creating SampleStep 2: Creating SampleKey principles

◦Specificity◦Comprehensiveness (vs Narrowness)◦Fidelity◦Practicality

Use Performance Attributes Matrix to:◦Ensure all KSAs are included in plan◦Weight all KSAs appropriately

Example: Example: Marketing SpecialistMarketing SpecialistPerforming market analysis

◦Have candidates perform a market analysis for a particular product/market

Developing appropriate marketing materials◦Critique current materials

Developing a marketing plan◦Develop part of a plan