ROI in HRM: Validity of Selection Procedures

Nicole BlakeEdgar NolascoDeena Desai

Employee Selection:Will Intelligence and Conscientiousness Do the

Job?

Orlando Behling

Key Terms

•General Intelligence “g”•Big Five•Meta-Analysis – combining research

results from existing studies that meet a specific criteria for to generate a general conclusion.

Summary

•Measuring general intelligence “g” in addition to the Big Five provides a stronger predictor of job performance in past jobs.

•Group Hypothesis: What have we learned since Behling?

Article

A Meta-Analytic Investigation of Conscientiousness in the Prediction of Job

Performance:

Examining the Intercorrelations and the Incremental Validity of Narrow Traits

By: Nicole M. Dudley, Karin A. Orvis, Justin E. Lebiecki and Jose M. Cotina

George Mason University

Nicole Blake

Key Terms

• Global Conscientiousness: A broad characteristic of being conscientious (careful, thorough).

• Four Narrow traits of conscientiousness:▫Achievement▫Order▫Dependability▫Cautiousness

Nicole Blake

Introduction:

•The Big Five

•Conscientiousness: the most important & the most valid personality predictor of job performance.

•Multiple meta-analyses from previous research examined and used in the current meta analyses.

•Goal of this studyNicole Blake

Hypothesis

•1) The narrow traits of conscientiousness are a better predictor than global conscientiousness alone.

•2) Narrow traits can be a useful predictor for specific occupations but not all occupations.

Nicole Blake

Method

•Validity coefficients and intercorrelations among the 5 predictors were obtained from previous studies.

•Criteria for inclusion

•2 moderators: type of performance criterion & occupational type.

•Hierarchical regression analysis

Nicole Blake

Results 1: Incremental Validity for Job Performance Criteria

Nicole Blake

Results 2: Overall Job Performance by Occupational Type

Nicole Blake

Resume Screening & Personality Testing

Recruiters Inferences of Applicant Personality Based on Resume Screening:

Do Paper People have a Personality?By: Michael S. Cole, Hubert S. Feild, William F.

Giles, Stanley G. Harris.

•Excludes potential candidates.

•Considerable error present in recruiters inferences.

Nicole Blake

Take Home Message

•Focusing on narrow traits is a better predictor of job performance than focusing on broad traits.

• Managers should consider narrow traits when the goal is to predict contextual performance or to predict overall performance for specific occupations ▫Saves money in the long run by hiring the

best fit applicants. Nicole Blake

Article

Situational Judgment Tests, ResponseInstructions, and Validity:

A Meta-Analysis

By: Michael A. McDaniel, Nathan S. Hartman, Deborah L. Whetzel and W. Lee Grubb III

Edgar Nolasco

Key Terms

• SJT- Situational Judgment Test▫Behavioral tendency ▫Knowledge

• Cognitive ability or g – It is a set of abilities, skills or processes that are part of nearly every human action.

• Big Five– Extraversion, Emotional Stability, Agreeableness, Conscientiousness and Openness.

 Edgar Nolasco

Introduction/Study• Two things being tested through this study

• Using a Meta-Analysis▫ The psychometric artifact method was used in the study

• Data set from the McDaniel (2001) study

• Previous research shows that SJTs predict performance because they measure job knowledge

Edgar Nolasco

Example of an SJTYou have just attended a training course delivered by your manager. While talking to the others privately after the course, several of them were critical of how your manager delivered the course. Your manager is unaware of this feedback and you felt the course delivery was adequate.Rate the effectiveness of each of the actions below on the following scale:1 = Very Ineffective, 2 = Ineffective, 3 = Effective, 4 = Very Effective.a. Tell your manager that you personally think the course went well and don’t mention the feedback from the other delegates.

b. Tell the other delegates that they should communicate their feedback to your manager if they feel strongly about it.

c. Tell your manager about the feedback from the other delegates, without mentioning them by name.

d. Don’t mention any feedback about the course to your manager.

Edgar Nolasco

Method

• Study uses:▫Data from McDaniel (2001)▫The psychometric artifact distribution meta-analysis

method (Hunter and Schmidt, 1990, 2004)• Criteria:

▫Employees or applicants only▫Paper-Pencil format▫Priority to supervisor ratings over job performance▫Hierarchical level, # of employees supervised, years

of management experience, and salary omitted

Edgar Nolasco

Results : Criterion-Related and Incremental validity

Edgar Nolasco

Take Home Message

• When looking for the candidate both behavior and knowledge based SJTs should be asked.▫SJTs that use behavioral instruction are easier

to manipulate because respondents know what to say.

• SJTs are unique: either serve assessment of typical or maximal performance

• SJTs offer job relevance, ease of use, flexible, are validated and are fair Edgar Nolasco

Further Evidence for the Validity of Assessment Center

Dimensions:

A Meta-Analysis of the incremental Criterion-Related Validity of Dimension

Ratings

John P. Meriac, Brian J. Hoffman David J. Woehr and Matthew S. Fleisher

Deena Desai

Key Terms

• Assessment Centers (AC) – objective method of measurement of skills, competencies and characteristics for a specific job

• Dimensions – Characteristics beyond the Big Five and cognitive ability

• Hierarchical Regression – adds factors at each stage

• Forward selection – testing individual variablesDeena Desai

Introduction

Job Performance

Personality

AC’s

Cognitive

ability

•Assessment Centers (AC’s)

•Weak construct validity

•Independent influence on job performance

Deena Desai

Hypothesis

How much if any, incremental variance in job performance do AC dimension ratings

account for over and above that accounted for by cognitive ability and

personality?

Deena Desai

Method

•Perform a Meta-Analysis

▫Retrieve studies that meet criteria▫Extract AC dimensions▫Input in one of the seven dimension

categories identified by Arthur et. al.▫Forward selection

Deena Desai

Sample Breakdown

Deena Desai

Sample Breakdown

Deena Desai

Results

Deena Desai

Results• AC dimensions have an incremental variance in

job performance above that of personality and cognitive ability

▫ The individual characteristics in addition to personality and cognitive ability have a significant relationship with job performance

• Cognitive ability + the big five + organization and planning = most significant impact on job performance

Deena Desai

Take Home Message

•AC’s are costly therefore should not be used for all positions

•AC’s must utilize up to date measures to be effective

•Further research is required to obtain alternative methods that effectively measure specific characteristics

Deena Desai

Group Take Home

•The research demonstrates that to date:▫Specific measurements of narrow traits

beyond personality and cognitive ability are the strongest predictors of job performance.

▫Assessing general characteristics for a variety of jobs increases the risk of litigation.