HIGHLANDS AFFILIATE CONFERENCE

July 24, 2013

On-going HAB Research Review

Reliability of the HAB: test-retest and internal consistency

Construct validity: Wonderlic Test (general intelligence) and Employee Aptitude Survey (verbal comprehension, visual speed/accuracy, spatial, word fluency)

Accuracy of self-estimates of abilities versus actual HAB scores

Factor Analysis: examining the relationship among the work samples over time

Cluster and Discriminant Analysis: examining the relationship between ability profiles and job choice

What is Reliability?

Test-Retest Internal Consistency

Time 1

Time 2

Test score

correlationr12 = 1.0

TEST-RETEST RELIABILITYCorrelation Coefficient

Vocabularyr12 = .90

Work Sample Test-Retest Internal ConsistencyAlpha Split-Half

Classification 0.78 .81a ( .70b) .73a ( .70b)Concept Organization 0.76 0.57 0.55Design Memory 0.80 0.76 0.71Extravert-Introvert 0.86 0.8 0.79Generalist-Specialist (written) 0.81 0.88 0.85Generalist-Specialist (oral) 0.81 0.88 0.86Idea Productivity 0.80 n/a n/a Number Memory 0.84 0.83 0.83Observation 0.76 0.73 0.71Pitch Discrimination 0.86 0.89 0.82Rhythm Memory 0.80 0.75 0.73Spatial Relations Theory 0.88 0.88 0.84Spatial Relations Visualization 0.71 0.74 0.72Time Frame 0.76 0.93 0.88Tonal Memory 0.83 0.89 0.85Typing Speed 0.94 n/a n/aVerbal Memory 0.80 0.89 0.83Visual Speed (6 min.) 0.81 .99 (n=41c) .99 (n=41c)Visual Accuracy (6 min) 0.72 .90 .91 Vocabulary 0.90 0.96 0.93

All work samples meet or exceed the accepted standard for test-retest reliability (r12 ≥ .70)

General Cognitive Ability timed (12 minutes) 50 questions

General Intelligence: The level at which an individual learns, understands instructions and

solves problems. It provides quantitative insight into how easily individuals can be trained,

how well they can adjust and solve problems on the job, and how satisfied they are likely to

be with the demands of the job.

Correlations Between the Wonderlic and HAB Work Samples

n = 40-45

Intercorrelation Matrix

Technical Manual

2013 Intercorrelation Matrix

The relationship among the work samples has not changed over time

FACTOR ANALYSIS

Spatial Relations T. 0.77

Spatial Relations V. 0.71

Concept Organization 0.67

Design Memory 0.67

Classification 0.63

Observation 0.48

Tonal Memory 0.80

Pitch Discrimination 0.67

Rhythm Memory 0.67

Idea Productivity 0.75

Time Frame 0.68

Vocabulary 0.48

Number Memory 0.69

Verbal Memory 0.67

Observation 0.38

Design Memory 0.37

Generalist-Specialist 0.62

Vocabulary 0.53

Extravert-Introvert -0.67

Visual Accuracy 0.81

Visual Speed -0.70

The basic factor structure has not changed over time.

2013 Factors

Technical Manual (p. 27)

HAB 1-Verbal 4-Visual Speed 5-Space 8-WordWork Sample Comprehension & Accuracy Visualization Fluency

Visual Speed (5 min) -0.05 0.46** 0.27 -0.01Visual Speed (6 min) -0.06 0.38* 0.18 0.00Visual Accuracy (5 min) 0.06 0.07 0.07 -0.20Visual Accuracy (6 min) 0.06 0.10 0.04 -0.16Typing 0.28 0.18 0.17 0.26Generalist-Specialist (written) 0.15 0.18 -0.08 0.26Spatial Reasoning Theory 0.40** 0.33* 0.68 0.17Idea Productivity 0.21 -0.06 0.20 0.21Verbal Memory 0.18 0.20 0.24 -0.04Design Memory 0.20 0.27 0.34* -0.14Tonal Memory 0.18 0.25 0.54** 0.04Rhythm Memory 0.22 0.21 0.43** 0.06Pitch Discrimination 0.43** 0.24 0.43** 0.21Spatial Reasoning Visualization 0.47** 0.19 0.61** 0.15Observation 0.02 0.20 0.21 0.01Concept Organization 0.43** 0.44** 0.60** 0.13Number Memory -0.27 -0.06 -0.29 0.10Time Frame 0.04 -0.02 -0.23 0.319*Classification 0.40** 0.39* 0.54** 0.26Generalist-Specialist (oral) 0.04 0.12 -0.08 0.24Vocabulary 0.64** 0.18 0.44** 0.39**Introversion-Extraversion 0.07 0.15 -0.07 0.15

** correlation is significant at the 0.01 level (2-tailed).* correlation is significant at the 0.05 level (2-tailed).

Correlations Between Employee Aptitude Survey (EAS)

and HAB Work Samples

CONSTRUCT VALIDITY

HAB 1-Verbal 4-Visual Speed 5-Space 8-WordWork Sample Comprehension & Accuracy Visualization Fluency

Visual Speed (5 min) -0.05 0.46** 0.27 -0.01Visual Speed (6 min) -0.06 0.38* 0.18 0.00Visual Accuracy (5 min) 0.06 0.07 0.07 -0.20Visual Accuracy (6 min) 0.06 0.10 0.04 -0.16Typing 0.28 0.18 0.17 0.26Generalist-Specialist (written) 0.15 0.18 -0.08 0.26Spatial Reasoning Theory 0.40** 0.33* 0.68 0.17Idea Productivity 0.21 -0.06 0.20 0.21Verbal Memory 0.18 0.20 0.24 -0.04Design Memory 0.20 0.27 0.34* -0.14Tonal Memory 0.18 0.25 0.54** 0.04Rhythm Memory 0.22 0.21 0.43** 0.06Pitch Discrimination 0.43** 0.24 0.43** 0.21Spatial Reasoning Visualization 0.47** 0.19 0.61** 0.15Observation 0.02 0.20 0.21 0.01Concept Organization 0.43** 0.44** 0.60** 0.13Number Memory -0.27 -0.06 -0.29 0.10Time Frame 0.04 -0.02 -0.23 0.319*Classification 0.40** 0.39* 0.54** 0.26Generalist-Specialist (oral) 0.04 0.12 -0.08 0.24Vocabulary 0.64** 0.18 0.44** 0.39**Introversion-Extraversion 0.07 0.15 -0.07 0.15

** correlation is significant at the 0.01 level (2-tailed).* correlation is significant at the 0.05 level (2-tailed).

Correlations Between EAS Tests and HAB Work Samples

Work Sample Self-Ratingr14-number memory r6-verbal memory r7-design memory r11-spatial v r1-visual speed r16-classify

W1SP5A -0.33 -0.05 0.01 -0.23 -0.03 -0.28W1SP5B 0.08 0.02 -0.03 -0.18 -0.18 -0.18W1SP6A -0.36 -0.11 -0.05 -0.29 -0.10 -0.36W1SP6B 0.01 0.00 -0.02 -0.22 -0.19 -0.22W1AC5A -0.07 0.00 -0.42 -0.36 -0.17 -0.03W1AC5B 0.18 -0.08 -0.23 -0.16 -0.31 0.07W1AC6A -0.01 0.02 -0.41 -0.36 -0.17 -0.02W1AC6B 0.18 -0.06 -0.24 -0.16 -0.34 0.04W6VMA 0.16 0.32 0.07 0.08 0.07 0.02W6VMB -0.18 0.32 0.05 0.07 0.19 -0.14W7DMA -0.08 0.11 -0.07 -0.11 -0.16 -0.15W7DMB 0.03 0.06 0.13 0.14 0.14 -0.11

W11SRVA -0.09 0.29 0.16 0.21 0.14 0.24W11SRVB -0.06 0.18 0.23 0.32 0.22 0.10W14NMA 0.59 0.30 0.21 0.20 0.08 0.18W14NMB 0.18 0.16 0.17 0.13 0.14 0.00

W16CA -0.14 -0.02 0.00 0.02 0.01 0.10W16CB -0.12 0.12 0.16 -0.07 0.11 0.23

n = 39

Ability Self-Estimates

Work Sample Self-Ratingr14-number memory r6-verbal memory r7-design memory r11-spatial v r1-visual speed r16-classify

W1SP5A -0.33 -0.05 0.01 -0.23 -0.03 -0.28W1SP5B 0.08 0.02 -0.03 -0.18 -0.18 -0.18W1SP6A -0.36 -0.11 -0.05 -0.29 -0.10 -0.36W1SP6B 0.01 0.00 -0.02 -0.22 -0.19 -0.22W1AC5A -0.07 0.00 -0.42 -0.36 -0.17 -0.03W1AC5B 0.18 -0.08 -0.23 -0.16 -0.31 0.07W1AC6A -0.01 0.02 -0.41 -0.36 -0.17 -0.02W1AC6B 0.18 -0.06 -0.24 -0.16 -0.34 0.04W6VMA 0.16 0.32 0.07 0.08 0.07 0.02W6VMB -0.18 0.32 0.05 0.07 0.19 -0.14W7DMA -0.08 0.11 -0.07 -0.11 -0.16 -0.15W7DMB 0.03 0.06 0.13 0.14 0.14 -0.11

W11SRVA -0.09 0.29 0.16 0.21 0.14 0.24W11SRVB -0.06 0.18 0.23 0.32 0.22 0.10W14NMA 0.59 0.30 0.21 0.20 0.08 0.18W14NMB 0.18 0.16 0.17 0.13 0.14 0.00

W16CA -0.14 -0.02 0.00 0.02 0.01 0.10W16CB -0.12 0.12 0.16 -0.07 0.11 0.23

n = 39

Ability Self-Estimates

DISCRIMINANT ANALYSIS

Used to predict membership in two or more mutually exclusive groups. A set of discriminant functions (equations) is used to estimate the probability of membership for each group.

P1 = .65

P2 = .30

P3 = .50

P4 = .10

1Sales

2Engineering

3Law

4Education

Job Groups Discriminant Functions

Cluster AnalysisCluster Analysis

A cluster is a group of individuals with similar abilities.

Maximize the similarity of ability profiles within groups and minimize similarity between groups.