Post on 27-Jun-2020
Paul Barrett
Psytech International (UK) University of Auckland University of CanterburyFaculty of Business Department of Psychology
paul.barrett@psytech.com paul.barrett@auckland.ac.nz paul.barrett@canterbury.ac.nz
Web: www.pbarrett.net
Paul Barrett
Psytech International (UK) University of Auckland University of CanterburyFaculty of Business Department of Psychology
paul.barrett@psytech.com paul.barrett@auckland.ac.nz paul.barrett@canterbury.ac.nz
Web: www.pbarrett.net
Consumer Personality & Research 2005 Conference, September 20-24th, 2005, Dubrovnik
Basic Definitions
A profile is defined by Collins English Dictionary (1991,3rd edition) as: "a graph, "a graph, table, or list of scores representing the extent to table, or list of scores representing the extent to which a person, field, or object exhibits various which a person, field, or object exhibits various tested characteristics or tendencies"tested characteristics or tendencies".
Cronbach and Gleser (1953) introduced three terms to describe a profile:
ElevationElevation: the mean of all scores within a single profile.
Scatter /Variability Scatter /Variability : the square root of the sum of squares of a single profile's deviation scores about the Elevation for that profile. Essentially the standard deviation of scores within a profile, multiplied by the square root of the number of attributes constituting the profile.
Shape Shape : the residual information left in each score of a profile, after equating for the elevation and scatter indexes by subtracting out the elevation and dividing the resultant deviation score by the scatter value.
Basic Definitions
The Vector ProfileTwo Profiles - Raw scale scores
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
2
4
6
8
10
12
14
16
18
20
22
24
26
Sca
le S
core
s
Scale Scores Target Scale Scores
Elevation Subtracted profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-8
-6
-4
-2
0
2
4
6
8
10
12
Elevation Subtracted Target Elevation Subtracted
The Vector Profile
Scatter Normalised profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-0.8-0.7-0.6-0.5-0.4-0.3-0.2-0.10.00.10.20.30.40.50.60.70.8
Scatter Normalised Target Scatter Normed
The Vector Profile
-2 -1.5-1 -0.5
0 0.5
1 1.5
2
Frequency in a Working Day-2
-1.5-1
-0.50
0.511.5
2Preference for Work-Style
-0.5
-0.5
-0.4
-0.4
-0.3
-0.3
-0.2
-0.2
-0.1
-0.1
0
0
0.1
0.1
0.2
0.2
0.3
0.3
0.4
0.4
0.5
0.5
Attr
ibut
e W
eigh
ting
Attr
ibut
e W
eigh
ting
3D Profile: showing bipolar nonlinear categorical scoring
-4-3
-2-1
01
23
Job P
erform
ance
-4-3
-2-1
01
23
Preference for
-0.6
-0.6-0.4
-0.4-0.2
-0.20
00.2
0.20.4
0.40.6
0.6
Attr
ibut
e W
eigh
t
Attr
ibut
e W
eigh
t
3D Attribute Weight x Job Performance x Preference for Work-Type
Two Profiles - Raw scale scores
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
2
4
6
8
10
12
14
16
18
20
22
24
26
Sca
le S
core
s
Scale Scores Target Scale Scores
Hypothesised, “idealideal” profileSingle empirical profile – the “star performerstar performer”Homogenous empirical profile – the
construction of a single “homogenous group homogenous group profileprofile” from a group of individuals who are all adjudged to be homogenous with respect to some external criterion (such as their job, level within a company, gender, team position, brand purchase etc.)
Constructing the Target Vector Profile
Constructing the Target Vector Profile
Although the vast majority of simple vector homogenous group profiling applications use a single point-estimate as the target magnitude, a ““homogenous with respect to a criterionhomogenous with respect to a criterion”” group target profile attribute should also take into account the “variability” of each estimate…
PointPoint--estimate and attribute variabilityestimate and attribute variability
Constructing the Target Vector Profile
The mean and median point-estimates and their disparity.
The interquartile and full magnitude range.A 10% modal range around the medianThe reliability/reproducibility of an
attribute score
And a suitable algorithm to incorporate the profile attribute reliability/variability into the matching procedures!
Profile Matching
TLabelY
BLabelY
PF Low Score Desc High Score Desc
A Reserved Outgoing
C Temperamental Calm-Stable
E Accomodating Assertive
F Cautious Enthusiastic
G Expedient Conscientious
H Retiring Socially Bold
I Factual Intuitive
L Trusting Suspicious
M Practical Conceptual
N Direct Restrained
O Confident Self Doubting
Q1 Conventional Radical
Q2 Group-orientated Self-Sufficient
Q3 Informal Disciplined
Q4 Relaxed Tense-Driven
1 2 3 4 5 6 7 8 9 10
15FQ Job Match
ComparisonFunction
Ranked Respondentsin terms of nearness to Target Profile
Target Profile
Profile Matching
1st rule of profile matching11stst rule of profile matchingrule of profile matching
Never use a matching coefficient without at first examining exhaustively how it will function on random, typical, and specifically degraded profile data.
Profile Matching
Transformation SensitivityTransformation Sensitivity
Determine the sensitivity of a coefficient to differences between the target and comparison profile elevationelevation(level) and scatterscatter (variability) whilst preserving an almost identical profile shape.
Profile Matching
The Sampling DistributionThe Sampling Distribution
Determine the expected average size and frequency distribution for a coefficient, using two kinds of applicationapplication--specificspecific* magnitude scaled data:
Random Profile VectorsTypical Profile Vectors
* Using the actual magnitude ranges for variables which are to beconsidered typical for your application.
Profile Matching
Systematic Profile Systematic Profile DegredationDegredation
Determine the sensitivity of a coefficient to systematically graduated systematically graduated disparitydisparity between a target and comparison profile.
Profile Matching
0.990.990.99De-Scatter comparison
0.780.790.99De-Elevation comparison
0.780.350.99Raw profile comparison
ICC Model 3
ICC Model 2
Pearson r
Two Profiles - Raw scale scores
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
2
4
6
8
10
12
14
16
18
20
22
24
26
Scal
e S
core
s
Scale Scores Target Scale Scores
Elevation Subtracted profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-8
-6
-4
-2
0
2
4
6
8
10
12
Elevation Subtracted Target Elevation Subtracted
Scatter Normalised profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-0.8-0.7-0.6-0.5-0.4-0.3-0.2-0.10.00.10.20.30.40.50.60.70.8
Scatter Normalised Target Scatter Normed
Transformation SensitivityTransformation Sensitivity
Profile Matching
0.990.9997.0%De-Scatter comparison
0.640.9985.7%De-Elevation comparison
-0.201.070.9%Raw profile comparison
CattellProfile
SimilarityCongruence
NormalizedEuclidean Similarity
Two Profiles - Raw scale scores
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
2
4
6
8
10
12
14
16
18
20
22
24
26
Scal
e S
core
s
Scale Scores Target Scale Scores
Elevation Subtracted profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-8
-6
-4
-2
0
2
4
6
8
10
12
Elevation Subtracted Target Elevation Subtracted
Scatter Normalised profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-0.8-0.7-0.6-0.5-0.4-0.3-0.2-0.10.00.10.20.30.40.50.60.70.8
Scatter Normalised Target Scatter Normed
Transformation SensitivityTransformation Sensitivity
Profile Matching
The coefficient distribution analyzer
The Sampling DistributionsThe Sampling Distributions
Profile Matching
Always work with the data that represent the profiles being compared. Transforming to rank values or standardized scores removes important discrepancy information.
2nd rule of profile matching22ndnd rule of profile matchingrule of profile matching
Profile Matching
www.pbarrett.net/statistics_corner.htmDocument = Euclidean Distance: Raw, Normalised, and Double-Scaled Coefficients.
Double Scaled Distance (DSD)
Profile Matching
A distance (or similarity) coefficient which is easily interpretable.
A fixed 0-1 metric achieved using linear transformations of discrepancy data.
Is comparable in meaning across all possible varying variable-range magnitudes.
Double Scaled Distance (DSD)
Profile Matching
DoubleDouble--Scaled Euclidean DistanceScaled Euclidean Distance
Step 1: Determine the maximum possible squared discrepancy for each variable comparison using the minimum and maximum possible values for this variable. Call these values md. Each variable will possess a minimum and maximum so the md for each variable is just:
mdi = (Max for variable i – Min for variable i)2
Profile MatchingStep 2. Compute a squared discrepancy, divide it by the maximum possible discrepancy for that particular comparison, then take the square root of the sum to produce the single-scaled variable Euclidean distance.
21 2
11
( )vi i
i i
p pdmd=
⎛ ⎞−= ⎜ ⎟
⎝ ⎠∑
where p1i = the attribute i value for person 1p2i = the attribute i value for person 2
d1 = the “scaled variable” Euclidean distancemdi = the maximum possible squared discrepancy
per variable i of v variables.
Profile Matching
Compute the scaled value from step 2 by dividing d1 by , where v = the number of variables.
v
21 2
12
( )vi i
i i
p pmd
dv
=
⎛ ⎞−⎜ ⎟⎝ ⎠=
∑
Profile Matching
When you can’t find a existing coefficient that appears to work the way you want it to – design a new one that does exactly what you want it to – and innovateinnovate.
3rd rule of profile matching33rdrd rule of profile matchingrule of profile matching
Profile Matching
www.pbarrett.net/factor_similarity.htmwww.pbarrett.net/factor_similarity.htm
Kernel Smoothed Distance (KSD)
More details can be found in the manual to the Orthosim_2Orthosim_2 Congruential Coordinate Similarity Program.
Profile Matching
A similarity coefficient which is easily interpretable, has a fixed 0-100% metric, and is comparable in meaning across all possible varying variable-range magnitudes.
I want to be able to calibratecalibrate this coefficient to give me values which best represent what I see visually when looking at profiles, and where I can adjust its characteristics to be more or less sensitive to observed profile discrepancies (in a false-positive/negative cost-benefit scenario analysis)
Profile Matching
The Standard Normal probability density function (the kernel kernel function) is:
( )222
where the population standard deviation = 1.0
the population mean = 0.0
the value of pi (3.141593)
euler's e (2.718282
12
= =
= e = )
x
ordz eμσ
σ πσμπ
⎡ ⎤−⎢ ⎥−⎢ ⎥⎣ ⎦=
Profile Matching: KSD with SD = 10
Profile Matching: KSD with SD = 30
Profile Matching
So, assume we have a target profiler value of 50, with an observed value (x) of 50, the ordinate expression for this difference is computed as:
( )22
50 502 10
where the population standard deviation = 10.0
the population mean (our target value) = 50.0
the value of pi (3.141593)
1 0.039894210 2
= =
=
ordz eπ
σμπ
⎡ ⎤−⎢ ⎥−⎢ ⎥⋅⎣ ⎦= =
an observed preference value (= 50 here also) = x
Profile Matching
( )
( )( )2
22
standard devation unit
person profile attribute value
target profile attribut
w
e v
here
alue
1 100 22
p t
KSD e
pt
σ πσσ π
σ
⎡ ⎤−⎢ ⎥−⎢ ⎥⎣ ⎦
⎡ ⎤⎢ ⎥= ⋅ ⋅ ⋅⎢ ⎥⎢ ⎥⎣ ⎦
===
Profile Matching
97.0%99.99%99.99%De-Scatter comparison
85.7%55.6%86.5%De-Elevation comparison
70.9%4.9%42.9%Raw profile comparison
1-DSDKSD with
SD=2KSD with
SD = 5
Two Profiles - Raw scale scores
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
2
4
6
8
10
12
14
16
18
20
22
24
26
Scal
e S
core
s
Scale Scores Target Scale Scores
Elevation Subtracted profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-8
-6
-4
-2
0
2
4
6
8
10
12
Elevation Subtracted Target Elevation Subtracted
Scatter Normalised profiles
Scal
e 1
Scal
e 2
Scal
e 3
Scal
e 4
Scal
e 5
Scal
e 6
Scal
e 7
Scal
e 8
Scal
e 9
Scal
e 10
Scal
e 11
Scal
e 12
-0.8-0.7-0.6-0.5-0.4-0.3-0.2-0.10.00.10.20.30.40.50.60.70.8
Scatter Normalised Target Scatter Normed
Transformation SensitivityTransformation Sensitivity
Profile Matching
Essentially, the KSD coefficient is a “designerdesigner--mediatedmediated” coefficient. That is, its value range is constructed or calibrated calibrated to suit the particular decision-requirements and error-costs defined for a specific application.
2-sten difference
S 1 S 2 S 3 S 4 S 5 S 61
2
3
4
5
6
7
8
9
10
11
Scale Scores Target Scale Scores
Cattell rp = 0.28Pearson r = 1.001-Alienation = 0.85Intraclass-2 = 0.721-DSD = 77.8%KSD (sd=1.67) = 48.7%
Systematic Profile Systematic Profile DegredationDegredation
All scales exactly the same - except the first (2 vs 10)
S 1 S 2 S 3 S 4 S 5 S 61
2
3
4
5
6
7
8
9
10
11
Scale Scores Target Scale Scores
Cattell rp = – 0.20Pearson r = 0.291-Alienation = 0.47Intraclass-2 = 0.271-DSD = 63.7%KSD (sd=1.67) = 83.7%
Systematic Profile Systematic Profile DegredationDegredation
All scores exactly the same - except the first (2 vs 10)
S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 S 100
2
4
6
8
10
12
Scale Scores Target Scale Scores
Cattell rp = 0.08Pearson r = 0.521-Alienation = 0.53Intraclass-2 = 0.501-DSD = 71.9%KSD (sd=1.67) = 90.0%
Systematic Profile Systematic Profile DegredationDegredation
All scores exactly the same - except for first (2 vs 10)
S 1 S 2 S 3 S 4 S 5 S 6 S 7 S 8 S 9 S 10 S 11 S 12 S 13 S 14 S 150
1
2
3
4
5
6
7
8
9
10
11
Scale Scores Target Scale Scores
Cattell rp = 0.28Pearson r = 0.631-Alienation = 0.64Intraclass-2 = 0.631-DSD = 77.0%KSD (sd=1.67) = 93.3%
Systematic Profile Systematic Profile DegredationDegredation
Classifier Predictive-Function Modelling
ClassifierFunction
The goal is to construct a classifier or some function of a set of variables which predict a predefinedpredefined criterion group membership
Inductive Homogeneity Analysis
Inductive ClassifierFunction
The goal is to discoverdiscoverhomogenous groups or clusters within a set of individuals or objects
What if we let the respondent define their own profile as a “conceptual wholeconceptual whole”or “attribute mapattribute map”, rather than the investigator reconstruct this post-assessment?
Graphical ProfilingGraphical ProfilingGraphical Profiling
Construct an assessment which consists of displaying a single, precisely semantically defined “construct” to a respondent, who then performs a rating of an object or person against it.
Allow a respondent to rate attributes on a magnitude scale, with immediate “visual relativityvisual relativity” maintained with all other previously or to-be-rated attributes.
Rate in 1 or 2 dimensions simultaneously.Marked response graduation is optional.Relevant only to non-cognitive preference,
self-report, or 360-style peer-review ratings.Only realistically possible via
computer/PDA-based administration.
Graphical ProfilingGraphical ProfilingGraphical Profiling
Graphical Profiling – 1-D Personality
Using the free 5-factor personality model item-bank at the International Personality Item Pool
http://ipip.ori.org/ipip/
Specifically using 1010 facets taken from the AB5C 45-facet personality questionnaire as “typical” personality test scales, spanning about 106 questionnaire items106 questionnaire items in total …http://ipip.ori.org/ipip/newAB5CTable.htm
Happiness
Impulse-Control
CalmnessFactor 4: Emotional StabilityOrderliness
Organization
Purposefulness
EfficiencyFactor 3: ConscientiousnessTalkativeness
Leadership
FriendlinessFactor 1: ExtraversionFacetFactor
Graphical Profiling – 1-D Personality
Extract the meaning of all the items in a scale, and compose a single rating statement that seems to best encompass the meaning not only of the scale name, but that embodied within the items.
Graphical Profiling – 1-D Personality
Construct the meaning of Construct the meaning of the construct to be ratedthe construct to be rated
Dislike talking about myself.X212Speak softly.H548Demand to be the centre of interest.H769Make a lot of noise.H531Never stop talking.H536Like to attract attention.H1150Make myself the centre of attention.H535Speak loudly.H527Talk too much.H1138Do most of the talking.H4
single item rewords? ... facet = TalkativeTalkative
Graphical Profiling – 1-D Personality
The Rating Statement: TalkativeTalkative“I have no problem in talking about almost anything. In fact, I find it hard to stop sometimes, especially if I've become the centre of attention! Frankly, I just like talking with people”.
Graphical Profiling – 1-D Personality
The Personality Profiler
Graphical Profiling – 1-D Personality
36.734.1Leadership
39.335.7Happiness
33.737.2Impulse-Control
33.536.4Calmness
32.532.9Orderliness
42.546.7Organization
44.441.5Purposefulness
41.537.7Efficiency
33.327.1Talkativeness
39.335.3Friendliness
Profiler Scores
AB5C Scales
Profiler -vs-
questionnairescale-score
MeansMeans
N=99 cases
8.86.9Leadership
6.76.9Happiness
11.36.9Impulse-Control
9.36.1Calmness
10.58.3Orderliness
10.87.3Organization
9.67.7Purposefulness
9.67.7Efficiency
9.97.3Talkativeness
8.16.8Friendliness
Profiler Scores
AB5C Scales
Profiler -vs-
questionnairescale-score
Std. Std. DevnsDevns..
N=99 cases
0.590.54Leadership
0.690.64Happiness
0.320.280.28Impulse-Control
0.620.54Calmness
0.800.75Orderliness
0.650.60Organization
0.610.56Purposefulness
0.680.63Efficiency
0.750.70Talkativeness
0.770.71Friendliness
Disattenuated
Actual Correlation
Profiler -vs-
questionnairescale-scores
CorrelationsCorrelations
N=99 cases
Predicting AB5C Scale "Leadership" by Profiler Score Plotting Prediction Residuals by Profiler Score
10 15 20 25 30 35 40 45 50 55
Personality Profiler Score
-20
-15
-10
-5
0
5
10
15
Raw
Res
idua
lsGraphical Profiling – 1-D Personality
Completion Times and Preference
Questionnaire: about 2020 minutesProfiler: about 55 minutes
•• 93%93% of individuals preferred completing the Profiler assessment over the paper and pencil Questionnaire.•• 87%87% felt they were able to describe themselves adequately using it.
Graphical Profiling – 1-D Personality
Graphical Profiling – 2-D Work Preferences
Graphical Profiling – 2-D Work Preferences
Graphical Profiling – 2-D Work Preferences
Graphical Profiling – 2-D Work Preferences
The movie
Graphical Profiling – 2-D Work Preferences
( ) ( ) ( )( ) ( ) ( )
1 1 2 2
1
100 0.25 0.25 0.5
10 30 60 600 50 0.25 80 0.25 80 0.5 80%
jki i j i k i j i k ij ik
jk
Match a a a a f f
Match
⎡ ⎤= − − ⋅ + − ⋅ + − ⋅⎣ ⎦⎡ ⎤= − − ⋅ + − ⋅ + − ⋅ =⎣ ⎦
The original Mariner7 person-to-person match coefficientP
refe
renc
e Fo
r
Frequency20
5080
20
50
80
30
70
100
Ambiguity p1
Ambiguity p2
Clarity p1
Clarity p2
40 6030 70
euclideandistance
euclideandistance
Using 2131 real-data cases, I compared every individual’s profile with every other individual (2.2million match coefficients) – and looked at the expected-value frequency distributions …
Mariner 7 scaled, weighted, unsigned simple discrepancy coefficient
Weighted normalised euclidean expressed as % similarity
Pref
eren
ce F
or
Frequency20
5080
20
50
80
30
70
100
Ambiguity p1
Ambiguity p2
Clarity p1
Clarity p2
40 6030 70
euclideandistance
euclideandistance
k1
k2
k1-freq
k2-freq
Weighted, normalised, doubly-degraded euclidean expressed as % similarity
Coefficient Density as a function of Profile Elements
Graphical Profiling – 2-D Work Preferences
Let the specific application requirements determine the optimal profile matching index.Algorithmic profiling solutions are always going to be optimal – but will require calibration and exhaustive scenario testing.Graphical Profiler assessment is “intriguing” but remains a relatively underdeveloped assessment methodology.