15fq+Review

&SelectionDevelopment Review

Volume 19 No. 2 April 2003ISSN 0963-2638

&SelectionDevelopment Review

P U B L I S H E D B Y T H E B R I T I S H P S Y C H O L O G I C A L S O C I E T Y

Downlo

aded

from

www.ps

yasia

.com

SDRC O N T E N T SWELCOME TO THE April Edition of SDR.We hope thisedition finds you all well and enjoying the longerdays of Spring.

Our first contribution to the review this monthcomes from Giles Burch. Giles explores some ofthe key competencies that organisations currentlylook for when selecting candidates. In particular,he looks at the areas of Creativity, Psychoticismand Emotional Intelligence and the links betweenthem. Secondly, Graham Tyler presents us with areview of Psytech International’s 15FQ+ question-naire. Graham’s paper is very comprehensive andcovers a number of aspects of the 15FQ+including its development and issues of reliabilityand validity, and comparisons to the original 15FQand Cattells 16PF.

With the increase in computer and internetaccess, the issues surrounding selection and devel-opment online become more and more pressing.As such,our final article written by Ruth Price andFiona Patterson from the Work Psychology Groupat City University is very timely.They have under-taken a study to explore the psychological effectsof receiving, completing and submitting on-lineapplication forms. In particular, five psychologicalissues were examined; privacy, support, feedback,dehumanisation and self-selection. Four usabilityissues are discussed along with the implicationsfor organisations and some practical recommen-dations for the future design of such forms.

We are also pleased to include two responses topast articles. Firstly, Hugh McCredie responds to apoint made by Michael Gray (February, 2003) onthe on-going issue of performance and person-ality. Secondly, K.W. Young takes up the debatesurrounding structured interviews in relation tothe article by Paul Taylor and Bruce Small(February, 2003). Finally, Christopher Ridgewayraises the question of whether there is a need fora verification procedure when using projectivemethods in occupational settings.

As John mentioned in the last edition, we stillstruggle each month to get enough articles.We arevery committed to making every effort possible toproduce each edition for you but we really doneed your help in doing this – no articles orpapers would ultimately mean the demise of SDRwhich would be a sad situation.Many thanks go,asalways, to those who have contributed to thisedition.We welcome your comments on any of thepapers featured here and look forward to hearingfrom you soon.

Philippa HainOn behalf of the Editorial team

Creativity, psycoticism and emotional 3intelligence: A conundrum for selection?Giles St. John BurchA review of the 15FQ+ Personality 7QuestionnaireGraham TylerOnline application forms: Psychological 12 impact on applicants and implications forrecruitersRuth E. Price & Fiona PattersonPersonality and Performance: 20Continuing the debateHugh McCredieStructured interviews: A response 21K.W. YoungQuestion format in the structured 22employment interview – statisticsappearing in the meta-analysesPaul Taylor & Bruce SmallProtective measures in occupational 23settings – the need for a verification procedureChristopher C. Ridgeway

Editorial

Selection & Development Review editorial teamDr John Boddy16 Tarrws Close, Wenvoe, Cardiff CF5 6BT. Tel: 029 2059 9233 Fax: 029 2059 7399E-mail: [email protected] Duff, Stephan Lucks & Ceri RoderickPearn Kandola Occupational Psychologists, 76 Banbury Road, Oxford OX2 6JT. Tel: 01865 516202 Fax: 01865 510182E-mail: [email protected] Hain98 Plymouth Road, Penarth CF64 5DL. Tel: 07816 919857 E-mail: [email protected] Tuvia MelamedThe Quo Group Ltd, 2 The Oaks, Clews Road, Redditch B98 7ST. Tel: 01527 540057 Fax: 01527 547527 Mobile: 0777 5643128.E-mail: [email protected] Editors: Dr S. Blinkhorn; Professor V. Dulewicz; Professor N. AndersonPublished six times a year by The British Psychological Society, St Andrews House, 48 Princess Rd East, Leicester LE1 7DR at £37(US $50 overseas) a year. Tel: 0116 254 9568. Fax: 0116 247 0787.E-mail: [email protected]. ISSN 0963-2638SDR is a review of current issues in selection, assessment anddevelopment. It is not intended to be an academic journal andwhile papers are subject to editorial review not all are referred toindependent referees. Any material intended for publicationshould be sent by e-mail if possible. The guideline maximumlength is 2000 words.The views expressed in articles do not necessarily representthose of the Editorial team or The British Psychological Society.

2 Selection & Development Review Vol. 19 No. 2 April 2003

Downlo

aded

from

www.ps

yasia

.com

Selection & Development Review Vol. 19 No. 2 April 2003 3

CONSCIENTIOUSNESS, ONE OF the ‘Big Five’ factors ofpersonality, characterised by ambition, discipline,a sense of competence, energy and perseverance,is widely regarded as a predictor of ‘effectiveness’for a majority of occupational roles (see Hough &Ones, 2001); whilst Agreeableness, characterisedby a concern to help others, altruism, trust andco-operation, is recognised as a predictor whenthe role involves interaction with other people(e.g. Mount, Barrick & Stewart, 1998). Morerecently, Witt, Barrick, Burke and Mount (2002)have demonstrated an interaction effect betweenConscientiousness and Agreeableness. This is intheoretical keeping with the notion of EmotionalIntelligence (EI) being a major determinant ofmanagerial and leadership effectiveness. Forexample, Higgs and Dulewicz (1999) show howAgreeableness and Conscientiousness maps ontotheir model of EI, and state how their research hasdemonstrated that ‘EI people (i.e. those with highemotional intelligence) tend to be agreeable’(p.30). Thus, in any selection process, thiscombined construct of Conscientiousness andAgreeableness appears to be of key importancewhen making predictions regarding potential oreffectiveness.

Whether or not the notion of ‘EmotionalIntelligence’ appeals, the evidence does supportthe predictive validity of these particular person-ality constructs of Agreeableness andConscientiousness, which appear to be closelyaligned with key elements of EmotionalIntelligence – this is not the issue for discussionhere. The point is that organisations are increas-ingly seeking to ensure that employees, particu-larly managers, are selected and developed with‘emotional intelligence’ in mind.

However, this is not all that the selection teamare concerned with, there are of course other keymanagerial competencies against which selectionpredictions are made, one of which is often‘creativity and innovation’. Organisations may alsowant their managers to be creative and innovative,and seek to make appropriate predictions relatingto individual creativity. This is evidenced by theincreasing number of tools available for selectingfor individual creativity, for example: theInnovation Potential Indicator (IPI; Patterson,1999) and the Team Selection Inventory (TSI;Anderson & Burch, in press; Burch et al., 2002),a measure of an individual’s preferred teamworking climate for innovation.Not only do organ-isation’s select against the more creative andemotionally intelligent competencies, they arealso investing much time and money developingcreativity and emotional intelligence competence.Thus, it appears that creativity and emotional intel-ligence are both high on the organisationalagenda, within the context of the predictivistselection paradigm – but alas,at this point a poten-tial paradox becomes apparent!

I have identified how Conscientiousness andAgreeableness are important constructs in theprediction of job success, and I have highlightedthe importance of individual creativity to theorganisation. At face value we may wish to ques-tion whether an emotionally intelligent employeecan also be a creative one, after all, there is muchanecdotal evidence for the ‘eccentric’ individual

SDRCreativity, psychoticismand emotional intelligence:A conundrum for selection

Giles St. John BurchInstitute of Psychiatry,King’s College London

Downlo

aded

from

www.ps

yasia

.com


locked up in R & D, with an apparent lack of EIcompetence. However, creativity does appear tobe a key element in the manager’s tool kit,suggesting that these two factors may go together– and indeed, this is what the organisation wantsand requires in many cases.

So far so good: creative managers who areconscientious and agreeable!

However, the personality literature may suggestotherwise. It has long been advocated thatcreativity and Psychoticism are closely related (e.g.Eysenck, 1995), with Psychoticism, a factor ofEysenck’s model of personality (see Eysenck,1992),reflecting a proneness to psychosis in the ‘normal’population.Such perspective runs concordant withthe widely held (folk psychological) belief thatcreativity is related to ‘madness’ where ‘…schizo-phrenia bears a remarkable resemblance to muchof the most sophisticated art, literature,and thoughtof the twentieth century, the epoch of modernism’(Sass, 1992, p.16). This correlation betweenPsychoticism and creativity has been noted in theresearch on several occasions (e.g. Woody &Claridge, 1977), although not consistently (e.g.Kline & Cooper, 1985; McCrae, 1987).Another keyfinding from the personality literature, germane tothe current discussion, is that Psychoticism is nega-tively related to both Conscientiousness andAgreeableness. This has been consistently demon-strated in the literature, where it has been shownthat Psychoticism negatively correlates withConscientiousness and Agreeableness (e.g. McCrae,1987) and (negatively) cross-loads onto these two‘Big Five’ factors (e.g. Zuckerman et al., 1993).

From these findings,whilst it would be incorrectto form any definitive conclusions, it does howeverraise an interesting question of whether, when weare selecting for Conscientious and Agreeableness,we are discriminating against those who are high inPsychoticism, with an associated consequence offailing to recruit those with the most potential forcreativity – or indeed vice versa,when selecting themost creative individuals, we exclude the most EIcompetent.Thus, this paradox – where the organi-sation desires a and also desires b, yet in opting for

a they are,by (possible) definition,rejecting b,or ofcourse, vice versa, in choosing b they are rejectinga – is the genesis of our conundrum.

This may be a real or an imagined paradox,and itis unlikely that it presents too much angst to thosewho use models of personality for the purposes ofselection and development. However, it may becomforting to find a solution to this paradox,havingraised it! In order to find such a solution, it is neces-sary to return to the personality literature.

The term Psychoticism is often used synony-mously with the term schizotypy, which is used todescribe ‘normal’ personality on a continuumbetween ‘normality’ and schizophrenia. However,recent research has shown that schizotypy iscomprised of at least four separate factors:(1) positive-schizotypy (reflecting the positivesymptomatology of schizophrenia); (2) negative-schizotypy (reflecting the negative symptom ofschizotype); (3) disorganised schizotypy/socialanxiety (reflecting ‘distractibility, difficulties withspeech and attentional difficulties’; Mason,Claridge & Williams, 1997, p.33); and (4) andasocial schizotypy (reflecting anti-social, tough-minded behaviour). It is with asocial-schizotypythat Psychoticism is most closely aligned, giventhat Psychoticism is particularly characterised byhostility, lack of empathy, lack of sensitivity andlack of feelings of guilt (Eysenck & Eysenck,1991).Thus, Psychoticism is only one syndrome of an all-encompassing concept of schizotypy.

Recently, my own research at King’s CollegeLondon,demonstrated that creativity (as measuredby Wallach and Kogan’s (1965) tests of divergentthinking) failed to correlate with either asocial-schizotypy or positive-schizotypy (Burch et al.,submitted). However, subsequent factor analysison this data, revealed that Openness toExperience (another of the ‘Big Five’, charac-terised by an openness to new and novel ideas,originality, imagination and intellectual curiosity)cross-loaded onto two factors, one reflecting whatwe labelled Anxious Positive-Schizotypy1 and onereflecting Creativity2, whilst failing to load ontoanother factor that we labelled Anti-Sociality

1 Anxious Positive-Schizotypy primarily comprised of the following loadings: positive-schizotypy, trait anxiety, neuroticism and openness.2 Creativity was comprised of the following loadings: creative personality, divergent thinking, IQ and openness.Dow

nload

ed fr

om w

ww.psya

sia.co

m

(comprising positive loadings of Psychoticism,Implusive Non-conformity (another scale ofasocial-schizotypy), and negative loadings ofConscientiousness and Agreeableness).Thus, thesefindings suggest that it is not ‘creativity’per se thatis related to schizotypy, but rather the essentialelements of Openness to Experience that corre-late with both creativity and schizotypy, and, mostimportantly, that this relationship is with positive-schizotypy rather than asocial-schizotypy (orPsychoticism).

The key point here then,is that creativity appearsto be linked more closely with positive-schizotypythan asocial-schizotypy, which is probably moreconsistent with Sass’s earlier quote regarding schiz-ophrenia and creativity. In addition, even this linkbetween positive-schizotypy and creativity appearsto be mediated by Openness to Experience, whichin turn, may be moderated by intelligence. If this isthe case, and clearly much more research isrequired, then we may, as human resource practi-tioners and occupational psychologists, feel a littlemore comfortable that when we select for creativity,we are not selecting for asocial-schizotypy, thus notcontradicting our desire to also recruit moreemotionally intelligent managers – thus the conun-drum is simplistically resolved … for now!

ReferencesAnderson, N. & Burch, G.St.J. (in press). The Team

Selection Inventory. Windsor:ASE;NFER-Nelson.

Burch, G.St.J., Hemsley, D.R., Corr, P.J. & Gwyer, P.(submitted). Creativity and schizotypy inrelation to latent inhibition and incidentallearning.

Burch, G.St. J.,Anderson, N.R., Banham, K., Larsen,S., Laursen, K., Sleap, S. & South, F. (2002).Selecting For Person-Team Fit. Paperpresented at The British PsychologicalSociety’s Division of Occupational PsychologyAnnual Conference, Blackpool. Abstract in theOccupational Psychology Conference Book ofProceedings, January 2002, pp.152–157.

Eysenck, H.J. (1992).The definition andmeasurement of Psychoticism. Personality andIndividual Differences, 13, 757–785.

Eysenck, H.J. (1995). Genius:The natural historyof creativity. Cambridge: Cambridge UniversityPress.

Eysenck, H.J. & Eysenck, S.B.G. (1991). Manual ofthe Eysenck Personality Scales. London:Hodder & Stoughton.

Higgs, M. & Dulewicz,V. (1999). Making sense ofemotional intelligence. Windsor: ASE:NFER-Nelson.

Hough, L.M. & Ones, D.S. (2001).The structure,measurement, validity, and use of personalityvariables in industrial, work, and organisationalpsychology. In N.Anderson, D.S.Ones,H.K.Sinangil & C.Viswesvaran (Eds.),Handbook of Industrial, Work andOrganisational Psychology. London: SagePublications.

Kline, P. & Cooper, C. (1985). Psychoticism andcreativity. Journal of Genetic Psychology, 147,183–188.

Mason, O., Claridge, G. and Williams, L. (1997).Questionnaire Measurement. In G.Claridge(Ed.), Schizotypy: Implications for illness andhealth. Oxford: Oxford University Press.

McCrae, R.R. (1987). Creativity, divergent thinkingand openness to experience. Journal ofPersonality and Social Psychology, 52,1258–1265.

Mount, M.K., Barrick, M.R. & Stewart, G.L. (1998).Five-factor model of personality andperformance in jobs involving interpersonalinteractions. Human Performance, 11,145–165.

Patterson, F. (1999). Innovation PotentialIndicator. Oxford: Oxford Psychologists Press.

Sass, L.A. (1992). Madness and Modernism:Insanity in the light of modern art,literature, and thought. Cambridge: HarvardUniversity Press.

Wallach, M.A. & Kogan, N. (1965). Modes ofthinking in young children. NewYork: Holt,Rhinehart & Winston.

Witt, L.A., Barrick, M.R., Burke, L.A. & Mount, M.K.(2002).The interactive effects ofConscientiousness and Agreeableness on jobperformance. Journal of Applied Psychology,87, 164–169.


Downlo

aded

from

www.ps

yasia

.com


Woody, E. & Claridge, G. (1977). Psychoticism andthinking. British Journal of Social andClinical Psychology, 16, 241–248.

Zuckerman, M.D., Kuhlman, M., Joireman, J.,Teta, P. & Kraft, M. (1993).A comparison ofthree structural models for personality:The BigThree, the Big Five, and the Alternative Five.Journal of Personality and Social Psychology,65, 757–768.

Giles St. John Burch, a Chartered OccupationalPsychologist, is an independent consultant andlecturer in organisational psychology andhuman resource management. He is alsoconducting doctoral research at the Institute ofPsychiatry, King’s College London, where he isinvestigating the relationship between creativity,personality and intelligence.E-mail: [email protected]

Downlo

aded

from

www.ps

yasia

.com

IntroductionTHE FOLLOWING PRESENTS a review of the 15FQ+questionnaire. The article presents a comparisonof the 15FQ+ with its predecessor, the 15FQ, intro-duces some of the new features of this assessmentand discusses practical issues, before reporting onthe development of the questionnaire and citinginternational data in relation to norm groups,reliability and validity.

Introduction to the 15FQ+The 15FQ+ is a normative, trichotomousresponse,personality test that has been developedby Psytech International as an update to the orig-inal 15FQ. Both versions of the 15FQ weredesigned for use in industrial and organisationalsettings. The original version of this assessmentwas first published in 1991 as an alternative to the16PF series of tests. The original 15FQ wasdesigned to assess 15 of the 16 personality dimen-sions that were first identified by Cattell and hiscolleagues in 1946. The 15FQ has been usedwidely throughout the world and now boasts animpressive array of norm groups, including appli-cants, non-applicants, management applicants,undergraduates, higher education workers and anumber of local and international norms.The UKgeneral population norm group consists of wellover 20,000 individuals. The 15FQ’s technicalmanual (which is available as a free download atPsytech’s website) provides extensive validity dataon a wide range of samples.

The updated 15FQ+The 15FQ+ is a full revision of the original 15FQ,with the authors developing and trialling acompletely new item set for the 15FQ+. Theauthors’ stated aim was to produce a relativelyshort yet robust measure of Cattell’s primary

personality factors. It had been known for sometime that reasoning ability (or intelligence) cannot be reliably measured by reasoning itemsincluded in untimed personality tests, as is thecase with Cattell’s Factor B. For this reason, FactorB was excluded from the 15FQ. However, for the15FQ+, the authors have decided to deal with thisproblem by redefining Factor B as a ‘metacognitivepersonality variable’ termed intellectance. Thisdoes not assess intelligence per se, but rather aperson’s confidence in their intellectual ability;defined in the 15FQ+ manual as:

‘…a self-reported superior level of intellectualcapacity, a preference for, and enjoyment of,complex arguments and ideas. A self-reportedsuperior level of: verbal ability, abstract reasoningability and numerical ability.’

New features of the 15FQ+In addition to the Intellectance scale, the 15FQ+now includes criterion-referenced scales for bothEmotional Intelligence (Goleman, 1996) and WorkAttitude (Ones & Schmidt, 1992).These scores arecalculated from a sub-set of 15FQ+ items that havebeen found to best predict well-validatedmeasures of the relevant constructs. Furthermore,the 15FQ+ now incorporates an extensive rangeof response style indicators that include a dedi-cated Social Desirability scale, non-dedicatedFaking Good and Faking Bad scales, and measuresof Central Tendency and Infrequency. As well asproducing a standard length test, which contains12 items per scale (200 items in total) the authorshave also produced a short form, containing sixitems per scale (100 items in total).


A review of the 15FQ+Personality Questionnaire

Graham Tyler

SDRDow

nload

ed fr

om w

ww.psya

sia.co

m


Development of the 15FQ+The 15FQ+ was developed following what theauthors term extensive item trialling. However, thisis not reported in great detail in the manual.According to the authors, the 15FQ+ has beenwritten in simple, clear and concise modernEuropean Business English (Psychometrics Limited,2002). While they report that the test items havebeen written to avoid culture,age and sex bias,onlyminimal data is reported in this regard in the tech-nical manual. The authors’ stated intention whendeveloping the 15FQ+ was to reflect the fullbreadth of Cattell’s original source traits, yet avoidproducing narrow, highly homogenous ‘cohesive’scales that measure no more than surfacecharacteristics.To this end they state that the item’sselection process was guided by the twin aims ofmaximising reliability, whilst maintaining thebreadth of the original personality factors.

Practical issuesThe 15FQ+ short-form takes approximately 15minutes to complete and the standard-form around30 minutes. It is possible to administer both formsin traditional paper-and-pencil formats,through theuse of self-scoring answer sheets and integralprofile charts,or through the use of the publisher’sGeneSysTM Integrated Assessment Software. Eitherway,administration is straightforward via the use ofdetailed, standardised instructions, and scoring iseither automated (when using the software) or amatter of collating scores from easy to use shadedboxes and transposing the item scores ontorespective sten score boxes and a graphical profilechart. Global scores are calculated through the useof a calculator and the simple instructionsprovided on every answer sheet.It should be notedthat for those who choose paper-and-pencil admin-istration and then manual scoring of the question-naire, some of the report options are not available,namely, the Fake Good, Fake Bad, EmotionalIntelligence and Work Attitude scores. However,these options can be made available through thepublisher’s bureau service or subsequent inputinto the GeneSysTM software.

The 15FQ+ Global Factors and Primary Scalesare reported in Tables 1 and 2 respectively.

15FQ+ NormsWhile the 15FQ+ continues to accumulate normgroups from around the world, in the currenttechnical manual, only one norm group isreported. However, this is a large sample of 1186individuals with a good gender breakdown of 561males and 621 females (four unknown) and anacceptable 10 per cent (n=111) representation ofethnic minorities. The age range of the reportedsample is 16–64, with a mean of 31.49 and astandard deviation of 11.15.

Reliability of the 15FQ+The 15FQ+ has been used on a variety of samples,although the technical manual currently onlyreports alpha coefficients for a professionalsample and two student samples. Table 3 (along-side) presents the alpha coefficients for each ofthe 16 personality factors for both the standard-(Form A) and short- forms (Form C) of the 15FQ+.All scales demonstrate good levels of internalconsistency, when the length of the scales is takeninto account. Most importantly, the alpha

Table 1: 15FQ+ Global Factors

E Extraversion IntroversionN Low Anxiety High AnxietyO Pragmatism OpennessA Independence AgreeablenessC Low Self-Control High Self-Control

Table 2: 15FQ+ Primary Factors

fA Distant Aloof Empathicß Low Intellectance High IntellectancefC Affected by Feelings Emotionally StablefE Accommodating DominantfF Sober Serious EnthusiasticfG Expedient ConscientiousfH Retiring Socially-boldfI Hard-headed Tender-mindedfL Trusting SuspiciousfM Concrete AbstractfH Direct RestrainedfO Confident Self-doubtingfQ1 Conventional RadicalfQ2 Group-orientated Self-sufficientfQ3 Informal Self-disciplinedfQ4 Composed Tense-driven

Downlo

aded

from

www.ps

yasia

.com

coefficients are not so high as to suggest thesefactors are measuring narrow surface traits. Thelower levels of reliability found in the short-formscales are to be expected, and reflect the relativebrevity (six versus 12 items) of the Form-C scales.

Table 4 continues to provide evidence of theacceptable levels of reliability for the 15FQ+scales. On this sample, both Factor ß(Intellectance) and Factor fM (Concrete-Abstract)fall slightly below UK acceptable levels of relia-bility. However, this may potentially reflect educa-tional and cultural factors The drop in alpha belowthe usually acceptable 0.70 level is minimal andthe mean alpha for this sample remains high forpersonality assessment at 0.75.

Psytech South Africa provide further evidenceof internal consistency reliability on their website.Overall, the 15FQ+ can be assumed to be a reliablemeasure of personality in South Africa, althoughalpha levels are generally lower than in UKsamples. Despite this, the alphas do comparefavourably to those obtained within South Africafrom other measures of personality Psytech SouthAfrica does acknowledge that literacy and educa-tional levels do, however, place constraints upon

Table 4: Reliability coefficients (alpha)for 15FQ+ administered in South Africa

to professional and managementdevelopment candidates

fA 0.71ß 0.67fC 0.76fE 0.75fF 0.71fG 0.81fH 0.82fI 0.71fL 0.75fM 0.68fN 0.73fO 0.81fQ1 0.80fQ2 0.72fQ3 0.77fQ4 0.78Mean Alpha 0.75(n=226)

Table 3: Reliability coefficients (alpha) for the 15FQ Scales based on a UK sample

Factor Form A Form A Form C Form CStudent Professional Student Professional Sample Sample Sample Sample(n=183) (n=325) (n=183) (n=325)

fA 0.83 0.78 0.64 0.64ß 0.77 0.80 0.62 0.71fC 0.80 0.77 0.60 0.63fE 0.80 0.79 0.60 0.66fF 0.75 0.78 0.63 0.63fG 0.85 0.81 0.60 0.64fH 0.85 0.81 0.68 0.68fI 0.74 0.77 0.64 0.63fL 0.78 0.77 0.66 0.62fM 0.80 0.79 0.64 0.64fN 0.79 0.78 0.67 0.67fO 0.82 0.83 0.67 0.69fQ1 0.81 0.79 0.60 0.72fQ2 0.82 0.78 0.67 0.62fQ3 0.78 0.76 0.66 0.63fQ4 0.84 0.81 0.60 0.62SD 0.72 0.70 Not quoted Not quotedn= 183 325 183 325


Downlo

aded

from

www.ps

yasia

.com


the test’s use and interpretation, and do notrecommend using the 15FQ+ for broad entry-levelscreening outside the UK; a point supported bythe current author during experience of using the15FQ+ in the United Arab Emirates and SaudiArabia.

Validity of the 15FQ+Table 5 provides data from 70 PsytechInternational course delegates who completedboth the 15FQ and 15FQ+ as part of their prac-tical experience. The table shows that ten of thecorrected correlations between the 15FQ andcorresponding 15FQ+ scale reach or approachunity, providing strong support for the constructvalidity of these factors. Of the remaining sixfactors, all but two correlate substantially withtheir respective 15FQ dimensions. The 15FQ+dimensions fA (Empathic) and fQ4 (Tense-driven)however, show only moderate correlations withtheir 15FQ counterparts. These modest correla-tions may reflect a subtle change in scale inter-pretation between the 15FQ and the 15FQ+ tests.Factor fA in the 15FQ+ measures a ‘warm-hearted,empathic concern for, and interest in, otherpeople’, rather than sociability and interpersonalwarmth as measured by the 15FQ dimension(Outgoing). A similar explanation is provided inthe 15FQ+ manual for the moderate correlationbetween 15FQ+ fQ4 its corresponding 15FQdimension.

The 15FQ+, as well as the original 15FQ, hasbeen developed to measure the original sourcetraits identified by Cattell and his colleagues.Therefore, one would expect to find evidence ofconstruct validity when comparing the 15FQ+with versions of the 16PF. Table 6 (alongside)provides data from a student sample of 183 indi-viduals, which further supports the constructvalidity of the 15FQ+.

All of the correlations in the above table aresubstantial and many of the corrected correlationsapproach unity.This demonstrates that the 15FQ+is measuring factors that are broadly equivalent tothose originally identified by Cattell andcolleagues.

In addition to the data referred to in Table 6, thetechnical manual quotes yet further construct

validity data. For example, relationships existbetween 15FQ+ factors and BAR-ON EQI scores,the Jung Type Indicator and the NEO PI-R.

Little criterion-related validity is available forthe 15FQ+. Whilst disappointing, this is to beexpected because of the recent publication of thistest. Two studies are reported by Psytech SouthAfrica (see www.psytech.co.za), one highlightsthe ability of the 15FQ+ to predict performanceappraisal outcomes for managers, supervisors andequity managers from a manufacturing company,and the other shows how various scales of the15FQ were able to predict insurance policy sales.

Summary and ConclusionsThe 15FQ+ is a relatively new, normative, factor-based measure of occupational personality, devel-oped as an update to the much used 15FQ, whichwas first published in 1991. The 15FQ+ hasdemonstrated, at an international level, more thanacceptable levels of reliability, as well as goodconstruct validity and appears to be measuringthe same source traits as those discovered byCattell. Users familiar with the 16PF series caneasily transfer their test interpretation skills to thisnew instrument. The 15FQ+ is distributed byPsytech International and is available to Level B(Intermediate) qualified users (or international

Table 5: Correlations between 15FQ+factors and the original 15FQ

15FQ+ 15FQ 15FQ Factor Uncorrected CorrectedfA 0.32 0.43ß – –fC 0.54 0.75fE 0.65 0.93fF 0.76 1.00fG 0.74 0.97fH 0.88 1.00fI 0.71 0.98fL 0.78 1.00fM 0.63 0.84fN 0.55 0.77fO 0.74 0.95fQ1 0.86 1.00fQ2 0.78 1.00fQ3 0.80 1.00fQ4 0.29 0.40

Downlo

aded

from

www.ps

yasia

.com

equivalent) without the need for conversion.Theassessment is currently being used throughout theworld, including:Australia, New Zealand, Malaysia,Singapore, South Africa,The United Arab Emiratesand the UK and Europe, as an alternative to the16PF series of tests. When administered to candi-dates for whom English is not their first language,particular care should be taken to ensure under-standing of the items and accurate interpretationof results. Despite this, the 15FQ+ has frequentlydemonstrated exceptional construct validity inter-nationally, as well as good criterion-related validityin South Africa. The 15FQ+ is at an advantagewhen compared with the 16PF5 due to its ease ofscoring, the acceptability of the language andimproved reliability on a number of the factors.Psytech report that they are continuously addingto international norms and progressing withfurther validation studies, foreign languageversions of the assessment and the provision ofcontrolled Internet administration.

ReferencesCattell, R.B. (1946). The description and

measurement of personality. New York:World Book Company.

Goleman, D. (1996). Emotional Intelligence:Why it can matter more than IQ.London: Bloomsbury.

Ones, D. & Schmidt, F. (1992). Paper presented atthe International Psychology Conference,Brussels, Belgium.

Psychometrics Limited (2002). The 15FQ+Technical Manual. Pulloxhill, Bedfordshire:Psychometrics Limited.

Further information:www.psytech.co.uk www.psytech.co.zawww.15fq.com www.genesys3.comwww.psytech.ws www.jungtype.com

Graham Tyler is a Registered (Organisational)Psychologist based in Brisbane, Australia.


Table 6: Correlations of the 15FQ+ factors with 16PF (Form A) and 16PF5

15FQ+ Factor 16PF (Form A) 16PF (Form A) 16PF5 16PF5Uncorrected Corrected Uncorrected Corrected

fA 0.31 0.37 0.55 0.70ß 0.10 – 0.34 –fC 0.59 1.00 0.81 1.00fE 0.68 0.99 0.82 1.00fF 0.72 0.98 0.81 1.00fG 0.55 0.89 0.791 0.75fH 0.78 0.99 0.88 1.00fI 0.50 0.75 0.47 0.56fL 0.29 0.52 0.60 0.79fM 0.26 0.65 0.79 1.00fN 0.30 0.70 0.25 0.31fO 0.68 0.99 0.83 1.00fQ1 0.29 0.43 0.60 0.84fQ2 0.51 0.85 0.81 1.00fQ3 0.30 0.50 0.572 1.00fQ4 0.69 0.94 0.69 0.89FG 0.49 0.72 – –FB 0.48 0.73 – –

1 Correlation with 15FQ+ Factor fQ3.2 Correlation with 15FQ+ Factor fG.

Reflects fact that the meaning of these two factors has been reversed in 16PF5 and provides further evidence that 15FQ+ is measuring original source traits identified by Cattell and colleagues.

Downlo

aded

from

www.ps

yasia

.com


APPLICATION FORMS REMAIN one of the most prevalentmethods of selection, especially in graduate andmanagement recruitment, and are implemented atthe first stage of the assessment procedure wherethe majority of the applicant pool is rejected(Robertson & Smith, 2001). A recent surveyrevealed that 78 per cent of UK graduaterecruiters now prefer their applicants to applyonline rather than on a paper-based form (Reed,2002), compared with 1998 when only 44 percent of UK graduate recruiters had the facility toaccept applications electronically (Park, 1999).Despite the importance placed on applicationforms in the selection procedure, the increasingprevalence of administering the forms online, andthe qualitative difference between paper-basedand online forms, little research has beenconducted on this development.

The initial research provided descriptive anddemographic accounts of the development and theuse of the Internet in selection (Bartram, 2002;Kuhn & Skuterud, 2000) and appeared to focus onthe benefits of online applications for the organisa-tion. These included: saving staff time and money(Bingham, Ilg & Davidson, 2002), easily sendingcandidate information to both local and interna-tional offices, appearing technologically advanced,incorporating psychometric tests at the start(KPMG, 2001), and ensuring that all of the candi-dates that apply are computer literate (Reed,2002).

Those few articles that have examined candi-date perspectives of online application forms havetended to take a broader perspective of onlinerecruitment in general (Feldman & Klass, 2002;

Foulis & Bozionelos, 2002). In addition, these articles have concluded that online recruitment isadvantageous, as the web provides a good sourceof information about employers, and applyingonline is a convenient and accessible method.

Whilst it is acknowledged by the researchersthat online application forms do have the poten-tial to be an advantageous selection device, thereare three omissions in the existing literature:(1) there has been no exclusive focus on the useof online application forms from the candidateperspective; (2) considering the difference in themedium of administration, the experience mayhave a psychological impact on the candidate;and (3) no study has considered the conse-quences and implications the psychologicalimpact of online application use may have onboth the applicants and organisations.The presentresearch aims to address this by: (1) identifyingand investigating the factors which may have apsychological impact on the candidate at eachstage of undertaking an online application form(inputs, processes and outcomes); (2) consideringthe implications this has for the organisation; and (3) providing practical recommendations for thedesign of future forms.

Online application forms:Psychological impact onapplicants and implicationsfor recruiters

Ruth E. Price & Fiona PattersonWork Psychology Group,City University, London

SDRDow

nload

ed fr

om w

ww.psya

sia.co

m

MethodTwenty university students (eight male and 12female) between the ages of 21 and 26 years(mean age 23.4 years) attended an individual 45-minute semi-structured interview, in whichthey discussed their experiences of online appli-cation forms as compared with paper-basedforms. Each participant had completed betweenone and six Internet application forms in theprevious 12 months (mean = three forms) and allhad experience of paper-based application forms,having completed, on average, two forms withinthe same time period.The data was analysed usingtemplate analysis, in which key themes were iden-tified and quantified.

1. Candidate reactions to the onlineapplication procedure(A) Inputs (i) Access to computers Seventy-five per cent of participants were usinguniversity computers and Internet service toavoid the cost of Internet charges,and they felt theprovision of computers at university was suffi-cient. However, five per cent of participants whohad to commute into university to use thecomputers identified their lack of easy access tocomputers as a factor which inhibited applyingonline.

(ii) CostThe participants who used home computers toundertake the forms (25 per cent) felt the costincurred from completing the application onlinewas a negative factor.

(iii) AttitudeNinety-five per cent of participants had a positiveattitude towards using computers and the Internetin general.

However, the negative attitude of the remainingfive per cent inhibited them from applying online.Box 1 displays the input factors and corre-sponding quotes that are perceived to inhibitonline application form use.

(B) Processes(i) Five psychological issues emerged from theprocess of applying online: privacy, support, feed-back, dehumanisation and self-selection.

PrivacyEighty-five per cent of participants wereconcerned about lack of privacy whencompleting an online form. This was due to thenecessity of completing it in a shared computerroom, as compared with completing a paper-based form in private.They felt this lack of privacyled to distractions, feelings of awkwardness, andthe potential for people to copy their responses.

SupportAlso in contrast to paper-based forms, 85 per centof participants thought that the support providedfor the online form was negative, because it wasdifficult to locate contact details, and thoseprovided were too general and offered no IT support.

Factor Quotation from participant

Lack of access to computers ‘It’s difficult if I can’t get into Uni, and it’s unfair as it assumes everyone has Internet access.’

Costs of Internet use ‘As a result of applying online my phone bill blew up…so I had to get a flat-rate Internet charge.’

Attitude ‘I find it stressful sitting in front of the[computer] screen.’

Box 1. Input factors perceived to inhibit online application form use


Downlo

aded

from

www.ps

yasia

.com


FeedbackAll of the candidates felt that it was essential to getacknowledgement that the online application hadbeen received, and indeed 85 per cent of thecandidates for the majority of their applicationshad received this feedback. The 15 per cent whohad not received acknowledgement felt this wasan important omission, given what they perceivedto be a greater uncertainty of safe delivery via e-mail rather than paper mail.

DehumanisationWhen compared with a paper-based form, 95 percent of candidates felt dehumanised by theprocess of applying online.The candidates felt thatthe process made them feel like they were notbeing treated as a personal applicant. In addition,candidates saw the process as ‘unreal’, as thoughtheir application forms would not be seen by anactual recruiter. In a number of cases this led topotential inaccuracy in responses: either viasocially desirable responding and the exaggerationof answers, or overly casual and carefreeresponses.

Self-selectionTwenty per cent of participants felt an onlineapplication form increased self-selection, and thatthis was fair, as the online process was reflectiveof a skill required in the job. However, 15 per centfelt that the relative ease of accessing a largevolume of application forms resulted in a decreaseof self-selection, and they applied to jobs becauseof the convenience, rather than genuine interestor suitability. These candidates felt that thislowered the quality of their application.

(ii) Four usability issues concerning online appli-cations emerged: practicality, technical unrelia-bility, clarity and restricted space/choice.

PracticalitySeventy-five per cent of candidates felt onlineapplications were far more convenient than thepaper-based forms; however, 40 per cent alsonoted less convenient aspects, namely the diffi-culty in ‘flicking through’ the form to check it,

reading from the screen, the inability to previewthe form before sending it, and having to providevery precise or repetitive information.

Technical Unreliability All candidates experienced some type of technicalunreliability whilst undertaking an online applica-tion form. Issues included connecting to theInternet initially, being disconnected from it andbeing unable to access the organisation’s website,or application form itself. Some candidates werealso logged off the form due to the page expiringwithout warning, and there were problems bothwith trying to save information and submitting theform. Some candidates felt the technical issues ledto spending a longer amount of time in front ofthe computer screen, which resulted in negativehealth effects such as eye strain and headaches.

ClarityFifty per cent of candidates felt there was a lack ofclarity in at least one aspect of the applicationform: it was either difficult to locate, or they feltthey were not given clear information about whatto expect, navigational instructions were unclearor they experienced problems with misleadinglinks.

Restricted Space/ChoiceFifty per cent of candidates felt the drop-downmenus were too restrictive, as the answer theyneeded was not always provided in the list, andthere was often no option to provide an alterna-tive. Additionally, 50 per cent of candidatescommented on the lack of space provided forfactual answers and, unlike a paper-form whereyou could attach an extra sheet, there was nospace to add additional relevant information.

Boxes 2 and 3 highlight the positive and nega-tive psychological and usability factors identifiedduring the process of completing an online appli-cation form.

(C) Outcomes The experience of using an online application wasperceived to affect the candidates in five differentways:Dow

nload

ed fr

om w

ww.psya

sia.co

m


(i) Attitude Forty per cent of candidates commented thatorganisations using online application formssuccessfully would be seen as innovative, forward-thinking and more appealing to work for.However, 70 per cent of candidates’ actual experi-ences with online application forms were nega-tive and this adversely affected their attitudetowards both the organisation and the adminis-tering of forms online. It is important to note that

15 per cent of candidates would still apply even iftheir image of the organisation were affected,because of their desire to find employment.

(ii) Motivation The technical difficulties encountered (see (B) ii)increased the length of time working at thecomputer. Half of the candidates believed this ledto a number of detrimental health effects, bothphysical and feeling emotionally drained.They felt


Dehumanisation ‘[A paper-based form allows you to] put more of yourself into it…online is a bit more cold.’

Support ‘Once online, [the company] expected the form to look after itself completely.’

Feedback ‘The automatic response is good, because you know it’s got there and isn’t lost in cyberspace.’‘Everyone else had e-mail acknowledgement, but I didn’t.So I rang up but they never got back to me.’

Privacy ‘[I felt] awkward using a form where people could easily look over my shoulder.’

Self-selection ‘Self-selection bias is not such a bad thing –if you can’t fill one in, then you really shouldn’t be applying.’‘I end up applying to jobs because of the ease of applying online, rather than because of my suitability for the job.’

Box 2. Psychological factors present with online application form use


Practicality ‘It’s easier to edit your responses online than on paper – you don’t have to re-write it out 10 times.’‘I hate having to scroll down the page all the time – you can’t see the answer you’ve typed in its entirety.’

Technical Issues ‘I spent hours typing information, and the page expired – my heart stopped. It hadn’t saved properly and I lost it all.I ended up in tears absolutely frustrated and didn’t apply in the end.’

Clarity ‘They should advise you if the page is going to expire,as otherwise you end up losing all the information.’

Restricted Space/Choice ‘It only gave you 50 letters for what topics you’d covered in the course.’‘Mine was a combined degree and I had to choose one [subject].’

Box 3. Usability factors present with online application form use

Downlo

aded

from

www.ps

yasia

.com


this lowered their motivation to complete some ofthe forms, especially if the job was not one towhich they were particularly committed.

(iii) Performance Sixty-five per cent of candidates felt their perfor-mance was affected by the factors that did notallow them to express their achievements and skillsaccurately, including restricted space and choice,privacy and support. 70 per cent of candidates alsofelt dehumanisation affected their performance asthey gave socially desirable responses or were lesscareful with their answers. This further distortedthe accurate portrayal of the candidate.

(iv) Fairness Twenty per cent of candidates felt that the onlineform was fairer as it is standardised, in that itrestricts word count for answers and candidatescannot add additional information, unlike with apaper-based form. However, the process ofapplying online was considered unfair by partici-pants who did not have a home computer and bythose who would incur costs from personalInternet charges. It was also felt that those factorswhich affected performance (as detailed above)were particularly unfair,as candidates thought thatthis decreased their prospects of being selected.

(v) Satisfaction Overall, although 35 per cent candidatesexpressed satisfaction with an online form due toaspects of practicality, 90 per cent felt dissatisfiedwith at least one element of the process, the mainfactors identified being technical problems, dehu-manisation, and a lack of space and support.Box 4outlines the psychological outcomes of onlineapplication form use.

2. Implications for organisationsThese psychological outcomes have a number ofimplications for organisations:

(i) Attracting quality candidates In the short-term, if online application formsgenerate negative attitudes towards an organisa-tion, and lower candidates’ motivation forcompleting the form, the quality of candidates

applying for vacancies may be affected. In thelonger-term, the success of the company may beaffected, as in a competitive market organisationshave to attract the best candidates to remainsuccessful (Robertson & Smith,2001).Moreover, forthose candidates eventually recruited, a negativeexperience with the process may influence psycho-logical contract formation prior to employment.

(ii)Recruiting the best candidates In addition to attracting quality candidates, theonline selection procedure must also discriminatebetween them and recruit the best candidates. Ifthe process of online application forms isreducing the accuracy of candidate responses andresulting in a false representation of the candi-date’s capabilities, then the reliability and thevalidity of the selection method may be affected. Ifvalidity is reduced, the company is much morelikely to recruit the ‘wrong’person for the job.Thiswill have serious consequences for both the candi-date and the organisation,especially as the averagecost of recruiting a graduate is estimated to be£5000 (Phillips, 2001).

(iii) Reputation A candidate’s reaction can contribute to the organ-isation’s reputation (Gilliland & Steiner, 1999) andan adverse reaction is likely to be publicised tocolleagues and peers, perhaps affecting custom. Inthis way, organisations may experience short-termfinancial gain by using online application forms interms of reduced recruitment costs, but in thelong-term the potential effect of a negative imageor profile may have hidden associated costs.

(iv) Fairness and Legal Challenges Discrimination between candidates must be basedon relevant and accurate information, or thecompany may face legal challenges.This is especiallythe case if there has been either no opportunity forthe candidate to demonstrate their relevant skills(e.g.due to the lack of space or forced-choice restric-tions) or the candidate has received no sympatheticinterpersonal treatment (such as support) during theselection procedure (Gilliland,1993).

In addition, the use of the Internet for recruit-ment may have an adverse impact on attracting aDow

nload

ed fr

om w

ww.psya

sia.co

m

representative candidate pool and the diversity ofthe workforce.Due to the ‘digital divide’and differ-ential access to ICT, the use of the Internet isrestricted in its demographic range in regard togender, age, race and income. For example, it isestimated that 75 per cent of job-search websitehits are from male users (Flynn, 2000).

(v) Ethical responsibility It is increasingly recognised that organisations havean ethical responsibility for the selection process(Hough & Oswald, 2000).Any responsible organisa-tion should want to design a form that is both easyto use and does not have a negative psychologicaleffect on the candidates, not least since they arededicating time and effort to the organisation inaddition to being potential employees. It is clearthat in their current state of development, there area number of issues surrounding online applicationform use, which need to be eliminated to ensurethe procedure is ethical.

(vi) Long-term costs To summarise the above implications, althoughonline application forms may save money in theshort term, the longer term impact may be farmore costly, via failure to attract, recruit and retainthe best candidates, a loss of reputation anddefence of legal challenges.

3. Preliminary practical guidelinesfor designThese initial findings suggest a number of prac-tical recommendations, which may be usefulwhen utilising online application forms in thefuture (see Box 5 overleaf).

Conclusions and future researchThis exploratory study investigated the psycholog-ical impact that online application form use has oncandidates.Although limited to a sample size of 20,the research highlighted that the method ofapplying online can lead to a variety of psycho-logical effects which have several important impli-cations for organisations. Factors that generated anegative psychological impact included a lack ofprivacy when filling in the form,restricted space todemonstrate their competencies, and a feeling ofdehumanisation.This negative psychological effectincluded feelings of unfairness and dissatisfaction,as well as attitudinal change to the organisation.Organisations using online application forms may,therefore, save money in the short term due to thereduced cost of the hiring method, but in the longterm may lose their reputation, quality employeesand money.

In the short term, given the increased usage ofonline application forms, future research urgentlyneeds to address: (1) whether the process does



Attitude ‘Using an online application form means they have got more scope to make themselves look bad. It puts me off the company if I have technical problems.’

Motivation ‘I got that sick of the technical problems that I just wanted to finish it and send it off.’

Performance ‘It’s easier to bang things out on computers, so I’m more casual with answers.’

Fairness ‘There was limited word space, but I prefer that.It means everyone is in the same position.’‘It feels like I haven’t done justice to myself in the space provided.’

Satisfaction ‘Online application forms have the potential to be better and quicker, but they are more frustrating, and I think they’ve got a long way to go on the design issues.’

Box 4. Perceived psychological outcomes of online application form use

Downlo

aded

from

www.ps

yasia

.com


FACTOR: RECOMMENDATION: HELPS TO ENHANCE:

Inputs ● Advertise in publications, not just online Fairness & Attitude● Offer paper based application form● Offer ‘print off and post’ form● Advise and allow ways of minimising cost,

e.g. type off-line, copy and paste

Process Privacy Psychological ● Offer paper-based application form Attitude, Performance &

● Offer ‘print off and post form’ Fairness● Offer ‘copy and paste’ function

Support● Provide details of support on the application form Attitude & Performance● Provide specific IT support

Feedback● Always give acknowledgement of receipt (via e-mail) Attitude

Dehumanisation● Provide a specific name and telephone Attitude & Performance

number/e-mail address for support● Send correspondence from to an individual

person from an individual person● Send correspondence by letter● Advise candidates’ to prepare, and inform

them of the consequences to applying

Self-selection● Provide a realistic job preview and Performance & Motivation

emphasise the competencies required to do the job ● Provide details of who the form will be

sent to and who will assess it

Process PracticalityUsability ● Design a centralised navigation system Attitude

● Offer the ability to preview and print the form● Offer spell-check and copy and paste function● Avoid rigid rules, e.g. having to enter the exact dates

Technical● Offer non-technical ways of applying Attitude, Motivation &● Advise and allow copy and paste function Performance● Provide IT support

Clarity● Ensure it is easy to locate the form from Attitude & Performance

the main webpage● Ensure the form is easy to navigate● Ensure wording of links is clear and accurate● Give instructions as to what to expect,

e.g. attach a CV, pages expiring

Space● Provide a flexible amount of space for factual answers Attitude, Performance &● Always provide the option of ‘other’ in Fairness

drop-down menus or similar● Allow a final text box for other relevant

additional information

Box 5. Preliminary practical recommendations for the design and use of online application forms

Downlo

aded

from

www.ps

yasia

.com


affect the reliability and the validity of the appli-cation form as a selection device; (2) the extent towhich online applications have an adverse impacton diversity and fairness; and (3) if these findingscan be generalised to all levels of employment.

It would be beneficial to conduct a series ofstudies, following the practical recommendationsgiven here, to discover if refining online applica-tion forms reduces the negative psychologicalimpact on candidates.This should help to ensurean online application form method that is reliable,valid, fair, convenient and cost-effective.

ReferencesBartram, D. (2002). Internet Recruitment and

Selection: Kissing Frogs to Find Princes.International Journal of Selection andAssessment, 8, 261–274.

Bingham, B., Ilg, S. & Davidson, N. (2002). Greatcandidates fast: Online job application andelectronic processing – Washington state’snew internet application system. PublicPersonnel Management, 31, 53–64.

Feldman, D.C. & Klass, B.S. (2002). Internet jobhunting: A field-study of applicant experienceswith online recruiting. Human ResourceManagement, 41, 175–192.

Flynn, G. (2000). Internet recruitment limitsdemographic scope. Workforce, 79, 85–87.

Foulis, S. & Bozionelos, N. (2002).The use of the Internet in graduate recruitment:The perspectives of graduates. Selection andDevelopment Review, 18, 12–15.

Gilliland, S. (1993).The perceived fairness ofselection systems:An organisational justiceperspective. Academy of ManagementReview, 18, 694–734.

Gilliland, S.W. & Steiner, D.D. (1999). Causes andconsequences of applicant perceptions ofunfairness. In R. Cropanzano (Ed.), Justice inthe Workplace, Vol. 2. Mahwah, NJ: LawrenceErlbaum.

Hough, L. & Oswald, F. (2000). Personnelselection: Looking toward the future –remembering the past. Annual Review ofPsychology, 51, 631–664.

KPMG. (2001). The KPMG guide to onlineapplications. KPMG International.

Kuhn, P. & Skuterud, M. (2000). Job searchmethods: Internet versus traditional.Monthly Labour Review, 3–11.

Park. (1999). Graduates in the eyes of employers.London: Park HR and The Guardian.

Phillips, C. (2001). How to recruit the bestgraduates. People Management, 7, 44–45.

Reed Executive PLC. (2002, March 25).Paper prejudice hits job seekers. RetrievedMay 14, 2002, fromhttp://cws.huginonline.com/R/132089/PR/200203/853957_10.html

Robertson, I.T. & Smith, M. (2001). Personnelselection. Journal of Occupational andOrganisational Psychology, 74, 441–472.

Ruth E. Price is a Research Associate at TheWork Psychology Group, City University,London.Fiona Patterson is Professor of OrganisationalPsychology at The Work Psychology Group, CityUniversity, London.

If you would like further information, or toparticipate in an on-going research programme,please contact: Ruth Price, The Work PsychologyGroup, Department of Psychology, CityUniversity, London, Northampton Square,LONDON, EC1V OHB, UK.E-mail: [email protected].

Downlo

aded

from

www.ps

yasia

.com


MICHAEL J. GRAY (February, 2003) seems to havedetected a note of ‘lament’ in my recent review ofperformance and personality research(October/December, 2002) that was neverintended. In fact, I thought that my concludingremarks were quite upbeat, i.e. ‘Perhaps thecurrent ‘bottom line’ for the contribution ofpersonality measures is to be found in Schmidtand Hunter’s (1998) attempt to combine bothGMA and conscientiousness in predicting perfor-mance. This combination was found to predict 36 per cent (R = 0.60) of variations in perfor-mance.This still leaves 64 per cent of performancevariation unexplained but it is a good (andeconomical) start to which can be added assess-ments of skill and previous performance’.

I take a lot of Gray’s comments on board and asan ideal it would be good to validate personalityinstruments for each role in every specificcontext.This would, undoubtedly produce highervalidity co-efficients. However, in the real worldeven the most conscientious in-house practitioneris likely to be up against the problem of samplesize and external consultants rarely have access toperformance or other criterion data. Research byBrown (February,1999) revealed that only two percent of test users sampled had done any in-housecriterion referenced validation. (I believe I wasone of four out of the 200 users that sheapproached who claimed to have done this.)

To try and put the generalisation versus speci-ficity argument into perspective, I offer MikeSmith’s (1994) ‘Theory of the validity of predic-tors’. Smith classified selection predictors intoUniversals, Occupationals and Relationals.Universals are likely to be predictive of success inany occupation and include General Mental

Ability, vitality and the centrality or importance ofwork in the life of the respondent. Occupationalsare job, or at least occupationally, specific attrib-utes and may include particular cognitive abilities(e.g. numerical, verbal, spatial), specialised knowl-edge and distinctive personality profiles.Relationals are context-dependent attributes thatcan occasionally subsume or militate against thebroader occupational requirements.

I undertook some preliminary comparison ofpersonality test means for sub-sets of the seniormanagement population in a substantial organisa-tion in which I was employed some years ago.I found almost no significant differences betweenthe means for managers in each of the four diversedivisions of the Group but a large number differ-entiating, sales/marketing, operations and financedirectors.This preliminary work was published inSDR between 1991 and 1993.

The table included in my most recent article forSDR showed some significant transcendingvalidity co-efficients emerging from the meta-analysis of Barrick, Mount and Judge (2001). Someof these co-efficients related to Occupationals,e.g.management and sales, whilst others were morelike Relationals e.g. training outcomes. These co-efficients were significant but they were bothunderstandably modest (rarely more than 0.3)since they transcended such a wide variety ofoccupations and contexts. However, for the samereason they were also very robust and can guidepractitioners who are unable to carry out statis-tical validations in-house.This leaves it for practi-

Personality andPerformance: Continuingthe debate

Hugh McCredie

SDRDow

nload

ed fr

om w

ww.psya

sia.co

m

tioner-researchers such as myself to try and movethe game further along the general-to-specificspectrum by investigating and publishing broadoccupational personality differences.

ReferencesBarrick, M.R., Mount, M.K. & Judge,T.A. (2001).

Personality and performance at the beginningof the new millennium:What do we know andwhere do we go next? International Journalof Selection and Assessment, 9, 1/2, 9–30.

Brown, R. (1999).The use of personality tests:A survey of usage and practice in the UK.Selection & Development Review, 15(1), 3–8.

Schmidt, F.L. & Hunter, J.E. (1998).The validityand utility of selection methods in personnel

psychology: Practical and theoreticalimplications of 85 years of research findings.Psychological Bulletin, 124, 262–274.

Smith, M. (1994). A theory of the validity ofpredictors in selection. Journal ofOccupational and OrganisationalPsychology, 67, 13–31.

Hugh McCredie is an independent manage-ment assessment and development consultant.He is undertaking part-time doctoral researchinto personality, behaviour and skill differencesbetween sales/marketing, operations andfinance director roles at UMIST. Hugh can becontacted at [email protected] website is www.coordinates.org.uk

I ENJOYED READING the article by Paul Taylor andBruce Small (February 2003) on Structured inter-views.

The argument on the relative merits of using‘past experience’ questions versus hypotheticalsituational questions with or without a behav-iourally-anchored scale, whilst not containinganything one could not decide from commonsense, did contain some interesting measures andwas a useful piece of teaching.

One thing I looked for and did not find in thearticle was any mention of the impact of skill inconducting ‘past experience’ behavioural inter-viewing. Though much more experienced in theuse of the ‘behavioural’ interviews in research intofactors leading to superior performance, when Ihave trained otherwise experienced interviewersin the skill of behavioural interviewing, I havenoted the initial low level of skill they possess and

hence their relative inability to gather informationfrom such an interview. This low level of skill inbehavioural interviewing probably applies to mostinterviewers since I believe that few interviewersare properly trained in the technique.

Whilst I cannot back up the assertion with thefigures I would think that the skill level of theinterviewers would rank at least equal to the ques-tion of the existence or otherwise of an anchoredscale in determining the predictive validity. Giventhe option I would give priority to developing theskill before developing the anchored scales.

E-mail:[email protected]


Structured interviews:A response

K.W.YoungKWY Associates

Downlo

aded

from

www.ps

yasia

.com


Below is a brief explanation of some of the statistical terms referred to in the meta-analysis.

Sample-weighted mean validity. Whencomputing mean validities and effect sizes, meta-analysts typically weight each study by its samplesize, or by a related statistic, in order to givegreater weight to studies with larger sample sizes.This practice is based on the principle that largersamples have less sampling error (i.e. their samplemeans are likely to be closer to the mean of thepopulation) than studies with smaller samples.

Correction for criterion unreliability and rangerestriction. Unreliability in criterion measuressystematically reduces (‘attenuates’) observed rela-tionships between other variables and the criterion.Meta-analysts often estimate the degree of unrelia-bility in the criterion, based on reliability data fromthe studies included,or a larger,similar set of studies,allowing them to correct for that unreliability.Such acorrection raises the validity coefficient or effect sizeto an estimate of what it would be had the criterionbeen measured with perfect reliability.

Similarly, validation studies are conducted onemployees who have been on the job long enough

so that job performance can be measured (typi-cally at least six to 12 months, and often longer).Because not all applicants have becomeemployees, and not all employees have lasted inthe job long enough for job performance to bemeasured, the range of scores obtained from asample of employees is typically restricted (i.e. smaller variance) in comparison to the rangeof the group to which the study is to be general-ized – applicants. The average difference in vari-ances between the restricted (employees) andunrestricted (applicants) samples is used by meta-analysts to correct validity coefficients for thisrestriction. Like corrections for attenuation (unre-liability), corrections for range restriction raise thevalidity coefficient to estimate what the validitieswould have been had the studies been performedon actual applicants.

Most meta-analyses concerning the criterion-related validity of selection methods report bothuncorrected and corrected validity estimates, aswe have done here.

Paul Taylor & Bruce SmallPaul Taylor’s e-mail is: [email protected]

In last month’s issue (Vol 19, No.1) we published the article ‘Question format in the structuredemployment interview’ by Paul Taylor and Bruce Small.This should have included a ‘box’ with expla-nations of the statistical terms used to help readers who were unfamiliar with the statistics used inthe meta-analytic study reported. Due to an oversight this ‘box’ did not appear with the article.We apologise for this omission and the missing explanation is printed below

SDRQuestion format in thestructured employmentinterview – statisticsappearing in the meta-analyses

Downlo

aded

from

www.ps

yasia

.com

THE USE OF projective methods; TAT, sentencecompletion, etc., in the occupational setting has, itwould appear, fluctuated over time. From the1960s onwards the growth of psychometrictesting, its associated distrust of subjective, impres-sionistic, whole person approach, and, its faith inthe statistical, seems to have significantly reducedthe use, in the occupational sphere, of the projec-tive test. However, to my knowledge, there are stilloccupational psychologists who use projectivemethods in selection assessment and/or for devel-opment purposes.

Those who use projective methods, it is hoped,received appropriate training. Additionally, itwould be expected that they will train others touse these methodologies. However, though thereis now considerable rigor in the validation ofpsychometric test trainers, it would appear that asimilar approach, at least in the occupational area,is not taken to projective training.

Reading the guidance on the certificates incompetency would not appear to preclude theinstrument(s) being projective, although thegeneral thrust would seem to suggest that it willbe easier to be successful if psychometricmeasures are selected.

It would be useful to know if anyone is trainingothers to use projective methods for occupationuse, and if they have been verified.

If there is demand for verification in projectivemethods, then it would be interesting to evaluateits strength, and thereby the usefulness of such aprovision.Any comments would be very valuable.

Chris can be contacted [email protected]


SDRProjective measures inoccupational settings:The need for a verificationprocedure

Christopher C. Ridgeway

Downlo

aded

from

www.ps

yasia

.com

15fq+Review

Documents

Transcript of 15fq+Review