Centre for Research in Equality and Diversity (CRED) The politics of assessment centres: Building...

Centre for Research in Equality and Diversity (CRED)

The politics of assessment centres:The politics of assessment centres:Building diversity in the UK judiciaryBuilding diversity in the UK judiciary

Gill Kirton, Geraldine Healy and Mustafa OzbilginGill Kirton, Geraldine Healy and Mustafa Ozbilgin

Paper to MMU Business School Paper to MMU Business School 1919thth January 2011 January 2011


Politics of assessment centresPolitics of assessment centres

Aim of studyAim of study To investigate the value of assessment To investigate the value of assessment

centres for increasing diversity in the UK centres for increasing diversity in the UK judiciaryjudiciary

Aim of talkAim of talk To put our investigation of judicial To put our investigation of judicial

selection centres in macro and micro selection centres in macro and micro political contextspolitical contexts


Structure of presentationStructure of presentation

Macro politics surrounding the Macro politics surrounding the introduction of assessment centres introduction of assessment centres for judicial appointmentsfor judicial appointments

Micro politics of recruitment and Micro politics of recruitment and selection, focusing specifically on selection, focusing specifically on assessment centres assessment centres

Research methodsResearch methods Research findings on assessment Research findings on assessment

centres for judicial appointmentscentres for judicial appointments


Macro politics of judicial selectionMacro politics of judicial selection

Demographic composition of judges pre-2005Demographic composition of judges pre-2005 Dominance of public school, Oxbridge educated – over Dominance of public school, Oxbridge educated – over

80% had attended Oxbridge80% had attended Oxbridge White male domination – 18% women; 4% BMEWhite male domination – 18% women; 4% BME

Socio-political issuesSocio-political issues Concern around public confidenceConcern around public confidence Concern around impact of judicial decisions on wider Concern around impact of judicial decisions on wider

societysociety Concern around impact of judicial decisions on BME Concern around impact of judicial decisions on BME

communitiescommunities Employment/HR issuesEmployment/HR issues

Job security of judicial post holders means once Job security of judicial post holders means once appointed, it’s more or less a ‘job for life’appointed, it’s more or less a ‘job for life’



The work of judges – competing The work of judges – competing rationalities rationalities Constrained autonomyConstrained autonomy Intensity of case loadsIntensity of case loads Impact of political influences at different Impact of political influences at different

levelslevels Professional judgement versus political Professional judgement versus political

influenceinfluence Professional judgement versus Professional judgement versus

consequential impact of decisionsconsequential impact of decisions



Traditional recruitment and selection Traditional recruitment and selection methodsmethods ‘‘Secret soundings’ with serving members Secret soundings’ with serving members

of the judiciaryof the judiciary Involved asking senior judges to rate Involved asking senior judges to rate

performance of lawyers (mainly barristers) performance of lawyers (mainly barristers) they had observed in courtthey had observed in court

Widely thought to preserve judicial Widely thought to preserve judicial independenceindependence

‘‘Suitable’ candidates invited to formal Suitable’ candidates invited to formal panel interview with Lord Chancellor’s panel interview with Lord Chancellor’s departmentdepartment



Peach Report 2001 charged with Peach Report 2001 charged with considering the considering the appropriateness and appropriateness and effectiveness of: effectiveness of: a) The criteria for judicial appointments, a) The criteria for judicial appointments,

and and b) The procedures for selecting the best b) The procedures for selecting the best

candidates candidates The extent to which candidates are assessed The extent to which candidates are assessed

objectively against the criteria for appointment objectively against the criteria for appointment The existence of safeguards in the procedures The existence of safeguards in the procedures

against discrimination on the grounds of race or against discrimination on the grounds of race or gender gender



Peach Report recommended ACsPeach Report recommended ACs The stated purposes were:The stated purposes were:

‘‘To ensure that the most able, gifted and suitable To ensure that the most able, gifted and suitable people, who meet all of the criteria for people, who meet all of the criteria for appointment, are selected for judicial posts’appointment, are selected for judicial posts’

‘‘To introduce selection methods and procedures To introduce selection methods and procedures which are seen to be fairer, more open and which are seen to be fairer, more open and transparent in which applicants, the legal transparent in which applicants, the legal profession and the public can feel more profession and the public can feel more confident and which demonstrably support the confident and which demonstrably support the Lord Chancellor’s policy statement on equal Lord Chancellor’s policy statement on equal opportunities’opportunities’


Micro politics of judicial selectionMicro politics of judicial selection

What are assessment centres?‘What are assessment centres?‘ A structured combination of assessment techniques that are A structured combination of assessment techniques that are

used to provide a wide-ranging, holistic assessment of each used to provide a wide-ranging, holistic assessment of each participant’ (Garavan and Morley 1997:160). participant’ (Garavan and Morley 1997:160).

The common features of assessment centres are: The common features of assessment centres are: Assessment in groups Assessment in groups Assessment by groups Assessment by groups The use of multiple techniques (e.g. ability tests, peer The use of multiple techniques (e.g. ability tests, peer

evaluations, performance tests)evaluations, performance tests) The use of simulation and situational techniques (e.g. role-The use of simulation and situational techniques (e.g. role-

playing exercises); assessment along multiple dimensions playing exercises); assessment along multiple dimensions (Garavan and Morley 1997). (Garavan and Morley 1997).

Multiple methods become a proxy for predictive evaluation Multiple methods become a proxy for predictive evaluation of subsequent job performance and therefore a more of subsequent job performance and therefore a more objective and valid basis on which to make appointment objective and valid basis on which to make appointment decisions (Iles 1992; Dayan et al. 2002; Tyler and Bernardin decisions (Iles 1992; Dayan et al. 2002; Tyler and Bernardin 2003). 2003).



ACs – a technocratic solution?ACs – a technocratic solution? Clear rules for both recruiters and Clear rules for both recruiters and

candidatescandidates Greater processual and decision Greater processual and decision

transparencytransparency More accurate selection decisions based More accurate selection decisions based

on ‘hard’ evidence on ‘hard’ evidence From a liberal equality perspective – From a liberal equality perspective –

reduces opportunity for recruiter reduces opportunity for recruiter subjectivity and biassubjectivity and bias



BUT, paradoxically ACs neglect macro and BUT, paradoxically ACs neglect macro and even micro political and social processeseven micro political and social processes The desired outcome was greater diversity, The desired outcome was greater diversity,

but do liberal equality (procedural) measures but do liberal equality (procedural) measures always achieve this?always achieve this?

A deep understanding of the socio-economic A deep understanding of the socio-economic causes of inequality is not taken into ACscauses of inequality is not taken into ACs

Potential for the exercise of recruiter bias Potential for the exercise of recruiter bias could remaincould remain


Research methodsResearch methods Analysis of documentation: DCA internal Analysis of documentation: DCA internal

documents, assessment centre (AC) guidance documents, assessment centre (AC) guidance notes and application forms, assessors’ notes and notes and application forms, assessors’ notes and evaluations, consultee (referee) reportsevaluations, consultee (referee) reports

Listening to tape recordings of AC activitiesListening to tape recordings of AC activities Interviews with recent judicial applicants (both Interviews with recent judicial applicants (both

successful and unsuccessful): 11 Black and successful and unsuccessful): 11 Black and minority ethnic (BME) women, 9 BME men, 10 minority ethnic (BME) women, 9 BME men, 10 white women and 10 white men, only 3 Oxbridge white women and 10 white men, only 3 Oxbridge graduates (39 interviews)graduates (39 interviews)

Interviews with AC assessorsInterviews with AC assessors Quantitative analysis of one judicial appointment Quantitative analysis of one judicial appointment

competition involving 386 applicantscompetition involving 386 applicants


Selecting judges through ACsSelecting judges through ACs Competency frameworkCompetency framework

Nine competencies identify both what is to be done and Nine competencies identify both what is to be done and how it is to be done (the work of a judge)how it is to be done (the work of a judge)

Diversity is incorporated into some of the competenciesDiversity is incorporated into some of the competencies Application form – self-assessment against the nine Application form – self-assessment against the nine

competencies (non-professional, ‘life’ examples are competencies (non-professional, ‘life’ examples are allowed and in theory encouraged)allowed and in theory encouraged)

‘‘Sift’ (shortlisting) process by panelSift’ (shortlisting) process by panel One-day Assessment CentreOne-day Assessment Centre

Individual interview by mixed panel of professionals (HR Individual interview by mixed panel of professionals (HR and management as well as legal)and management as well as legal)

Legal case assessment – written testLegal case assessment – written test Written test of legal knowledge Written test of legal knowledge Role play – managing a court caseRole play – managing a court case


The dynamics of micro political The dynamics of micro political control in assessment centrescontrol in assessment centres

Assessor practices and perspectivesAssessor practices and perspectives Use of discretion – controlling and Use of discretion – controlling and

manipulating the interview and its outcomemanipulating the interview and its outcome ‘‘In ‘building relationships’ … sometimes you get a In ‘building relationships’ … sometimes you get a

better response from women candidates, particularly better response from women candidates, particularly when you ask, “members of the judiciary have when you ask, “members of the judiciary have sometimes been criticised for being too distant”, do sometimes been criticised for being too distant”, do you agree with that?’ ….. Usually then women come you agree with that?’ ….. Usually then women come out much better ….. I try to get them off that and out much better ….. I try to get them off that and move them on somewhere else because that’s move them on somewhere else because that’s advantaging them.’advantaging them.’

‘‘Well, again, if you ask them what they are doing in Well, again, if you ask them what they are doing in their community because, I think of an Asian their community because, I think of an Asian candidate, telling me he was a member of Rotary candidate, telling me he was a member of Rotary wasn’t actually what I was looking for.’wasn’t actually what I was looking for.’



Assessor practices and perspecitvesAssessor practices and perspecitves Use of stereotypes – influencing competency Use of stereotypes – influencing competency

ratingsratings ‘‘Again, a complete mixture of response … many Again, a complete mixture of response … many

[women] are very, very good; quite surprising really. [women] are very, very good; quite surprising really. Some of them really empathise [with minorities]. But Some of them really empathise [with minorities]. But you would expect that, from their experiences and so you would expect that, from their experiences and so on, you would expect them to be good.’on, you would expect them to be good.’

‘‘Most of them [BME candidates] weren’t the sort of Most of them [BME candidates] weren’t the sort of black people that I would meet in my court. They were black people that I would meet in my court. They were highly educated. Some public school, grammar school, highly educated. Some public school, grammar school, graduates so that they weren’t ordinary and they’d graduates so that they weren’t ordinary and they’d never had to struggle with diversity because of their never had to struggle with diversity because of their colour…’colour…’



Candidate perspectivesCandidate perspectives Main views on interviewsMain views on interviews

(Im)propriety of questions(Im)propriety of questions Unevenness of ‘diversity questions’Unevenness of ‘diversity questions’ Assessor behaviour manipulating process and outcomesAssessor behaviour manipulating process and outcomes

‘‘I don’t think any of the questions were inappropriate. I I don’t think any of the questions were inappropriate. I think the tone was occasionally inappropriate and I don’t think the tone was occasionally inappropriate and I don’t know if it was deliberately trying to be confrontational, know if it was deliberately trying to be confrontational, but I think it was slightly putting down, you know, so it but I think it was slightly putting down, you know, so it wasn’t the questions that were inappropriate, it was the wasn’t the questions that were inappropriate, it was the tone.’ (BME candidate)tone.’ (BME candidate)

‘‘I find it strange that they would even have to ask the I find it strange that they would even have to ask the [BME] person about diversity issues, you know l live it, I [BME] person about diversity issues, you know l live it, I live the life. So I don’t criticise the questions, it was part live the life. So I don’t criticise the questions, it was part of the process, but I think it’s unnecessary, it’s like asking of the process, but I think it’s unnecessary, it’s like asking someone who is in a wheelchair how they can relate to someone who is in a wheelchair how they can relate to other people in wheelchairs, why on earth would you need other people in wheelchairs, why on earth would you need to do that?’ (BME candidate)to do that?’ (BME candidate)



Candidate perspectivesCandidate perspectives Final verdict on assessment centresFinal verdict on assessment centres

ProceduralProcedural justice is served? justice is served? ‘‘I think the assessment centre approach is better because I think the assessment centre approach is better because

… somebody … with a less middle class type background … somebody … with a less middle class type background doesn’t do so well in my view in these interviews. But it doesn’t do so well in my view in these interviews. But it doesn’t matter who you are, if you’re going to be good at doesn’t matter who you are, if you’re going to be good at doing your job, you get the opportunity to demonstrate it doing your job, you get the opportunity to demonstrate it in other parts of the assessment day.’ (Female candidate)in other parts of the assessment day.’ (Female candidate)

Only to some extent because subtle processes of bias Only to some extent because subtle processes of bias and discrimination infuse the assessment centreand discrimination infuse the assessment centre

‘‘I think the people who do best at the assessment centre I think the people who do best at the assessment centre are probably the criminal practitioners … Now, most of are probably the criminal practitioners … Now, most of them are white and male, so in that sense it would prefer them are white and male, so in that sense it would prefer them because they are more able to deal with particularly them because they are more able to deal with particularly some of the academic issues and the role playing, which I some of the academic issues and the role playing, which I found terrifying.’ (Female candidate)found terrifying.’ (Female candidate)


Do ACs lead to greater Do ACs lead to greater numerical diversity?numerical diversity?

Based on analysis of one DDJ Based on analysis of one DDJ competition:competition: Women (both white and BME) more likely Women (both white and BME) more likely

to be successful at the sift stage and also to be successful at the sift stage and also more likely to be recommended for more likely to be recommended for appointmentappointment

Ethnicity did not have any influence at Ethnicity did not have any influence at the sift nor at the AC stagesthe sift nor at the AC stages

BME women had greater success rates at BME women had greater success rates at sift than BME mensift than BME men


ConclusionsConclusions• ACs increase recruiter and to some extent candidate ACs increase recruiter and to some extent candidate

perceptionsperceptions of procedural justice of procedural justice• Plenty to indicate that ACs are an improvement over ‘secret Plenty to indicate that ACs are an improvement over ‘secret

soundings’soundings’• But, a single HR intervention cannot tackle wider relations But, a single HR intervention cannot tackle wider relations

of power that enter the micro processes of the AC or of power that enter the micro processes of the AC or structural inequalities that influence the pool of ‘competent’ structural inequalities that influence the pool of ‘competent’ candidatescandidates

• Even this purportedly more objective selection method Even this purportedly more objective selection method allows ample opportunity for manipulation by designers and allows ample opportunity for manipulation by designers and recruitersrecruiters

• ACs are social processes not places and competencies are ACs are social processes not places and competencies are socially constructed, not neutral and objectivesocially constructed, not neutral and objective

• Thus, political control at the micro level of ACs rests with Thus, political control at the micro level of ACs rests with recruiters and existing power holders from the design recruiters and existing power holders from the design through to executionthrough to execution


PostscriptPostscript In 2007 the responsibilities of the DCA were In 2007 the responsibilities of the DCA were

passed to the Ministry of Justice and to the Judicial passed to the Ministry of Justice and to the Judicial Appointments CommissionAppointments Commission

ACs are still in useACs are still in use Candidate seminars are held to explain the application Candidate seminars are held to explain the application

and AC processand AC process Sample tests are provided on the websiteSample tests are provided on the website Potential candidates can volunteer to take part in ‘mock’ Potential candidates can volunteer to take part in ‘mock’

ACs as part of assessor trainingACs as part of assessor training JAC data shows an increase in the appointment of women JAC data shows an increase in the appointment of women

in most categories of judicial appointment post in most categories of judicial appointment post introduction of ACs and some increase in appointment of introduction of ACs and some increase in appointment of BME candidatesBME candidates

DDJ 2001/02 = 35% women and 5% BME; 2007/08 = 58% DDJ 2001/02 = 35% women and 5% BME; 2007/08 = 58% women and 8% BMEwomen and 8% BME

Centre for Research in Equality and Diversity (CRED) The politics of assessment centres: Building...

Documents

Transcript of Centre for Research in Equality and Diversity (CRED) The politics of assessment centres: Building...