Clutterbuc Kjan2011

© Clutterbuck Associates 2010

Measuring the Effectiveness of Mentoring

Professor David Clutterbuck and Lis Merrick


The RoI of Coaching & Mentoring

How HR can achieve greatest value from investment in coaching & mentoring

Professor David Clutterbuck


What issues do you need to address with regard to Coaching

RoI, and why?

© 2010 Clutterbuck Associates


The coaching spectrum

• Line manager coaching• Off-line internal skills and behavioural coaching• Internally resourced executive coaching• Externally resourced executive coaching• Mentoring



How does responsibility for developing talent divide out between:

• Line manager?• HR?• Sponsor?• Mentor?• The learner ?

Discussion: How clearly do people in your organisation understand their own and other players’ responsibilities?



Problems measuring RoI in coaching

Problems in measurement include:

• Short-term v long-term outcomes may conflict• Goals are often emergent, not fixed at the beginning• Easy fixes are the most easily measurable, but typically the least

important in terms of value• Client self-assessment is not a reliable measure



Some elements of VFM

• Change is acknowledged and sustained• Change affects more than just the client, but those they

work with (i.e. it’s systemic)• Cost is related to impact on the business• Coaches’ relative competence is related to the level

at which they operate



Measuring the effectiveness of Line Manager coaching



What drivers and impediments do you see for line manager

coaching in your organization?



Like a dance, effective coaching requires the active, informed cooperation of at least

two people. Training a line manager as a coach and expecting the team to pick it up as they

go is like a tango, where only one of the partners knows the steps!



Elements of a systemic approach to coaching within the team

• Everyone learns at least the basics of coaching • Everyone learns how to be coached• Learning about coaching takes place over time, with

opportunities to experiment and practise • There is a positive psychological contract• Everyone may be coached by anyone

(including the leader/manager)• There is ample time for reflection together



Measuring line manager coaching

• Coaching behaviours of line manager• Coaching behaviours of coachee/team member• Relationship quality• Broad individual outcomes: performance, learning, enabling,

emotional• Skinny individual outcomes: specific skills or behaviours• Outcomes for other stakeholders: the team and other

colleagues



Measurement: coachingProcess Performance

Organisation system

Team System

Individual



Assessing external executive coaches



How do you know how the executive coaches you use compare to

“world class” coaches?



Why assess executive coaches?

• Coaching qualifications are not reliable indicators of competence and “fit”, in general

• Coach provider organisations cannot guarantee the suitability of all their coach pool

• Client feedback is not a reliable indicator of coach performance

• The coaching market is evolving rapidly – someone considered an effective, leading edge coach three years ago may be considered mediocre today, unless they have invested heavily in CPD



Why an assessment centre?

• Assesses coaches from several perspectives (what they do, what they say, how they think)

• Provides a consistent benchmark, against which to compare candidate coaches

• Indicates that the organization is serious about the quality of the executive coaches it hires

• Educates people in the business about what to expect from good coaching

• Overcomes ad hoc hiring of “man in the bar”• Enables HR to set bands of coaching capability and fees• Major cost savings, both immediate and long term



What makes an effective assessment centre?

• Clarity of the competences to be assessed• Multiple perspectives on each competence • Assessors are trained to ensure consistency of scoring• Participants are clear about the purpose of each activity and

what is being assessed• Assessors have sufficient time between activities to reflect

and confer• Volunteers are briefed thoroughly about their role• Confidentiality and participant safety• Participant reflection is taken into account in the overall

assessment• Feedback to participants



The key elements of a coach assessment centre

• Robust application process to identify suitable candidates• Helping selected coaches prepare • Psychological interview, to identify relevant personality and

or other dysfunctions, which may cause concern• Panel interview, to assess knowledge, CPD, ethicality, use of

supervision, “organisational fit” and other aspects of practice• “Real play” using executive volunteers• Reflections by coachee• Reflections by coach• Feedback to coach



Some critical questions

• How do we ensure the appropriate level of applicants?• What do we do with existing executive coaches?• What experienced resources (coaches, coaching psychologists

etc) do we have in-house and what do we need to buy in?

• How will we manage the politics (internal and external)?• What is our communication plan (internally and externally)?



Where coaches most often fail to perform

• Use of supervision• Managing boundaries• Relevance and depth of CPD• Commercial awareness – linking issues with the business

context• Defining their personal philosophy of coaching• Over-dependence on simplistic models (e.g. GROW)• Too narrow a portfolio of coaching approaches



After the assessment centre

• Ranking the candidates• Feedback to participants• Induction of accredited pool• Long-term supervision and development of the pool• Internal communications• Coach promotion



Some lessons we’ve learned

• Don’t over-promise on the amount of feedback• Maintain broad support from both HR and the line• Have a process to ensure the safety of volunteers,

who experience poor or dangerous coaching• Professional organisations can be highly supportive

and helpful• Calibrate frequently



What have we achieved?

• Removed poor and dangerous coaches• Increased confidence of HR and the line in the quality

of coaching provision• Regularised executive coaching fees into bands according

to level of client and intervention• Potentially, reduced the average number of coaching sessions

required (to be measured after a year)• Raised the profile and awareness of coaching across the

organisation• Stimulated coaching pool providers to vet their coaches

more thoroughly and regularly• Contributed to raising coaching standards generally



Evolution of coach competence

Coaching approach

Style Critical questions

Models-based Control How do I take them where I think they need to go? How do I adapt my technique or model to this circumstance?

Process-based Contain How do I give enough control to the client and still retain a purposeful conversation? What’s the best way to apply my process in this instance?

Philosophy-based Facilitate What can I do to help the client do this for themselves? How do I contextualise the client’s issue within the perspective of my philosophy or discipline?

Managed eclectic Enable Are we both relaxed enough to allow the issue and the solution to emerge in whatever way they will? Do I need to apply any techniques or processes at all? If I do, what does the client context tell me about how to select from the wide choice available to me?



What have you done/could you do to give internal coaches equal status

with external?



Measuring mentoring• Longitudinal measures better than single point• Individual outcomes: Career, developmental, enabling

and emotional• Organizational outcomes: retention, job commitment• Programme quality: International Standards for Mentoring

Programmes in Employment



Sustaining RoI

Some of the essential ingredients include:

• An influential head of coaching and mentoring• Continued top level backing• Links to HR systems and developmental priorities (e.g. diversity)

and to CSR• Opportunities for continuous development of internal coaches

& mentors, and levels of qualification• Measurement of relationship quality and coaching/mentoring

behaviours, as well as outcomes



Measuring mentoring• Longitudinal measures better than single point• Individual outcomes: Career, developmental, enabling

and emotional• Organizational outcomes: retention, job commitment• Programme quality: International Standards for Mentoring

Programmes in Employment



Sustaining RoI

Some of the essential ingredients include:

• An influential head of coaching and mentoring• Continued top level backing• Links to HR systems and developmental priorities (e.g. diversity)

and to CSR• Opportunities for continuous development of internal coaches

& mentors, and levels of qualification• Measurement of relationship quality and coaching/mentoring

behaviours, as well as outcomes



When to measure

• Before the programme start (Climate for mentoring + general participant expectations)

• At the beginning (Specific participant expectations)

• After 2/3 meetings (Is this a good match?)

• After 4-6 months (Are relationships working? What support do people need?)

• After 12 months (Short-term - mainly developmental - outcomes)

• After 24 months (Longer-term – career – outcomes)

© Clutterbuck Associates 2010© Coach Mentoring Ltd 2008

12 April 2023 32

1. Individual Mentee and Mentor Development

• Through the organisation’s performance management system

• Promotion and retention figures• Attitude questionnaires/360 degree feedback


12 April 2023 33

2. Mentoring Relationships

• Number and frequency of meetings• Average length of relationships• Number of requests for new mentors• Number of relationships with agreed contracts (if

that is a voluntary process)• Confidential 3rd party feedback


3. Evaluation of a Mentoring Programme

Programme processes,Selection criteria,Proportion of successes/failures,Training, Skills, Programme support

Programme outputs, Retention, Promotions, Improved assessments, Significant progress

Relationship process, Rapport, Committed objectives, Meeting frequency/relevancy/value, Learning

Relationship outputs, Learning milestones reached, Performance improvements, Confidence level, Line manager reaction

Rel Level

Prog Level

Process Outputs


12 April 2023 35

Final Evaluation

• Has the programme achieved its objectives?• Have you evaluated change in skills/benefits

obtained/feelings of participants/process effectiveness?• How well did your process meet your needs?• Return to organisation - Has it contributed to your overall

development process?


Validated scales

• Organisational context• Goal clarity, commitment and alignment• Mentoring behaviours (by both parties)• Relationship quality• Outcomes – career, developmental, enabling and

emotional


Formal assessment: the ISMPE

The International Standards for Mentoring Programmes in Employment cover:

1. Clarity of purpose2. Stakeholder training and briefing3. Selecting and matching4. Measurement and review5. Ethics and pastoral care6. Support systems and administration


What does it mean to achieve Gold standard?

• Three levels of award – Bronze (85%), Silver (90%), Gold 95% + compliance

• Gold schemes are among the most effective and well-managed in the world

• It can be used as a benchmark for other programmes


The assessment process

• Programme coordinator and team review practice against the standards, and gathers initial evidence

• Assessor reviews the evidence and identifies areas, where improvements would increase the score

• Changes are made!• Second assessment, with more detailed evidence and

interviews with stakeholders• Report is reviewed by supervisory assessor• Award is made!


Outcomes

Hard measures of mentoring effectiveness

• Retention• Appraisal scores• Increased sales/outputs• Proportion of proposals based

on customer value-added• Employee engagement data• Diversity statistics• Quality improvement data• Actual promotions

• Quality improvement data• Actual promotions• Process• Frequency of meetings• Numbers of pairs that stay

the course


Outcomes

Soft measures of mentoring effectiveness

• Achievement of relationship goals (as measured by mentee and mentor)

• Significant changes in behaviour or competence, as seen by mentee or via 360 feedback

• Increased self-confidence• Improved work-life balance,

stress etc

• Improved work-life balance, stress etc

• Process• Perception of relationship

quality• Depth and openness

of mentoring dialogue


Informal, ad hoc monitoring

• What’s going well and less well?• What would you like your mentor/mentee to do more/less of?• How does this relationship compare with your ideal mentoring

relationship?• What specifics (e.g. More frequent meetings) would make a

positive difference?• What additional support would be helpful?


Continuous improvement

• Measurement encourages good practice in mentoring dyads• Communicate survey data to participants and sponsors – ask

them to help contextualise responses and use them to initiate programme improvements

• Look for patterns in the data – e.g. Why do male mentees appear to be less trusting of their mentors than females?

• Conduct an annual programme review and make it part of your report to senior management

• Where appropriate, benchmark against other similar organisations


Discussion: How would you now respond if your CEO

asked you to prove the RoI for coaching and mentoring in your organisation?



Clutterbuck AssociatesTel: +44 (0)1628 606850

Email: [email protected]


Clutterbuc Kjan2011

Documents

Transcript of Clutterbuc Kjan2011