Theresa Hammond 12th Annual Coaching & Mentoring Conference Oxford Brookes University January 14 th 2016 Context Pastoral Ministry My personal experience Beneficial environment of coaching & mentoring: Missing Patchy Design was questionable lacking & haphazard delivery mostly via current hierarchal structures Increasingly, Coaching & Mentoring are offered However, it has been noted that it is the exception and these relationships are often neglected No standardized practice Making monitoring and evaluation effectiveness difficult Literature Review Confirmed the need and call for Coaching & Mentoring in the pastoral ministry for: Training Support Ongoing Development However, of the handful of empirical studies of Coaching & Mentoring in the pastoral ministry, none focused specifically on the design of the Coaching & Mentoring Aim of Research To identify what different coaching and mentoring designs are currently in use in the pastoral ministry To evaluate the efficacy of the different coaching and mentoring designs Evaluation of Coaching & Mentoring Notoriously difficult Not-for-profit settings difficult to establish desired outcomes Ministers own sense of efficacy arising out of his/her learning, growth and development Long term nature of growth and development difficult & less tangible What and How to Assess Evaluation can be approached in two ways (Ely, et al., 2010): Outcomes summative evaluation Process formative evaluation Context & aim of research call for a process or formative evaluation To do so is to ask: Does the coaching/mentoring environment provide the conditions that either promote or hinder growth and development & therefore his/her sense of efficacy & resilience? What and How to Assess We might be able to define the conditions necessary to facilitate growth & development i.e. Rogerian conditions, etc However, it is not what is believed to be delivered, but what is subjectively experienced and can therefore be accessed This necessarily calls for a subjective assessment Research Methodology Due to the requirement of subjective data Qualitative research methods were considered however, although this would have produced rich data in terms of the experiences of individual participants it could not necessarily link this data with coaching and mentoring designs Instead Q Methodology was chosen a combined research method coined a qualiquantological approach (Stenner & Stainton, 2004) Q Methodology.. a science of subjectivity, introduced in 1935 by psychologist/physicist William Stephenson.. it offers a a qualitative analysis through a quantification of patterned subjectives (Shemmings, 2006) At the heart of a Q investigation A person is presented with a set of statements (Q-Set) about a specific topic and asked to rank-order them, an operation referred to as Q- sorting (Brown, 1993) These statements are subjective and are ranked from the participants point of view onto a grid of prearranged frequency distribution Sorting Grid used for this study Most Disagree Most Agree The forced ranking of the statements result in Q-Sorts: configurations of items (statements), as statements relative to one another These are: whole configurations, patterns of items, which reveal viewpoints as a whole, and can be subjected to factor analysis In factor analysis the Q-sorts are correlated with each other to identify the most common subjective viewpoints... those which are highly correlated may be considered to have a family resemblance (Brown, 1993), and are referred to as Factors The aim of this study being to identify these Factors (patterns of subjective experiences and viewpoints) and how these might correlate with coaching & mentoring designs Broader Q Methodological study involves The following steps: 1. Definition of the Concourse 2. Development of the Q-Set 3. Selection of the Participant Set 4. Q-sorting 5. Analysis and interpretation (Van Exel & de Graaf, 2005) Definition of Concourse First step is defining the concourse The flow of communicability surrounding any topic is referred to as a concourse The concourse is usually drawn from a wide range of sources, the ordinary conversation, commentary, and discourse of everyday life (Brown, 1993, p.94) Interviews, casual conversations, newspapers, magazines and documents are all possible sources for a concourse Statements can also be drawn from academic and theoretical literature Definition of Concourse In this study, due to time constraints, collecting subjective statements was unrealistic Instead, a concourse was created using theoretical underpinnings Guided by: conditions & environment that promote or hinder growth and development which could mediate the ministers efficacy and resilience the outcomes that would indicate the ministers efficacy and resilience Definition of Concourse To evaluate the subjective experience of coaching & mentoring environment statements were drawn from: Professional competencies as outlined by ICF (ICF, 2015) Rogers conditions for growth (Rogers, 1967) Herons interventions (Heron, 2001) Herons degenerative interventions (Heron, 2001) Krams mentoring functions (Kram, 1985b) Eby et als Meta-themes of negative mentoring (Eby, et al., 2000) Definition of Concourse To evaluate the impact of coaching & mentoring statements were drawn from: Objectives & outcomes of coaching/mentoring (Allen, et al., 2004; Eby & Lockwood, 2005) Burnout Engagement Dimension (Maslach, et al., 2001) Areas of Worklife (Maslach, et al., 2001) Resilience CD-RISC Scale (Conner & Davidson, 2003) Development of Q-Set from Concourse Final concourse resulted in 61 statements: 36 statements for the experience of coaching/mentoring 25 statements for the impact of coaching/mentoring Recommendation to generate a large concourse, which can be refined and reduced to produce a manageable Q-Set The Q-Set is the set final set of statements presented to participants that they rank order onto the sorting grid Two characteristics make a good Q-Set: Broadly representative of all possible subject positions Balanced not biased Development of Q-Set from Concourse Paring down of Concourse is done by piloting the statements in a Q- Sort procedure Piloting highlights the pertinent issues concerning the concourse statements and aids with reducing and refining a Q-Set: Statements with double or multiple propositions Statements in essence repeating others Statements with negative proposition Procedure too long and cumbersome The final Q-Set consisted of 25 statements covering both the experience and impact of coaching/mentoring, which were to be sorted onto the grid shown earlier Selection of Participant Set Selected on basis of typical case sampling (Bryman, 2012) Research question demanded the sample was restricted to those in pastoral ministry or training for pastoral ministry Three contexts: Elim Pentecostal Church London School of Theology Personal contacts Q-Sorting Procedure To be able to reach these geographically dispersed participants I created an online procedure Consisted of three elements of data collection: Demographic information of recipient & coach/mentor Design and delivery of coaching/mentoring Matching Context Configuration Who commissioned C/M Training or orientation Length of alliance Clarity of purpose, location, frequency, etc Ending Q-Sorting procedure Analysis 45 respondents producing 45 Q-Sorts Q-Sorts subjected to factor analysis to identify the most common subjective viewpoints Done by entering Q-Sorts into a computer programme PQMethod 2.35 (Schmolck, 2014) A centroid analysis is performed to extract factors Further probing done by rotating the data to further sharpen and focus each factor In final analysis: three-factor solution seemed a best fit Two of these factors were bi-polar bi-polar opposite experience Factors identified Factor 1+Skilled, Challenging & Transformative Factor 1-Unskilled & Manipulative Factor 2Directive, Heavy-handed & Educational Factor 3+Supportive & Restorative Factor 3-Unsupportive, Manipulative yet Educational Interpretation For the majority (75%) coaching & mentoring was beneficial and satisfactory (Factors 1+ and 3+) For some (8%) coaching & mentoring delivered some benefits, but was not necessarily satisfactory (Factors 2 and 3-) For one participant (2%) coaching & mentoring was neither beneficial nor satisfactory (Factor 1-) In Q post-sorting interviews can be used to add depth & richness to the Factors Unrealistic in this research most not face-to-face even though comments were invited Instead attending to the demographic and design data identified the pertinent issues relating Factors and design Relevant themes to Factors & Design Continuum of Learning: instrumental transformative Continuum of Ownership of Learning: Organisational Individual Ownership Support for stress amelioration & resilience building Skill of Coach/Mentor: expertise as minister skill as coach/mentor Context & Purpose Unsatisfactory coaching/mentoring & toxicity Design Implications Clarity on context & purpose Nuanced approach to matching taking into account: Context and purpose Continuum of learning Continuum of ownership of learning Nuanced approach to delivery of coaching/mentoring: Formal C/M - yet imitating informal C/M to foster self-directedness Informal C/M - yet aided & facilitated by organisation to maximise effectiveness and protect from toxicity Training themes and issues identified in this study could be used to foster effective coaching & mentoring programmes and culture Q Methodology as Evaluation Tool Research demonstrates Q Methodology as a useful evaluation tool Although significant preparation is required to design the Concourse and the Q-Set, I found the Q-Sort elegant and easy to administer Approximately 10 minutes to perform the Q-Sort Procedure elicits rich &informative qualitative data in minutes Development of computer software means calculations possible Implications for Practice The Q-Set developed for this research could be used to evaluate coaching/mentoring interventions Other Q-Sets could be designed for different purposes and foci, for example, designed to evaluate experiences of coaches/mentors A singular Q-Sort could be undertaken and evaluated on its own merit A coach/mentor could use a Q-Sort procedure as an evaluation tool for their own practice Limitations The findings of Q Methodology are not considered to be generalizable (but are considered stable) Due to time constraints the concourse was produced theoretically rather than by collecting subjective statements which may have influenced the factors identified Some participants found the forced ranking difficult and this may have put off those who had conflicted experiences This study only considered the perspective of the recipients of Coaching & Mentoring and did not include the perspectives of Coaches and/or Mentors or the sponsoring Organisations Further Research Longitudinal study could be undertaken to appreciate how these coaching/mentoring environments might correlate with objective outcomes Useful to create a concourse sourcing subjective statements in the more traditional manner of interviews, conversations, etc might identify other factors Different Q-Sets could be generated to evaluate different aspects i.e. coach/mentor behaviours, or from coach/mentor perspective