Evaluator Personality Preferences – Implications for Influencing Evaluation Design

and Utilisation

Eve Barboza

Wholistic Learning Pty Ltd

Contents

• Abstract• Defining Programs Projects and Evaluation• Project Management and Evaluation• Human Dynamics and Project Management

Challenges• Personality Preferences• Relative Importance of Team Activities• Personality Preferences and Project Types and

Teams

ABSTRACT: This 60 minute mini workshop will explore the questions can the personality preference of the evaluator influence the design and utilisation of evaluation? Moreover can any differences in these personality preferences between evaluator and client / audience of the evaluation explain some of the controversies in evaluation practice? This will be a highly interactive session that investigates two possible approaches to using personality preference in evaluation practice. Firstly how personality preferences could be drawn on to inform the design of evaluation will be explored. Secondly, how personality preferences could influence the implementation and utilisation of evaluation findings will be discussed. This discussion will draw on some positive and negative experiences of the presenter and any that the participants wish to share. Participants will have the opportunity to briefly identify their personality preferences and explore the role that these preferences may play in shaping the design of evaluations and the development of key evaluation questions.

Program / Project Definitions

Program: Any set of procedures, activities, resources and /or management strategies designed to achieve some common goal(s) or objectives. Note that no arbitrary limit is set on how wide or narrow the procedures, activities (etc.) have to be. The interests of the observer set the boundaries of the program at a particular time.

Project: a temporary endeavour undertaken to create a unique product or service – outcome driven, the ultimate product or service assessed against performance measures that determines project success

Defining Program / Project EvaluationThe process of making judgements about a program’s or project’s

• appropriateness • effectiveness • efficiency• objectives• client needs• procedures• activities• resources• management strategies• outcomes

• APPROPRIATENESS – Does the project or program objectives address client needs and are project or program operations acceptable to clients?

• EFFECTIVENESS - Did the project or program achieve its results or outcomes and cause the desired impact?

• EFFICIENCY - Did the project or program make efficient use of resources - can outputs be achieved more cheaply?

Project Management and Evaluation

Evaluations of program or projects are usually conducted as a project with the following features:

• Every project has a definite beginning and a definite end

• Projects desired product or service is different in some way from other products and services that the organisation produces

• A project team seldom outlives the project – once the project has been completed, the team is disbanded and members move on to other projects and other teams

Evaluation Project Management (1)

Initiation Phase• The project is defined and authorised and

generally results in a project charter, a needs statement, and outcome measures

• Often handled by the project sponsors – those who are commissioning the project – who also select the project manager

• Deliverable: Project charter establishes the mission, driving goals, scope, boundaries and objectives of the project

Evaluation Project Management (2)

Planning Phase• The team translates the charter or needs statement

into a project plan with milestones, tasks schedule, and resource assignments

• The project team is usually selected and responsibilities are assigned during this phase

• Deliverables: Project management Plan and Work Breakdown Structure – outlines tasks, activities and deliverables; the resources assigned against them the time projected to complete each; and the milestone deliverable that will result

Evaluation Project Management (3)Execution Phase• The project manager mobilises both people and

resources to implement the plan.• Team members take actions and complete tasks

that help achieve milestone delivery using the resources assigned

• Deliverable: Organisation chart – describes the roles and relationships between people on the project

• Deliverable: Stakeholder Management Plan – describes how relationships and communication with internal and external interests are initiated, maintained and ended

Evaluation Project Management (4)Monitoring and Control Phase• The team, sometimes assisted by external independent

reviewers, measures and monitors progress against the plan to identify any corrective action needed to keep the project on track

Deliverables:• Risk Management Plan – addresses how risks will be

managed• Risk Matrix – tracks risks through the project life cycle• Status Reports – Regularly produced by the project

team – typically weekly, monthly, or quarterly – these summarise progress made, interim deliverables, and key risks and task interdependencies identified since the previous report

Evaluation Project Management (5)

Closing Phase• The team formalises delivery and acceptance of the

final service or product and brings the project to an end

• Ideally, this includes a project review meeting (often called a ‘post mortem’) where best practices and lessons learned that can be carried on to other projects are discussed

• Deliverable: In evaluation projects these are typically reports with key findings and recommendations. Occasionally there is a presentation of the report to project sponsors who commissioned the evaluation project

Ideal vs. Actual Project Management

• Idealised version of phases and activities represent best practice across multiple projects over time

Actual projects• In the rush to begin the

“real” work of a project, the initiation and planning phases are often collapsed into one

• Under pressure to stay on schedule and within budget despite changing needs the monitoring and control phase can turn into crisis management

Human Dynamics & Project Management

Conflict on a project is often a direct result of:• Scope creep: uncontrolled growth of product/

service needs without an increase in time and budget

• Lack of role clarity: uncertainty as to who holds accountability for what, and how to describe different roles in terms of project objectives

• Misalignment of authority and responsibility: giving responsibility for tasks to those without the authority to act

Common Project Challenges

• Inability of project team to succinctly communicate the project’s mission and driving goals

• Poor client / customer management and communication skills

• Failure to manage scope creep• Failure to differentiate between stakeholder groups• Lack of team and meeting management skills• Overemphasis on risk management

Benefits of working with Personality Preferences

• Provides a framework and process that can quickly yield greater self-awareness, which can lead to better self management

• Establishes a shared vocabulary for an important set of mental activities, allowing people to communicate more effectively about needs, expectations and preferences

• Helps people become aware of these differences and through the appreciation of those differences, reduce conflict

• Applicable to a wide range of both personal and work place functions, including leadership, team development, conflict management, decision making, problem solving and customer service – as well as project management

• Reveals possible strengths and blind spots for both individuals and team, in a way the remains positive and development orientated

Personality Preferences - MBTI

• Remember that the MBTI and personality preferences are not a test there are no ‘right’ or ‘wrong’ answers or types.

• Furthermore personality preferences is not an indicator of abilities, likelihood of success, intelligence or skills.

• People of any type can be successful in a variety of project management of project delivery roles

• Therefore the MBTI assessment should never be used as a tool for selecting employees, assigning tasks or evaluating individual performance

Extraversion (E) “Lets talk about it”

People who prefer Extraversion• Are energised by the external world of

people, activities and things• Prefer to talk things through with others – an

idea isn’t real until it has been expressed to and discussed with others

• Tend to become drained when working alone prefer to interact with people

• Process information externally, this what they say may not be the final word – or even the intended message

Introversion (I) “Let’s think it through”

People who prefer Introversion• Are energised by the internal world of ideas and

experiences• Prefer to think things through before sharing with

others – an idea isn’t real until it has been well considered internally

• Tend to become drained by excess interaction with others, preferring to work independently

• Process information internally, thus may not sufficiently share their thinking or decisions with others

Sensing (S) “Let’s look at the facts”

People who prefer Sensing:• Prefer specific information – a precise and

detailed view• First perceive literal, concrete, and

sequential details about the here and now• Generally begin with facts and figures, which

is then combined to create a larger picture• Value past experience and a focus on what is

actually known

Intuition (N) “Let’s look at the possibilities”

People who prefer Intuition• Prefer general information – a view of the big

picture• First perceive the patterns and connections

in data received• Generally begin with the possibilities and

meaning underlying information and fill in the details later

• May present information using metaphor, valuing abstraction and a theoretical spin

Thinking (T) “Let’s keep this objective”

People who prefer Thinking:• Prefer decision making that is objective.

Logical and cause-effect orientated• Tend to consider the problem or issue first

and the impacts on people second• Generally favour impartiality and analysis,

even if it causes conflict• Believe it’s more important to be right than

liked

Feeling (F) “Let’s focus on the people”

People who prefer Feeling:• Prefer decision making that is subjective,

empathetic and values orientated• Tend to consider the people involved first and

the problem or issue second• Generally prefer to be appreciative and

maintain harmony• Avoid conflict wherever possible to keep the

peace

Judging (J) “Let’s get to closure”

People who prefer Judging:• Make decisions with the goal of achieving

closure• Are directive and strive to maintain order in

the external world• May initially resist new information when it

disrupts a plan or schedule• May change their mind or direction after

reflection and internal consideration of new information

Perceiving (P) “Let’s keep our options open”

People who prefer Perceiving:• Communicate their perceptions to others,

with the desire to remain open• Are generally flexible, adaptable, and

nondirective with others• Generate options easily – the more

discussion, the more options• Often make decisions internally, meaning

that sometimes those decisions aren’t shared with others

Relative Importance of Team Activities• Extraverted (E) teams tend to rank “conduct project

team meetings” higher that do Introverted (I) teams• Sensing (S) teams tend to rank “review the financial

status” higher than do Intuitive (N) teams• Feeling (F) teams tend to rank “conduct one on one

meetings with key project team members” higher than do Thinking (T) teams

• Judging (J) teams tend to rank “assess progress of activities against he work breakdown structure” higher than do Perceiving (P) teams.

• Perceiving (P) teams tend to rank “respond in real time to emerging project crisis” higher than do Judging (J) teams

Extraverted (E) Project TypeTeam members may:• Spend lots of time in

meetings to explore ideas, discuss process and review progress

• Actively engage external stakeholders groups to sell the project or involve them in the process

• Rely on verbal communication techniques more than written ones

When Extraversion is over emphasised members may:

• Struggle with scope creep and expectation management because ideas are processed externally, audiences may assume a decision has been made when ideas are really just being explored

• Engage in too many meetings and excess outreach, leading to confusion between discussion and action.

• Take on too many tasks with insufficient depth due to their wide breadth of interests

Introverted (I) Project TypeTeam members may:• Work according to an

individual contributor model

• Have more structured communication and rely more heavily on e-mail and written reports

• Like to have things figured out before sharing externally

When Introversion is over emphasised members may:

• Assume that others know what is being worked on, or that they do not need to know, due to the fact that project tasks or work streams may be sliced.

• Make changes and decisions in small groups or between individuals, which may not be adequately communicated across the project to all those affected

• Assume rather than actively solicit, customer needs and desires leading to poor expectations management

Primary Risk and Possible Response

Extraversion (E)• Over communication may lead to

confusion, scope creep and wasted resources

• Ask regularly: “Do we really need to talk about this?”

Introversion (I)• Under communication may lead to

assumed buy in and surprises due to inadequate involvement of others.

• Ask Regularly: “Who needs or would want to know about this?”

Sensing (S) Project TypeTeam members may:• Want to discuss specifics about

the project’s current state and the practical benefits the end product or service will provide

• View the big picture by stacking individual building blocks

• Show an interest in benchmarking against what other projects have done, trusting past experience and concrete evidence and focusing on efficient business processes

• Have a clear set of performance measures that define project success and actually measure progress against them

When Sensing is over emphasised members may:

• Analyse the scope and work breakdown structure in such find detail that coordination of the intricate schedule becomes a project unto itself

• Miss possible opportunities to streamline across tasks thus time is spent fighting fires without identifying the underlying cause

• Struggle to succinctly state the project’s overarching vision and future possibilities – lack of buy in by external stakeholders

Intuition (N) Project TypeTeam members may:• Use big-picture visionary

descriptions of the project, as well as metaphor and figurative language

• Start projects with a big-picture concept and fill in the details later

• Develop new, innovative processes, technologies, and approaches for managing work rather than relying on established policies and procedures

When Intuition is over emphasised members may:

• Craft a project that addresses tomorrow’s problems or future possibilities but not today’s practical need

• Underestimate the effort involved in individual tasks and over conceptualise or reinvent the project’s scope rather than deliver on it

• Rely on a small set of general signs of progress and not be able to point to clear outcomes or products

Primary Risk and Possible Response

Sensing (S)• Too much emphasis on the here and now

may result in missing new possibilities and patterns that emerge from the data

• Take the time to ask: “What does it all mean? What are the trends?”

Intuition (I)• Too much theory and not enough hand-on

action can result in missing the delivery and communication of concrete outcomes

• Take the time to focus on present needs and think through the details

Thinking (T) Project TypeTeam members may:• Use a clear set of objectives

criteria for decision making and for making trade-offs in scope, schedule and budget

• Engage in logical cause-effect analysis with formal decision making tools, particularly in the area of risk management

• Have a stakeholder management plan but approach stakeholders from a “problem first, people second” perspective

• Give an impersonal, problem-focused demeanour to the project, encouraging critique and debate

When Thinking is over emphasised members may:

• Neglect the need to “sell” the project to audiences and may underestimate the impacts of change on the ultimate customers

• Lump various stakeholders into one group and not acknowledge the specialised interests of distinct audience segments

• Neglect personal, interpersonal and team development needs of members, risking team dissatisfaction and a lack of connection and team loyalty

• Fail to align project assignments with team member interests risking having people leave for better opportunities

Feeling (F) Project TypeTeam members may:• Engage in congenial discussions

that stress inclusion, collaboration and consensus building

• Engage in decision making that is customer and team centred and that differentiates between stakeholder groups

• Determine project assignments in part by professional development interests and what team members like to do

• Be unlikely to embrace risk management as a practice, and may avoid topics that likely involve conflict or blame external parties to shift the conflict elsewhere

When Feeling is over emphasised members may:

• Consider team cohesion to be a measure of project success overall – result in discouraging dissent risking the loss of better outcomes through debate and even conflict

• Make decisions on the premise that the project is unsuccessful if the sponsors and users aren’t happy. - Scope creep trying to please everyone

• Deliver against the desires of a representative user rather than objective specification driven by project scope and resources

• Avoid assigning unpleasant tasks, leaving important actions undone

Primary Risk and Possible Response

Thinking (T)• Focusing too much on the product may

result in ignoring people’s needs• Ask how both team members and

customers might perceive decisions and directions

Feeling (F)• Focusing too much on the people may

result in ignoring tough trade-offs• Ask how team members would decide if

they weren’t concerned about what other people think and feel

Judging (J) Project TypeTeam members may:• Use the work breakdown

schedule (WBS) as a central management tool

• Generally begin and end meetings on time and govern them through an agenda

• Conduct project discussions that seem more decision driven than process driven

• Attempt to even out the pace of work over time to avoid the need for a last minute rush of activity

When Judging is over emphasised members may:

• See project management as an end unto itself, where management activities overtake quality and content

• Discourage innovation (intentionally or not) by overemphasising adherence to schedule and the desire for minimised risk

• Appear rigid and inflexible to customer groups resulting in questions about whether they are listening to audiences

• Inadvertently encourage scope creep by making statements that sound like decisions but really aren’t

Perceiving (P) Project TypeTeam members may:• Treat the WBS if one exists

as a guideline – open to change as work proceeds

• Proceeds with projects that weren’t formally chartered but rather emerged from ongoing operations or other efforts ad became their own entity

• Appear generally comfortable with uncertainty – knowing it is business as usual anyway – and open to evolving needs as the project unfolds

• Experience intense periods of production and effort prior to key deadlines and delivery dates

When Perceiving is over emphasised members may:

• See the process and learning aspect of the project as being as important as its outcome, resulting in the product or service never actually being delivered

• Take action without overly stating the decision to proceed leading to confusion and questions about how a decision was made and who made it

• Be perceived by stakeholders as chaotic and disorganised which may lead to concerns about their ability to deliver

• Explore so many different options for filling the requirements that none can be completed before time and resources run-out possibly resulting in scope creep

Primary Risk and Possible Response

Judging (J)• The team may

ignore vital new information and resist tasks that add time now but save time later

• Ask what buffers will allow for new information and needs

Perceiving (P)• The team may jeopardise the future

completion of the project by exploring new options now

• Reduce the number of options to help push for closure

Personality Preferences and Project Teams

• Effective projects are ones whose team members recognise and maximise the benefits of their preferences while also integrating opposite preferences into project operations and processes

• Most effective project teams both recognise and balance all eight preferences mitigating the risks that can come from over- relying on any one preference