FORCE FIELD ANALYSIS - Clearwater · PDF fileClearwater Evaluation Workshop: EVALUATION TOOLS...

Clearwater Evaluation Workshop: EVALUATION TOOLS

FORCE FIELD ANALYSIS Source: Context Pty Ltd, evaluation consultants to the Clearwater Program

The “Force Field Analysis” was developed by Kurt Lewin in the 1970s and is one of the most useful open-ended frameworks for problem solving. The Force Field Analysis look at the ‘drivers’ and ‘contratints’ for a particular issue or project. This tool enables you to actively involve stakeholders in identifying these processes, and provides useful data for evaluating your project. Stakeholders that can affect the success of any given project can add significant complexity. This complexity can either hinder or help a project. Clearwater Application: The Force Field Analysis was applied in the early stages of developing the Clearwater Program. It was conducted six times across the state as part of the Clearwater Roadshows involving participants in assessing the drivers and constraints for implementing sustainable urban water management in Victoria. The data provided a useful framework for designing a capacity building program to support sustainable urban water management in Victoria. How To Construct A Force Field Analysis For Your Project Draw up two columns on the whiteboard or flip chart / butchers paper, with the two headings “Drivers” and “Constraints” (as below):

Ask the participants to brainstorm all the driving that are driving the implementation of “best practice evaluation techniques”. Then, similarly, brainstorm the factors that are constraining the implementation of “best practice evaluation techniques” (examples next page).


These could include:

��Individuals or groups;

��Procedures, structures, policies, regulations, reporting arrangements;

��Availability of information or data;

��Technologies;

��Culture – tradition, status, hierarchy, decision-making;

��Systems or processes, how performance is measured, etc.

��Current economic or business conditions, legislation and conformance. The main thing is not to edit or limit any suggestions yet. Include all the driving or restraining forces you can think of at this stage. Once you have the two lists, give participants a few minutes to review the points and prioritise these factors, deciding which they think are most important then:

��Each participant is then given TWO votes for each column (2-3 votes depending on the number of participants).

��Run through each of the drivers, and ask the participants to raise their hand to vote for their choice.

��Write the number of votes for each driver on the board.

��Repeat this process for the constraints. Note: Keep a copy of the output for later analysis.

This process should be completed in 20-30 minutes. Analysis The information generated can be useful in understanding barriers and drivers for your stakeholders. In addition, this information can be further analysed by assessing the relative strength of each of the forces. Allocate a total of 100 points to the driving forces side and 100 points to the restraining forces side. Using the top five forces identified (based on the votes) assign a weighting to each. Depending on how you perceive their criticality it is possible that a single force might receive a weighting of as much as 90. Review the RESTRAINING forces you have identified:

��Which of these should you dismiss as not being amenable to change?

��Which of the restraining forces are most amenable to change?

��Which of the restraining forces are not only amenable to change, but have also been assigned a significant weighting?

��Of those restraining forces you have identified as both amenable to change and significant, which do you think you should attempt to influence?


FISHBOWL FORUM QUESTIONNAIRE Developed by R. Brown, C. Chesterfield and J. White, 2005

Clearwater Application: The Fishbowl Questionaire was developed as an online survey tool for the Clearwater “Water and Sustainable Development – Tools for Change” seminar. It was designed to target a range of stakeholders in the water sector to understand their role in sustainable urban water management and their assessment of their ’s performance stakeholder group compared to other stakeholder groups. The survey data was disseminated as part of the event in the form of a presentation and fishbowl forum. This enabled participatns to debate the responses and and futher analyse the data through a facilitated discussion. Evaluation Tool: Please take a few minutes to respond to the questions below. Your feedback will be used to structure the ‘Fishbowl Forum’ – the afternoon session of the Water and Sustainable Development seminar. DEFINITIONS:

‘Sustainable urban water management (SUWM)’ – involves the management of the total water cycle, including wastewater, water supply, stormwater and groundwater resources.

Question 1: Which area of water management do you primarily work in? (please select) ��stormwater / waterways ��domestic wastewater ��land development ��water supply / sewerage ��Other (please describe): Question 2: What stakeholder group do you primarily represent? (please select)

��Local Government ��Water Authority ��Consultants ��State Regulatory Authority ��Developer ��Research Institution ��Other (please describe): Question 3: How many years have you been involved in water management? (please select):

��0-2 ��3-5 ��6-10 ��11-20 ��21-30 ��31+


Question 4: What do you believe to be the top 3 barriers to meeting the challenge of sustainable urban water management? Please select 3 barriers only, and then rate each barrier 1, 2 or 3, where 1= most significant barrier

��Lack of community awareness and support of the need for SUWM ��Insufficient information on SUWM options and management alternatives ��Insufficient technical information on the operation and maintenance of SUWM practices ��Insufficient information on how to estimate the capital and maintenance costs of SUWM alternatives ��SUWM is not economically viable ��Lack of standards to guide the detailed design and regulatory approvals processes ��Lack of political will to support the improved on-ground implementation of SUWM ��Lack of appropriate regulatory and administrative support (such as the incorporation of WSUD

obligations in local planning schemes) for the on-ground implementation of SUWM ��Insufficient professional capacity across the water sector to design sustainable urban water

management practices ��Insufficient organisational capacity (ie local government and regulatory authorities) to assess and

approve proposed schemes ��Sustainable urban water management is still too risky – there is not enough evidence to

demonstrate that it is effective and reliable in the long term.

��Other (please describe):

The following two questions ask you to rate each stakeholder’s:

• commitment to advancing the principles of sustainable urban water management, and then; • level of on-ground implementation of sustainable urban water management practices.

“Stakeholders” relates to local government, water authorities, state regulatory authorities, developers and land development consultants. Question 5: Rate the level of COMMITMENT of each of the following stakeholder groups to

advancing the principles of sustainable urban water management. Please rate from 1 to 6 as suggested: While there can be diversity within the stakeholder types please rate each as a whole to the best of your working knowledge.

1= not committed 2= some commitment from individuals – but not organisational commitment 3= some organisational commitment and senior support 4= major organisational departments and internal champions committed 5= organisation fully committed and advocates for sustainable urban water management 6= don’t know

Local Government Water Authorities

State Regulatory Authorities Developers

Other (describe): Consultants


Question 6: Rate the extent of on-ground IMPLEMENTATION of sustainable urban water management by the following stakeholder groups. Separately rate greenfield and urban renewal practice from 1 to 6 as suggested: While there can be diversity within the stakeholder types please rate each as a whole to the best of your working knowledge.

1= none 2= one-off projects (primarily traditional water management) 3= ad-hoc projects (some level of sustainable implementation but mainly traditional water management) 4= majority of major projects integrating sustainable urban water management 5= wide-spread implementation as part of core business 6= don’t know (a). Greenfield:



Other (please describe): Consultants

(b). Urban renewal:



Other (please describe): Consultants

Question 7: What do you consider to be the single most important initiative needed to substantially improve the on-ground implementation of sustainable urban water management?

REQUEST FOR YOUR INVOLVEMENT: The last session of the seminar will be a facilitated “fishbowl forum”. This is an opportunity for you to contribute to a public discussion and debate based on responses to the above questions. For example, comparing the different perspectives of individuals within their place of work with their perception of other stakeholder groups. I am willing to participate in the fishbowl forum: �� Yes – Fishbowl forum participant

�� No – Observer only

THANK-YOU FOR YOUR FEEDBACK

All survey responses are anonymous and the identification of participants will remain confidential.


CLEARWATER LEADERSHIP PROGRAM Source: Context Pty Ltd, evaluation consultants to the Clearwater Program

Clearwater Application: This evaluation tool was developed for the Clearwater Leadership Program to benchmark and map the confidence and skills of participants, identified as the future leaders (‘champions’) of the water sector. The tool is used at each Leadership event as it provides solid benchmark data on:

��the preferred choices champions apply on a regular basis to gather support for change;

��the range of choices or tactics they deploy; ��the similarities or patterns of tactical choice across the broader audience of

co-ordinators. The advantage of this approach is that the evaluation tool can be used on further occasions to build up a picture of the range of and change in skills and tactical approaches of champions. This provides comparative data that can be used to determine the effectiveness of the Leadership Program, and the degree of capacity improvement. This information can then be factored into future program design at several levels, from the design of future leadership programs, to strategic choices relating to the design and deployment of Clearwater program components generally. It also provides information, which can be presented to the Clearwater Steering Committee as a way of identifying future program activities. Confidentiality: All of the information gathered through questionnaires such as this is confidential. No individuals can be identified. No individual data finds its way back to employers. All data analysis is conducted off-site by the evaluation consultants, and is only presented back to the Clearwater program in a collated report format. It is important to communicate this to your participants prior to completing the evaluation form. Question 1: Tasks To Support Change In Behaviours Or Outcomes There are three broad areas of activity that define the role of champions in a typical program where one is trying to change behaviour or outcomes:

Interpersonal: all those activities that involve working with people to support the change;

Informational: those activities relating mostly to communication and information, either sending or receiving;

Decisional: those activities that involve analysing information, identifying alternatives and making choices.


Successful program delivery requires a good balance of these three areas. This evaluation tool is intended to help you, and Clearwater, identify what is the balance of these activities required for champions working in sustainable urban water management. This information will assist in designing further materials and programs to build the confidence and performance of people in similar roles. Instructions Below are 12 statements, four under each of the activity areas described. Beside each of the statements are two scales; the first identifies the need for this type of activity in your job, the second asks you to identify how confident you feel to undertake that particular activity. Use the scale to record how much you agree with each statement by circling one of the numbers that corresponds with your situation. For example:

My job has a big need for this 0 1 2 � 4 5 Meeting with influential people in my organisation to encourage support for my program. I am confident of my skills in

this area 0 � 2 3 4 5

1. INTERPERSONAL

My job has a big need for this 0 1 2 3 4 5

Meeting with influential people in my organisation to encourage support for my program.

I am confident of my skills in this area 0 1 2 3 4 5


Getting staff or stakeholders together to discuss the program.



Encouraging other agencies, departments or organisations to support the program.



Clarifying the project’s priority in the organisation – giving it visibility.


N/A Completely

trueCompletely untrue

N/A Completely true

Completely untrue


2. INFORMATIONAL


Representing the program at outside events/ presentations.



Developing and disseminating brochures, newsletters, reports, articles, etc.



Reporting about program progress to others in the organisation, especially Executive Management or Steering Committees.



Sharing concerns, analyses of project results with staff, managers or supervisors


3. DECISIONAL

My job has a big need for this 0 1 2 3 4 5 Searching for additional funding or resources for the program.


My job has a big need for this 0 1 2 3 4 5 Developing a budget and allocating resources for the program.


My job has a big need for this 0 1 2 3 4 5 Designing appropriate new directions or goals as the program unfolds (eg. refining outcomes, negotiating improvements, etc.)


My job has a big need for this 0 1 2 3 4 5 Removing organisational barriers to implementing the program.


N/A Completely

trueCompletely untrue

N/A Completely true

Completely untrue


Question 2: Feedback – Clearwater Leadership Program

Poor Average Excellent

1 2 3 4 5 6 What you learned

How much has this day helped you learn about your own approach to leading and the leadership approaches of those around you?

What can be applied

How valuable was the day in terms of helping you think about leadership approaches that can be applied back in your own work situation?

Personal challenge

`How much did the day challenge you to think about your personal approach to leading a change initiative

The facilitators

Facilitation style

Capacity to pull together ideas and comments

Overall effectiveness

Overall Evaluation

Relevance

Focus on helping you apply what you have learned in your organisation

Value for time invested

What aspect of the Clearwater Leadership Program did you gain MOST value? What aspect of the Clearwater Leadership Program did you gain LEAST value? Are there any topic areas that you would like to see included in future programs of this sort? General comments?


WHO CARES ABOUT THE ENVIRONMENT? Source: Context Pty Ltd, evaluation consultants to the Clearwater Program

Clearwater Application: This evaluation tool was developed for a Clearwater seminar/workshop. Many of the participants were new to Clearwater and represented a huge diversity of professions, stakeholder groups and focus of environmental / sustainability issues. The evaluation tool devleoped therefore had to be generic to cover this diversity yet still produce reliable and useful data for Clearwater. Evaluation Tool: 1. What is your main job role? (Please tick one only):

� Environment / Education Officer � Planner � Other__________________

� Researcher � Engineer

2. What sector do you primarily work in? (Please tick one only): � Local Government � Water authority � State Government � Private industry

� Non-Government Organisation � Other: ________________________________

3. How would you rate the usefulness of the seminar and workshop? (Circle one only)

(Poor) 1 2 3 4 5 (Excel lent )

4. Will you be able to use the information shared today in your work? (Circle one only)

Yes A l i t t le No 5. How could Clearwater assist you in further integrating today’s information into your work?

� Further events � Email network

� Resources � Web based discussion group

� Other (Please describe): ____________________________________________________________

6. Have you visited the Clearwater InfoExchange website: www.clearwater.asn.au ? (Please tick)

� No � Yes

Usefu l components 1.___________________ 2 . ___________________ 7. How did you hear about this event? (Please select one only)

� Clearwater Email � Website � Work colleagues/managers

� Other (please describe): __________________________________________________________

8. Additional comments?


WATER SENSITIVE URBAN DESIGN ROADSHOWS “DESIGNING VICTORIA”

Developed by Context Pty Ltd, S.Lloyd and J.White

Clearwater Application: The Clearwater WSUD Program aims to facilitate the delivery of information, education and training to local government and industry professionals in sustainable urban stormwater management. This evaluation questionnaire provides you with the opportunity to tell us what training topics are important to you, and how you would like them delivered. Your responses will be used to guide us in the development of the Clearwater WSUD Program – training content and calendar. Evaluation Tool: Question 1: The table below contains a list of key WSUD issues identified through earlier local government and industry consultation. We are proposing to address these issues through the Clearwater WSUD Program. We would appreciate your input into which issues areas are of greater importance to you.

Please take the time to read through the list, and number your top 5 issues (1=HIGHEST, 5=LOWEST).

Priority Issue Lack of collaboration within and between industry

Incorporating WSUD planning provisions in local planning schemes

Enforcement and compliance

Sharing of experiences and information dissemination

Dealing with conflicting stakeholder views in developing a concept design

Lack of standards to guide detailed design and approval process

Construction issues, including improved site practices

Insufficient information on operation and maintenance

Performance indicators - how do we know its working?

WSUD principles & practices – technical skills to design, assess or approve schemes

Dealing with split economic incentives – investors versus those who benefit

Costs – Estimating capital and maintenance costs

Cost/Benefits – Procedures for assessing environmental & social costs/benefits

Community attitudes and perceptions to WSUD

Marketing tools

Other (describe):


Question 2: The following topics are currently proposed to be part of the Clearwater WSUD training calendar. Please indicate your top 4 areas of interest, with a rating of 1 indicating that you wouldn’t miss it for anything!

Case studies: hurdles, challenges, innovation & success

Developing concept and detailed technical designs

Dealing with construction, operation and maintenance issues

Costing procedures available and their application

Amending regulatory frameworks

Applying assessment tools (eg. MUSIC training)

Engaging the community and marketing WSUD

Question 3: Do you feel there are any important areas we have missed that should be included into our seminar program?

Question 4: To assist us in delivering an effective, tailored Clearwater WSUD Program, what approaches to training and communication would you prefer? Please indicate your top 3 preferences (1=HIGHEST, 3=LOWEST).

Presentations from peers about their own experiences - organisational barriers (policy, funding, attitudes), managing change and external parties

Provision of information in document form (hard copy)

Provision of information via web

Site visits – field demonstrations

Workshops - apply theoretical learning to real situations

Illustrated lecture style presentations (like today!)

Name: (optional) Position or occupation:

Thank-you for your feedback


STORMWATER ACTION SURVEY Source: Context Pty Ltd, evaluation consultants to the Clearwater Program

Clearwater Application: This evaluation tool was developed for the Clearwater “Stormwater and Sustainability in Industrial Areas” workshop. It was designed to assist Clearwater assess the capability and interest of participants in the workshop theme, and in addition, the capability and willingness of participants to establishing an ongoing network. Evaluation Tool: Today has been about providing information to participants about approaches and resources that can be mobilised to improve stormwater quality in industrial areas. The questions below will help Clearwater to understand how effective today has been for you. They will also help Clearwater learn more about the skills and interests of participants with regard to action on Industrial Stormwater Quality. Question 1: Skills and Interests Please take a few minutes to complete the survey. Circle the number between 0 and 5, which most accurately reflects how each statement applies to you.

0 = this statement is irrelevant 1 = this statement is never true for me 2 = this statement is rarely true for me

3 = this statement is sometimes true for me 4 = this statement is often true for me 5 = this statement is always true for me

I have expert knowledge on industrial stormwater issues which I can use to help others

0 1 2 3 4 5

I am able to help people set up education programs by accessing resources or funding

0 1 2 3 4 5

I am in a position to do something positive to improve the way my organisation deals with stormwater

0 1 2 3 4 5

I know what needs to be done to get better quality stormwater in industrial situations

0 1 2 3 4 5

I know where to go to get help on improving the quality of stormwater in the areas for which I am responsible

0 1 2 3 4 5

I am responsible for helping others develop a better response to stormwater quality issues

0 1 2 3 4 5

I am responsible for delivering knowledge or technology to organisations that want to improve the way they deal with stormwater

0 1 2 3 4 5

I am already involved in a stormwater education program for industrial situations

0 1 2 3 4 5

I can help put people who want to do something about stormwater quality in their organisations in touch with people who can help them resource and deliver an education and improvement program

0 1 2 3 4 5


Question 2: Characteristics Of A ‘Capable’ Network Understanding capability is a more tricky aspect of evaluation as capability is something, which emerges over time. However, this checklist technique will, in the first instance, provide a baseline assessment of what it takes to make a successful network. Using a checklist will also help answer the background question of what education and training resources are required (i.e. is establishing a network a useful, or more appropriate tool than, for example, roadshows or demonstration site visits?) Please answer yes ‘Y’ or no ‘N’ to the following questions (circle):

1. Is there a capacity to get administrative things done:

a. arrange meetings; Y / N

b. take minutes and record action points; Y / N

c. undertake secretarial activities.

Y / N

2. Can the group identify and access expert support:

a. from local and state government sources; Y / N

b. from private practitioners and consultants.

Y / N

3. Do those involved have:

a. time to spend in a network dedicated to stormwater information; Y / N

b. the capability to arrange funding; Y / N

c. access to decision-makers; Y / N

d. capacity to mobilise others; Y / N

e. a commitment to the topic. Y / N

FORCE FIELD ANALYSIS - Clearwater · PDF fileClearwater Evaluation Workshop: EVALUATION TOOLS...

Documents

Transcript of FORCE FIELD ANALYSIS - Clearwater · PDF fileClearwater Evaluation Workshop: EVALUATION TOOLS...