Evaluation Technical and Financial Plan Revised
-
Upload
bryan-otieno-amata -
Category
Documents
-
view
217 -
download
3
description
Transcript of Evaluation Technical and Financial Plan Revised
Evaluation Plan Impact of KLIP activities on governance and policy making in Kisumu
Date: 21st July 2014
Trim Ref #: 09/
Evaluation Manager:
Contact Person: Name:
Position title:
Contact Phone:
Business Unit:
Table of Contents
Contents
1. Background ......................................................................................................................... 1
1.1 Project/Program/Initiative Definition ........................................................................................... 1
2. Evaluation Governance ........................................................................................................ 1
2.1 Purpose of the Evaluation .......................................................................................................... 1
2.2 Core Evaluation Questions ......................................................................................................... 2
2.3 Evaluation Stakeholders ............................................................................................................. 2
2.4 Evaluation Management Structure ............................................................................................. 3
2.5 Evaluation Governance Group Membership .............................................................................. 3
2.6 Evaluation Governance Group Terms of Reference .................................................................. 4
2.7 Reporting Requirements ............................................................................................................. 4
3. Evaluation Design ................................................................................................................ 5
3.1 Evaluation Framework ................................................................................................................ 5
3.2 Information and Collection .......................................................................................................... 5
4. Evaluation Resources and Timeframe ................................................................................. 5
4.1 Budget and Staff Resources ....................................................................................................... 5
4.2 Evaluation Staff Roles and Responsibilities ............................................................................... 7
4.3 Time Schedule ............................................................................................................................ 7
Appendix 1 Evaluation Plan Checklist ........................................................................................... 11
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
1
1. Background
1.1 Project/Program/Initiative Definition
KLIP is part of a consortium of organizations funded by Swedish International
Development Cooperation Agency (Sida) and Mistra Urban Futures (M-UF) with the sole
aim of promoting sustainable urban development. KLIP therefore strives to achieve the
set out goals of Fair, Green and Dense city of Kisumu by incorporating two thematic
pillars of: a) Eco-Tourism and b) Market Places. The two themes aims at addressing
urban challenges like food insecurity, climate change, poverty, urban insecurity among
others within Kisumu city and its environs. The partnership between Kisumu LIP and
participating stakeholders seeks to overcome the threats to food security, urban
insecurity, and skewed distribution of factors of production in a changing environment;
exploring new ways of helping vulnerable urban communities adjust to global changes
and explore other factors of production other than fishing along the adjacent beaches.
This consultancy exercise is a result of the collaboration between KLIP Sida and Mistra-
Urban Futures and its main purpose will be to evaluate the flagship programs of Eco-
tourism and Market places under the perspective of the various stakeholders, who
include KLIP Trustees, Kisumu Local Government Leaders, Various Private Sector
organizations and the local residents. The whole process will establish the impact of the
activities since their inception and also model future activities based on the results
obtained.
2. Evaluation Governance
2.1 Purpose of the Evaluation
The purpose is to identify the societal impact that has been achieved so far, or has the
potential to be achieved, through the support of KLIP project and capacity building
activities
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
2
2.2 Core Evaluation Questions
In fulfilling the purposes of the evaluation, the following core questions were addressed:
How has the activities carried out by KLIP, influenced governance and policies of
collaborating public and private sector organizations?
How has KLIP impacted in the society so far, and what can it achieve in the future?
How can the programs in KLIP be further improved for achieving the organization’s vision and
mission?
2.3 Evaluation Stakeholders
KLIP TRUSTEES
Under the banner of SIDA and M-UF, KLIP Trustees formulated programs that were targeted at:
1) Supporting Ecotourism and Market Places as the flagship projects
2) Strengthening of the Secretariat through participation in the strategic development of Mistra
Urban Futures; and facilitating the establishment of the KLIP Trust.
3) Promoting Knowledge and Technology Transfer as well as Capacity Building for County
Governments via the.
In this evaluation process, we will try to establish the reasons behind formulation of the programs
by the trustees, their methods of implementation, possible success and failure stories, as well as
the future scope of the projects from the trustees’ point of view
Kisumu Local Government Leaders
For any project to succeed, participation of local leaders is paramount. And so KLIP partnered
with various local leaders holding various positions, in the implementation of the various projects,
with special focus in Eco-Tourism and Market place.
As a means of evaluation, we will establish the impact of the programs on the governance in
Kisumu. We will find out, how the partnership was carried. Possible success and failures
throughout the partnership and future modifications that will foster better partnership in the
various projects.
Kisumu Private Sector Organizations
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
3
One of the goals of KLIP is to bridge the gap between the private sector and the local
government. Through the various programs, we will evaluate the collaborations created in the
process between the private sector and the local government. We will find out, how these ties
have improved the region’s economy. In addition, the evaluation will highlight possible
drawbacks encountered through the partnerships created, and how these can be improved for
future collaborations
Kisumu Residents
KLIP always puts the needs of the locals in the programs and projects it formulates. The goal of
KLIP is cogeneration of knowledge, practical application and dissemination for the purposes of
sustainable urban development. Through the various activities of Eco-tourism and Market
places, the evaluation will establish the following aspects:
Acceptance and attitude of the locals to the various projects
Training involved in the various projects
Impact of the projects on the locals lifestyles
Future proposals for the projects and programs
2.4 Evaluation Management Structure
Position Name
Assistant-Director General Dr.
Director Dr.
Project/Program/Initiative Manager Mr. Owuor
2.5 Evaluation Governance Group Membership
Position Name
Research Assistant Mr.
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
4
Data Analyst Mr.
2.6 Evaluation Governance Group Terms of Reference
The roles and responsibilities of the Evaluation Governance Group include:
Roles and responsibilities
Project Consultant – Overall in charge. Supervises the project.
Project Manager- Supervises day to day activities in the evaluation cycle. Reports to consultant
Research Assistant – In charge of data collection
Data Analyst – Analyses the data
Field Guide – Assists in data collection
Field Support staff – provides as need service to the team
2.7 Reporting Requirements
Date Report type Writer/s Audience
08/08/2014 Progress Report Project Manager Evaluation Steering Committee
10/08/2014 Draft Report Project Consultant Evaluation Steering Committee
12/08/2014 Quality Assurance Review
Contracted Consultant Evaluation Steering Committee
13/08/2014 Final Report Project Consultant KLIP Director
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
5
3. Evaluation Design
3.1 Evaluation Framework
The Guide below shows the framework to be applied during the evaluation process
3.2 Information and Collection
Core evaluation
question to
address
Information
required
Information
sources
Data collection and
analysis
What are the reasons behind setting up the various programs and activities in KLIP?
KLIP Programs and Activities
Literature available since 2010
Literature review
Interviews
What is the impact of the KLIP programs on governance in Kisumu?
Influence of KLIP activities on Governance
Key stakeholder – Local Leaders Interviews
Questionnaire
What is the impact of KLIP activities in fostering private sector – government collaborations
Influence of KLIP on fostering private sector – local government relations
Key stakeholder – Private organization partnered in the activities - Interviews
Questionnaire
How has the activities impacted on impact local residents lifestyle
Economy elevation of the locals
Key stakeholder Interviews
Literature reviews
Questionnaire
4. Evaluation Resources and Timeframe
4.1 Budget and Staff Resources
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
6
Budget for the evaluation study
Internal DET staff
resources
Position Classification
level
Approximate time
and / costs required
Dr. Oketch Team Leader Top level 80 hours – Kshs
150, 000
Project Manager Middle level 50 hours – Kshs 75,000
Research Assistant Middle level 50 hours – Kshs 75,000
Data Analyst Middle level 50 hours – Kshs 50,000
External human resources Approximate time and cost required
Data Entry Technician 40 hours – Kshs 35,000
Field Guide 80 hours – Kshs 20,000
Field Support Staff 20 hours – Kshs 5,000
Other resources required
Cost estimates
Program Logic Modelling session
Data tools generation Kshs 5,000
Questionnaires booklets
Kshs 2,000
Interview recording devices (4)
Kshs 15,000
Video recording devices (1)
Kshs 23,000
Stationary Kshs 5,000
Totals Kshs 45,000
Accommodation & Travel Committee Members
5
Bus Return Ticket Kshs 5,000
Kshs25,000
3 Day Hotel bed & breakfast
Kshs 4,000
Kshs20,000
Total Kshs 45,000
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
7
Allowances for Evaluation Committee Food Allowances Kshs 10,000
Airtime Allowances Kshs 10,000
Transport Allowances Kshs 10,000
Miscellaneous Kshs 15,000
Total allowances Kshs 45,000
4.2 Evaluation Staff Roles and Responsibilities
Internal DET
evaluation team
members
Position Roles and responsibilities
Dr. Oketch Project Manager Overseeing project evaluation process
Mr. Owuor Research Assistant Supervision of data collection
Mr. Oloo Data Analyst Data cleaning and analysis
External
consultant(s)
Position Roles and responsibilities of the
consultant(s)
Dr. Okecth Project Consultant Consultation on project activities
4.3 Time Schedule
Time/Activities 8:00-10:00AM 11:00-12:00AM 2:00-4:00PM 5:00-7:00PM
31/07/14 Evaluation
Commencement
Approval
process for the
evaluation
Project
Committee 1st
meeting
Project goals
reviewed
01/08/14 Program Logic Model generated Specific Objectives for team
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
8
members formulated
02/08/14 HOLIDAY
03/08/14 HOLIDAY
04/08/14 Data Tools Generated Data tools reviewed and revised
05/08/14 Data Collection (interviews) Data Collection (interviews)
06/08/14 Data Collection (Questionnaires) Data Collection (Questionnaires)
07/08/14 Data Entry Draft Project progress Report
08/08/14 Draft Project Report Presented to
Evaluation Governance Team
Feedback from Evaluation
Governance Team
09/08/14 Data Cleaning and Analysis Quality Assurance consultant
contracted
10/08/14 Data Analysis Complete Progress Report to evaluation
steering committee
11/08/14 Draft Project Report Presented to
Evaluation Governance Team
Report compilation
12/08/14 Review/Endorsement of report by
the Evaluation Governance Team
Report revision
13/08/14 Final Report approved by
evaluation steering committee
Presentation of final report to KLIP
trustees
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
9
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
10
Major milestones Due date
Consultation with Evaluation Services regarding support and approval process for the evaluation
31/07/14
Program Logic Model (or evaluation framework) generated 01/08/14
Evaluation Governance Group established with endorsed Terms of Reference
04/08/14
Draft Evaluation Proposal presented to Evaluation Governance Group for feedback
05/08/14
Evaluation Proposal endorsed by Evaluation Governance Group 06/08/14
Evaluation Proposal approved by Evaluation Steering Committee 07/08/14
Consultant contracted (if applicable) or Evaluation Team appointed
08/08/14
Progress Report to Evaluation Steering Committee 08/08/14
Data gathering and analysis complete 10/08/14
Progress Report to Evaluation Steering Committee 11/08/14
Draft Evaluation Report presented to Evaluation Governance Group for feedback
12/08/14
Draft Evaluation Report endorsed by Evaluation Governance Group
12/08/14
Final Evaluation Report approved by Evaluation Steering Committee
13/08/14
Brief with any recommendations to the Policy and Performance Committee with any recommendations
13/08/14
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
11
Appendix 1 Evaluation Plan Checklist
This checklist has been included to assist in the development of the Evaluation Plan.
Socio-political Factors The team should identify and effectively address affected/concerned groups
Involvement Whose sanction and support is required?
How will it be secured?
Internal communication How will key audience needs for information on the evaluation’s progress be determined and met?
How will communication be maintained between the program team, the evaluation team and the Evaluation Steering Committee (ESC)?
External credibility Will the evaluation be free of bias?
Realistic expectations How will the evaluation team make it clear to stakeholders that realistically only a subset of their information needs will be addressed?
Security What provisions will assure security of the data?
Protocol What communication channels will be honoured and employed?
Public relations How will stakeholders be consulted and kept informed about the intents and results of the program evaluation?
Stakeholder confidence What checks will be made to ensure that the evaluation plan and the composition of the evaluation team are responsive and acceptable to the key stakeholders?
Contractual/Legal Arrangements The evaluation team and participants should establish clear working agreements to ensure efficient collaboration and protect involved parties’ rights
ESC, evaluation team, and other roles
Who is the sponsor
Who is on the evaluation team
Who are the other audiences
How are they related to the program?
Realistic commitments What clarifications assure that the evaluation can proceed while making reasonable efforts to serve a broad audience but not becoming bogged down in over identifying and consulting with stakeholders?
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
12
Contractual/Legal Arrangements The evaluation team and participants should establish clear working agreements to ensure efficient collaboration and protect involved parties’ rights
Delivery schedule What is the schedule of evaluation services and products?
Editing reports Who has authority for editing evaluation reports?
Access to data What existing data may the evaluation team use
What new data may they obtain?
Access to stakeholders Are there sufficient safeguards to assure that the evaluation team may contact involved stakeholders?
Pre-release reviews Will the ESC be provided appropriate opportunities to review the draft report for clarity and fairness prior to their finalisation and release?
Release of reports Who will release the reports
What audiences may receive them?
Responsibility and authority
Have the system personnel and the evaluation team agreed on what persons and groups have both the responsibility and authority to perform the various evaluation tasks?
Finances What is the schedule of payments for the evaluation
Who will provide the funds?
Technical Design The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical plan
Objectives What is the program intended to achieve/produce
In what terms should it be evaluated?
Variables What classes of information will be collected, e.g., context, inputs, processes, outcomes?
Program description Will the program be described sufficiently, so that stakeholders will understand its nature?
Investigatory framework Under what conditions will the data be gathered, e.g., experimental design, case study, survey, site review, examination, etc.?
Instrumentation What data gathering instruments and techniques will be employed
How will the evaluator assure that they address the key evaluation questions?
Sampling What samples will be drawn
How will they be drawn
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
13
Technical Design The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical plan
Will they meet both utility and technical requirements?
Data gathering How will the data gathering plan be implemented
Who will gather the data?
Have multiple modes of data gathering been planned to ensure triangulation?
Data storage and retrieval
What format, procedures, and facilities will be used to store and retrieve the data?
What strategies are in place to ensure the data complies with ethics and codes of conduct?
Data analysis How will the data be analysed?
Sources of interpretation Who is charged to interpret findings, e.g., the evaluators, various stakeholders, a regulatory body, etc.?
Bases for interpretation What bases will be used to interpret findings, e.g., objectives, assessed needs, contractual specifications, laws and regulations, democratic ideals, social norms, performance by a comparison group, technical standards or criteria, polls, judgments by reference groups, etc.?
Methods of interpretation What methods will be used to assign value meaning to findings, e.g., focus groups, a Delphi study, advocacy and adversary reports, etc.?
Reports What reports will be used to disseminate the evaluation findings?
Reporting media Considering the preferences of the audiences, what are the most appropriate means of reporting findings, e.g., detailed technical reports, summaries, press conferences, study sessions, memos and letters, video presentations, web assessable documentation, etc.?
Reporting language Will reports need to be presented in different languages: technical or non-technical, simple English or other language(s) to meet the needs of different audiences?
Reporting format Will reports be carefully formatted to enhance their readability, e.g., use of white space?
Responsive design What ongoing evaluation planning process and resource plan will assure flexibility for adding to or otherwise revising the evaluation questions and obtaining unanticipated, pertinent information?
Delimited design Is there a clear delimitation of the design, including the
Evaluation Plan for KLIP Evaluation Manager: Dr. Oketch TRIM Number: 00900
14
Technical Design The evaluation team should convert a general evaluation plan to a detailed, yet flexible technical plan
purpose of the evaluation and the questions that will be answered?
Attention to trade offs How will the evaluation address trade-offs between comprehensiveness and selectivity at each stage of the evaluation: planning; budgeting; and collecting, organizing, analysing, interpreting, and reporting information?
Technical adequacy What are assurances that the findings will be reliable, valid, and objective?
Moral/Ethical Imperatives The evaluation team and clients/stakeholders should clarify and confirm the evaluation’s role in ethically serving some socially valuable purpose
Philosophical stance Will the evaluation be value based, value plural, or value free?
Evaluation team’s values Will the evaluation team’s technical standards and values conflict with the client system’s and/or sponsor’s values; will the team face any conflict of interest problems; what will be done about possible conflicts?
Judgments Will the evaluation team judge the program; or obtain, analyse, and report the judgments of various reference groups?
Objectivity How will the evaluation team avoid being co-opted and maintain their objectivity?
Equity How will the evaluation team make sure to address and honour the needs and rights of all stakeholders equitably, taking appropriate account of their gender, ethnicity, and language backgrounds?
Cost effectiveness Compared to its potential payoff, will the evaluation be carried out at a reasonable cost?
NB the above listed prices and costs are quoted in their net values.