1

NORAD/OED Trust Fund for Evaluation Activities

Nils FostvedtSenior Advisor to the Director-General,

Operations EvaluationMay 26, 2005

2

Financial Times May 20, 2005

• “Watchdog tells World Bank to focus on growth rather than health and education.”

• “The hard-hitting report by the Bank’s Operations Evaluation Department criticized the emphasis placed on education and health spending.”

• “Just getting spending up in the social sector is not enough for poverty reduction.”

3

The Bank Group has Three Independent Evaluation Units

• Bank:Operations Evaluation Department (1973)IFC: Operations Evaluation Group (1996)MIGA: Operations Evaluation Unit (2002)

• Overseen by the Director-General, Operations Evaluation (DGO), who reports directly to the Board through the Board’s Committee on Development Effectiveness (CODE)

• Findings and Recommendations are not subject to approval by Bank Management

• Budgets are approved by the Board• Work Programs are set by the DGO, with Board

consultation and endorsement

4

Major OED Evaluation Products

• Ex post project evaluations• Country Assistance Evaluations (over 70 CAEs)

– New Country Assistance Strategy (CAS) Completion Report Reviews

• Sector/thematic and global evaluations– Extractive Industries, Investment Climate, HIV/AIDS

• Process evaluations– Global Programs, Africa Capacity Building, PRSs (Poverty

Reduction Strategies)

• Corporate reviews– Annual Review of Development Effectiveness (ARDE)– Annual Report on Operations Evaluation (AROE)

5

Project Evaluations

• Self-Evaluation – Implementation Completion Reports (ICRs) followed by independent evaluation

• Objectives-based methodology

• Ratings

• Ratings database

6

Independent Project Evaluations

• All projects (ICRs) subject to OED desk reviews• Subsequent in-depth Project Performance

Assessment Reports (PPARs) for 25%• Internal quality review process • Send drafts to Bank staff and to Borrowers for

comments• All PPARs are now disclosed

7

Key Ratings: Project Outcome

• Rating Scale– Highly Satisfactory– Satisfactory– Moderately Satisfactory– Moderately Unsatisfactory– Unsatisfactory– Highly Unsatisfactory

• Based on three components– Efficacy: To what extent were objectives

achieved?– Efficiency: Cost effectiveness of operation?– Relevance: Were these the right objectives?

8

Outcome Ratings

By Project

20%

40%

60%

80%

100%19

90

1991

1992

1993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

*

Exit FY

Perc

enta

ge S

atis

fact

ory

Out

com

e

Note: 2004* Partial (dashed line)

9

Interactions with Management

• Get comments on evaluation reports from Bank management before sending to CODE; draft Management Response accompanies evaluation.

• Review management’s actions committed in Management Response and report to the Board in the Management Action Record (MAR).

• Respond to management comments on evaluation products at CODE/Board Meetings.

10

Reporting to the Board

• Meet with Board Committee (CODE) and the full Board on major corporate reviews (ARDE, AROE)

• Present sector, thematic and process reviews to CODE

• Discuss CAEs and methods with CODE Subcommittee

• Do not negotiate content of reports with CODE/Board

• Provide comments to Board and participate in CODE discussion of selected Management reports and proposals

11

Part of the Bank’s Evaluation and Control System

• Independent Evaluation (Reports to Board)– OED, OEG, OEU

• Quality Assurance (Reports to Management)– QAG, QACU (Bank, at entry, safeguard compliance)– Quality Assurance Teams, Credit Review (Bank, IFC)– OPCS, CPR, CES (Bank, reviews; IFC, risk and compliance)

• Financial Probity (Reports to Management)– Internal Audit Department (effectiveness of controls)– Institutional Integrity Department (fraud, corruption)

• External claims of noncompliance– Inspection Panel (Bank, investigative, reports to Board)– Compliance Advisor Ombudsman (IFC, MIGA to

management)

13

NORAD/OED Trust Fund for Evaluation Activities

• NKr15 million over three years

• Funding supports selected OED evaluation studies and training activities

• The funds are used for consultants, dissemination activities, evaluation capacity development (ECD)

• Funds are can be used for consultants from any member country

• But we wish to strengthen evaluation partnership with Norway

14

OED’s Use of Consultants

• About $5 million per annum, mostly for individual consultants

• Consultants typically work in teams led by an OED Task Manager

• They are identified based on specific requirements, which depend on work program

• Also use senior individuals on advisory panels for major evaluations

15

Possible consulting requirements next three years include– Impact evaluations– Evaluation capacity development– Judicial reform, agriculture, environment, economic sector

work, public sector reform, small & medium enterprises, fiduciary systems and infrastructure

– Also work on evaluations of projects and country programs– Need specialists in these areas with an understanding of

evaluation– Experience in developing countries an advantage

16

If you think you have something to offer:

• Look at OED’s website for types of products http://oed.worldbank.org/index.shtml

• Contact Patrick Grasso pgrasso@worldbank.org or Nils Fostvedt nfostvedt@worldbank.org

• And don’t hold your breath

17

Thank You