IASC Inter – Agency Real Time

Evaluations(IA RTEs)

Outline

• What are IA RTEs?

• What are they good for?

• Where do IA RTEs come from?

• How do IA RTEs work?

• In which ways are IA RTEs different from standard evaluations?

What are IA RTEs?

Definition• IA RTEs are ‘Inter-Agency (IA)’ in that the RTEs are instigated,

managed and used by a variety of international organizations

• The term ‘Real-Time (RT)’ refers to the early implementation stages of a humanitarian emergency operation, to the rapid feedback of IA RTE findings at the field level (within the two first month) and consists of remote monitoring and the IA RTE mission.

• ‘Evaluation’ (E), as applied in the term IA RTE, describes a participatory review process. IA RTEs differ from other humanitarian evaluations regarding speed, coverage, methods, and outputs.

What are IA RTEs good for ?

Purpose IA RTEs seek to:

• identify gaps, access constraints to the “humanitarian space”,

• assess the relevance, quality and timeliness of the response,

• unlock inter-stakeholder coordination problems or operational bottlenecks,

• propose appropriate strategic re-orientation

• provide real time learning to the Humanitarian Country Team.

• identify strengths and weaknesses of the response through the assessment of the response to the affected population.

The primary end users of IA RTEs are in-country agencies and staff engaged in the humanitarian response

SUM UP• IASC mandated evaluation tool that is managed

and used by a variety of international agencies for real time learning

• Focus is on inter-agency coordination and management.

• Carried out at the early implementation stages of a humanitarian operation

• Rapid feed back of findings at the field levels for immediate use

• Seek to identify gaps, access constraints & potential threats and assesses the quality of the response

Where do IA RTEs come from?

2003 2004 2005 2006 2007 2008 2009 2010 2011

Situation in SudanCluster

Approach

Evaluation

Phase 1/ CERF

Humanitarian

Reform

Humanitarian

Response

Review (HRR)

Key Events & Origins of IA Evaluations

IA Real Time

Evaluation

Cluster Approach

Evaluation: Phase 2

Towards IA RTEs

• Humanitarian Response Review

• Humanitarian Reform

• Clusters and CERF

• RTE

2005 Humanitarian Response Review (HRR)

ERC Jan Egeland

Assess the humanitarian response capacities of the UN, NGOs, Red Cross/Red Crescent Movement and other key humanitarian actors

Identify the gaps

Make recommendations to address gaps

Humanitarian Response Review

Results of the HRR (excerpt from recommendations):

The IASC and the donors should:

establish a limited set of benchmarks (and indicators) to be implemented in the first period of a new emergency up to a maximum of 3 months and addressing in priority

• access and coverage of population in need

• identification of responsibilities in delivery of assistance and in coordination

• resources mobilization (human, assets, financial)

The IASC should:

identify and assign lead organizations with responsibility at sectoral level, especially in relation to IDP protection and care and develop a cluster approach in all priority sectors. (Action: ERC/IASC – immediately)

Humanitarian Reform 2005 / 2006

Cluster Evaluation

s

IA RTEs

Inter-Agency Standing

Committee (IASC)• Unique inter-agency forum for coordination,

policy development and decision-making involving the key UN and non-UN humanitarian partners.

• Established in June 1992 in response to United Nations General Assembly Resolution 46/182 on the strengthening of humanitarian assistance.

• General Assembly Resolution 48.57 affirmed its role as the primary mechanism for inter-agency coordination of humanitarian assistance.

• Develops humanitarian policies, agrees on a clear division of responsibility for the various aspects of humanitarian assistance, identifies and addresses gaps in response, and advocates for effective application of humanitarian principles.

IASC WG requests IA RTE Pilot Phase

IASC WG requests IA RTEs in 2007:

• At its 64th meeting held in March 2007, the IASC Working Group endorsed a one-year pilot to inform the developing and testing of approaches to inter agency real time evaluation (IA RTE), with the objective of eventually establishing a regular standby mechanism. At this time, the IASC also endorsed nine tenets / principles of IA RTE and committed to ensuring agency-specific and IASC-wide management response and action to the recommendations arising out of future IA RTEs.

1. Inter-Agency RTE should be used to address broader, cross-cutting and sector wide concerns.

2.The primary purpose of the IA-RTE is to support field management decision-making; a secondary purpose is to provide information to senior managers in agencies to help better understand and support the programmes they are in charge of directing.

3.The adopted approach should be light, flexible and responsive to the needs of field actors and the changing nature of the emergency response.

4.There needs to be a consistent and timely trigger for IA-RTEs – ideally IA-RTEs should already be indicated in a flash appeal/CAP.

IASC WG formulates 9 Basic Principles

5. IA-RTEs should not replace agency-specific RTEs - the target audience is therefore the broader humanitarian community and not a single agency.

6. There needs to be a strong focus on the affected population which must inform the method of the IA-RTE.

7. IA-RTE teams should be lean and reflect a mix of agency expertise, methodological skills and sectoral expertise as is appropriate for the emergency.

8. IA-RTE are management intensive and need a full time lead management agency, supported by an inter-agency management group

9. IA-RTE need to result in immediate use and action. This also requires a strong commitment from all participating agencies from field upwards to headquarters to respond to and take seriously the recommendations made by the team.

IASC WG formulates 9 Basic Principles

• 74th IASC WG agreed to move beyond the pilot phase to regular implementation and requested to develop a set of operating procedures:

Criteria for triggering IA-RTE

A Clear Purpose Statement for IA RTE

A standard ToR & methodlogies

IASC WG request to move from pilot phase to regular

implementation

IASC IA RTE Interest Group develops an IA RTE

Procedures and Methods Guide

IA RTE Procedures and Methods Guide is endorsded by the IASC during its WG meeting in July 2010!

How do IA RTEs work?

How do IA RTEs work?

• Who are the main Stakeholders in IA RTEs?

•How and when to trigger an IA RTE?

•What is the Evaluation Framework for IA RTEs?

IA RTE Stakeholders

OCHA Focal Point

IA Advisory Group

(optional)

Humanitarian Country Team

IA SupportGroup

IA Management

Group

ERC/ HC

IA RTE

Criteria Lists for Triggering IA RTE

Essential IA RTE Trigger Criteria (Automatic)

• > 1 million people affected?

AND

Flash Appeal > than $50 million or CAP >than $350 million?

Desirable IA RTE Trigger Criteria

(By request)

•IASC WG, SG members, ERC, RC/HC or a majority of HCT members requested IA RTE •Inter-agency coordination & management to be examined cover a specific period & a defined geographical area? •Opportunities for RTE learning?• Capacity exists to conduct an IA RTE?

IA RTE Framework & Bottom-Up

Approach

IA RTE Standard ToR

• Focus on participatory approach

• Independent consultants team deployed during the initial response phase with arrival no later than two months after the disaster

• Assess initial response via desk reviews, field visits, interviews with key stakeholders, especially affected population

• & provide real time feedback & learning to the HCT

• Consultants serve as facilitators to encourage & assist field personnel to critically examine operations & to find creative solutions to problems

• Overarching questions include:

• What were the main operational results, and the positive & negative outcomes for all segments of the affected population, during each phase?

• Have appropriate common standards been developed within the coordination systems (globally & for each Cluster) & to what degree have these been met?

In which ways are IA RTEs

different from

standard evaluations?

IA RTEs are different in the following way:

• An attempt at system-wide assessment at an early stage

• Real-Time Learning for the HCT

• Bottom-Up Approach

• Specific Trigger Mechanism

• IA Stakeholders

IA RTEs ARE DIFFERENT FROM STANDARD EVALUATIONS

DEFINITION:

• IA RTEs are ‘Inter-Agency (IA)’ in that the RTEs are instigated, managed and used by a variety of international organizations. These are represented at a global level in the IA RTE Support Group and for a specific IA RTE, through the IA RTE Management Group, the Humanitarian Country Team (HCT) and the in-country IA RTE Advisory Group, when established.

• The term ‘Real-Time (RT)’ refers to the early implementation stages of a humanitarian emergency operation, to the rapid feedback of IA RTE findings at the field level, and to the possibility of an iterative, multi-phased approach depending on the scale of the disaster. A one-phase approach of an IA RTE would be carried out within the two first month and consists of remote monitoring and the IA RTE mission […].

• ‘Evaluation’ (E), as applied in the term IA RTE, describes a participatory review process. IA RTEs differ from other humanitarian evaluations regarding speed, coverage, methods, and outputs.

IA RTEs ARE DIFFERENT FROM STANDARD EVALUATIONS

PURPOSE:

• Intended primarily for learning in emergency operations with field-level ‘inter-agency coordination and management at the core, especially in the context of sudden-onset disasters, or protracted crises undergoing a phase of rapid deterioration or escalating violence.

• Defined as an evaluation carried out at the early implementation stages of a humanitarian operation which almost simultaneously feeds back its findings for immediate use by the broader humanitarian community at the field level.

• Seek to identify gaps, access constraints and potential threats to the “humanitarian space”, assess the relevance, quality and timeliness of the response, unlock inter-stakeholder coordination/ collaboration problems or operational bottlenecks, propose appropriate strategic re-orientation and provide real time learning to the field on same.

• Principle aim is to identify strengths and weaknesses of the response through the assessment of the response to the affected population.

• Primary end users of IA RTEs are in-country agencies and staff engaged in the humanitarian response, especially those involved in inter-agency coordination and management.

Challenges

Inter-Agency Challenges

• Funding

• Coordinating IA Stakeholders

• IA-RTE in Complex Emergencies

• Responsibility “Sharing”

LinksInter-agency Real Time Evaluations

http://oneresponse.info/Coordination/IARTE

Inter-agency Standing Committee (IASC)

http://www.humanitarianinfo.org/iasc/pageloader.aspx

OCHA & Inter-agency Evaluation Reports

http://ochaonline.un.org/ToolsServices/EvaluationandStudies/ESSReports/tabid/1325/language/en-US/Default.aspx

Active Learning Network for Accountability & Performance in Humanitarian Action (ALNAP)

http://www.alnap.org/

Andreas Schuetz

Associate Expert in Humanitarian AffairsPolicy Development and Studies Branch PDSBEvaluation and Guidance Section EGSPalais des Nations, Geneva

Tel. +41 22 917 19 29tel. +41 76 271 31 47fax. +41 22 917 00 20E-mail: schuetza@un.org

Claude Hilfiker

Senior Evaluation OfficerPolicy Development and Studies Branch PDSBEvaluation and Guidance Section EGSPalais des Nations, Genevatel. +41 22 917 11 96fax. +41 22 917 00 20E-mail: hilfiker@un.org

Questions?