THE CITIES ALLIANCE RESULTS FRAMEWORK AND RESULTS … · visionary global agenda to date. For the...

THE CITIES ALLIANCE RESULTS FRAMEWORK AND

RESULTS-BASED SYSTEM

Document Version First Draft

Prepared by Cities Alliance Secretariat

Reviewed by

Approved by Management Board [pending]

Date 03/12/2018

of 20

Table of Contents

I. Background 4

II. The Results Framework 4

Principles 4

Narrative 6

Assumptions 7

III. The Corporate Core Indicators 9

Hierarchy 9

Indicators List 10

Indicators Definitions 13

IV. Period 2018-21. Universe, Frequency, Methods and Targets/Performance Standards 18

V. Corporate Reporting 19

Results-Based Annual Report 19

Corporate Scorecard 19

of 20

Acronyms

CA Cities Alliance

CP Country Programme

IBSA India-Brazil-South Africa

JWP Joint Work Programme

KPI Key Performance Indicator

RBM Results Based System

RF Results Framework

SDG Sustainable Development Goals

SOCR State of the Cities Report

TA Technical Assistance

UR Urbanization Review

WUF World Urban Forum

of 20

I. Background

1. As per its Charter, the main objective of the Cities Alliance is to reduce urban poverty and promote the

role of cities in sustainable development. To assess the extent to which its efforts and those of partners are making

progress toward that objective, the Cities Alliance monitors, evaluates and publicly reports its activities within an

agreed-upon corporate performance and results framework and through a dedicated result-based management

system.

2. The Results Framework is approved by the Management Board as part of its responsibility for setting the

strategic direction of the Cities Alliance, and for reviewing and evaluating the organisation’s overall performance.

The Cities Alliance Results Framework was initially developed while hosted at the World Bank and approved at

the 2013 Annual Meeting in Ouagadougou. With the adoption of the new Strategic Plan 2018-21, the Results

Framework has been revised to align it to the new objectives set by the Plan. 3. The Cities Alliance corporate Results Framework follows international standards and practices in

development cooperation on results-based management. In its original formulation, it has highly benefitted from

the World Bank expertise and advice on corporate reporting which has influenced the overall approach and,

notably, the structure of the scorecard.1 It was further refined based on the experience with the hosting agency,

UNOPS.

4. The adoption of the 2030 Agenda for Sustainable Development provided the most comprehensive and

visionary global agenda to date. For the first time, the role and importance of cities was acknowledged by member

states, impacting a significant number of the 17 Goals and 169 targets. The Sustainable Development Goals

(SDGs) have guided the overall frame and several metrics of the current version of the Results Framework.

II. The Results Framework

Principles

5. The Charter and four-year Strategic Plans establish the developmental objectives of the Cities Alliance, its

approach and the type of activities it supports. The Cities Alliance will track and report progress through a

comprehensive Results Based Management (RBM) system. The Results Framework is a tool at the core of RBM

approach as it defines an organisation’s theory of change and the core metric.

6. The Results Framework articulates the different tiers of results expected by Cities Alliance interventions.

It lays out the products and services that will be delivered over a period of time (Tier IV), the changes these

products and services will generate in the short/medium time (Tier III), the effect on the main clients (Tier II),

and the overall influence on the national and global developmental goals (Tier I). The vertical progression across

tiers approximates the chronological and causal/logical progression across a result chain: from inputs to outputs,

to intermediate outcomes to outcomes and, lastly, to impact. Nevertheless, it should be noted that, for corporate-

wide aggregating exercises, this vertical progression - differently from project logframes - should not be taken to

be neither scientific nor always attributable. As per Figure 1 below and international practises, there is also no full

correspondence between the Tier scale and the results chain as the definition of what constitutes an ‘output’, an

‘outcome’ or rather an ‘impact’ is arguable.

7. The Results Framework is defined operationally by selected indicators to help measure and document

progress and performance at the portfolio level across the various tiers of results at different intervals of time. The

Results Framework indicators aggregate information from the various programmes. The indicators are core,

meaning they have standard definitions and each programme is required to report on them. The Corporate Results

Framework strategically informs and guides programming. However, it does not substitute the need for each

programme to have their own individual logframe tailored to their specific objectives and the requirements of the

programme’s donors.

8. The Results Framework also defines organisation-wide standards for baselines, milestones and targets,

1 See Section V below.

of 20

Att

ributi

on

Contr

ibuti

on

Infl

uen

ce

data sources as well as the tools and frequency for data collection. The Results Framework is not only about

monitoring, controls and tracking emerging results; it is also about learning – for both clients and the Cities

Alliance as a partnership – and applying the experiences learned in the planning and design of new activities. The

data collected will also provide a sound basis for future evaluations.

9. The Cities Alliance Result Framework is based on different attribution vs. contribution approaches in

relation to the various tiers of results. The Cities Alliance Secretariat is responsible, and should be held

accountable, for effectively delivering Tier IV. These input/output levels form the basis of the Terms of Reference

of the Cities Alliance Secretariat. At Tier III, the Secretariat has a direct (when implementing) and/or shared

responsibility with its international and country institutions (when financing partner organisation) to deliver the

desired change. Attribution of results can be established to a reasonable degree. Moving further up the chain, the

level of control decreases, and the attribution gap and risk increases. Tier II is the core and raison d’etre of the

organisation. However, results as this level are delivered primarily by client cities and communities with the

support of the Cities Alliance and its partners. They are partly beyond the control of the Cities Alliance and as

such only contribution factors – if any – can be established. Tier I Results and progresses are well beyond the

control of the Cities Alliance, and the improvements observed in Tier I indicators are rather the outcome of

collective efforts by countries and their development partners. However, since development effectiveness is more

than a ‘flag-planting’ exercise, it is in the Cities Alliance’s and common developmental interest to track progress

against the macro context.

Figure 1. Results Framework Structure

10. Tier I. Development Goals. As per standard international practise, this tier is primarily contextual and

reports on the long-term development goals that partner countries are achieving. The universe of measurement is

the priority countries where Cities Alliance has long-term engagements. Developmental impact is measured in

terms of a subset of Sustainable Development Goals (SDG)2, i.e. those which relate most to the Cities Alliance

mission and livelihood of the target population – the urban poor – across three aspects: poverty, tenure and gender

representation. As mentioned, it is still important to include Tier I in the result framework to encapsulate Cities

Alliance interventions within the broader developmental picture and define what the organisation is, ultimately,

trying to influence.

11. Tier II. Client Results. This level reports on the clients’ results as promoted, supported and/or influenced

2 While for measurement purposes we focus on a small subset of SDGs indicators, more generally, and as per its Strategic Plan 2018-21, the Cities Alliance remains firmly committed to the progress of those 10 SDG Goals and 55 SDG Targets which relate to cities.

IMPACT

OUTCOMES

OUTPUTS

INPUTS

TIER I

This tier is primarily contextual and reports

on the long-term development goals that countries are achieving.

TIER II

Client’s results influenced by Cities

Alliance’s technical assistance and

operations.

TIER III

Results of programme activities financed

by the Cities Alliance and typically implemented by Members and Partners, or

the Secretariat.

TIER IV

Metrics (KPIs) tracking the performance of

the Secretariat

of 20

by the Cities Alliance. The city (broadly defined) is the client of the Cities Alliance. In fact, the Cities Alliance

provides technical assistance services, exemplary projects and facilitates the leverage of the financing that helps

cities to be more effective, participatory, and able to deliver improved, responsive services to the urban poor.

Partner cities and national governments are the primary responsible for results at this level.

12. Tier III. Cities Alliance Programmatic Results. This tier covers the result of the programme activities

of the Cities Alliance. Through the Secretariat, its members and partners, the Cities Alliance provides technical

assistance support for upstream diagnostics, planning, and policy advice and development. It also supports long-

term institutional strengthening and capacity development by engaging and investing in national, regional, local

and community institutional structures, including universities. Cities Alliance contributes to infrastructure

development in slums, typically delivered through community engagement and dedicated funds. Specific attention

is paid downstream to project preparation, aligning the Cities Alliance support to maximise the mobilisation of

domestic and international financial resources. The Cities Alliance, through its Secretariat, implementing

members and partners – is responsible and accountable for delivering these outcomes. It is the Partnership’s Terms

of Reference.

13. Tier IV. Cities Alliance Organisational and Operational Preference. This tier covers the overall

performance of the Cities Alliance Secretariat through its two operating window (global window and country

window) and across four different areas: partnership, volume, efficiency, and sustainability. The Secretariat is

responsible and accountable for delivering these outputs. It is the Secretariat’s Terms of Reference.

Narrative

14. In any Results Framework, it is important to be clear about the theory of change, i.e. the vertical integration

of the various Tiers in causal and chronological relation. While in recent times there is more relaxation about the

rules governing such vertical logic with regard to agency-wide exercises - especially if compared to traditional

project logframes – in favour of a more flexible approach, a results framework should still be built around a

pathway of change that articulates inputs into longer-term development impacts.

15. As mentioned, the city (broadly defined) is the client of the Cities Alliance while the ultimate beneficiaries

of Cities Alliance activities are the urban poor and women. The Cities Alliance’s aim is to enable cities to run

more effectively, deliver improved and responsive services to the urban poor, be inclusive and foster equal

economic opportunities. These objectives are outlined in the description and indicators of Tier I and Tier II.

16. Tier III captures the typology of technical assistance and physical interventions of the Cities Alliance which

aim at directly influencing local authorities and the living conditions of the urban poor. While, in the organisations’

strategy, this is achieved through investments in low income settlements that address the symptoms of slums –

most obviously the lack of services, poorly-located settlements, insecure tenure and poor housing – tackling the

root causes of social exclusion and urban poverty is equally important and requires a more comprehensive,

citywide approach that include financial, governance and policy interventions. In the long term, it is well run and

resourced cities that will allow citizens to exercise their rights and responsibilities as part of a shared vision for

their city. Typically, the technical assistance provided at Tier III does not result in single projects but rather in

more complex programmes, which target vertically three levels of governance and spatial territory (national, city

and community) as well as operate horizontally multi- development partners and sectors.

17. In terms of the theory of change, the critical juncture is the relation between Tier II (changes at cities level)

and III (Technical Assistance and exemplary infrastructure). It is within this space - the outcome level - that the

seeds for sustainability, institutionalisation and structural change are planted. This is even more crucial for an

organisation such as the Cities Alliance which primarily provides technical assistance with limited physical

interventions. Thus, the ability to help client cities in a change-process relies on the ability to catalyse additional

investments and trigger long term policy impact. To strengthen the relation between Tier II and Tier III and

achieve tangible development results, the Cities Alliance pursue three main approaches:

The first – which is the most direct approach - is through facilitating the financing of physical

interventions, typically managed through community development/upgrading funds, which directly

affect livelihood and access to services for low income households;

The second approach is to promote larger-scale infrastructure investments directly linked and leveraged

by the technical assistance activities - diagnostic, planning and project preparation - supported and

provided through the Cities Alliance;

of 20

The third approach - more indirect - focuses on the legal and policy frameworks and the associated

strengthening of local and national institutions. This implies encouraging significant changes in the ways

cities are managed, their capacity and financial resources, including increased fiscal transfers from higher

tiers of government and/or the better management of local revenue sources. These legal and policy

changes should have significant direct and indirect long-term impacts on the way cities are run and

develop.

18. None of the products and services at Tier III can be delivered outside effective day-to-day operations in the

Secretariat. Successfully delivering the mission and the 2018-21 Strategic Plan requires constant organisational

changes aimed at optimising the efficiency and flexibility of the Secretariat. Changes in the international aid

architecture, a more crowded city space, and evolution in funding patterns with more complex funding modalities

require a more attractive, transparent, and cost-effective business model – one that leverages the best of its United

Nations and UNOPS institutional operating environment, communicates effectively, and provides excellence in

delivery and results. Tier IV description and indicators allow the organisation to continuously track its

performance and provide amelioration to its systems and processes.

19. Relation to the business model. With the implementation of the new Strategic Plan, the Cities Alliance

works through two main funding windows: a technical assistance and operational country/city window centred on

the Country Programme model, and a normative global window, centred on the analytical and collaborative

approaches of global thematic programmes, such as the Joint Work Programmes (JWPs). Because of the nature

of the work they support, the Country Programmes will have more direct and attributable/contributable effects on

Tier II. However, the same logic and narrative of change applies to the global window work, although in a more

indirect fashion. The assumption is that all global programmes - through their diagnostic work, advocacy and

coordination efforts - contribute to raise the profile of cities and operationalise global issues to local contexts.

Their campaigns and new technical approaches help to draw attention and corrective actions to the way cities are

managed, their capacity and financial resources. The tools and expertise these programmes produce are meant to

find direct and practical application in local contexts, especially secondary cities.

Assumptions

20. Critical assumptions are generally referred to as conditions which can affect the achievement of results in

the Results Framework, but over which Cities Alliance has no direct control. Critical assumptions may also be

expressed as risks or vulnerabilities as they are opposite sides of the same coin. In fact, risks are negative

statements about what might go wrong, whereas assumptions are positive statements about the conditions that

need to be met if the results are to be achieved. Understanding and analysing these assumptions/risks is an

important part of the planning process in order to improve wherever possible the robustness of developmental

intervention.

21. The critical assumptions for Secretariat’s operations (Tier IV) are fully captured and detailed in the Cities

Alliance Risk Management Framework. Without an effective response to these assumptions/risks, the Secretariat

would not be able in the first instance to operate and run any programmatic funded activity and deliver against its

annual budget and work plan.

22. With regard to the programmatic results (Tier III) and their effects on cities (Tier II), the assumptions are

captured in the relevant Cities Alliance documents which define multi-year global/country programmes

(Programme Framework Documents). The analysis of risks is carried out against standard risk categories and

dimensions (stakeholder, country, institutions, governance, capacity, fraud/corruption, and project design risks).

The assumptions/risks for each programme are generally very context-specific. Nevertheless, a few are typical

assumptions/risks on which programmes of the Cities Alliance, and ultimately the overall developmental logic of

the organisation is strictly contingent upon. The most recurrent assumptions/risks are the following:

Stability of national governments and the operating environment, including consistency in government

approaches and policies;

Consistency in local governments commitment and ownership with smooth transfer of power through

electoral cycles;

of 20

Existing social cohesion and social capital able to ensure that local communities can shape policy and

act as a catalyst for change;

Ability and capacity of partners and stakeholders to operate at standard level of effectiveness and deliver

on commitments.

Ability to leverage follow up investments so that the Cities Alliance’s technical assistance can be

effective on the ground translating into infrastructure and/or other types of interventions.

of 20

III. The Corporate Core Indicators

Hierarchy

TIER I. Developmental Context (Macro level indicators)

Selected SDGs

TIER II. Client Level Results (City and Neighbourhood level indicators)

Result Area 1: Municipal Government

Result Area 2: Services

Result Area 3: Citizenship

Result Area 4: Local Economy

TIER III. Programme Results (Programme level indicators)

Result Area 1: Urban policies

Result Area 2: Local strategies and plans

Result Area 3: Infrastructure and investment

Result Area 4: Capacity development

Result Area 5: Civil society and communities

TIER IV. Secretariat Results (Key Performance Indicators)

Performance Area 1: Partnership

Performance Area 2: Volume

Performance Area 3: Efficiency

Performance Area 4: Sustainability

of 20

Indicators List

TIER I - DEVELOPMENT CONTEXT

RESULT

DESCRIPTION INDICATORS CRITERIA

I.1 Improved quality

of life, socio-economic

condition and inclusion of the urban

poor.

I.1.1 Proportion of urban population living in slums, informal settlements or inadequate

housing (SDG indicator 11.1.1)

%

I.1.2 Urban poverty gap at national poverty lines (%) (SDG indicator 1.1.1) %

I.1.3 Proportion of total adult population with secure tenure rights to land (SDG indicator 1.4.2)

%

1.1.4 Proportion of seats held by women in (a) national parliaments and (b) local

governments (SDG indicator 5.5.1)

%

TIER II - CLIENT RESULTS

RESULT


TARGETS3

[2021]

II.1 Cities

increasingly

characterised by effective local

government,

active citizenship, and

delivering

improved and responsive

services to the

urban poor.

II.1.1 [Municipal Government] Average municipal expenditures per person

per year

US$ % increase on

baseline

II.1.2 [Municipal Government] Total municipal revenue per year

US$ % increase on baseline

II.1.3 [Municipal Government] Average number of municipal employees as a percentage of the total population

% % increase on baseline

II.1.4 [Municipal Government] Average number of women among municipal

employees

% % increase on

baseline

II.1.5 [Municipal Government] Proportion of municipal employees with

post-secondary education.

% % increase on

baseline

II.1.6 [Services] Proportion of population in slum and/or low-income areas

with regular access to safely managed drinking water services (SDG 6.1.1)

% % increase on

baseline

II.1.7 [Services] Proportion of population in slum and/or low-income areas

using safely managed sanitation services

% % increase on

baseline

II.1.8 [Services] Proportion of population in slum and/or low-income areas with regular electricity connections


II.1.9 [Services] Proportion of population in slum and/or low-income areas with regular access to solid waste collection


II.1.10 [Citizenship] Average percentage of voter participation in most recent

local elections

% of all eligible

voters

% increase on

baseline

II.1.11 [Citizenship] Average ratings on participatory planning process in

place (budgetary or other) (SDG 16.7.2)

Scale [0-2] 1,5

II.1.12 [Local Economy] Average rating of the informal economy working

environment

Scale [0-2] 1,5

3 Specific targets for TIER II are defined for each country programme as they are very contextual. Typically they imply a 5% positive change over bassline value.

of 20

TIER III - PROGRAMME RESULTS

RESULT


TARGETS4

[2021]

III.1 National policy frameworks developed

and/or enhanced to

address urban development needs.

III.1.1 Number of urban policies at the national level developed and/or updated

Unit (aggregate from scale: values = or >

2)

7

III.1.2 Number of urban dialogues which delivered

strategic, policy and/or normative influence

Unit 12

III.2 Local strategies and

plans developed towards

effective urban development

III.2.1 Number of local strategies/plans developed Unit (aggregate from

scale: values = or >

2)

60

III.3 Infrastructure and

leveraging of funds

III.3.1 Number of beneficiaries of infrastructure projects

Unit 1.8mn

III.3.2 Amount of funds leveraged for investments in cities

US$ total value (,000) USD 420mn5

III.4 Capacities

strengthened in city

governance and

management areas such as strategic planning,

financial management,

and human resources management.

III.4.1 Number of urban institutions (Cities Alliance

members, local governments, national public

organisations, universities, training institutions,

associations of cities, etc.) with strengthened capacities

Unit (aggregate from


2)

676

III.4.2 Number of people (professionals in the national

and local governments, community representatives, civil

society, etc.) with strengthened capacities

Unit 7,000

III.4.3 Number of toolkits and other TA products with

evidence of uptake by the stakeholders and/or

beneficiaries

Unit 20

III.5 Mechanisms

developed to engage

citizens in city/urban governance

III.5.1 Number of regularly functioning mechanisms

developed to engage communities and civil society in

urban governance

Unit (aggregate from


2)

60

TIER IV - SECRETARIAT RESULTS

RESULT


YEARLY

PERFORMANCE

STANDARD

IV.1 Partnership IV.1.1 Multi-member new programmes/initiatives per year Unit 2

IV.1.2 Total co-financing per programme per year US$ total value (,000)

700

IV.1.3 Members’ impression of Secretariat effectiveness Scale AVG score 4 (out of 5)

IV.2 Volume IV.2.1 Number of TA activities approved Unit 30

IV.2.2 Total value of TA activities approved US$ (,000) 10,000

IV.2.3 Knowledge products produced with grant financing by

members, partners and the Secretariat

Unit 15

IV.2.4 Policy dialogues and formal learning events that are financed by grants and implemented by members, partners and the Secretariat

Unit 10

4 Targets for Tier III are calculated on a minimum of 7 fully active country programmes covering a universe of 60 cities over the four-year

period of the Strategic Plan 2018-21. Targets are also calculated on rate of delivery averages and previous results obtained from earlier country

programmes.

5 Calculated on seven times the initial TA investment

6 Calculated on all universe of cities (60) and at least one national institution per country

of 20

IV.3 Efficiency IV.3.1 Grant Making: Average time from initial submission of

proposal to approval of grant

Days 70

IV.3.2 Grant Making: Average time from approval of grant to grant

agreement

Days 30

IV.3.3 Grant Making Efficiency: Average time from grant agreement

to first disbursement

Days 10

IV.3.4 Grant Making Efficiency: Average time from final

disbursement to closing

Days 120

IV.3.5 TA activities effectively supervised % of total reports received

90%

IV.3.6 Audience access to knowledge products Unique Visitor Access

70,000

IV.4

Sustainability

IV.4.1 Secretariat staff capacity on Gender Mainstreaming % positive feedback

ratings

75%

IV.4.2 Secretariat Greenhouse Gas Emissions performance (tonnes

CO2 equivalent)

Average emissions

per staff

5

IV.4.3 Secretariat Delivery Performance % completed

activities

100%

IV.4.4 Cities Alliance revenue growth rate % 15%

of 20

Indicators Definitions

TIER I: DEVELOPMENT CONTEXT

I.1. Improved quality of life, socio-economic

condition and inclusion of the urban poor

I.1.1 Proportion of urban population living in

slums, informal settlements or inadequate

housing (SDG indicator 11.1.1): The urban

population living in slums, informal settlements or

inadequate housing (numerator) divided by the total

urban population (denominator), expressed as a

percentage.

Sources: https://unstats.un.org/sdgs/metadata/

I.1.2 Urban poverty gap at national poverty lines

(%) (equivalent to SDG indicator 1.1.1): Urban

poverty gap at national poverty lines is the urban

population's mean shortfall from the poverty lines

(counting the nonpoor as having zero shortfall) as a

percentage of the poverty lines. This measure

reflects the depth of poverty as well as its incidence.

Sources: https://data.worldbank.org

I.1.3 Proportion of total adult population with

secure tenure to land (SDG indicator 1.4.2): Indicator is composed of two parts: (A) measures the

incidence of adults with legally recognized

documentation over land among the total adult

population; while (B) focuses on the incidence of

adults who report having perceived secure rights to

land among the adult population.


I.1.4 Proportion of seats held by women in local

governments (SDG indicator 5.5.1): Indicator

measures the proportion of positions held by women

in local government. It is expressed as a percentage

of elected positions held by women in legislative/

deliberative bodies of local government.


TIER II: CLIENT RESULTS

II.1. Cities increasingly characterised by effective

local government, active citizenship, and delivering

improved and responsive services to the urban poor.

II.1.1 Average municipal expenditure per person

per year [Municipal Government]. Numerator:

Total operating expenditures of municipality in a

given year. Denominator: total population of

municipality in same year. Average expressed in

US$.

Sources: Finance department of municipality;

national population census and population

estimates.

II.1.2 Total municipal revenue per year

[Municipal Government]. Total annual revenue

generated by the local government from sources

other than direct central government transfers.

Figure expressed in USD.

Sources: Finance department of municipality.

II.1.3 Average number of municipal employees as

a percentage of the total population [Municipal

Government]. Numerator: Total number of

employees directly or indirectly employed by the

municipality in a given year. Denominator: Total

population of municipality in same year. Figure

expressed as a percentage.

Sources: Human Resources department of

municipality; national population census and

population estimates.

II.1.4 Average number of women among

municipal employees [Municipal Government].

Numerator: Total number of women directly or

indirectly employed by the municipality in a given

year. Denominator: Total number of municipal

employees in same year. Figure expressed as a

percentage.




II.1.5 Proportion of municipal employees with

post-secondary education [Municipal

Government]. Numerator: Number of well-trained

employees (engineers, technical experts, etc.) in a

municipality in a given year. Denominator: Total

number of municipal employees in the same year.

Figure expressed as a percentage.




II.1.6 Proportion of population living in slums,

informal settlements or inadequate housing areas

with access to safely managed drinking water

services (equivalent to SDG indicator 6.1.1)

[Services]. Proportion of population using safely

managed drinking water services is currently being

measured by the proportion of population using an

improved basic drinking water source which is

located on premises, available when needed and free

of faecal (and priority chemical) contamination.

‘Improved’ drinking water sources include: piped

water into dwelling, yard or plot; public taps or

standpipes; boreholes or tubewells; protected dug

wells; protected springs; packaged water; delivered

water and rainwater. Numerator: Population living

in slums, informal settlements or inadequate housing

of 20

areas with access to safely managed drinking water

services. Denominator: Total population living in

slums, informal settlements or inadequate housing

areas. Figure expressed as a percentage.

Sources: Municipal water/sanitation departments;

surveys.



using safely managed sanitation services

(equivalent to SDG indicator 6.2.1) [Services].

‘Safe’ sanitation facilities include: flush or pour

flush toilets to sewer systems, septic tanks or pit

latrines, ventilated improved pit latrines, pit latrines

with a slab, and composting toilets. Numerator:

Population living in slums, informal settlements or

inadequate housing areas with access to safely

managed sanitation services. Denominator: Total


inadequate housing areas. Figure expressed as a

percentage.

Sources: Municipal water/sanitation departments;

surveys.



with regular electricity connections (equivalent

to SDG indicator 7.1.1) [Services]. Access to

electricity refers mainly to connection to the grid,

but it also includes other reliable off-grid electricity

sources such as solar energy. Numerator: Population

living in slums, informal settlements or inadequate

housing areas with connection to electricity.

Denominator: Total population living in slums,

informal settlements or inadequate housing areas.

Figure expressed as a percentage.

Sources: Municipal/local electricity supply agency;

surveys



with access to regular solid waste collection

(either publicly or privately) (equivalent to SDG

indicator 11.6.1) [Services]. Regularly Collected

Municipal Solid Waste refers to municipal solid

waste that is routinely collected from specific

addresses or designated collection points. Waste

collection is conducted directly by municipal

authorities or private contractors

licensed/commissioned by municipal authorities

with a regular schedule of the day of the week and

time of collection. In some cases, private waste

collection companies have contracts with clients

individually and provide collection services.

Numerator: Population living in slums, informal

settlements or inadequate housing areas that are

served by regular solid waste collection (either

publicly or privately). Denominator: Total


inadequate housing areas. Figure expressed as a

percentage.

Sources: Municipal sanitation departments; surveys

II.1.10 Average percentage of voter participation

in most recent local elections [Citizenship]. Numerator: Number of eligible voters who voted in

most recent local elections. Denominator: Number

of eligible (or registered) voters in municipality for

the same election. Figure expressed as a percentage.

Sources: GCIF, election registers.

II.1.11 Average rating of participatory planning

processes in place (budgetary or other)

[Citizenship]. Participatory planning is a tool for

identifying the collective needs of all individuals

within a community, a way of building consensus,

and a means of empowering disadvantaged or

disenfranchised groups (World Bank). Rating scale:

Sources: Surveys and interviews

II.1.12 Average rating of the informal economy

working environment [Local Economy]. Informal

economy as described by ILO. This indicator

measures the extent to which the informal economy

is recognised and supported the national and local

government. Rating scale:

Sources: Surveys and interviews

TIER III: PROGRAMME RESULTS

III.1 National policy frameworks developed and/or

enhanced to address urban development needs.

III.1.1 Number of urban policies at the national

level developed and/or updated [Urban Policies]. The indicator counts the maturity stage and number

of urban-related policies at the national level

developed and/or updated through the Cities

Alliance. Policies on urban development may

include sectoral policies covering some or all of the

following aspects: housing, slum upgrading,

transport, land, fiscal decentralisation.

0 Little or no participatory planning

1 Participatory planning processes are in place

but are ad hoc and irregular

2 Participatory planning processes are in

place, formalised and used regularly.

0 No support to the informal economy /

hostility towards the informal economy

1 Ad hoc and unsystematic support to the

informal economy

2 Systematic and regular support to the

informal economy

http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTSOCIALPROTECTION/EXTSF/0,,contentMDK:20706647~menuPK:1561737~pagePK:148956~piPK:216618~theSitePK:396378,00.html

https://www.ilo.org/ilostat-files/Documents/description_IFL_EN.pdf

of 20

Unit: Number (#) and Rating scale

0 Policy not developed

1 Policy development/update in process

2 Policy development/update completed

Sources: Copies of the official policies; Secretariat

records.

III.1.2 Number of urban dialogues which

delivered strategic, policy and/or normative

influence [Urban Policies]. The indicator counts

the number of urban dialogues shaped by the Cities

Alliances at global, national and local level which

have had some form of influence on urban

policies/thinking. Influence is captured in one or

more of the following broad dimensions: increased

interest/knowledge on urban issues, increased

alignment and partnerships, increased visibility and

prominence of urban issues, significant quantity and

profiles of attendees, significant follow-up actions,

shift in certain values/beliefs (e.g. on forced

evictions), adoption of policy recommendations.

Sources: Secretariat records, feedback survey,

interviews, fact-finding stories.

III.2 Local strategies and plans developed towards

effective urban development

III.2.1 Number of local strategies/plans

developed [Local Strategies/Plans]. The indicator

measures the maturity stage and number of

strategies/plans developed in cities in which Cities

Alliance works such as city development strategies

(CDSs), slum upgrading strategies, resilience plans,

investment plans, etc.


0 Strategy/Plan not developed

1 Strategy/Plan development in process

2 Strategy/Plan development completed

Sources: Copies of the strategies/plans, and

Secretariat records.

III.3 Infrastructure and leveraging of funds

III.3.1 Number of beneficiaries of infrastructure

projects [Infrastructure and Investment].

This indicator counts the number of people who

have directly benefitted from infrastructure projects

implemented through Cities Alliance funding such

as Community Upgrading Fund (CUF) projects.

Unit: Number (#)

Sources: Secretariat records

III.3.2 Amount of funds leveraged for

investments in cities [Infrastructure and

Investment].

This indicator measures the amount of co-, parallel

and follow up funds committed by other partners

(local and international) towards urban projects as a

result of investments by the Cities Alliance.

Unit: USD


III.4 Capacities strengthened in city governance and

management areas such as strategic planning,

financial management, and human resources

management.

III.4.1 Number of urban institutions (Cities

Alliance members, local governments, national

government units/agencies, public organisations,

universities, training institutions, associations of

cities, etc.) whose capacities have been

strengthened [Capacity Development].

This indicator counts the number of institutions

(local governments, national public organisations,


cities, etc.) whose capacities have been strengthened

in city governance and management areas such as

strategic planning, financial management, and

human resources management. It also monitors the

extent to which the engagement of Cities Alliance

members in country-based and/or global

programmes have contributed to a change in

members’ corporate practices and policies.


0 Institutional capacity not strengthened

1 Institutional capacity strengthening in

process

2 Institutional capacity strengthening

completed

Sources: Secretariat records, programme-based

member survey

III.4.2 Number of people (local and national

government officials and technicians, community

representatives, civil society, etc.) whose

capacities have been strengthened [Capacity

Development].

This indicator counts the number of people (local

governments, national public organisations,


cities, etc.) whose capacities have been strengthened

in city governance and management areas such as

strategic planning, financial management, human

resources management, community project

management etc. Unit: Number (#)


III.4.3 Number of toolkits and other TA products

with evidence of uptake by the stakeholders

and/or beneficiaries

The indicator counts the number of toolkits or

similar knowledge products on urban issues that

have been developed, synthesises and/or updated by

of 20

the Cities Alliance and show sign of uptake by the

stakeholders/beneficiaries. Toolkits are understood

as thematic guidelines and practitioners’ materials to

inform technical assistance programmes. These

toolkits may derive for example from a global

review of case studies, national and local diagnostic

work and/or a review of existing practices and

guidelines, etc. Unit: Number (#)

Sources: Copies of the toolkits and Secretariat

records.

III.5 Mechanisms developed to engage citizens in

city/urban governance

III.5.1 Number of participation mechanisms

developed to engage citizens in city governance

[Civil Society and Communities].

This indicator rates the degree of participation by

citizens - with a specific focus on slum dwellers,

informal workers and civil society - in city

governance by counting governance mechanisms

such as social accountability mechanisms, slum

development committees, informal workers’

associations, municipal fora, etc.


0 Mechanism not developed

1 Development of mechanism in process

2 Development of mechanism completed

Sources: Secretariat records.

TIER IV: SECRETARIAT RESULTS

IV.1.1 Multi-member new programmes and/or

initiatives per year [Partnership]. Indicator

measures the number of formalised cooperation

frameworks involving two or more members in a

given year as a measure degree of the success of the

Secretariat convening process. Forms of formalised

cooperation may be: framework documents for

Country Programmes; MOUs; resolution of

partners; statement of agreement. Multi-member is

defined as two or more Cities Alliance members.

Source: Secretariat records

IV.1.2 Scaling: Total co-financing per

programme per year [Partnership]. Indicator

measures total co-funding contributed in a given

year to a specific programme by partners directly

and/or jointly fundraised. It also calculates the value

ratio of the total funds per Secretariat funding.

Source: Secretariat records.

IV.1.3 Members’ impression of Secretariat

effectiveness [Partnership]. Average rating by

members in a given year. Scale of five (1 – very

unsatisfactory; 5 – very satisfactory) on selected

statements.

Sources: Cities Alliance Secretariat yearly survey

of members

IV.2.1 TA activities (CP, JWP and Innovation

Fund) approved [Volume]. Indicator measures the

total number of TA activities [both grants and

contracts] approved in a given year following the

appraisal process.

Source: Secretariat records.

IV.2.2 Total value of TA activities (CP, JWP and

Innovation Fund) approved [Volume]. Indicator

measures the total cumulative US$ value funded by

the Cities Alliance of TA activities [both grants and

contracts] approved in a given year following the

appraisal process.

Source: Cities Alliance Secretariat records.

IV.2.3 Knowledge products that are financed by

the Cities Alliance and produced by members,

partners and/or the Secretariat [Volume]. Indicator measures the total number and cost of

knowledge products developed with Cities Alliance

financing, as well as the alignment of the knowledge

products and strategy, and demonstrates clear and

proactive management of the delivery of Cities

Alliance knowledge to targeted audiences.

Knowledge products may include: thematic

publications, published diagnostic studies such as

the CEE ratings, State of the Cities Report (SOCR)

or Urbanisation Review (UR); toolkits; and other

guides, policy papers etc. produced by members and

partners with Cities Alliance Secretariat support and

funding. Generally, a knowledge product should

have a Cities Alliance logo.


IV.2.4 Policy dialogues and formal learning

events that are financed by Cities Alliance and

implemented by members, partners and/or the

Secretariat [Volume]. Indicator measures the total

number of Policy Dialogues, Advocacy and

Knowledge and Learning events that are financed by

grants and carried out by member and partners.

Policy dialogues may include: (i) formal

consultation events with members and/or relevant

institutions (e.g., IBSA; Policy Advisory Forum,

100RC); (ii) Advocacy/ Communications events

(e.g., seminars/workshops at Africities, WUF).

Formal learning exchanges could include: peer-to-

peer events and study tours, learning workshops and

seminars.

Source: Cities Alliance Secretariat records

IV.3.1 Grant Making - Average time from initial

submission of proposal to approval of grant

[Efficiency]. Average time, in days, from initial

submission of proposal to approval of grant for

projects completing this phase in a given year.

of 20


IV.3.2 Grant Making - Average time from

approval of grant to grant agreement

[Efficiency]. Average time, in days, from approval

of grant to signature of grant agreement for projects

whose agreement was signed in a given year.


IV.3.3 Grant Making - Average time from grant

agreement to first disbursement [Efficiency].

Average time, in days, from signature of grant

agreement to first disbursement for projects

receiving first disbursement in a given year.


IV.3.4 Grant Making - Average time from final

disbursement to closing [Efficiency]. Average

time, in days, from final disbursement to closing for

projects closed in a given year.


IV.3.5 TA activities effectively supervised

[Efficiency]. Indicator measures quality of

supervision. Percent of grants and contracts with

progress and completion reports that include

information on process and results achieved in a

given year. Numerator: number of grants/contracts

with at least 75% of all required progress and

completion reports. Denominator: Total number of

TA activities supervised.


IV.3.6 Audience access to knowledge products

[Efficiency]. Indicator measures the effective

distribution of knowledge products via the Cities

Alliance website (number of unique visitors to the

CA website on specific knowledge

pages/downloads from targeted countries). Total

number of unique visitors to the CA website from

targeted countries.


IV.4.1 Secretariat staff capacity on Gender

Mainstreaming [Sustainability]. Average

feedback rating by staff in a given year on selected

statements evaluating workshops and other capacity

development activities focused on gender.

Source: Cities Alliance Secretariat feedback and

evaluation forms.

IV.4.2 Secretariat Greenhouse Gas Emissions

performance [Sustainability]. Average emissions

per Cities Alliance staff (tonnes CO2 equivalent)

calculated on the following sources: Air travel, On-

site Electricity, On-site Refrigerants, Public

transport during official travel, Purchased

heat/steam, CFC/HCFCs.

Source: UNOPS GHG Annual Inventory as part of

Greening the Blue initiative.

IV.4.3 Secretariat Delivery Performance

[Sustainability]. Indicators measures the rate of

completed activities against the approved annual

work plan in a given year.

Source: Cities Alliance Secretariat Annual Work

Plan reviews.

IV.4.4 Cities Alliance revenues growth rate

[Sustainability]. Revenue Growth Rate measures

the year-over-year percentage increase in revenue.

Revenue Nominator: Revenue current year.

Denominator: revenue previous year.

Source: Cities Alliance Secretariat accounting

records.

http://www.greeningtheblue.org/

of 20

IV. Period 2018-21. Universe, Frequency, Methods and Targets/Performance Standards

23. This phase of the RF/RBM implementation is for the period of the current Strategic Plan from 2018 to 2021.

Currently it covers 2 countries, 21 cities of operations, and 8 multi-year TA programmes. As new operations are

rolled out, the universe of measurement will be enlarged.

RF Tier Universe Type of indicators Data Collection

Methods

Frequency of data

update

Targets

(Y/N) (when)

I Country of operations Quantitative Mainly desk review of SDG’s indicators

from UN Stats and

National Institute of Statistics in partner

countries

Every year [provided availability of SDG

data].

Yearly results are included in the annual

report and scorecard

Not applicable since this is the macro

developmental

context

II Cities and neighbourhood of

sustained operations

Quantitative and maturity scales

Primary data studies/surveys with

private/public entities

contracted through public procurement

In cities with large scale programmes

once every year.



Yes (end-line year in 2021/2022 in

accordance with the

current Strategic Plan) – typically is a

5% positive change

III All portfolio

programmes and projects

Quantitative and

maturity scales

Mainly through desk

reviews of project reports and

documentation

Every year.



Yes (end-line year in

2021/2022 in accordance with the

current Strategic

Plan)

IV All Secretariat

transactions

Quantitative KPIs

Internal through

Cities Alliance databases

Every year.



Yearly performance

standards

of 20

V. Corporate Reporting

24. As said above, the Results Framework is a central part of a results-based management system, which

operationalises the performance indicators into baselines, milestones and targets, data sources as well as tools and

frequency for data collection, and operates across all programmes. The data gathered in the RBM will be reported

through two tools: a result-based annual report and a corporate scorecard.

Results-Based Annual Report

25. As part of its accountability requirements to its Governing Bodies and related commitments to development

partners, clients and other relevant stakeholders, the Cities Alliance Secretariat shall prepare a yearly report that

outlines progress made towards programmatic objectives and stated results at Tiers II (client results), III

(programme results) and IV (secretariat results). Grounded in the data collected through the projects and

programmes' baseline studies and progress and completion reports, the annual report should provide an aggregate

account of progress along with evidence-based explanations of variances in reported achievements, whether

positive or negative7. The annual report is issued at the end of Q1 of a given year and retrospectively covers the

calendar year before.

26. While reported achievements should be tied in with the actual work for which the Cities Alliance and its

Members/Partners are accountable (Tiers III & IV), results reports should also speak to the Alliance’s overall aim

of enabling cities to be more effective, participatory and able to deliver improved, responsive services to the urban

poor (Tiers I & II). In other words, annual results reports should provide an overview of sectoral and city-wide

progress towards stated development results (Tiers I and II), while accounting for how the Cities Alliance and its

partners contributed to those results (Tiers III and IV). The narrative of the annual report will allow tying the

various tiers together when a coherent causal story of impact can be traced. As such, the annual results report is

the primary instrument through which the Cities Alliance communicates its story to the Assembly and

Management Board, beneficiaries, partners, and to the wider public. It should offer a snapshot of the Alliance’s

overall performance, facilitate decision-making, and any significant changes in the internal and external context

that either affected or will ultimately affect the portfolio effectiveness. While data should provide the backbone

of the Annual Report, they should also be complemented by the most significant results (MSC) that have been

achieved in the calendar year. According to the MSC theory, these are results which are not captured by indicators

but may still be highly noteworthy.

Corporate Scorecard

27. Increasingly, international organisations such as the World Bank group and various UN agencies are relying

more and more on scorecard indexes and dashboard or ‘traffic light’ systems to showcase their contributions. The

reasons for this are many. First, scorecards provide a quantitative approach that is fairly rigorous. It relies on

objectively identifiable indicators that can be reliably measured. Second, by associating quantitative results with

a universally recognised colour-coding system (i.e., the dashboard or ‘traffic light’ system component) an observer

can readily appreciate areas where progress is on track versus areas where further improvements are warranted.

As the World Bank’s own experience demonstrates, the scorecard approach can facilitate strategic dialogue

between Management and the Board on progress made and areas that need attention. Finally, the scorecard

approach helps to create a living document that can be used to continuously monitor progress towards results and

ultimately improvements over time as an organisation’s ability to report on results increases, leading to the

development of more refined outcome indicators that can give a more accurate measure of progress made.

28. Quantifiable indicator measures are used along with a corresponding colour coding system to facilitate

analysis and draw the reader’s attention to emerging concerns. Green or blue are used to indicate areas where

progress is on track; yellow points to issues that need to be watched more closely and where performance is

7 Most multilateral organizations issue at least two yearly reports, an annual report that is presented at annual executive meetings and includes

detailed financial information in addition to a description of activities undertaken during the year under review, and more recently an annual results report, that concentrates on the results obtained from their operations and normally includes or is based on an organizational results

framework. Given the size of its operations, the Cities Alliance opted for a combined yearly report, which should serve as the main prospectus

of the Cities Alliance’s work and effectiveness contributions, to which traditional information pertaining to financials, human resources, and other relevant issues are appended.

of 20

improving, relative to baseline data; and red is used to highlight areas where performance is either off track or not

improving. The table (below) provides a description of the ‘traffic-light’ system and the correspondent coding for

quantitative values. The Aggregate Results column indicates the mean distribution of results and corresponding

colour scheme for qualitative/maturity scale in instances where the performance of two or more country

programmes are collated and averaged out for indicators based on ratings. Numerical ratings should be used to

aggregate results only. Final scorecard balance sheets should show the colour only. The definitions used in the

table borrow heavily from the World Bank’s approach. The Corporate Scorecard is published on an annual basis

in conjunction with the Annual Report.

Scorecard Rating System

Quantitative

coding

Aggregate Results

for Maturity scales

Definition

<50% 0 – 0.7 CHALLENGE. For indicators based on targets (Tiers II&III), indicator shows a decrease from baseline and/or has failed in achieving the established target. For

indicators based on performance standards (Tier IV), indicator is significantly far under

the established performance standard.

50-75% 0.7 – 1.5 WATCH. For indicators based on targets (Tiers II&III), indicator shows no significant

increase or decrease from baseline and/or has not yet achieved the established target. For

indicators based on performance standards (Tier IV), indicator is under the established

performance standard although within tolerance.

> 75% 1.5 – 2 ON TRACK. For indicators based on targets (Tiers II&III), indicator shows significant

increase from baseline and/or has achieved the established target. For indicators based

on performance standards (Tier IV), indicator meets/exceeds the established

performance standard.

Sustained >100% 2 (sustained) SUSTAINABLE. Targets/Performance standards are consistently achieved and

mechanisms/processes underlying change are institutionalised and/or maintained

without external assistance.

N/A White NOT APPLICABLE. There is insufficient data to establish a trend, or there is no target

or performance standard.

THE CITIES ALLIANCE RESULTS FRAMEWORK AND RESULTS … · visionary global agenda to date. For the...

Documents

Transcript of THE CITIES ALLIANCE RESULTS FRAMEWORK AND RESULTS … · visionary global agenda to date. For the...