MAPP Marcos C. Holanda marcosholanda2@gmail

MAPPMarcos C. [email protected]

Monitoring and Evaluation of Priority

Programs

Banco Interamericano

de Desenvolvimento

Management for Development Results at

Sub-national Governments


de Desenvolvimento

RBM - MAPP

RBM – Governance StrategyRBM – Governance Strategy

MAPP – Operational MAPP – Operational InstrumentInstrument


de Desenvolvimento

It is important to differentiate

PlanningPlanning Projects selection Projects selection

AdministrationAdministration Projects execution Projects execution

ManagementManagement Systemic Systemic coordination of projects coordination of projects based on based on resultsresults


de Desenvolvimento

What RBM is About1.1. Do less but do Better.Do less but do Better.2.2. Outcomes prevail over outputs.Outcomes prevail over outputs.3.3. Outcomes put citizens first. Outputs put Outcomes put citizens first. Outputs put

government first.government first.4.4. Outcomes as driving force to programs Outcomes as driving force to programs

integration.integration.5.5. Outcomes to differentiate good from bad Outcomes to differentiate good from bad

programs.programs.6.6. A permanent cycle of learning: Planning-A permanent cycle of learning: Planning-

Budgeting-Monitoring-Evaluation.Budgeting-Monitoring-Evaluation.7.7. An instrument for recognition and compensation An instrument for recognition and compensation

of good performance of government employees.of good performance of government employees.


de Desenvolvimento RBM

GOVERNMENT STRATEGIC OUTCOMES

DEPARTMENT STRATEGIC OUTCOMES

DEPARTMENT PROGRAMS

DEPARTMENT OUTPUTS

DEPARTMENT PROJECTS AND ACTIVITIES

DEZ

IGN

IMPLEM

ENTA

TION

GOVERNMENT PLANGOVERNMENT PLAN


de Desenvolvimento

Requirements for a successful RBM

To have information To have information – outcome and output – outcome and output indicatorsindicators

To know how to use the information To know how to use the information – – information information affect budget decisionsaffect budget decisions

To have a budget based on results To have a budget based on results – RBM – RBM defines defines the budget and not the contrarythe budget and not the contrary

To know what it is and how to do RBM To know what it is and how to do RBM – – Institutional and operational capacity to implement Institutional and operational capacity to implement RBMRBM


de Desenvolvimento

Guidelines for RBM

1.1. Operational flexibility in searching for Operational flexibility in searching for results.results.

2.2. Medium term budget. Medium term budget.

3.3. Increments in the budget are based on Increments in the budget are based on increments in the results.increments in the results.

4.4. Look for “good or better practices” and not Look for “good or better practices” and not for “best practices”.for “best practices”.


de Desenvolvimento

MAPP

MAPP

Planning

Monitoring

Evaluation


de Desenvolvimento

MAPP

1. PLANNING Selection of Projects Application Application CardCard

2. MONITORING Outputs MAPP MonitoringMAPP Monitoring

Macro EvaluationMacro Evaluationa) Government Outcomesb) Integration Matrix

3. EVALUATION Outcomes Micro EvaluationMicro Evaluationa) Design Evaluationb) Impact Evaluation


de Desenvolvimento


de Desenvolvimento

PLANNING Project Project Application CardApplication Card

1.1. The problem to be faced.The problem to be faced.2.2. The government outcomes to be targeted.The government outcomes to be targeted.3.3. The department outcomes to feed the The department outcomes to feed the

government ones.government ones.4.4. The outputs to be delivered.The outputs to be delivered.5.5. The outcomes and outputs indicators.The outcomes and outputs indicators.6.6. The targeted population or region.The targeted population or region.7.7. The time frame of expected outcomes.The time frame of expected outcomes.8.8. The sources of financing.The sources of financing.9.9. The expected efficiency (benefits-costs The expected efficiency (benefits-costs

analysis).analysis).10.10. The expected integration with others projects.The expected integration with others projects.


de Desenvolvimento

PLANNING Project Application Project Application CardCard


de Desenvolvimento

MONITORING MAPP MAPP MonitoringMonitoring


de Desenvolvimento

MONITORING MAPP MAPP MonitoramentoMonitoramento


de Desenvolvimento

MONITORING MAPP MAPP MonitoramentMonitorament

oo


de Desenvolvimento

EVALUATION Macro Macro EvaluationEvaluation

Government Outcomes

Indicadores Sinalização

Nota SAEB e ESPAECE ▬Distorção Idade Série ▲Taxa Escolar Líquida: Ensino Fundamental ▼Taxa Escolar Líquida: Ensino Médio ▲Taxa de Analfabetismo

(>15 anos) ▲(de 7 a 14 anos) ▬

Taxa Mortalidade Infantil ▲Razão Mortalidade Materna ▲Taxa Internação AVC (> 40 anos) ▼Taxa de Roubos ▼Taxa de Furtos ▬Taxa de Homicídio ▼Renda Domiciliar Per Capita (R$) ▲Porcentagem Extrema Pobreza ▲Índice de GINI ▲


Saldo de Empregos Formais (Novos Postos) ▲

Taxa de Crescimento do PIB (%) ▬

Fluxo de Comércio Exterior ▲

PIB Interior / PIB RMF (%) ▼

Consumo de Energia Residencial: Interior / RMF (%)

▲

Empregos Interior / Empregos RMF (%) ▬


Div / PIB (%) ▲

Div / RCL (%) ▲

Resultado Primário ▲

Custeio / RCL (%) ▲

Pessoal / RCL (%) ▲

Gastos Finalísticos / RCL (%) ▬

Investimento / RCL (%) ▼

Número de Acessos às Páginas WEB do Governo ▲

Social OutcomesSocial Outcomes Economic OutcomesEconomic Outcomes Governance OutcomesGovernance Outcomes


de Desenvolvimento

EVALUATION Macro Macro EvaluationEvaluation

Integration MatrixPrograms Integration Matrix

Government Outcome # 1 Government Outcome # 2

Government Outcome # 3

Economic Growth

Program1

Program2

Program3

Program4

Program5

Program6

Result Based Budget


de Desenvolvimento

EVALUATION Micro Micro EvaluationEvaluation

Design Evaluation

Inputs Activities Outputs Outcomes Impacts

Logical FrameworkLogical Framework

MonitoringMonitoring Outputs deliveredOutputs deliveredXX

Planned OutcomesPlanned Outcomes


de Desenvolvimento


Design Evaluation

a) Education: . Primary education schooling rate . Secondary education schooling rate . 4th grade promotion rate.

b) Health: . Infant mortality rate . Cerebral Vascular Accident (CVA) admittance rate

c) Living Conditions: . Proportion of urban domiciles with regular water supply . Proportion of urban domiciles with sewage

d) Employment and income: . Average residential energy consumption . Formal employment quality index . Average establishments’ size

e) Rural Development:.Gross value of the agricultural production per rural establishment . Proportion of energy consumption in rural areas

OUTCOMES OUTPUTS

a) Education:. Proportion of teachers in primary education with bachelor degrees. Proportion of teachers in secondary education with bachelor degrees. Libraries, reading rooms and computer labs per public school; and. Computer equipments per public school.

b) Health:. Prop. of women assisted during the first 3 months of pregnancy. Proportion of children (2 years old or younger) assisted. Proportion of the population assisted by the Family Health Program

c) Living conditions:. Urban water supply coverage rate. Urban sewage coverage rate

d) Employment and income:. Secondary enrollment – total enrollment ratio . Health professionals per 1,000 inhabitants. Paved roads – municipality area ratio.

e) Rural development:. Mean value of rural loans. Assisted producers per rural establishment


de Desenvolvimento


Impact Evaluation

Probability (Recidivism) = F(age, gender, approach period, activity, per capita family income, guardian education, benefit)

Variables Coeficient dY/dX P Value

Age 0.0136 0.0035 0.002

Gender 0.2493 0.0619 0.000

Approach period 0.1606 0.0441 0.001

Activity 0.2590 0.0664 0.000

Per Capita Family Income -0.0050 -0.0013 0.000

Guardian Education -0.1698 -0.0013 0.000

Benefit -0.5001 -0.1344 0.000

Constant -0.8609 - 0.000

Number of observations 8397 Pseudo R2 0.0639Source: IPECE

Program Evaluation: Criança Fora da Rua Dentro da Escola


de Desenvolvimento

LESSONS

1.1. Strategic leadership(Governor, Mayor) and Strategic leadership(Governor, Mayor) and operational leadership(line secretary) are operational leadership(line secretary) are crucial.crucial.

2.2. It is very Important a clear understanding of It is very Important a clear understanding of RBM concepts . RBM concepts .

3.3. It is important to show that RBM is about It is important to show that RBM is about opportunity and not a threat.opportunity and not a threat.

4.4. The selection of output and outcome indicators The selection of output and outcome indicators can be very tricky.can be very tricky.


de Desenvolvimento

LESSONS

5. It is important to explain the difference 5. It is important to explain the difference between Monitoring and Evaluation:between Monitoring and Evaluation:Monitoring Monitoring Good Accounting Good Accounting

Evaluation Evaluation Good Economics Good Economics

6. Before impact evaluation, ask for design 6. Before impact evaluation, ask for design evaluation (quick evaluation) for the main evaluation (quick evaluation) for the main Program/Project.Program/Project.

7. 7. Keep the RBM policy as simple as possible.Keep the RBM policy as simple as possible.

8. 8. RBM requires 3 P’s ( a different kind of PPP): RBM requires 3 P’s ( a different kind of PPP): Patience + Persistence + PassionPatience + Persistence + Passion


de Desenvolvimento

Thanks!

[email protected]@gmail.com

MAPP Marcos C. Holanda marcosholanda2@gmail

Documents

Transcript of MAPP Marcos C. Holanda marcosholanda2@gmail