Document of The World Bank · SAERJ Rio de Janeiro State Education Evaluation System (Sistema de...

i

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: RES14017

RESTRUCTURING PAPER

ON A

PROPOSED PROJECT RESTRUCTURING

OF

RIO DE JANEIRO

RENOVATING AND STRENGTHENING PUBLIC MANAGEMENT TECHNICAL

ASSISTANCE PROJECT

August 26, 2010

TO THE

STATE OF RIO DE JANEIRO

WITH THE GUARANTEE OF THE FEDERATIVE REPUBLIC OF BRAZIL

October 30, 2014

Governance Global Practice

Country Management Unit for Brazil

Latin America and Caribbean Region

This document has a restricted distribution and may be used by recipients only in the performance of their official

duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

ABBREVIATIONS AND ACRONYMS

AF Additional Financing

AGE State Audit Agency (Auditoria Geral do Estado)

CPS Country Partnership Strategy

DPL Development Policy Loan

DRM Disaster Risk Management

EMF Environmental Management Framework

GDP Gross Domestic Product

GORJ Government of Rio de Janeiro

IBRD International Bank for Reconstruction and Development

IDA International Development Association

IDEB Basic Education Development Index (Índice de Desenvolvimento da Educação

Básica)

IDERJ Rio de Janeiro Education Development Index (Índice de Desenvolvimento da

Educação Básica do Estado do Rio de Janeiro)

IFR Interim Unaudited Financial Reports

INEA State Environmental Institute (Instituto Estadual do Ambiente)

ISR Implementation Status Report

ORAF Operational Risk Assessment Framework

PAHI Program to Support Interior Regional Hospitals (Programa de Apoio aos Hospitais

Regionais do Interior)

PDO Project Development Objective

RioPrevidência State of Rio de Janeiro Single Social Security Fund

SAERJ Rio de Janeiro State Education Evaluation System (Sistema de Avaliação da

Educação do Estado do Rio de Janeiro)

SEEDUC State Secretariat of Education (Secretaria Estadual de Educação)

SEFAZ State Secretariat of Finance (Secretaria Estadual da Fazenda)

SEPLAG State Secretariat of Planning and Management (Secretaria Estadual de Planejamento

e Gestão)

SES State Secretariat of Health (Secretaria Estadual de Saúde)

SISPAT State Real Estate Management System (Sistema de Gestão do Patrimônio

Imobiliário do Estado)

SUAS Social Assistance System (Sistema Único de Assistência Social)

TERV Waiting time for patients with “green” risk level classification (Tempo de espera do

paciente classificado como risco verde desde o acolhimento)

UGP Project Management Unit (Unidade Gestora de Projeto)

UPA Urgent Care Facility (Unidade de Pronto Atendimento)

Regional Vice President: Jorge Familiar

Country Director: Deborah L. Wetzel

Senior Practice Director: Mario Marcel Cullel

Practice Manager: Arturo Herrera Gutiérrez

Task Team Leaders: Lorena Viñuela and Roland N. Clarke

iii

BRAZIL

RIO DE JANEIRO

RENOVATING AND STRENGTHENING PUBLIC MANAGEMENT TECHNICAL

ASSISTANCE PROJECT RIO DE JANEIRO

CONTENTS

DATA SHEET .................................................................................................................... 4

A. SUMMARY ................................................................................................................ 13

B. PROJECT STATUS.................................................................................................... 13 C. PROPOSED CHANGES ............................................................................................ 14 ANNEX 1: RESULTS FRAMEWORK ........................................................................... 19 ANNEX 2: OPERATIONAL RISK ASSESSMENT FRAMEWORK (ORAF) ............. 30

4

DATA SHEET

DATA SHEET

Brazil

Brazil - Rio de Janeiro Renovating and Strengthening Public Management (P106768)

LATIN AMERICA AND CARIBBEAN

0000009063

Report No: RES14017

Basic Information

Project ID: P106768 Lending Instrument: Investment Project

Financing

Regional Vice President: Jorge Familiar Calderon Original EA Category: Not Required (C)

Country Director: Deborah L. Wetzel Current EA Category: Not Required (C)

Senior Global Practice

Director: Mario Marcel Cullell Original Approval Date: 26-Aug-2010

Practice

Manager/Manager: Arturo Herrera Gutierrez Current Closing Date: 30-Jun-2015

Team Leader: Lorena Vinuela

Roland N. Clarke

Borrower: Secretaria de Planejamento e Gestão Rio de Janeiro

Responsible

Agency:

Restructuring Type

Form Type: Full Restructuring Paper Decision Authority: Board Approval

Restructuring

Level:

Level 1

Financing ( as of 23-Oct-2014 )

Key Dates

Project Ln/Cr/TF Status Approval

Date Signing Date

Effectiveness

Date

Original

Closing Date

Revised

Closing Date

P106768 IBRD-79550 Effective 26-Aug-2010 22-Nov-2010 27-Dec-2010 30-Jun-2015 30-Jun-2015

Disbursements (in Millions)

Project Ln/Cr/TF Status Currency Original Revised Cancelle Disburse Undisbur %

5

d d sed Disburse

d

P106768 IBRD-79550 Effective USD 18.67 18.67 0.00 7.08 11.60 38

Policy Waivers

Does the project depart from the CAS in content or in other significant

respects? Yes [ ] No [ X ]

Does the project require any policy waiver(s)? Yes [ ] No [ X ]

A. Summary of Proposed Changes

Two additional safeguard policies OP/BP4.04 (Natural Habitats) and OP/BP4.12 (Involuntary

Resettlement) are being triggered as a precaution. The restructured project is expected to have positive

indirect environmental impacts, given the focus on capacity building of key institutions of the Government

of Rio de Janeiro including the State Institute of the Environment, and may include policy changes or

actions for disaster risk management that entail temporary or permanent preventive evacuation. The

safeguard policy Environmental Assessment (OP/BP 4.01) was triggered in the original project. An

Environmental Management Framework and a Resettlement Policy Framework prepared by the Borrower

are publicly available.

Change in Implementing Agency Yes [ ] No [ X ]

Change in Project's Development Objectives Yes [ ] No [ X ]

Change in Results Framework Yes [ X ] No [ ]

Change in Safeguard Policies Triggered Yes [ X ] No [ ]

Change of EA category Yes [ X ] No [ ]

Other Changes to Safeguards Yes [ ] No [ X ]

Change in Legal Covenants Yes [ ] No [ X ]

Change in Loan Closing Date(s) Yes [ X ] No [ ]

Cancellations Proposed Yes [ ] No [ X ]

Change to Financing Plan Yes [ X ] No [ ]

Change in Disbursement Arrangements Yes [ ] No [ X ]

Reallocation between Disbursement Categories Yes [ X ] No [ ]

Change in Disbursement Estimates Yes [ X ] No [ ]

Change to Components and Cost Yes [ X ] No [ ]

Change in Institutional Arrangements Yes [ ] No [ X ]

Change in Financial Management Yes [ ] No [ X ]

Change in Procurement Yes [ ] No [ X ]

Change in Implementation Schedule Yes [ X ] No [ ]

Other Change(s) Yes [ X ] No [ ]

Appraisal Summary Change in Economic and Financial Analysis Yes [ ] No [ X ]

Appraisal Summary Change in Technical Analysis Yes [ ] No [ X ]

6

Appraisal Summary Change in Social Analysis Yes [ ] No [ X ]

Appraisal Summary Change in Environmental Analysis Yes [ X ] No [ ]

Appraisal Summary Change in Risk Assessment Yes [ ] No [ X ]

B. Project Status

Implementation progress has accelerated with the completion of major procurement processes and is now

satisfactory. The project has disbursed to date USD 7.1 million and with the completion of several

activities the Government of Rio de Janeiro anticipates that another USD 5.5 million would be disbursed

by December 2014, reaching 67 percent of the loan disbursed.

C. Proposed Changes

Development Objectives/Results

Project Development Objectives

Original PDO

The objective of the project is to support the State of Rio de Janeiro to introduce performance-based

management and information technologies to drive service delivery improvements in key public services,

including secondary education and hospital care.

Change in Project's Development Objectives

Change in Results Framework

Explanation:

The Results Framework is being revised to reflect the changes in subcomponents and the removal of

indicators related to transferred subcomponents that were eliminated as they could no longer be financed

with the project’s resources. Other indicators have been changed to allow for better monitoring of project

activities, improving methodologies used to calculate them and clarifying responsibilities for data

collection and reporting.

Compliance

Change in Safeguard Policies Triggered

Explanation:

Given the focus on capacity building of key institutions of the GORJ, including the State Institute for the

Environment, the restructured Technical Assistance Project is expected to have positive indirect

environmental impacts. The fourth component "Development of Institutional and Technological Capacity

for Preventing and Performing during High-intensity Weather Events," which entails the strengthening of

disaster risk management systems and the acquisition of two weather radar systems may have indirect

environmental implications.

The restructured Project would finance technical assistance activities for disaster risk management and the

preparation of an Action Plan that may include policy changes and actions that may entail temporary or

permanent preventive evacuation. OP/BP 4.12 (Involuntary Resettlement ) and OP/BP 4.04 (Natural

Habitats) are being consequently triggered in a precautionary mode to ensure that actions for handling

adverse events through preventive evacuation will: (i) take into consideration the impacts on the

preventively relocated population, the population that may continue living at the sites because the areas at

high risk that cannot be controlled by any other measure but relocation is not always entirely coterminous

7

with the territory of the community settled at the sites, and the host population in the neighboring

communities of the new settlements; and, (ii) include proper principles and guidelines to be applied by the

Borrower when any land acquisition might be undertaken for the preventive relocation of people living in

risk-prone areas and/or for the post-disaster relocation of affected people in places other than their original

residences and places of business; and, abandoned lands that reverted to the state for relocation were

encumbered by pre-existing use claims.

Additionally, the project's Environmental Assessment category has been revised from "C" to "B".

Operational Policy 4.01 (Environmental Assessment) was triggered under the original project to ensure

that proper consideration is given to potential future negative environmental impacts, as well as to positive

ones that may result from the adoption of sustainable environmental practices (particularly in relation to

management of health and other wastes). In 2010, the borrower prepared a document entitled "Protection

and Guarantees" describing the project components and the potential safeguards issues concerning hospital

waste management. The document summarizes the international hospital facility accreditation

methodology that was to be applied under the project to ensure the proper management of medical wastes.

The subcomponent dealing with hospital accreditation and, consequently medical waste management, was

removed from the project under this restructuring as it could not longer be financed with the existing

resources.

An Environmental Management Framework (EMF) was prepared by the Borrower for the Additional

Financing, later approved as a separate project, the "Strengthening Public Management and Integrated

Territorial Development" Technical Assistance Project (P126735). This EMF covers activities of the

original and the new project, including the activities for the strengthening of disaster risk management

under Component 4. A Resettlement Policy Framework (RPF) was also completed by the Government of

Rio de Janeiro to set the principles and guidelines to be followed by the project and during the preparation

of the Plan for Disaster Risk Management. The EMF and the RPF were disclosed on the Bank's website on

December 19, 2011 and disseminated in Brazil on December 20, 2011.

The Environmental and Social Safeguards Specialists assigned to the project verified during the last

mission of September 2014 that the Borrower's safeguards performance continues to be satisfactory and

that both the EMF and the RF continue to be suitable to guide the implementation of activities related to

disaster risk management under Component 4. The updated Integrated Safeguards Data Sheet discusses

the specific risks involved triggering the safeguard mentioned above.

Current and Proposed Safeguards Policies Triggered: Current Proposed

Environmental Assessment (OP) (BP 4.01) Yes Yes

Natural Habitats (OP) (BP 4.04) No Yes

Forests (OP) (BP 4.36) No No

Pest Management (OP 4.09) No No

Physical Cultural Resources (OP) (BP 4.11) No No

Indigenous Peoples (OP) (BP 4.10) No No

Involuntary Resettlement (OP) (BP 4.12) No Yes

Safety of Dams (OP) (BP 4.37) No No

Projects on International Waterways (OP) (BP 7.50) No No

Projects in Disputed Areas (OP) (BP 7.60) No No

8

Change of EA category

Current EA Category: Not Required (C) Proposed EA Category: Partial Assessment (B)

Explanation:

As part of the preparation of the Additional Financing, later approved as a separate ("Strengthening Public

Management and Integrated Territorial Development Technical Assistance Project" P126735), a new

Environmental Assessment was conducted for both the original and the new project. The Environmental

Assessment was carried out by the Borrower, assessing all sub-projects for potential environmental

impacts. The assessment identified potential positive but indirect environmental impacts resulting from the

strengthening of the State's disaster risk management system which includes the acquisition and

installation of two meteorological radar systems.

The installation of the radars would involve construction activities with limited adverse environmental

impacts. The project does not include conversion or degradation of critical natural habitats, and no

significant impact on natural habitats is expected. However, due to potential unforeseen activities OP/BP

4.04 (Natural Habitats) is being triggered on a precautionary basis.

No direct negative social impacts are expected from project activities. Nevertheless, due to activities

related to the proposal of a Plan for Disaster Risk Management, OP/BP 4.12 (Involuntary Resettlement

Policy) is being triggered to ensure that proper consideration is given to proper principles to guide

handling actions that may require the preventive relocation/evacuation of people living in high-risk areas.

For these reasons, the project's Environmental Assessment has been re-categorized from "C" to "B". An

Environmental Management Framework and a Resettlement Policy Framework (publicly available at:

http://www.progestao.rj.gov.br/salvaguardas_ambientais.asp) were prepared by the Borrower to address

activities' screening and eligibility and to provide guidelines for the effective and timely implementation of

environmental and social safeguards during project implementation.

Financing

Change in Loan Closing Date(s)

Explanation:

The extension request is justified for the unexpected delays in the initiation of activities encountered as a

result of the Borrower’s limited capacity to manage technical assistance and international tendering

processes and the capacity building activities required to overcome this limitation. The time needed to

implement activities related to the acquisition and installation of complex weather radars and information

systems for strengthening disaster risk management, which were introduced under the first restructuring of

2011, was underestimated and 24 additional months are needed to finalize them. An extension from June

30, 2015 to June 30, 2017, if granted, would allow for enough time to complete and deliver ongoing

contracts and activities related to disaster risk management. With this extension, total implementation

period would be 6 years and 6 months, below the average project range.

Ln/Cr/T

F Status

Original Closing

Date

Current Closing

Date

Proposed Closing

Date

Previous Closing

Date(s)

IBRD-

79550 Effective 30-Jun-2015 30-Jun-2015 30-Jun-2017 30-Jun-2015

Change to Financing Plan

Explanation:

9

The activities financed with the Borrower's own resources no longer constitute part of the project as a

result of the transfer of subcomponents to the second Technical Assistance Project "Rio State

Strengthening Public Management and Integrated Territorial Development” (P126735).

Source(s) At Approval Current (from AUS) Proposed

BORR 9,160,000.00 9,160,000.00 46,682.50

IBRD 18,673,000.00 18,673,000.00 18,673,000.00

Total 27,833,000.00 27,833,000.00 18,719,682.50

Reallocations

Explanation:

The Government of Rio de Janeiro has requested the reallocation of funds to better meet the funding

requirements of the project's activities. The change implies transferring US$ 550,000 from the operating

cost category to the first cost category, thus increasing the funding available for the contracting of services

and acquisition of goods to support the implementation of Components 1, 2, 3 and 4.

Ln/Cr/TF Currency Current Category of

Expenditure Allocation

Disbursement % (Type

Total)

Current Proposed Current Proposed

IBRD-79550 USD GO, NCS, CS & TRGN 18,026,317.

50 18,576,317.

50 100.00 100.00

OPERATING COSTS 600,000.00 50,000.00 100.00 100.00

Designated Account 0.00 0.00 0.00 0.00

FRONT END FEE 46,682.50 46,682.50 0.00 0.00

Total: 18,673,000.

00

18,673,000.

00

Disbursement Estimates

Change in Disbursement Estimates

Explanation:

The disbursement schedule was revised to better reflect the implementation schedule that was initially

delayed because of the complexity of tendering and contracting processes.

Fiscal Year Current (USD) Proposed (USD)

2011 63,000.00 46,682.50

2012 3,800,000.00 164,356.54

2013 4,810,000.00 1,293,339.40

2014 8,500,000.00 4,630,364.19

2015 1,500,000.00 8,728,257.37

2016 0.00 3,810,000.00

Total 18,673,000.00 18,673,000.00

Components

10

Change to Components and Cost

Explanation:

The Level 2 restructuring is introducing two new components to better organize the activities and

objectives added in the first restructuring. The new components are (i) Component 4, titled “Development

of Institutional and Technological Capacity for Preventing and Performing during High-intensity Weather

Events”, with US$7.1 million (the activities corresponding to disaster risk management currently under

component 1 will be transferred to this component), and (ii) Component 5, titled “Project Management”,

with US$ 0.2 million.

Component 4 would support the strengthening of the State’s early warning system for high rainfall events.

Activities supported by this component include: (i) the development and implementation of institutional

and operational arrangement for the disaster risk management; and (ii) the acquisition and installation of

two weather radar systems. The activities aim at substantially improving the State’s early warning system

through the use of meteorological radars. This hardware will enable the tracking of storms (volume, speed,

and direction) before it reaches areas at risk. While the ultimate goal is to prevent and reduce the negative

effects of disasters, these activities would also help improve critical service delivery and strengthen public

financial management by reducing the fiscal impact generated by extreme weather events. This component

would support the GORJ in developing a core management system for disaster risk management as well as

to acquire needed equipment. The component would be the responsibility of the State Institute for the

Environment (INEA).

Component 5 would support activities related to the overall management of the project. Under the original

project, US$0.6 million had been allocated to the expenditure category “operating costs” in order to cover

project management activities associated with strengthening of the GORJ’s capacity for project execution,

monitoring, supervision and evaluation. However, some of the required expenditures are related to goods,

consultant services, non-consultant services, and training. Therefore, a new component is being created

and part of the operating costs reallocated to adequately reflect the project’s management needs. The

component would be the responsibility of the State Secretariat of Planning and Management (SEPLAG)

and managed by its Project Management Unit.

Three subcomponents are being eliminated as they could no longer be financed with the loan proceeds.

These include subcomponents 2.2 “School-level Performance Targets for State Schools”, 3.1.

“Accreditation System for UPAs”, and 3.2 “Cost-Accounting Methodology for UPAs and Hospitals”.

The project's budget allocations have been revised to reflect the changes made to accommodate the

introduction of the disaster risk management activities and the transfer of activities that could no longer be

financed by this project to the second Technical Assistance Project "Rio State Strengthening Public

Management and Integrated Territorial Development," which was originally designed as an Additional

Financing to this project, but later approved separately.

Current Component

Name

Proposed Component

Name

Current Cost

(US$M)

Proposed

Cost (US$M) Action

1: Core Government

Systems and

Management Tools for

Improved Performance

and Efficiency

11.87 6.87 Revised

2: Strengthening

Education Management 2.77 2.39 Revised

11

Capacity, Teacher

Instructional Practice and

Incentives for School

Effectiveness

3: Health Management

and Information Systems 3.21 1.89 Revised

4: Development of

Institutional and

Technological Capacity

for Preventing and

Performing during High-

intensity Weather Events

7.10 New

5: Project Management 0.20 New

Total: 17.85 18.45

Other Change(s)

Change in Implementation Schedule

Explanation:

The schedule was extended by two years to provide sufficient time to complete the implementation of

activities related to the acquisition and installation of complex weather radars and information systems for

strengthening disaster risk management, which were introduced under the first restructuring of 2011.

Other Change(s)

Explanation:

In addition to the proposed Level 1 restructuring triggering two additional safeguard policies, a Level 2

restructuring has been approved. The Level 2 restructuring includes: (i) the introduction of two new

components; (ii) the elimination of three subcomponents; (iii) the reallocation of loan proceeds across

components and expenditure categories; (iv) the reduction of the counterpart funding from US$ 9.16

million to US$ 0.047 million; (v) the change of the Environmental Assessment (OP/BP 4.01) safeguards

category from "C" to "B"; and (iv) the extension of the closing date resulting in a new date 24 month

beyond the original closing date and the corresponding adjustment of disbursement estimates.

Change(s) in Appraisal Summary

Appraisal Summary Change in Environmental Analysis

Explanation:

Two additional safeguard policies OP/BP4.04 (Natural Habitats) and OP/BP4.12 (Involuntary

Resettlement) are being triggered as a precaution. The restructured project is expected to have positive

indirect environmental impacts, given the focus on capacity building of key institutions of the Government

of Rio de Janeiro including the State Institute of the Environment, and may include policy changes or

actions for disaster risk management that entail temporary or permanent preventive evacuation. The

safeguard policy Environmental Assessment (OP/BP 4.01) was triggered in the original project.

The Environmental Management Framework and the Resettlement Policy Framework prepared by the

Borrower address subcomponents screening and eligibility and provide guidelines for the effective and

timely implementation of environmental and social safeguards during project implementation.

13

BRAZIL

RIO DE JANEIRO

RENOVATING AND STRENGTHNING PUBLIC MANAGEMENT

TECHNICAL ASSISTANCE PROJECT

A. SUMMARY

1. This restructuring paper seeks the approval of the Executive Directors to trigger

two additional safeguard policies, Operations Policy 4.04 Natural Habitats and Operational

Policy 4.12 Involuntary Resettlement. The project’s development objective is to assist the

Government of Rio de Janeiro (GORJ) to introduce performance-based management and

information technologies to drive service delivery improvements in key public services,

including secondary education and hospital care. This objective is being achieved through the

adoption of core government systems and management tools, strengthening of the education

management capacity, improvement of health management and information systems, and the

development of institutional and technological capacity for preventing and managing high-

intensity weather events. Operations Policy 4.04 (Natural Habitats) and Operational Policy 4.12

(Involuntary Resettlement) are to be triggered during the implementation of activities to

strengthen the Borrower’s disaster risk management practices. An Environmental Management

Framework and a Resettlement Policy Framework prepared by the Borrower to guide the

implementation of these activities are publicly available.

2. In addition to the proposed Level 1 restructuring, other changes have been

approved. The changes include the following: (i) the introduction of two new components; (ii)

the elimination of three subcomponents; (iii) the reallocation of loan proceeds across components

and expenditure categories; (iv) the reduction of the counterpart funding from US$ 9.16 million

to US$ 0.047 million; (v) the change of the safeguards category from “C” to “B”; and (iv) the

extension of the closing date resulting in a new date 24 month beyond the original closing date.

3. There are no changes to PDO, institutional arrangements or overall design.

B. PROJECT STATUS

4. The loan supporting the project, in the total amount of US$ 18.673 million, became

effective on December 27, 2010 and has to date disbursed US$7 million, or 38 percent of the

total amount. Project implementation progress is satisfactory. The GORJ expects to have

concluded all major tendering processes and several subcomponents by the end of December

2014. The GORJ estimates that an additional US$6 million would be disbursed by December of

2014. Overall, the implementing team is very committed and prepared to conduct the project’s

activities.

5. The original operation was restructured for the first time in February 2011 in

response to an emergency request of the GORJ to introduce activities to improve the State

14

capacity to reduce disaster risks. New intermediate outcome indicators to measure the effects

of the added activities were included at that time. These activities were designed in the wake of

unprecedented disasters caused by extreme weather events that exposed weaknesses in the

State’s disaster risk management system. With the restructuring, the State Environmental

Institute (INEA) was incorporated as an executing agency of activities aiming to strengthen

disaster risk management. However, adding the new activities reduced the resources available to

complete the original activities.

6. The preparation of an Additional Financing to the loan was initiated in 2011 to

supplement funding and incorporate the changes proposed in this restructuring, which was

later approved as a separate project. As part of the preparatory work, an Environmental and

Social Assessment of all activities of the original project was completed by the Borrower with

World Bank support. The Assessment identified the need to trigger additional safeguard

policies. At the request of the Borrower, the Additional Financing was approved as the stand-

alone project “Rio State Strengthening Public Management and Integrated Territorial

Development Technical Assistance Project” in February 12, 2014, making necessary the

proposed restructuring.

C. PROPOSED CHANGES

New Safeguard Policies

7. The restructured loan is expected to have positive indirect environmental impacts,

given the focus on capacity building of key institutions of the GORJ, including the State

Institute for the Environment. The fourth component “Development of Institutional and

Technological Capacity for Preventing and Performing during High-intensity Weather Events”,

which entails activities for the strengthening of disaster risk management systems and the

acquisition of two weather radar systems, may have indirect environmental implications.

8. Operational Policy 4.04 (Natural Habitats) and Operational Policy 4.12 (Involuntary

Resettlement) are being triggered in a precautionary mode. An Environmental Management

Framework (EMF) was prepared by the Borrower for the Additional Financing. The EMF

covers the activities for the strengthening disaster risk management in the State of Rio de Janeiro

included under Component 4. The EMF was disclosed on the Bank’s website on December 19,

2011 and disseminated in Brazil on December 20 of the same year. A Resettlement Policy

Framework (RFP) was also prepared by the GORJ to set the principles and guidelines to guide

the preparation of the Action Plan for implementing the new institutional arrangements for

disaster risk management. The RPF has been published and disseminated at the same time as the

EMF. The updated Integrated Safeguards Data Sheet (ISDS) of Setember 26, 2014 discusses the

specific risks involved and the features of the policy environment and capacity of the State to

implement the principles and guidelines. During the update of the ISDS, the Bank’s Safeguards

team verified that both the EMF and the RF continue to be suitable to guide the implementation

of activities related to disaster risk management.

9. The acquisition and installation of two meteorological radar systems involve

construction activities with limited adverse environmental impacts. For this reason, the

project Environmental Assessment has been re-categorized from “C” to "B". Operational

15

policy OP/BP 4.01 (Environmental Assessment) was triggered under the original project to

ensure that proper consideration is given to potential future negative environmental impacts, as

well as to positive ones that may result from the adoption of sustainable environmental practices.

In 2010, the Borrower prepared a document entitled "Protection and Guarantees" describing the

project components and the potential safeguards issues concerning hospital waste management.

The document summarizes the hospital facility accreditation methodology that was to be applied

under the project to ensure the proper management of medical wastes. The subcomponent

dealing with hospital accreditation and, consequently medical waste management, was removed

from the project under the present restructuring as it could not longer be financed with the

existing resources.

10. The project would not include conversion or degradation of critical natural habitats,

and no significant impact on natural habitats is expected. However, due to potential

unforeseen activities related to disaster risk management Operational Policy 4.04 is being

triggered on a precautionary basis.

11. The project would support technical assistance for improving disaster risk

management and the preparation of an Action Plan that may include policy changes or

actions entailing temporary or permanent preventive evacuation. Operational Policy 4.12

(Involuntary Resettlement) is being triggered as a precaution to ensure that actions for handling

adverse events through preventive evacuation would: (i) take into consideration the impacts on

the preventively relocated population, the population that may continue living at the sites

because the areas at high risk that cannot be controlled by any other measure but relocation is not

always entirely coterminous with the territory of the community settled at the sites, and the host

population in the neighboring communities of the new settlements; and, (ii) include proper

principles and guidelines to be applied by the state when any land acquisition might be

undertaken by the state for the preventive relocation of people living in risk-prone areas and/or

for the post-disaster relocation of affected people in places other than their original residences

and places of business; and, abandoned lands that reverted to the state for relocation were

encumbered by pre-existing use claims.

12. The safeguard performance continues to be satisfactory and the Government of Rio

de Janeiro is making significant progress to improve its environmental management system

and involuntary resettlement framework. The EMF proposes institutional capacity

strengthening measures and institutional arrangements to implement the project. The State

Environmental Institute (INEA), the agency that is responsible for implementing Component 4,

is fully capable of overseeing the implementation of the Framework and the resulting plans.

Additionally, the State of Rio de Janeiro has notably improved its involuntary resettlement

legislation and procedures in recent years and is capable of handling, monitoring, and

implementing the social safeguards issues that may indirectly arise from future projects and

programs or changes in the disaster risk management policy that involve resettlement of

populations living in areas at high risk of landslides. Decree 41,148/2008 introduces as its main

guidelines: (i) the participation of the affected people in all states of the resettlement process; (ii)

the provision of resettlement options; (iii) strong mechanisms of communication with the

population; and (iv) extensive consultation processes with the main stakeholders to address their

concerns during implementation. These procedures are in accordance with most of the provisions

of the Bank's resettlement policies.

16

Other Changes

13. Project Components. The Level 2 restructuring is introducing two new components

to better organize the activities and objectives added in the first restructuring. The new

components are (i) Component 4, titled “Development of Institutional and Technological

Capacity for Preventing and Performing during High-intensity Weather Events”, with US$7.1

million (the activities corresponding to disaster risk management currently under component 1

will be transferred to this component), and (ii) Component 5, titled “Project Management”, with

US$ 0.2 million.

14. Component 4 would support the strengthening of the State’s early warning system

for high rainfall events. Activities supported by this component include: (i) the development

and implementation of institutional and operational arrangement for the disaster risk

management; and (ii) the acquisition and installation of two weather radar systems. The

activities aim at substantially improving the State’s early warning system through the use of

meteorological radars. This hardware will enable the tracking of storms (volume, speed, and

direction) before it reaches areas at risk. While the ultimate goal is to prevent and reduce the

negative effects of disasters, these activities would also help improve critical service delivery and

strengthen public financial management by reducing the fiscal impact generated by extreme

weather events. This component would support the GORJ in developing core management

system for disaster risk management as well as to acquire needed equipment. The component

would be the responsibility of the State Institute for the Environment (INEA).

15. Component 5 would support activities related to the overall management of the

project. Under the original project, US$0.6 million had been allocated to the expenditure

category “operating costs” in order to cover project management activities associated with

strengthening of the GORJ’s capacity for project execution, monitoring, supervision and evaluation.

However, some of the required expenditures are related to goods, consultant services, non-

consultant services, and training. Therefore, a new component is being created and part of the

operating costs reallocated to adequately reflect the project’s management needs. The component

would be the responsibility of the State Secretariat of Planning and Management (SEPLAG) and

managed by its Project Management Unit.

16. Three subcomponents are being eliminated as they could no longer be financed with

the loan proceeds. These include subcomponents 2.2 “School-level Performance Targets for

State Schools”, 3.1. “Accreditation System for UPAs”, and 3.2 “Cost-Accounting Methodology

for UPAs and Hospitals”.

17. The Results Framework is being revised to reflect the changes in subcomponents. In

addition, several indicators are being changed to improve the capacity of the Government and the

World Bank to assess the impact of the project activities, refining methodologies used to

calculate them and clarifying responsibilities for data collection and reporting.

18. Changes to Allocation of Resources. Table 1 below presents the proposed

reallocation. Component 1 has a total allocation of US$11.87 million to support core

government systems and management tools for improved performance and efficiency, including

disaster risk management. With the restructuring, activities to strengthen the performance of the

disaster risk prevention system would be included in Component 4, leading to a reallocation of

17

US$7.1 million from components 1, 2, and 3 to component 4. The funding of Component 2 and 3

would be reduced to US$ 2.39 and 1.89 million respectively, as activities have been eliminated

in each of them. Component 5 would receive US$ 0.2 million dedicated to project management.

Table 1: Proposed Reallocation Project Financial Resources to Components

Original 1st

Restructuring

2nd

Restructuring

Components US$ millions

1. Core Government Systems and Management Tools for

Improved Performance and Efficiency

7.65 11.87 6.87

1.1. Design and Application of Public Sector Performance

Indicators

1.72 5.94 0.5

1.2. Real Estate Property Management 3.50 3.5 3.0

1.3. Management Information System for Social Security 2.43 2.43 3.37

2. Strengthening Education Management Capacity, Teacher

Instructional Practice and Incentives for School

Effectiveness

4.83 2.77 2.39

2.1. Building SEEDUC’s Technical Capacity for Conducting

Studies and Evidence-based Policy

2.35 1.0 0.5

2.2. School-level Performance Targets for State Schools

[ELIMINATED]

0.98 0.27 0

2.3. Developing the Escola SEEDUC as an Effective In-Service

Teacher Training Network 1.51

1.5

1.89

3. Health Management and Information Systems 5.37 3.21 1.89

3.1. Accreditation System for UPAs [ELIMINATED] 1.50 0.75 0

3.2. Cost-Accounting Methodology for UPAs and Hospitals

[ELIMINATED]

2.02 0.5 0

3.3. Information System for Health Supply Chain Management 1.81 0.87 1.09

3.4. Evaluation of Financial Incentive Programs for

Municipalities

0.05 1.09 0.8

4. Development of Institutional and Technological Capacity

for Preventing and Performing during High-intensity

Weather Events [NEW]

7.10

4.1. Design of the Institutional and Operational Arrangement for

the Disaster Risk Management

0.5

4.2. Acquisition and Installation of Two Weather Radar Systems 6.6

5. Project Management [NEW] 0.20

Price Contingencies 0.440 0.176 0.176

Front-end Fee 0.047 0.047 0.047

19. Counterpart Funding and Expenditure Categories. Counterpart funding is reduced

from US$ 9.16 to US$ 0.047 million and resources in the amount of US$ 0.55 million from

the expense category 2 “Operating Costs” to category 1. The activities co-financed with

counterpart resources have been removed from the Project and/or transferred to the second

Technical Assistance Project. The Borrower’s contributions are being accordingly updated. The

Table of Category of Expenses of the Loan Agreement of Item 2 of Section IV of the Loan

Agreement are being modified in the following way:

18

Table 2: Category of Expenses

20. No changes will be made to procurement methods and arrangements.

D. APPRAISAL SUMMARY

21. Two additional safeguard policies OP/BP4.04 (Natural Habitats) and OP/BP4.12

(Involuntary Resettlement) are being triggered as a precaution. The restructured project is

expected to have positive indirect environmental impacts, given the focus on capacity building of

key institutions of the Government of Rio de Janeiro including the State Institute of the

Environment, and may include policy changes or actions for disaster risk management that entail

temporary or permanent preventive evacuation. The safeguard policy Environmental

Assessment (OP/BP 4.01) had been triggered in the original project. An Environmental

Management Framework and a Resettlement Policy Framework prepared by the Borrower to

guide the implementation of thease activities are publicly available.

Original 2nd Restructuring

Category Amount of the Loan

Allocated (expressed in

USD)

Amount of the Loan

Allocated (expressed in

USD)

Percentage of

Expenditures to be

financed (inclusive of

Taxes)

(1) Goods, Non-

consultant services,

consultant services and

training under the

Project

18,026,317.50 18,576,317.50 100%

(2) Operating Costs 600,000.00 50.000,00 100%

(3) Front-end Fee 46,682.50 46.682,50 Amount payable

pursuant to Section 2.03

of this Agreement in

accordance with Section

2.07 (b) of the General

Conditions

TOTAL AMOUNT 18,673,000.00 18.673.000,00

19

ANNEX 1: RESULTS FRAMEWORK

Project

Name:

Brazil - Rio de Janeiro Renovating and Strengthening Public

Management (P106768)

Project

Stage: Restructuring Status: DRAFT

Team

Leader: Lorena Vinuela

Requesting

Unit: LCC5C Created by: Lorena Vinuela on 08-Mar-2014

Product

Line: IBRD/IDA

Responsible

Unit: GGODR Modified by: Lorena Vinuela on 03-Oct-2014

Country: Brazil Approval FY: 2011

Region: LATIN AMERICA AND

CARIBBEAN

Lending

Instrument: Technical Assistance Project

Project Development Objectives

Original Project Development Objective:

The objective of the project is to support the State of Rio de Janeiro to introduce performance-based management and information technologies to

drive service delivery improvements in key public services, including secondary education and hospital care.

Results

Core sector indicators are considered: Yes Results reporting level: Project Level

Project Development Objective Indicators

Status Indicator Name Core Unit of Measure Baseline Actual(Current) End Target

Marked for

Deletion

Waiting time for UPA patients

with #green# risk classification

Minutes Value 59.60 58.00 52.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

20

(TERV) Comment The

subcomponent

related to this

indicator has been

removed from the

project.

Marked for

Deletion

UPAs with at least ten

#treatment protocols# adopted

by health care staff

Number Value 2.00 5.00 9.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment Activities have

been completed

without project

financing.

Marked for

Deletion

Secondary school completion

rate

Percentage Value 33.40 45.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The methodology

for calculating

this rate has been

changed and it is

no longer possible

to measure

progress towards

the agreed end

target.

t-2 = 2012

Marked for

Deletion

RioPrevidência permanent

personnel

Number Value 350.00 262.00 300.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The reduction of

personnel is not

21

considered to be

an adequate

indicator of

increased

efficiency.

Marked for

Deletion

Real Property Management

Days Value 65.00 22.00 15.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment This indicator is

no longer being

measured as a

result of

difficulties in

reproducing the

methodology

used to end

target.

New Number of the State's

properties regularized

Number Value 363.00 1988.00 2468.00

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

Revised Average number of days to

record debt owned by renters of

State Government's properties

Days Value 120.00 120.00 30.00

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

Rio Previdência performance

Days Value 110.00 110.00 30.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment Average

elapsed time

As part of the

restructuring, the

22

(days) to

collect Social

Security

employment

tax debts

measurement of

this indicator

changes from

"days" to "hours".

Revised Points improved in the Index of

Educational Development of

Rio de Janeiro (IDERJ) by the

30 percent worst performing

schools


Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

Schools with #critical# IDEB

scores (2.9 or lower)

Number Value 372.00 200.00

Date 30-Jun-2009 05-Feb-2014 30-Jun-2015

Comment (2009 IDEB

results)

As part of the

restructuring, this

indicator will be

revised to:

Increase in the

Rio de Janeiro

Education

Development

Index (IDERJ) for

secondary school.

The IDEB and

IDERJ use the

same

methodology, but

the IDERJ is

more useful for

program

monitoring

(2013 IDEB

results)

23

because it is

measured

annually, rather

than every 2

years.

New Percentage of false alerts

issued by the Flooding

Warning System

Percentage Value 56.00 56.00 44.00

Date 31-Dec-2013 31-Dec-2013 30-Jun-2017

Comment

Intermediate Results Indicators

Status Indicator Name Core Unit of Measure Baseline Actual(Current) End Target

Marked for

Deletion

Casa do Educador - teachers

who have taken a face-to-face

course


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The design of the

program is being

changed

substantially

based on a

consultancy

financed by the

project and the

indicator is no

longer considered

adequate to

measure progress

towards the

activity'

objectives.

24

Marked for

Deletion

Casa do Educador - teachers

who have taken an online

training course


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015


program is being

changed

substantially

based on a

consultancy

financed by the

project and the

indicator is no

longer considered

adequate to

measure progress

towards the

activity'

objectives.

Marked for

Deletion

Casa do Educador - Website

Downloads


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015


program is being

changed

substantially

based on a

consultancy

financed by the

project and the

indicator is no

longer considered

25

adequate to

measure progress

towards the

activity'

objectives.

Marked for

Deletion

UPAs reporting the full cost of

medical services by Cost

Center in their Annual Reports


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The

subcomponent

affecting this

indicator has been

removed from the

component.

Marked for

Deletion

UPAs with completed quality

accreditation by Joint

Commission Methodology


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The

subcomponent

affecting this

indicator has been

removed from the

component.

Marked for

Deletion

Schools with annual targets for

learning improvement

established with SEEDUC

Number Value 0.00 1255.00 400.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The target has

been achieved in

2011 with the

passing of a

26

decree that made

mandatory for all

schools to have

annual targets.

The number of

1255 schools is

equivalent to 96%

of all state

schools. The total

number of state

school has

decreased.

Marked for

Deletion

Schools that use SAERJ results

in school-level Improvement

Plans


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment The Government

of Rio de Janeiro

has already

implemented this

system for all

state schools.

Marked for

Deletion

Real Property Management:

Responses provided to requests

from public bodies concerning

the availability for use of a

GORJ property


Date 30-Jun-2009 31-Dec-2013 30-Jun-2015

Comment A new indicator

has been added to

replace this and

better reflect the

content of

activities aiming

to improve asset

27

management.

Revised Number of State secretariats

and agencies with signed

results agreements

Number Value 0.00 4.00 12.00

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

#Results Agreements# between

the Governor and line

departments or other

government bodies


Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment When

restructured, the

indicator will be

changed to

"Contracting

Instruments

(Result

Agreements)

signed".

Revised Average number of working

days to issue a reminder letter

Days Value 90.00 5.00

Date 30-Jun-2014 31-Dec-2013 30-Jun-2017

Comment

RioPrevidência: Average

number of working days to

prepare and deliver information

demanded for judicial

proceedings

Days Value 20.00 20.00 10.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment This indicator

will be revised

and the

measurement

changed from

"working days" to

28

"hours" to be

aligned with the

Public

Administration

core indicator.

Marked for

Deletion

RioPrevidência: Average

elapsed time (working days) to

transfer financial data to

SIAFEM-RJ (Sistema

Integrado de

AdministraçãoFinanceira para

Estados e Municípios)

Days Value 15.00 15.00 7.00

Date 30-Jun-2009 25-Feb-2014 30-Jun-2015

Comment

New Number of public real estate

properties with records

registered and updated in

System of Real Estate

Management

Number Value 728.00 2176.00 3376.00

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

New Percentage of public servants

in strategic positions of the

Secretariat of Education

receiving performance

evaluation

Percentage Value 0.00 0.00 100.00

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

New Number of hospitals

participating of the Program

PAH I with targets monitored

through the System of

Management of Indicators and

Targets

Number Value 0.00 0.00 70.00

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

New Percentage of unmet requests


29

of the State Hospital "Rocha

Faria" for hospital supplies

Date 30-Jun-2009 31-Dec-2013 30-Jun-2017

Comment

30

ANNEX 2: OPERATIONAL RISK ASSESSMENT FRAMEWORK (ORAF)

Brazil: Brazil - Rio de Janeiro Renovating and Strengthening Public Management (P106768)

Project Stakeholder Risks

Stakeholder Risk Rating Low

Risk Description: Risk Management:

If the State Government changes as a result of the October

2014 elections, there is a risk of a loss of political support

for the project from State government. Given the

advanced stage in the implementation of all activities this

risk is low.

State Secretariats of Planning (SEPLAG) is strongly committed to managing and

implementing project activities and has created a technically strong Project Management

Unit, staffed with civil servants. The Unit has been active in defining project activities

and indicators and is committed to reporting on progress throughout the project life. The

Operation Manual clearly defines the roles and responsibilities of each implementing

agency and those of the Coordination Committee. The Vice-Governor’s Office oversees

the Coordination Committee responsible for coordinating deliverables and

communication with the Bank. The World Bank is working closely with the

implementing agencies to ensure that the necessary transition arrangements are made to

warrant the continuity of the ongoing activities.

Resp: Status: Stage: Recurrent: Due Date: Frequency:

Both In Progress Implementation

Implementing Agency (IA) Risks (including Fiduciary Risks)

Capacity Rating Low

Risk Description:

State governments generally have a poor track record in

Risk Management:

The project has been under implementation since its approval in 2010 and during this

period the Project Management Unit of the Secretariat of Planning and Management has

31

implementing information technology (IT) systems and

face difficulties in handling managerial and procedural

requirements for project implementation, including

fiduciary and safeguard issues.

acquired considerable experience working with World Bank procurement procedures.

The unit also supports the other agencies participating in the project, which have a more

uneven capacity in Bank procedures and safeguards.

The Secretariat of Finance has an adequate capacity to generate and report on financial

data.

The Bank team is working closely to build capacity in safeguards. Both the UGP staff

and secretariat staff has been offered safeguards training.



Risk Management:

The Bank team is working closely to build capacity in the implementing agencies. Both

the UGP staff and secretariat staff has been offered procurement, fiduciary, and

safeguards training. Additional training and support will be provided as needed.


Bank In Progress Implementation

Governance Rating Low


Complex institutional framework of the State Government

and competing priorities would create a challenge to

project oversight.

However, SEPLAG has been successful in overseeing the

original project. The Vice- Governor’s office supports

project coordination and oversight.

The Coordination Committee ensures communication and support to the four

implementing Secretariats at all stages to ensure the successful design of appropriate

governance measures. All implementing Secretariats have agreed to the terms set out in

the Operational Manual detailing governance mechanisms.



Risk Management:

The Operational Manual contains an adequate procurement section and guidelines. Line

secretariats responsible for procurement have adequate capacity to do so and guidance is

32

given to Implementing Agency on how to verify compliance with procurement rules and

identify red flags. The Implementing Agency provides assurances that supervision is

undertaken, serious issues are being flagged, and action taken. Bidders and other local

stakeholders have access to information on procurement procedures, sanctionable

practices, and on the sanctions imposed by the World Bank and the government.



Risk Management:

Forms of direct social control over government, through participatory budgeting and

transparent expenditure notices, are evolving in Brazil.


Client In Progress Implementation

Project Risks

Design Rating Low


The multi-sectoral nature of the project makes clear

implementation, coordination and oversight rules crucial.

There is a risk of fragmentation in project implementation.

Regular quarterly meetings are held to facilitate coordination and identify potential

problems at an early stage between SEPLAG and the line secretariats. There are clear

implementation and coordination mechanisms in place that are being used regularly.



Social and Environmental Rating Moderate


The activities are not expected to generate negative

environmental or social impacts. Nonetheless, Component

4 that entails the acquisition of two weather radar systems

The State has engaged in strong dialog and consultation with civil society and groups of

citizens affected by the landslides of 2011 during the preparation of the second

Technical Assistance Project preparation, originally designed to be an Additional

33

will be closely monitored to ensure that they do not lead to

future safeguard risks. The Natural Habitats (OP/BP 4.01)

and Involuntary Resettlement (OP/BP 4.12) have been

triggered in a precautionary mode and the project has been

re-categorized from “C” to “B”.

Financing to this project. This has been continued through implementation and will

continue to be given high priority in the remaining time. The activities of Component 4

will be closely monitored to ensure that they do not have adverse impacts. An

Environmental Management Framework and a Resettlement Policy Framework have

been developed for the project by the Government of Rio de Janeiro with support from

Bank specialists. These documents are available to the public at:

http://www.progestao.rj.gov.br/salvaguardas_ambientais.asp



Program and Donor Rating Low


The project is part of a broader Government and Bank

engagement on integrated territorial development in Rio

de Janeiro, including three DPLs already closed and a

fourth in implementation. Several Trust Funds have also

supported advisory services to the state including PSIA,

SFLAC funds and Korean Trust Fund. Low risk of

withdrawal of funds or programmatic support for the

project.


Delivery Monitoring and Sustainability Rating Low


The number of components and Secretariats involved in

this project increase the risk that monitoring progress and

delivery of information will be delayed. Some

implementing Secretariats may find it difficult to oversee

the implementation of activities outside of their own

Secretariat.

Mechanisms and responsibilities for delivering project documents and information are

clearly defined in the Operational Manual, which describes in detail all implementing

Secretariats responsibilities and the rules guiding the Coordination Committee. An

emphasis has been placed on state-wide integrated planning and the implementation of

information management systems in the project activities, encouraging the availability

and integration of databases across sectors. This will increase the sustainability of

improvements made in public management and disaster risk management.

34



Other (Optional) Rating



Other (Optional) Rating



Overall Risk

Overall Implementation Risk: Rating Low

Risk Description:

Given the advanced state of implementation of all components and subcomponents the overall risk is low. The World Bank team will continue to

work closely with the Project Management Unit and provide implementation support as needed.

Document of The World Bank · SAERJ Rio de Janeiro State Education Evaluation System (Sistema de...

Documents

Transcript of Document of The World Bank · SAERJ Rio de Janeiro State Education Evaluation System (Sistema de...