Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: PAD874

INTERNATIONAL DEVELOPMENT ASSOCIATION

PROJECT APPRAISAL DOCUMENT

ON A

PROPOSED CREDIT

IN THE AMOUNT OF SDR 19.5 MILLION (US$30.0 MILLION EQUIVALENT)

TO

MONGOLIA

FOR AN

EDUCATION QUALITY REFORM PROJECT

May 5, 2014

Education Sector Unit Human Development Sector Department East Asia and Pacific Region

This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS (Exchange Rate Effective March 31,, 2014)

Currency Unit = Mongolian Tugrik (MNT) MNT1,789 = US$ 1 US$1.5456 = SDR 1

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

BD Bidding Document IPPF Indigenous Peoples Planning Framework

CBA Cost Benefit Analysis IRR Internal Rate of Return

CPS Country Partnership Strategy ISP Implementation Support Plan

CQS Consultants Qualification Selection M&E Monitoring & Evaluation

DA Designated Account MES Ministry of Education and Science

DLI Disbursement-Linked Indicator MNT Mongolian Tugrik

EA Environmental Assessment MOF Ministry of Finance

EDCM Education Donors' Consultative Mechanism NCB National Competitive Bidding

EEC Education Evaluation Center NPV Net Present Value

EE Eligible Expenditure ORAF Operational Risk Assessment Framework

EFA-FTI Education for All Fast Track Initiative PAD Project Appraisal Document

EMIS Education Management Information System PDO Project Development Objective

FA Financing Agreement POM Project Operation Manual

FID Finance and Investment Department QBS Quality-Based Selection

FM Financial Management QCBS Quality- and Cost-Based Selection

GoM Government of Mongolia READ Rural Education and Development Project

GPA Government Procurement Agency SBD Standard Bidding Document

GPE Global Partnership for Education SC Steering Committee

IBRD International Bank for Reconstruction and Development

SOE Statement of Expenditures

ICB International Competitive Bidding TA Technical Assistance

IDA International Development Assessment TOR Terms of Reference

IFR Interim Financial Report TTL Task Team Leader

IP Indigenous Peoples UB Ulaanbaatar

IPF Investment Project Financing UNESCO United Nations Educational, Scientific and Cultural Organization

WB World Bank

Regional Vice President: Axel van Trotsenburg, EAPVP Country Director: Klaus Rohland, EACCF

Sector Director: Xiaoqing Yu, EASHD Sector Manager: Luis Benveniste, EASHE

Task Team Leader: Prateek Tandon, EASHE

MONGOLIA Education Quality Reform Project

TABLE OF CONTENTS

Page

I.  STRATEGIC CONTEXT .................................................................................................1 

A.  Country Context ............................................................................................................ 1 

B.  Sectoral and Institutional Context ................................................................................. 1 

C.  Higher Level Objectives to which the Project Contributes .......................................... 3 

II.  PROJECT DEVELOPMENT OBJECTIVES (PDO) ....................................................4 

A.  PDO............................................................................................................................... 4 

B.  Project Beneficiaries ..................................................................................................... 5 

C.  PDO Level Results Indicators ....................................................................................... 5 

III.  PROJECT DESCRIPTION ..............................................................................................5 

A.  Project Components ...................................................................................................... 5 

B.  Project Financing .......................................................................................................... 8 

C.  Lessons Learned and Reflected in the Project Design .................................................. 9 

IV.  IMPLEMENTATION .....................................................................................................10 

A.  Institutional and Implementation Arrangements ........................................................ 10 

B.  Results Monitoring and Evaluation ............................................................................ 11 

C.  Sustainability............................................................................................................... 11 

V.  KEY RISKS AND MITIGATION MEASURES ..........................................................12 

A.  Risk Ratings Summary Table ..................................................................................... 12 

B.  Overall Risk Rating Explanation ................................................................................ 12 

VI.  APPRAISAL SUMMARY ..............................................................................................13 

A.  Economic and Financial Analysis ............................................................................... 13 

B.  Technical ..................................................................................................................... 14 

C.  Financial Management ................................................................................................ 14 

D.  Procurement ................................................................................................................ 15 

E.  Social (including Safeguards) ..................................................................................... 15 

F.  Environment (including Safeguards) .......................................................................... 16 

Annex 1: Results Framework and Monitoring .........................................................................17 

Annex 2: Detailed Project Description .......................................................................................20 

Annex 3: Implementation Arrangements ..................................................................................30 

Annex 4: Operational Risk Assessment Framework (ORAF) .................................................44 

Annex 5: Implementation Support Plan ....................................................................................48 

Annex 6: Disbursement Linked Indicators and Verification Protocol ...................................51 

i

.

PAD DATA SHEET

Mongolia

MN Education Quality Reform Project (P148110)

PROJECT APPRAISAL DOCUMENT.

EAST ASIA AND PACIFIC

EASHE

Report No.: PAD874.

Basic Information

Project ID EA Category Team Leader

P148110 B - Partial Assessment Prateek Tandon

Lending Instrument Fragile and/or Capacity Constraints [ ]

Investment Project Financing Financial Intermediaries [ ]

Series of Projects [ ]

Project Implementation Start Date Project Implementation End Date

01-Jul-2014 30-Jun-2019

Expected Effectiveness Date Expected Closing Date

01-Jul-2014 31-Dec-2019

Joint IFC

No

Sector Manager Sector Director Country Director Regional Vice President

Luis Benveniste Xiaoqing Yu Klaus Rohland Axel van Trotsenburg .

Borrower: Government of Mongolia

Responsible Agency: Ministry of Education and Science

Contact: Ts. Bayarkhuu Title: Head, Finance and Investment Division

Telephone No.: 976-11-265696 Email: bayarkhuu@mecs.gov.mn .

Project Financing Data(in USD Million)

[ ] Loan [ ] Grant [ ] Guarantee

[ X ] Credit [ ] IDA Grant [ ] Other

Total Project Cost: 30.00 Total Bank Financing: 30.00

Financing Gap: 0.00 .

Financing Source Amount

ii

BORROWER/RECIPIENT 0.00

International Development Association (IDA) 30.00

Total 30.00.

Expected Disbursements (in USD Million)

Fiscal Year

2015 2016 2017 2018 2019 2020 0000 0000 0000

Annual 2.50 7.60 9.00 7.00 3.90 0.00 0.00 0.00 0.00

Cumulative

2.50 10.10 19.10 26.10 30.00 30.00 0.00 0.00 0.00

.

Proposed Development Objective(s)

The Project Development Objectives are to improve the quality of education for primary school children in Mongolia, with particular emphasis on improving native language and mathematical skills, and strengthen school-level planning. .

Components

Component Name Cost (USD Millions)

Improving learning outcomes 14.90

Pre- and in-service professional development of teachers 3.30

Implementation of a school support program 6.10

System management, monitoring and evaluation 5.70.

Institutional Data

Sector Board

Education .

Sectors / Climate Change

Sector (Maximum 5 and total % must equal 100)

Major Sector Sector % Adaptation Co-benefits %

Mitigation Co-benefits %

Education Primary education 100

Total 100

I certify that there is no Adaptation and Mitigation Climate Change Co-benefits information applicable to this project. .

Themes

Theme (Maximum 5 and total % must equal 100)

Major theme Theme %

Human development Education for all 100

iii

Total 100 .

Compliance

Policy

Does the project depart from the CAS in content or in other significant respects?

Yes [ ] No [ X ]

.

Does the project require any waivers of Bank policies? Yes [ ] No [ X ]

Have these been approved by Bank management? Yes [ ] No [ X ]

Is approval for any policy waiver sought from the Board? Yes [ ] No [ X ]

Does the project meet the Regional criteria for readiness for implementation? Yes [ X ] No [ ] .

Safeguard Policies Triggered by the Project Yes No

Environmental Assessment OP/BP 4.01 X

Natural Habitats OP/BP 4.04 X

Forests OP/BP 4.36 X

Pest Management OP 4.09 X

Physical Cultural Resources OP/BP 4.11 X

Indigenous Peoples OP/BP 4.10 X

Involuntary Resettlement OP/BP 4.12 X

Safety of Dams OP/BP 4.37 X

Projects on International Waterways OP/BP 7.50 X

Projects in Disputed Areas OP/BP 7.60 X .

Legal Covenants

Name Recurrent Due Date Frequency

Verification of Achievement of DLI Target for the DLI Period

Description of Covenant

No withdrawal shall be made in respect of DLI Values allocated to DLIs for the DLI Period until and unless the Recipient has furnished evidence satisfactory to the Association that the DLI Target for the respective DLI Period has been fully achieved or is in process of being achieved. .

Conditions

Source Of Fund Name Type

IDA Engagement of Verification Consultant Disbursement

Description of Condition

No withdrawal shall be made for DLI Expenditures until and unless the Recipient has engaged a verification consultant with terms of reference and qualifications satisfactory to the Association.

iv

Team Composition

Bank Staff

Name Title Specialization Unit

Carol Chen Ball Consultant Preparation & implementation support

EASHD

Minneh Mary Kane Lead Counsel Lead Counsel LEGES

Juan Prawda Consultant Preparation Support SASED

Feng Ji Senior Environmental Specialist

Environmental Safeguards

EASCS

Anna Coronado Program Assistant Administrative and Client Support

EASHD

Jun Zeng Social Development Specialist

Social Safeguards EASCS

Jianjun Guo Senior Procurement Specialist

Procurement EASR2

Prateek Tandon Senior Economist Team Lead EASHE

Takiko Koyama Jr Professional Officer Education EASHE

Dulguun Byambatsoo Consultant Financial Management EASFM

Rabia Ali Young Professional Education EASHE

Ulziimaa Erdene E T Temporary Administrative and Client Support

EACMF

Pagma Genden E T Consultant Education EASHH

Non Bank Staff

Name Title Office Phone City

Atsuko Muroga Operations Officer New York

Robert Prouty Washington .

Locations

Country First Administrative Division

Location Planned Actual Comments

Mongolia Ulaanbaatar Ulaanbaatar Hot X

1

I. STRATEGIC CONTEXT

A. Country Context

1. Mongolia is the least densely populated country in the world. It is the second largest landlocked country in the world and has a large nomadic population. The number of urban inhabitants has grown rapidly in recent years; almost half the population now lives in the capital city, Ulaanbaatar (UB). This has resulted in an education system that is characterized by a large number of medium- or small-sized schools that are geographically isolated. Unit costs increase significantly as the distance from UB increases. 2. Mongolia is divided administratively into UB and 21 aimags (provinces). UB is divided into districts and khoroos (city wards). The aimags are divided into districts known as soums, and soums are divided into baghs (villages). The rural areas continue to be largely agricultural, though the recent development of extensive mineral deposits has increased industrial production in these areas. The incidence of poverty remains high. Urban poverty is also growing, and both rural and urban areas have significant service delivery challenges, although these are very different in nature. 3. While the first decade following Mongolia’s transition in 1990 to a market-based economy saw sluggish growth, the past decade has seen a dramatic turnaround, driven largely by the burgeoning mineral sector. Mongolia’s Gross Domestic Product has grown at an annual rate of eight percent for ten years, and is projected to experience double digit growth for the next several years. However, this period of growth has also been characterized by a rise in perceptions of income inequality and persistent labor market stagnation. 4. Mongolia has a young population; about 60 percent are under thirty years of age. Fertility rates have fallen sharply—more so than in any other country in the world—resulting in a declining number of school age children over the past decade, this trend is reversing, and the number of school-aged children is expected to grow over the next decade. Mongolian is spoken by 95 percent of the population; a variety of dialects are spoken across the country. In the west of the country, Kazakh and Tuvan are also spoken. B. Sectoral and Institutional Context

5. The education system in Mongolia has been experiencing a period of reform over the past decade. The entrance age into primary school was lowered during this period from eight years of age to six. The length of the basic education cycle was increased from ten to twelve years. The education system now consists of five years of primary school, four years of lower secondary school, and three years of upper secondary school. Upper secondary school is free but not compulsory. Many schools also offer pre-school programs of two or three years in length. The first cohort of students to have completed the full twelve-year cycle will graduate in 2020. 6. Mongolia has done well in extending access to pre-primary and basic education since its transition. Basic education enrollment has recovered with the economy. The 1990s saw a drastic decline in school enrollment ratios; the 2000s witnessed its recovery to the pre-

2

transition level in pre-primary through lower secondary education; the current decade will likely see an extension of access in pre-primary and post-primary education. The national net enrollment rates of 90 percent and 82 percent for primary and secondary education, respectively, compare favorably with middle income countries around the world. Figure 1 displays the trends in enrollment in various subsectors in education.

Figure 1: Changes in Gross Enrollment Ratios between 1991 and 2011

Source: UNESCO Institute of Statistics

7. The issue of gender in Mongolian education is in many ways the reverse of what is seen in most countries at a similar stage of economic development. Both genders show roughly 100 percent completion rates for primary education, but girls achieve slightly more secondary schooling and much more tertiary education than boys. Momentum over the past decade has swung in favor of boys. Boys’ primary school enrollment rates are now slightly ahead of girls, and the difference in secondary enrollment rates has declined from over 10 percent to 1 percent in less than a decade. Girls still represent 60 percent of tertiary enrollments, although here too the differences are declining rapidly. Recent achievement tests show little difference in learning outcomes for boys and girls, although for both genders, learning levels are worrisomely low. The area of clearest gender bias in the education sector is that of employment: men represent only 4 percent of the primary school teaching force but 45 percent of school-level management (school directors and teaching managers).

8. Having successfully extended access to basic education, the Government has now placed a much sharper focus on improving the quality of primary and secondary education. With nearly 30 percent of the population under the age of 14 and 12 percent under the age of 51, and in light of the rapid changes and emerging issues the country is facing – managing its mineral wealth, transitioning to middle-income status, handling increasing rural to urban migration, and responding to the structural changes in its labor market – Ministry of Education and Science (MES) has stated its intention that Mongolia’s education system be a provider of high quality skills and a key driver of growth. As such, Mongolia’s education system

1 2011 Mongolia Population Census

3

will need to deliver not just a greater quantity of skills (the share of the population with a certain level of education and training), but also greater quality skills (better cognitive, non-cognitive, and technical skills that will help its citizens deal with complex and unexpected tasks and to continue to learn throughout their lives). This imperative implies a sharpened focus on increasing the quality of education, starting from the earliest ages. Key Issues Facing the Education Sector 9. Several challenges must be addressed if the Government is to deliver on its promise of higher quality education. A brief summary is given below:

The lack of learning materials in urban areas has inhibited the student experience. Available information on student learning outcomes shows alarmingly low

performance, and disparities exist between Ulaanbaatar, aimag centers and soums. The lack of a systematic and institutionalized comparable assessment of student

performance, starting from the earliest years is a large obstacle in improving the quality of basic education.

The quality of teaching in primary and secondary schools in Mongolia is an area of prime concern.

Preliminary findings from a Bank-commissioned study on how teachers are groomed, retained, and maintained reveals a teacher training system that is not very dynamic, and does not appear to be reaching a high level of quality in terms of preparing teachers for future service.

Finally, instructional time in Mongolian schools is relatively low. 10. To address these issues, MES has recently introduced a rigorous reform agenda to improve the quality of basic education. This agenda – “The Educational Quality Reform Program” – centers on three pillars. The first pillar is called “The Book Project” (the scale-up of the IDA-financed Rural Education and Development (READ) project). The second pillar is a Teacher Development and Upgrading Program. The third pillar is a so-called “Talent Program,” which is a school support program to increase discretionary spending at the school level to provide further educational enrichment opportunities and increase educational quality. C. Higher Level Objectives to which the Project Contributes

11. Mongolia adopted a Comprehensive National Development Strategy in 2008. This document established a first-phase target of achieving the Millennium Development Goals by 2015, and a second-phase target of transitioning to a knowledge-based economy by 2021. The World Bank has developed a Country Partnership Strategy (CPS) for Mongolia (FY 13-17), discussed by the Board on May 17, 2012, that is aligned with the Comprehensive National Development Strategy. 12. The World Bank’s CPS for Mongolia identifies three areas which the Bank will support between FY13 and FY17: (i) enhancing Mongolia’s capacity to manage its mining economy, (ii) building a diversified basis for economic growth and employment; and (iii) addressing vulnerabilities through improved access to services, safety net provision, and disaster risk

4

management. Better delivery of basic education services is identified as one of the outcomes of the third area of intervention. The CPS notes that improved education services are essential to the country reaching its income targets. It calls for both growth- and equity-enhancing investments, with policy and institutional frameworks and delivery systems geared to efficient use of incremental resources. It further notes significant disparities in educational access by location, and the need to improve learning outcomes. 13. The project is consistent with both the Comprehensive National Development Strategy and the CPS. It aims to improve the delivery of education services with a focus on both quality and equity. Its strategy of scaling up evidence-based approaches to improve learning outcomes, while building system capacity to provide classroom-level support, is well aligned with the CPS call for policy and institutional frameworks and delivery systems to promote the more efficient use of new resources. 14. The interventions are also aligned with the Bank’s twin goals of eradicating extreme poverty and increasing shared prosperity. The literature on human capital has documented the positive correlation between human capital investment and social and economic development. Investments in education significantly contribute to poverty eradication and wealth creation, as higher educational attainment significantly increases the probability of getting a paid job and has a large and positive impact on monthly wage earnings. This well-known contribution laid the foundation for one of the landmark pledges at the 2000 UNESCO-sponsored World Education Forum in Dakar that provided a sustainable and well-integrated sector framework – Education for All – that clearly linked education with poverty elimination and social and economic development. 15. Since the Dakar forum international evidence has shown that access to and permanence in the education system, while necessary, are not sufficient to achieve reduced poverty and greater shared growth. Basic education systems need to be of sufficient quality to ensure that their graduates acquire the basic, soft, and sector-specific skills needed to function efficiently and effectively on the job, adapt as workers or managers in a rapidly evolving technological society, and innovate and compete as entrepreneurs not only in developing new products but also in adopting those produced elsewhere.

16. The project is expected to make an important contribution to enhance the achievement of basic education reforms already realized under READ. The project will build on the successes of READ by providing resources to further support the Government’s efforts to improve reading and math skills for primary students. The project will use Disbursement–Linked Indicators (DLIs) to ensure that project-supported activities deliver the intended impact as envisioned by the Government through its reform agenda. II. PROJECT DEVELOPMENT OBJECTIVES (PDO)

A. PDO

17. The Project Development Objectives (PDO) are to improve the quality of education for primary school children in Mongolia, with particular emphasis on improving native language and mathematical skills, and to strengthen school-level planning.

5

B. Project Beneficiaries 18. The project beneficiaries will be primary school children, primary school teachers, and education policymakers in Mongolia. C. PDO Level Results Indicators 19. The key performance indicators will be:

(i) The average number of words read correctly with comprehension per minute at the end of grade 2;

(ii) At the end of grade 2, the average number of correct additions and subtractions completed per minute; and

(iii) Of the total number of school financing recipient schools in the project, the percentage that have successfully achieved the objectives and targets agreed in the school financing proposal

III. PROJECT DESCRIPTION

A. Project Components

20. The project has four components: Component 1: Improving learning outcomes Component 2: Pre- and in-service professional development of teachers Component 3: Implementation of a school support program Component 4: System management, monitoring and evaluation 21. Component 1: Improving learning outcomes (estimated cost: US$14.9 million). The purpose of this component is to ensure that by the end of grade 2 of primary school, Mongolian students are able to read fluently and acquire basic numeracy and math skills (like oral counting, additions and subtractions and identification of missing numbers in a sequence). This purpose is to be achieved through the implementation of the following two sub-components. 22. Sub-component 1.1 would involve the scaling up of several key interventions of the READ project to achieve national coverage. Specific activities to be financed include the provision of books and materials, creation of school libraries, a public reading campaign, the development of scientific and mathematics-focused children’s books, content development for special needs children, including those with disability, and an impact evaluation/assessment. 23. Sub-component 1.2 will support the national rollout of early grade native language and mathematical skills assessments for grades 1 and 2 students as well as the national assessment for grade 5. With respect to early grade native language and mathematical skills assessments, the project will support: (i) the development and validation of the assessment instruments; (ii) the training of teachers in appropriately using these assessment instruments and interpreting results; (iii) the on-site monitoring and support on the correct use of these instruments; and (iv) the

6

development of corrective and/or preventive policy and tailored-made interventions derived from the administration of these instruments by teachers in order to improve learning outcomes in reading and mathematics in the first two grades of primary schools in the entire country.

24. The project will finance under component 4 the carrying out of an impact evaluation study to assess the effectiveness of this project in improving learning outcomes in reading and mathematics at the end of grade 2. This impact evaluation study will be conducted through a qualified consulting firm or individual under Terms of Reference (TORs) satisfactory to the Bank. 25. With respect to the rollout of the national assessment for grade 5, the project will support: (i) strengthening the Education Evaluation Center (EEC) to administer and analyze national assessments for grade 5; (ii) developing and validating the standardized assessment test for grade 5; (iii) providing training to school administrators and teachers in preparation for the nationwide administration of the national assessment test for grade 5 including: (i) administering the assessment tests to grade 5 primary school students; (ii) scoring and grading the assessment tests; (iii) data capturing and entry; (iv) developing and administering quality control mechanisms of the results; and (v) using the results of the assessments to support system and policies for pedagogical and curricular improvement. 26. Component 2: Pre- and in-service professional development of teachers (estimated cost: US$3.3 million). The purpose of this component is to upgrade teacher quality by helping teachers develop the appropriate tools to provide individualized support for all children to improve their learning outcomes, in particular reading and mathematical skills for grades 1 and 2. This purpose is to be achieved through the implementation of the following two sub-components. 27. Sub-component 2.1 will support primary education Teacher Training Institutions throughout the country. The project will support: (i) the provision of high quality teaching and learning equipment and materials especially associated to early grade reading and mathematics; (ii) the provision of in-service training to teacher trainers; and (iii) a review of the practicum programs for teacher trainees, in particular as they relate to the teaching and learning practices of early grade reading and mathematics. Both pre- and in-service training programs will develop: (a) sample scripts that would help teachers initiate children into reading, with a focus on phonemic awareness and print exposure; and (b) teaching methods and materials to improve oral counting, rational counting, number recognition, comparisons of quantity, number sequencing, mathematical reasoning and the operations of addition and subtraction, skills that have been shown to be highly predictive of later mathematical skills. 28. Sub-component 2.2 will support pedagogical in-service training for teachers and the improvement of academic leadership of school principals and aimag methodologists. The sub-component will finance the: (i) provision of pedagogical training to teachers especially concerning the use of early grade learning materials for reading and mathematics; (ii) provision of in-service training and hands-on support to school principals; and (iii) strengthening the capacity of and providing enabling support to district and provincial education officers, methodologists, school principals and teachers. The in-service professional development

7

activities will be designed to create a much stronger instructional leadership function within schools, with teachers working as a team under the instructional leadership of the school director to increase subject matter skills particularly for reading and mathematics. Teachers will be given the knowledge and opportunity to carry out meaningful classroom-level assessments of achievement. Formative assessments will be developed based on a simplified version of the early grade native language and mathematical skills assessment testing protocols under Component 1 to help teachers to understand expected rates of learning in class, track the progress of all children, self-evaluate performance based on the benchmarking of competencies. 29. Additionally, the project will support the carrying out of classroom observations using the internationally comparable Stallings instrument to measure teachers’ use of instructional time, materials and core pedagogical practices and assess the effectiveness of the training provided and derive lessons learned to feedback into the pre- and in-service teacher training programs. 30. Component 3: Implementation of a school support program (estimated cost: US$6.1 million). The purpose of this component is to scale up MES’s “Talent Program” of school financing, currently being rolled out in 100 pilot schools, to the rest of the primary schools in the country. This component will provide about US$3,000 per school per block of financing for them to implement an approved school quality enhancement proposal. The financing represents a transfer of funds from MES to the schools to be spent autonomously by schools as per an approved school proposal including objectives, targets and a budget. Under the project, all primary schools nation-wide will have the opportunity to apply for school financing, whose broad purpose will be to increase the quality of education, including provision of enrichment opportunities for students. Independent annual audits will be conducted to ensure the use of the financing complies with approved school financing proposals. 31. The component will include support for increasing the time available for learning, along with the development of a school-level planning function. The School Support Handbook, an attachment to the Project Operation Manual (POM), has been developed and identifies broad areas of eligible expenditures such as co-curricular activities, management, staff development, services, teaching materials, furniture and equipment, and minor maintenance. 32. One key feature of this component warranted to be continuously monitored is the impact of the school financing in increasing the instructional time in school during the school year. 33. Component 4: System management, monitoring and evaluation (estimated cost: US$5.7 million). The purpose of this component is to support MES to effectively implement the project with the support of qualified technical assistance (TA). The TA will include: (i) chief technical advisor, financial management and procurement specialists attached to the office of the Project Director; and (ii) qualified consultants attached to the various working groups that have been established for the implementation of this project as indicated in Annex 3 of the PAD. This component will also finance the carrying out of an impact evaluation to assess the effectiveness of the project in improving reading and mathematics learning outcomes in primary schools. The objectives of this component will be achieved through the implementation of the following two sets of interventions: (i) supporting the Department of Monitoring and Evaluation of the MES

8

and the EEC to integrate student-learning outcomes into the tracking of learning trends; and (ii) supporting the capacity building of the MES in: (i) financial management and internal audit; and (ii) procurement.

B. Project Financing

Lending Instrument

34. The lending instrument is Investment Project Financing. To sharpen the focus on impact, the project will use DLIs. 35. Support to two of the four components and activities will be provided through disbursements made against evidence that a particular DLI has been achieved and eligible expenditures (EE) incurred. To facilitate measuring progress towards DLI achievement and disbursement, Annex 6 includes a table with the following DLI-specific information under each component: (i) DLI targets and amounts allocated; (ii) DLI achievement measures, verification protocol and disbursement rules; and (iii) a timeline for DLI achievement.

36. The DLIs and project activities will be monitored through independent validation conducted by a third party verifier hired by MES. Regular Bank implementation support missions jointly undertaken with the Government will supervise the validation process. The third party verifier will be selected competitively based on TORs acceptable to the Bank.

37. While DLIs have annual targets, verification will be done semi-annually. MES will report to the Bank no later than July 1 of each year on the status of the results framework and the DLIs (covering DLIs achieved from July 1 to December 31 of the previous year), as will a third-party verifier for selected DLIs.2 MES will also report to the Bank no later than December 1 of each year on DLI achievement, covering the first six months of that year (January 1 to June 30). If the Bank is satisfied that the DLI requirements have been met and the verification protocol properly followed, it will authorize the withdrawal of proceeds in accordance with the disbursement rules associated with each DLI. Project Cost and Financing

Project Components Project

cost (US$M)

IDA Financing (US$M)

% Financing

1. Improving learning outcomes 14.9 14.9 100

2. Pre- and in-service professional development of teachers 3.3 3.3 100

3. Implementation of a school support program 6.1 6.1 100

4. System management, monitoring and evaluation 5.7 5.7 100

2 The third-party verifier assessment will cover the DLIs, as indicated in Annex 6 of this PAD

9

Total Costs 30.0 30.0 100

Total Project Costs 30.0 30.0

Total Financing Required 30.0 30.0

C. Lessons Learned and Reflected in the Project Design 38. Key lessons learned incorporated into the project design include:

A focus on student access to primary schooling has given way to concerns over educational quality. Student outcomes and academic achievement need to be systematically measured and monitored in order to appraise the effectiveness, progress and impact of the MES education reform agenda.

DLIs, when striking the right balance between feasibility, ambition, and flexibility, can sharpen a government’s focus on results. By using DLIs, this project will support MES’s reform agenda, as the disbursement of funds will be based on the achievement of specified results agreed as DLIs. Each DLI, which contains a specific set of disbursement rules and a verification protocol, will guide the timing and amount of disbursement as in Annex 6.

Familiarizing ministries, teachers, school managers and parents with children’s literature

and its benefits to classroom learning is a critical condition in ensuring local support and the usefulness of the resources to the classrooms.

Evidence from the READ project has shown that changing the way reading is taught had large and measurable effects on reading skills of students. The project will provide an avenue to build upon these positive findings by introducing and scaling up the early grade native language assessment methodologies nationwide.

A cascade training model in which classroom teachers are trained on how to use books to

teach reading and writing, how to integrate classroom libraries into their daily classroom routines and how to encourage students’ use of classroom libraries can help ensure sustainability. Evidence from the READ project shows that “core schools,” in which teacher development was a main focus, were effective in reaching more teachers and disseminating innovative pedagogical practices that lasted beyond the life of that project.

Distribution costs can be reduced by distributing books to teachers at the end of teacher training workshops, which also serve to make them active participants in the implementation process, instead of passive recipients of books.

The implementation of projects in Mongolia can be jeopardized without adequate capacity and management oversight of the implementing agency. Projects with many activities in Mongolia are particularly vulnerable due to weak institutional settings. There is a need for adequate and qualified staff to oversee the implementation of the

10

project components to ensure that the resources reach the appropriate schools and classrooms.

Effective contract management is essential to quality service delivery. Contracts and deadlines can be managed well by implementing a set of measures to prevent delays. These measures include: (i) agreeing on detailed milestones to monitor the implementation of contracts; (ii) making amendments to the contract if the expected final completion date is modified; (iii) enforcing penalties for failure to achieve milestones; and (iv) setting and monitoring cash flow requirements.

IV. IMPLEMENTATION

A. Institutional and Implementation Arrangements

39. Overall responsibility for the project supporting the MES reform agenda will be vested with MES. Project implementation will be mainstreamed using the MES structure at the central and decentralized (aimag) levels supported by qualified TA to be financed by the project. Education departments and relevant government officials at aimag (province) and soum (village) levels and schools will be responsible for the activities executed at the decentralized level over the five-year project period. 40. At the highest level, a Steering Committee (SC) will be established by the Minister, chaired by the MES’ State Secretary and comprising the heads and or designated representatives of: (i) the Strategic Policy and Planning Department; (ii) the Policy Implementation and Coordination Department including the Pre-school and Primary Unit; (iii) the Monitoring and Evaluation and Internal Auditing Department; and (iv) the EEC. The committee will approve annual activities, oversee implementation progress, and provide overall guidance to the Project Director and the project-related working groups. 41. The Minister of Education will designate a Project Director. The Project Director will be accountable to the Director of the Department of Strategic Policy and Planning. The Project Director, supported by a Chief Technical Advisor and Financial Management and Procurement specialists that will be externally selected by MES following World Bank procedures, will be responsible for the overall strategic direction for the project. 42. The Chief Technical Advisor, externally selected by MES, will support the Project Director on a daily basis for contract management and recruitment of various TA for implementation of project components and implementation assistance. Annex 2 contains a more detailed description of this role.

43. The Financial Management Specialist, in close collaboration with the Finance and Investment Department (FID) of the MES, will support the Project Director and will be responsible for the overall financial management functions of the project. The Procurement Specialist, in close collaboration with the Government Procurement Agency (GPA), will support the Project Director and coordinate all project procurement activities, including the preparation and regular updates, if needed, of the project Procurement Plan.

11

44. Various working groups will be established for implementation of project components. MES’ Policy Implementation and Coordination Department including its Pre-school and Primary Education Unit will be responsible for implementation of Components 1 and 2, except the national assessment for grade 5, in close collaboration with the working groups established for that purpose. The EEC will be responsible for managing the collected data through early grade native language and mathematical skills assessments and the rolling out of the standardized national assessment for grade 5 in close collaboration with the working group established for that purpose. 45. The MES’ Strategic Policy and Planning Department will be responsible for the implementation of Component 3 in close collaboration with the working group established for that purpose. 46. The MES’ Monitoring and Evaluation Department will be responsible for the implementation of the M&E activities under Component 4 in close collaboration with the working group established for that purpose. 47. Ad hoc Working Groups, comprised by existing officials of the participating departments and units within the MES, will be established and supported by coordinators and appropriate TA selected under TORs satisfactory to the Bank and recruited following Bank procedures. The Working Groups/TA will take the lead of carrying out the overall tasks necessary for planning, preparation, coordination, logistics, implementation, supervision and monitoring and evaluation of the component activities of the project. B. Results Monitoring and Evaluation

48. The relevant departments of MES will monitor the indicators of the project. The monitoring indicators are provided in Annex 1. The data related to output indicators will be provided semi-annually, while MES will provide outcome-related data annually, using its EMIS. The progress in meeting the monitoring indicators will be reviewed by the Bank semi-annually. M&E will include monitoring of DLIs as described in the Lending Instrument subsection, and an impact evaluation for early grade native language assessment activities noted in the Project Description subsection. Where appropriate, data collected will be disaggregated by gender, in order to understand and respond to specific issues. C. Sustainability 49. The project will be sustainable because of the track record of the Government in implementing projects to improve the quality of education to benefit all children. For this project, only the key activities for achieving MES’s reform agenda have been selected as interventions, some of which MES has already begun implementing using its own funds. The Ministry has budgeted in its Mid-Term Expenditure Framework forecast that it will assume responsibility for each project activity in the outer years. This provides additional assurance that these costs are embedded in the public expenditure framework and will be sustained after project closure. 50. To assist with capacity for executing project activities, a detailed POM has been developed to guide project implementation. This manual will provide MES staff with a

12

comprehensive set of written policies and procedures for, among other things, routine tasks, administrative activities, records management, and documentation. Moreover, individual technical and fiduciary consultants, whose terms of reference are included in the POM, will complement the fiduciary training provided by the Bank and will assist MES in daily implementation activities. The dissemination of national assessment results, their discussion at the national fora to improve sector functioning, and the integration of early grade native language and mathematical skills assessment data into the Education Management Information System (EMIS) will foster public accountability that will further contribute to sustainability. Finally, the overwhelming enthusiasm of parents and communities for high quality primary education has provided an additional insurance that the project will be sustained from the demand side. Since there is full technical and operational alignment with MES’s reform goals, a financial commitment already displayed by the Government in its financial forecasts, and an embedding of the project activities into the current public expenditure framework, there is a relatively low sustainability risk. V. KEY RISKS AND MITIGATION MEASURES

A. Risk Ratings Summary Table

Risk Category Rating

Stakeholder Risk Low

Implementing Agency Risk

- Capacity Substantial

- Governance Moderate

Project Risk

- Design Low

- Social and Environmental Low

- Program and Donor Low

- Delivery Monitoring and Sustainability Moderate

Overall Implementation Risk Moderate

B. Overall Risk Rating Explanation

51. The overall implementation risk of this project is moderate because of the governance and capacity risks, particularly in financial management and procurement. This is despite the fact that the project has wide support from parents, teachers, communities, local governments, development partners and that the Government’s and civil society’s interests are closely harmonized. Moreover, MES is thoroughly familiar with the proposed financing modality, and many of the activities have been implemented as pilots before. Given that this is the first operation in the sector to use DLIs, an additional risk may be that in the event of MES failing to meet the DLI targets, some of the funds issued to MES may have to be repaid to IDA at the end of the project. To mitigate this risk, the DLIs have been formulated with the following principles in mind: simplicity in number and scope to facilitate supervision and disbursement; alignment

13

with the Results Framework to ensure a harmonized focus on outputs; traction to contribute to a steady implementation pace for the DLI-related components; and flexibility to ensure liquidity to operate. The project’s strong emphasis on national and international TA will also help mitigate this risk, as well as the capacity risks associated with the project. Detailed mitigation measures are discussed in the ORAF. VI. APPRAISAL SUMMARY

A. Economic and Financial Analysis

52. The project’s development impact. The project will have significant development impact through generating better educational attainment and quality which leads to the increased private returns. The Bank team prepared a cost-benefit analysis (CBA) of the project’s interventions based on the actual quantity and costs of the project interventions as implemented under the READ project and a reasonable projection of costs moving forward. Component 1 (the scaling up of the READ to achieve national coverage through the provision of scientific and mathematics-focused books for primary school children via classroom libraries in schools located in UB and aimag centers) and Component 2 (improving teachers’ quality through a cascaded scheme of in-service teacher training) are amenable to this type of analysis and were appraised in a CBA. The project’s direct costs were estimated based on current levels. The CBA allows the PAD to calculate indirect costs and benefits both with the project and without the project. 53. The results of this economic analysis demonstrate that the financial outcomes from the READ expansion and teacher training are significant. The CBA results show a combined net present value (NPV) of US$ 2.09 million and an internal rate of return (IRR) of 12.9 percent. Individual assessments of these components also show promising returns. The NPV from higher educational attainment resulting from access to books via classroom libraries is estimated at US$ 0.95 million. The IRR from this classroom libraries scheme is 12.26 percent. Similarly, the in-service teacher training scheme yields an NPV of US$1.14 million, with an IRR of 13.92 percent. These results have been tested for robustness with changes in individual assumptions; the results remain robust to changes in virtually all of the critical variables. 54. Beyond private financial benefits computed, other outcomes are likely to be positively affected. Exposure to higher quality primary education services has been documented to have impacts on improving school outcomes such as lower repetition and dropout rates, higher enrollment rates, better long-term cognitive development, and greater school attainment. In addition, improved educational quality could support the easing of socio-economic inequality by addressing inequality in opportunities, fostering improved nutritional outcomes, and providing other benefits. 55. Rationale for public sector financing. The rationale for public financing for improving the quality of basic education is strong. Investing early in the education cycle is efficient since waiting to invest in human capital until adulthood is more costly; the older one becomes, the higher the opportunity cost of keeping that person in remedial programs such as second chance education programs. International evidence demonstrates that the rate of return to each dollar of

14

education investment decreases with age.3 Studies have also found that education encourages active citizenship as measured by voter participation4 and increases female participation in the labor market by delaying the age of childbearing.5Moreover, the public sector is responsible for managing about 80 percent of all schools in Mongolia, with over 90 percent of the student population. Teachers in privately managed schools receive public financing as well. The provision of financing through this project would help ensure that public financing results in optimal impact. 56. The Bank’s value added. The Bank brings a great deal of experience and expertise gained in part through the management of READ in Mongolia and other countries. The Bank also brings the pooled expertise gained from supervising at least twenty projects worldwide over the past decade that focus on improved learning outcomes and school-level planning. The project includes the scale up and continuation of many of the activities which were successfully tested under the previous IDA projects. In particular, core activities such as student assessment in reading and mathematics and the implementation of school support schemes are areas where the Bank has extensive knowledge and operational technical experience, both regionally and globally. B. Technical

57. The project draws on best-practice reform efforts in Mongolia and also other countries. The components are embedded within the Ministry’s national reform agenda. The project was prepared through a lengthy consultative process. Innovations—such as provision of classroom libraries, demand-side incentives (a public reading campaign), strengthening decentralized education quality improvement planning and teacher professional development, and school improvement grants—have all been piloted by MES during the past few years. C. Financial Management

58. MES is the principal implementation agency for the project but the project implementation will be mainstreamed using the MES structure not only at the central level but also at the decentralized levels. Thus the education departments and relevant government officials at aimag and soum levels and schools will be responsible for the activities executed at the decentralized level. However, the financial management functions for the project will be centralized at the MES. MES will designate a Project Director, who will be supported by an externally selected Financial Management Specialist Consultant in charge of managing the project’s financial management systems. A financial management capacity assessment for the project has been conducted by the Bank for MES and actions to strengthen the project’s financial management capacity have been identified. The FM assessment has concluded that with implementation of the actions identified, the financial management arrangements will satisfy the

3 Heckman, J. J. & Lochner, L. (1999). Rethinking Education and Training Policy: Understanding the Sources of Skill Formation in a Modern Economy. Mimeo. 4 Hall, J. C. (2006). Positive externalities and government involvement in education. Journal of Private Enterprise, 21(2), 165-175. 5 Behrman, J. R., Murthy, A., Quisumbing, A., Ramakrishna, U. & Young, K. (2006). What is the real impact of education on age of first parenthood and family formation? Background paper for the World Bank World Development Report 2007.

15

Bank’s requirements under OP/BP 10.00. Annex 3 of the PAD provides additional information on financial management. D. Procurement

59. Procurement at the central level will be conducted by the GPA in collaboration with MES. GPA will have overall responsibility for procurement cycle management, while MES will provide the technical inputs for procurement including, packaging of contracts, drafting of the procurement plan, preparation of Terms of Reference and Technical Specifications; and setting of evaluation and qualification criteria. MES will also participate in bid/proposal evaluation and will be responsible for contract supervision and management. Procurement under the matching grants will be conducted by the participating schools using very simple procurement methods described in the School Support Handbook. An assessment of the capacity of the GPA and MES to implement procurement under the project has been carried out. The key risks identified are: (i) new staff members of MES and the GPA are relatively new to procurement under a Bank-financed project; and (ii) possible lack of coordination between the GPA and MES which could lead to delays in processing procurement. Risk mitigation measures have been discussed and agreed with GPA and MES. Key mitigation measures include: (i) MES to hire/assign a Procurement Specialist with qualifications and under TOR acceptable to the Bank to coordinate procurement activities with the GPA within three months of effectiveness; (ii) training in procurement under Bank-financed projects to be provided by the Bank (or by training institutions acceptable to the Bank) to key staff of GPA and MES during project preparation and periodically during project implementation; and (iii) GPA and MES to agree on standard processing times for procurement along with a detailed procurement plan with dates for key milestones including submission of TORs and technical specifications. The draft procurement plan for the project has been prepared by MES and it will be updated annually or as required to reflect project implementation needs. A brief summary of the procurement capacity assessment and project procurement arrangements is provided in Annex 3. E. Social (including Safeguards)

60. While there are significant differences in achievement and enrollment levels between rural and urban students, there are no aggregate differences in enrollment rates between boys and girls within schools. Additionally, there is no aggregate difference in completion rates for boys and girls. However, in rural areas, male participation rates are lower than female rates, as boys who herd have responsibilities which keep them out of school. Parental perceptions of declining school quality in rural areas and that a high level of education is not necessary for herder boys may act as a disincentive for boys to remain in school. The project will attempt to address the disparities between rural and urban schools (and thus gender) by supporting activities that will specifically benefit rural schools and provide them with the opportunity to improve their educational services. Teachers who lack the resources and appropriate training to prepare lessons based on principles of literacy are direct project beneficiaries. Project activities take into account specific constraints and needs faced by rural schools and have been designed under consultation with rural teachers, other development partners, and NGOs who have worked in rural schools.

16

61. OP/BP 4.10 on Indigenous People. Overall, this project aims to improve the quality of education for primary schools in Mongolia. It is a nationwide project which will bring significant social benefits to local people. 62. Though largely an ethnically homogenous nation, with 80-86 percent of the population estimated to be of Khalkh Mongol origin, Mongolia is home to more than 10 ethnic minorities that have historically resided within current state boundaries. These ethnic minorities include Kazakh, Tuva, Dukha, Chantuu, Khoton and others. At the preparation stage of the project, the exact locations of project activities cannot be determined.

63. According to the Bank’s OP4.10, an Indigenous People Planning Framework (IPPF) has been prepared by the client. The main purpose of the document is to: (i) ensure ethnic minority groups can benefit from the project; (ii) guide “free, prior, and informed consultation” with ethnic minority communities; and (iii) ensure issues raised by ethnic minority communities are incorporated into subcomponent designs. The IPPF was disclosed locally on February 20, 2014 and the English version of the IPPF was disclosed at the Infoshop on March 13, 2014. 64. OP/BP 4.12 on Involuntary Resettlement. The project will not finance civil works, but support capacity building activities for primary schools in Mongolia, including the scaling up of the READ project (i.e., development and provision of books and classroom libraries, teacher training, and an impact evaluation), and Talent Program (i.e., management, staff development, services, teaching materials, sports, furniture and equipment, and maintenance). Therefore, no involuntary resettlement is expected for the purpose of the project implementation. F. Environment (including Safeguards)

65. The project will not finance civil works but will support capacity building activities for primary schools in Mongolia. The project will have minimal or no adverse environmental impacts. The Environmental Assessment policy is triggered due to the need to assess and take into account, in an integrated manner, the social aspects of the project particularly any impacts on Indigenous People communities. The project is not required to prepare an instrument under OP 4.01.

17

Annex 1: Results Framework and Monitoring

MONGOLIA: EDUCATION QUALITY REFORM PROJECT .

Project Development Objectives .

PDO Statement: The Project Development Objective is to improve the quality of education for primary school children in Mongolia, with particular emphasis on improving native language and mathematical skills, and to strengthen school-level planning.

Project Development Objective Indicators

Cumulative Target Values Data Source/

Responsibility for

Indicator Name Core Unit of

Measure Baseline YR1 YR2 YR3 YR4

End Target

FrequencyMethodology Data

Collection

PDO Indicator 1: The average number of words read correctly with comprehension per minute at end of grade 2.

Number To be

collected in Y1

Establish baseline

Baseline + 5%

Baseline + 10%

Baseline + 10%

Baseline + 20%

Annual

End-of-year grade 2 Teacher Reports (spot-

checked by aimag

methodologist)

Education Evaluation

Center

PDO indicator 2: At the end of grade 2, the average number of correct additions and subtractions per minute

Number

To be collected

in Y2

Establish baseline

Baseline + 5%

Baseline + 5%

Baseline + 10%

Annual

End-of-year grade 2 Teacher Reports (spot-

checked by aimag

methodologist)

Education Evaluation

Center

PDO indicator 3: Of the total number of school financing recipient schools in the project, the percentage that have successfully achieved the school proposal objectives and targets agreed in the school financing

Percentage 0 0 10%* 30%* 70%* 70%* In Y2, Y3

and Y4 Project Reports MES

.\

Intermediate Results Indicators

18

Cumulative Target Values Data Source/

Responsibility for

Indicator Name Core Unit of

Measure Baseline YR1 YR2 YR3 YR4 End Target

Frequency

Methodology Data Collection

Intermediate results indicator 1: Percentage of grade 2 teachers reporting native language assessments results

PercentageTo be

collected in Y1

Establish baseline

Baseline +20%

Baseline +40%

Baseline +60%

Baseline +80%

AnnualSample Survey

Education Evaluation

Center

Intermediate results indicator 2: Percentage of grade 2 teachers reporting assessment results on: (a) additions and subtractions; and (b) identification of missing numbers in a sequence

PercentageTo be

collected in Y2

Establish baseline

Baseline +20%

Baseline +40%

Baseline +60%

AnnualSample Survey

Education Evaluation

Center

Intermediate indicator 3: Of the total number of grades 1 and 2 primary school teachers to be trained during the life of the project, the cumulative percentage that achieved intended competency levels in native language assessment methodology

Percentage 0 0 10%* 35%* 60%* 60%*

In Y2, Y3 and

Y4

Administered tests at the end of the training

Education Evaluation

Center under the guidance of the

MES

Intermediate indicator 4: Of the total number of grades 1 and 2 primary school teachers to be trained during the life of the project, the cumulative percentage that achieved intended competency levels in early grade mathematical skills assessment methodology

Percentage 0 0 0 10%* 35%* 60%* In Y3, Y4 and

Y5

Administered tests at the end

of training

Education Evaluation

Center under the guidance of the

MES

19

Intermediate indicator 5: Number of additional qualified primary teachers resulting from project interventions**

Number 500

2,100 3,500 4,500 6,000 7,400 Annual Project Reports

MES

Intermediate indicator 6: Of the total number of school financing recipient schools in the project, the percentage that use the proceeds for pedagogical-related activities (as defined in the School Support Handbook attached to the Project Operational Manual)

Percentage 0 0 20%* 25%* 30%* 30%*

In Y2, Y3 and

Y4 Project Reports MES

Intermediate indicator 7: Roll out of grade 5 National Assessment

AssessmentNo yet

rolled outNot yet

rolled out No yet

rolled out Assessment rolled out

Assessment rolled out

Assessment rolled out

Y3 National

Assessment Education

Evaluation Center

Intermediate indicator 8: Average percentage of the school year allocated for instruction in the classroom

Percentage

To be collected

in Y1

Establish baseline

Baseline Baseline Baseline Baseline

+ 6% Y1 and

Y4

Stallings- Type

Assessment Consultant

Intermediate indicator 9: System for learning assessment at the primary level*

Yes/No No

No No Yes Yes Yes AnnualNational

Assessment

Education Evaluation Center

Intermediate indicator 9 (Sub-Indicator): Utility of the learning assessment system*

Number Sub-Type Supplemental

1

1 1 2 2 2 Annual

National Assessment

Education Evaluation Center

Intermediate indicator 10: Direct project beneficiaries (number), of which female (percentage)

Number (percentage

) 0

44,000 (52%)

44,000 (52%)

44,000 (52%)

44,000 (52%)

44,000 (52%)

Annual Project Reports MES

20

Annex 2: Detailed Project Description

MONGOLIA: EDUCATION QUALITY REFORM PROJECT 66. The project would support MES’s Educational Quality Reform Program to help improve the quality of basic education for all Mongolian children. The project will improve student learning by creating a consistent results focus within the education system and strengthening the system’s capacity to provide classroom-level support for teaching and learning. A national assessment system would be created, providing real-time feedback to teachers and administrators about learning levels; proven interventions to improve reading would be taken to national scale. The project’s interventions complement the existing interventions of Mongolia’s development partners and will build capacity within the Ministry to sustain the interventions once the project closes. The project activities would be consistent with Mongolia’s stated intent to achieve the Millennium Development Goals by 2015 and transition to a knowledge-based economy by 2021. The project activities also closely align with the framework of the World Bank’s CPS and its call for stronger policy and implementation frameworks to provide more effective educational services. 67. The project will aim to achieve the development objective through four dimensions that will nurture educational quality improvements. This strategy will involve: (i) extending access to high quality learning materials; (ii) establishing and institutionalizing the national assessment monitoring system; (iii) improving the pedagogical skills of the current and future cohort of Mongolia’s teaching workforce; and (iv) lengthening the school day. 68. The theory of change underpinning the project is schematically depicted in Figure 1 below. First, classroom libraries will be provided to all primary schools in Mongolia to improve the classroom experience. To help improve learning, an independent national assessment system will monitor progress and provide specific feedback to educators and policymakers to guide the education system towards improved student participation and academic achievement outcomes. These quality interventions will be complemented by professional development efforts to enhance principal and teacher technical capacity to deliver academic programs more effectively. Finally, the school financing supported by the project will lengthen the school day and support increased instruction time.

21

Figure 1: Mongolia Education Sector Reform Project: Towards Better Student Outcomes

69. The project comprises four components: 70. Component 1: Improving learning outcomes (estimated cost: US$14.9 million). The purpose of this component is to ensure that by the end of grade 2 of primary school, Mongolia students are able to read fluently and acquire basic numeracy and math skills (like oral counting, additions and subtractions and identification of missing numbers in a sequence). This purpose is to be achieved through the implementation of the following two sub-components. 71. Sub-component 1.1 would involve the scaling up of several of key interventions of the READ project to achieve national coverage. Specific activities to be financed include the provision of books and materials, creation of school libraries, a public reading campaign, the development of scientific and mathematics-focused children’s books, content development for special needs children, including those with disability, and an impact evaluation/assessment. 72. The sub-component will specifically finance:

(i) Technical assistance to support development of technical and pedagogical

specifications for new books and learning materials, especially focused on math and science;

(ii) Development and provision of science- and math-focused books and learning materials;

(iii) Technical assistance to support development of books for children with special needs;

(iv) Development and procurement of books for children with special needs;

Extending Access to Quality Learning Materials

Improved Student

Outcomes

Establishing a National Assessment System

Education Quality

Improvement

Improving Teacher Professional Development

Increasing Instruction Time

22

(v) Creation of classroom libraries; (vi) Local expert as consulting firm or individual consultant to design a reading

campaign; (vii) Implementation of public reading campaign; (viii)Development of training modules to train teachers on the use of all developed

materials and delivery of the training to teachers and teachers of teachers in the pre- and in-service primary education training schools; and

(ix) On-site monitoring of the use of new books and learning materials by teachers and brainstorming workshops to develop corrective and/or preventive measures to improve the effective use of the new books/learning materials and feeding these recommendations to teachers and the pre- and in-service primary education training schools.

73. Sub-component 1.2 will support the national rollout of early grade native language and mathematical skills assessments for grades 1 and 2 as well as the national assessment for grade 5. With respect to the early grade native language and mathematical skills assessments, the project will support: (i) the development and validation of the assessment instruments; (ii) the training of teachers in appropriately using these assessment instruments and interpreting results; (iii) the on-site monitoring and support on the correct use of these instruments; and (iv) the development of corrective and/or preventive policy and tailored-made interventions derived from the administration of these instruments by teachers in order to improve learning outcomes in reading and mathematics in the first two grades of primary schools in the entire country. 74. The project will finance under component 4, the carrying out of an impact evaluation study to assess the effectiveness of this project in improving learning outcomes in reading and mathematics. This impact evaluation study will be conducted through a qualified consulting firm or individual under TORs satisfactory to the Bank. 75. With respect to the rollout of the national assessment for grade 5, the project will support: (i) strengthening the EEC to administer and analyze national assessments for grade 5; (ii) developing and validating the standardized assessment test for grade 5; (iii) providing training to school administrators and teachers in preparation for the nationwide administration of the national assessment test for grade 5 including: (iv) administering the assessment tests to grade 5 primary school students; (v) scoring and grading the assessment tests; (vi) data capturing and entry; (vii) developing and administering quality control mechanisms of the results; and (viii) using the results of the assessments to support system pedagogical and curricular improvement. 76. The sub-component will specifically finance:

(i) Technical assistance: one for early grade native language and mathematical skills

assessments and another one for standardized learning assessment systems for grade 5;

(ii) Development of the early grade native language and mathematical skills assessments instruments, adaptation workshops and pilots for validation as for grades 1 and 2 well as the national assessment test for grade 5 (part of the TA task);

23

(iii) Development of training modules to train teachers in the appropriate use of the early grade native language and mathematical skills assessments as well as the administrators of the national assessment tests (part of the TA task);

(iv) Delivery of the training to teachers and methodologists participating in the early grade native language and mathematical skills assessments and personnel administering the standardized national assessment test for grade 5 including staff from the EEC;

(v) National roll-out of the early grade native language and mathematical skills assessments for grades 1 and 2 and national assessment instruments for grade 5;

(vi) On-site monitoring of the use of the early grade native language and mathematical skills assessments instruments by participating teachers and brainstorming workshops to develop corrective and or preventive measures to improve the effective use of the early grade native language and mathematical skills assessments instruments and feeding these recommendations to teachers and the pre- and in-service primary education training schools;

(vii) Experts for sampling, data management, data analysis and spot-checking in collaboration with EEC staff;

(viii)Provision of office equipment for administration of the national assessment test for grade 5;

(ix) National symposia for dissemination of findings and development of policies to improve learning outcomes in the primary education levels; and

(x) Integration of early grade native language and mathematical skills assessments and national assessment results of grade 5 into the EMIS in collaboration with the Department of Monitoring and Evaluation of MES.

77. Component 2: Pre- and in-service professional development of teachers (estimated cost: US$3.3 million). The purpose of this component is to upgrade teacher quality by helping teachers develop the appropriate tools to provide individualized support for all children improving their learning outcomes, in particular reading and mathematical skills for grades 1 and 2. This purpose is to be achieved through the implementation of the following two sub-components. 78. Sub-component 2.1 will support primary education Teacher Training Institutions throughout the country. The project will support: (i) the provision of high quality teaching and learning equipment and materials especially associated to early grade reading and mathematics; (ii) the provision of in-service training to teacher trainers; and (iii) a review of the practicum programs for teacher trainees, in particular as they relate to the teaching and learning practices of early grade reading and mathematics. Both pre- and in-service training programs will develop: (i) sample scripts that would help teachers initiate children into reading, with a focus on phonemic awareness and print exposure; and (ii) teaching methods and materials to improve oral counting, rational counting, number recognition, comparison of quantity, number sequencing, mathematical reasoning and the operations of addition and subtraction, skills that have been shown to be highly predictive of later mathematical skills. 79. Sub-component 2.2 will support pedagogical in-service training for teachers and the improvement of academic leadership of school principals. The sub-component will finance the: (i) provision of pedagogical training to teachers especially concerning the use of early grade

24

learning materials for reading and mathematics; (ii) provision of in-service training and hands-on support to school principals; and (iii) strengthening the capacity of and providing enabling support to district and provincial education officers, school principals and teachers. The in-service professional development activities will be designed to create a much stronger instructional leadership function within schools, with teachers working as a team under the instructional leadership of the school director to increase subject matter skills particularly for reading and mathematics. Teachers will be given the knowledge and opportunity to carry out meaningful classroom-level assessments of achievement. Formative assessments will be developed based on a simplified version of the early grade native language and mathematical skills assessments testing protocols under Component 1 to help teachers to understand expected rates of learning in class, track the progress of all children, self-evaluate performance based on the benchmarking of competencies. 80. Additionally, the project will support the carrying out of classroom observations using the internationally comparable Stalling instrument to assess the effectiveness of the training provided and derive lessons learned to feedback into the pre- and in-service teacher training programs. 81. The component will specifically finance:

(i) Expert and technical assistance for pre- and in-service primary school teacher

training schools to review and develop training modules associated to the training aimed by these two sub-components;

(ii) Technical assistance to develop sample scripts for initiation into reading; (iii) Teaching and learning equipment and materials; (iv) In-service training to teacher trainers; (v) Technical assistance to develop assessments for teachers to track reading and

math progress of children and self-evaluation; (vi) Delivery of pedagogical training to teachers; (vii) In-service training and support to school principals; (viii)Experts of the technical, data management, and sampling, including Stallings

instrument for classroom observation; (ix) Development of instruments for time on task surveys to collect baseline and

endline data and the piloting for validation; (x) Undertaking of the data collection using the developed instrument in sample

schools; (xi) On-site monitoring including classroom observations to ascertain the application

of the training provided in early grade primary school classrooms and brainstorming workshops to develop corrective and/or preventive measures to improve the training modules associated with the training aimed by these two sub-components; and

(xii) National symposia on the findings and lessons learnt. 82. Component 3: Implementation of a school support program (estimated cost: US$6.1 million). The purpose of this component is to scale up MES’s “Talent Program” of school financing, currently being rolled out in 100 pilot schools, to the rest of the primary schools in the

25

country. This component will provide about $3,000 per school per block of financing for them to implement an approved school quality enhancement proposal. The block financing represent a transfer of funds from the MES to the schools to be spent autonomously by schools as per an approved school proposal including objectives, targets and a budget. Under the project, all primary schools nation-wide will have the opportunity to apply for school financing, whose broad purpose will be to increase the quality of education, including provision of enrichment opportunities for students. Grants will be made on the basis of a sub-agreement between MES and the school, and arrangements will be specified in the School Support Handbook. Independent annual audits will be conducted to ensure use of the financing complies with approved school financing proposals. 83. The project will include support for increasing the time available for learning, along with the development of a school-level planning function. As per the School Support Handbook, an attachment to the POM, there are seven broad areas of eligible expenditures: co-curricular activities, management, staff development, services, teaching materials, furniture and equipment, and minor maintenance. 84. A key feature of this component that will be continuously monitored, is the impact of the school financing in increasing the instructional time in school during the school year. 85. The component will specifically finance:

(i) Technical assistance: one for the school financing component to develop

orientation materials for the prepare school financing proposals and another for designing the awareness campaign about the school support program;

(ii) Implementation of an awareness campaign; (iii) Provision of orientation materials and the School Support Handbook; (iv) Delivery of the orientation training to participating primary schools and Aimag

education authorities; (v) Implementation of the school financing by participating primary schools; and (vi) On-site monitoring of the implementation of school financing; and fine-tuning of

the school support component for the remaining batches of primary schools. 86. Component 4: System management, monitoring and evaluation (estimated cost: US$5.7 million). The purpose of this component is to support MES in the timely and effective implementation the project with the support of qualified TA. The TA will include: (i) chief technical advisor, financial management and procurement specialists attached to the office of the Project Director: and (ii) qualified consultants attached to the various working groups that have been established for the implementation of this project as indicated in the Implementation Arrangement of Annex 2. This component will also finance the carrying out of an impact evaluation study to assess the effectiveness of the project in improving reading and mathematics learning outcomes in the first two grades of primary schools. The objectives of this component will be achieved through the implementation of the following two sets of interventions:

(i) Supporting the Department of Monitoring and Evaluation of the MES to integrate student-learning outcomes into tracking of learning trends.

26

(ii) Supporting the capacity building of the MES in: (i) financial management and internal audit; and (ii) procurement.

87. The component will specifically finance:

(i) Technical assistance by individual consultants: for the Chief Technical Advisor,

Financial Management and Procurement Specialists; (ii) Coordinators for Working Groups (books, early grade native language and

mathematical skills assessments, national assessment, teacher training, school support, and EMIS);

(iii) Assistance to verify the achievement of DLIs (third-party verifier for selected DLIs);

(iv) Expert to support strengthening the EMIS; (v) Provision of equipment needed for installing new system; (vi) Delivery of training on new EMIS; (vii) Annual audit of the project; (viii)Impact Evaluation of early grade native language assessment activities; (ix) Monitoring of project activities by MES for urban schools and by aimag offices

for rural schools; (x) Diagnostic workshops to improve performance of early grade native language and

mathematical skills assessments rollout; (xi) Project management, monitoring and evaluation; (xii) Experts on disability survey, sampling, and data management; (xiii)Development of instruments of disability survey and pilot for validation; and (xiv) National symposium on the findings and lessons learnt.

.

Table A2.1 Costs and Quantities of Activities Financed by IDA6

Line item Unit Quantity

(estimate) Unit cost

(USD) Cost (USD)

Component 1: Improving Learning Outcomes (estimated cost: US$14.9 million)

A. S

cali

ng

up

of

RE

AD

pro

ject

National TA to develop scientific and mathematics-focused books

person_month 6 3,800 22,800

1Translation and quality control of scientific and mathematics-focused books

lump sum 1,500 30 50,000

Printing/publishing of scientific and mathematics-focused books developed

lump sum

3,822,100

International TA to develop books for special needs children

lump sum 87,000

Local TA (to work with international TA to adapt content)

person_month 3 3,800 11,400

6 For activity and costing details, see Costab output in costing annexes in the POM.

27

Editing page 150 5 750

Printing/publishing of books/material developed

Each 375 100 37,500

Bookshelves for classroom libraries Each 5,300 160 848,000 Delivery of bookshelves lump sum 1 100,000 100,000

Local designer to develop public reading campaign

person_month 12 3,800 45,600

TV advertisements for reading campaign airtime 250 180 45,000

Posters for reading campaign page 5,000 1 5,000

B. T

each

er

trai

nin

g Local firm to conduct teacher training on READ scale-up activities

lump sum 74,220

Teacher per diems, monitoring, and other operating costs

lump sum 939,960

C. E

arly

Gra

de

Nat

ive

Lan

guag

e A

sses

smen

ts

International TA to oversee activities lump sum 330,000

Adaptation workshop of trainers and MES/Education Evaluation Center on early grade native language assessments

lump sum 5,750

Local training firm to pilot and administer full-scale roll out of the early grade native language assessment instrument

lump sum 444,800

Teacher per diems and other operating costs lump sum 1,253,025

7. National symposia package 3 5,050 15,150 8. Monitoring of early grade native language assessment use at schools

152,000

D. I

mp

act

Eva

luat

ion

International technical assistance to oversee activities

lump sum 330,000

Local firm to implement activities lump sum 2 576,000 1,152,000

Per diems for consultations and workshops lump sum 18,000

E. E

arly

Gra

de

Mat

hem

atic

al

Sk

ills

Ass

essm

ents

International TA lump sum 330,000

Adaptation workshop of trainers and MES/Education Evaluation Center on early grade mathematical skills assessments

lump sum 5,750

Local firm to conduct pilot and full-scale rollout of early grade mathematical skills assessment instrument

lump sum 437,200

Pilot teacher per diems and other training costs

lump sum 1,205,025

Monitoring of early grade mathematical skills assessments at schools

lump sum 124,400

F. N

atio

nal

as

sess

men

t fo

r 5t

h g

rad

e

International TA lump sum 330,000 Office equipment lump sum 1 24,100 24,100 Local firm to develop the instrument, pilot and administer the NA

lump sum 470,250

Per diems and other training costs lump sum 7,400 Contingencies lump sum 2,173,389

Component total 14,857,569

28

Component 2: Pre- and in-service professional development of teachers (estimated cost: US$3.3 million) A

. Su

pp

ort

pri

mar

y te

ach

er

trai

nin

g co

lleg

es

International TA for review of teaching practice (practicum programs) for teacher trainees and dissemination workshop

lump sum 171,550

Training to teacher trainers/professors at (MSUE) in charge of primary curriculum

Package 1 5,050 5,050

Printing of all new materials developed for teacher trainers

Each 500 15 7,500

teaching and learning equipment lump sum 27,500

B.

Tea

cher

tr

ain

ing Local firm to provide pedagogical training to

teachers lump sum 119,330

Per diems and operating costs lump sum 829,960

C. I

mp

rove

le

ader

ship

of

pri

nci

pal

s

International TA to develop training lump sum 87,000

Local firm to provide training to school principals to improve academic leadership

lump sum 21,760

Per diems and other training costs lump sum 52,700

D. T

ime

on

task

su

rvey

s International TA to provide oversight lump sum 330,000

Local firm to develop the survey instrument, pilot and administer survey in sampled schools

lump sum 899,800

E. D

isab

ilit

y su

rvey

International TA to work with local firm to develop trainings

lump sum 165,000

Local firm to pilot and implement the survey, and train experts

lump sum 58,265

Per diems and other operating costs for training and deployment of expert team

lump sum 70,875

Contingencies lump sum 449,122 Component total 3,295,412

Component 3: Implementation of a school support program (estimated cost: US$6.1 million)

A. O

rien

tati

on o

n s

choo

l su

ppo

rt s

chem

e

International expert on school financing to disseminate School Support Handbook and oversee training of school personnel

lump sum 330,000

Local firm to provide training to schools lump sum 18,020

Per diems and transport for school personnel lump sum 128,700

Printing of School Support Handbook Each 1,950 15 29,250

School Support Grant 1,750 3,000 5,250,000

Local firm to design and implement awareness campaign (including production of TV advertisements and posters)

lump sum 103,200

Monitoring and supervision of grants and grant activities

29

Diagnostic workshop lump sum 6,730 Audits Package 5 15,000 75,000 Monitoring and supervision lump sum 31,200

Contingencies 78,983 Component total 6,051,083

Component 4: System management, monitoring and evaluation (estimated cost: US$5.7 million)

A. P

roje

ct m

anag

emen

t

Chief Technical Advisor lump sum 1,245,000 National working group coordinators Early grade native language and mathematical skills assessments working group

person_month 60 3,800 228,000

Books working group person_month 60 3,800 228,000

Teachers pre- and in-service working group person_month 60 3,800 228,000

School Support working group person_month 60 3,800 228,000 EMIS working group person_month 60 3,800 228,000 NA for grade 5 working group person_month 60 3,800 228,000

Secretarial support to project Steering Committee

Office equipment Package 5 24,100 120,500 Vehicle Each 1 50,000 50,000 Operating costs Package 300 100 30,000

Technical assistance to project director

Procurement specialist lump sum 825,000 Financial management specialist lump sum 825,000 FM and procurement trainings to MES lump sum 11,600

B. P

roje

ct m

onit

orin

g an

d e

valu

atio

n International TA to support strengthening of

EMIS and integration of Early grade native language and mathematical skills assessments into EMIS

lump sum 165,000

Support equipment needs at MES to install new system

lump sum 1 24,100 24,100

Support training of relevant school staff members on new EMIS data collection

lump sum 290,606

DLI third-party verification person_month 4 30,000 120,000 Annual audit Package 5 15,000 75,000 Mid-term review and Implementation Completion Report preparation

Package 2 40,000 80,000

Contingencies lump sum 526,130

Component total 5,755,936

PROJECT TOTAL 30,000,000

30

Annex 3: Implementation Arrangements

MONGOLIA: EDUCATION QUALITY REFORM PROJECT Project Institutional and Implementation Arrangements 88. Overall responsibility for the project supporting the MES reform agenda will be vested with the MES. Project implementation will be mainstreamed using the MES structure at the central and decentralized (Aimag) levels supported by qualified technical assistance (TA) to be financed by the project. Education departments and relevant government officials at Aimag (province) and Soum (village) levels and schools will be responsible for the activities executed at the decentralized level over the five-year project period. 89. This project will follow the implementation structure below (Figure A3.1):

Figure A3.1 Proposed Organogram of Implementation Reporting Relationships

90. At the highest level, a Steering Committee (SC) will be established by the Minister, chaired by the MES’ State Secretary and comprising the heads and or designated representatives of: (i) the Strategic Policy and Planning Department; (ii) the Policy Implementation and

31

Coordination Department including the Pre-school and Primary Unit; (iii) the Monitoring and Evaluation and Internal Auditing Department; and (iv) the EEC. 91. The SC will: (i) approve annual activities and budget allocations; (ii) oversee the physical and financial implementation progress of the project and compliance with the Project Operation Manual (POM); (iii) ensure that policy decisions related to the project are well coordinated and harmonized with other programs of stakeholders, partners and donors in the sector; (iv) monitor and advise if the project’s development is embedded within the Ministry’s overall strategic program and budget; (v) commission and approved required accounts and reports; (vi) ensure agreed audit requirements are satisfied; and (vii) provide the overall guidance with the Project Director and review the work of all the project-related Working Groups. 92. The Minister of Education will designate a Project Director. The Project Director will be accountable to the Director of the Department of Strategic Policy and Planning. The Project Director, supported by a Chief Technical Advisor and Financial Management and Procurement Specialists that will be externally selected by MES following World Bank procedures, will:

(i) Provide the overall strategic direction for the project; (ii) Prepare the annual work and procurement plan for the consideration of the SC; (iii) Make day-to-day decisions to ensure the timely implementation of the approved

annual work and procurement plan attuned to the Financing Agreement and the POM;

(iv) Guide and coordinate the activity of the different Working Groups; (v) Ensure the required articulation of the various project working groups; (vi) Act as the secretary to the SC ensuring the undertaking of regular SC meetings,

preparing reports and documents to be presented to the SC, keeping a record of SC meetings minutes, and following on the implementation of agreed actions;

(vii) Prepare quarterly IFRs and biannual physical implementation progress reports; (viii)Liaise with other key project-related Ministries, like Economic Development and

Finance; and (ix) Ensure appropriate follow up on recommendations agreed during the Bank

supervision missions. 93. The Chief Technical Advisor will be externally selected by MES and support the Project Director along two dimensions. 94. The first dimension will be support for contract management and recruitment of technical advisors for implementation of the component activities of the project. On a day-to-day basis, the Advisor will:

(i) Enable project staff and relevant MES officials to understand and apply project goals and implementation commitments, including personnel and workplace considerations;

(ii) Assist in the implementation, monitoring and evaluation of project progress, including ensuring fluid communication and integration amongst various MES departments and Working Groups;

32

(iii) Assist in the preparation of all matters related to the implementation of the project, including recruitment, supervision and appraisal of project-financed local and international technical assistance; liaising with and provide reports on activities to the World Bank; and assuring compliance with Bank regulations and guidelines; and

(iv) Perform other duties as needed.

95. The second dimension of support will:

(i) Provide training and transfer of competencies to MES staff, particularly regarding Bank procedures and procurement mechanisms;

(ii) Support development by MES staff of networks of technical expertise that can be called on as needed;

(iii)Provide support to MES on managing donor and civil society interactions, including effective communications with the EDCM;

(iv) Provide training and guidance to MES staff for monitoring timely implementation of project components and achievement of DLIs;

(v) Support to Bank supervision missions and timely inputs to annual review meetings of the EDCM;

(vi) Organize and deliver additional training sessions to MES staff, or study visits, as needed and agreed with MES and the Bank TTL.

96. The Project Director will be also supported by Financial Management and a Procurement Specialist Consultants that will be externally selected by MES following Bank procedures. These consultants will be accountable to the Project Director. 97. The Financial Management specialist, working in close collaboration with the FID of the MES, will:

(i) Provide an oversight to the overall financial management functions of the project, including recording, reporting, budgeting, and banking and treasury activities;

(ii) Elaborate the project financial plans and budgets; (iii) Monitor the project budgets, expenditures and costs; (iv) Provide the required financial reporting as per the Financing Agreement and the

POM; (v) Prepare and submit to the FID withdrawal applications from the project

designated accounts; (vi) Maintain financial management policies and procedures including adequate

internal controls to ensure safeguarding of the projects assets and funds and ensure that these are adequately implemented;

(vii) Prepare periodic cash flow forecasts estimating the costs to be incurred in respect of each item set out in the approved project budget;

(viii)Ensure that all contracts and other commitments are promptly entered in a commitment control record so that the uncommitted/unspent balance on each activity and line item is readily known at any point of time;

33

(ix) Maintain records and process payments for all project transactions up to the end of the four months disbursement grace period following the project closure;

(x) Based on the assessment and advice of the achievement and verification of relevant DLIs by a third-party verifier, process the advised disbursement in close collaboration relevant Working Groups; and

(xi) Provide FM training as needed and ensure transfer of related competencies to MES staff.

98. The Procurement Specialist in close collaboration with the Procurement Unit of the GPA will:

(i) Coordinate all procurement under the project ensuring that the procurement is carried out in accordance with the Financing Agreement including by reference, the Procurement Guidelines, the Consultant Guidelines and the procurement plan agreed with the World Bank;

(ii) Establish and update regularly a project Procurement Plan in the format agreed with the Bank and ensure that the plan and its updates are disclosed to the public;

(iii) Ensure adequate publicity to business opportunities through the preparation and publication of the General Procurement Notice, Specific Procurement Notices and Requests for Expressions of Interest (REOI) in the format and timing acceptable to the Bank;

(iv) Prepare complete Bidding Documents (BD) for goods and services and Requests for Proposals (RFP) for consultants services based on applicable World Bank Standard BD and Standard RFP and in compliance with the provisions of the Guidelines and of the Financial Agreement;

(v) Establish and maintain a procurement tracking system and follow–up as required with the GPA and user departments to monitor progress throughout the procurement process for goods and services included in the project components, while ensuring compliance with the provisions of the Financial Agreement and the World Bank Guidelines;

(vi) Assist in the management of contracts under execution, including the preparation of documentation required to certify progress, conclusion or acceptance of goods and services;

(vii) Ensure the timely submission of procurement documentation, reports and procurement decisions to the Bank for no objection in all cases of mandatory prior review;

(viii) Establish and maintain complete and accurate procurement records including all actions and documents for review by the Bank's supervision missions, including advertisement, bidding documents, requests for proposals, invitation for bids, REOI, minutes of pre-bid meeting/pre-proposal submission conferences, requests for clarifications and responses provided, record of bid/proposal submissions, bids/proposals/quotations, bid/proposal opening, complaints and responses, bid/proposal evaluation reports, contract award and performance of the contracts;

(ix) Facilitate procurement post review by the Bank and/or procurement audits by independent Auditors; and

(x) Provide procurement training as needed and ensure transfer of related competencies to MES staff.

34

99. MES’ Policy Implementation and Coordination Department including its Pre-school and Primary Education Unit will be responsible for the implementation of Components 1 and 2, except the national assessment for grade 5, in close collaboration with the working group established for that purpose. 100. MES’ Strategic Policy and Planning Department will be responsible for the implementation of Component 3 in close collaboration with the working group established for that purpose. 101. MES’ Monitoring and Evaluation Department will be responsible for the implementation of the M&E activities of Component 4 in close collaboration with the working group established for that purpose. 102. The EEC will be responsible for the rolling out of the standardized national assessment for grade 5 in close collaboration with the working group established for that purpose. 103. Ad hoc Working Groups, comprised by existing officials of the participating Departments and Units within the MES, will be established and supported by appropriate technical assistance selected under term of reference satisfactory to the World Bank and recruited following Bank procedures. The Working Groups/TA will take the lead of carrying out the overall tasks necessary for planning, preparation, coordination, logistics, implementation, supervision and monitoring and evaluation of the component activities of the project. Specifically, the Working Groups will:

(i) Implement project activities as per the POM; (ii) Ensure appropriate coordination with the concerned MES line departments and

units, aimags, schools and other governmental agencies during the project’s implementation cycle;

(iii) Develop all technical inputs such as the terms of reference and technical specifications and preparation of all necessary procurement documentation as their pertain to their component;

(iv) Prepare annual activity-based work plan, procurement and financial plans, technical and operational reports to be presented for the Project Director;

(v) Monitor progress against DLIs; and (vi) Implement the Monitoring and Evaluation of the relevant component activities

against the indicators of the project results framework. Financial Management, Disbursements and Procurement

Financial Management 104. The project financial management risk is High and the mitigating measures below have been identified for lowering the FM residual risk to Substantial. 105. The FM capacity assessment identified the following principal risk associated with the implementation arrangements for the project: readiness of the Project Director’s office as well as

35

that of the project financial management systems. In order to help address these risks, a financial management section will be included in the POM as a tool for strengthening the project financial controls. Furthermore, once a qualified candidate is identified for the Financial Management Specialist’s position, the Bank FM staff will provide the necessary training to this person.

Significant weaknesses Actions Responsible

Person Completion

Date A Project Director’s office has not yet been established and staffed with a qualified Financial Management staff.

- Once identified, the Project Director should hire a qualified financial staff following the Bank’s consultant selection procedures.

MES Within one month of

effectiveness

Readiness of the project’s financial management systems, particularly the accounting and financial reporting arrangements

- An FM section of the POM has been prepared to lay out details of the project financial management procedures e.g. chart of accounts, including account description and use. - Design manual accounting registers to manually record project transactions until the accounting and financial reporting system is implemented. - Interim Financial Reports formats need to be developed and agreed by the Bank.

MES/Project FM specialist

Within one month of

effectiveness

Budgeting 106. The Project Director’s office will prepare a work plan and budget for the entire implementation period of the project and then on an annual basis with quarterly breakdowns, which will identify the detailed project activities. The annual work plan and budget will need to be submitted to the relevant project authorities, i.e., the Bank and the project SC, for approval. Moreover, the work plan and the budget will have to be consistent with the agreed format of the project IFRs. 107. The Project Director’s office is required to conduct regular budget vs. actual variance analysis within the computerized accounting system to be purchased, report them as part of the quarterly project IFRs, provide reasons why significant differences occur between the planned (budget) and actual expenses, if any, and take necessary actions. Accounting and Financial Reporting 108. Using computerized accounting software, the Project Director’s office will set up the project accounting and reporting in accordance with the Bank requirements, which obligate

36

Borrowers to prepare financial statements in accordance with acceptable accounting standards. Prior to the purchase of the accounting software, as an interim arrangement, the project will maintain manual accounting records using MS Excel. 109. The Bank does not mandate a particular format for annual financial statements. However, where a Borrower prepares financial statements on the modified cash basis, the Bank encourages the adoption of formats laid out in the International Public Sector Accounting Standards (IPSAS), and Financial Reporting under the Modified Cash Basis of Accounting, in order to monitor any non-cash transactions. The Project Director’s Office will adopt the modified cash basis of accounting for preparing financial statements. When doing so, it will account for advances made to the decentralized level of MES. The project financial statements will include the following:

Balance Sheet of the Project; Statement of Sources and Uses of Funds by Project Components; Statement of Implementation of Credit Proceeds; Statement of Designated Account for the Credit; and Notes to the financial statements

110. The Project Director’s Office will prepare IFRs on the quarterly basis directly from the accounting software where the project accounting records are maintained and submit them to the Bank for review within 45 days after the end of each calendar quarter. The Bank task team will monitor the project’s accounting and financial reporting processes, especially during the initial stages of project implementation, in order to ensure complete and accurate financial information is available to the relevant project stakeholders in a timely manner. Internal Control 111. To facilitate adequate internal control practices, a FM section of the POM that lays out procedures related to proper authorization for payment requests, segregation of duties, monitoring fulfilment of the DLIs and other internal control procedures and practices relating to financial management of the project will be prepared. The procedures identified in the FM section of the POM will have to be closely followed by all the parties involved in the project implementation. In addition, regular oversight by the project SC, periodic supervision missions by the Bank’s task team and annual financial audits by independent external auditors will serve as mechanisms for ensuring the project financial management systems function effectively. Audit Arrangements 112. The Bank requires the project financial statements to be audited in accordance with auditing standards acceptable to the Bank. Accordingly, same with other Bank-financed projects in Mongolia, the Mongolian National Audit Office will appoint an independent external auditor acceptable to the Bank to conduct an annual audit of the project accounts in accordance with International Standards on Auditing and under terms of reference satisfactory to the Bank. The annual project audits will be financed from the credit proceeds.

37

113. The auditors will: (i) express an opinion on the project financial statements; (ii) determine whether the Designated Account (DA) has: (i) been correctly accounted for, and (ii) been used in accordance with the legal agreement; and (iii) determine adequacy of the supporting documents and controls surrounding the use of Statement of Expenditures (SOEs) as the basis for disbursement. The auditors will also furnish a separate Management Letter, which will: (i) identify significant weaknesses in accounting and internal control as well as asset management; (ii) report on the degree of compliance with financial covenants of the FA, and (iii) communicate matters that have come to the attention of the auditors which might have a significant impact on the implementation of the project. 114. The annual audit report on the project financial statements will be due to the Bank within 6 months after the end of the reporting date. This requirement is stipulated in the FA. The responsible agency and timing are summarized as follows:

Audit Reports Submitted by Date Due Project Financial Statements MES June 30 of each calendar year

Disbursement and Funds Flow Arrangements 115. Four disbursement methods, advance, reimbursement, direct payment, and special commitment, will generally be available for the project. The primary methods of disbursement for the project will be advance and reimbursement. Supporting documents for Bank disbursements will be the WB Withdrawal Application, statements of expenditures (SOEs) or records like contracts and invoices. The detailed requirements will be laid out in the project disbursement letter to be issued by the Bank. 116. The Project Director’s Office will maintain and manage the DA in US$ at a commercial bank, on terms and conditions satisfactory to the Bank, including appropriate protection against set-off, seizure and attachments. The DA will be used for disbursing funds for eligible project expenditures upon appropriate authorization by MES and MOF. The ceiling for the DA will be discussed and agreed by the Bank and will also be specified in the Bank issued disbursement letter for the project. Accounts of Bank-financed projects may move into the Treasury Single Account System within the Government Financial Management Information System (GFMIS). If/when such a change takes place; the DA arrangements will be revised accordingly.

117. The project shall maintain ledger accounts to track eligible expenditures paid for each project component. Applications of reporting eligible expenditures paid from the DA and reimbursement for category 1 “expenditures with DLIs” and category 2 would be submitted separately. 118. In order to make payments for expenses that are of recurrent/operating nature (including goods, training, and consultants’ services) and incurred in the local currency, Mongolian Tugrik (MNT), MES can open a sub-account in MNT at the Treasury in accordance with the Treasury guidelines for opening and maintaining project accounts at the Treasury for the execution of activities related to Components 2 and 4. The ceiling of this account will have to be discussed

38

and agreed between the Bank and the MES and be specified in the Bank-issued disbursement letter.

119. Authorized signatories for the sub account will be designated officials from the MES/Project Director’s Office. Furthermore, the Project Director’s Office can utilize petty cash up to the ceiling to be established and approved under the project’s internal control procedures. The project Financial Management staff will be responsible for conducting regular reconciliation of the DAs, sub account and petty cash balances. 120. The project proceeds will be disbursed against eligible expenditures according to the following table:

Category Amount of the Financing Allocated (expressed in SDR)

Percentage of Expenditures to be Financed (inclusive of Taxes)

(1) Goods, non-consulting services, consultants’ services, training and incremental operating costs for Components 2 and 4 of the Project 5,900,000 100

(2) DLI Expenditures 13,600,000 100

TOTAL AMOUNT 19,500,000

121. The Project Director’s office will be responsible for preparing withdrawal applications. Withdrawal applications submitted to the Bank will be signed off by authorized representatives from MES and Ministry of Finance (MOF). These documents should be made available for review by auditors and Bank supervision missions. If the auditors or the Bank find any expenditure that is not justified by the supporting documentation; hence declare the expenditure ineligible, the Bank may take further actions necessary as per the related policies. 122. Disbursement-linked indicators. For Components 1 and 3 (Category 2), the EEs may be paid from the DA. Applications for reporting eligible expenditures paid from the DA would be submitted to the Bank, with support of: (i) statements of expenditure (SOE) and other records evidencing eligible expenditures as required; and (ii) evidence of verification of DLI achievement. The application will be accepted and the advance in the DA will be documented in the amount of the lesser of: (i) the reported eligible expenditures; or (ii) the amount allocated to the achieved DLIs in accordance with the FA. By the end of the project, any undocumented advances in DA are to be refunded to the Bank.

39

Procurement

123. Capacity Assessment. The procurement will be conducted by GPA in collaboration with MES. Major risks include: (i) staff of MES and the GPA are relatively new to procurement under a Bank-financed project; (ii) possible non-compliance arising from influence by domestic rules; (iii) weak planning and possible lack of coordination between MES and the GPA which could lead to delays in processing procurement; (iv) possible contract issues like suspension, termination, consultant's non-performance and over-pay to consultants during contract implementation. Mitigation measures include: (i) MES to hire/assign a Procurement Specialist with qualifications and under TOR acceptable to the Bank to coordinate procurement activities with the GPA; (ii) training to MES and GPA staff; (iii) early involvement by the Bank team in procurement; (iv) MES and the GPA to agree on standard processing times for procurement along with a detailed procurement plan with dates for key milestones including submission of TORs and Technical Specifications; and (v) the Bank team to closely monitor the progress and deliverables of consultants’ assignments during implementation. The overall procurement risk is rated Substantial. 124. Applicable Guidelines. Procurement will be carried out in accordance with the “Guidelines: Procurement of Goods, Works and Non-Consulting Services under IBRD Loans and IDA Credits & Grants by World Bank Borrowers” dated January 2011, “Guidelines: Selection and Employment of Consultants under IBRD Loans and IDA Credits & Grants by World Bank Borrowers” dated January 2011, and the provisions stipulated in the Financing Agreement.

125. Procurement of Works. Works procurement is not anticipated in the project.

126. Procurement of Goods. Goods procured under this project will include furniture, equipment, books and materials required for teaching and learning in the schools. Procurement will be conducted using the Bank’s Standard Bidding Documents for all ICB; bidding documents satisfactory to the Bank will be used for NCB and shopping. 127. Selection of Consultants. Consulting services will include various assignments with firms and individuals for improving learning outcomes for school children, professional development of teachers, monitoring and evaluation, and implementation services required for the project components to achieve the PDO. The Bank’s Standard Request for Proposals will be used for services costing more than US$100,000 or equivalent. Simplified Requests for Proposals will be used for services costing less than US$100,000 or equivalent. 128. School Support. Procurement under the school support will be conducted by the participating schools using very simple procurement methods described in the School Support Handbook. Procurement under the school support scheme will be subject to procurement post review by external auditors and/or by the Bank. 129. Training and Workshops. Plans for training and workshops will be developed by MES, and be included in project annual work plan for Bank review. Actual expenditures incurred in accordance with the agreed plans and cost ceilings will be disbursed on the basis of SOE.

40

130. Procurement Plan. A Procurement Plan for the whole project implementation period will be prepared by MES. It will be made available on the Bank’s external website. The procurement plan will set forth the thresholds for procurement methods and prior review. The Procurement Plan will be updated annually or as required to reflect implementation needs and improvements in institutional capacity. 131. Frequency of Procurement Supervision. In addition to prior review, the Bank mission will also carry out at least one procurement supervision annually. 132. Thresholds for Procurement Methods and Prior Review: The indicative thresholds are shown in the table below.

Thresholds for Procurement Methods and Prior Review

Expenditure Category Contract Value

Threshold (USD)

Procurement/Selection Method

Prior Review Threshold (USD) 1/

1. Goods and Non-Consulting Services

--

<300,000

<100,000

--

ICB NCB Shopping Direct Contracting

All First 2 NCB contracts; all contracts >=200,000 All

3. Consultants Services --

<300,000

--

--

--

QCBS/QBS CQS Individual Consultant Single-Source Selection (firm) Single-Source Selection (individual)

First contract; all contracts >=100,000 First contract, and all contracts >=100,000 Only in exceptional cases. All ≥20,000

1/ A contract whose cost estimate was below the Bank prior review threshold is subject to prior review if the price of the lowest evaluated responsive bid (or, in the case of consulting services, the financial offer of the selected firm) exceeds such threshold at the bid/proposal evaluation stage. TORs for all firm and individual consultants shall be prior reviewed by the Bank. 2/ The threshold for shortlisting comprising only national consultants: US$100,000.

41

Advance Contracting

133. The procurement plan shall set forth those contracts which will be procured in advance together with the relevant Bank review procedures. Environmental and Social (including safeguards) 134. There are no major social and environmental safeguards issues associated with the project, and no land acquisition, loss of lands and/or resettlement concerns are expected, since no civil works will be financed under the project outside the existing school or MES premises. The project will not allow the purchasing of new land. The project will ensure the use of vacant and idle land owned by the Government that is free from all encumbrances, habitation, dispute, claim or controversy. Thus the use of the land will neither require a resettlement plan nor a resettlement framework. The project’s Environmental Category is rated as “B”. 135. Environmental. The project will not finance civil works, but support capacity building activities for primary schools in Mongolia, including scaling up of the existing READ project (i.e., development and provision of books and classroom libraries, teacher training, and an impact evaluation), and Talent Program, or a school support program (i.e., management, staff development, services, teaching materials, sports, furniture and equipment, and maintenance). The project will have minimal or no adverse environmental impacts. The Environmental Assessment policy (OP 4.01) is triggered due to the need to assess and take into account, in an integrated manner, the social aspects of the project particularly any impacts on other vulnerable groups of people other than IP communities. 136. The project is not required to prepare an instrument under OP 4.01. Relevant social aspects of OP 4.01 are embedded in the project design such as, the project will attempt to address the disparities between rural and urban schools (and thus gender) by supporting activities that will specifically benefit rural schools and provide them with the opportunity to improve their educational services. In addition, parental perceptions of declining school quality in rural areas and that a high level of education is not necessary for herder boys may act as a disincentive for boys to remain in school. The project will attempt to address the disparities between rural and urban schools (and thus gender) by supporting activities that will specifically benefit rural schools and provide them with the opportunity to improve their educational services. 137. Social. While there are significant differences in achievement and enrollment levels between rural and urban students, there are no aggregate differences in enrollment rates between boys and girls within schools. Additionally, there is no aggregate difference in completion rates for boys and girls. However, in rural areas, male participation rates are lower than female rates, as boys who herd have responsibilities which keep them out of school. Parental perceptions of declining school quality in rural areas and that a high level of education is not necessary for herder boys may act as a disincentive for boys to remain in school. The project will attempt to address the disparities between rural and urban schools (and thus gender) by supporting activities that will specifically benefit rural schools and provide them with the opportunity to improve their educational services. Teachers who lack the resources and appropriate training to prepare lessons based on principles of literacy are direct project beneficiaries. Project activities take into account

42

specific constraints and needs faced by rural schools and have been designed under consultation with rural teachers, other development partners, and NGOs who have worked in rural schools. 138. OP/BP 4.10 on Indigenous People. This project aims to improve the quality of education for primary schools in Mongolia, and it is a nationwide project which will bring significant social benefits to local people. 139. Though largely an ethnically homogenous nation, with 80-86 percent of the population estimated to be of Khalkh Mongol origin, Mongolia is home to more than 10 ethnic minorities that have historically resided within current state boundaries. These ethnic minorities include Kazakh, Tuva, Dukha, Chantuu, Khoton and others. At the preparation stage of the project, the exact locations of project activities cannot be determined. 140. According to the Bank’s OP 4.10, an Indigenous People Planning Framework (IPPF) has been prepared by the client. The main purpose of the document is to: (i) ensure ethnic minority groups can benefit from the project; (ii) guide “free, prior, and informed consultation” with ethnic minority communities; and (iii) ensure issues raised by ethnic minority communities are incorporated into subcomponent designs. The IPPF was disclosed locally on February 20, 2014 and the English version of the IPPF was submitted and disclosed at the Infoshop on March 13, 2014. 141. OP/BP 4.12 on Involuntary Resettlement. The project will not finance civil works, but support capacity building activities for primary schools in Mongolia, including the scaling up of the existing READ project (i.e., development and provision of books and classroom libraries, teacher training, and an impact evaluation), and Talent Program (i.e., management, staff development, services, teaching materials, sports, furniture and equipment, and maintenance). Therefore, no involuntary resettlement is expected for the purpose of the project implementation. Monitoring & Evaluation 142. Strengthening monitoring and evaluation is at the heart of the project. The use of DLIs will aim to improve government reporting mechanisms on project implementation and impact. The project provides for the reporting of intermediate results indicators on an annual basis as per the results-based financing arrangement. Most reporting will be integrated into standard statistical and finance reporting requirements within MES, and some special reporting on school support program actions and deliverables will be required under the project, as defined in the POM. Annual third-party verification and ‘spot check’ verification during Bank supervision will also provide valid measures of progress in meeting the project’s development objective. Information on reporting of data related to DLIs can be found in Annex 6. 143. M&E will be carried out at different levels by different departments:

At the school level, the principal compiles data on student and school performance, and

report these data to the provincial or municipal departments of education. The provincial departments of education aggregate data from the schools, baghs (villages),

soums (districts), and aimag centers and report to the Department of Informatics,

43

Monitoring and Evaluation in MES. The UB municipality’s departments of education do the same.

The Department of M&E compiles these data and publish an annual report on education statistics of the country. MES tracks the progress of education through a set of monitoring indicators that are generally related to students, teachers and school facilities.

44

Annex 4: Operational Risk Assessment Framework (ORAF)

MONGOLIA: EDUCATION QUALITY REFORM PROJECT

Project Stakeholder Risks

Stakeholder Risk Rating Low

Description: The principal stakeholders are Government, other development partners, civil society organizations, parents, and teachers. Government ownership and commitment for READ and EFA-FTI was strong. The ownership and commitment for the GPE project remains strong. The scaling up of the Government reform agenda (the basis for the interventions in this project) is a high priority for MES. The recipient and Bank interests, objectives, and motivation for undertaking this project are well aligned. In previous education projects in the country, teachers, parents, NGOs, and development partners have expressed strong support for the interventions under this project. The risk of major donor disagreements over the Education Sector Plan and GPE remains low particularly with donors who are EDCM members. 10 donors and development partners are members of the EDCM.

Risk Management:

The project will include measures of outreach to parents and teachers during preparation including consultations on interventions. During implementation, the sharing of budget information and feedback surveys will increase the flow of transparent information.

Resp: Both

Stage: Preparation and Implementation

Recurrent:

X

Due Date: Frequency: Status: In Progress

Risk Management:

Donor coordination and communication and consultation mechanisms with the Government will be maintained to ensure ongoing sequenced implementation of the project through bi-monthly meetings of the EDCM. Active discussion and engagement between donors and government will be conducted at these meetings to identify the progress made on key sector reform agendas.

Resp: Client

Stage: Preparation and Implementation

Recurrent:

X

Due Date: Frequency: Status: In Progress

Implementing Agency (IA) Risks (including Fiduciary Risks)

Capacity Rating Substantial

Description: Risk Management:

MES has successfully implemented a number of Bank-financed projects in the education sector, including the recently closed READ and EFA-FTI projects, as well as the ongoing GPE project. Ministry officials are familiar with the interventions, as most have already been implemented by the government as part of its reform agenda.

This project does not finance any construction activities. Moreover, the READ methodology of service delivery has been successfully executed in the past and institutionalized. There is no reason to change that delivery system. Nevertheless, capacity building will be continued to address the fiduciary and project management risk issues. Training for Procurement, FM and auditing staff will be provided on an ongoing basis to help ensure necessary controls are

45

They are also familiar with Bank procedures and have received training on financial management, procurement, and also substantive sector issues. Based on past experience, a risk may be that MES and its technical teams may have insufficient in-house capacity to carry out the expected large volume of procurement activities, as well as having insufficient FM and auditing processes that meet acceptable standards to link financial information with the project’s physical progress. This situation has been present in previous projects in the sector that featured a large amount of construction. Moreover, there may be a risk that some procurement for education inputs (school equipment, learning materials, and textbooks) may be slow due to insufficient contract management or the limited number of suppliers.Additionally, the fact that this will be the first financing within the sector using DLIs, the fact that country systems are not strong, and the fact that project activities will be fully streamlined into MES departments for the first time all contribute to the elevated, but manageable, risk level. A further risk may be that in the event of MES failing to meet the DLI targets, some of the advance issued to MES may have to be repaid to IDA at the end of the project.

in place and that sound procurement and financial management practices are being implemented. In addition, International and/or National Individual Technical Consultants for these areas will be engaged by the project. The project will be audited annually, and the Bank will also closely supervise the project during implementation. Further a detailed POM will be developed to guide project implementation. This manual will provide MES staff with a comprehensive set of written policies and procedures for, among other things, routine tasks, administrative activities, records management, and documentation. The project will obtain written consent from MES for public release of pilot and main assessment results, and inform the public of this consent, prior to the commissioning of the field test. To mitigate the risk around failing to meet the DLI targets, the DLIs have been formulated with the following principles in mind: simplicity in number and scope to facilitate supervision and disbursement; alignment with the Results Framework to ensure a harmonized focus on outputs; traction to contribute to a steady implementation pace for the DLI-related components; and flexibility to ensure liquidity to operate. The DLI targets are designed to be feasibly achieved. The project’s strong emphasis on national and international TA will also help mitigate this risk.

Resp: Both

Stage: Preparation and Implementation

Recurrent:

X Due Date: Frequency: Status:

Governance Rating Moderate

Description: Risk Management:

There is strong ownership by the Government to the concept of the project, MES has driven the initiation of the project concept and its evolving design, and the project’s goals are fully aligned with and based upon the current sector strategy. MES has clarified its responsibilities and oversight processes through its Good Governance Framework (GGF). The GGF is an essential anti-corruption action plan to strengthen the governance capacity in projects in the portfolio. The GGF includes various elements for improving transparency and accountability through strengthened procurement arrangements, strengthened financial management, enhanced public disclosure, involvement of civil society, a complaints mechanism, a code of ethnic conduct, and

The Bank team will work closely with the MES team to ensure the satisfactory implementation of the GGF for the whole life of the project.

46

sanctions. With this in place, as well as the previously adopted Financial Management Manuals, and Operational Manuals for previous projects in the sector, MES has clearer lines of who will make decisions for the project. Previous Bank-financed projects in the sector in Mongolia have demonstrated that MES is structured in such a way that various levels of its management can take decisions related to their respective areas of expertise.

Resp: Both

Stage: Preparation and Implementation

Recurrent:

X Due Date: Frequency: Status:

In Progress

Project Risks

Design Rating Low

Description: Risk Management:

The design is a straightforward IPF, a modality with which MES is thoroughly familiar through previous IDA projects. Moreover, almost all of the interventions being considered for this project have been piloted in previous IDA projects in the sector, and this project builds on the foundation established by previously deployed interventions. Some interventions being considered, however, are new for MES, and there is a risk that some non-piloted interventions will not be punctually implemented. Previous experience from IDA projects has shown that some activities have not progressed on time (e.g., large scale of civil works that were non-piloted interventions). Moreover, the complexity of the project with multiple implementing departments within MES continues to represent a risk to the overall quality of delivery. Difficulty in achieving agreed results in a results-based operation could mean low or slow disbursement. Under results-based financing, inputs will have to be already purchased, installed or used and showing results before disbursement can take place. Another key risk to the project implementation may be that poor performance on the pilot test may cause the recipient to rethink the value of participation in assessment activities.

The implementing agency and the project team will work together with MES to provide additional implementation support in the early stage of implementation period with so as to avoid implementation delays of those activities. In addition, the fiduciary training provided to MES will anticipate these issues and possible solutions to them before they arise. Disbursement-linked indicators/key results will be developed in very close collaboration with key stakeholders (MES officials and staff, key MES implementing units) responsible for delivering on results, as well as other agencies/units responsible for resource allocation and provision and in implementation monitoring and oversight. The task team will ensure clear assignment of responsibilities and accountabilities of all key actors. Regarding assessment, a prior commitment from the Recipient and Implementing Agency for full implementation as well as informing the public from the outset, as a pre-condition of grant award about the purpose, the amount, the effectiveness date of the grant, the schedule of implementation, the results of pilot tests, and implementation results, may mitigate potential risks of non-implementation due to political considerations. A dedicated website could enhance further public accountability.

Resp: Bank

Stage: Preparation and Implementation

Recurrent:

X Due Date: Frequency: Status:

In Progress

Social and Environmental Rating Low

Description: Risk Management:

47

There is a risk that the most disadvantaged children, particularly those with disabilities and/or those from poor families, may not be reached or are not allowed to access to the education interventions.

The project will incorporate proven strategies based on successful experiences from other education projects for disadvantaged communities in Mongolia to find and encourage the participation of these groups. Additionally, stakeholder consultations about the project will occur frequently during preparation to raise awareness about the project’s interventions.

Resp: Client

Stage: Preparation and Implementation

Recurrent: Due Date: Frequency: Status:

Program and Donor Rating Low

Description: Risk Management:

The Education Donors’ Consultative Mechanism, which includes some 10 donors and civil society organizations, has agreed on a division of labor between development partners to reduce overlap and build coherence. They will not object to the project. Donor collaboration continues to be good. The project interventions are firmly in line with the MES reform agenda.

The team will continue to promote effective donor coordination through active dialogue with other donors as an ongoing agenda for the bi-monthly EDCM. The project will keep donors updated about the project through joint missions, frequent consultation, and meetings.

Resp: Bank

Stage: Preparation and Implementation

Recurrent:

Due Date: Frequency: Status: In Progress

Delivery Monitoring and Sustainability Rating Moderate

Description: Risk Management:

Previous experience from IDA projects in the sector have shown that contract management for MES may present a risk for timely implementation and delivery of interventions. Secondly, monitoring and evaluation using the new results-based monitoring framework is new for MES. Though the government has mainstreamed many of key project activities which have shown positive impacts on education outcomes, mainstreaming key activities of the project into the Program Budget may be a challenge.

Intensive supervision missions, frequent review of the implementation plans, and diligent following up with MES on the state of its communication and enforcement of penalties with contractors will help the implementing agency improve its contract management. Additionally, the supervision missions will afford the project with external procurement oversight. The Task Team will continue to work in partnership with concerned departments and DPs to improve the quality of results-based monitoring framework through participation in the MES-led results monitoring consultations and exercises. Additionally, component 4 of the project will help MES strengthen its monitoring framework for the project. For this project, only key activities for supporting achieving the MES reform agenda as interventions. Hence there will be low risk in sustainability of activities.

Resp: Client

Stage: Implementation

Recurrent:

X Due Date: Frequency: Status:

Not yet due

Overall Risk

Overall Risk Rating: Moderate

48

Annex 5: Implementation Support Plan

MONGOLIA: EDUCATION QUALITY REFORM PROJECT

A. Strategy and Approach for Implementation Support 144. This implementation support plan has been developed taking into account the risks identified and the agreed risk mitigation measures included in the ORAF (see Annex 4). Implementation support will focus on the capacity risks related to sectoral and technical aspects of the project, as well as those related to fiduciary concerns. In addition to close and frequent communication and follow-up, formal supervision missions will be conducted semi-annually. Additionally, supervision missions will afford the project external procurement oversight. The Implementation Support Plan (ISP) covers the following dimensions:

(a) Project Supervision Missions: Implementation support from the Bank will be provided through supervision missions conducted every six months. These missions will be conducted in close collaboration with MES and, to the extent possible, will be coordinated with the other donors including the local education group. At least one of these missions each year will include field visits. MES is responsible for preparing a detailed six-month implementation progress report to be shared with the Bank one month prior to the supervision mission. These reports will help identify key implementation issues and bottlenecks as well as the necessary corrective measures agreed between MES and the Bank to be discussed during the mission. (b) Sectoral and Technical Oversight: Knowledgeable Bank staff and technical consultants will provide advice and exchange views to ensure:

(i) Component 1 – Scale-up of the READ project and rigorous review, effective administration and use of results of early grade native language and mathematical skills assessments annually. (ii) Component 2 – Improvement in teacher quality and pedagogical practices, establishing the systems for data collection and classroom observation.

(iii) Component 3 – Administration of school support program. Experience and expertise is available within the Bank in terms of strategies to enhance accountability.

(iv) Component 4 – Set up of the monitoring and evaluation systems and handling project management smooth execution of project management in a way that generates capacity in the implementing agency.

(c) Financial Management oversight: This aspect will ensure adherence to the project implementation manual and provide training on proper financial management. Capacity building will be continued to address the fiduciary risk issues. In addition to formal training sessions for Procurement, FM and auditing staff, support and training will also be provided on an ongoing basis.

49

(d) External Audits: An independent external auditor will be engaged to conduct the audit of the entire project annually, DLI verification audits will be conducted annually, and school support audits will occur annually. These reports should be submitted to the Bank within 6 months of the end of each calendar year, and be shared with MES, MOF and the Bank, and would be posted on the project website as appropriate. (e) Procurement: Training on procurement procedures acceptable to the Bank will be provided to the implementing agency as well as schools which won the School Support. (f) Local Technical and Fiduciary Support: A local Bank education specialist will provide technical support for the annual planning and implementation of the project, and liaise with other education partners in the field. In consultations with the Task Team Leader (TTL), this specialist would also work with local Bank FM and Procurement staff to ensure project receives timely fiduciary support.

145. Bank Implementation Support Team: The composition of Bank’s team consists of a senior education specialist/TTL, local education specialist, operations officer, social development specialist, environmental specialist, financial management specialist and procurement specialist. Other staff members with specific expertise would also participate in the supervision missions at the request and/or needs of MES. B. Implementation Support Plan 146. The project has designed the following implementation support at different stages of the project:

Time Focus Skills Needed Resource

Estimate (per year)

First 12 months

Project launch workshop with the implementing agency and the stakeholders

Project Supervision Team

3 Staff Weeks

Technical training and supervision: review the POM prepared by MES; development and validation of early grade native language and mathematical skills assessments instruments, training for early grade native language and mathematical skills assessments, national rollout for grade 5 assessment, sub-project selection and management, etc.

Education Specialists

8 Staff Weeks

Financial Management training and supervision: Financial Management section of POM

Financial Management Specialist

4 Staff Weeks

50

Procurement training and supervision: Procurement section of POM and preparation of bid documents

Procurement Specialist

4 Staff Weeks

Environmental and Social Safeguards training and supervision: Review of Environmental/Social Assessment

Environmental and social safeguards specialists

2 Staff Weeks

Monitoring and evaluation training and supervision: M&E framework for the project developed and first semi-annual review completed

M&E and impact evaluation specialist

4 Staff Weeks

12-60 months

Mid-Term Review: Assessment of project progress completed

Education specialists, FM, Procurement and Safeguards, M&E, impact evaluation specialists

4 Staff Weeks

Project supervision Education specialists and operations officer

4 Staff Weeks

Procurement: Periodic reviews as required by Bank policy and as requested by the implementing agency

Procurement specialist

2 Staff Weeks

Financial Management: Annual Financial Statement review and auditing report review

Financial Management specialist

2 Staff Weeks

II. Skills Mix Required

Skills Needed Number of Staff Weeks

Number of Trips (per year)

Economist and Task Team Leader 7 2 Education Specialist 4 2 Operations Officer 3 1 FM specialist 5 1 Procurement Specialist 5 2 Social Development Specialist 2 2 Environmental Specialist 2 2 Monitoring and Evaluation Specialist 2 2 Impact Evaluation Specialist 4 1

51

Annex 6: Disbursement Linked Indicators and Verification Protocol

MONGOLIA: EDUCATION QUALITY REFORM PROJECT Criteria:

a) Simplicity – reduced number of DLIs, thus facilitating supervision and disbursement procedures b) Alignment with the RF and with the some key components c) Traction to contribute to a steady implementation pace for the most important (not all) components and sub-component (as per the amount allocated of the total Credit) d) Ensure liquidity to operate

Year 1 Year 2 Year 3 Year 4 Year 5 DLI 1 (US$ 14.9 million) Improved learning outcomes

Target: MES has prepared TORs and selected TAs for early grade native language assessment and development of reading books (“Books TA”)

Initial early grade native language testing has been completed and national baseline established MES has prepared TORs and selected TAs for early grade mathematical skills assessment

Initial analysis of early grade native language assessments results disseminated and discussed; Initial early grade mathematical skills assessment testing has been completed and national baseline established

Full set of new reading books available in 80% of schools in country

Early grade native language and mathematical skills assessments have become grades 1 and 2 teacher’s tools to assess progress in learning outcomes in reading and math in at least 50% of the primary schools in the country.

Allocated amount:

US$ 1.5 million

US$ 4.0 million US$ 4.5 million US$ 3.5 million

US$ 1.4 million

Disbursement rule: Y1: allocate US$1.5 million Y2: allocate US$4.0 contingent on verification of 100% compliance with Y1 DLI (early grade native language assessment TA and Books TA contracts have been signed); if selection of early grade native language assessment TA and/or Books TA is in process, allocate 50% of the US$4.0 million; Y3: allocate US$4.5 million if initial early grade native language assessment testing has been completed, national baseline established and

the early grade mathematical skills assessment TA contract is signed; if the selection of the early grade mathematical skills assessment TA is in process but contract is not yet signed, allocate 65% of the US$4.5 million;

Y4: allocate US$3.5 million if initial analysis of early grade native language assessment results

52

have been disseminated and discussed at a national forum and the initial early grade mathematical skills assessment testing has been completed and national baseline established; the initial early grade mathematical skills assessment testing is in process, allocate 50% of the US$3.5 million;

Y5: allocate US$1.4 million if the full set of new reading books are available in 80% of schools in country; allocate 50% of the US$1.4 million if the acquisition of new books is in process to the point of bid evaluation; allocate 30% of the US$1.4 million if the procurement process is about to start (bidding documents ready and not objected to by the World Bank).

DLI achievement measure: TORs for early grade native language and mathematical skills assessment TA no-objected by the Bank Contracts signed by the early grade native language and mathematical skills assessment consultant(s) Early grade native language and mathematical skills assessment baselines Sample of early grade native language and mathematical skills assessment findings (school reports) Evidence of implementation of corrective and/or preventive measures by grades 1 and 2 primary school teachers to improve learning outcomes in reading and math Bidding documents for the acquisition of new books no objected by the World Bank Procurement process for the acquisition of new books in progress New books in schools DLI verification process: Review of TORs and contracts Review of bidding documents; School early grade native language and mathematical skills assessment reports; On sight observations in a random sample of primary schools to observe (a) new books utilization; (b) administration of the early grade native language and mathematical skills assessment instruments; and (c) implementation by grades 1 and 2 teachers of corrective and/or preventive measures to improve learning outcomes in reading and math. EE: Technical assistance Development and validation of assessment instruments; Development and delivery of training modules for the utilization of assessment instruments; Administration of assessment instruments; Acquisition of books; Carrying out of training workshops; Carrying out of on-sight supervisions.

Year 1 Year 2 Year 3 Year 4 Year 5 DLI 2 (US$ 6.1 million) Implementation of a school support program

Target: MES has adopted a final School Support Handbook MES has prepared TORs and contracted

First batch of schools (200) are implementing their school proposal

Second batch of schools (450) are implementing their school proposal

Third, and last batch of schools (450) are implementing their school proposal

School support completion reports have been submitted to MES

53

TA to support the implement-ation of the school support program (“School Support TA”)

Allocated amount:

US$ 0.5 million

US$ 1.6 million

US$ 2.0 million

US$ 1.5 million

US$0.5 million

Disbursement rule: Y1: allocate US$0.5 million Y2: allocate US$1.6 million contingent on the official adoption by the MES of the School Support Handbook and

School Support TA contract has been signed; if selection of the School Support TA is in process, allocate 50% of the US$ 1.6 million;

Y3: allocate US$ 2.0 million if the first batch of schools are implementing their school proposal or are about to start the implementation of their approved school proposal; 50% of the US$ 2.0 million if the first batch are in the process of preparing their school proposal to be submitted for approval to MES; 20% of the US$ 2.0 million if the orientation on how to use the School Support Handbook has been or is being provided to the first batch of participating schools. Y4: allocate US$1.5 million if at least 10% of the first batch of school has satisfactorily completed the implementation of their school proposal and

if the second batch of schools are implementing their school proposal or are about to start the implementation of their approved school proposal and; if the second batch are in the process of preparing their school proposal to be submitted for approval, allocate 50% of the 1.5 million; if the orientation on how to use the School Support Handbook has been or is being provided to the second batch of participating schools, allocate 20% of the US$ 1.5 million.

Y5: allocate US$0.5 million if 100% of the school support completion reports required have been submitted to MES. DLI achievement measure: TORs for TA no-objected by the Bank Contracts signed by the consultant School Support Handbook adopted by the MES Orientation to participating schools completed/batch Evidence of implementation of school proposals School support completion reports/school

54

DLI verification process: Review of TORs and contracts On sight observations in a random sample of primary schools School completion reports

EE:

Technical assistance Drafting of a School Support Handbook Development and delivery of orientation modules Orientation workshops Implementation of the school financing reflected in the acquisition of goods, services and technical assistance as per approved school financing proposal On-sight supervisions.