Document of The World Bank

Report No.: 86027-NG

PROJECT PERFORMANCE ASSESSMENT REPORT

FEDERAL REPUBLIC OF NIGERIA

STATE EDUCATION SECTOR PROJECT (P096151)

April 16, 2014

IEG Public Sector Evaluation Independent Evaluation Group

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

Currency Equivalents (annual averages)

Currency Unit = Nigerian Naira

2006 US$1.00 NGN128.65 2007 US$1.00 NGN125.81 2008 US$1.00 NGN118.55 2009 US$1.00 NGN148.90 2010 US$1.00 NGN150.30 2011 US$1.00 NGN154.70 Abbreviations and Acronyms

ADB Asian Development Bank CCT Conditional Cash Transfer ECD Early Childhood Development CUBE Capacity for Universal Basic Education DfID Department for International Development EMIS Education Management Information System ESSPIN Education Sector Support Education Sector Support Programme in Nigeria HDI Human Development Index ICR Implementation Completion and Results Report IEG Independent Evaluation Group IEGPS IEG Public Sector Evaluation JICA Japan International Cooperation Agency JSS Junior Secondary School LGAs Local Government Authorities M&E Monitoring and Evaluation MDGs Millennium Development Goals PPAR Project Performance Assessment Report SESP Nigeria State Education Sector Project SUBEB State Universal Basic Education Boards UNICEF United Nations Children's Fund USAID U.S. Agency for International Development Fiscal Year

Government: January 1 – December 31

Director-General, Independent Evaluation : Ms. Caroline Heider Director, IEG Public Sector Evaluation : Mr. Emmanuel Jimenez Manager, IEG Public Sector Evaluation : Mr. Mark Sundberg Task Manager : Ms. Susan Ann Caceres

iii

Contents Principal Ratings ................................................................................................................. v Key Staff Responsible ......................................................................................................... v Preface ............................................................................................................................... vii Summary ............................................................................................................................ ix 1. Background and Context................................................................................................. 1

Socio-Economic Context of Nigeria ............................................................................... 1 Education in Nigeria ....................................................................................................... 2

2. Objectives, Design, and their Relevance ........................................................................ 5 Relevance of Objective ................................................................................................... 5 Design ............................................................................................................................. 6 Implementation Arrangements........................................................................................ 9 Monitoring and Evaluation Design ................................................................................. 9 Relevance of Design ..................................................................................................... 11

3. Implementation ............................................................................................................. 13 Implementation Experience ...................................................................................... 14

Safeguards ..................................................................................................................... 15 Fiduciary ....................................................................................................................... 16 Implementation of Monitoring and Evaluation ............................................................. 16

4. Achievement of the Objectives ..................................................................................... 17 Improve the Quality of Basic Education ....................................................................... 17 Improve Girls Participation ........................................................................................... 23

5. Efficiency ...................................................................................................................... 27 6. Ratings .......................................................................................................................... 28

Outcome ........................................................................................................................ 28 Bank Performance ......................................................................................................... 30 Quality of Supervision .................................................................................................. 31 Borrower Performance .................................................................................................. 32 Monitoring and Evaluation ........................................................................................... 33

7. Lessons .......................................................................................................................... 34 References ......................................................................................................................... 37

This report was prepared by Susan Ann Caceres, who assessed the project in May and June 2013. The report was peer reviewed by Xiaoyan Liang and panel reviewed by Denise Vaillancourt. Viktoriya Yevsyeyeva provided administrative support.

iv

Annex A. Basic Data Sheet ............................................................................................... 39 Annex B. School Conditions in Project LGAs in Kwara, Kaduna, and Kano 2009-2011 41

KWARA SCHOOL CONDITIONS ............................................................................. 41 KADUNA SCHOOL CONDITIONS ........................................................................... 42 KANO SCHOOL CONDITIONS................................................................................. 43

Annex C. List of Persons Met ........................................................................................... 53 Annex D. Borrower Comments ........................................................................................ 57 Tables

Table 1. Poverty Distribution across Geo-political Zones, 1999 ........................................ 2 Table 2. Gross Enrollment by Level by Year (%) .............................................................. 3 Table 3. Gross Primary and Secondary Enrollment by Geo-political Zone, 2006 (%) ...... 3 Table 4. Gross Enrollment by Level and Gender, 2006 ..................................................... 4 Table 5. Activities within each Component and Planned Costs ......................................... 7 Table 6. Project Results Framework ................................................................................. 10 Table 7. Appraisal and Actual Cost of Project by Component ......................................... 13 Table 8. Reported Use of Grant Funds (% of surveyed schools) ...................................... 18 Table 9. Primary Public Enrollment Total and Girls (number and percentage) 2009-2011........................................................................................................................................... 25 Table 10. JSS Public Enrollment, Total and Girls (number and percentage) 2009-2011 . 26 Table 11. Comparison of Project and Government Unit Costs (Naira) ............................ 28 Table 12. Primary School (2009-2011) ............................................................................ 41 Table 13. Junior Secondary Schools (2009-2011) ............................................................ 41 Table 14. Primary School (2009-2011) ............................................................................ 42 Table 15. Junior Secondary Schools (2009-2011) ............................................................ 42 Table 16. Primary School (2009-2011) ............................................................................ 43 Table 17. Junior Secondary Schools (2009-2011) ............................................................ 43 Table 18. Public Primary School Enrollment - Kaduna ................................................... 45 Table 19. Public Junior Secondary School Enrollment – Kaduna .................................... 46 Table 20. Public Primary School Enrollment - Kwara ..................................................... 46 Table 21. Public Junior Secondary School Enrollment - Kwara ...................................... 47 Table 22. Public Primary School Enrollment – Kano ....................................................... 48 Table 23. Public Junior Secondary School Enrollment – Kano ........................................ 49 Table 24. Percentage of Schools with no Toilets – Kwara ............................................... 50 Table 25. Percentage of Schools with no Toilets – Kaduna ............................................. 50 Table 26. Percentage of Schools with no Toilets – Kano ................................................. 51 Table 27. States in Reading Comprehension and Mathematics 2007 and 2011 (mean percentage score) .............................................................................................................. 51

v

Principal Ratings ICR* ICR Review* PPAR

Outcome Moderately Satisfactory Moderately Satisfactory Moderately Unsatisfactory Risk to Development Outcome

Moderate Moderate Significant

Bank Performance Moderately Satisfactory Moderately Satisfactory Moderately Unsatisfactory Borrower Performance Moderately Satisfactory Moderately Satisfactory Moderately Satisfactory

* The Implementation Completion and Results (ICR) report is a self-evaluation by the responsible Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. Key Staff Responsible

Project Task Manager/Leader Division Chief/ Sector Director Country Director

Appraisal Halil Dundar Laura Frigenti Hafez M.H. Ghanem Completion Olatunde Adetoyese Adekola Peter Nicolas Materu Marie Francoise Marie-Nelly

vi

IEG Mission: Improving World Bank Group development results through excellence in independent evaluation.

About this Report

The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons.

To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate.

Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public.

About the IEG Rating System for Public Sector Evaluations

IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org).

Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.

Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable.

Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.

Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.

vii

Preface This is a Project Performance Assessment Report of the State Education Sector Project in the Federal Republic of Nigeria, financed through IBRD Loan No. 42950, in the amount of US$65 million equivalent (SDR43.3 million). The project was approved on April 26, 2007, became effective on April 25, 2008, because of the time it took with Nigeria’s Federal processes approving and ratifying a World Bank project. It closed on July 1, 2011 without any extension.

This report was prepared by Susan Caceres, Senior Education Specialist, IEG. Data analyses of the Annual School Census Reports was provided by Ms. Segen Teklu Moges, under the supervision of Susan Caceres. The findings are largely based on a two-week mission to Nigeria from May 28, 2013 to June 7, 2013, conducted by Susan Caceres, Senior Education Specialist. During the same time period, Ms. Elena Bardasi, Senior Economist, also conducted a mission to examine the Community Based Poverty Reduction Project (P069086). The mission met with education authorities in the implementing agency, the State Ministry of Education and State Universal Basic Education Board in Kwara and interviewed representatives from the States of Kano and Kaduna. Representatives from the Federal Ministry of Education were also interviewed. The mission only visited schools in Kwara, as security conditions did not permit visits to Kaduna and Kano. During the mission in Kwara, a representative of the Federal Ministry of Finance accompanied the visits to schools and the interviews. The mission also met with key staff and managers of the Bank in Washington and Abuja, as well as key staff and managers of DfID. The list of persons met is in Annex C. Preparation for the mission and report involved examination of: (a) World Bank project files, (b) Project related reporting documents and evaluations, and (c) Education studies with data from the government and other development partners, as well as the relevant research literature.

The IEG team gratefully acknowledges the logistical assistance and support of the staff in the Abuja Office of the World Bank and the support from the Kwara State Ministry of Education personnel, particularly Mr. Rasaq Alabare.

Following standard IEG procedures, a copy of the draft report was sent to the relevant government officials and agencies for their review and feedback. Their comments are presented in Annex D.

IX

Summary

Context

In Nigeria national school enrollment data mask differences across the six geo-political zones. The North East and North West have respective primary gross enrollment rates of 67 percent and 64 percent, while rates in the North Central, South East, South West, and South South are 1 00 percent. There are also inequalities in enrollment in terms of gender. In the Northern Regions enrollment rates are the lowest in the country for girls. At the start of the project the gap in primary enrollment between boys and girls was 17 percent in Kano, while the difference between boys and girls enrollment in Kaduna was 11 percent and 3 percent in Kwara (World Bank 2007).

State Education Support Project

The objective of the US$65.40 million equivalent State Education Support Project (2008-2011), as stated in the Credit Agreement (p. 5) was "to improve the quality of basic education in targeted Local Government Areas (LGAs) in the participating States, focusing particularly on girls' participation", as measured by improved learning conditions 1, increased transition and completion rates for boys and girls in targeted LGAs. The same objective was noted in the appraisal documents.

The activities to attain the objective of improving the quality of basic education included: provision of school grants, textbooks, rehabilitation of targeted schools, teacher training, state and local government capacity building, reform of the inspection services, and an Early Childhood Development pilot. Design focused on improving inputs into learning, but neglected others that more needed to improve education quality and learning outcomes such as teachers and their unequal distribution among schools, particularly rural ones. Textbooks were printed in English, despite a national policy specifying a gradual transition to English as the medium of instruction during the primary grades (fourth to sixth). There were few targeted activities to increase the participation of girls. The conditional cash transfer pilot implemented in Kano after the midterm review, only supported the first grant payment to girls. The pilot continued with the support of ESSPIN and the Kano State Ministry of Education, but its future was uncertain2.

The relevance of objectives is high given the commitment, consistent with the Federal Ministry of Education's Ten Year Strategic Plan, to promote delivery of quality education and establishment of quality assurance procedures. The project's objective was also highly relevant to each State Education Sector Plan, which aimed to accelerate progress toward the key education Millennium Development Goals (MDGs), creating the basis for sustainable development by emphasizing improvements to quality in areas with low schooling participation levels, especially among girls. The Bank's current Country Partnership Strategy (20 10-2013, current at lEG Mission) stresses a state (rather than

1 This key performance indicator was subsequently changed to the measurement of student learning. 2 The Bank is supporting a conditional cash transfer program for poor households. This program is not targeted at girls.

X

federal-level) approach to improve access, quality, efficiency, and equity in education, with a particular emphasis on girls' education.

During the project, 3 million textbooks were distributed to primary and junior secondary schools within the three States, which achieved a 1: 1 student to textbook ratio in the core subject areas in Kwara and Kaduna, but their effective use has not been assessed. School renovations improved the physical conditions for learning and helped reduce overcrowding in classrooms. Professional development was provided to 6,456 teachers in the States of Kaduna, K wara, and Kario, but there were problems related to the relevance and impact of training, and many teachers were subsequently transferred to other schools, which head teachers reported was a barrier to sustaining pedagogical changes. The percentage of girls enrolled in primary schools in Kano increased slightly more in project LGAs (47 percent in 2009 to 50 percent in 2011) than in non-project LGAs (47 percent to 48 percent in 2011). The rate remained steady in primary schools in Kwara and Kaduna in both project and non-project LGAs.

The project's outcome is rated Moderately Unsatisfactory, and is based upon high relevance of the objectives, modest relevance of design, modest achievement in improving quality, and modest achievement of increasing girls' participation. Efficiency was rated substantial because of the cost savings the project demonstrated from its use of competitive bidding for civil works and textbooks. The risk to development outcome is significant, given weak institutional management capacity, teacher posting policies, and changing government priorities. The performance of the Bank is rated moderately unsatisfactory at entry, during supervision, and overall. The Borrower's performance is rated moderately satisfactory. The Government and States remained committed to improving education and increasing girls' participation, but there were some shortcomings in performance. Both Government and implementing agency performance were moderately satisfactory.

Lessons

Based on the experience of this project, several lessons can be drawn:

• Learning and sustainability of pilot activities are limited when there is an absence of clear design, an explicit mechanism for evaluating, and a sufficient implementation time frame. Initiating the girls CCT during the last year of the project provided insufficient time to implement the pilot. As well, it is difficult to determine the benefits and effectiveness of the pilot ECD activities, given the absence of the monitoring and evaluation system and implementation plan. Without strong design and evaluation, there is no way to identify whether the pilot activity is worthy of scaling-up or sustaining. Pilots generate further interest in sustaining or scaling-up when robust data are collected.

• Setting up a monitoring and evaluation system takes more time than Governments and TTLs often recogniz;e and provide. An overly complex system was designed and replaced with a simplified one. In environments with low capacity, understanding the critical data needs of the Government is essential, as well as the amount of technical assistance that will be required. During this

xi

project, it took considerable time for the s ·tates to collect and analyze the Annual School Census.

• A prior unsuccessful project can result in learning, but it depends upon an accurate understanding of performance issues. This project avoided some of the shortcomings in the predecessor Bank project, Universal Basic Education Project. DfiD's TAwas aligned with the objective of this project. However, each partner worked separately to complete respective responsibilities within the project. Better coordination between the work of the Bank and DfiD could have enhanced the EMIS, state and local capacity building efforts and teacher professional development.

• Even with a State-focused project, the involvement of the Federal Ministry of Education is critical given its central role in policy. The difficulties related to the Federal implementation in the Universal Basic Education project were not fully understood. During preparation, instead of designing approaches to overcome the challenges experienced by the Federal Ministry of Education with the Universal Basic Education Project, the Federal role was minimal. This reduced the buy-in by the Federal Ministry of Education and likelihood of replicating project activities in other states.

• Improving inputs into learning (facilities, textbooks, etc.) needs to be accompanied by defining and addressing quality requirements in terms of minimum standards for student learning. While this project improved the inputs and physical conditions for learning, these may not translate into improved student learning, as the key issue of teachers' capacities was insufficiently addressed by the project. When teachers' numeracy, literacy, and language skills are low, children's learning will be compromised. This is particularly true in rural areas as more skill deficiencies were found among rural teachers. There also continue to be issues in how teachers are allocated amongst schools.

6~~~ M,·~~ -fv~

Caroline Heider Director-General

Evaluation

1. Background and Context 1.1 This Project Performance Assessment Report (PPAR) reviews the experience and achievements of World Bank support for basic education in the Nigeria State Education Sector Project (SESP). It was approved on April 26, 2007, and became effective on April 25, 2008, because of the time it took with the Federal processes for approving and ratifying a World Bank project. The project was financed by an International Development Association (IDA) credit of US$65.40 million equivalent. The British Department for International Development provided US$1.67 million in parallel financing for the components related to Education Management Information System (EMIS), Reform of the Inspectorate, and Capacity Building for Planning and Management.

1.2 This project was selected for a field-based assessment since it focused on improving the quality of education, increasing the participation of girls, and also implemented pilot activities to support early childhood development. The information gained from this field-based study will be utilized as an input into IEG’s upcoming evaluation of Bank support for early childhood development and nutrition programs.

Socio-Economic Context of Nigeria

1.3 In the period 2001-2009 Nigeria experienced rapid economic growth. Growth has averaged 9.2 percent annually, then slowed in the later years to 6.4 percent (World Bank 2011). Despite this robust growth, there has not been significant improvement in social indicators. In the midst of dynamic economic growth, the poverty headcount ratio increased from 63.07 percent in 2004 to 67.98 percent in 2010. In 2011 Nigeria’s Human Development Index, HDI, was 0.459, ranking 156 out of a total of 187 countries, sharing that position with Senegal, and just one position above Haiti (0. 454), both countries with a much lower per capita gross domestic product. Nigeria’s 2011 HDI revealed that life expectancy at birth, mean years of schooling, and expected years of schooling are lower than those of other Sub Saharan African countries. The levels of stunting and wasting in Nigeria are 41% and 26.7% respectively, which are higher than those of other Sub Saharan African countries such as Gambia, Uganda, Togo, and Cote d’Ivoire.

1.4 Across the six geo-political zones, there are sharp socioeconomic disparities. The southern region has lower levels of poverty than the more populated Northern region (See Table 1). The north western and north eastern zones have the highest children malnutrition rates of 35 and 34 percent respectively, while the south east and south south regions have the lowest rates of 10 percent and 12 percent respectively (World Bank 2011). The three States covered by this project are Kwara, Kano, and Kaduna. Kwara is part of the north central Zone, while Kano and Kaduna are in the north west.

2

Table 1. Poverty Distribution across Geo-political Zones, 1999

Zone Non-poor Moderately Poor Core Poor North East 29.9 35.7 34.4 North West 22.8 39.9 37.3 Central 35.4 36.7 28.0 South East 46.5 35.3 18.2 South West 39.1 33.4 27.5 South South 41.8 34.8 23.4 Source: Federal Office of Statistics 1999 in World Bank 2003. Education in Nigeria

1.5 The education system includes primary (six years), junior secondary (three years), senior secondary (three years), and tertiary education (various years with at least four years for university). Basic education is composed of early childhood care and development, primary, and junior secondary education. State Universal Basic Education Boards (SUBEB) and the State Ministry of Education work with Local Government Authorities (LGAs) to manage and provide basic education. States also have responsibility for secondary education. Tertiary education is managed and financed by both the Federal and State governments. However, more than three-quarters of students attend federal institutions. Federal policies have not been uniformly implemented across the States. At all levels, the Federal government is responsible for policy setting and monitoring of education standards.

1.6 In 2004 the Federal Government introduced the Universal Basic Education law, which has been ratified by the States. The intent of this legislation was to provide nine years of free compulsory basic education (six years for primary and three years at junior secondary)3. The law established the Universal Basic Education Commission which oversees the allocation of the guaranteed funding of not less than two percent of the Consolidated Revenue Fund to the States.

1.7 Most of the funding for basic education is from the local governments' allocation from the Federation Account. The Federal government is not responsible for basic education, but provides financial support through the Education Trust Fund, Virtual Poverty Fund, and Universal Basic Education Commission. Because there are differences between States and local governments in their efforts to fund education, the total share of education expenditure varies across States (World Bank 2003). National enrollment figures show an increase and then decline in primary education, while secondary is steadily growing (See Table 2). Some respondents interviewed for this report speculated that the decline may be related to unreliability in data, as well as the discontinuation of the school feeding program. The gross enrollment rate for pre-primary has remained around 14 percent; however, private providers

3 There are efforts to add one year of pre-primary.

3

accounted for more than 80 percent of preschool enrollment (World Bank 2008). Official data show that gross primary enrollment reached 100 percent in 2005, but declined to 83 percent in 2010. Secondary enrollment has steadily increased. Data are viewed by many as unreliable.

Table 2. Gross Enrollment by Level by Year (%)

Level 2000 2005 2010 Pre-primary na 14 14 Primary 98 102 83 Secondary 24 34 44 Tertiary na 10 na Source: WDI. 1.8 National enrollment data mask differences across geo-political zones. As Table 3 shows, primary enrollment was over 100 percent in the North Central, South East, South West, and South South. In comparison, the North East and North West had respective primary enrollment rates of 67 percent and 64 percent. In 2006 the state of Kano had an even lower rate for primary education (48 percent), while Kwara (80 percent) and Kaduna (66 percent) were higher. The Junior Secondary School enrollment rate was 42 percent in Kaduna, 48 percent in Kwara, and 27 percent in Kano. Thus, at the beginning of the project, Kwara had the highest enrollment rates in primary and junior secondary, while Kano had the lowest.

Table 3. Gross Primary and Secondary Enrollment by Geo-political Zone, 2006 (%)

Zone Primary Secondary North East 67 45 North West 64 41 North Central 114 81 South East 124 95 South West 116 96 South South 114 99 Source: World Bank 2007 data from household survey (CWIQ). 1.10 There are inequalities in enrollment in terms of gender. Across the country, the gap in enrollment between boys and girls is largest at the postsecondary (10.7), with primary (7.6) and junior secondary levels (6.7) having slightly smaller gap (See Table 4). At the senior secondary level, the difference is the smallest (4.3), since many boys enter the labor market and drop out. In the Northern Regions enrollment rates are the lowest in the country for girls. In Kano the gap in primary enrollment between boys and girls was 17 percent, while the difference between boys and girls enrollment in Kaduna was 11 percent and 3 percent in Kwara (World Bank 2007). At the junior secondary level, Kano also had the largest enrollment gap between boys and girls (23 percent), while the difference between boys and girls in junior secondary enrollment was 13 percentage points in Kaduna and

4

Kwara. Thus, increasing girls’ enrollment in the Northern geo-political zones was key for Nigeria to meet the Millennium Development Goals.

Table 4. Gross Enrollment by Level and Gender, 2006

Level Female Male Difference Primary 88.5 96.1 7.6 Junior Secondary 67.9 74.6 6.7 Senior Secondary 65.2 69.5 4.3 Postsecondary 22.1 32.8 10.7 Source: World Bank 2008.

1.11 Private schools comprise a low percentage of total enrollment at the primary (13 percent for male and 11 percent for female) and secondary levels (10 percent of male and 13 percent of female) (World Bank 2008). However, private schools are increasing in the south where there is demand and ability to pay for education. There are a large number of non-formal religious schools in the north, but data is not available, since these schools fall outside the monitoring of the state government.

1.12 Data and monitoring capacity are challenges in the education sector. While educational data are available, it is not viewed as reliable. There is also the need to strengthen the data from schools. Monitoring and evaluation capacity is low, according to respondents. Inadequate monitoring from the inspectorate services in the State Ministry of Education is another limitation.

1.13 There are inadequate conditions for learning. There are shortage of core textbooks and teaching and learning materials. Before the project, approximately three students had to share core textbooks in Kwara, Kaduna and Kano (World Bank 2007). The physical conditions of learning is poor, since there is overcrowding in primary schools in Kaduna, Kano, and Kwara and insufficient number of classrooms to meet the demand to achieve universal enrollment. Buildings are based on outdated construction standards. Schools are poorly maintained. Across States there are large differences in the deployment of teachers with the teacher-student ratio ranging from 1:36 in Kwara, 1:88 in Kaduna and 1:100 in Kano (Abe, 2011).

1.14 Improvements are needed in the teaching-learning process. Teachers utilize few methods to develop literacy skills. Many teachers also do not have a good command of the English language and so they are unable to explain lessons and effectively utilize English textbooks. Teachers’ rely on chalk and talk techniques and do not understand how to engage students actively in learning. Students do not spend enough time on productive learning activities (Adekola 2007). Teachers do not have tools and techniques to monitor students’ progress. Primary school teachers do not regularly plan their lessons or prepare instructional materials for use in their lesson (Adekola 2007).

1.15 Learning assessment data in primary grades have shown low student achievement. In 2003 the Universal Basic Education Commission carried out a criterion referenced test in the subject areas of English, mathematics, science, and social studies, for a sample of students in

5

grades four, five, and six. Students in fourth grade answered 25-50 percent of the questions correctly. Fifth grade scores ranged from 25-39 percent across the four subjects, while scores of sixth graders were 21-40 percent (World Bank 2008). In cross-national studies Nigerian children also did worse in comparison to other African countries (Adekola 2007). Reasons for this relate to students’ limited fluency in English and basic knowledge of the concepts (Adekola 2007).

2. Objectives, Design, and their Relevance 2.1 According to the Financing Agreement (World Bank 2007 p. 5) and the Project Appraisal Document (World Bank 2007 p. 9), the objective of the project was "to improve the quality of basic education in targeted Local Government Areas (LGAs) in the participating States, focusing particularly on girls’ participation." The project operated in the States of Kano, Kwara, and Kaduna. While the objectives remained the same during the course of the project, one of the key performance indicators was changed during a project restructuring.

2.2 The project selected the three States based on their demonstrated commitment to the education sector, quality of Sector Plan, and poor educational indicators; all three were northern States, but not those with the highest level of poverty, or the lowest education indicators. Within the States, Local Government Areas were targeted (e.g. six in Kaduna, nine in Kano, and six in Kwara) using criteria such as poverty, readiness and willingness of key officials to implement reforms, enrollment rate in basic education, per capita public spending on primary and secondary education, and absence of other externally funded projects. Gender disparity was another criterion used to select the 21 Local Government Areas from which 1,523 schools were selected.

Relevance of Objective

2.3 Relevance of Objective is rated High. Nigeria was significantly behind its goal to achieve universal basic education, with low primary and junior secondary enrollment rates in the Northern areas, especially among girls. In Kano the gross junior secondary enrollment rate was 56 percent for boys and 31 percent for girls (World Bank 2007). In Kaduna the gross junior secondary enrollment rate was 74 percent for boys and 61 percent for girls (World Bank 2007). In Kwara the gross junior secondary enrollment rate was 82 percent for boys and 69 percent for girls (World Bank 2007). Moreover, improving the quality of education was a priority within each state. Thus, the focus on girls’ participation and improving quality was highly relevant for each of the States.

2.4 The Country Partnership Strategy (2010-2013) at project closure stressed a state (rather than federal-level) approach to improve access, quality, efficiency, and equality in education, with a particular emphasis on girls’ education. The Country Partnership Strategy also promotes the use of school-based management committees as a means to improve governance in the education sector to ensure proper use of resources, and hold teachers and schools accountable for good quality education. This is consistent with the project’s use of school grants and school management committees to improve quality and girls’ participation.

6

2.5 The project's objective is also highly relevant to each State’s Education Sector Plan, which aim to accelerate progress toward the key education Millennium Development Goals (MDGs), creating the basis for sustainable development by emphasizing improvements to quality in areas with low schooling participation levels, especially among girls. It also aligns with the Federal Ministry of Education Ten Year Strategic Plan to promote delivery of quality education, as well as the establishment of quality assurance procedures. The strategy affirms that “every Nigerian shall have a right to equal education opportunity irrespective of gender, social status, age, religion, ethnic background, geographical locations and any peculiar challenges”.

Design

2.6 Components. The project had four components. Table 5 summarizes the main activities of each component. The first component provided school grants. The second component focused on teacher training, textbooks, and school rehabilitation, expansion, and upgrading. The third component strengthened aspects of the State Ministry of Education such as school inspection services, data collection and monitoring, and developed an Education Management Information System. The fourth component supported project implementation, monitoring, and an information strategy to communicate the benefits of girls’ education. The subsequent section provides an overview of the design.

7

Table 5. Activities within each Component and Planned Costs

Component 1: School Development Scheme (US$19.48 million equivalent)

Component 2: Quality Improvement in Basic Education in Targeted LGAs (US$32.42 million equivalent)

Component 3: Institutional Development for Key Functions at State Ministries of Education and Local Government Education Authorities (US$4.98 million equivalent)

Component 4: Project Management and Monitoring and Evaluation (US$2.50 million equivalent)

1a. Schools in targeted LGAs receive grants to improve the quality of teaching and learning and mange resources.

2a. Professional development activities

to benefit 18,900 teachers to implement

the basic education curriculum.

3a. Strengthening the EMIS at the state level to plan, manage, and monitor the education

sector more effectively.

4a. Support for project implementation and

information and communication strategy

highlighting the benefits of education

for girls. 2b. Textbook,

instructional materials, and equipment to attain

agreed upon ratios in core subjects.

3b. Strengthening management planning,

budgeting, financial management, and

establishing a mechanism to publicize

educational data.

4b. Establishment of a robust M&E system at

the state level.

2c. Expansion, rehabilitation, and upgrading of basic school facilities in

targeted LGAs.

3c. Reform of inspection services and training for inspectors

and supervisors.

Source: World Bank 2007. 2.7 The project focused on the basic inputs as the means to improve quality. It provided school grants, in-service training to teachers, learning materials/textbooks to classrooms, and infrastructure enhancements. School committees were provided training to develop school plans and manage the school grants, which were intended to be used for investments to improve education quality (as well as improve girls’ participation). The involvement of the community was to create transparency in the use of the funds. The teacher training related to learner-centered activities. Professional development included: training for teachers, follow-up mentoring visits, peer assisted learning, as well as leadership support for head teachers, school principals, and school supervisors.

2.8 Another strategy the project employed to improve education quality was enhancing the functioning of each State’s school inspection services and building State and LGA capacity. A Basic Education Inspectorate was to be developed to improve accountability for the delivery of education services. The process was to ensure the appropriate linkages between the federal and state level reforms and the rationalization of units currently

8

providing inspectorate services. New inspection instruments were to focus on education quality and learning, rather than the physical infrastructure in the school.

2.9 The project focused on building state-level capacity to collect better data and ensure that budgetary allocations were properly directed towards activities and recurrent costs. State Sector Plans and Operational Plans were to be developed and used to improve budgeting and financial management and teacher recruitment/ deployment. Given the unreliability of data, the project focused on collecting and disseminating data, as well as developing an Education Information Management System for State Ministry of Education and State Universal Basic Education Board. These activities were intended to improve strategic planning and budgeting.

2.10 By improving quality, it was also thought that more parents would send their children (and daughters) to school. Thus, the design of the project utilized a whole school approach to improve quality and girls’ participation and integrated a focus on girls within the main activities. Respondents told the IEG Mission that improving quality was an appropriate strategy for Nigerian schools, since access to learning was what was needed.

2.11 To address attitudes towards girls' enrollment the project employed sensitization and training for selected school committee members. It was thought that with the training, they would then identify areas within their school plans and resources from the school grant scheme to encourage girls' participation. However, changing cultural attitudes towards girls' enrollment would likely need additional interventions to foster behavioral changes among men, the decision-makers within the families, as well as additional advocacy efforts beyond the project's information and awareness strategy may have also been needed.

2.12 Schools were encouraged to spend at least 35 percent of the grant annually on activities to benefit girls (World Bank 2007). Interventions that schools could finance with their grants and outlined in the manual were: school uniforms, after-school activities, and vocational training. Given the lack of decision-making of women and their constrained status, it seems optimistic to expect school-management committees would prioritize resources for girls’ participation, considering the dilapidated school conditions that were noted in the Aide Memoires during project preparation.

2.13 After the midterm review, two activities were added to the project from the unallocated funds and savings from exchange rate fluctuation, which combined amounted to US$ 20 million. A pilot project was implemented in the state of Kano to improve the participation of girls through a Conditional Cash Transfer program. Since this activity was added and needed to be timed with the start of the school year it began in September 2010, which permitted funding only one round of grants4 to support the transition of girls from primary to junior secondary school during the project. A local information dissemination campaign of the Conditional Cash Transfer (CCT) program was included so that the community would know about the program and enroll their daughters in it. Second, early

4 The second round of funding did not get to the Implementing Agency’s account before the closing date of the project.

9

childhood education5 activities were implemented in each state, which supported both project objectives of improving girls’ participation and education quality. While project documents report an Early Childhood pilot, the implementation was not designed to test the effectiveness of an approach or strategy that could later scaled-up, if warranted. However, each State developed a list of activities and corresponding budgets. Activities included: (1) providing additional grant money in schools that offered early childhood services, (2) examining pre-service training institutes within the States to identify capacities and constraints, (3) developing state plans for early childhood education, and (4) study tours.

Implementation Arrangements

2.14 The project was implemented by the State Ministry of Education in the States of Kaduna, Kano, and Kwara in close coordination with the state parastatal organization (SUBEB) and federal parastatal organization, Universal Basic Education Commission (UBEC), as well as the 21 participating Local Government Agencies. The State Ministry of Education oversaw all activities under the project, with support from a project implementation unit that assisted with the coordination and fiduciary management. The unit was filled with staff on secondment from the State Ministry of Education. The project implementation unit collaborated with each State technical department to monitor project implementation and ensured smooth disbursement of funds. A Consultative Steering Committee was planned to share knowledge about the project at the State and Federal level.

Monitoring and Evaluation Design

2.15 The Project Appraisal Document provided a detailed plan for monitoring and evaluation. It specified anticipated uses for the project’s data including empowering communities to better manage their schools, informing policy, planning and management of basic education, ensuring effective quality and supervision and public accountability. There was a clear timeline for data collection and analysis, and explicit assignment of responsibility for various types and levels of data collection. At the time of preparation, baseline enrollment data was not available by LGA, and the State figures were reported to be unreliable.

2.16 The Results Framework specified a mix of outcome and output indicators to measure attainment of the objectives (See Table 6). Key performance indicators were (i) improved learning conditions from 2007 baseline, (ii) increased primary and junior secondary completion rates (disaggregated by gender) and (iii) increased transition rate from primary to junior secondary (disaggregated by gender). During restructuring, improved learning conditions was replaced with a new key performance indicator, the measurement of student learning. Output indicators included: number of school implementing school development plans, number of teachers trained, pupil-to-textbook ratio, teacher-to-textbook ratio, pupil-to-classroom ratio, number of schools with upgraded facilities, annual education statistics report and sector analysis, school report card, quality assurance inspection in schools, and 5 Early childhood education interventions “provide opportunities for children to interact with responsive adults and actively learn with peers to prepare for primary school entry; generally refers to interventions for children age 36-83 months of age” (World Bank forthcoming).

10

establishment and implementation of a results-based monitoring and evaluation system (See Relevance of Design for analysis of weakness in the Results Framework).

Table 6. Project Results Framework

Quality Girls’ Participation Outcomes Improve learning conditions in project

schools based on 2007 baseline benchmarks.

Later changed to the measurement of

reading and mathematics learning.

Increased primary completion rates for boys and girls in targeted LGAs.

Increased junior secondary completion rates for boys and girls in targeted LGAs.

Increased transition rates from primary to junior secondary for girls and boys in targeted LGAs.

Intermediate Outcomes Outputs Number of target schools implementing

approved School Development Plans. Number of head teachers trained that

demonstrate enhanced school management and leadership skills.

Number of teachers trained. Pupils have access to and use of key

core subject textbooks on a pupil to book ratio of 3:1 in primary and 4:1 in JSS in target schools.

Teacher to book ratio of 1:1 for teacher guides and workbooks in target schools.

Pupil to classroom ratio reduced in primary and JSS targeted schools.

Increase number of schools with newly upgraded learning facilities.

Annual Education Statistics Report produced by EMIS and disseminated to stakeholders for planning and monitoring purposes.

School report cards produced and disseminated annually for school planning and monitoring.

Education sector analysis produced by 2008 and leading to improved strategic planning and budgeting.

All primary and junior secondary schools have quality assurance inspection resulting in Annual Basic Education Report.

Results-based Monitoring and Evaluation system established and implemented continuously.

Annual Education Statistics Report produced by EMIS and disseminated to stakeholders for planning and monitoring purposes.

School report cards produced and disseminated annually for school planning and monitoring.

Source: Author adapted from World Bank 2007.

11

2.17 An impact evaluation was designed for the pilot conditional cash transfer program to provide evidence of the results, and monitor fidelity of the implementation. The study was designed to provide information related to effectiveness of various operational mechanisms, cost-effectiveness, and measure whether the poorest were reached. Since two groups of students (e.g. students with junior secondary schools in surrounding area and those without) would be evaluated, ( the evaluation was designed to test whether the CCT could counteract school supply or distance constraints to secondary education. The study was financed by DfID’s Education Sector Support Programme in Nigeria (ESSPIN) and Bank provided technical assistance (See Achievement of Objectives, para 4.35, for preliminary evaluation findings).

2.18 Project documents indicate plans for an impact evaluation of the Early Childhood Development (ECD) pilot, but project records provide no evidence of its design. No impact evaluation was conducted of the ECD pilot activities; however there was low feasibility of completing an impact evaluation with what was implemented. Indicators could have been developed, but none were.

Relevance of Design

2.19 Relevance of design is modest. The project's objectives, components, sub-components, and outcome measures were linked logically, but there were weaknesses.

2.20 The project design (See Components) provided a cohesive and consistent framework of common technical guidelines, activities, and monitoring, but there were appropriate allowances for variations across States in implementation of the activities. The variations were based on the needs identified in each State’s education Sector Plan. For example, there were differences across the States in length of training time, use of different service providers, and provision of follow-up support. This was consistent with the Federalized responsibility for education, as well as consistent with the Education Sector Support Plan developed by each State.

2.21 There were a number of design shortcomings, especially in relation to the underlying results chain. There were no targeted activities to increase girls’ participation, with the exception of a planned information education campaign to highlight the benefits of girls’ education. While the project appraisal document noted that culture and poverty were the main reason why girls did not participate in school, there were no demand-side interventions. As the project was ending, a pilot conditional cash transfer was implemented in one state to address poverty constraints to girls' enrollment. The conditional cash transfer program was targeted to girls (fifth, sixth, and JSS1) from the poorest families in an effort to encourage continued participation into junior secondary school and counter early marriage, which was common reason for girls dropping out of school. The highest rate of girls' primary school drop-out occurs in sixth grade, according to the Annual School Census.

2.22 Design focused on improving inputs into learning (i.e. grants, textbooks, renovations etc.), and neglected others that would be needed to improve education quality and learning outcomes such as teachers and their unequal distribution among schools, particularly rural ones. Inservice training would not be sufficient to address teachers’ literacy, numeracy, and

12

English language skills, as many of them did not have sixth grade levels when tested. The project did not produce materials for students or teachers in languages other than English. There was no implementation plan for the early childhood pilot and it as well as the girls conditional cash transfer pilot were implemented too late in the project’s life (World Bank 2012). Grants were to be allocated to improving quality and girls’ participation, which did not match local priorities.

2.23 Important project activities were to be supported by DfID’s Capacity for Universal Basic Education (CUBE) and specified in the project appraisal document, yet CUBE was scheduled to end in July 2008, around the time the project was just effective. Specific roles included: capacity building for LGEAs, and States Ministries of Education, design and development of the teacher professional support program and materials, provision of professional architectural and engineering expertise, and support to States related to EMIS, reform of inspectorate and capacity development for management and planning. While DfID was expected to continue working with the Bank in Nigeria beyond 2008 (and has continued working in Nigeria), the scope of DfID’s follow-up project would not have been known at the time of preparation. Technical assistance was important, given the lack of State capacity. The complexity of the task and the time it would take was underestimated. Integrating another development partner’s project which was slated to end during the early stage of implementation meant that the States had to utilize IDA resources to cover aspect’s not originally planned, as an agreement for a more limited scope of work was reached between ESSPIN and the Bank.

2.24 The design provided a very limited role for the Federal Ministry of Education in the implementation of the project. A Consultative Forum was established (no funding was allocated, but the appraisal document stated that DfID’s CUBE would subsidize expenses for workshops). Given the Federal Ministry’s responsibility to coordinate policy across the States, a role in the project may have resulted in improved synergies between the States and Federal involvement in the project. Respondents indicated that this was a poor decision and subsequent Bank projects have defined a role for the Federal Ministry of Education.

2.25 Several of the key performance indicators were not defined and thus making them open to interpretation and difficult to measure. For example, the intent of “improved learning conditions from baseline survey” and “measurement of student learning” were not clear. Was the original indicator aimed at changes in physical infrastructure in a classroom or school, changes in how teachers taught, or changes in student learning? Was the goal of the revised indicator to establish a system of student performance, measure the learning at one point in time from a sample of students, or improve the learning of students? The lack of precision in the wording of both of versions of indicators creates challenges in determining the attainment of the objective (improved quality of basic education). These indicators do not define the minimum conditions for learning.

2.26 There were also shortcomings with the key performance indicators to measure girls’ participation. Some of these issues related to data limitations, but also due to the choice of

13

indicators. Considering the unreliability in the Nigerian Population Census 2006 6, enrollment rates (e.g. overall and by gender) were logically not selected as a key performance indicator. Completion and transition rates were to provide measures of girls staying longer in school. However, the Annual School Census did not report all the information needed to calculate these rates by LGA (e.g. repeaters by LGA and the total number of public enrolled children of official graduation age). Thus, the primary completion rate can reliably only be calculated on a state basis, making the comparison between project LGAs and non-project not feasible. Other proxies to measure the increasing duration of girls in school were not selected. The other key performance indicator, transition rate between primary and secondary, was sensitive to space constraints in junior secondary schools, which was beyond the scope of the project’s interventions.

2.27 Several of the intermediate outcome indicators were also poorly defined making them difficult to measure (i.e. Results based monitoring and evaluation system established and implemented continuously, primary and junior secondary schools have quality assurance inspection resulting in Annual Basic Education Report, education sector analysis produced by 2008 leading to improved strategic planning and budgeting, number of head teachers and principals trained that demonstrate enhanced school management and leadership skills).

3. Implementation 3.1 The project was approved on April 26, 2007, became effective on April 25, 2008, and closed on July 1, 2011. Despite the delay in effectiveness due to the revisions in the Federal Government’s process for approving World Bank projects, there were no extensions. During preparation and prior to project effectiveness, training in monitoring and evaluation, procurement and financial management, and sensitization workshops were financed by DfID for officials in the 21 participating Local Government Agencies. During this time, Terms of References, Request for Proposals, and bidding documents were also prepared.

Table 7. Appraisal and Actual Cost of Project by Component

Component Appraisal Estimate Actual Cost Percentage of Appraisal School Development Scheme 19.48 19.09 98 Quality Improvement 32.42 38.41 118 Institutional Development for State and LGAs 4.98 3.65 73 Project management, monitoring and evaluation 2.5 4.25 170 Total Baseline Cost 59.38 Physical Contingencies 6.82 Price Contingencies 2.31

Total Project Cost 68.51 65.40 95 Source: World Bank 2012.

6 In Kano and Kaduna, population census of primary age children reported fewer than the number of children enrolled.

14

3.2 Throughout the implementation, the project was rated satisfactory for implementation progress and progress on the development objective in the Implementation Status Reports. Disbursement took place at a pace close to the planned schedule. Due to exchange rate fluctuations, some of the revised credit went undisbursed; however, 95 percent of the original Credit, US$65.4 million disbursed. As Table 7 shows, the components where spending was higher were Quality Improvement and Project Management (World Bank 2012).

IMPLEMENTATION EXPERIENCE

3.3 The predecessor Bank project, Universal Basic Education Project, closed two years early at the request of the Government with cancellation of 35 percent of the credit. While the state activities performed better than the federal ones, the project had several implementation challenges. The technical assistance provided by DfID was not coordinated with the project objective and there was the need for more supervision of the TA by the Bank (IEG 2008). There were fundamental flaws in the design of the project, and weak readiness for implementation (IEG 2008). After that experience, all parties reported the desire to learn from the mistakes and avoid them again.

3.4 During preparation, technical assistance was provided by DfID to the States Ministries of Education, as well as LGAs and SUBEB. This support directly addressed key aspects of implementation (i.e., procurement, monitoring and evaluation, development of sector plans). CUBE consultants worked with each state team. For example, the criteria to select schools for infrastructure enhancements were developed, lists of schools receiving grants and civil works were identified, as well as Terms of Reference for the teacher professional development providers, among other things. Because of DfID’s assistance through CUBE, each state was ready to implement and was able to accelerate the procurement schedule, despite the delay in effectiveness.

3.5 Each State Ministry of Education initiated its own parallel civil works projects to renovate and build additional classrooms. In the State of Kwara SUBEB constructed 534 new primary classrooms and 140 new classrooms in junior secondary schools, as well as renovated 65 primary classrooms and 17 classrooms in junior secondary schools between 2008 and 2011, according to Ministry data. In addition, the Kwara State Ministry of Education renovated 40 classrooms in Grammar Schools and 8 classrooms in junior secondary schools. The number of additional classrooms created outside the project in Kaduna and Kano could not be confirmed.

3.6 There were delays in establishing the State Education Information Management System. The system was to be a tool to monitor progress of the project and outcomes of the State Education Sector Plan. The system was expected to track data on teacher training, school enrollment, and school facilities, but by mid-term review was not functioning in any of the States. A system was developed, but it was never operationalized because of its complexity. There was resistance from the Federal Ministry of Education, which left the States unsure how to proceed with the development of the management information systems and collect school census data. As a result, for much of the project the States were not able to report project outcome data except intermediate indicators, until the States published the

15

2009/2010 school census. This was the only annual school census that was published during project implementation, resulting in limited data to assess attainment of objectives.

3.7 While a Consultative Steering Committee was planned, as a means to share knowledge about the project at the State and Federal level, this Committee was not established until the final year of the project. Only one meeting was held. As a result, there was limited involvement in the project from the Federal Ministry of Education.

3.8 There were political factors in Kwara which assisted the State in making further implementation advances. Leaders in the state of Kwara made personnel changes in the SUBEB and the LGA and supported the redefinition and restructuring of organization roles and responsibilities. In Kwara a needs assessment of teachers was conducted, and reform of the preservice teacher education colleges was done. This facilitated further changes in Kwara, not experienced in the other States, which may be one explanation why project implementation and results were noted in supervision reports to have been more advanced.

3.9 There were several other donors that were involved in education in Nigeria, including: DfID, USAID, ADB, JICA, and UNICEF. The Bank partnered with DfID which provided parallel financing for capacity development, EMIS, and reform of the inspectorate. DfID and the Bank largely worked separately related to their areas of responsibility, but there was a joint midterm review and joint supervision missions. While JICA and USAID worked within similar States, the Bank did not coordinate its activities with them, instead advised the State teams to engage these donors with their education sector support plans. After mid-term review, when the project implemented activities related to ECD, the Bank partnered with UNICEF and USAID related to this work to complement efforts and avoid duplication.

3.10 The mid-term review occurred in November 2009, at which time about 70 percent of the project funds had been disbursed or committed, according to project records. At the review, plans for the new activities (ECD and CCT) were discussed. Other topics noted at the meeting were the Consultative Steering Committee, Education Management Information System, and implementation progress.

Safeguards

3.11 The project was classified as category B for Environmental Assessment (OP 4.01) due to enhancement/rehabilitation activities in schools in the States of Kaduna, Kwara, and Kano. The government disclosed the Environmental and Social Management Framework (ESMF) on January 10, 2007, prior to appraisal. This document contained a screening mechanism to identify adverse impacts from proposed construction activities (World Bank 2008). The implementing agency in each state monitored the Framework. Supervisory firms were hired to ensure contractors complied with environmental protections. Satisfactory ratings for safeguard compliance were noted in Implementation Supervision Reports throughout the project and no issues were noted in project records or reported. By the close of the project each implementing agency submitted to the Bank an Environmental Report detailing its safeguard compliance. At the time of the IEG Mission, there were three unfinished civil works in the State of Kwara. The completion status of all civil works activities was reported by Kwara to the Bank before the close of the project. The IEG

16

Mission did not visit any of the three sites. The safety impact to students from the unfinished buildings in these three schools cannot be determined.

Fiduciary

3.12 Project ratings for fiduciary compliance were satisfactory. Acceptable financial arrangements were established in Kaduna and Kwara, but it took longer to establish them in Kano. Financial statements from each state were submitted on time and were unqualified, but interim financial reports were delayed. There were no outstanding financial reports.

3.13 Procurement compliance was rated satisfactory, but there were a few shortcomings. There were issues in the quality of tender documents, execution of bid documents and contracts, and maintenance of proper documentation. It took longer to receive No Objections from the Bank because incomplete documents were submitted. The Bank provided training for the officers in the Project Implementation Unit. The Bank addressed procurement issues with the Implementing Agency via dialogue and monthly meetings. Respondents in Kwara noted that once they became familiar with the Bank’s procedures, they were able to easily follow its guidelines even though this was the first Bank project that they had implemented.

Implementation of Monitoring and Evaluation

3.14 The Project Implementation Unit was responsible for monitoring project performance indicators. Baseline data were updated in 2010, since the data contained in the appraisal document were unreliable and based on state-wide numbers. The Results Framework was not updated to provide measures in relation to the activities added after the midterm review (ECD and CCT). However, an impact evaluation was implemented to assess the effectiveness of the Conditional Cash Transfer Program. Preliminary findings were shared with IEG, but not the final report (See Achievement of Objectives, para 4.35).

3.15 Given the delay in establishing the State-level EMIS (partly related to delays in Federal EMIS) and the delay for States to produce and analyze the data, project outcome data were not available until the end of project implementation. However, intermediate indicators were tracked throughout the project. ESSPIN supported school censuses in each project state.

3.16 Consultants conducted evaluations of the main project activities. These included: School Development Scheme Assessment, Learning Assessment (pre and post), and Teacher Professional Development Evaluation. These reports provided descriptive and qualitative data to assist in making determinations of the effectiveness of the interventions.

3.17 Ratings in Supervision Reports for M&E were satisfactory and moderately satisfactory over the course of the project, but one was rated moderately unsatisfactory. The low rating was given after delay in updating baseline data and questions on the reliability of what was submitted. A consultant was hired to analyze and review collected data from the school census survey. By the end of the project, there were improvements in M&E capacity at the State level, as well as improvements in the quality of data.

17

4. Achievement of the Objectives Improve the Quality of Basic Education

4.1 The objective of improving the quality of basic education was modestly achieved.

4.2 There were seven activities that the project supported to improve quality: provision of school grants, textbooks, rehabilitation of targeted schools, teacher training, state and local government capacity building, reform of the inspection services, and top-up grants for pre-primary classrooms. The evidence of the implementation of these activities and their impact is presented below.

Outputs 4.3 Schools grants. School grants were provided and utilized by schools, which tended to use them primarily for construction and repair and not for originally intended activities aimed at improving quality and student learning. Over the course of two years (2009-2011), school grants were awarded to 1,974 schools, with 499 of them receiving two years of funding. Grants in one year averaged N597,127 (US$ 3,852) per school, which represented approximately N41,154 per student (US$265) (Balarabe, Zakariaya, & Garba 2011). Schools7 developed the plans according to the guidelines and each school established a committee (Balarabe, Zakariaya, & Garba 2011). Schools welcomed the opportunity to manage their own resources. Based on reports to the IEG Mission, community participation increased and the resources were a catalyst for leveraging addition funding for complementary investments in some schools, which is consistent with finding of grant programs in other settings (Krishnaratne, White, Carpenter 2013). Minimal cases of grant misuse were reported in project records. School management committees enhanced the accountability of the resources to the community; however, female participation in the committees was much lower than male. Since the closure of the project, ESSPIN has found it necessary to create women’s committees as a way to facilitate the participation of women in decision-making.

4.4 After the first round of grants to 499 schools, the Bank requested that States ensure that schools use the money to improve quality and student learning; however committees primarily used the grants for construction and repair. Additional sensitization training for school committees was conducted so that they identified interventions to support learning. However, construction continued to be prevalent, rather than learning materials (See Table 8). The most common type of general renovation was replacement and repair of windows and doors, construction and repair of latrines, and flooring and patching of walls and floors (State of Kwara Ministry of Education Science and Technology 2009). The school grants helped to improve the adequacy and supply of facilities (Abe 2011). They also improve the dilapidated conditions with the rehabilitation and painting of classrooms and construction of toilets, as reported by respondents.

7 849 schools in 9 LGAs in Kano, 500 schools in 6 LGAs in Kaduna, and 627 schools in 6 LGAs in Kwara.

18

Table 8. Reported Use of Grant Funds (% of surveyed schools)

Activity Kano Kwara Classroom renovation 62 20 Headmaster office, staff room, dormitory 27 5 Furniture for students or teachers 47 49 Toilet or borehole 14 36 Renovation to ceiling, floor, roof, or painting 26 51 Library 3 0.8 Science or computer lab 3 5 Source: Balarabe, Zakariaya, & Garba 2011. 4.5 Committee members reported that there was a conflict between the grant guidelines on how to spend the money and the actual needs of the school (Balarabe, Zakariaya, & Garba 2011). They felt that schools had pressing renovation needs, while the guidelines indicated that no more than 40 percent should be spent on renovation, provision of water and sanitation. Survey respondents reported that the grants contributed to improving quality, since the classrooms became more conducive for teaching and learning (Balarabe, Zakariaya, & Garba 2011). Similarly, they reported that parents were more inclined to send their children to the schools after the repairs. If schools do not have adequate protection, students can lose valuable instructional time due to leaking roofs (White 2004). For this reason, community members indicated to the IEG Mission that the conditions of the school were of primary importance and improved the environment where their children learned.

4.6 School Renovations. School renovations improved the physical conditions for learning and helped reduce overcrowding in classrooms. 98 schools were provided with upgraded learning facilities, which improved the physical conditions for learning. These schools were selected during preparation, based upon a needs assessment. This work prioritized the building of classrooms, laboratories, libraries and toilets, aspects to support learning through a whole-school approach. It also selected schools with serious overcrowding issues. By the end of the project, there was construction or upgrading of 640 classrooms, and installation of 720 toilets and 75 boreholes. Given the poor conditions that were reported in the appraisal document and the fundamental necessity of ensuring adequate learning environment, these investments were needed. Deprivation of school resources affects the educational attainment, as well as future returns to education (Case & Yogo 1999). School inputs such as adequate sitting and writing space for children, education materials such as textbooks, and other learning resources contribute to student learning (Glewwe, Hanushek, Humpage & Ravina 2011). In schools visited by the IEG mission, construction appeared to be of adequate quality with no evident deficiencies. However, in Kwara all civil works activities were not completed by the close of the project, as three projects were still unfinished at the time of the IEG mission.

4.7 While classroom overcrowding was reduced within each state, this is largely attributable to SUBEB, and less so to the project. Annual School Census shows a decrease in the classroom to pupil ratios in 2011 (i.e., average 59:1 for Kaduna, 87:1 for Kano, and in

19

Kwara 34:1 ) from the start of the project as noted in the appraisal document (i.e., 93:1 for Kaduna, 192:1 for Kano, and 48:1 in Kwara). The scale of new classrooms constructed by this project was much smaller than what SUBEB conducted on its own. For example, in Kwara 135 new classrooms were constructed by this project. While SUBEB in Kwara constructed 534 new primary classrooms and 140 new classrooms in junior secondary schools, according to State Ministry of Education data.

4.8 Quality improvements in the conditions of schools and classrooms were noted by respondents, which are likely attributed to SUBEB and the project. Kwara Annual School Census Data overall demonstrate a trend in project LGAs in terms of decreasing percentages of classrooms needing major repair, having insufficient seating, and containing no source of water (See Anne B). For example, the percentage of classrooms in primary schools in need of major repair declined in 5 out of 6 project LGAs in Kwara and 4 out of 6 project LGAs in Kaduna. The percentage of schools without a source for water declined from 2009-2011 in primary schools in two out of six project LGAs in Kwara and four out of six project LGAs in Kaduna. Unfortunately, data from the Annual School Census Reports in Kano showed no consistent pattern, suggesting possible inaccuracies in its collection. (See Annex B)

4.9 Provision of Textbooks. The provision of textbooks under the project enabled the achievement of 1:1 student-to-textbook and teacher-to-textbook ratios in the targeted states, but their effective use has not been assessed. 3 million textbooks were distributed to primary and junior secondary schools within the three States, which achieved a 1:1 ratio in the core subject areas in Kwara and Kaduna. This was a significant improvement from the baseline in Kwara, where the ratio was 3.3:1 in 2006. Since no baseline was established in Kano and Kaduna the extent of improvement is not determinable. Within each of the three States, the primary teacher-to-textbook ratio improved from 5:1 in 2006 to 1:1 in 2011, so that now all teachers have curriculum guides and core subject textbooks. Learning materials help students follow along and support teachers in their lessons. There are also positive impacts in mathematic scores, when learning materials are provided (Krisnaratne, White, Carpenter 2013). However, no data were collected during the project to assess the effective use of the textbooks by teachers and students, despite the fact that this was noted in the appraisal document. During the IEG mission to schools in Kwara, which were selected by the Ministry of Education8, textbooks were available in the classrooms. During approximately 5 out of 8 observations in primary and JSS classrooms, students were using the textbooks during their learning activities. It was beyond the scope of this mission to analyze the content of the textbooks.

4.10 While national policy specifies a gradual transition to English as the medium of instruction during the primary grades four through six, it has not been implemented in the States. English has become the default language of instruction at all levels of primary school (World Bank 2007), even though there are benefits to teaching children in a language they understand. During IEG mission, English language instruction was observed in all pre-primary and primary classrooms, while two JSS classrooms utilized some mother tongue. Lack of textbooks in languages other than English is one of the reasons the mother-tongue 8 IEG requested to visit schools that had civil works, but the specific schools were selected by the State Ministry in Kwara.

20

policy has not been implemented. Local language textbooks were not evident during IEG’s visits to schools and classroom. Yet, the project printed textbooks in English, which was a decision of the State Ministry textbook committee.

4.11 Another challenge to implementing the mother-tongue policy is the difficulty in finding teachers able to instruct in all of the main Nigerian languages, as a classrooms can be filled with students speaking more than one language. There are three dominant languages: Yoruba, Hausa, and Igbo.

4.12 On the other hand, a consequence of having teachers instruct in English, is the difficulty recruiting teachers (particularly in rural areas) with adequate English language competencies. Given that children may only be exposed to English during school, they need an appropriate English speaking and writing model, which may be lacking particularly in rural schools. In the classrooms visited by the IEG Mission in urban and rural schools (which were selected by the State Ministry of Education), teachers had adequate command of English; however, teachers’ spelling errors could be seen in a few classrooms.

4.13 Teacher Inservice Training. Teacher inservice training targets were exceeded, albeit with some issues related to the quality and impact of the training. The relevance of the training and the large number of trained teachers transferred out of project schools indicated issues requiring further follow-up. Professional development was provided to 6,456 teachers in the States of Kaduna, Kwara, and Kano, which is more than what was planned (5,808). Inservice addressed topics such as: teaching methods to promote student-centered learning, use of assessment methods to monitor learners’ progress, and content within the areas of science, social studies, mathematics, and English. However, the predominant topic was student-centered learning, based on content analysis of the training modules. The modules could have benefited from more emphasis on subject area content. The materials developed by the project were only written in English.

4.14 The training was followed up by mentoring visits and teacher meetings to give additional opportunities to share experiences and support, which is needed for teachers to be able to apply what was learned (Garet and others 2001). As well, the mentors provided assistance to other teachers in the school who did not attend the training, which was a way of furthering the impact of the training.

4.15 Project records and the evaluation of the training program pointed to issues with the training. Head teachers had limited involvement in the selection of teachers. Other concerns about the training were that more men than women participated. At the initial stage of the project, there was conflicting information between what was presented in workshops and quality assurance supervisors’ advice, suggesting a lack of coherence between the training for teachers and inspectors. Respondents believed that many of these issues were addressed during the project.

4.16 After the first round of training, some changes were made to the program and its delivery. Bank staff reported that the training was adapted to accommodate teachers with lower levels of education. As well, the state of Kwara conducted an assessment of teachers to identify their needs in relation to teaching literacy and numeracy. This was used to

21

identify effective teachers for future appointment as lead teachers instead of appointment by political patronage. Even with the changes made to the professional development program, teachers who were surveyed perceived the training to have limited relevance (Abe 2011)9.

4.17 In basic instructional skills such as setting the purpose of the lesson, ensuring the participation of children in the activity and reinforcing was what learned, there was minimal improvement from baseline (Abe 2011). However, the same teachers were not assessed from baseline. No data were collected to demonstrate that trained teachers were effectively using the textbooks and guides, despite being noted in the appraisal document. During the IEG visit to schools selected by the Kwara State Ministry, teachers applied learner-centered techniques. In several of the observations, teachers