Document of The World Bankdocuments.worldbank.org/curated/.../pdf/...PUBLIC.pdf(Exchange Rate...

Document of The World Bank

Report No: ICR00003094

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-77200 TF-94998)

ON A

LOAN

IN THE AMOUNT OF US$20 MILLION

AND A TRUST FUND IN THE AMOUNT OF US$0.4 MILLION

TO THE

PEOPLE’S REPUBLIC OF CHINA

FOR A

GUANGDONG TECHNICAL AND VOCATIONAL EDUCATION AND TRAINING PROJECT

June 23, 2015

Education Global Practice East Asia and Pacific Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS (Exchange Rate Effective June 7, 2015)

Currency Unit = Chinese RMB

RMB 1.00 = US$0.16 US$1.00 = RMB 6.20

FISCAL YEAR

January 1-December 31

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank ARCD Assessment Report for Course Development BA Beneficiary Assessment CPS Country Partnership Strategy CBT Competency Based Training CANO China National Audit Office DA Designated Account DO Development Objectives DOF Department of Finance DOLSSP Department of Labor and Social Security DRC Development Reform Commission EMF Environmental Management Framework FB Financial Bureau FMM Financial Management Manual GDFB Guangdong Finance Bureau GDHRSS Guangdong Human Resources and Social Security GDP Gross Domestic Product GLIS Guangdong Light Industry Secondary and Tertiary School GNI Gross National Income GUCS Guangdong Urban Construction Secondary and Tertiary

School IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report ICT Information Communication Technology IE Impact Evaluation IFR Interim Financial Report ISR Implementation Status and Results Reports M&E Monitoring and Evaluation MOE Ministry of Education MOHRSS Ministry of Human Resources and Social Security MOLSS Ministry of Labor and Social Security MOF Ministry of Finance NDRC National Development and Reform Commission

iii

PAD Project Appraisal Document PMO Project Management Office RMB Renminbi TVET Technical and Vocational Education and Training YTS Yangjiang Secondary and Tertiary Technical School

Vice President: Axel van Trotsenburg, EAPVP

Country Director: Bert Hofman, EACCF

Practice Manager: Harry Anthony Patrinos, GEDDR

Project Team Leader: Liping Xiao, GEDDR

ICR Team Leader: Liping Xiao, GEDDR

iv

PEOPLE’S REPUBLIC OF CHINA Guangdong Technical and Vocational Education and Training Project

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ........................................................... 1

2. Key Factors Affecting Implementation and Outcomes .......................................................... 5

3. Assessment of Outcomes ...................................................................................................... 14

4. Assessment of Risk to Development Outcome ..................................................................... 19

5. Assessment of Bank and Borrower Performance ................................................................. 19

6. Lessons Learned .................................................................................................................. 22

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners ....................... 23

Annex 1. Project Costs and Financing ...................................................................................... 24

Annex 2. Outputs by Component ............................................................................................. 25

Annex 3. Economic and Financial Analysis ............................................................................. 33

Annex 4. Bank Lending and Implementation/Supervision Support ......................................... 45

Annex 5. Beneficiary Survey Results ....................................................................................... 47

Annex 6. Stakeholder Workshop Report and Results ............................................................... 50

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ................................. 51

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ................................... 63

Annex 9. List of Supporting Documents .................................................................................. 64

MAP .......................................................................................................................................... 65

v

A. Basic Information

Country: China Project Name:

Guangdong Technical and Vocational Education and Training Project

Project ID: P096707 L/C/TF Number(s): IBRD-77200,TF-94998

ICR Date: 06/23/2015 ICR Type: Core ICR

Lending Instrument: SIL Borrower: CHINA

Original Total Commitment:

USD 20.40M Disbursed Amount: USD 19.99M

Revised Amount: USD 20.40M

Environmental Category: C

Implementing Agencies: Guangdong Provincial Department of Human Resources and Social Security

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 03/23/2007 Effectiveness: 10/08/2009

Appraisal: 10/06/2008 Restructuring(s): 07/31/2012

Approval: 06/02/2009 Mid-term Review: 05/21/2012 05/21/2012

Closing: 12/31/2014 12/31/2014 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Low or Negligible

Bank Performance: Satisfactory

Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Satisfactory

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Satisfactory

vi

C.3 Quality at Entry and Implementation Performance IndicatorsImplementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Public administration- Education 5 5

Vocational training 95 95

Theme Code (as % of total Bank financing)

Education for the knowledge economy 100 100 E. Bank Staff

Positions At ICR At Approval

Vice President: Axel van Trotsenburg James W. Adams

Country Director: Bert Hofman David R. Dollar

Practice Manager/Manager:

Harry Anthony Patrinos Eduardo Velez Bustillo

Project Team Leader: Liping Xiao Dingyong Hou

ICR Team Leader: Liping Xiao

ICR Primary Author: Franco Russo, Sandra F. Beemer F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The project development objective was to improve the quality and relevance of technical education in three schools and produce lessons from this experience as a guide for future school reforms and policy development. The project objective will be achieved with the support of: (i) school-based reforms and innovations to improve quality and relevance of technical and vocational education, and (ii) policy development and capacity building that help capture lessons drawn from the three school sites.

vii

Revised Project Development Objectives (as approved by original approving authority) Not applicable (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percent of graduates who pass skill certification exams Value quantitative or qualitative)

70% 85% 90.37% Source: School records

Date achieved 04/30/2009 12/31/2014 12/31/2014 Comments (incl. % achievement)

Target exceeded.

Indicator 2 : Percent of graduates finding initial employment within 6 months Value quantitative or qualitative)

86% 56% (target for 3 months)

92% (target for 6 months)

98.51% Source: PMO

Date achieved 04/30/2009 12/31/2014 12/31/2014 12/31/2014 Comments (incl. % achievement)

Target exceeded. This indicator was modified during the 2012 restructuring to change the period of time from 3 months to 6 months.

Indicator 3 : The starting salary of graduates (graduates with competency based curricula versus trainees without this curricula)

Value quantitative or qualitative)

￥950 ￥1,187 ￥2,625 Source: PMO


Target exceeded.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of instructors trained in industry attachments (in the same specialties)

Value (quantitative or qualitative)


Date achieved 04/30/2009 12/31/2014 12/31/2014 Comments (incl. %

Target exceeded

viii

achievement)

Indicator 2 : Percentage of industrial experts and enterprises participated in teaching in schools




Target exceeded

Indicator 3 : Percent of students participating in internship programs (in the same specialties




Target exceeded.

Indicator 4 : Percent of school annual performance goals met Value (quantitative or qualitative)

70% 85% 96% Source: School records


Target exceeded.

Indicator 5 : Percent of students enrolled in modular CBT curricula Value quantitative or qualitative)

0 77% 81.66% Source: School records


Target exceeded.

Indicator 6 : Percent of students assessed with outcomes-based student assessment Value quantitative or qualitative)



Target exceeded.

Indicator 7 : Percent of instructors using new CBT curricula Value quantitative or qualitative)


Date achieved 04/30/2009 12/31/2014 12/31/2014

ix

Comments (incl. % achievement)

Target exceeded.

Indicator 8 : Percent of students utilizing upgraded instructional technology aligned with new CBT




Target exceeded

Indicator 9 : Number of key policy studies disseminated Value quantitative or qualitative)

0 3 3 Source: NDRC


Target met. The three policy studies were completed and disseminated by NDRC through workshops and policy briefs in 2014.

Indicator 10 : Disclosure to the public of student learning outcomes (e.g. number and percentage of students passing certificate exams, employment etc.)


0 3 3 Source: Schools


Target met. All three schools disclose student learning outcomes on their school websites.

Indicator 11 : Enhancing provincial capacity for planning and management as measured by the number of tasks


0 2 2


This indicator tracks the number of new policies or regulations issued by the Guangdong HRSS during project implementation. This indicator has been met.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 08/05/2009 Satisfactory Satisfactory 0.00 2 05/13/2010 Satisfactory Satisfactory 0.70 3 04/26/2011 Satisfactory Satisfactory 4.50 4 02/22/2012 Satisfactory Satisfactory 9.83 5 09/17/2012 Satisfactory Satisfactory 12.07

x

6 04/14/2013 Satisfactory Satisfactory 16.34 7 10/23/2013 Satisfactory Satisfactory 16.34 8 05/31/2014 Satisfactory Satisfactory 16.34 9 12/16/2014 Satisfactory Satisfactory 19.95

H. Restructuring (if any)

Restructuring Date(s)

Board Approved

PDO Change

ISR Ratings at Restructuring

Amount Disbursed at

Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO IP

07/31/2012 N S S 12.07

The proposed change in the indicator is from "the percent of graduates finding initial employment within 3 months in the specialties in which they are trained" to "the percent of graduates finding initial employment within 6 months".

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country context. Since 1978, China experienced rapid economy expansion that led to an annual growth rate of 9 percent, lifting some 400 million people out of poverty. The expansion shifted employment into manufacturing and service industries with growth rates twice those of agriculture; 10 percent and 4 percent respectively. This growth rate accelerated industrialization and technological change, increasing the demand for new skills. China responded with increased investment in compulsory education up to 9 years, expansion of enrollments in vocational secondary education with improvements in quality and teaching reforms, and expansion and improvements in the quality of higher education. 2. In 2006, Guangdong Province, one of China’s centers of growth, accounted for approximately 12.4 percent of national GDP. The province’s economic boom began in the early 1990s and by 2009 the economy was based on manufacturing and exports. Its GDP in 2009 was almost 5 trillion RMB equivalent to US$0.63 trillion. Guangdong was part of the labor intensive manufacturing powerhouse that China had become with its exports accounting for 31 percent of the nation’s total exports. Gains in agricultural productivity had created a large pool of surplus labor that led to rural to urban migration. As a result, Guangdong needed to introduce changes in the structure of employment and reduce the importance of older labor-intensive industries using large quantities of low wage, unskilled labor. 3. Sectoral context. In the late 1990s, state-owned enterprises began to shift responsibility for skilled worker education and training to the Ministry of Education (MOE) and Ministry of Human Resources and Social Security (MOHRSS) (formerly the Ministry of Labor - MOL). In addition, the market opened to private providers of skills, but their capacity was only beginning to emerge and did not ensure uniform access. Government's strategy had been focused on upgrading school infrastructure and addressing issues of access to education by allocating more resources to rural and poor areas and setting up student aid systems. Spending on education in 2005 increased from 2.8 percent of GDP to 3.0 percent in 2006.

4. Secondary education nationally was separated into general, vocational, and technical schools with the first two under the direction of the MOE and the third under the MOHRSS. Overall, enrollments in vocational and technical schools in Guangdong had increased from 15 percent of all secondary enrollments in 2000 to 15.7 percent in 2006. Between 2000 and 2006, new enrollments in vocational and technical secondary schools grew by 55 and 158 percent, respectively, while the increase in general secondary schools grew by approximately 35 percent. This was evidence that more students were pursuing a secondary education and a larger share were pursuing skills through vocational and technical education.

2

5. Higher education included a mix of institutions covering universities, colleges (including technical colleges and senior vocational training colleges), and adult colleges. Several of the senior secondary technical and vocational schools also delivered two years of post-secondary education, and thus had a 3 + 2 year structure. Enrollments in higher education had surged since 2000, from 299,500 to 1,008,000 in 2006. Similar to higher education in other industrializing countries, vocational content was expanding in the first two years as the demand for more advanced skills continued to grow along with the economy. The expansion of secondary and higher education enrollments was consistent with an economy that was becoming more knowledge-based. 6. In Guangdong, placement rates on graduation were generally higher than the national average of 70 percent, further increasing demand. The investment in public vocational education had come at a time when the training capacity of state-owned enterprises had been diminished. Like many other areas with rapid growth in China, Guangdong had begun to face new challenges as economic forces had begun to introduce changes in the structure of its employment and reduce the importance of older labor-intensive industries using large quantities of low wage, unskilled labor. These industries had begun to feel competitive pressures on exports from rising labor costs and appreciation of the RMB. Wages, long kept low by the flow of surplus labor from rural areas, including from within Guangdong itself, had begun to rise, with wages in manufacturing and services outpacing those in agriculture. As a result, the mix of industries had started to shift with the expansion of new knowledge-based industries leading to a growing demand for skills. Demand for vocational and technical education was strong, measured by the increase in enrollments. 7. Public provision of vocational and technical education filled an important gap, to respond to the demand for advanced skills in Guangdong against a background of limited private provision. In 2005, there were 123 private vocational senior secondary schools in Guangdong. These schools enrolled 80,171 students with the average school size about half that of public vocational secondary schools. An Asian Development Bank (ADB) study in 2006 reported that the quality of private vocational schools in China was variable and, on average, lower than that of public schools. Private secondary vocational schools account for just over 7 percent of vocational and technical enrollments in the province. Public vocational and technical secondary schools face capacity constraints resulting in as many as 40 percent of applicants being turned away. While demand was strong, there were issues of quality and relevance in these schools. Equipment for practice was limited, curricula were often outdated and disconnected from industry requirements, instructors lacked adequate industry experience and teaching skills, and managers lacked the skills needed for functioning in a rapidly changing economy. The strength of the labor market nevertheless assured jobs for graduates. School-based reforms for quality and relevance were needed to support the transition to a more knowledge-based economy. 1.2 Original Project Development Objectives (PDO) and Key Indicators 8. The project development objective was to improve the quality and relevance of technical education in three schools and produce lessons from this experience as a guide

3

for future school reforms and policy development 1 . The project objective was to be achieved with the support of: (i) school-based reforms and innovations to improve quality and relevance of technical and vocational education, and (ii) policy development and capacity building that help capture lessons drawn from the three school sites. The key indicators as stated in the Project Appraisal Document (PAD) were: (i) percent of graduates who pass skill certification; (ii) percent of graduates finding initial employment within three months in the specialties in which they were trained; and (iii) the starting salary of graduates (graduates with competency based curricula versus trainees without this curricula). 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. The PDO was not modified over the life of the project however, one of the key indicators was revised in the 2012 restructuring. The indicator was modified from percent of graduates finding initial employment within three months in the specialties in which they were trained to percent of graduates finding initial employment within six months to align with the government’s standard indicator of employment. 1.4 Main Beneficiaries 10. The PAD identified the three senior secondary and tertiary institutions in Guangdong province that primarily served rural populations as the main beneficiaries of the project. The three schools were Guangdong Light Industry Secondary and Tertiary School (GLIS); Guangdong Urban Construction Secondary and Tertiary School (GUCS), and Yangjiang Secondary and Tertiary Technical School (YTS). Students, faculty and school administers were to be the primary beneficiaries within the three schools. In addition, provincial HRSS bureau staff was to benefit from capacity development activities and national level administrator’s capacity was to be strengthened through involvement in the development of the policy studies supported by a grant from the “Trust Fund for Bank-Korea Partnership on Poverty Reduction and Socio-Economic Development” (KTF). Direct project beneficiaries having benefitted from project interventions reached approximately 9,146 students who enrolled in CBT classes, of which 26.06 percent were female. The project also benefitted teachers and school administrators, by supporting training for 23,826 person times. This does not include the indirect beneficiaries such as parents and decentralized ministry staff who benefitted from knowledge transfer and other forms of capacity building.

1 There is a slight difference in the loan agreement PDO which was “the objective of the project is to assist Guangdong Province in improving the quality and relevance of technical education in selected schools and using the experience as a guide to further school reforms and policy development”. The PDO in the PAD is used for the purpose of this evaluation to assess project achievement because it was used in all the supervision reports and is the official PDO as recorded in the Data Sheet.

4

1.5 Original Components 11. Component 1. School-Based Reforms and Innovation (IBRD-US$19.73 million; Counterpart-US$24.65 million equivalent) 2 . This component was to: (i) strengthen the linkage between schools and industry; (ii) improve school management; (iii) introduce an updated modular, Competency-Based Training (CBT) curriculum; (iv) train instructors to use the new curriculum; (v) improve student assessment and quality assurance at the school level; and (vi) expand and upgrade facilities and equipment. 12. Component 1 had the following sub-components: (i) sub-component 1.1: strengthen the linkage between schools and industry aimed to strengthen and institutionalize linkages between the schools and local industry to promote a more efficient flow of information between the parties affecting the quality and relevance of training offered; (ii) sub-component 1.2: improvement of school management was designed to build capacity needed to ensure the effective use of school resources and produce the learning outcomes needed by a market-led economy; (iii) sub-component 1.3: curriculum reform aimed to introduce a new CBT curriculum with competency standards established by industry to promote improvements in quality and relevance; (iv) sub-component 1.4: improvement of student assessment and quality assurance aimed to improve quality by helping establish a competency-based benchmark for measuring student and school performance; (v) sub-component 1.5: improvement of instructor quality aimed to strengthen instructional capacity and improve the quality of student learning assessments to highlight areas where instructor performance in the classroom and workshop with the new CBT curricula could be improved and train instructors through pre-service and in-service measures to use the competency- based curriculum; and (vi) sub-component 1.6: upgrading of instructional equipment and facilities to improve teaching facilities and equipment for CBT specialties in the three schools. 13. Component 2. Policy Development and Capacity Building (IBRD-US$0.22 million; Counterpart-US$0.37 million equivalent and KTF grant-US$0.40 million3). This component was to: (i) support policy research and knowledge sharing at the national level, (ii) improve monitoring and evaluation, and (iii) enhance project management and coordination. 14. Component 2 had the following sub-components: (i) sub-component 2.1: policy research and knowledge sharing that supported policy research for carrying out policy studies on key issues in technical and vocational education and training, and learning from

2 The overall project cost included a Bank Front-end fee of US$0.05 million and counterpart fund contingencies in the amount of US$2.55 million. 3 The KTF was secured to finance policy studies at the national level, as well as impact evaluations for three provinces per the initial project design. The total amount was US$600,000. Once the decision was made to prepare a separate project for Liaoning and Shandong, funds were allocated as follows: (i) US$300,000 for NDRC to conduct three national policy studies, and (ii) US$100,000 for each province to conduct an impact evaluation. Therefore, US$400,000 were managed through the Guangdong TVET project but for ease of administrative purposes, the full amount of the KTF remained linked to this project.

5

the experience of implementing the project and disseminating such lessons learned; (ii) sub-component 2.2: improvement of monitoring and evaluation by laying the foundation for moving to evidence-based management and policy development; and (iii) sub-component 2.3: enhancing project management and coordination aimed to enhance capacity for project management and coordination by the provincial project management office. 1.6 Revised Components 15. The components were not revised over the life of the project. 1.7 Other significant changes 16. On July 31, 2012, the project restructuring was countersigned by the government to: (i) modify the definition of a PDO level indicator; and (ii) to modify the baseline and target for the redefined indicator. The indicator wording was changed from “percent of graduates finding initial employment within three months in the specialties in which they were trained” to “percent of graduates finding initial employment within six months”. The modification was made to align the indicator with the national and provincial level indicators that measure graduate initial employment within six months of finding a job rather than the three month period. The 2009 baseline employment rate for the newly defined indicator was revised to 86 percent and the target to 92 percent. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 17. Project preparation. In July 2006, the Chinese State Council approved the proposals from Guangdong, Shandong and Liaoning provinces to borrow from the Bank for a potential TVET operation. It was agreed that each province would borrow US$20 million for a combined loan of US$60 million. The government’s intention was to borrow for all three provinces under one project. The provinces were chosen based on their economically more advanced status, having stronger capacity and being able to serve as a model to other provinces. Lastly, the project was intended to serve as a pilot and to explore innovative approaches for reforming the current TVET system in China. 18. With this in mind, preparation began in October 2006 with the National Development and Reform Commission (NDRC) inviting the three provinces to present their preliminary project proposals. It became apparent that despite the provinces advanced capacity, their proposals focused mainly on infrastructure activities and equipping model schools rather than reviewing curricula and improving school management. In an effort to broaden the provinces’ intervention focus and to bring together stakeholders from central, provincial and school levels4, a workshop was organized in Singapore to share experience and jointly discuss the potential project design. The workshop included presentations from

4 Ministry of Finance, NDCR, Ministry of Education and Ministry of Labor and Social Security.

6

international TVET experts and visits to a number of Singaporean technical education institutes and professional organizations. As a result, a consensus was reached on a preliminary project design and objectives.

19. The central authorities including NDRC and MOF decided to pilot a new cooperation model of decentralizing the Bank loan from ministries to provincial departments in the area of education. As a result, the project was to be prepared without a central government component, and without a central government coordinating agency. This was the first time such an approach was taken in the education sector in China and was in response to the client’s decision to allow these more mature provinces to take the lead in the development of the project. This project was also the first education project to sign a project agreement directly with beneficiary provinces in China and to provide for fund transfers directly to participating schools. However, the absence of a coordinating agency among the three provinces slowed initial preparation activities as provinces clarified how their project proposals could fit within the overall project design. The Bank (with the support from other development partners, such as Korea and Singapore) organized additional workshops and study tours, which helped guide the three provinces in improving their respective proposals. Despite intensified support, progress remained uneven, with Guangdong improving their proposal following the workshop while Liaoning and Shandong were facing challenges due to weak provincial management offices (PMO) and beneficiary school selection. In May 2008, the government agreed to allow Guangdong to move ahead separately as a pilot operation and identified GLIS, GUCS and YTS. This allowed the Bank team to provide much more focused support to Guangdong and the three schools it had selected to be included in the project, so that appraisal could take place five months later. 20. Project preparation also benefitted from experiences made under the Australian-supported “Chongqing Vocational Education and Training Project”, especially in terms of introducing Competency-Based Training (CBT) to support the intended curriculum reform. Specifically, the Australian-supported project provided lessons on the need to establish school-industry linkages and set-up advisory/coordination committees with industry participation to establish competency standards, support school management improvements and teacher training. To support policy development aspects of the proposed project at the national level and ensure knowledge gathering and dissemination could take place, the Bank team secured US$600,000 from the KTF, of which US$400,000 were allocated for this project. Lastly, project design was also informed by studies on TVET in China, which highlight quality and relevance as the biggest challenges5. 21. As mentioned earlier, the preparation team also ensured that project interventions would represent (as requested by the client) a “public good” and would pilot innovative approaches and gather evidence to help inform TVET school operation in other provinces. To this end, the KTF grant was instrumental to earmark funding specifically for analytical work and evidence gathering. At the request of the delegation during negotiations, the

5 “Technical and Vocational Education and Training (TVET) in China”, Copenhagen Development Consultants A/S, 2005.

7

project also included a retroactive financing clause up to US$4 million to cover expenditures incurred after April 1, 2009 mainly to cover teacher training expenditures and which has allowed for activities to start as quickly as possible. 22. Again, these experiences and lessons learned helped the teams prepare and appraise the proposed project in five months once agreement was reached to allow Guangdong proceed without the other two provinces. The Bank approved the loan on June 2, 2009 in the amount of US$20 million and it became effective on October 8, 2009. Although the KTF grant agreement was signed on June 2, 2010, the grant had already been incorporated into the original design and was clearly identified in the component costing, making these funds an integral part of the project from the beginning. Program preparation was thoughtful, collaborative and technically sound and benefitted from a deliberative process between the national and provincial offices in charge of finance, education, development and reform, and labor and social security, and with the support of the Guangdong PMO. This was a sizable undertaking and might have contributed to a longer-than-average preparation period but proved crucial to ensure buy-in at all levels and a clear understanding of the project. The Bank team was actively involved in facilitating communication and helping the province in finalizing the design with the support of other donor partners. The Bank preparation and appraisal teams consisted of technical experts that were appropriate for the development of the project and included peer reviewers that provided sound guidance on the project design. 23. Project Design and Quality at Entry. The project design was fully aligned with the government’s 11th Five-Year-Plan for 2006-2010, which included strengthening market institutions through reforms that would enhance performance and accountability of TVET institutions but also increase relevance of technical and vocational education. The project was also viewed as aiding Guangdong in achieving its goal of becoming southern China’s financial center and logistics hub by producing the skilled workers necessary to support the growth of the new knowledge-based industries. The design recognized the pilot nature of this project and purposefully included training opportunities for teachers and other key staff at project schools to learn from other countries such as Singapore’s “teaching factory” or Germany’s “dual-system” teaching process. This was highly relevant as it marked a shift from the traditional TVET instruction methodology applied in China in the past and strengthened the piloting of a new approach through this project. The project was also fully aligned with the Bank’s 2006-2010 Country Partnership Strategy (CPS). Specifically, the project supported the CPS’ Pillar 5 improving public and market institutions, by improving the performance and accountability of TVET institutions. The project also contributed to Pillar 2 of the CPS reducing poverty, inequality, and social exclusion as the almost three quarters of all enrollees in the three project schools were from rural areas and low-income households. As mentioned previously, the program design took into consideration comments from peer reviewers and donor partners provided during the concept and appraisal review meetings. 24. Overall responsibility of project implementation was delegated to the province through the establishment of a Project Steering Committee (PSC) to be led by the Director General of the Guangdong Department of Human Resources and Social Security (DHRSS)

8

(formerly Labor Bureau). The PSC included representatives of the provincial department of finance (DOF) and development and reform commission (DRC) and was to serve as a coordination body, provide substantive guidance on overall project implementation and facilitate inter-agency coordination. The provincial PMO was in charge of the day-to-day implementation and was to coordinate with the three project schools’ project implementation units (PIU). DOF, which had experience with World Bank-financed projects, was supporting the PMO in terms of fiduciary, auditing and monitoring of project activities. This institutional set-up was appropriate given the pilot nature of project interventions and the need for more flexibility with a bottom-up approach. To support the shift of TVET instruction from a teacher-focused to a student-centered approach and the strengthening of school-industry partnerships, the design included the establishment of advisory committees. These committees were to be established at the school level to facilitate communication with enterprises and seek their input for the development of competency-based standards and teaching modules. The committees were also charged with guiding teachers (who were identified as the main drivers of the school-based reform) in the development of competency-based standards and teaching modules, which was a new role for teachers. This proved highly relevant and was instrumental for the successful implementation of the school-based reforms and CBT curriculum. 25. The PDO was in line with the government’s priorities as expressed in the 11th Five-Year Plan of improving quality and relevance of technical and vocational education provision and was also reflective of the project’s activities. The results framework was appropriate for measuring project progress: the PDO was measured by three key outcome indicators, and eleven intermediate outcome indicators were selected to measure outcomes of the project components. In addition to the outcome indicators, the design team selected output indicators to be monitored and reported regularly. As previously mentioned, the KTF was specifically secured to support knowledge generation and lessons learned, and to help inform future policy decisions.

26. Indicator formulation was guided by extensive consultations between the project schools, the PMO and the Bank. Given that this was the first education project to be prepared without a central coordination component, intensive technical assistance was provided by the Bank team. This allowed the project team to agree on and finalize the results framework in time for the Bank’s pre-appraisal mission in May 2008. The results framework development was guided by the Quality Enhancement Review (QER) held on September 15, 2008, which provided sound guidance to the team and allowed the project to be appraised by October 2008. As mentioned above, the results framework was revised in July 2012 following the mid-term review (MTR) to adjust the timeframe within which graduates would find employment and be in line with the province’s own monitoring target. 27. The project design was comprised of two main components that focused on school-based reforms and innovation and policy development and capacity building. The project included a baseline and follow on survey to measure the impact of the project’s interventions through an Impact Evaluation (IE). The project also included a Beneficiary Assessment (BA) to be conducted at mid-term and project closing and an Assessment Report for Course Development (ARCD) in 2012. As mentioned previously, the design

9

recognized the importance of teachers in carrying out the school-based reforms and therefore put in place the necessary support structure (advisory committees, trainings at TVET schools abroad and other provinces). Lastly, the preparation team identified project risks and proposed adequate mitigation measures. 2.2 Implementation 28. The project had a good start in regards to Component 1. Within the first year of implementation, all three schools: (i) established School Industry Advisory Committees and Specific Sector Advisory Committees; (ii) undertook extensive management and teacher trainings through international, national and school-level exchanges; (iii) developed new competency-based standards and curriculum packages; and (iv) introduced new teaching methods focusing on student-centered pedagogy. Civil works and procurement of equipment was also underway albeit slower than anticipated due to the PMO and PIUs being unfamiliar with Bank procurement processes and issues related to delays in mobilizing counterpart funds for one project school. Component 2 activities were also progressing but slower than originally planned: Policy research and knowledge sharing at the national level and improving monitoring and evaluation moved at a slower pace. This was mainly due to the time it took to complete the signing of the KTF, which, in turn, delayed the signing of contracts with the relevant research companies. Given that the KTF was to finance national-level policy studies as well as impact evaluations in two additional provinces (Liaoning and Shandong provinces), it took time to agree on the most appropriate implementation arrangement. Once an agreement was reached between the different parties, the grant agreement was signed. Despite the initial slow start, actual disbursements were at US$2.7 million (approximately 14 percent) by September 2010 (15 months after approval), or US$9.7 million (48.5 percent) when including commitments. 29. By the 2012 MTR, however, most activities were either on track to achieve their targets or had already exceeded them: (i) construction at GUCS and YTS was underway; (ii) industrial experts were teaching at schools; (iii) more students benefitted from internships in enterprises with which schools had established a cooperation; (iv) practice spaces had been established, either on campus or at enterprises to further enhance students’ practical knowledge; (v) new assessment tools had been developed to evaluate student learning and performance; (vi) intensive capacity building of teachers and school administrators was being carried out; and (vii) activities related to the policy studies and evaluation had been launched. In addition, the three schools had applied (and been accepted) to participate in the MOHRSS’s research program to allow teachers to summarize lessons learned in developing and implementing CBT and conduct research to further inform CBT content and implementation. The MTR confirmed that the program remained relevant, the PDO was still valid and achievable, and disbursements were at approximately 50 percent of loan proceeds. As mentioned earlier, the MTR identified the need to modify the wording of one PDO indicator and to adjust the baseline and end target to accommodate the new definition, which were approved in the July 2012 restructuring.

30. At mid-term, some challenges were also highlighted, namely: (i) GLIS construction delays due to permit problems and lengthy approval processes, which slowed the

10

acquisition of new equipment and led to crowded teaching spaces; (ii) slower-than-anticipated roll out of reforms in all project schools due to capacity constraints of teachers and administrators to develop new textbooks and learning material, and deliver the new CBT courses; and (iii) the need to strengthen the role and involvement of school-industry and industry-specific sector advisory committees to help increase enterprises’ awareness of the benefits of CBT and increase their involvement in developing competency-based courses. These findings were also reported in the BA and ARCD, which proved to be timely and highly relevant to helping schools improve the quality and relevance of courses and inform material development. Delays also continued to affect the production of the three policy studies funded by the KTF, which were managed by the Social Development Department of NDRC. 31. However, the project was able to overcome these challenges and complete all envisaged activities as designed and disburse 98 percent of combined loan and grant proceeds. This was a substantial achievement given some of the initial delays, the newness of the implementation arrangements and innovative nature of CBT implementation. When the project closed, all intermediate indicators had been exceeded. In addition: (i) each school had operational school-industry advisory bodies in place, established guidelines and standard contract forms for school-industry partnerships, further strengthening school-industry linkages; (ii) teachers and administrators benefitted from capacity strengthening through a variety of trainings (school-based, domestic and international); (iii) CBT syllabi, curriculum standards and textbooks had been developed, trialed and rolled out in all three schools; (iv) all schools had significantly expanded training spaces for students to practice their newly acquired skills; (v) MOHRSS used the results and lessons from the project to inform the development of two national level guidelines related to TVET; and (vi) the management information system (MIS) in each school was strengthened and is being used to collect student tracking information. Project savings of approximately US$415,000 stemmed from fewer overseas training activities due to government travel regulations. However, this did not undermine CBT capacity building of school staff as evidenced by achievements made under the project, which are substantial. 32. There were seven Bank review and implementation support missions from December 2010 through December 2014, including one MTR and one final implementation mission. The makeup of the Bank teams was consistently appropriate and well balanced. Each team composition reflected the needs of the respective mission and included specialists from the areas of education, financial management, procurement, construction, environment, social, and monitoring and evaluation as necessary. There were different team leaders (TTLs) involved during preparation and project launch, however, the Beijing-based TTL was involved from the first full implementation mission until project closing. This allowed for continuity during implementation and informal visits in-between official missions. As a result, the Bank was able to provide the intensive technical assistance required at the beginning of the project and quickly respond to issues and challenges as they arose, further contributing to the project’s success.

11

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 33. Design. The M&E design was three-fold. First, the monitoring of the results framework and project outputs, as well as process evaluations were to be done by the PMO. As stated previously, the original results framework was appropriate for the project design to measure achievement of the PDO. The intermediate indicators were linked to the components and were appropriate to measure intermediate level achievements. The results framework included 11 intermediate outcome indicators as well as output indicators. Secondly, the project included the strengthening of school-level M&E mechanisms, such as their MIS, student tracer systems, two beneficiary assessments and an ARCD. Finally, the design included an IE aimed at assessing the impact of the curricula reforms on improving the quality and relevance of technical and vocational education. This was the first time such an independent evaluation was being proposed for a Bank-supported education project in China and served as a learning experience for the PMO. As indicated above, the results framework was reviewed during the MTR and adjusted through the 2012 restructuring. The M&E design was relevant for the overall design of the project and appropriate for monitoring project impact. 34. Implementation and Utilization. The PMO regularly provided Bank missions with data and information that allowed teams to make judgments on progress toward achieving the PDO and the project’s overall implementation. This information was consistently reported on by the Bank in aide-memoires and Implementation Status and Results (ISRs) reports. The PMO also monitored the project outputs, and schools used the information to make the appropriate adjustments in their annual plans. As mentioned above, the project conducted two BAs and an ARCD, which were available for the MTR, while the PMO prepared an MTR progress report. These documents, especially the BA and ARCD, provided useful suggestions on how to improve CBT which included: (i) deepening school-industry cooperation to further align CBT with market demands; (ii) strengthening the advisory committees so that they would become more active and better able to support teachers during curriculum development; and (iii) improving schools’ marketing campaign regarding the types of graduates and skills they produce to attract more enterprises. The recommendations were acknowledged and incorporated in the school plans, which was evidenced in the end-line BA and government completion report. (See Annex 2 for details on outputs related to these recommendations.) 35. All three schools’ monitoring systems were strengthened through the project and all are able to collect and analyze student tracking data. Additional aspects of the management systems were developed in each school, including education management information, teacher quality assessment, course scheduling, student performance and graduate employment tracking.6 Schools collect and analyze information related to school-industry collaboration, teacher effectiveness and promotions, student performance and

6 Because of the improved MIS systems, one school applied for funding from the “China Association of Workers and Staff Education” to further analyze and assess its tracking system. The research is still ongoing but it is expected that the results will be disseminated at the ministerial level and inform guidelines for system development and standards.

12

number of industry experts teaching at schools. Results are being published online and shared through “Weixin”, China’s version of “Twitter”. 36. The IE was to be conducted by an independent firm, hired by the PMO. The baseline study was carried out in 2010 but the approach had to be modified due to constraints in finding comparable non-project schools. In the end, a simple “before and after” analysis was proposed, and a second round of data collection was completed in 2014. The province and schools learned valuable lessons with regard to how IEs are designed, conducted, and data is analyzed7. 37. These diverse ways of collecting, analyzing and documenting information has led to changes at school, provincial and national levels as evidenced (amongst other things) by: (i) institutionalization of school-industry cooperation, including standardized agreements; (ii) use of project school CBT textbooks by the provincial education research division to promote CBT development in other TVET schools as well as provincial school-industry collaboration guidelines for Guangdong; and (iii) the issuance of two national guidelines on April 6, 2013.

2.4 Safeguard and Fiduciary Compliance 38. Environment and social safeguards. The project was rated a Category “C” operation, and an Environmental Management Framework (EMF) was developed. The project’s compliance was rated “satisfactory” throughout its life. In terms of environmental safeguards, each project school worked with the PMO and representatives from local environmental protection agencies to ensure compliance with the EMF and the relevant environmental laws and regulations. At the time of the MTR, a number of improvements were suggested such as timely removal of solid wastes, acceleration of greening works and strengthening of noise and dust mitigation measures. As a result, environmental requirements were added to bidding documents and contracts, payment retention clauses were included, public consultation prior, during and after construction were intensified and the local environmental monitoring agencies were entrusted to closely monitor overall environmental safeguards compliance. This was routinely verified during subsequent implementation support missions so that by project closing, the Bank’s environmental safeguards specialist confirmed that civil works were completed in compliance with the EMF and relevant environmental laws, and that all MTR suggestions had been taken on board. No social safeguards policies were triggered but the project did assess stakeholder satisfaction, which included students from poorer backgrounds (i.e., farmers and migrant workers), through the two BAs and the IE. Based on the end-line BA, the majority of students felt that their basic learning needs had been met, the curriculum was considered relevant and teachers’ capacity had increased.

7 The IE clarified that: (i) the response rate was 46.5 percent for 2010 baseline survey and 45.5 percent in 2014; (ii) GUCS reported that all graduates participated in CBT and that data for comparison purposes was not available; and (iii) continued changes and improvements to subject structure and content did not allow for a simple “before-and-after” comparison as the variables had changed over the life of the project.

13

39. Financial management. Financial management (FM) remained satisfactory throughout the life of the project. During the initial stages of project implementation, the project encountered the following challenges: (i) delays in civil works, which led to disbursements lagging original projections; and (ii) confusion by project schools regarding eligible expenditures for innovation activities. It should be noted that although a designated account (DA) was established at the PMO level, each school had an operating account (OA) from which innovation-related expenditures were paid. This was a new way of operating and required each school to strengthen their fiduciary systems. With intensified support from the Bank and PMO, the following was achieved: (i) aspects related to innovation activity financing were clarified and relevant guidelines issued by early 2011; (ii) school-level financial management capacity had increased and internal controls were strengthened; and (iii) collaboration between different stakeholders, including provincial departments of finance, DRC and MOF, and the PMO had been established, contributing to smooth implementation and accurate record keeping. By mid-term, overall disbursements had accelerated and remained largely on track until project closing, especially once delayed construction of GLIS had gotten underway. Bank implementation support missions consistently confirmed that the project had a satisfactory financial management system in place, which produced timely and accurate information, and all audits were unqualified. 40. Procurement. Procurement was consistently rated satisfactory, except in January 2012 as the project experienced delays in civil works and innovation activity procurement. As mentioned before, the new building at GLIS continued to face challenges, which were due to government-related permit issues and the fact that certain “Asia Games” venues were being held in Guangzhou in 2010. This meant that city officials were focused on ensuring the necessary infrastructure was in place and ready in time for the games, delaying all non-game related processes. Nevertheless, all procurement packages were completed prior to project closing, including goods for the new GLIS building, and are in use. This was partly due to the continued capacity building efforts shown by the Bank and PMO to school procurement office staff and the establishment of a procurement management manual. Also, quality assurance mechanisms were put in place by the PMO to review procurement packages and annual implementation/procurement plans prepared by schools.

2.5 Post-completion Operation/Next Phase 41. Expanding CBT and school-industry cooperation continues to be promoted through exchanges and workshops, the two national policy documents, the three national policy studies and different provincial directives provide a strong foundation for expansion. Furthermore, the Bank-supported Guangdong Social Security Integration and Rural Worker Training project (SSIRWT), which was approved in May 2013, continues a number of activities that were piloted under this project, including: (i) the development and roll-out of CBT; and (ii) strengthening school-industry linkages. In fact, the GLIS was selected to be part of the SSIRWT project as a lead school so that the other two participating schools could benefit from GLIS’ experience acquired under this project. The SSIRWT project is also being managed by the same PMO due to its proven management track record. There also exist a number of TVET projects in China, which are financed by the Bank and

14

which include the same components/activities, the latest operation having been approved by the Bank’s Board of Directors in May 2015 for Xinjiang province.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 42. Relevance of Objectives. The project development objectives were highly relevant to the country’s sectoral needs when the project was developed. The objectives fit squarely into the government’s 11th “Five-Year-Plan” for 2006-2010 and the aligned 2006-2010 CPS. They all shared a common vision of strengthening market institutions through reforms that will enhance performance and accountability of TVET institutions but also increase relevance of technical and vocational education. The project continued to be aligned with the 2013-2016 CPS, which again is aligned with the country’s 12th Five-Year-Plan, both focusing on promoting more inclusive development by strengthening skills development programs. The objectives’ high relevance continues to be acknowledged by: (i) the “China 2030: Building a Modern, Harmonious and Creative Society” study conducted jointly by the Bank and the government, which proposes expanding opportunities for all to access decent jobs, accelerate pace of innovation and increase technical and cognitive skills of graduates; and (ii) the government’s “Development of Modern Vocational Education Plan 2014-2020” and the “Medium to Long-term Education and Talent Development Plans for 2010-2020”, proposing increased access, quality and relevance improvement through stronger school-industry linkages, improved TVET curricula and flexible transitions between secondary and vocational education. It should also be noted that policy studies, financed by the KTF, have provided a good foundation in the area of vocational education and training as the government prepares its 13th Five-Year-Plan. 43. Relevance of Design. As mentioned above, the project design was highly relevant. The PDO was appropriate in its focus on quality and relevance of TVET education, and the results framework, project components and activities were appropriate for measuring achievement of the PDO. The 2012 restructuring ensured that the identified PDO level indicator was more consistent with the national and provincial level indicator of tracking employment of graduates after six months rather than three months, which was an appropriate alignment. Moreover, the 20 percent allocation of the project funds for technical assistance ensured adequate resources for teachers and administrators to develop new curricula and focus on the paradigm shift from teacher-center to student-centered teaching. The focus on strengthening the school information systems has improved their ability to collect and analyze data, especially in terms of tracking student and teacher performance, school-industry collaboration and curriculum reforms. The inclusion of the KTF-financed policy studies, IE, BAs and ARCD was highly relevant and instrumental in informing reforms at school, provincial and national levels. The project risks and mitigation measures were also appropriate. 44. Relevance of Implementation. The relevance of project implementation is also judged to be high. As indicated above, the project schools and PMO were able to

15

implement all project activities as planned. Where they encountered implementation challenges, such as with financing eligibility criteria for innovation activities, they were able to work with Bank staff to find solutions and resolve the issues. The lessons learned are now being incorporated into the ongoing work of the DHRSS to further harmonize TVET instruction in Guangdong and have also influenced policy discussions at the national level. The high implementation relevance is also reflected in the fact that the SSIRWT operation in Guangdong has adopted the same approach to implementing migrant worker skills training, with the GLIS as the model for the other two project schools. In fact, the government has drawn upon the implementation relevance of this project when it requested Bank support for TVET projects in Liaoning and Shandong, Yunnan and Xinjiang. These are positive developments and an indication of overall project relevance. 3.2 Achievement of Project Development Objectives 45. The project development objective was to improve quality and relevance of technical education in three schools and produce lessons from this experience as a guide to future school reforms and policy development. This section evaluates the outcomes against the PAD results framework that was adjusted in July 2012. More details on project outputs can be found in Annex 2. 46. It should also be noted that during the project period all three project schools were upgrading within the context of China’s TVET system. There are a number of national and provincial criteria that are used for determining the levels of TVET schools which included, but are not limited to, teacher qualifications, curriculum development, number of m2 per student, course specific equipment available for student use etc. During the project, YTS was upgraded twice from an ordinary technical school to an advanced technical school and ultimately to a technical college in December 2014. GLIS and GUCS were also upgraded, from advanced technical schools to technician colleges. In the case of YTS, the school was able to offer a higher level technician certification. In the case of GLIS and GUCS, the schools were able to offer certification from MOHRSS and a diploma equivalent to an associate’s degree. While there are many factors that may have contributed to the upgrading of the schools, the project inputs of facilities, equipment, curriculum development and teacher training made a significant contribution to the upgrading of the schools. This is an important factor to consider when evaluating the achievement of the PDO as well as the overall impact of the project. 47. Quality improvement was to be measured by percent of graduates who pass skills certification exams. There was considerable progress in this PDO level indicator. The overall percentage of students that passed the skill certification exams increased from 70 percent in 2009 to 90.37 percent in 2014 exceeding the target of 88 percent. The impact evaluation conducted showed that the proportions of secondary technical graduates of the class of 2013 from YTS and GLIS, who received competency-based coursework and passed skills examination for the first time at the time of graduation, were 96.8 percent and 96.6 percent respectively, both of which were higher than that of secondary technical

16

graduates8 from such schools who did not receive competency-based coursework (92.9 percent and 87.2 percent respectively).9

48. Relevance was to be measured by: (i) gains in the percentage of graduates finding initial employment within 6 months; and (ii) starting salary of graduates (graduates with competency-based curricula versus trainees without this curricula). The overall percentage of graduates from the project schools finding employment within six months increased from 86 percent in 2009 to 98.51 percent in 2014, exceeding the target of 95 percent. In addition to the PDO indicators, the impact evaluation showed that the relevance between jobs and majors10 with respect to college technical graduates in 2013 from project schools was 59 percent compared to 48 percent in of 2010, while the relevance between jobs and majors with respect to secondary technical graduates in 2013 was 46 percent compared to 45 percent in 2010. 49. The data reported for starting salaries was the average starting salaries for all graduates in the three schools. This data shows an increase from RMB 1,744 in 2009 to RMB 2,625 in 2014.11 However, the impact evaluation provided an analysis of salaries for GLIS and YTS12 based on graduates that participated in the competency-based courses compared to the non-competency based courses. The impact evaluation results showed that 2013 college technical graduates participating in CBT courses from GLIS received RMB 3,149 while non-CBT received RMB 3,302; and from YTS the CTB graduates received RMB 2,744 and non-CBT received RMB 2,383. With respect to secondary technical graduates, GLIS CBT graduates received RMB 3,681 and non-CBT RMB 3,099 and YTS CBT graduates received RMB 2,751 and non-CBT received RMB 2,565. All data provided exceeds the 2014 results framework target of RMB 1,18713. 50. Produce lessons from this experience as a guide to future school reforms and policy development was to be measured by the intermediate outcome indicator “number of key policy studies disseminated”. There were substantial achievements related to this indicator. At the national level, and with support from the KTF, the Social Development Department of NDRC produced three policy studies in the areas of: (i) financing of TVET

8 The three project schools can provide two kind of programs: secondary level program and college level program. The secondary technical graduates are the graduates of the secondary program. 9 The secondary technical graduates is the appropriate comparison because YTS did not have college technical program before implementation. 10 Relevance between jobs and majors is defined as: graduates who are "employed full-time" and includes two categories: firstly, those engaged in full-time employment relevant to their majors; secondly, those engaged in full-time employment not relevant to their majors. Major relevancy = number of graduates engaged in full-time employment relevant to majors/number of graduates engaged in full-time employment (including those relevant not relevant and not relevant to majors. 11 The salary calculations are adjusted for inflation and do not include pension, medical insurance and public housing fund. 12 GUCS claimed all graduates participated in CBT courses so there are is no non-pilot program data for this school available to make the comparison. 13 Note that these figures differ from Figure 3 in Annex 3 due to the fact that Figure 3 reports on starting salaries of secondary technical and college technical graduates combined for comparison purposes with other sectors and provincial average.

17

education; (ii) quality improvement of the TVET system particularly as it relates to curriculum development and school-industry collaboration, and (iii) long-term governance structure. These policy studies provided a good foundation for the preparation work for the 13th Five-Year-Plan in the area of vocational education and training. The results of the policy studies have been disseminated to other provinces, which is expected to also facilitate the preparation of the provincial level 13th Five-Year-Plans. Moreover, MOHRSS used the results and lessons from the project to inform the development of two national level guidelines related to TVET. These guidelines are: (i) “Guideline for the Development of National Training Standard for High-skilled Talents”; and (ii) “Technical Specifications for the Development of Integrated Curriculum”. Finally, TVET experts in China indicated that the three policy studies are expected to also inform revisions to China’s Vocational Education Law. 51. At the provincial level, the Provincial Department of Human Resource and Social Security issued guidelines that were informed by the project entitled “Suggestions on Strongly Promote the Connotation Development of Technical Education in Guangdong” in November 2014, which include project schools’ experiences and recommendations related to competency-based curriculum development and implementation. The Guangdong PMO worked with project schools and the Provincial Department of Human Resource and Social Security to produce a collection of the competency-based curriculum reform outcomes and published the “Distinguished Papers on World Bank Loan: Guangdong Technical and Vocational Education and Training Project (2009-2014) (ISBN 978-5100-5567-6) which included 35 papers. At the school level there were many activities related to policy research and knowledge sharing that included essays, survey reports, textbook development and training plans. 3.3 Efficiency 52. Overall, the evidence suggests that wages improved for graduates at both the secondary and college levels. In particular, wages after six months of graduation for secondary level graduates who took CBT courses were between 7 and 18 percent higher than those for graduates who took non-CBT courses (depending on the school). Although wages for college graduates at GLIS were four percent lower, their employment rates were higher than for GLIS students who did not take part in CBT courses. 53. The evidence also points to different impacts for college graduates than for secondary graduates. In GLIS and YTS, college graduates of CBT courses had higher employment rates than others. While for secondary graduates, there was either no impact or a slight decline (depending on the school). 54. Variation across project schools in terms of outcomes could be explained by the fact that in some schools, roll out of reforms may have been slower than anticipated. This may have been due to capacity constraints of teachers and administrators to develop new textbooks and learning materials, and a lag time inherent in learning how to deliver the new CBT courses as explained above. Based on this, efficiency is rated substantial.

18

3.4 Justification of Overall Outcome Rating Rating: Satisfactory 55. The overall outcome rating of the project is satisfactory. The relevance of design was high. The design focused on the government’s priorities in the 11th Five-Year Plan that focused on improving quality and relevance of technical and vocational education provision and was also reflective of the shift in TVET instruction from a teacher-focused to a student-centered approach. The relevance of implementation was high. While there were challenges during implementation, the PMO and project schools, working with the Bank team, made necessary adjustments to successfully complete the activities and draw lessons for future interventions. Efficacy is rated satisfactory because the PDO was achieved as measured by the achievement of the PDO indicators. Finally, the program was implemented efficiently. Table 1. Project Rating

Original Project – 10/8/2009 – 7/31/ 2012 – 59.4 percent disbursement-net project funds1

Project Relevance Achievement of PDO (Efficacy)

Efficiency Overall Rating

High Satisfactory Substantial Satisfactory

Original Project – 8/1/2012 – 12/31/2014 – 38.6 percent disbursement-net project funds1



High Satisfactory Substantial Satisfactory

Overall Project Ratings – 98 percent disbursement of total loan as of 12/31/20142



High Satisfactory Substantial Satisfactory2

1The net amount for the total project is US$19.99 million. 2 Overall rating = (5)(.594) +(5)(.386) = 4.9 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 56. The project was to contribute to reducing poverty and inequality. During the project period, the occupational categories of families of graduates changed substantially. From 2010 to 2013, the advanced technical graduates from farmer and migrant worker families increased from 38 to 52 percent while the intermediate technical graduates increased from 42 to 50 percent. The increase in graduates from farmer and migrant worker families along with an increase in wages for graduates of the three schools implies the project made a contribution in these areas within the project schools, for the graduates and Guangdong province. (See Annex 3 for the wage analysis.) (b) Institutional Change/Strengthening Institutional change and strengthening is covered throughout the document.

19

(c) Other Unintended Outcomes and Impacts (positive or negative) Not Applicable 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops See Annex 5 for details. 4. Assessment of Risk to Development Outcome Rating: Low 57. There were three issues identified as risks during the original project preparation with an overall rating of moderate. The ratings were primarily related to: (i) over-emphasis on upgrading technology and infrastructure at the school level without due attention to CBT curriculum reforms; (ii) technical capacity to implement and manage the CBT curriculum; and (iii) willingness of teachers and students to adopt the new CBT approach. The mitigation measures incorporated in the project: (i) tied the provision of buildings and equipment to the CBT curriculum reforms; (ii) frontloaded targeted TA and training for the reform; and (iii) linked school-based innovation with the policy studies, which was appropriate and ultimately responsible for ensuring smooth project implementation. The risks associated with financial management were identified as a moderate risk. Going forward, the financial management risk is likely to be low given the capacity development training provided by the project at the provincial and school level. The overall risks for maintaining the TVET and CBT reforms should be low given the government’s continued focus on competency-based skills for workers and the expansion of similar projects in other provinces in China (i.e., Shandong, Liaoning, Yunnan and Xinjiang). In Guangdong specifically, the risk should be low as well due to the continuation of many of the project activities in the new Bank-supported SSIRWT project and the willingness exhibited by project schools to expand the CBT courses beyond the original majors. All of these factors, along with the increased capacity within the provincial education bureau to implement TVET projects, future risks should be low. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 58. The project preparation team ensured that the project design was closely aligned with the 11th Five-Year-Plan and the 2006-2010 CPS. The design: (i) was appropriate to address the challenges identified by the province and project schools; (ii) included a PDO that was pertinent; and (iii) identified key indicators that were relevant to measure progress towards achieving project objectives. The design took into consideration the recommendations of the QER, comments from peer reviewers and lessons learned from other Bank and donor-supported projects. The inclusion of the retroactive financing clause was appropriate to ensure a quick project implementation start-up and continued ownership

20

of the project. The preparation team identified the appropriate risks and incorporated design features to mitigate them and included the relevant technical specialists to support project development. In the absence of a central coordinating agency, the Bank team provided intensive and continuous support, which was critical especially for project schools: as they faced challenges in preparing their respective project proposals, the Bank team’s support helped ensure that proposals were relevant and fit within government priorities. Emphasis was also placed on the project being innovative and producing lessons learned through the inclusion of different assessments and evaluations. The Bank team also helped organize various workshops, trainings and study tours, which were instrumental for the finalization of the design, content, and implementation arrangements of the project. Based on this, the quality at entry is rated satisfactory. (b) Quality of Supervision Rating: Satisfactory 59. As stated previously, there were seven Bank review and implementation support missions including a MTR and final implementation mission. The Bank was actively engaged in supporting the PMO and project schools in their efforts to implement the project. The in-country presence of the task team leader allowed for continued interactions between the PMO, project schools, and the Bank through informal meetings and exchanges in addition to official implementation support missions. The provincial departments as well as school directors repeatedly mentioned the country-based TTL as having been instrumental in facilitating early identification of potential bottlenecks and mobilizing the necessary technical expertise to resolve them. The mobilization of additional technical expertise was particularly pertinent for the development of the three national policy studies and review of CBT curricula. In addition to the task team leader, the in-country presence of the fiduciary and safeguards team members further enhanced the Bank’s ability to promptly respond to client requests and resolve challenges (i.e., helped clarify eligibility criteria for innovation activities, organized financial management and procurement workshops as well as provided on-the-job training, ensured safeguards were being complied with etc.).The Bank worked actively with the government to ensure that the results framework was revised to ensure its alignment with provincial targets. Whenever implementation challenges arose, the Bank team worked with the clients to find solutions that would not compromise the integrity of the design. The Bank team supported schools in expanding research, knowledge gathering and sharing beyond what was originally envisaged, which was instrumental in informing provincial and national policy dialogue. The implementation support teams also consistently reported on financial management and procurement progress during missions and worked with the PMO and project schools to build their capacity in these areas. As already indicated, they also systematically documented project progress in aide-memoires, back-to-office reports and ISRs, all of which kept Bank management informed of progress. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 60. Based on the analysis above, overall Bank performance is rated satisfactory.

21

5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 61. At the time of preparation, the government was fully committed to the project. Although initially proposing to combine Guangdong, Shandong and Liaoning provinces under this project, the decision was made to allow Guangdong province to move ahead separately because their proposal was more advanced. This helped to accelerate final project preparation, appraisal and approval. Government representatives actively participated during the initial stages of project identification and design and supported the knowledge exchanges and study tours. The three national policy studies were carried out in a timely manner and findings have been disseminated to inform future policy discussions. As mentioned previously, the government has embraced the essentials of competency-based skills development as the mode of instruction in TVET schools across China, using the experiences and materials produced by this project as a model to guide other provinces. All relevant central and provincial actors remained actively engaged throughout the life of the project and continuously strengthened their collaboration to facilitate project implementation. Lastly, lessons learned from monitoring and evaluation activities have been used to inform the development of competency-based skills programs in other provinces. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 62. The PMO was responsible for coordinating all aspects of implementation in close collaboration with project schools. When faced with implementation challenges, the PMO and schools were able to work effectively with the other provincial departments and the Bank to resolve them. The PMO was also instrumental in ensuring training was being provided, supporting materials produced to facilitate implementation at the school level (i.e., manual clarifying innovation expense eligibility) and information collected for monitoring and reporting purposes. The PMO supported project schools to explore innovative ways to develop and roll-out CBT, and schools showed strong commitment to implement the proposed reforms by: (i) going beyond the initial target of majors being taught through CBT; (ii) collecting their experiences and best-practices on the development of CBT, which has been compiled into a book and made publicly available as a resource guide; and (iii) remaining flexible in the way they developed and implemented CBT as well as developing school-industry partnerships according to each school’s characteristics. These are substantial achievements and show the increased capacity of the province and schools. This is further supported by the fact that the PMO has been chosen to support the implementation of the SSIRWT project, and the inclusion of the GLIS as a model school. The project’s financial management was consistently rated satisfactory. As mentioned above, procurement was also rated satisfactory throughout the life of the project except in December 2011 when the project experienced delays in civil works and procurement of innovation activities. However, these challenges were quickly addressed and the procurement rating was upgraded again to satisfactory by May 2012. Based on these aspects, the implementation agency performance is rated satisfactory.

22

(c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 63. Based on the above assessment, overall borrower performance is rated satisfactory. 6. Lessons Learned 64. Successful policy development and reforms should be linked to school-based innovation and reforms. The documentation and dissemination of project school experiences in developing new curricula and student-centered teaching methods were studied and reviewed at the provincial and national levels. Lessons learned from these experiences have been used to inform the development of TVET guidelines at both the provincial and national levels. In addition, some of the project experience is being used to inform the skills training approaches in the government’s next Five-Year-Plan. 65. Awareness raising of all stakeholders is critical when introducing new reforms. The project supported the development of school marketing plans to promote the new method of skills training being piloted under the project. The marketing plans had a substantial impact in developing school-industry connection related to curriculum development, industry lecturers and increased provision of in-service training requested by industries. 66. Connecting provision of training, buildings, and equipment to progress in developing new curriculum ensures focus on reforms. The project frontloaded technical assistance and training to quickly build capacity of teachers to develop the new curricula and then tied the provision of buildings and equipment to the new courses that were developed. This design moved the focus of the schools away from an initial emphasis on upgrading facilities and equipment toward the reforms efforts envisioned and agreed to in the project design. 67. Exposure of relevant stakeholders to international best-practice can facilitate the development and implementation of a reform. School administrators and teachers repeatedly cited that study tours to countries with experience in CBT and student-centered learning was instrumental in increasing their ability to implement the curriculum reform outlined in the project design. Teachers and school administrators were the main actors in developing and implementing new curricula. Therefore, exposure to the new concepts and learning from other countries was important for carrying the reforms forward within their schools.

68. Strong coordination between project schools and sharing of experiences can lead to strong ownership. Schools routinely shared experiences amongst each other, encouraging more teachers to participate in CBT and increasing their commitment to and interest in the reform process. Sharing of experiences with non-project schools also raised their interest in developing and applying CBT beyond Guangdong province. This has proven to be an effective mechanism to raise awareness and interest in introducing school reforms and using a CBT-type approach.

23

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The borrower and implementing agencies confirmed that they do not have any comments. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) Not applicable

24

Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ million equivalent)

Components Appraisal Estimate

(US$ millions)

Actual/Latest Estimate

(US$ millions)

Percentage of Appraisal

Component 1: School-based reforms and innovation

19.73 19.35 98.00

Component 2: Policy development and capacity building

0.62 0.59 95.20

Total Baseline Cost 20.35 19.94 97.99

Physical Contingencies

0.00

0.00

0.00

Price Contingencies

0.00

0.00 0.00 Total Project Costs 20.35 19.94 97.99

Front-end fee IBRD 0.05 0.05 100 Total Financing Required 20.40 19.99 98.00

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(US$ millions)

Actual/Latest Estimate

(US$ millions)

Percentage of Appraisal

KOREA, Rep of: Economic Development Cooperation Fund

0.40 0.37 92.50

International Bank for Reconstruction and Development

20.00 19.62 98.10

25

Annex 2. Outputs by Component 69. Initial progress towards achieving project outcome and meeting output targets was measured by three PDO indicators and 11 intermediate indicators. This annex uses the combined indicators as listed in the revised results framework to assess output achievements. 70. Component 1: School –Based Reforms and Innovation (IBRD-US$19.73 million; Counterpart-US$22.10 million equivalent). The aim of this component was to introduce reforms addressing issues of inadequate instructional space, a disconnect of current training programs with competencies sought by local industries, a need for more efficient use of public resources at the school level, weak instructional capacity, and a need for improved student assessment and quality assurance in three selected institutions: (i) Guangdong Light Industry Technician School (GLIS); (ii) Guangdong Urban Construction Advanced Technician School (GUCS) and (iii) Yangjiang Advanced Technician School (YTS). The six operational areas were: (i) strengthen the linkage between schools and industry; (ii) improve school management; (iii) introduce an updated modular, competency-based curriculum; (iv) train instructors to use the new curriculum; (v) improve student assessment and quality assurance at the school level; and (vi) expand and upgrade facilities and equipment.

71. The intermediate outcome indicators selected to measure strengthen linkage with industry were: (i) percent of instructors trained in industry attachments, which increased from 10 percent in 2009 to 71.79 percent in 2014 exceeding the target of 50 percent; (ii) percent of industrial experts from enterprises that participated in teaching in schools, which increased from 2 percent in 2009 to 26.28 percent in 2014 exceeding the target of 12 percent; and (iii) percent of students that participated in internship programs, which increased from 42 percent in 2009 to 88.2 percent in 2014 exceeding the target of 85 percent. These are all substantial achievements for the project.

72. The three project schools initially identified 17 total occupational programs for which to develop the modular, competency-based curricula—GLIS-7; GUCS-5 and YTS-5. By the end of the project, the schools had successfully developed curriculum standards, professional standards, and course materials (including textbooks) for all 17 occupation programs. In addition, GLIS had added four more occupational programs to their list of competency-based programs for a total of 21 by project closing. 73. Sub-component 1.1: Strengthen the linkage between schools and industry (IBRD-US$0.79 million; Counterpart-US$0.13 million equivalent). This sub-component was designed to strengthen and institutionalize linkages between schools and local industry to promote a more efficient flow of information through: (i) establishing and strengthening school-industry advisory bodies; (ii) developing standard industry cooperation agreements; (iii) engaging industry representatives in technical advisory committees to set competency standards, develop and adapt new curricula, validate testing and certification systems, and promote sharing of industry equipment and skilled personnel

26

with schools; and (iv) preparing publicity and marketing materials for schools. Outputs from this sub-component are as follows: 74. Number of school-industry advisory bodies established. During project implementation GLIS established three advisory bodies; GUCS established six, and YTS established five. The requirement for the advisory bodies was that 50 percent of the members be from industry in order to ensure adequate industry representation and program relevance to the market needs. These advisory bodies provide schools with guidance on: (i) establishing and/or improving industry collaboration; (ii) developing guidelines for school-industry partnerships; (iii) developing contracts and agreements for school-industry collaboration; and (iv) setting competency standards and developing and adopting new curricula for occupational the selected occupational areas. 75. Number of reports of proposals submitted by the school-industry advisory boards. The number of reports or proposals submitted by the advisory bodies is as follows: GLIS-8; GUCS-16 and YTS-4. These reports and proposals represent the teaching plan proposals for courses that were developed, reviewed and approved by the advisory boards with representatives from industry. The proposals were not approved unless they were judged by the boards to have relevance to the various industries, which ensured continued relevance of the courses and programs being offered by the three schools. 76. Number of completed guidelines for school-industry partnerships. During the project period, there were a total of five guidelines developed for school-industry partnerships. They included obligations for schools and participating enterprises to provide adequate training spaces, promote enrollment into CBT courses to ensure sufficient student participation, ensure student safety and well-being and provision of equipment by enterprises, wherever possible. The project design also included the requirement for schools to develop standard industry cooperation agreements (including for requested training) so as to ensure a formal understanding of requirements and responsibilities of the school and industry relationships. Over the life of the project, the three schools signed a total of 280 school-industry cooperation agreements based on the needs for specialty development--GLIS signed 82, GUCS signed 102 and YTS signed 96 agreements. 77. Number of marketing packages prepared. The project schools prepared marketing packages to promote their schools for both pre-service and in-service training for enterprises. Over the life of the project a total of 38 marketing packages were development—GLIS-15 packages, GUCS-16 and YTS-8. 78. Number of expert and industry technical committees established. The project was to develop industry committees so as to: (i) engage industry representatives and other technical specialists in the establishment of competency standards; (ii) adapt new curricula; (iii) validate testing and certification systems; and (iv) promote sharing of industry equipment and personnel with schools. During the project period, a total of 39 committees were established—GLIS-15; GUCS-15 and YTS-8. See Table 2 for additional cooperation.

27

79. Development and completion of uniform contract templates. During the project period at total of 16 uniform contract templates were developed, covering (among other things) aspects of contract duration, types of possible collaborations, negative list of collaboration projects based on their risks, potential negative environmental impacts and arrangements for collaboration evaluations. Table 2. Additional School-industry Cooperation Cooperation Activity GLIS GUCS YTS Number of industry representatives participating in technical committees

135 78 103

Number of industry instructor employed by project schools

135 78 103

Percentage of school faculty sent to industries for training

49.6 45.2 25.2

Joint practical training facilities created 82 55 35 1 Dual class is one of the school-industry partnership models. Students enrolled in dual classes have double identities; that is, students of TVET institutes and employees for the enterprises. 80. Sub-component 1.2: Improvement in School Management (IBRD-US$0.99 million; Counterpart-US$0.26 million equivalent). This sub-component was to build capacity to ensure the effective use of school resources and production of the learning outcomes needed by a market-led economy through: (i) foreign and domestic study tours; and (ii) provision of long-term and short-term training for school administrators and lead instructors. 81. The intermediate outcome indicator selected to measure improved management capacity was percent of school annual performance goals met which was defined as “percent of school annual plans finished”. During the project period, the percent of annual plans finished increased from 70 percent in 2009 to 96 percent in 2014, exceeding the target of 90 percent. The outputs for this sub-component were: 82. Number of people (managers and staff) trained. During the project period, a large number of school administrators and project school teachers were trained overseas and domestically. See Table 3 for the numbers trained. Table 3. Summary of Trainee Numbers at Project Schools School Overseas

Training1 (person times)

Domestic Training2

(person times)

Campus Training2

(person times)

Total Person Times

GLIS 121 893 1,866 2,880 GUCS 101 2,099 7,871 10,071 YTS 98 1,475 9,302 10,875

1Person times for overseas training general refers to one person trained one time overseas. 2Person times for domestic training can be one person trained several times.

28

83. These training activities contributed to increased understanding of how to develop and teach the competency based curricula as well as improved management of the schools. The improved understanding and acceptance of the competency-based curricula led to the successful development of competency-based curricula for the 21 majors identified for project support. Moreover, the improved management capacity led to improved campus management. This was especially true in the case of information technology. School information systems included one card pass for campus management, multi-media resource databases for teaching resources, computer software for development, teaching and evaluation of CBT curricula as well as information platforms for training and employment, school-industry collaboration, online examinations, and graduate tracking systems. 84. Sub-component 1.3: Curriculum Reform (IBRD-US$0.64 million; Counterpart US$0.28 million equivalent). This sub-component was to introduce a competency-based training curriculum with competency standards established by industry to promote improvements in quality and relevance through: (i) acquisition and/or development of modular CBT curriculum; (ii) acquisition of instructional materials; and (iii) revision of testing and certification instruments with priority given to skills in high demand as determined from consultations with industry.

85. The intermediate outcome indicator selected to measure developed and introduced modular, competency-based curricula and instructional material was “percent of students enrolled in modular CBT curricula (number of students enrolled in modular CBT curricula/total students enrolled in the same specialties)”. During the project period, the percent of students increased from 0 percent in 2009 to 81.66 percent in 2014 exceeding the target of 60 percent. The outputs for this sub-component were: Table 4. Output of Competency-Based Curriculum Reform

Schools Curriculum Plan Curriculum Standards

School-based Textbooks

Published Textbooks

GLIS 23 162 100 6 GUCS 4 64 36 10 YTS 5 57 35 1 Total 32 283 171 17

86. Sub-Component 1.4: Improvement in Student Assessment and Quality Assurance (IBRD-US$0.16 million; Counterpart-US$0.05 million equivalent). This sub-component was to improve quality by helping establish a competency-based benchmark for measuring student and school performance by providing goods and services to: (i) realign the student assessment system to the new CBT curriculum; and (ii) develop instruments for reporting to the public about aggregate student performance on competency exams as an indicator of school quality. 87. The intermediate outcome indicator selected to measure improved assessment and quality assurance was “percent of students assessed with outcome-based student assessment system (number of students assessed with outcomes-based student assessment systems/total students studying in the same specialties)”. During the project period, the

29

percent of students increased from 0 percent in 2009 to 83.1 percent in 2014 exceeding the target of 70 percent. The specific output indicator selected to measure improved student assessment and quality assurance was “number of specialties utilizing CBT curricula”. By the end of the project there were a total of 28 specialties that were using CBT curricula—GLIS-14; GUCS-11, and YTS-3.

88. The 2014 impact evaluation, conducted by the MyCos Company, also found that 11 majors in the three schools were using a CBT assessment to evaluate their students (GLIS-6; GUCS-2 and YTS-3). This assessment includes: (i) self-assessments; (ii) peer-to-peer evaluation for both students and teachers; (iii) evaluation for each course unit; (iv) industry representative assessment of students; and (v) publication of student assessment results. The publication of student assessments are done on a daily and/or weekly basis so that they can know how and where to improve their performance. This is a substantial change from the original practice of assessing students once at the end of the semester. 89. Sub-Component 1.5: Improvement in Instructor Quality (IBRD-US$1.54 million; Counterpart-US$0.57 million equivalent). This sub-component was designed to strengthen instruction capacity and the quality of training by: (i) carrying out pre-and post-assessment of instructor performance with the CBT curricula; and (ii) train instructors through pre-service and in-service measures to use the competency-based curriculum.

90. The intermediate outcome indicator selected to measure improved quality of instruction was “percent of instructors using new CBT curricula (number of instructors using new CBT curricula (for a minimum of 20 hours)/total instructors in the same specialty)”. The percent of instructors using the CBT curricula increased from 0 percent in 2009 to 75.29 percent in 2014 exceeding the target of 70 percent. The total number of instructors trained with CBT curricula was: GLIS-549, GUCS-686 and YTS-384. The overall percentage of teachers able to teach the CBT curriculum at GLIS, GUCS and YTS is 50, 68 and 62 percent respectively. The training of instructors in the CBT curriculum has transformed instruction from teacher-centered to student-centered at all three schools and each of the schools continues to expand this approach through thematic workshops, training sessions and lectures, 91. Sub-Component 1.6: Upgrading of Instructional Equipment and Facilities (IBRD-US$15.61 million; Counterpart-US$20.81 million equivalent). This sub-component was designed to expand instructional facilities where there was evidence of market demand determined in consultation with industry. The expansion included civil works, equipment and technical assistance to: (i) upgrade and expand teaching facilities and related infrastructure; and (ii) equip facilities to conform to the needs of the new modular CBT curricula.

92. The intermediate outcome indicator selected to measure upgraded instructional equipment, facilities and technology was “percent of students utilizing upgraded instructional technology aligned with new CBT curricula (students utilizing upgraded instructional technology/total students studying in the same specialties)”. The percent of students increased from 0 percent in 2009 to 71.97 percent in 2014 exceeding the target of

30

70 percent. The specific output indicators selected are: (i) number of specialties equipped with CBT related equipment; and (ii) number of student places for skill training or learning/number of new enrollments. The number of specialties equipment with CBT equipment were: GLIS-5; GUCS-18, and YTS-8. 93. In addition to the indicators, the project supported the procurement of IT equipment and specialty-specific equipment at all schools as well as the construction of school facilities (see Table 5 for details). An important aspect of the project support for upgrading of facilities at all three schools was the contribution to increasing the schools overall standards. This ultimately led to the reclassification/upgrading of the three schools. Table 5. List of Project Construction

School Construction Project Financing Net additional m2

GLIS Practical Training Building (Bank

financing) Bank 13,850.0

GUCS Practical Training Building (Bank Bank 21,501.3YTS 13-Story Instructional Complex Bank 12,500.0YTS 4 practical training buildings

(counterpart) Counterpart 10,000.0

YTS 1 dining hall counterpart 3,000.0YTS 6 student dormitories counterpart 20,000.0Total 60,851.3

94. Component 2: Policy Development and Capacity Building (IBRD-US$0.22 million; Counterpart-US$0.37 million equivalent; KTF Grant-US$0.40 million equivalent). The objectives of this component were to contribute to the national and provincial policy development and building of a learning culture to guide future school management and policy decision making by focusing on policy studies and on monitoring and evaluation that supported a greater reliance on evidence based decision making in school management and policy development. The component: (i) supported policy research and knowledge sharing at the national level; (ii) improvement in monitoring and evaluation; and (iii) supported capacity building for planning and management. 95. Sub-component 2.1: Policy Research and Knowledge Sharing (IBRD-US$0.0 million; Counterpart-US$0.06 million equivalent; KTF US$0.30 million equivalent). This sub-component provided financing to support: (i) policy studies on key issues in technical and vocation education and training; and (ii) learning from the experience of implementing the project and disseminating lessons learned and the results of the policy studies. Central government agencies played a lead role in policy development with support from the KTF through research and development of a series of policy studies. These studies were to assist in formulating better policies for equipping workers with skills for a growing knowledge based economy. Dissemination of the results of these studies together with dissemination by Guangdong province of the project lessons formed the basis of policy research involving TVET.

31

96. The intermediate outcome indicator selected to measure progress of this sub-component was “number of key policy studies disseminated”. This indicator was achieved as evidenced by the discussion below. 97. At the national level and with support from the Korean grant, the Social Development Department of NDRC produced three policy studies in the areas of: (i) financing of TVET education; (ii) quality improvement of the TVET system particularly as it relates to curriculum development and school-industry collaboration, and (iii) long-term governance structure. During the preparation of the policy studies, NDRC conducted a study tour to Europe to learn from other experiences, facilitated revision of research reports from three consulting firms, and held a completion workshop in December 2014 in Shanghai. During the workshop, the three revised policy reports were reviewed by the TVET experts in China and concluded that the research would be used in the revision of China’s Vocational Education Law. In addition, the NDRC indicated that the policy studies provided a good foundation for their preparation work for the 13th Five Year Plan in the area of vocational education and training. The Social Development Department of NDRC also invited DRC officials from 15 provinces to join this workshop. The results of the policy studies have been disseminated to other provinces, which are expected to also facilitate the preparation of their provincial level 13th Five Year Plan. 98. In addition to the policy research by NDRC, the MOHRSS used the results and lessons from the project to inform the development of two national level guidelines related to TVET. These guidelines are: (i) “Guideline for the Development of National Training Standard for High-skilled Talents”; and (ii) “Technical Specifications for the Development of Integrated Curriculum”. 99. At the provincial level, the Provincial Department of Human Resource and Social Security issued guidelines that were informed by the project entitled “Suggestions on Strongly Promote the Connotation Development of Technical Education in Guangdong” in November 2014, which is comprised of experiences and suggestions for the development of CBT curricula. The Guangdong PMO organized capacity building activities and coordinated knowledge sharing activities among the three project schools and with non-project schools in the province. They also worked with the project schools to produce a collection of the competency-based curriculum reform outcomes and published the “Distinguished Papers on World Bank Loan, Guangdong Technical and Vocational Education and Training Project (2009-2014) (ISBN 978-5100-5567-6), which included 35 papers. 100. At the school level, there were many activities related to policy research and knowledge sharing that included essays, survey reports, textbook development and training plans. Specific outputs were: (i) project experience and research results published - GLIS-130 volumes, GUCS-21 volumes and YTS-26 volumes; (ii) six policy research seminars with industry and other provincial schools in areas related to the multi-evaluation system, curriculum and textbook development; and (iii) seven seminars related to key policies shared with provincial and national decision makers.

32

101. Sub-component 2.2: Improvement of Monitoring and Evaluation (IBRD- US$0.22 million; Counterpart-US$0.15 million equivalent; KTF US$0.10 million equivalent). This sub-component provided financing to lay the foundation for moving to evidence-based management and policy development by providing services to: (i) design and implement project beneficiary assessments and tracer studies; and (ii) develop and implement a school-based management information system for monitoring school performance. 102. The intermediate outcome indicator selected to measure progress of this sub-component was disclosure to the public of student learning outcomes, e.g. number and percentage of students passing certificate exams, employment. This was accomplished through schools’ respective websites. 103. In 2014, the PMO hired an independent firm to conduct an impact evaluation of the project. The province and schools learned valuable lessons with regard to designing and conducting IEs, and how to analyze the data. The PMO used some of the results to inform the borrower portion of the ICR and will continue to review the results to inform provincial TVET reform policies. The project also included two beneficiary studies: one was conducted in 2012 at project mid-term and one in 2014 prior to project completion. In addition to the overall project beneficiary studies, the school departments responsible for student management and graduate employment have adopted and institutionalized the beneficiary study research methodology in their graduate tracer system. These are all substantial achievements of the project. 104. The three project schools also developed management information systems. The designs within institutions vary and include but are not limited to: (i) education information management platform; (ii) network teaching resources platform; (iii) graduate employment tracking service system; (iv) school-industry collaboration information platform; (v) multi-evaluating system; (vi) examination database; (vii) teaching quality assessment system; (viii) distant teaching platform; and (viii) graduate employment tracking system. 105. Sub-component 2.3: Enhancing Project Management and Coordination (IBRD-US$0.0 million; Counterpart-US$0.16 million equivalent; and KTF US$0.0 million equivalent). This component used counterpart funds to support the provincial PMO in the coordination school activities that would benefit from uniformity of approach and cost savings in design, procurement and implementation. The intermediate outcome indicator selected to measure progress of this sub-component was “number of activities completed”. The total number of activities completed by the PMO and school implementation offices was approximately 87. These activities included TVET reform workshops and seminars as well as procurement and financial management training. Overall capacity at the provincial level and school level is judged to have increased substantially.

33

Annex 3. Economic and Financial Analysis Project context 106. At the time the project was designed, Guangdong needed to introduce changes in the structure of employment and reduce the importance of older labor-intensive industries using large quantities of low wage, unskilled labor. Enrollments in higher education had surged since 2000, increasing from 299,500 to 1,008,000 in 2006. As mentioned previously, wages had begun to rise, with wages in manufacturing and services outpacing those in agriculture. Demand for technical and vocational education was strong as measured by the expansion of enrollments in TVET schools. Economic and Financial Analysis at the PAD stage 107. The economic analysis in the PAD sought to:

(i) Demonstrate the demand for TVET in the province (ii) Examine the costs and benefits of the proposed project interventions.

108. According to the PAD employment trends were shifting away from agriculture to manufacturing, information and communication technology (ICT) and services. At that time, annual growth rates in wages were as follows: manufacturing wages grew by eight percent, ICT wages grew eight percent, and wages in the services sector grew by 9 percent. In addition, at the time of the PAD, the project schools were growing as were the employment rate and entry level wages of their graduates. (See Table 6 extracted from PAD below). Table 6. Average annual increase in project school graduates, placement rates and starting wages, 2003-2007 2003 2004 2005 2006 2007 Total graduates 516 631 880 1026 1311 % graduates employed within 6 months 78 77 79 85 87 Entry level monthly wages 700 708 747 869 919

109. The cost-benefit analysis conducted in the PAD relied on standard modelling assumptions and compared the wages of graduates from project schools to assumed (yet conservative) counterfactual wages of graduates of non-project schools. According to the model, the first cohort would not start until 2011 and it would take until 2013 for all 3 cohorts to be enrolled in project schools. In addition, the model assumed that only 10 cohorts would benefit from the project (another conservative assumption). The stream of benefits was assumed to accrue from graduation at age 22 to retirement at age 60. With these assumptions in place, according to the cost-benefit analysis in the PAD the internal rate of return ranged from 11 to 19 percent (depending on additional robustness checks). 110. Similarly, an evaluation was designed which would compare graduates of project schools to graduates of non-project schools over time along key dimensions such as employment rates after six months and starting salary. However the original approach had

34

to be modified due to constraints in finding comparable non-project schools. In the end, the evaluation was conducted using outcomes for students from the project schools enrolled in the CBT modules (those developed under the project) relative to the outcomes of students enrolled in non-CBT modules (those that already existed before the project).14

Economic and Financial Analysis at the ICR stage 111. Against this background, the economic and financial analysis in this ICR examines trends in key outcome indicators as reported by administrative sources and as documented by the impact evaluation.15 Similarly, given the fact that non-project schools are no longer a relevant comparison group for project beneficiaries, this section does not attempt to recreate the cost-benefit analysis in the PAD. Instead, it assesses whether the assumptions in the PAD materialized and, if so, to what extent. Specifically, these assumptions centered on the growing demand for TVET graduates as measured by the number of graduates being produced by these schools and on the assertion that starting wages for graduates would continue to rise. As noted in the main text, this economic and financial analysis cannot and does not attempt to attribute causality to the project. Findings from administrative data 112. The ICR team collated administrative data on the number of graduates as well as their employment rate after six months and the starting salary they received. The number of graduates in the project schools fluctuates over time. In the case of the GLIS, data are not disaggregated by the type of courses students took. The number of graduates in 2014 is 3,327, substantially smaller than the number of graduates in 2009 (4,881). It is therefore no possible to assert whether or not graduation rates of students enrollment in in CBT courses was higher than for non-CBT students. In contrast, the number of CBT graduates at YTS grew over the period 2009-2014, while the number of non-CBT graduates remained fairly unchanged: there was a sharper increase in rates of graduates for CBT graduates relative to non-CBT graduates (albeit the absolute numbers were much smaller). When the number of overall graduates shrunk, more students enrolled in CBT graduated compared to those enrolled in non-CBT courses (See figure 1). 113. However, the trends in CBT and non-CBT graduates followed different paths. In the case of GUCS, there was a growth in the number of graduates between 2009 and 2012 followed by a sharp decline in 2013, followed again by an equally sharp increase in 2014. In both episodes, the growth rate of CBT graduates was faster than that of non-CBT graduates. Again – the absolute number of graduates from CBT courses is smaller than that

14 As mentioned before, the PMO hired an independent firm to conduct this evaluation. The baseline study was conducted in 2010 and follow-up data were collected in 2014. 15 For a full discussion of the impact evaluation, please see World Bank Guangdong Vocational and Technical Education and Training Project Impact Evaluation Report. The report notes that: (i) the response rate was 46.5 percent for 2010 baseline survey and 45.5 percent in 2014; (ii) GUCS reported that all graduates participated in CBT and that data for comparison purposes was not available; and (iii) continued changes and improvements to subject structure and content did not allow for a simple “before-and-after” comparison as the variables had changed over the life of the project.

35

from non-CBT courses. It should be noted that GUCS is much smaller than in the case of the YTS school mentioned earlier. (See Table 1).

Figure 1. Total Graduates by School

48814523

4969

3227 31133327

0

1000

2000

3000

4000

5000

6000

2009 2010 2011 2012 2013 2014

GLIS

697

9741109

1445

717

1517

502

756

9981145

405

1023

0

500

1000

1500

2000

2009 2010 2011 2012 2013 2014

GUCS

GUCS Non‐CBt GUCS CBT

278 235 278376 385 364

1077

770

1117

1274

1046 1053

0

200

400

600

800

1000

1200

1400

2009 2010 2011 2012 2013 2014

YTS

YTS CBT YTS Non‐CBT

36

Table 7. Percentage of graduates from CBT and non-CBT courses Panel A

YTS

Growth in graduation between 2009-2012

Decline in graduation between 2012-2014

Non-CBT graduates 18 -17

CBT graduates 35 -3

Panel B

GUCS

Growth in graduation

between 2009-2012

Decline in graduation

between 2012-2013

Growth in graduation

between 2013-2014

Non-CBT graduates 107 -50

115

CBT graduates 128 -65

152

114. The second assumption centered on the growth rate of wages in Guangdong. Figure 2 shows the growth in average monthly wages for Guangdong as a whole between 2008 and 2013. It also shows the starting wages of project school graduates from CBT and non-CBT courses between 2009 and 2014. In terms of evaluating the project, it is important to remember that CBT courses began operation around 2011. Figure 2. Monthly Wages

1087 1211 1275 14011694

2089

20632249

25662958

34513803

19102154

24362799

31683564

1733 18632055

23482614

3009

0

500

1000

1500

2000

2500

3000

3500

4000

2008 2009 2010 2011 2012 2013

Guangdong's average monthly wages (RMB)

Agriculture (farming)

农业Manufacturing

制造业Construction

建筑业Services (Hotel & Restaurant)

服务业（酒店&餐馆）

Project start

37

Monthly Wages (continued)

Note: GLIS data not reported as it is not available by CBT and non-CBT graduates. All wage data are in nominal terms. 2011 is start of project.

115. Comparing growth rates of wages of project school graduates against the growth rates of wages in the province by sector of employment would suggest that some graduates – particularly those who graduated from CBT courses taken at YTS experienced sharp increases in starting salaries over the period 2009-2014. Across the board, GUCS graduates also experienced growth in wages. While this was slightly lower than the average of Guangdong, CBT graduates reported greater growth than non-CBT graduates. For GLIS graduates, wage growth was not as strong a growth as that posted by the graduates of the other two schools. CBT graduates from this school did 0.7 percentage points better than non-CBT graduates. This comparison is necessarily limited and not claiming a causal connection. It is important to note that the data from the schools pertains to 2009-2014 while the data from the administrative sources is 2008-2013 (Figure 3). As such this is not a like-for-like comparison at the same point in time but the resulting numbers are suggestive.

1,159

1,5001,813

2,000

2,370

2,700

1,2851,568

1,804 1,914 2,0212,311

0

500

1000

1500

2000

2500

3000

2009 2010 2011 2012 2013 2014

YTS: Entry level monthly wages

CBT Graduates Non‐CBT Graduates

20822336 2364 2364 2413

2820

20102200 2100 2210 2350

2710

0

500

1000

1500

2000

2500

3000

2009 2010 2011 2012 2013 2014

GUCS: Monthly wages

CBT Non‐CBT

38

Figure 3. Wage growth rates (secondary and technical graduates combined)

Note: Data from schools is annualized over the period 2009-2014 while the industry level data are annualized over 2008-2013. Monthly wage data was reported for 2009 and 2014. Annual rates of growth reported.

116. The relative success of the subset of YTS graduates might be explained by that fact that some data suggests that the school had a higher rejection rate than other project schools. In fact, data suggests that over time, YTS became more selective. Perhaps this reflects an early and accurate assessment of candidates’ potential, which in turn manifested itself in more successful job-market candidates. Figure 4. YTS rejection rate

117. One last aspect that can be assessed from administrative data is the percent of graduates employed after six months. Data on this indicator are somewhat difficult to interpret because: (i) employment levels for all schools are already high to begin with; and (ii) there seems to have been no change in employment rates over time for graduates from

5.6%

6.3%

9.1%

11.2%

12.4%

12.6%

14.7%

16.0%

16.9%

17.3%

18.4%

26.6%

0% 5% 10% 15% 20% 25% 30%

GLIS Non‐CBT

GLIS CBT

GUCS Non‐CBT

GUCS CBT

All industries in Guangdong

ICT

Services

YTS Non‐CBT

Manufacturing

Construction

Agriculture

YTS CBT

Growth in monthly wages, annualized

5%

7%

5%

8%

9%

11%

0.00%

2.00%

4.00%

6.00%

8.00%

10.00%

12.00%

2009 2010 2011 2012 2013 2014

YTS: Applicant rejection rate

39

GUCS - virtually all of them get a job (99 percent of CBT graduates) versus 97 percent of non-CBT graduates). This does not vary before or after the start of CBT courses. In the case of GLIS, the employment rates grew at a steady rate of 2 to 3 percent from 2009-2014, suggesting that there was little deviation from the pre-existing trend over the life of the project. Figure 5. Employment trends

85%

92%

88%

93%96% 96%

88% 88%

93% 93% 93%95%

75%

80%

85%

90%

95%

100%

2009 2010 2011 2012 2013 2014

YTS: Empoyment rate within 6 months

CBT Graduates Non‐CBT Graduates

96%

97%

97%

98%

98%

99%

99%

100%

2009 2010 2011 2012 2013 2014

GUCS Employment rate within 6 months

CBT Non CBT

40

Employment trends (continued)

A summary of findings from the impact evaluation report 118. An assessment of project impact was undertaken by surveying graduates from project schools in 2010 and 2014. The survey covered those who had benefited from CBT courses and those who had not. The data below reflect information provided by survey respondents and may not be identical to the information reported above which is from administrative sources. The evaluation found the following:

(i) Project schools enrolled more students from socially disadvantaged families. Comparisons of the family background of graduates in 2010 and 2014 show that a substantial percentage of students enrolled in the three project schools mainly came from farmer and rural migrant worker families and that this percentage increased over time from 38 percent to 52 percent.

(ii) Six month employment rates of graduates of college technical majors who

received CBT were higher than those who did not receive such coursework. With respect to college technical majors in project schools, the employment rate of graduates who received competency-based coursework in YTS after six months was 90 percent, higher than that of college technical graduates who did not receive competency-based coursework (87.9 percent). Similarly, the employment rate of graduates who received competency-based coursework at GLIS after six months was 88.9 percent, higher than that of college technical graduates who did not receive competency-based coursework (86.3 percent).

(iii) Evidence on the six-month employment rates of graduates of secondary technical majors who received competency-based coursework is more mixed. With respect to secondary technical majors in project schools, the employment rate of secondary technical graduates who received competency-based coursework in YTS was 77.8 percent, the same as that of secondary technical graduates who did not receive competency-based coursework (78.4 percent). Meanwhile, the employment rate of secondary technical graduates

86.05%

90.80% 91.59% 92.56%95.07%

98.91%

75.00%

80.00%

85.00%

90.00%

95.00%

100.00%

2009 2010 2011 2012 2013 2014

GLIS: Employment rate within 6 months

41

who received competency-based coursework in GLIS was 69 percent, lower than that of secondary technical graduates who did not receive competency-based coursework (82.7 percent).

(iv) Overall monthly income after six months was higher for both college and secondary technical graduates. Monthly income for college technical graduates of the class of 2013 from the Guangdong project schools six months after graduation was RMB 3,230, higher than that of baseline college technical graduates of the class of 2010 (RMB 2,475). The monthly income for secondary technical graduates of the class of 2013 after six months was RMB 2,945, also higher than that of baseline secondary technical graduates of the class of 2010 (RMB 2,187). With monthly income of the class of 2010 as a benchmark and adjusted for inflation, monthly incomes for advanced and secondary technical graduates of the class of 2013 after six months were RMB 2,911 and RMB 2,654 respectively, up RMB 436 and RMB 467 respectively compared with the class of 2010.

(v) As with employment rates for secondary majors, the data suggest a mixed story for wages of college graduates who took CBT courses compared to those who did not. With the exception of GLIS, CBT graduates reported higher monthly wages than non-CBT graduates: monthly income for college technical graduates who received competency-based coursework from YTS after six months (RMB 2,744) was higher than that for college technical graduates who did not receive competency-based coursework (RMB 2,383). In terms of GLIS, monthly income after six months for college technical graduates who received competency-based coursework was RMB 3,149, lower than that for college technical graduates who did not receive competency-based coursework (RMB 3,302).

(vi) Secondary graduates who took CBT courses reported higher monthly wages than non-CBT graduates in all schools. Secondary technical graduates who received competency-based coursework from GLIS reported that their wages after six months was RMB 3,681, higher than that for secondary technical graduates who did not receive competency-based coursework (RMB 3,099). Similar patterns of monthly income for secondary technical graduates were reported by those who attended YTS: graduates who received competency-based coursework reported that their wage after six months was RMB 2,751, higher than that for secondary technical graduates who did not receive competency-based coursework (RMB 2,565).

(vii) The evaluation also found that project beneficiaries had fewer difficulties

during their job search, got more offers and experienced lower costs in finding a job. Specifically, college graduates obtained four job offers in 2014 compared to two in 2010, while secondary graduates received 3 job offers on average in 2014 compared to two.

42

Overall assessment 119. Taken together, the evidence would suggest that wages improved for graduates at both the secondary and college levels. In particular, wages after six months for secondary level graduates who took CBT courses were between 7 and 18 percent higher than those of graduates who took non-CBT courses (depending on the school). The one exception is wages for college graduates at GLIS whose wages were actually 4 percent lower. However, even for this group, employment rates were higher than those of their peers who did not take part in CBT courses. 120. The evidence also points to different impacts for college graduates than for secondary graduates. In GLIS and YTS, college graduates of CBT courses had higher employment rates than others. While for secondary graduates, there was either no impact or a slight decline (depending on the school). 121. Possible explanations for the variation across project schools in terms of outcomes include the fact that in some schools, roll out of reforms may have been slower than anticipated. This may have been due to capacity constraints of teachers and administrators to develop new textbooks and learning materials, and a lag time inherent in learning how to deliver the new CBT courses. Financial Analysis: Efficiency 122. Efficiency can also be assessed by examining how well funds were spent. The table below captures what percent of the funds allocated to each component were actually spent:

Table 8. Percent of funds committed/spent (excludes US$0.05 million Front-end fee)

Allocated Actual Percent Component 1 19.73 19.35 98.00 Component 2 0.62 0.59 95.20 Total 20.35 19.94 97.99

123. Efficiency of the project is also assessed using unit rate costs. Each of the figures capture unit costs at each of the schools participating in the project. There are several patterns in the data:

(i) Publication costs are fairly uniform across the project schools ranging from 26,000 – 30,000 per textbook. Equipment costs are fairly uniform as well – ranging from 4,800- 5,175;

(ii) Construction costs are very different across schools ranging from 1,900 to 3,150 per square meter. But this may be a function of how the costs are reported – some capture more components than others; and

(iii) Teacher training costs also have quite a wide range – 1,000 – 3,000 per person per time.

43

Unit Cost comparison for GLIS Activity

Unit Cost Explanation of what costs included Comparable Program Unit Costs 1

Construction (cost per m2)

3,150 Construction, décor/painting, wiring/plumbing (floor height 1st and 2nd floor is 5.4 meters

In accordance with provincial standards.

Teacher training

3,025.63 per person/time

Training, transportation, accommodation

Equipment

5,175 per computer (Dell laptops for teachers)

Computer equipment/furniture expense

Textbook publishing

29,800 per textbook Layout, cover design and printing

1If the PMO or schools have any other unit costs that can be compared based on government loans or programs that they have with the project schools and/or other schools in Guangzhou or Guangdong province. 2Based on the list of equipment provided in the borrower portion of the ICR and any comparisons that can be made with other PMO purchases. Perhaps the easiest would be computer equipment. Unit Cost comparison YTS Activity Unit Cost Explanation of what costs included Comparable

Program Unit Costs 1

Construction (cost per m2)

1873 per m2 Including: 1、Construction RMB/m2: 1776 2、Field visit and design RMB/m2: 44 3、Supervision and management fee RMB/m2: 16 4、Measurement fee RMB/m2: 10 5、Bidding process RMB/m2: 4

In accordance with provincial standards.

44

6、Other RMB/m2: 23 Teacher training 1,014 Training expense, accommodation and travel, meals (domestic and

international teacher training)

Equipment2 4,877 RMB/unit

Auto repair training equipment: 116, Model production equipment: 38, digital equipment: 17, pedagogical equipment: 147, projector: 150, laptops/desktops: 260, School monitoring network equipment: 1,630（not including textbooks)

Textbook publishing Not available None

1If the PMO or schools have any other unit costs that can be compared based on government loans or programs that they have with the project schools and/or other schools in Guangzhou or Guangdong province. 2Based on the list of equipment provided in the borrower portion of the ICR and any comparisons that can be made with other PMO purchases. Perhaps the easiest would be computer equipment. Unit Cost comparison for GUCS

Activity

Unit Cost

Explanation of what costs included

Comparable Program Unit Costs 1

Construction (cost per m2) 2,432.38 Construction expense In accordance with

provincial standards

Teacher training 1,670 Training, travel, meals, transportation

Equipment2 5,384 G25 设备费用（351 units台）

Textbook publishing 25,600 Publishing fee 1If the PMO or schools have any other unit costs that can be compared based on government loans or programs that they have with the project schools and/or other schools in Guangzhou or Guangdong province. 2Based on the list of equipment provided in the borrower portion of the ICR and any comparisons that can be made with other PMO purchases. Perhaps the easiest would be computer equipment.。

45

Annex 4. Bank Lending and Implementation/Supervision Support (a) Task Team members

Names Title Unit Responsibility/

Specialty Lending Dingyong Hou Sr. Education Specialist ECSH2 Team Leader Arvil Van Adams Consultant AFTSE TVET specialist Fook Yen Chong Consultant EASHD TVET specialist Xiaowei Guo Sr. Procurement Specialist EASR2 Procurement

Jian Hou Program Assistant EACCF Administrative Support

Feng Ji Sr. Environmental Specialist EASCS Safeguards

Haixia Li Sr. Financial Management Specialist

EASFM Financial Management

Margaret Png Lead Counsel LEGLE Lawyer

Cristobal Ridao-Cano Country Sector Coordinator EASHS Education Specialist

Sabrina Gail Terry Program Assistant EASHD Administrative Support

Chaogang Wang Sr. Social Development Specialist

MNSSU Safeguards

Liping Xiao Sr. Education Specialist EASHE Education Zhihua Zeng Sr. Economist AFTFE Economic AnalysisKin Bin Wu Sr. Education Specialist EASHE Education Supervision/ICR Xiaowei Guo Sr. Procurement Specialist EASR2 Procurement

Haixia Li Sr. Financial Management Specialist

EASFM Financial Management

Chaoying Liu Results Measurement Specialist

CDIAS Monitoring and Evaluation

Tao Su Program Assistant EACCF Administrative Support

Sabrina Gail Terry Program Assistant EASHD Administrative Support

Yiren Feng Sr. Environmental Specialist GENDR Environmental safeguards

Dingyong Hou Sr. Education Specialist GEDDR Education Rahman Mokhlesur Sr. Operations Officer GEDDR Education Han Xu Consultant GEDDR Education Jin Zhang Consultant GEDDR Education

46

Haiyan Wang Sr. Finance Officer CTRLN Financial Management

Liping Xiao Sr. Education Specialist GEDDR Task Team Leader Anya Wang TVET consultant GEDDR TVET Franco Russo Operations Officer GEDDR ICR Co-author Sandra F. Beemer Consultant GEDDR ICR Co-author Amer Hasan Education Economist GEDDR ICR Economist (b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks US$ thousands (including

travel and consultant costs)Lending

FY06 2.45 47.96 FY07 25.71 133.00 FY08 66.03 393.45 FY09 19.08 117.82

Total: 113.27 692.23 Supervision/ICR

FY10 12.96 60.23 FY11 18.85 76.60 FY12 14.25 94.52 FY13 18.62 65.47 FY14 24.93 80.10 FY15 8.09 61.18

Total: 97.70 438.10

47

Annex 5. Beneficiary Survey Results (Summary from BA submitted by the PMO. Summary is verbatim to preserve authenticity. The full report is available on file) Benefits to teachers: Various trainings delivered by the project provided teachers with more training

opportunities. The data collected through questionnaires showed that most of the teachers participated in the project trainings, which were quite effective and contributed to broaden their horizons.

When teachers were asked about how helpful the trainings were: 66.97 percent of the teachers indicated that the trainings were helpful or very helpful, while 33.3 percent regarded them as helpful. Conclusions could be drawn that the trainings were quite effective and recognized by the teachers.

Results of the field survey also supported the conclusions drawn from questionnaires. The field survey showed that teachers got more opportunities for various trainings. Some training focused on advanced education concepts and pedagogy while the others emphasized on the technical placement. The increased training opportunities and diversified types of trainings broadened their horizons. The feedback all showed that various trainings contributed a lot to the improvement of their quality and capacities.

Teachers’ ability in curriculum development was strengthened: First of all, the degree of participation of teachers in school-based curriculum development was very high, which helped to enhance their ability of curriculum development. When they were asked about whether they participated in the school-based curriculum development, 82.05 percent teachers said yes while only 17.95 percent said no. In general, most of the teachers took part in the school-based curriculum development.

Teachers’ curriculum development capacity was improved significantly: When they were asked about how much their capacity in curriculum development was enhanced, 68.42 percent of the teachers held positive attitudes and believed there was much more improvement made; 28.95 percent thought there was improvement; only 2.63 percent thought there was no improvement made in their capacity of curriculum development.

It was found that teachers involved had made great improvement in their capacity of curriculum development through attending different kinds of trainings. In this process, teachers have constantly improved their ability in curriculum development; that is, starting from having no idea about curriculum reform to being able to develop practical curriculum modules.

The data showed that the respondents held positive attitudes on their capacities in delivering this new pedagogy. Furthermore, most of them believed they were much more capable of adopting the student-centered teaching approach.

The improvement of utilizing modern education technology and multimedia showed that 92.11 percent teachers believed that much or more improvement was made in this aspect. The survey data indicated that most of teachers held positive attitudes and thought they had made good progress in utilizing modern education technology and multimedia and benefited a lot from this project.

Teachers’ practical ability has been improved: Only 2.63 percent of teachers said there was little or no improvement at all.

48

Teachers’ capacity for innovation has been enhanced. Most teachers showed positive attitudes towards the improvement made in their innovation ability. Students thought highly of teachers’ abilities, generally reflecting that teachers’ capacities had been significantly improved. In particular, teachers developed better practical skills to provide better guidance for students as they carried out continual enterprise survey and went to enterprises for more placements.

Benefits to students: With the implementation and progress of the project, great improvement was made to

the school teaching conditions and equipment. The equipment purchased with the project loan was put into use gradually. Only 10.05 percent of student felt equipment was somewhat able to meet and 5.03 percent not at all able to meet students’ learning needs. Students generally recognized that the teaching conditions and equipment were significantly improved. The facilities purchased with project funds have been utilized in teaching successively.

The curriculum used to focus on subject knowledge before the project implementation. Teaching was mainly lecturing while students only were passively receiving and participating in the teaching. After project implementation, CBT curriculum was delivered in selected majors in three project schools with emphasis was put on student-centered teaching. When students were asked about the fact that the teacher considered their interests during teaching, only 0.5 percent of the students disagreed.

With the implementation and progress of this project, the content of the courses got more relevant to work, which laid down a good foundation for students’ future work. Students generally held a positive attitude towards the teaching and curriculum, recognizing that the teaching and curriculum reform contributed to improve the relevance of curriculum and teaching to actual production and students’ adaptability to their future jobs.

The implementation of CBT curriculum promoted the enhancement of student occupational quality and competence: When students were asked about how much the teaching helped them to improve their competence and quality, the population mean was between “Helpful” and “Very helpful”.

Benefits to enterprises: After the project implementation, the school-industry collaboration and the quality of

talent training has been greatly improved. As a result, enterprises’ demand for talents was well met. The deepened school-industry cooperation contributed to the great improvement of training quality so as to meet the enterprise’s demand for qualified employees.

The survey on enterprises’ satisfaction about the graduates of pilot classes was between ‘Satisfied’ and ‘Very Satisfied’. The valuation on the graduates’ basic quality showed that enterprises generally thought highly of them on their professional theoretical and practical knowledge and skills, social morality, working attitudes, loyalty, and compliance with the industry and enterprise regulations.

By expanding and deepening the school-industry cooperation scope, enterprises got more and more engaged in student trainings, while the schools’ capacity in serving the enterprises was also strengthened. As a result, the enterprises got more benefits through

49

school-industry collaboration. Enterprises’ satisfaction towards project schools was between Satisfactory and Highly Satisfactory.

Enterprises were willing to establish partnership with TVET schools in other areas apart from talent cultivation such as providing in-service trainings, technical development and so on. School-industry cooperation was deepened and expanded from the original personnel training to a wider range of collaborations. In addition, the technical consultancies, employee training and other services provided by the TVET institutions were all regarded highly by the enterprises.

50

Annex 6. Stakeholder Workshop Report and Results Not applicable

51

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 124. The Technical & Vocational Education and Training Project between Guangdong Provincial Government and World Bank was signed on July 23rd, 2009. The project is under the cooperative implementation by the Human Resource & Social Security Bureau of Guangdong Province (HRSS), which is at the provincial level as well as three technical vocational schools, which are Guangdong Light Industry Advanced Technical and Vocational School (later upgraded to be the Provincial Light Industry Technician School, hereafter as Provincial Light Industry), Guangdong Urban Construction Advanced Technical and Vocational School (later upgraded to be Provincial Urban Construction Advanced Technician School, hereafter as Provincial Urban Construction) and Yangjiang Technician School (later upgraded to be YTS, hereafter as Yangjiang Advanced Technician). The pilot schools of the project have conducted a series of school-based reforms and innovations on the intermediate and advanced skill development. The total investment of the project is US$45.62 million, including the World Bank loan of US$20 million, a counterpart fund of US$25.52 million and a grant of US$400,00016 from South Korea. It has taken 5 years to implement the project, and it has been successfully completed on December 31, 2014. 125. With the support and assistances from the World Bank project TTL and other team experts, the project has achieved significant outcomes after 5 years’ practice and exploration. The project implementation schedule has met the expectations, and it has met or exceeded all the targets according to the planned indicators. During the past 5 years, the three schools have not only successfully accomplished the one basket of reform tasks with the school-enterprise cooperation as the central line, but also carried out the innovations to achieve significant progress in many aspects of the work based on the previous foundations. The quality and the quantity have both exceeded the PAD target a lot, and there is a significant improvement in the construction of the software and hardware facilities. The size of the school, the impact of vocational education and the interior development, have been improved greatly. 1. The Provincial HRSS Bureau has paid much attention to the project implementation, and set up a provincial PMO for project management with establishing a sound management framework. During the project implementation, the PMO has carried out the following: One, produce various financial management and procurement management rules under the established system and effectively facilitate the project implementation; Two, focus on the cost-effectiveness, all the tasks in goods, civil works, reform as well as the newly added tasks under the project have been completed. The civil works implementation has met the safeguard requirements. The project requires the participating schools to complete two major components: One, school-based reform and innovation; Two, policy development and capacity building. Specific task requirements are also raised under these two major components.

16 As mentioned earlier, US$300,000 were allocated for the three national studies carried out by NDRC and US$100,000 for the IE.

52

126. Component One. School-Based Reforms and Innovation. The tasks completed by the three schools were:

(i) Set up and strengthen the Consulting Committee for the school-industry cooperation. In the past 5 years, the three schools have established 14 school-industry consulting institutions. 5 of them were established by YTS, 6 of them were established by the GUCS, and 3 of them were established by the Provincial Light Industry;

(ii) Compose the standardized school-industry cooperation agreement. In the past

5 year, the three schools have signed 280 school-industry cooperation agreements according to the needs of specialty development. 82 of them were signed by the GLIS, 102 of them were signed by the GUCS and 96 of them were signed by YTS (See Appendix 1); and

(iii) Employ the industry representative to join the technical consulting committee to set up capacity standards; develop and adopt the new standards to verify the assessment and certificate evaluation system; promote the sharing of the equipments and staff between the school and the industry.

127. During the period of project implementation, the three schools have employed the industry representatives to join the school-industry cooperation, and developed a training team including the instructors from both schools and industry. To be more specific, there were 135 industry representatives joining the cooperation activities in the GLIS; 78 for the GUCS; and 103 for YTS.

128. Also, the GLIS has made 23 training plans for the professional technicians; the GUCS has made 4 training plans; Yangjiang Advance Technician School has made 5 plans. The GLIS has made 162 curriculum standards, 101 of them are relating to the World Bank project; the GUCS has made 64 curriculum standards, 46 of them are relating to the World Bank project; YTS has made 57 curriculum standards, 24 of them are relating to the World Bank project. 129. With the lead by the GUCS, the three schools have discussed and agreed to carry out 17 management regulations on their campuses GUCS summarized these 17 regulations into the “Summary of School-Industry Cooperation Management Regulations”. The consultants on law affairs were also employed to guide the school-industry cooperation at the same time. The schools also shared staff and equipment between schools and industry. In addition, GUCS established an information platform of school-industry collaboration meant to strengthening the information communication between the school and the enterprise, assisting the two-way options of cooperative target between the school and the industry, and promoting the practical training and employment for the graduate.

53

130. The innovations conducted by the participated schools. Schools did the following:

(i) set up school-industry cooperation;

(ii) collaborative training with the dual-system between the school and the industry which is a concept of “dual-system” that comes from a document of the Guangdong HRSS Bureau, which means a combination of the training system in the industry and the training system in the schools;

(iii) jointly developed the team of “double-instructors” which meant employing the technicians and advanced technician talents from the industry to be the part-time instructors, which is in accordance with the rules and principles of personnel training of the technical & vocational schools GLIS inviting 135 industry instructors; the GUCS 78 industry instructors; and YTS 103 industry representatives;

(iv) jointly participate in the teaching with 43 enterprises having participated in the teaching in GUCS, and 15 in YTS;

(v) jointly establish practical training bases with schools providing the sites and equipment which could also be offered by the enterprises with GLIS building up 82 training bases; GUCS 55 training bases; and YTS 35 training bases;

(vi) jointly develop the project research and technical training service with GLIS

cooperating with enterprises to apply for 36 national patents, 28 of them were applied by the teachers and 8 of them were applied by the students and GUCS conducted 28 service projects such as the technical consulting, technical training and technical assessment;

(vii) jointly set up the technical master studios that brought high-end talent to train

the advanced technicians; and

(viii) jointly implement the teaching assessment with industry. 131. Strengthening the training, enhancing the management capacity. The agreed tasks related to this were: (i) the designs and implementations of domestic and overseas study tours; and (ii) long-term and short-term training provided for the school’s administrators and specialty leaders. The completions implemented by the three schools were: (i) training abroad; administrators were sent to the countries or regions which have advanced vocational education systems such as Germany, the United States, the UK, Singapore and Hong Kong SAR, China to learn the vocational education mode; (ii) participating the training and education training at home, including the training on the practical capacities received by the school teachers in the enterprises; and (iii) training on campus which included two ways: inviting experts to conduct trainings on campus,

54

establishing a team of trainers (instructors) which is composed of the experts who have been sent out of the province for training or been widely recognized within the school, and they organized and provide highly relevant professional trainings according to the different categories. Table 9. Total number of trainees in the three schools (person times)

School Total Training abroad

Training at home

Training on campus

YTS 10,875 98 1,475 9,302

Guangdong Light Industry Technician School

2,880 121 893 1,866

Guangdong Urban Construction Technician School

10,071 101 2,099 7,871

Total 23,826 320 4,467 19,039

132. The school administrators and specialty leaders are two crucial leading groups in the school, who are playing the roles of coordinating and modeling in every important working field. There is no doubt that giving trainings to these two groups of people is very crucial to enhance the school’s management capacity. Over the past 5 years, all of these three schools paid high attention to the training to the school administrators and specialty leaders, with YTS in particular, who sent as many as 202 school administrators for training, and as many as 68 specialty leaders for training. It can be seen that with the strong support of the World Bank project loans, the trainings to the school administrators and specialty leaders have been intensively implemented in all of the three schools, which laid a solid foundation for the significant improvement in the school’s management capacity.

133. The innovation extension tasks conducted by the participated schools. The three schools have innovated in the following two aspects: One, teachers as the trainees. The statistics on the trainee numbers of these three schools include the teachers as the trainees; Two, improve the IT management capacity. With the support of World Bank loan, the information technology construction of the three schools has received breakthrough in progress and the advancement level. 8 management information systems have been built in GUCS, which are: Education Informational Management Platform, Network Teaching Resources Platform, Graduate Employment Tracking Service System, School-Industry Collaboration Information Platform, Students’ Psychological Archive System, Multi-Evaluating System, Examination database, and Resources Database. 7 information systems in YTS, which are Network Intelligent Office System, Information Management System, Teaching Quality Assessment System, Distant Teaching Platform, Yangjiang Advanced Technician Further Education Network, Course Scheduling System, and Graduate Employment Tracking Service System. The information OA systems for their administration, One Card Pass for campus management, the multimedia resource database

55

for teaching resources construction, the computer software for the development, teaching and evaluation of CBT curriculum, as well as the information platforms in these three schools on training and employment, school-industry collaboration, online examination and graduate tracking are under the proper and sound operation with certain good effects. 134. Reform and Innovations on the Competency Curriculum. The tasks by the project agreement were: (i) modular CBT curriculum; (ii) teaching materials; and (iii) consulting with enterprises, prioritize the amendment of methods assessment and verification for skills which are are greatly demanded by enterprises. 135. Modular CBT curriculum. Modular CBT curriculum refers to the competency-based curriculum which has been widely used in the international advance industrial countries. This kind of curriculum is quite striving in the countries with the market economic system due to its combination with knowledge and capacities, as well as its emphasis on the training of professional qualities. The CBT curriculum which has been jointly introduced with the World Bank is quite suitable to China’s actual situations, and it is more appealing to the economic developed province such as Guangdong. With the coordination of World Bank PMO at the provincial level, the three schools made the “Suggestions on Strongly Promote the Connotation Development of Technical Education in Guangdong” by means of the school-industry collaboration. The reform directions of these three schools have been therefore uniformed, and the working process as well as the operating methods of the competency-based curriculum reform has also been regulated. During the period of the curriculum reform, the three schools not only comply with the “Guiding Manual”, but also achieve breakthroughs and rich outcomes based on their real situations.

Table 10. A summary of the CBT curriculum development outcomes in the three schools over the past 5 years.

School Training Plans (No.)

Curriculum Standards (No.)

Textbooks on Campus (copy)

Published Textbooks (copy)

YTS 5 57 35 1

GLTS 23 162 100 6

GUCS 4 64 36 10

Total 32 283 171 17

136. The table below shows the number of majors, classes and students involving into the competency-based curriculum reform in these three schools over the past 5 years. The three columns of the data demonstrate the coverage of the competency-based curriculum reform: 21 majors, 169 classes and 9,146 students were involved (see Table 11 on following page).

56

Table 11. A summary of the competency-based reforms conducted by the three schools over the past 5 years

School Number of Majors participating the reform

Number of Classes participating the reform

Number of students participating the reform

YTS 5 21 1,716

GLTS 11 122 6,123

GUCS 5 26 1,307

Total 21 169 9,146

137. Consulting with enterprises, prioritize the amendment of methods assessment and verification for skills which are are greatly demanded by enterprises. In addition to the national stipulations on the skills assessment and appraisal, the school and the industry would jointly make plans for and implement the assessments with the practical skills teaching as the core element. Over the past 5 years, the schools were able to jointly develop evaluation systems with enterprises and form a multi-evaluation system. 138. Reforms and Innovations on Students Evaluation. The project was to: (i) adjusting the students evaluating mode, make it related to the competency-based curriculum; (ii) developing the tools to release students’ assessment results which is regarded as one of the indicators of the school’s teaching quality.

(i) Adjusting the students evaluating mode, make it related to the competency-based curriculum. The adjustment job is led by the GLIS, and the three schools jointly decided the assessment criteria which have been put into use. The GLIS gave the evaluations on the teaching effects of the curriculum reformed classes in every semester. The GUCS depended on the uniformed assessment criteria which have been decided by the three schools to improve its own student’s evaluation system with the consideration of the real situation in their school. The students’ stage learning outcomes were displayed to be voted by students and teachers with a multiple way of assessment. According to the survey conducted by Mycos Company and the statistics provided by the schools, the three participating schools have adopted CBT assessment to evaluate their students’ learning effects. Among them, GLTS has assessed the highest number of the majors under the CBT assessment (6 majors), and GUCS has 2 majors and YTS has 3 majors assessed under the CBT assessment respectively.

(ii) Developing the tools to release students’ assessment results which are

regarded as one of the indicators of the school’s teaching quality. All three schools publish the assessment results on-line to the public. In addition, the GLIS released the “Students Assessment Results List” every week, so that every student could be clearly aware of their individual situations all the time, and then make constant improvements. This can enhance the students’ self-

57

discipline and self-management capabilities. The GUCS included the students’ daily performances, class attendance and learning attitude into the comprehensive assessment in addition to the students’ works, which made the assessments on student capabilities to be linked with the social employment needs.

139. Reforms and Innovations on the Improvements in the teaching quality. The project was to: (i) enhancing the teachers’ teaching capacities for the new competency-based curriculum by means of the students’ learning evaluation; and (ii) conducting the pre-employment training and in-service training to the teachers with the new competency-base curriculum

(i) Enhancing the teachers’ teaching capacities for the new competency-based curriculum by means of the students’ learning evaluation. The reform on the students learning evaluation method, especially the involvement of the students, learning group and the industry representatives into the teaching quality assessment, have “forced” the teachers to take the reform on the teaching methods for the competency-based curriculum seriously. The modular, competency-based curriculum proposed by the World Bank project and the reform on the students’ learning evaluation in particular, made them to have “two closes, three mean-times, and one enhancing”, that is, close to the industry, and students; reforming, teaching and textbooks composing implemented in the meantime; enhancing the teaching capacities by means of reform.

(ii) Conducting the pre-employment training and in-service training to the

teachers by means of the new competency-base curriculum. The data from the “World Bank Project Impact Research Report” provided by Mycos Company shows that: among the three participating schools and out of their sample, GLIS has the largest amount of teachers to be trained for the competency-based curriculum (139 teachers), followed by YTS (125 teachers) and GUCS (12 teachers) respectively (in the two majors under the curriculum reform).

140. The Improvements in the School Facilities. The project was to: (i) improve and expand the teaching facilities and the relevant infrastructures; (ii) equip with the facilities to meet the needs of new modular, competency-based curriculum. The implementation by the three schools:

(i) Improving and expanding the teaching facilities and the relevant infrastructures. The improvements in the teaching sites of the three schools: As the proverb says, “The earth will tremble and the mountains will sway if the foundation is not solid”. The foundations of the technician schools include the software and hardware conditions. The World Bank loan has played a significant role in improving the teaching facilities of these three schools.

58

141. Policy Research & Knowledge Sharing at National Level. The project was to: (i) conduct policy research on the key issues of Technical & Vocational Education and Training reform; (ii) conduct policy research on the key issues of Technical & Vocational Education and Training reform; to learn lessons from the project implementation, and distribute experiences as well as the previously mentioned research outcomes. The Project Management Office (PMO) of GD HRSS Bureau organized the three participating schools to summarize and produce a collection of the competency-based curriculum reform outcomes, and published the “Collection of Excellent Papers of World Bank Technical & Vocational Education and Training Project (2009-2014)” (ISBN 978-5100-5567-6), which has included 35 excellent papers from the participating schools. The collection was promoted within the province and issued publicly. (Table X provides additional information.) Table 12. The Survey List on the policy development and capacity building conducted by the three participating schools

Name of School GUCS YTS GLTS

Times of the seminars on policy research 6

Quantity of the publications on the project experiences and research outcomes (number of copies)

21 26 130

Quantity of policies and researches shared with stakeholders and policy makers (number)

7

Quantity of the Applied Management Information System (number)

22 6 2

Number of people participating the monitoring and evaluation (number)

42 11 3

Quantity of the capacity building activities (number) 45 11 3

Source: Mycos Company impact evaluation 142. Project Monitoring & Evaluation. The project was to provide: (i) the evaluation of project beneficiaries and the design & implementation of the following-up survey; (ii) The baseline survey and impact evaluation conducted by Beijing Mycos Consulting Co, Ltd as the third party; (iii) set up annual monitoring indicator framework, and collect data annually; and (iv) develop and implement the school-based management information system to monitor school’s performance.

(i) The evaluation of project beneficiaries and the design & implementation of the following-up survey. The project beneficiary analysis was conducted at twice: the mid-term evaluation in October 2014 and final evaluation in May 2015. The degree of the benefits received from the the project implementation by the three types of beneficiaries, who are teachers, students and enterprises, the successful experiences drawn from the project implementation and the

59

exiting problems have been objectively evaluated by analyzing the implementations of reform measures which have been stipulated by the World Bank project in the three participating schools. The report has proposed effective suggestions for the project, and summarized a series of key index and indicators during the project implementation process. It also provided fundamental materials for the World Bank expert team’s project evaluation, as well as the experiences which can be used as the reference by other World Bank’s project implementation and promotion;

(ii) The baseline survey and impact evaluation conducted by Beijing Mycos

Consulting Co, Ltd as the third party. According to the purchasing requirements of the World Bank consultation activities, and with the support of Korean donation, the provincial PMO selected Beijing Mycos Company as the third party institution to conduct an impact evaluation. The baseline report aimed to present the initial situation of the project before the intervention, and to evaluation the project impact when project was completed. The project impact investigation at the final stage of the project was conducted from April 2nd to June, 15th, 2014. It can be seen that the majority of the project indicators have met or exceeded the expectations by comparing the baseline report and impact report. The project is proved to be a huge success;

(iii) Set up annual monitoring indicator framework, and collect data annually.

The framework is composed of the project output indicators, project final outcome indicators and mid-term output indicators, and it is collected systematically at the end of every year; and

(iv) Develop and implement the school-based management information system

to monitor school’s performance. YTS took the lead to jointly implement the graduate employment tracking service system with the other two participating schools, for the purpose to assess the schools’ performances.

143. The Management by Provincial Project Management Office (PMO). Since the start of this VET project, the provincial project management office (PMO) has experienced institutional and personnel adjustment, yet it kept the spirit of active exploration, significant innovation and diligent hard working. PMO strictly stayed with the World Bank project management requirements, focus on the constant regulations on financial management, procurement management, environmental management as well as the monitoring & evaluation. It also strictly complied with the initially designed task outline and logical framework, conducting a concrete promotion on various project work, overcoming various limited conditions and difficulties, and finally reached the ideal target with the significant achievement. Over the past 5 years, PMO organized the relevant staff from the participating schools to take part in various trainings. In the meanwhile, PMO actively focused on its own self-improvement as well as the quality enhancement by organizing as many as 1,000 people to participate in various training classes on project management, financial management and procurement management under the World Bank project. PMO paid high attention to each time of project supervision by the World Bank

60

experts team and the annual auditing by the relevant provincial departments. PMO insisted on the principles of getting the materials prepared in advance, ensuring the logistical support during the work and adjusting & correcting afterwards. The Lessons of Experience in the project implementation of school-based reform and innovation. 144. Experience one: Innovation in collaborative training system by both schools and enterprises. The nature of the dual school-enterprise system is the school-enterprise collaboration with significance in depth. The collaboration between the school and the enterprise is more than the simple internships or employment recommendations. For one thing, the collaborative classes have been recognized and paid high attention by both school and enterprise, which led to a sound effectiveness. Also, it can be easily accepted by the vocational schools based on their previous familiarity. The classes with the similar names but different contents can led the schools which have already conducted the “Pre-contracted class” and “Title sponsorship class” to have a further reform and innovation; Fourth, two innovative measure have been conducted to work with the “School-Enterprise Collaborative Class”, which are: the improvement of the organizational institutions and the establishment of the information platform for the school-enterprise collaboration. 145. Experience Two: Exploring a collaborative training system by means of the league of schools. This is the experience of the GUCS, which has been implemented since March, 2014, including: a league with four parties of government, administrations, schools, enterprises. The Guangdong School-Enterprise Collaborative & Development League on Urban Construction” was established under the guidance of the Provincial Department of HRSS, which was led by the GUCS and included four parties of government, administration, enterprises and schools. Three industrial associations, 33 enterprises and 5 schools have joined the league. Combine the league and the industrial park to co-create a fashionable and intellectual industrial park as the training base. The industrial park took the co-training on the highly skilled technical personnel as the basic task, and built up a deeply integrated model to link the industry and education. 146. Experience Three: Exploring the multiple training method for the management capacity. The three schools have explored the experiences in the multiple training methods on how to make use of the World Back loans to conduct effective training and enhance the school’s management capacity. This was done through: (i) organizing triple-level training, which is training at the school level (on campus), the training at the national level (domestic) and the training at the international level (abroad); (ii) selecting trainees from the leaders of schools, middle level leadership, specialty leaders and key teachers; (iii) conducting the dual-track training in terms of the training targets for both teaching and administration; and (iv) organizing the training for the group of instructors within the school. 147. Experience Four: Making good use of the Specialty Leaders. The highly skilled teachers guided the technicians of the enterprise on the project research and development

61

on the site of advanced devices, expanding a new path for the specialty construction and development with the innovation. 148. Experience Five: Exploring the multiple developing method for the competency-based curriculum. The three schools are different from each other when they are developing and implementing the same process of the competency-based curriculum which is related to the development idea of the integrated curriculum promoted by the Ministry of HRSS. The GLIS took a combined method with the collective course planning, large-sized class practice and public model lessons on teaching methodology to intensively promote the teaching of competency-based curriculum after they confirmed the school-enterprise collaborative courses. The GUCS took the measures such as the small-sized class teaching, action-guided curriculum teaching competition, separated examination from the teaching, a multiple evaluation and so on, to intensively expand its confirmed action-guided curriculum which was co-developed by the school and the enterprise. YTS guided the development and the implementation of their competency-based curriculum based on the integrated curriculum developing regulations by the Ministry of HRSS and was highly recognized by the experts from the Ministry of Education by using the teaching methodology to promote the reform of the curriculum. 149. Experience Six: Exploring the multiple evaluation method for the teaching outcomes. The three schools have all conducted a multiple evaluation method, with the following characteristics: (i) appoint a dual evaluation subject for the evaluation criteria, which can be appointed by the school and a third party; (ii) multiple evaluation subject: the students, the supervisor, the practical experts from the enterprises and the parents; (iii) self-assessment, peer review within the group, teacher’s evaluation, and the evaluation on the information platform; (iv) information-based evaluation: the school has set up an information platform for the evaluation on the teaching effects. The students, supervisors, practical experts from the enterprises and the parents can have their own access to give the evaluation separately. The evaluation results will be processed automatically by the computer; and (v) the release of the students’ learning results. The results of the students performance are released weekly to the students within the stipulated scope. The exploration on the multiple ways of evaluations on the teaching outcomes not only significantly improve the teaching quality but also lay a solid foundation for the tracking service of the graduate employments. 150. Experience Seven: Exploring the ways to improve teachers' capabilities. This was done by: (i) transferring ideas through schools organizing and conducting a great deal of trainings at various levels, such as the trainings on campus, and the trainings in domestic and abroad, to transfer teachers’ educating ideas in the technical &vocational education, development of specialty construction, curriculum development, teaching and evaluation; (ii) enhancing the capacity of curriculum reforms. Teachers were encouraged to participate in the curriculum reform, during which they got the ideas further transferred and abilities enhanced; (iii) enhancing the professional practical capacity; (iv) organizing teaching contests; and (v) stimulations by raising the class pay coefficient of the teachers who were involved into the curriculum reform to encourage more teachers to join the team of the integrated curriculum reform and teaching.

62

151. Experience Eight: Exploring the construction of digital campus The digital campuses of the three schools have been basically completed with the strong support from the World Bank project loans.

63

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable

64

Annex 9. List of Supporting Documents Distinguished Papers on World Bank Loan: Guangdong Technical and Vocational Education and Training Project (2009-2014) Project Implementation Completion Report, World Bank Project Management Office, December 2014 Impact Evaluation Report Assessment report of course development, May 2012 Beneficiary Assessments, 2012 and 2014 Country partnership strategy Aides-memoire

65

MAP

Document of The World Bankdocuments.worldbank.org/curated/.../pdf/...PUBLIC.pdf(Exchange Rate...

Documents

Transcript of Document of The World Bankdocuments.worldbank.org/curated/.../pdf/...PUBLIC.pdf(Exchange Rate...