Final Report (Vol. III) Case Study Annexes external... · Final Report (Vol.III) External...

Final Report (Vol. III) Case Study Annexes

External evaluation of the

European Training Foundation Contract no. 2001 – 3355/001-001

November 2002 Submitted by

Lion House, Ditchling Common, Ditchling

HASSOCKS, West Sussex, BN6 8SG United Kingdom

Telephone: +44 1444 248088 Fax: +44 1444 248763 Email: [email protected]

Website: www.itad.com

Final Report (Vol.III) External evaluation of the European Training Foundation

ITAD Ltd November, 2002 i

Final Report – Case Study Annexes for the External evaluation of the European Training Foundation

Contract no. 2001 – 3355/001-001

Table of contents

Acknowledgements ...............................................................................................................................................................iii

List of abbreviations and terms used...............................................................................................................................iv

1. Introduction....................................................................................................................................................................1

2. Lithuania and the National Observatory case study ...........................................................................................2

2.1 Purpose.....................................................................................................................................................................2

2.2 Approach and Methodology..................................................................................................................................2

2.3 Structure of the National Observatory in Lithuania ..........................................................................................3

2.4 The Functions of the National Observatory .......................................................................................................4

2.5 Stakeholder perceptions.........................................................................................................................................5

2.6 Sustainability and future strategies for the National Observatory..................................................................8

3. Case study 2: The Delphi programme case study .............................................................................................. 10

3.1 Purpose of Case Study.........................................................................................................................................10

3.2 Approach and Methodology................................................................................................................................10

3.3 Background on Delphi I Project .........................................................................................................................10

3.4 ETF perceptions of its added value to the Delphi I programme ...................................................................11

3.5 Management Roles and Systems ........................................................................................................................11

3.6 ETF Project Management Oriented to Objectives ..........................................................................................13

3.7 ETF Project Management: Content /Qualitative Monitoring........................................................................14

3.8 ETF Project Management and Delivery of Outputs........................................................................................15

3.9 ETF Project Management and Dissemination .................................................................................................16

3.10 ETF Project Management: Synergy with Other Initiatives ...........................................................................17

3.11 ETF Project Management: Added Value to EC ..............................................................................................18

3.12 Continuity from Delphi I To Delphi II..............................................................................................................18

3.13 Conclusions on ETF’s Project Management of Delphi I ...............................................................................19

Appendix 1: Project Synopsis from Final Report of Contractor 30 November 2001.........................................21

4. The Algeria case study .............................................................................................................................................. 25

4.1 Introduction and background..............................................................................................................................25

4.2 How is ETF perceived?........................................................................................................................................26

4.3 Using Title 3 funds ...............................................................................................................................................27

4.4 Support to the Commission.................................................................................................................................27

4.5 The observatory function.....................................................................................................................................28


ITAD Ltd November, 2002 ii

4.6 The Advisory Forum............................................................................................................................................28

4.7 ETF’s future role in Algeria ................................................................................................................................29

5. The Social Partners case study ............................................................................................................................... 31

5.1 Introduction............................................................................................................................................................31

5.2 Description of the Social Partners Project........................................................................................................31

5.3 Social Partners: An Awareness-Raising Project..............................................................................................34

5.4 Dissemination of Project Outputs ......................................................................................................................35

5.5 Information Exchange and Stakeholder Cooperation.....................................................................................36

5.6 Networking ............................................................................................................................................................37

5.7 Management of the Social Partners Project .....................................................................................................40

5.8 Added value...........................................................................................................................................................41

5.9 Social Partners links to a Centre of Expertise..................................................................................................42


ITAD Ltd November, 2002 iii

Acknowledgements This report draws on the results of a series of visits to ETF in Turin, Commissions services and the European Parliament in Brussels, and to Lithuania, Russia and Algeria. During the evaluation we have interviewed a large range of stakeholders, all of whom have been willing and helpful contributors.

The evaluation team would like to thank DG EAC (and in particular, our task manager Olivia Dorricott) and the Evaluation Steering Committee for their patience, support and guidance in steering the evaluation towards its conclusion.

The evaluation team would also like to thank the staff of ETF for their open and constructive approach to the questions and challenges we have posed to them, and for the practical assistance they have provided to us in arranging interviews, accessing documents and undertaking country visits. This assistance has been invaluable. In particular, we would like to thank Peter Greenwood and Sylvia Molteni of the Planning, Monitoring and Evaluation Unit for their unstinting efforts.

In Lithuania, we would like to thank the following who were very helpful with the organisation of the visit as well as giving valuable advice and practical support: Giedre Beleckie, Vincentas Dienys, Natalija Zimina, and Lina Vaikute.

In Moscow, similar helpful support and valued insights were given by Olga Oleynikova.

In Algeria, we would like to thank Osmane Meslouh and Belkacem Djoudad for their practical assistance and kind hospitality, which made our visit there both illuminating and enjoyable. During our visit, we were fortunate to spend time with Jean-Marc Casteljon, Algeria Country Manager, and Gerard Mayen, Teamleader for Observatory function development in the Meda region. Their comments and insights were a valuable contribution to our findings and conclusions about ETF effectiveness in Algeria and the Meda region.


ITAD Ltd November, 2002 iv

List of abbreviations and terms used ABB Activity Based Budgeting

AF Advisory Forum

AIDCO EuropeAid Cooperation Office

CARDS Community Assistance for Reconstruction, Development and Stability

C&TS Computer and Technical Services Unit within ETF

CC Candidate Countries

CEDEFOP European Centre for the Development of Vocational Training

CMT Change Management Team

CVT Continuing Vocational Training

DG EAC Directorate General for Education and Culture

DG ELARG Directorate General for Enlargement

DG EMPL Directorate General for Employment and Social Affairs

DG RELEX Directorate General for External Relations

ESF European Social Fund

ETF European Training Foundation

EURYDICE Information network on education in Europe

GB Governing Board

ICT Information and Communications Technology

I&CU Information and Communications Unit within ETF

ILO International Labour Organisation

JAP Joint Assessment Paper

LLL Lifelong Learning

LM Labour Market

MB Management Board

MEDA The programme financed by the European Communities to support the Euro-Mediterranean Partnership

MEP Member European Parliament

MTP Management Training Programme

NIS New Independent States

NOb National Observatory

OECD Organisation for Economic Cooperation and Development

Phare One of the three pre-accession instruments financed by the European Communities to assist the applicant countries of central Europe in their preparations for joining the European Union

PMEU Planning, Monitoring and Evaluation Unit


ITAD Ltd November, 2002 v

SME Small and Medium Scale Enterprises

SMT Senior Management Team

Tacis EU programme for provision of assistance to partner states in Eastern Europe & Central Asia

TOR Terms of reference

UNESCO United Nations Educational, Scientific and Cultural Organisation

VET Vocational Education and Training

WP Work Programme


ITAD Ltd November, 2002 1

1. Introduction 1. The terms of reference required case studies of “three typical projects in the areas of policy

advice, information provision and project implementation”. In fact, after consultation with DG EAC and the operational departments of ETF, the Evaluation Steering Committee agreed to the selection of four case studies. Case studies are considered to be representative of the different regions, types of activity, thematic areas of work, and period of implementation. The following were selected:

• National Observatories, which represent good examples of capacity building in partner countries and the development of products and services in close collaboration with local experts. A visit was undertaken to Lithuania between 7th and 19th May where the focus was on the National Observatory, but where the full range of ETF activities was also examined.

• The Delphi project in the Russian Federation, which represents a good example of ETF’s project management role. A visit was undertaken to Moscow between 19th and 25th May to look in particular at the Delphi project, but also at the National Observatory.

• Algeria providing an example of the new ‘centre of expertise’ relationship between ETF and the Commission in a new geographical area. A visit was undertaken to Algeria between 2nd and 7th June looking at all ETF activities there

• Social Partners where ETF is helping to enhance social dialogue on VET strategies at national and international level with a wide range of stakeholders. This was treated as a desk study with no requirement for country visits.



2. Lithuania and the National Observatory case study

2.1 Purpose 2. The National Observatory (NOb) is regarded as a main management instrument for ETF,

especially for the exchange of information between countries and ETF, who assigns them “a key role in its Work programmes and medium term strategy.” Its functions are to identify priorities and advise on policies for the further development of vocational training by:

• Collecting data on, and carrying out structured assessments of vocational education, training and labour market reform in their country

• Dissemination of information and good practice on vocational education, training and labour market systems from EU members states and other partner countries

• Developing co-operation with international networks and organisations from EU member states.

3. ETF have set up 25 National Observatories since 1996 in co-operation with the national authorities in 24 partner countries, to build a capacity within a country to perform these information and clearing house functions and provide a range of services and products to assist VET reform, as well as support the work of ETF in individual countries and for cross-country studies and activities. Since 1996, ETF has provided varied training support to staff of the Observatories to enhance their management and technical capacity.

4. Formally National Observatories are independent bodies, although they are often hosted in the Ministry of Education or the Ministry of Labour, which constrains their independence. Each has a steering committee to oversee their work. Development targets were set for the National Observatories in 1998 for 1999-2000 and a five-year plan for the Network. As part of the change of institutional medium-term strategy for ETF, the funding relationship between the National Observatories and ETF has changed from 2002. ETF now provides only core funding for mandatory outputs/activities related to their expertise and clearing house functions. NOb budgets are now more project-oriented, recognising their increasing need to sustain themselves from other sources.

5. The National Observatory network was evaluated in 2001 and the current evaluation does not duplicate this. Its purpose is to examine the function of the National Observatory as an example of its value as a management instrument for ETF in a wider context and its role as a source of information and expertise for customers.

2.2 Approach and Methodology 6. A case study was undertaken to Lithuania from 7th to 19th May. The purpose of the visit was to

examine the Lithuanian National Observatory in detail, and to look more generally at the range of ETF activities in Lithuania. This annex presents findings with regard to the Lithuanian National Observatory. Findings concerning this, together with the wider range of ETF activities in Lithuania, the National Observatory in Russia, and the observatory function in Algeria (addressed during separate visits to Russia and Algeria) are presented in the main report.

7. With regard to the Lithuanian National Observatory, the following issues were examined:



• Functions of the National Observatory

• Function of the transfer information and knowledge to the wider users in-country

• Way in which this knowledge and information is used by its end users or customers

• Extent to which this meets their needs and expectations

• Services and products it provides to the users or customers

• How customers for its services and products perceive their value to meet their needs

8. A list of the people and the documents consulted are presented in Annexes 3 and 10 respectively in Volume II to the main report.

2.3 Structure of the National Observatory in Lithuania 9. The National Observatory is located within the Methodical Centre for Vocational Education and

Training (MC), which is under the Ministry of Education and Science (MES). The contractor for the ETF in respect of the National Observatory is the Methodical Centre whose Director has overall responsibility for its activities. The Centre is primarily a research and resource centre in vocational education, and within it the National Observatory has independent status and its own Steering Committee and Team Leader. It has four staff, of which two are professional experts in the field and mathematicians by training, with one of them as Team Leader for the NOb. The NOb has a network of up to 60 local experts that it can draw upon for various studies and reports, either for ETF, MES or other customers. The MES funds the two main professional staff, provides accommodation and support within the Methodical Centre.

10. Its objectives are those common to all National Observatories as listed in para.6, although the NOb in Lithuania redefined them in its Annual Report 2001 as:

• To observe and analyse changes in VET, labour market and business systems, to investigate the causes of such changes and their effect on further development of these systems.

• To give analytic opinions and recommendations to politicians and practitioners involved in the area of VET and employment, as well as to the employers, trade unions.

• To promote an open dialogue between the business world and education regarding training of specialists, implementation of the Lifelong learning concept and development of human resources in Lithuania.

• To accumulate and store information received though the network of national observatories, through the UNEVOC network, or from international organisations, such as CEDEFOP, OECD, etc.

• To give proposals regarding classification of information according to the EU standards and requirements (statistic indicators, regulations) and application of the EU experience for the needs of Lithuania.

• To assist Lithuanian and international organisations to find contacts or partners in Lithuania and abroad, to organise workshops or training courses, and to implement the projects in Lithuania.

11. They now govern the main functions and activities of the National Observatory from 2002. It is too soon to assess the progress in this respect, for this evaluation.

12. These objectives reflect the priorities set by the Government, as part of its overall policy for national Human Resource Development, which is evident in the relevant government documents. The objectives also reflect priority issues for the Candidate Countries as outlined by the EC and, also, the direction of ETF as a centre of expertise within its new strategy. They move away from the original function for the NOb of collection, organisation and dissemination of information to



one that has an emphasis on services that can give added value to the whole spectrum of stakeholders involved in an education and employment within the broader context of Lifelong learning. The latter has now been set as a major priority for Government.

2.4 The Functions of the National Observatory

2.4.1 Providing Policy Advice and Identifying Priorities for VET and Labour Market Reform

13. The National Observatory (NOb) collects data and formulates structured assessments of vocational education, training and labour market reform. With a staff of only four, of which two are professional experts in the field, it has to draw upon its network of some 60 national experts, many of whom are well recognised within the country as high level specialists, drawn from institutions, government and research organisations.

14. The main users of NOb reports and analyses acknowledge that these have contributed to policy debate and development in VET reform, although opinions varied as to the degree of contribution. The main user is the MES, which perceived the data and the analysis of trends found in the annual reports as useful inputs into policy development and planning. Various reports such as on regulated professions and sectoral studies in 2000 fed into Phare 2001 programming, and contributed to the process of building information for planning and policy purposes. Examples of other inputs were its contribution with ETF of methodological support to MES in the drafting of the 1999 White Paper on VET in Lithuania. The NOb participated in the drafting of VET standards, and also made an important contribution to the 1997 report on labour market information system, which led to a joining together of the databases of MES and the Ministry of Social Security and Labour (MSSL). Staff of the NOb contribute to national working groups, for example, in preparation for the national employment strategy and the JAP. The NOb produced the national report on the Consultation on Lifelong Learning in 2001 for DG EAC. National Observatory staff are sometimes delegated to represent the MES, such as part of the working group for Special Preparatory Programme II.

15. The main product of the National Observatory is the annual reports produced for ETF, which are perceived as useful by all users consulted, and as well-regarded inputs into policy development and planning of activities by the MES.

2.4.2 Dissemination of Information and Best Practice 16. The National Observatory disseminates information and good practice on VET and labour

market reform from EU member states and partner countries to various customers in Lithuania, which are listed on its database. Where it receives that material electronically from ETF it is able to transfer using the same medium, although many of the reports that it receives from ETF are printed. The NOb does not have the resources to copy and distribute. If a customer visits the NOb then they are provided with a copy if that person is aware of the report. A VET web page on the Methodical Centre’s website provides news of publications. Users recognise that they can access the web sites of the EC direct.

17. MES has an operational need for information on EU and best practice in VET and labour market, and the National Observatory meets this need. While policy makers can access the Internet, this is time consuming – large amounts of data have to be sifted for specific needs, and they need to know where in EU the data can be found. Such data provides a comparative basis for analysis of different educational issues. This service is performed effectively for the MES by the NOb, a service which is perceived as essential. If the NOb did not exist, then no such analysis would take place.



2.4.3 Networking 18. The NOb provides limited support and promotion to networking between the different

stakeholders and initiatives in VET reform within Lithuania. Indeed, no single body facilitates this task unlike the example of Estonia where one institution co-ordinates the different EU programmes.

19. If the NOb is making a study or review, then it circulates drafts to the different players for comment and this is a way of building local networks. Similarly so when it transmits information and good practice as outlined above. Where VET institutions are approached for information for the annual country report prepared for ETF, then it is more effective for the NOb to route this request through the Ministry of Education, National Labour Exchange (NLE) or Statistics Lithuania. Where seminars are held in the counties, then they are used to bring together all entities involved in VET and thus foster national and local networks and contacts.

20. With regard to networking with other partner countries, the NOb has made links with the other candidate country National Observatories, especially in the Baltic region. These regional links arose from direct contact by the NOb and also via the Phare Programme Management Units (PMUs). These were facilitated through the Phare VET Reform project and the PMU, and the regional networks facilitated by ETF.1

21. The NOb does draw upon the regional networks for information and source of best practice on VET and labour markets. They are useful for preparing reports for ETF and in respect of other studies. A standard methodology is used for ETF reports, therefore it is easier to draw upon comparative experience using common parameters. If such regional and ETF networks did not exist then such comparative analysis would not be possible.

22. Before the change to its new strategy, where ETF provides financial support for mandated activities only, NObs used to meet twice a year to share mutual experiences and compare the feasibility of different approaches and activities in VET reform. Comparative studies were possible and ETF would meet ‘ad hoc’ requests for information. All such support ended in 2001, and any links are now maintained only through projects – for example, on the teacher-training project with Latvia in which CVER in Kaunas has a leading role.

2.5 Stakeholder perceptions

2.5.1 Use of Knowledge and Information by NOb Customers 23. Annual country reports are perceived as containing expert evaluation of changes, trends and

national indicators. They are seen in varying degrees as helpful to planning the work of users. Mainly they are regarded as “useful” but for many it must be questioned how far they are more than just background.

24. Main demand for NOb services comes from the ETF, Ministry of Education and, to some extent, NLE. ETF requires information on VET and labour markets in Lithuania. The Ministry of Education usually requires analyses on specific VET issues.

25. Local customer demand in government, institutions and private sector has not been purposely analysed hitherto as the NOb claim it has been occupied with meeting the different requests from ETF. A basis for the provision of specific national activities have arisen more from the understanding of NOb staff as to which needs have to be met by the NOb. It also has advice from its Steering Committee.

26. Stakeholders in VET have tended not to articulate their annual needs clearly enough as a basis for reports. Services provided by the NOb do not specifically relate to any needs for skills planning as part of national economic planning and skills forecasting process.

1 ETF has provided funding to the Baltic States sub-regional network, for annual meetings to share information.



27. The NOb is responsive to demand rather than pro-actively assessing needs, primarily because it has only a small staff. There was an attempt in 1998 to assess demand through self-analysis in an inter-ministerial group of Ministry of Education, Ministry of Social Security and Labour and the Labour Market Training Authority. Based on its own assessments and also responding to pressure from ETF, the NOb puts forward proposed projects to its Steering Committee, on which are represented the main stakeholders, who then choose what it considers the most useful to undertake. This is not really a demand led process in terms of assessing market needs for its services and products.

28. Its main customers see the NOb as providing a framework in the transfer of ideas and knowledge so that they can be interpreted and used in a local situation. This can help generally to promote the process of change in VET reform.

29. Information and services from the NOb have a potential for use by the private sector/employers and social partners. It can help identify their needs, for example, in human resource development and the need for pre-qualifications and standards. There is a general lack of national capacity to assess demand by institutions coupled with a lack of sectoral analysis. Institutions and schools work individually to assess the demand of companies. Hitherto, assessment of labour demand is not expected of the NOb; rather it is a general context that is provided. It is difficult to forecast the needs of employers as the reform process is underway in all sectors. The redirection of export markets and the need to conform to EU standards in management and product quality standards poses challenges for companies. For example, the NOb report of 2001 on human resource management in private enterprises showed that only 8% of employers are ready to invest in education. More have now understood the need to invest in human resource development as a key element in being competitive.

2.5.2 The Added Value of the National Observatory 30. Reports collate and present the various data in one coherent form, which assists analysis and use,

and is related to needs. The reports and analyses provide consistent reference tools that systematise information from different sources and training levels and allow comparison using the key indicators contained in the annual reports. The collection and collation of data provides useful aggregated summary of data, which saves time for MES officials in having to locate it themselves. The NOb performs different analyses according to specific requirements of the MES, often on an ‘ad hoc’ basis and this is a flexible and useful resource. MES has neither the personnel nor a support institution to carry out these functions. All these aspects provide added value according to users within the MES and the National Labour Exchange.

31. The only research centre for VET in Lithuania is CVER in Kaunas, which also provides graduate courses for VET practitioners. CVER does not duplicate the functions of the NOb. Rather its emphasis is on innovation and research on teaching and learning. In this regard it participates with the NOb in the research and collection of knowledge. Yet the CVER cannot undertake in the same way the function for dissemination of knowledge and information as carried out by the NOb. Both centres are complementary in the expertise and transfer of knowledge and information.

32. The National Labour Exchange has used the information and services of the National Observatory, which have improved its work. NOb data and analyses have assisted in the anticipation of labour market demands, such as demographic trends, and numbers of people entering and leaving the labour market. Information provided on enrolments in vocational training institutions is aggregated by entry level, qualifications and data on graduates. Together with information on numbers leaving secondary schools, numbers entering the labour market who are unqualified and those going on to further education, it enables the NLE to project the likely demands on its own services and activities. Issues relating to youth covered in NOb reports assist in the organisation of measures in the active youth labour market and the formulation of policy decisions. The NLE collects data on graduates of higher education institutions who are registered with it. The NOb is able to combine this information with data from the education



system, and graduate numbers by academic/training programme and region. This enables the NLE to calculate the numbers of unemployed in this category against type and level of qualification and region

33. In relation to the private sector, the 2001 report on the management of human resources for the private sector reportedly made companies "think of these issues where before they had not". Such added value is not easily measurable, but is felt to provide benefits as it channels information from the EU and the network of National Observatories to social partners and organisations. Information can be disseminated to members, indicating for example the requirements of EU standards in management and the use of human resources, which will be needed if companies are to participate competitively in EU markets.

34. Statistics Lithuania is a provider of data to NOb rather than a user, and has the capacity for co-ordination, collection and dissemination of data. Sectoral analysis is lacking and the NOb could fill this gap, as well as the occupational classification of data needs evaluation – for example, the NOb could analyse trends in labour market demands and make proposals for how to address these needs.

35. The National Observatory has co-operated with other EU programmes or agencies, thereby providing added value to their operations. There has been a degree of co-operation with the National Agency for the Leonardo programme, albeit limited. It supplied data and analysis to help the Agency with the production of an Information Catalogue to help with vocational guidance for secondary school graduates. This support was perceived by the Agency as valuable. Data came from Statistics Lithuania and the Labour Market Training Authority. The Agency could have sourced it themselves but as an Agency that is essentially a management unit administering grants and projects, it has constraints on its staffing, time and capacity. The NOb made a coherent order to fragmented data and presented it in a way that the Agency could use for its specific purpose of vocational guidance.

2.5.3 Conclusions on the Functions of the National Observatory 36. The annual country reports are the main product of the National Observatory and for ETF and are

perceived as useful to all the users. Data is systematically collected from different sources, collated, analysed and summarised in a form that the user can then develop further. The data and the analysis of trends they contain are perceived as necessary and useful and well-regarded inputs to policy development and the planning of activities by the main user, the MES. The MES would not otherwise have access to such a service, and it has neither the staff nor the resources to collect and collate the data itself. The Institute for Labour and Social Research, which was set up independently from the MES in 1991, also has the Ministry as the main client for its studies and research. The use made of the NOb by the MSSL is therefore limited.

37. The added value to the customer is not just the data and information products themselves, but the process of collection, analysis and then systematic collation and presentation. This function is necessary for the MES, as it has no other support institution that can provide this service. In contrast, the Ministry of Social Security and Labour has the Labour Market Training Authority on which it can draw for such analyses. Indeed, although the NOb’s reports and analyses are generally perceived by the main users and stakeholders as more descriptive and factual than analyses of a high standard, this is actually what is needed according to the MES, providing the basis for further analysis and interpretation by the users themselves.

38. The annual country reports are regarded as useful also at the local and institutional level, for example, in assisting the process of matching outputs of schools to the labour market. All educational institutions need such information for their own planning purposes, e.g., courses to meet trends in the labour market in a county as was mentioned at the Regional Training Centre in Marjampole and Vilnius College in Higher Education.

39. NOb reports have contributed to the process of building information for planning and policy purposes. For example, sectoral studies on the retail sector done by the NOb in 2000 for the MES



were valued as their data and analysis assisted the MES in forecasting trends in labour market, optimising networks and resources, and the formulation of policy and strategy for VET provision. However, from examination of the range of publications produced by the NOb, this customer orientation is a recent development. Prior to 2000 reports were produced more to meet the direct needs of ETF.

40. The NOb is not perceived as a high-level research body, but the MES requires practical analysis for operational needs and this is provided by the NOb. There is though a perception that the NOb could involve more experts with a higher level of academic expertise. The NOb is not regarded as the main source of best practice and knowledge on VET and labour market. Rather it uses expertise from a variety of sources. It is known for vocational training and not labour market knowledge. There are other institutions better known and used for the latter.

41. Ministries and institutions often take different approaches to VET and labour market reform issues. There can be a competitive element whereas co-operation would serve common goals.

42. The NOb can provide objective analysis, and is an intermediary between the two parallel systems run by MES and the Labour Market Training Authority (under MSSL). Information is not fully shared between the two in order to maximise its value and potential for policy development. Users in the education system need to know what is happening in the labour market so they can plan; and vice-versa with the outputs of the school system. By disseminating information from both sources to national users, the NOb provides a service that neither on their own would provide. By doing so the NOb can also contribute to a process of synergy between all the entities involved in the process of reform.

43. The NOb is able to contact social partners and convey the substance of the process of social dialogue, when the latter is weak in Lithuania. This can then support national initiatives as well as efforts by ETF to facilitate stakeholder consultation, although this role is more for national actors in VET reform than for ETF.

44. NOb has an adequate networking function within the country and to the EU, although the latter is via the ETF. While it remains linked to ETF, the function of a NOb is that of an intermediary to draw on EU best practice and networks for expertise. The range of information and knowledge that comes from the EU, ETF and other partner countries needs to be adapted to needs of the in-country user and the specificity of the change process for VET, labour markets and human resource development and within the context of national initiatives for pre-accession. The different needs of users at different levels have to be recognised. Therefore, it is important to have an intermediary body that can facilitate that adaptation, manage the knowledge and disseminate information accordingly. The NOb goes some way to fulfilling this function, but requires guidance from ETF to consolidate this role.

2.6 Sustainability and future strategies for the National Observatory

45. There is a broad consensus that the function of the NOb is necessary, but a key issue is to how to finance it. There is also a common perception that the NOb could help itself by more actively promoting its services, perhaps in co-operation with other institutions.

46. To do so, the NOb will need to assess market needs and develop a strategy to meet them. This imperative is underlined by the change in the funding from ETF as part of its new strategy. The extent to which more self-sustainability is feasible for a NOb will depend on national circumstances and where it is hosted or whether it is an independent body. This will influence the degree to which the National Observatory is proactive in marketing and promoting its services and products.

47. An underlying issue is whether the National Observatory has the services and products for which there is a market. The potential to market its services must be seen within the context of the gradual recognition that information has a value within a market economy, which in itself is new.



That recognition is more prevalent in the private sector but is slowly being accepted in the public sector. It all depends on the willingness of the user to pay – the MES budget is already stretched, and priorities for educational users are likely to be more the payment of utilities rather than information, which hitherto they have had free.

48. Many of the key informants interviewed expressed doubts as to the commercial value of NOb products, as the only potential customers are limited to government bodies. One option raised is for the NOb to become a public non-governmental institution funded by the MES. A comparison is with the Institute for Labour and Social Research but this does have core funding from the range of activities supported by the MSSL, and the range of work it does for other donors. Even if such an approach were feasible, the NOb still has to assess what it can offer, and to develop a plan. This means a marketing and promotion strategy, for which the NOb has weak skills.

49. Sustainability for the NOb has to be seen in the context of the needs of customers. Reports from the NOb generally address the needs of policy makers. Although potential customers can often access direct information from a variety of sources, it is the service of organising and analysing data that often pose constraints. It is most unlikely that private sector organisations would pay for services unless their specific needs were addressed.

50. Potentially saleable products and services include:

• best practice information

• the methodologies that it has developed

• analysis services

• conduct of sectoral studies

• forecasting of labour demands for social partners

• analysis of vocational training related to labour market needs, as a basis for the development of training

• collection and analysis of data for special purposes within the context of labour market needs, and co-operation with the National Labour Exchange.

• the analysis and informing Government of the needs in local training systems.

• advisory services through its local expert network.

51. The National Observatory does have a basis from which to plan its future. It has a recognised and valued role; funding from ETF; good links with VET institutions in Lithuania and with the EU, and a degree of security and support from MES. What it now needs is a strategy for its future and for the marketing of its services to potential customers. This is gradually being recognised, but requires a cultural change within the NOb to take on this new role and to stand more confidently as and entity independent of ETF.

52. If the National Observatory succeeds in becoming more independent and self-sustaining, then ETF will become one of its clients rather than its parent institution. This is the logic of the approach of the new ETF strategy. CEDEFOP will be conducting its own assessments of potential candidates for REFER this year, and no decision has yet been made. The National Observatories represent a considerable investment and a valuable resource, and while they may yet be incorporated into REFER there is little time for the 2004 accession countries in general, and Lithuania in particular, to pave the ground for this. If the National Observatory is not selected, then its future role becomes uncertain.



3. Case study 2: The Delphi programme case study

3.1 Purpose of Case Study 53. The aim of the case study is to assess the project management by ETF for the EC of Delphi I, as

an example of the service that ETF could offer its main customer. ETF’s management of the Delphi project was assessed according to its effectiveness in meeting the needs and expectations of stakeholders/beneficiaries in partner countries, and providing added value to the project outputs.

3.2 Approach and Methodology 54. The case study visit to Moscow took place between 19th and 25th May 2002. Prior to the visit,

ETF staff were interviewed to gain their perspective, and to assist in planning the visit. Interviews were conducted in Moscow with key informants, most of who were suggested by ETF. The persons suggested included not only the Ministry of Education2 and Ministry of General and Professional Education but also those involved in the Presidential Initiative and national experts who had been project component managers. The evaluation team also consulted others who could give an independent view, such as the Tacis Monitoring team. No meetings were possible with the EU contractor as the project was completed and they had left Russia. Meetings were held with the EC Delegation in Moscow, and subsequently with Tacis Unit staff in Brussels, to gain their perspective as the customer. Interviews with the National Observatory provided an informed overview of the project and a valuable perspective on the different issues covered by the evaluation.

3.3 Background on Delphi I Project 55. Delphi I had a total budget of €6m, started in December 1998 and ended in December 2001. It

was implemented in Moscow and five regions.

56. Project objectives for Delphi I and its four components with their specific objectives are summarised below, and details with outputs and activities are in a synopsis in Annex II:

Wider Objectives:

Education system responds to the assessed needs of economy and society in the fields of Economics, Business Administration and Law Teaching and Vocational Education and Training using improved Open and Distance Education.

Specific Objectives:

Component I Education Management:

To support the Ministry for General and Professional Education (MGPE) in formulating education strategy, policy planning and implementation in co-ordination with other Ministries and social partners.

Component II Teaching of Economics, Business Administration and Law:

New, innovative education and training solutions are provided to regional labour market needs in Applied Economics, Business Administration and Commercial Law. Stakeholders have been trained in expressing labour market need and describing professional profiles; new teaching and learning methods are applied and new networks established according to the effective needs in the selected economic sectors.

2 Ministry of Education as used in text covers both ministries.



Component III Vocational Education and Training:

Efficient and applicable VET reform model based on the labour market needs of different economic sectors has been developed.

Component IV Development of Open and Distance Learning for Dissemination of Education:

Increased access to management training and economics education through improved Open and Distance Learning system methodologies and learning and teaching materials.

3.4 ETF perceptions of its added value to the Delphi I programme

57. ETF considers that its project management of the Delphi I exemplified the following added value. Questions in interviews were oriented to an assessment of whether this added value was indeed provided. Responses were incorporated into the report whose structure is based on analysis of the following issues:

• The adaptation and incorporation into Delphi I of ETF experience gained from other activities, such as the evaluation of previous management training in Russia, the Northwest Russia project, the National Observatory and others

• Continuity of ETF staff and corporate memory was better than the EC

• Dissemination was built into project activities, which include the monitoring of dissemination, which the EC does not undertake

• Synergy between ETF-managed projects - e.g. NW Russia and elements of the Management Training Programme-through case studies and exchange visits

• ETF complement Tacis monitoring information with qualitative/content monitoring

• Documentation of guidelines and lessons, making them available for subsequent project design/management, which is not done by the EC

• Close and direct relationships with local partners, which build strong relationship over time, and which the EC is unable to do given the frequency of internal changes.

3.5 Management Roles and Systems 58. Initially, the EC in Brussels received four separate project fiches from the Russian authorities.

The EC then decided to combine all four together as one programme, and asked ETF to draft the TOR for the preparation of the programme. ETF had a real challenge to persuade the Ministry of Education to accept one rather than four projects. ETF were given only one month to prepare the TOR (later extended), during which time the EC task manager changed.

59. The Ministry of Education rejected the TOR first presented by ETF. This appears to have been due to their concerns over the feasibility of one rather than four projects (a concern shared by ETF), and their perception that ETF had prepared the TOR based on their own experiences in other countries rather than based on a full assessment of needs and context in Russia. Consultations over a period of time of all parties led to the final TOR, and ETF was then involved in the tender evaluation process to select the EU contractors. Task management for project implementation was subsequently delegated to ETF from EC Brussels. ETF’s responsibilities included supervision of the EU contractor, keeping the Delegation informed of progress, approving contractor’s progress reports, and procuring equipment.

60. Stakeholders feel that ETF’s previous involvement and experience with projects in Russia helped it to establish viable modalities for project management. ETF evaluation of previous management



training and findings and experience in Russia fed into its management of Delphi, as did experiences from other projects and activities. Many of the issues pertinent to realisation of the project objectives were known to ETF. It had established contacts with the Federal Ministry of Education and so was aware of its capacity. Therefore, it did not make the MoE the only formal partner for management but looked to a wider set of stakeholders in the regions. ETF was able to use the National Observatory and its network. The National Observatory provided advice to ETF throughout the project on the strengths and weaknesses of the different stakeholders as well as possible action for implementation of the project components that might be feasible given the Russian national and regional context. ETF was able to draw upon knowledge of best practice from within ETF, its networks and the National Observatory, especially in components II and III, to provide pertinent content input for the project.

61. As the project was originally four separate projects into one in the TOR, there was a recurring concern as to how they would cohere as one project and be managed accordingly. The Tacis Monitoring team advised at the start of the project that this would cause problems in implementation and their subsequent reports to some extent bore out this assessment. As project managers ETF tried to ensure coherence and assisted all the project partners to understand how they could participate jointly in project implementation. The long involvement and experience with Russia of the ETF project managers was perceived as a positive factor in the establishment of project management modalities, and advice to the EU contractors.

62. As a result of ETF’s continuity of experience with projects in Russia, management systems were structured in a way that emphasised the role of the beneficiaries and Russian partners, and did not just rely on the EU contractors. This meant all the partners were involved and they were facilitated by periodic regional meetings during the project. At these meetings, progress on the components in each region was shared.

63. While the EU contractor was seen as responsive and focused on practical results, there were a number of problems and delays related mainly with the procurement of equipment for components III and IV. These affected negatively the end of project assessment by the Tacis Monitoring team. The final April 2002 report concluded, “Tacis projects of this nature should not be awarded to academic bureaucracies who lack flexibility and speed of response to operate Tacis Supply Procedures.” However, problems with procurement are common for many Tacis projects.

64. ETF’s efforts to resolve this issue were generally commented upon favourably in interviews. While with hindsight, it is easy to say that ETF could have foreseen these problems earlier (as some key informants commented), there is also a general acknowledgement that delays in procurement of equipment is common for many Tacis projects for a number of reasons that relate to the procedures themselves and to the operational environment in Russia. Nonetheless, the evaluation found the considered view that ETF were seen as more content oriented in their project management, and had left the resolution of procurement problems more to the contractor. Overall, procurement problems were the only significant ones identified with ETF project management.

65. One key element to the management system set up by ETF was the link set up with the National Observatory, both a source of expertise, and as support for ETF in its project management role. The TOR required the contractor to work with the National Observatory, and stakeholders regarded it and its good regional network of local experts as essential to implementation of component III on VET reform. This contribution is acknowledged in the contractor’s final report of November 2001.

66. A positive feature of the project management by ETF and the contractor generally acknowledged by both stakeholders and the national experts was the involvement of Russian partners and regional managers in project management. Each EU expert had a counterpart at the federal level, and the selection of regional managers also assisted the management process. Their selection was perhaps 50:50 in terms of institutional as against individual representation and the consequent authority they could bring to their role.



67. National experts considered that they were involved as partners in the management systems as set up by ETF and the team of EU experts in Moscow. Regular meetings were held between the EU contractor experts, the national experts, and the Ministry of Education. In respect of component II (management training) the national experts were involved in management planning and implementation of activities. Local experts planned and prepared the papers that were the project outputs. There was perceived “real partnership” with EC experts for component II. Special regional initiatives were mounted to develop management training; training of trainers; training of trainers for managers; design of programmes and materials and their implementation. One expert was responsible for each region, and disseminated the results in other regions. This was regarded as real teamwork and characterised the way that the management of component II was organised. ETF was also instrumental in achieving this. The national experts interviewed were fully involved in both an expertise and management role for components II, III and IV. The Team Leader of the National Observatory was an expert for component III (VET).

68. Professional support to activities and achievement of outputs was provided for component IV (Distance Learning). Resource centres with audio-visual equipment are functioning and are a source of ideas on distance learning for their region. Seminars were held and the results disseminated. Teams for the component were trained and local needs analysed. A database was set up and training materials designed in the regions. The major problem was the procurement of equipment and the late delivery.

69. In terms of the supervision by ETF of the EU contractor, it is evident from the numerous and detailed monitoring and management reports produced for each visit by ETF project managers that they visited regularly and attended all the relevant project meetings. This indicates the degree to which ETF had to advise the contractor and component managers on specific project modalities and procedures. ETF’s approach was perceived by some as being ‘at a distance’, not interfering too much in the day-to-day management by the contracted team in Moscow. While this has the virtue of allowing the contractor to do their job, there was a perception that ETF could have been more attentive as to how the project was actually delivered, especially in the regions. For example, more validation of regional reports was necessary, as they were then the basis for reports by the contractor and ETF. Therefore the perception was that reports tended to reflect more the view from Moscow and those of the EU experts.

70. Taking all reservations into account, consultations found that there was an overall view by the EC Delegation that the supervision by ETF of the EU contractor, whose project manager and team dealt with implementation and day-to-day management, was “appropriate, timely and effective.” The task management of ETF was assessed as “incisive and stable” [Tacis Monitoring report August 2001].

3.6 ETF Project Management Oriented to Objectives 71. A key issue for assessment of effectiveness of the project management role of ETF was the

extent to which its supervision of the contractor and support to the project ensured that the objectives were maintained and its outputs were delivered.

72. The perception of the main stakeholders was that ETF sought to maintain orientation to project objectives with defined targets, and attempted to ensure that the EU contractor met those targets. This is evidenced by various reports of ETF and memoranda exchanged between ETF and the contractors and EU team in Moscow. They indicate the extent to which the ETF project manager endeavoured to ensure that project activities developed by the contractors and experts were aligned with the objectives. This was despite certain activities and outputs being modified as compared to the original TOR. The one target not met relates to the procurement problems described earlier.

73. Steering committees for the components were reminded by ETF at each meeting of the need to maintain objectives and a demand-led approach to their realisation. Changes that were made to components were done in consultation with the EC, Ministry of Education, national and EU



partners or to engender a complementarity with related initiatives. For example, component II had an original target group of university students for management training identified by the Ministry of Education. This was not the focus of the Management Training Programme under the Presidential Initiative. Discussions as to possible synergies led to the changes agreed with the Ministry and key stakeholders as outlined in the Inception Report for the project. The emphasis for component II was changed to post-experience training with a market led focus and more linked to companies and their needs. It then reflected the common theme of a demand led approach to all components in line with EC objectives and the national action plan.

74. Materials and methodologies were disseminated in the regional meetings for all partners - ‘Delphi weeks’ - to indicate what results needed to be achieved and to give guidance as to how to achieve them. The longer-term internalisation by beneficiaries of the methodologies that were introduced, in order to enhance their own technical and management capacity, is difficult to assess and is more a task for Delphi II.

75. As part of its project management role, ETF tried to bring coherence to the four components of the project. It is clear that ETF consistently reminded all partners of the need to achieve this coherence and this contributed to the delivery of outputs. It presented methodological materials that demonstrated means whereby results could be achieved. This was especially the case with component II on management training as evidenced by the feedback from the national experts involved in management of project components.

76. Coherence between the objectives of the four components was not achieved initially and only happened in the last six months of the project. The end of project assessment report of April 2002 noted that “[there was a] lack of synergy between components; the platform for the synergy between the components was not well thought out.” In respect of component III, synergy was regarded as “virtually impossible because, traditionally, there have not been any business links between classical universities and professional training institutions except in some special cases.” [Comments on contractor’s report of November 2001: Jan. 2002].

77. However, from the reports and interviews there is a perception that ETF did try to create a synergy between the heterogeneous elements to produce a coherent project and a related set of outputs. Such synergy which emerged in the final phase of the project was perceived as due to their efforts. Regional synergy was attempted through ‘Delphi weeks’ in each of the five regions where the results for each component were shared in order to stimulate coherence. Co-ordination between component managers was encouraged and would appear to have been achieved. There was some doubt expressed as to whether the networks and links set up between the regions would be sustained.

3.7 ETF Project Management: Content /Qualitative Monitoring 78. ETF claim added value from their content monitoring as compared to the EC, which is

exemplified by its qualitative monitoring of the Delphi I project. Each stakeholder/national expert/manager for a component had different experiences, which influences their perception of any content monitoring by ETF.

79. It is instructive that a main stakeholder - the Ministry of Education - was unaware of ETF providing any content monitoring or reports in this respect. It was only aware of the monitoring by the Tacis Monitoring team and from its reports that were sent by ETF. Monitoring of the project was done for the EC by the regular visits and reports of the Tacis Monitoring team and not ETF. All their reports were provided to both ETF and the EU contractors in Moscow. There was a perception that, in some respects, the Monitoring Team was more informed as to activities in the regions. The contractor had no long-term experts in the five regions. Their reports gave a picture from Moscow rather than the regions, and were the basis for ETF reports. There was a view that the overall picture given by ETF based on contractors’ reports did not always correspond with the reality in the regions. On the other hand, the EC Delegation has the perception that ETF provided content monitoring which was more evident early in the project



than later. Feedback came from ETF at the beginning and end of the project. It was able to suggest changes in component strategies and did not take for granted what was reported to them.

80. National experts for component II considered that ETF provided content monitoring. This appears mainly to relate to the input of the ETF project manager, who was highly regarded as a management specialist and with whom there had been long standing contact. The periodic Tacis Monitoring reports were seen as the more reliable source of information on activities in the regions.

81. While there was a general perception that ETF can add value to project management through its capacity to provide content/qualitative monitoring, this was not evident to all parties involved. However, this perception is perhaps more in terms of what is provided through recognised individual capacities for a specific component rather than as an institution.

3.8 ETF Project Management and Delivery of Outputs 82. Substantive detail on the outputs and specific results for each component is found in the reports

from the final conference in October 2001 to examine the results of Delphi, and the final report of the contractor in November 2001. The bottom up approach of combining a federal project with a high level of regional and local activities was regarded as successful. Three kinds of main outputs were identified as “policy papers, methodological guides and learning resources in which resources have been refined for dissemination supporting the education system involving the demand side as well as defining mid to long term education policies.” The key result was that: “Delphi supported the reform process in the Russian Federation with the objective to bring education closer to the demand, increase access to education and to enhance co-operation within the Russian education system at federal and regional levels.” [Final report].

83. The issue is to what extent can these results be attributed to ETF’s management strategy and the effectiveness of its project implementation.

84. The end of project assessment report of the Tacis Monitoring team [April 2002] concluded, “the project produced all planned results (against agreed outputs). The quality and quantity of outputs was considerably higher and the final project results were more fundamental than expected.” Specific objectives were achieved: “each institution took on board what was the most feasible and appropriate for the current situation.”

85. From the perspective of a main stakeholder – the Ministry of Education – the main output was that each project component made participants understand the need for change in attitudes to an approach based on labour market demand. The project generated evident results in orienting education to customers, and enhancing the context of training programmes. Recent documents of the Federal Ministry of Education on VET policy and strategy have reflected the outputs of Delphi I (as well as the NW Russia project). They correlate with the main strategic aims for VET in Russia. Realisation of project outputs was perceived as a team effort by both EU and Russian experts, with a substantial contribution from beneficiaries. The project was regarded as a catalyst for the realisation of national ideas and initiatives.

86. However, it is difficult to locate the major stimulus for this, whether it was EU and/or national experts, ETF project managers, management systems which ETF set up or the EU contractor that allowed such ideas to be articulated and developed; or whether a combination of all of these interventions. To assess the validity would require a time frame and extended regional visits outside the agreed framework for this evaluation.

87. On balance, the considered assessment of the key informants interviewed is that ETF recognised the potential catalyst role of Delphi, and set in place with the contractor the structure and systems to allow such initiatives and ideas to be developed, especially at the regional level. The project supported local initiatives by providing case studies of success, while regional managers gained real professional competency and were helped to find common themes and solutions and to speak the same ‘language’.



88. The policy advice component was considered not to have achieved its goals fully at the federal level for the Ministry of Education. Tacis monitoring reports point to minimal impact in relation to the policy advice to the Federal Ministry of Education. Contributing to this issue was the high turnover in senior policy makers within the Ministry, reducing the capacity of the national partner to take advantage of the potential the programme offered, and weakening the relationship with EU experts. One view was that ETF as project manager could have been more pro-active so that action was taken but this is difficult to substantiate.

89. It was acknowledged that the progress and results of the project improved dramatically when the focus moved from the federal Ministry of Education to the regions. As the institutions and administrations in the regions became stronger, they stimulated the Ministry of Education to react and move more quickly in its own decisions. The EU contractors tried to ensure delivery of outputs and this was put into effect more in the second phase of the project. Reports and materials were produced. It would appear that the expectations of the groups of stakeholders were met through the delivery of the different outputs for each component. In the view of the Ministry of Education and the National Observatory, feedback from the five regions and the results were positive as to expectations being met notwithstanding the delay in procurement of equipment for some institutions.

90. Overall, it was an agreed view that the main outputs were delivered. However, there was also a broad perception that it owed a great deal to the efforts of the ETF project management to secure these results in the latter phases of the project.

3.9 ETF Project Management and Dissemination 91. The perception of the main stakeholders was that dissemination of results, materials and

publications for the four components took place. A dissemination strategy was developed and guided by ETF as evident in their reports. The Ministry of Education considered general support was evident for the process of change, and that new approaches were introduced under the project with a gradual acceptance of new methodologies by beneficiaries. Regional seminars, ‘Delphi weeks’, dissemination events in five regions and policy advice seminars facilitated the sharing of new ideas and approaches as well as the dissemination of results, methods and materials. However, the degree of their full acceptance is difficult to estimate especially at regional level. Materials from EU sources and the demand approach are used within the Ministry of Education.

92. Consultations with national experts found that the dissemination of results for component II continued as the regions face similar problems. This is evident at the various conferences they attend. One constraint is that the papers of the experts cannot be downloaded, as the web site is not maintained. Due to the lack of funds available, new materials cannot be printed in addition to those already printed and distributed. Therefore no formal post-project dissemination is currently being carried out. There is a serious concern therefore as to whether all the good results for component II will be sustained and disseminated to support continued improvements in management training.

93. In terms of the acceptance of the new ideas for management based on a market approach, the evaluation was advised that there was initial resistance to the transfer of this new knowledge under the project. A factor is that the grant-funded Tacis projects are seen by beneficiaries (as are other donors) as ‘money transfer’ rather than for the purpose of transferring of knowledge and skills. The Ministry of Education was also unclear as to the needs and expectations. Now acceptance can be seen in the plans that the experts see being produced which reflect an internalisation of the ideas. Dissemination of ideas within an institution depended on the level of the regional manager within the institution as well as the degree of interest. It also depended on finding a market niche in the region.

94. Under component III (VET reform) new methodologies were introduced, for example, labour market analysis through the pilot schools. A strong point of the project was the quality and



dissemination of the materials produced. Methodologies and materials were written by Russian experts and not by experts from ETF or the contractor – the National Observatory and the Ministry of Education considered the materials for VET were good. They were disseminated to the main libraries for all 89 regions and most VET schools, and the changes introduced are being accepted as the National Observatory has in the regions where both attitudes and mindsets are changing in respect of VET.

3.10 ETF Project Management: Synergy with Other Initiatives 95. Both the main stakeholder and national experts considered that the project management had

endeavoured to develop synergy with other related initiatives of Tacis. This was as an added value that ETF brought to the project and its management.

96. Delphi was not an isolated initiative as other Tacis projects have contributed to education reform. As the EC urged at the final conference in October 2001, Delphi also needed to be seen within the broader context of economic reform and the development of civil society, and the range of projects that Tacis supports. For example, Delphi I used the results of the White Paper on education in the Russian Federation under the Tacis Education Management Project, while component III drew its ideas mainly from the project for NW Russia VET which is an ETF project.

97. Other examples include:

• ETF and the contractors developed contacts with the parties involved in the World Bank loan for reform in education in order to find any potential synergies.

• ETF drew upon its National Observatory Network to facilitate for the contractor a study visit to the National Observatory in the Czech Republic for beneficiaries especially in relation to the concept of social partnership, which Delphi developed and promoted with the project partners.

98. Most synergy was achieved with the Presidential Initiative in respect of the Management Training Programme under the Federal Commission on Organisation of Managers and Executives Training of Enterprises of the National Economy of the Russian Federation and which also has Tacis funding. Consultations found concurrence with the claim of ETF as to the contributions made to the MTP as follows:

• Adaptation of the target group of Component II (from undergraduate students as proposed by the MoE to post-experience training for managers with a clear market and demand led focus to management).

• Participation in the identification of gaps in the quality of teaching in the Russian management training (i.e. provision of experts to the Tacis/ETF Bistro project BIS99/092/005).

• Contribution to filling gaps and material quality improvement; the three best-selected materials were awarded.

• Staff development of Russian Institutions taking part in the Presidential Initiative through participation in competitive tender for DELPHI support for regional projects in management training.

• Contribution to organisation of two conferences in Moscow and Novosibirsk to disseminate the outputs of the Bistro project BIS99/092/005 to the whole Presidential Programme.

• Training of teachers to fill the gaps identified by the quality analysis under Dephi I.

99. Certain activities still need to be done. It is assumed they will be part of Delphi II:



• Training with regional project partners, selected institutions from the Presidential Programme and additional teachers from the Presidential Programme covering all of the gaps identified by the quality analysis by the Bistro project BIS99/092/005.

• Joint activities with the Management Development Support Fund to ensure synergy of work done so far by Bistro/Delphi with the intention of MTP to undertake further gap analysis.

• Contribution to improved service which participating DELPHI-Presidential training institutions will give to managers in preparation for work placements abroad sponsored by the MTP.

• Possible synergy with MTP by sending teachers on internship in EU.

3.11 ETF Project Management: Added Value to EC 100. Based on the overall comments from the experience of Delphi I, and as outlined in the relevant

sections of this report, there is clear evidence to support ETF claims that they provided added value as project managers.

101. Stakeholder perceptions of this added value are more weighted towards a reflection of individual contributions and continuity of related experience than in terms of institutional capacity. It was not clear if many respondents differentiated the two in their assessment. While it is acknowledged that the individual is a representative of an institution and provides services on its behalf, this nuance is not always apparent to interlocutors in the context of managing a project. This was particularly so in relation to the links between component II on management training and the Management Training Project (MTP) under the Presidential Initiative. There was added value to the MTP as claimed by ETF but MTP would have continued even if there had been no Delphi project.

102. There was a consensus that ETF could give added value to the management of projects compared with the EC through the creation of synergy between ETF-managed projects such as Delphi and other initiatives such as MTP. ETF endeavoured to establish links within Delphi and to MTP. While there were formal arrangements with institutions and on the surface an acceptance of links and common goals that bespoke coherence, it was not clear whether this approach had been fully internalised.

103. ETF were seen to have taken a more considered approach to issues such as objective orientation and synergy, whereas the EC is more concerned with the formal arrangements. ETF looks at the content of the TOR and seeks to make sense of its meaning in the reality of the project partners’ environment. In comparison, the EC looks at the formal requirements of a TOR and the outputs to be delivered. Their main concern is delivery according to time and financial parameters, and monitoring by Tacis is in those terms and relates to performance and outputs.

3.12 Continuity from Delphi I To Delphi II 104. Delphi II is intended to build on the results of Delphi I. Delphi II will work in new regions and

will be more linked to the reform process. There is a basis for Delphi II from the outputs achieved for Delphi I, and the Ministry of Education indicated that ETF and the experts in Delphi I would have an understanding that would be useful for Delphi II.

105. ETF was providing a bridge from Delphi I to II by writing the TOR for the latter. Concerned stakeholders considered that ETF could be more pro-active and visible in this bridging role, in order to maintain the dialogue and momentum and encourage a response from the Ministry of Education to the draft TOR (which is delayed) in order that momentum for project preparation might be maintained.



3.13 Conclusions on ETF’s Project Management of Delphi I 106. Stakeholders agree that ETF’s previous involvement with and experience of projects in Russia

helped it to establish viable modalities for project management. As a result of this continuity, the management systems were structured on a basis that emphasised the role of the beneficiaries and Russian partners and did not just rely on the EU contractors. ETF tried to ensure coherence between components, and assisted all the project partners in understanding how they could participate jointly in project implementation. The National Observatory for the Russian Federation was both a source of expertise and support in the management role.

107. Project management of ETF sought to maintain the objectives set for the project with defined targets and which broadly met stakeholder expectations. Supervision by ETF was significant to the achievement of the final results and any coherence between the project components. The Tacis monitoring reports pointed to the lack of synergy between the components.

108. While there was a general perception that ETF can add value to project management through its capacity to provide content/qualitative monitoring compared with the EC, this was not evident to all parties. Each stakeholder/national expert/manager for a component appears to have different experiences, which influences their perception of any content monitoring by ETF. There is a need to distinguish whether this is because of the capacity of individuals who have built up a standing and track record with the project beneficiaries (as is the case with Delphi) or the capacity of ETF. The perception of the added value of content monitoring by ETF is perhaps more in terms of the former.

109. Overall, it was an agreed view that the main outputs were delivered. There was a general perception that this owed a great deal to the efforts of the ETF project management in the latter phases of the project, viz., “the project produced all planned results (against agreed outputs). The quality and quantity of outputs was considerably higher and the final project results were more fundamental than expected.” [Tacis end of project assessment report April 2002].

110. While results were achieved and most expectations met, ETF were seen to have had some project management shortcomings. It is not clear to what extent this is because of the procurement problems or the overall management by ETF, but the procurement problems clearly contributed to a negative perception by some national stakeholders. However, this does not square with the consistent opinions in the various monitoring reports, which are positive. In August 2001 management was regarded as “incisive and stable.” [Tacis monitoring]. Nonetheless, had ETF acted earlier on the procurement problems that the Monitoring Unit had reported consistently to the EC, then this negative perception may well not have arisen.

111. However, the common positive assessment of the project management of ETF seems to reflect more on the individual ETF manager than ETF as an institution. It was the efforts of the former that were perceived as the key factor that allowed the project to come to a relatively successful conclusion in terms of the good outputs delivered.

112. The main tasks of the day-to-day management fell to the contractor and ETF had a supervisory role for the latter. Notwithstanding some misunderstanding over the separation of roles between the contractor and ETF, the overall view was that ETF’s supervision was “appropriate, timely and effective.”

113. ETF built dissemination into project activities to project partners and beneficiaries, and both the main stakeholders and national experts considered that the project management had developed synergy with related Tacis initiatives and especially MTP. Networks set up under Delphi I have good potential for management. ETF built networks under the project and has a good standing in relation to management training. This is an added value they can bring to the EC.

114. ETF needs to be more pro-active in promotion of its services and the added value it can bring to project management for customers such as the EC. It needs to identify clearly and objectively its strengths, and market them as part of an overall institutional strategy that is focused on the needs of its customers. Such a project management role might be more in respect of strategy and



support to policy development rather than technical content except where there are recognised strengths such as in management training.



Appendix 1: Project Synopsis from Final Report of Contractor 30 November 2001

Project Title: Delphi - Development of Educational Links and Professional and Higher Education Initiatives

Project Number: EDRUS9706

Country: Russian Federation

Wider Objectives: Education system responds to the assessed needs of economy and society in the fields of Economics, Business Administration and Law Teaching and Vocational Education and Training using improved Open and Distance Education

Component I Education Management

Specific Component Objectives: To support the Ministry for General and Professional Education (MGPE) in formulating education strategy, policy planning and implementation in co-ordination with other Ministries and social partners

Planned Outputs:

IA) Forum for structural dialogue established at federal and regional levels between the Ministries of Education, Labour, Economics and Finance, representatives from employer and employee organisations and from individual companies

IB) Methodologies for social dialogue and labour market assessment in EU and Central and Eastern European Countries investigated, developed and implemented in Russia

IC) MGPE is aware of issues like equal access to education, social and cultural education and indicators for educational planning

ID) MGPE is aware of major international co-operation programmes in education

IE) Experience of DELPHI disseminated to at least 30 regional administrations

Project Activities:

IA1) Definition of long term agenda and desired outputs of the Forum

IA2) Organisation of regular meetings of key actors in the education dialogue institutionalised at federal and regional levels

IA3) Development and implementation of a strategy plan for involving social partners in education

IB1) Investigation of different methodologies for social dialogue and assessment of labour market needs

IB2) Development of a model for labour market needs analyses and methodologies for social dialogue

IB3) Implementation of regular labour market needs analyses and social dialogue in the Russian Federation

IC1) Organisation of workshops

IC2) Policy advice and recommendations

ID1) Analysis of donor programmes

ID2) Considering the establishment of an MGPE data base

IE1) Promotion Conference of the DELPHI programme

IE2) Information about the progress and the outcomes of the DELPHI programme

IE3) Dissemination of experiences at regional level



Component II Teaching of Economics, Business Administration and Law

Specific Component Objectives: New, innovative education and training solutions are provided to regional labour market needs in Applied Economics, Business Administration and Commercial Law. Stakeholders have been trained in expressing labour market need and describing professional profiles, new teaching and learning methods are applied and new networks established according to the effective needs in the selected economic sectors.

Planned Outputs:

IIA) Management Training. Staff of institutions that are involved in the Presidential Initiative, plus others where possible, in ail five regions, are engaged in a development plan. Staff acquires the skills to solve the human resource problems the companies in the respective regions, assess the tra ining needs of the managers and executives and work with them to meet their requirements. Economics staff of higher education institutions in the field of economics is aware of the successful experience of economics teaching in Russia, including courses and materials developed, and are able to deliver courses that suit the contexts of the economic reality in Russia and the needs of the regional economy. The target groups also include teachers who are involved in the Presidential Initiative in the subject area of ‘Finance’. There will therefore be overlap with the Management Training group. Law staff of higher education institutions in the field of law is able to deliver courses that address the needs of enterprises and legal specialists in the private and public sector. Special attention will be applied to commercial law. There may be synergy with the Management Training and Economics subject areas.

IIB) Academic managers (Deans, programme designers and course leaders) in the institutions are capable of establishing and maintaining effective relations with companies, creating opportunities for internships and joint projects. Managers should create incentives to ensure the participation of teachers in university-industry liaison, networks with other institutes to allow benchmarking of teaching standards and management. The regional local administrations, managers of human resources are aware of and facilitate the dissemination of new approaches to skills -based training in the future and have upgraded skills relevant to their job.

Project Activities:

IIA1) Identification of areas in which teaching and learning methods should be improved in management training (‘Bistro’ quality analysis).

IIA2) Preparation for and imp lementation of training activities to implement the teaching and learning methods.

IIA3) Re-design courses and diversify teaching approaches.

IIA4) Acquisition and use of previously produced materials will be facilitated by means of a database produced collaboratively with the Bistro project. A Bistro/Delphi peer review will identify the best of those materials and will provide an incentive to the authors if improvement is needed.

IIA5) Two conferences in Moscow and Novosibirsk on the implementation of change plans for the teachers in the Presidential Initiative.

IIB1) Upgrading of educational management by training institutes' staff in managing relationships with enterprises, practical placements industry and commerce, alumni and fund-raising activities, etc.

IIB2) Upgrade teachers' skills in marketing educational consultancy and training services.

IIB3) Identification of joint work between the institutions and a sample of local companies, e.g. analysing organisation changes, needs for restructuring, s trategic development issues, etc.

IIB4) Introduction of sound financial management of courses with cost-control and budgeting systems to ensure sustainability.

IIB5) Run mid-term workshops in the regions with stakeholders to stimulate networking activity and evaluate lessons learned.



Component III Vocational Education and Training

Specific Project Objectives: Efficient and applicable VET reform model based on the labour market needs of different economic sectors has been developed

Planned Outputs:

IIIA) New occupational profiles identified on the basis of labour market assessment, curricula and teaching materials developed and training courses implemented for a limited number of occupational profiles selected in each of the sectors

IIIB) Teachers and trainers are trained to apply and continuously develop curricula, new teaching materials and methods

IIIC) Policy makers have relevant tools to develop, monitor, evaluate and stimulate the VET reform process at federal and regional levels

IIID) A new approach for developing vocational standards has been approved and applied

IIIE) The federal and regional education authorities and institute managers are able to apply new methods in financial planning of VET, in particular cost estimation and accounting

IIIF) New links between administrators and social partners as well as between vocational training institutions and enterprises have been created and new possibilities for such co-operation identified and applied

IIIG) Continuing training activities have been integrated in the day-to-day activities at institutional level in one of the regions involved

Project Activities:

IIIA1) Collecting and analysis of skills needs from the labour market

IIIA2) Improving and Developing Curricula

IIIB1) Training of trainers

IIIB2) Training of teachers

IIIB3) Feed back to policy and strategy: Evaluation of new methods and materials

IIIB4) School Management

IIIC1) Workshop on VET reform

IIIC2) Support the design and establishment of an assessment and evaluation system IIIC3) Staff development programme

IIID1) Introductory workshop

IIID2) Development of integrated model of standards in VET

IIID3) Implementation of integrated model standards in VET

IIIE1) Comparative analysis of methods of financing VET

IIIE2) Development of plans

IIIE3) Review of the implemented plans

IIIF1) Workshop on co-operation in education

IIIF2) Establishment of systematic networks between VET institutes, administrations, enterprises and social partners and set up of structural models for co-operation

IIIF3) Staff training

IIIG1) Capacity building for the selected institutes

IIIG2) Development of 2 new continuing training courses

IIIG3) Feasibility Study



Component IV Development of Open and Distance Learning for Dissemination of Education

Specific Project Objectives: Increased access to management training and economics education through improved Open and Distance Learning system methodologies and learning and teaching materials.

Planned Outputs:

IVA) Materials from Tacis projects in management training and economics have been analysed, relevant learning and training materials selected and by materials developers adapted for open and distance learning purposes

IVB) Teachers and tutors have been trained to apply newly developed curricula teaching materials and methods

IVC) A database on existing ODL learning and teaching materials and newly trained teachers and tutors has been set-up

IVD) Networks of the federal and regional training institutes, enterprises and European training partners have been developed

Project Activities:

IVA1) Adaptation of materials for ODL purposes

IVB1) Training of teachers and tutors on ODL

IVB2) Support to delivery of ODL courses by local training institutions

IVB3) Marketing and training

IVC1) Development of databases of materials and trainers/tutors

IVD1) Network building

IVD2) Dissemination

IVD3) Policy advice and quality development

Project starting date: 02.12.98

Project duration: 36 months



4. The Algeria case study

4.1 Introduction and background 115. ETF’s geographical mandate was extended to the Meda region in 1998 (Council Regulation No

1572/98), coinciding with the start of a period of major organisational reform within the European Commission, and with the structured dialogue that took place between ETF and the Commission in 2000. In this context, it was clear that ETF would not be required to manage projects on behalf of the Commission, but should rather develop a new role more in line with the then emerging consensus that the Foundation should strengthen its role as a centre of expertise. This offered the Foundation the opportunity to take a fresh approach to developing its role within the region.

116. In preparation for the extension of ETF’s mandate, the Commission prepared a memorandum of understanding, which set out the following framework for ETF activities in the region:3

• To establish contacts and communication channels with relevant national and multilateral actors in the region, including major donors;

• To define further the role of the Foundation and the type of services to be provided to the Commission in the context of the MEDA Programme and the Euro-Mediterranean policy;

• To develop Foundation knowledge and understanding of the Mediterranean partners’ specific needs, expectations and capacities;

• To implement pilot activities in a number of Mediterranean partners.

117. In 1999, ETF was requested by the Commission to pay particular attention to Algeria, where the EC was resuming its activities after some years during which the security situation had militated against any extensive external aid activities. To-date, the following activities have been funded by ETF in Algeria:

• Facilitation of the participation of Algerian stakeholders in review seminars in 1999 and 2001 on Tunisian VET system reform and progress

• The preparation of a Country Report in 2000, to provide an overview of the Algerian VET system and reform process

• A Maghreb seminar in September 2000 on financing mechanisms for Continuing Education and Vocational Training, to support the launch of the Fonds National pour l’Apprentissage et la Formation (FNAC)

• Provision of support to the launching in 2001 and ongoing development of an inter-institutional observatory network, linked to initiatives in other Meda countries to develop a regional network

• The organisation in 2001 of a study visit for Algerian stakeholders to Sweden and France, to expose them to best practice for the ‘Baccalaureat Professionel’, which is planned to be introduced in Algeria in 2002.

118. In September 2001, the Regional Framework for Foundation activities in the Mediterranean Region during the period 2002-2004 was agreed between ETF and the diverse Commission services concerned (RELEX, AIDCO, EAC). This framework committed ETF to helping the individual Meda partners to develop socially balanced and cost effective training systems, which are geared to the employment market. The specific services to be provided by ETF were set out as follows:

3 ETF Annual Report, 1999



• To support the European Commission to develop and implement MEDA assistance in vocational education and training.

• To support and reinforce the capacity of partners to analyse and anticipate labour market needs and develop and implement the vocational education and training system strategies, policies and programmes accordingly (the Observatory Function).

• To support the transfer of European and Meda region best practice in vocational education and training system development.

119. In addition to the above, and at the request of EuropeAid, ETF were commissioned in 2001 to undertake the technical preparation of a €60m VET reform programme, co -financed by the Algerian Government to the tune of €48m. This involved the preparation of an Identification Report, a Feasibility Study, and a draft Financing Proposal. The programme has been approved and implementation is due to commence in early 2003.

120. The resources expended by ETF on their activities in Algeria are summarised as follows:

Figure 1: Expenditure on ETF activities in Algeria

Staff Mission Operational1999 8,000 2,000 23,000 33,000 2000 37,000 9,000 170,000 216,000 2001 47,000 9,000 105,000 161,000 Total 92,000 20,000 298,000 410,000

CostsTotalYear

4.2 How is ETF perceived? 121. ETF has successfully achieved a high degree of credibility among Algerian stakeholders. It is

perceived to have made a number of significant contributions to the ongoing VET reform process, and to the Commission’s cooperation within that process.

122. From the start of its involvement in Algeria, ETF is seen to have provided a vital window on best practice and innovations elsewhere in the region and in Europe. This has enabled the Algerian authorities to improve their knowledge and understanding of VET reform issues and practices and the options available to them, as well as improving their self-confidence in addressing the challenges that they face.

123. Examples of the added value that ETF are perceived to have brought to the Algerian authorities include:

• The Country Report produced in 2000 not only provided a comprehensive analysis to consolidate that already done by the Algerian authorities, it also moved the internal debate on, exposed them to the analysis techniques and methods used, and gave them greater confidence to discuss their problems and concerns with other countries in the region.

• The conference on funding mechanisms for Continuing Education and Vocational Training in 2001 not only facilitated a useful exchange of experiences, but also provided a stimulus to greater cooperation and openness between the many institutions and stakeholders concerned with VET reform in Algeria, as well as transferring practical knowledge and skills in the organisation and management of such events. It also helped considerably in raising the profile of ETF in Algeria.

124. ETF’s most direct link with the Algerian authorities has been through the Ministry of Vocational Training, and the Institute of Vocational Training (IFP) based in Algiers.4 The two Algerian

4 There is an IFP for each of Algeria’s regions



Advisory Forum members are based in these institutions. This relationship has been particularly strong, and has provided a mutual reinforcement of each other’s role in the Algerian context: IFP has provided ETF with the opportunity to develop an effective relationship with the key stakeholders involved in the reform process; while ETF has helped to improve the confidence and capacity of IFP and its standing with other Algerian stakeholders. The current relationship can be characterised as one of friendship, trust, and common purpose.

4.3 Using Title 3 funds 125. There is a recognition that the Title 3 funds available for ETF activities in Algeria are limited,

and this had led to some concern that the fruits of Algeria’s cooperation with the Foundation were slow in coming. However, with the growing credibility of ETF and the qualitative benefits that were perceived to have come from the small range of ETF ‘projects’, there has been an apparent acceptance on the part of the Ministry of Vocational Training in particular, and the wider stakeholders in general, as to the role and importance of ETF. Certainly within the Ministry of Vocational Training there is an understanding that while the Foundation does not have the material means to do much in the way of ‘concrete’ activities (i.e. projects), its impact as a catalyst for reform and capacity building has been profound.

126. It is important to note that all of these activities have a coherence that has contributed to the value added provided by ETF. All activities appear to be building towards the Meda VET reform programme and to the wider reform process – for example, the Country Report provided a good basis for identification of the Meda programme, while the observatory network, and capacity building for FNAC are all elements that are included in the Meda programme design. This would appear to attest to a successful strategic approach in allocating the limited title 3 funds available.

127. Despite the fact that ETF has demonstrated its ability to generate added value from limited funds, and the broad agreement that its role is not that of a funding agency, a number of stakeholders would like to have seen more funds being made available – in particular to support the Fonds National pour l’Apprentissage et la Formation. Since the seminar on financing mechanisms for Continuing Education and Vocational Training in September 2000, ETF has not provided any further funding to build FNAC’s capacity. While ETF might take the view that this was a conscious and pragmatic decision on their part to direct their resources elsewhere, the perception on the part of Algerian stakeholders is that limited but carefully applied funding could have helped FNAC to develop its capacity in preparation for the forthcoming Meda VET reform programme. As things stand, there is serious concern that FNAC faces serious internal capacity constraints represent a significant risk to programme implementation.

4.4 Support to the Commission 128. Much of ETF’s credibility with Algerian stakeholders comes from the perceived quality of the

expertise it has been able to mobilise, both internally and externally, as well as the seriousness with which they have approached their tasks. This has been exemplified by the manner and quality with which ETF have prepared the Meda VET reform programme.

129. From the Commission’s perspective, the programme has been well-prepared. Its design has met the requirements of Project Cycle Management, and offers an example of good practice for similar programmes elsewhere – in particular, stakeholders were closely involved thus ensuring a high degree of national ownership of the programme. Specific expertise in VET reform is not present in the same depth in Commission services in Brussels and the Delegation (requiring as they do a broader expertise in a range of sectors), nor was it present in the EC Meda team that previously existed. ETF has been able to bring to bear this specific expertise, which building on their continuity of involvement since 1999, has contributed to what is acknowledged as a sound programme analysis and design.



130. This has been borne out by the stated perceptions of Algerian stakeholders that the design reflects national priorities, and that at all stages in its preparation the programme design has been validated with them. The overall approach of ETF has been seen as pro-active and sympathetic, avoiding the conditionality-driven approach that sometimes accompany negotia tions for donor-funded programmes.

131. Some concerns have been expressed about the feasibility of the implementation structure and modalities, with the view expressed that with hindsight the ETF team could have benefited from more practical experience in implementation issues – particularly in how to deal with the critical capacity constraints that exist among Algerian institutions. Nonetheless, there is a degree of optimism on the part of the Ministry of Vocational Training that with ETF’s assistance and support, programme implementation issues will be addressed as they arise.

4.5 The observatory function 132. The primary objective of ETF funding for the observatory function is to reinforce Algerian

capacity to collect and analyse information, and to forecast needs for employment, the labour market, qualifications and training. The approach adopted for the development of the observatory function in Algeria is very different to the historical approach applied in other regions of ETF activity, and represents a pilot version of the approach being applied in all other Meda countries. The key new aspects of the approach are as follows:

133. The observatory function is to undertaken by a coordinated network rather than primarily by a dedicated institution, as is more commonly the case in Tacis and Phare countries.

134. There has consequently been a greater focus right from the start on embedding the function in the existing institutional landscape in Algeria, rather than on creating a new institution; as well as on ensuring that the observatory function will serve directly the needs of Algerian VET reform and labour market development.

135. The regional dimension will be addressed through convergence of approaches rather than through benchmarking of labour market performance in the various countries.

136. The approach in Algeria builds in particular on two elements – the experiences gained by ETF through its involvement with the National Observatories in the Phare, Tacis and Cards regions; and the priority expressed on the part of the Algerian authorities to more closely link service provision by the various VET and labour market institutions to real labour market needs.

137. A working group was established in mid 2001 to formulate and implement an agreed approach to development of the observatory function, and met for the first time in June that year. The group comprises representatives of a range of organisations concerned with vocational education and training from government and the private sector, and has operated with the support of the ETF Country Manager for Algeria, and other internal and external experts. Over the period since its creation, the group has formulated all aspects of the proposed network, from its strategic objectives through the information ‘domains’ to be addressed, to the costs and responsibilities for making the network operational. Notwithstanding the usual teething problems that one would expect with the creation and start-up of such a group, there appears to be a wide recognition among the members of the working group that the initiative has been a success in terms of the ownership felt by group members, and the political importance assigned to it by the Ministry of Vocational Training. In a relatively short period of time therefore, the observatory function in Algeria has gone through a successful design stage and is clearly embedded within the priorities of the institutions involved.

4.6 The Advisory Forum 138. The Advisory Forum members are the key contact for ETF in Algeria. The current members have

held their positions since the start of ETF’s involvement in Algeria, thus providing excellent



continuity and doubtless contributing to the strong relationship that now exists between ETF and the Ministry of Vocational Training.

139. Contact between the Advisory Forum and ETF has been regular, and has facilitated the maintenance of close contacts between ETF and other stakeholders in the VET sector. Initially (and as stated previously), ETF faced a challenge in establishing its profile as a support to policy analysis and formulation rather than as a funding agency. The mutual reinforcement of roles between ETF and the Advisory Forum has helped to address this challenge, and to position ETF where it would like to be.

140. The Algerian Advisory Forum members were appointed one year into the previous three-year cycle, and were confirmed as members again for the current cycle ending in 2003. They have therefore had experience of the previous thematic composition of the Forum, and the current regional/sub-regional composition. Perhaps given their period of relative isolation until 1999, the Advisory Forum members expressed their appreciation of the previous composition, which exposed them to a wider geographical range experiences and therefore arguably a more diverse learning environment. Whilst appreciating the benefits of the current regional composition, they feel that the commonality of problems between the Maghreb countries does not generate as rich an information exchange as was previously the case. With full plenary meetings of the Advisory Forum (comprising all regional groups) now taking place only every three years, they feel they are missing out on opportunities for further reflection and learning – for them, this is one of the major strengths that ETF offers. Their perception is that the change in composition was made more for budgetary rather than technical reasons.

141. Although the relationship between the Algerian Advisory Forum members and ETF is strong, contact between Advisory Forum members (in the Meda region and more broadly) between meetings is not so good, depending as it does on the motivation of the membership – something that has proved a bit of a disappointment to the Algerians. They would like to see a more structured approach, perhaps through smaller cross-regional thematic meetings between the main regional meetings.

4.7 ETF’s future role in Algeria 142. ETF has established a strong platform from which to build for its future activities in Algeria. It

has clearly demonstrated that with its combination of regional and country knowledge, its corporate memory, networking capabilities, and sympathetic approach, it can make a significant contribution.

143. However, in a peculiar way it may be in danger of becoming a victim of its own success. Its acknowledged role as a catalyst and valued partner has raised expectations on a number of fronts – expectations that may not be consistent either with its role as a centre of expertise, or with its capacity to respond. The challenge is twofold – to clarify the Foundation’s future role vis-à-vis the Meda VET reform programme; and secondly to maintain its engagement in the wider education reform process without spreading itself too thinly.

144. With regard to the Meda VET reform programme, it is clear on the part of the Commission and the Foundation that ETF does not have a direct role to play in its implementation, nor that of other EC-funded programmes in the Meda region. Yet in Algeria there appears to be a range of possibly competing expectations as to ETF’s role:

• Some expressed the desire to see a strong role for ETF in helping to deal with implementation problems, and to help the Algerian authorities to steer the programme.

• Some would like to see ETF monitoring the impact of the programme at the policy level, leaving content monitoring to others.

145. The dividing line between these two functions (implementation support, and monitoring) is perhaps not as clear as might first seem – especia lly as the size and scope of the programme



mean that it forms a significant element of the government’s overall reform programme, and the fact that the activities of most of the Title 3 funded projects will be subsumed into the Meda programme.

146. One example of this lack of clarity is the role currently being undertaken by the ETF Country Manager in preparing for start up of implementation next year. He is working with the Algerian authorities to assist in defining the operational modalities, and putting in place the practical arrangements necessary. This arrangement only came into being after an attempt to appoint a suitable consultant proved unsuccessful, and it also demonstrates ETF’s flexible response to national needs,5 as well as the Ministry’s confidence in its capabilities. Nonetheless it does appear to the evaluator as more of a hands-on role than the ‘centre of expertise’ should normally take, and will clearly have an opportunity cost in terms of the activities that the Country Manager could otherwise become involved in.

147. A second potential problem is the observatory function. The observatory working group has received much advice and support from ETF, and yet the observatory function forms part of the Meda programme. How will ETF maintain its support to development of the observatory function without becoming embroiled in an implementation role? In fact it will continue to fund the regional element separately from the programme, and much support will be required before the Observatory function becomes operational. Nonetheless the potential for raising (and disappointing) expectations is clearly there.

148. It appears that ETF’s role during implementation of the Meda VET reform programme has not yet been discussed in any detail. Given the above issues, this urgently requires clarification before further expectations are raised. This must in turn be clearly communicated to Algerian stakeholders, and expectations as to the Foundation’s role should be more systematically managed – especially with regard to differentiating from the role of the Commission. The lesson of the criticism of ETF’s perceived lack of continuity of support to FNAC should be learned – even if the decision not to provide further direct support was a sound one, it appears not to have been well-communicated to FNAC or to other key stakeholders. Given ETF’s position of trust in the Algerian context, open and transparent discussion of its role and decisions should not present a problem.

149. With regard to a possible role in the wider reform context and the education sector as a whole, there is a desire at senior levels within the Ministry of Vocational Training to see ETF make a contribution. Specific areas for future support might include developing dialogue with other Meda partners, EU Member States, and international bodies; and assisting the Algerian authorities in further developing its policies and plans. The limited Title 3 budget is clearly seen as a constraint to this, and many would like to see the budget for ETF activities increased.

150. In conclusion therefore, ETF has achieved a significant amount with limited resources, and has established a position of credibility and trust with Commission and partner country stakeholders. These achievements provide a solid basis for further developing its role, and for continuing to make a difference in the longer term. However, ETF is operating at close to full capacity in Algeria, and faces the risk of raising expectations that it cannot meet, and therefore of damaging its excellent reputation. The period prior to start up of the Meda VET reform programme is a vital opportunity to discuss and agree the Foundation’s medium term role for 2003 onwards, and in particular to clarify its role with regard to the Meda programme.6

5 ETF see work as an important means of ensuring that the philosophy and strategy of the programme are transferred to the

Algerian implementation team, to assist in its subsequent steering and management. 6 Subsequent to the submission of the first draft of this report - check against new date etc and tense of this may have to

shift , ETF have clarified that the role of ETF in Algeria will focus on:

• the experience gained in preparation of the MEDA VET reform programme, and how this will be fed into the preparation of future projects for the EC

• assisting in the implementation of current reforms, and the monitoring of their impact



5. The Social Partners case study

5.1 Introduction 151. The aim of the case study of the Social Partners Project was not to evaluate the project but to

look at issues of effectiveness specifically in terms of the management and organisation of the project and the capacity ETF has in managing such projects. It also reviews how this role fits into ETF’s new institutional strategy.

152. The main means of data collection was document review to which the Social Partners Project provided a comprehensive list of documents (see bibliography). A series of subsequent questions were put to representatives from the EU, ETF, National Observatories and social partner organisations via email. Although not ideal, the method of email survey saved time and to some extent lessened potential language difficulties. Questions and areas of interest of the Case Study fitted in with the overall objectives of the Evaluation of ETF and centred mainly on issues of effectiveness.

5.2 Description of the Social Partners Project 153. In 1996 a sub-group7 of the Advisory Forum to ETF focused its discussion on the role of social

partners in the development of VET in the Candidate Countries8 of Central and Eastern Europe. Conclusions from this meeting recommended further ETF action in this area. As a result ETF developed a project proposal for action in this area which was subsequently included in ETF’s Work Programme for 1997-9, utilising Title 3 funds.

154. The proposal specified the following activities:

• To observe, register and analyse the partner country9 initiatives in relations to social partners and VET.

• Convince the policy decision-makers of the importance of social dialogue regarding VET.

• Provide technical and methodological support for the social partners who commit themselves in this direction.

• Support the involvement of social players from the partner countries in ‘Leonardo’ projects set up by European Union social partners.

155. The proposal also suggested the means by which some of these initiatives could be achieved which included the involvement of the National Observatories, the development of country monographs, case studies, study missions and programmes of co-operation within the partner countries and EU Member States.

156. The project proposal highlighted three overall objectives which were:

7 Sub-group B 8 Also referred to in the text as partner countries 9 Romania, Bulgaria, Czech Republic, Estonia, Hungary, Lithuania, Latvia, Poland, Slovenia and the Slovak Republic



• To develop an action-oriented, learning and self-sustainable network with a view to reinforcing the regular communication links among the social partners in the involved countries as well as improving their competencies in addressing policy or technical aspects with regard to VET;

• To promote the culture of social dialogue on VET in the partner states participating in the project; and

• To reinforce the institutional capacity in these countries for developing effective practices of social dialogue in VET.

157. It also had three immediate objectives, as follows:

• To provide a first platform of knowledge and information which will enable the comparative analysis of significant experiences of social partners’ involvement in VET. This platform would be mainly provided by the country reports as well as the final report of the planned international workshop;

• To raise the awareness of both decision makers and social partners in the partner states about the main challenges relating to the development of the social dialogue in VET in their countries. The preparation and distribution of the National Reports as well as the discussion and exchange of views on them among all the participants in the international workshop are expected to contribute to this objective; and

• To contribute to the establishment of the above-mentioned network. The establishment of such a network is conceived primarily as the aim of the envisaged international and national workshops.

158. There were four stages to the project which were outlined in the project proposal. These are described alongside a brief description of the project outcomes (which will be developed in more detail throughout the text).

Phase One

Activities (as per project proposal): The preparation of Country Reports10 and a cross-country synthesis a cross-country synthesis which would outline the general situation regarding the involvement of social partners in VET in each country, to be submitted by September and October 1997 respectively.

Outputs: In 1997 background reports for the ten candidate countries and a synthesis comparative report were compiled. The reports investigated levels of social dialogue and key future challenges.

Phase Two

Activities (as per project proposal): An International Workshop to involve social partners from partner countries and Member States, the EC, CEDEFOP, ILO and the Dublin Foundation.11 The overall aim of the workshop was to enhance the discussion and exchange of knowledge among the participants on the basis of the information material included in the country papers of Phase One. The workshop was to be organised by December 1997 and the final report for the workshop was to be prepared by January 1998.

Outputs: A Conference was held in December 1997 in Turin bringing together 54 participants from social partners and government organisations in Candidate Countries and Member States, representatives from international organisations, the EU, ETF and CEDEFOP. A Final Report for the workshop was prepared.

11 About 40 participants were indicated.



Phase Three

Activities (as per project proposal): The publication of the Country Reports and the Final Report of the Workshop to be completed by February 1998.

Outputs: National Reports and the Final Report of the Workshop were published.

Phase Four

Activities (as per project proposal): The organisation of six national workshops between February and December 1998. The aims of the workshops were to disseminate in each country the outcomes of Phases One and Two, to fit the results of the project in with other related activities and strengthen the synergy between them, and to establish regular links of communication between the persons or networks involved in the above activities.

Outputs: National thematic workshops took place in Romania, Hungary, the Czech Republic and following a request by Lithuania, Latvia and Estonia 1998-9; there was a combined workshop for the three Baltic States.

159. The combined budget for Phases One, Two and Three was 100,000 ECU and the Phase Four budget was also 100,000 ECU. The first three phases were funded through the 1997 budget and the fourth through the 1998 budget. ETF hired a Project Manager who has been in place since 1997. A number of activities were subcontracted. These included the preparation of the cross-country report in 1997-98 to a EU expert, the preparation of the National Reports by local experts through the National Observatories and the logistical organisation of national workshops to the National Observatories.

160. In addition to the activities above a number of subsequent initiatives were identified. These are described with main outputs below.

• National Workshops for Poland, Slovenia, Bulgaria, Slovak Republic in 2000. For these countries follow up reports to the 1997 National Reports were also required. The structure of the reports was designed in line with similar studies undertaken with CEDEFOP and ILO. Expected outcomes included an evaluation of the capacities and institutional possibilities of social partner organisations in candidate countries to play an effective role in the development/implementation of national employment strategies in conformity with the requirements of the European employment guidelines.

Outputs: Four country workshops took place and four National Reports were produced in 2000.

• A study visit for Social Partners representatives12 from the Baltic region to an EU Member State was identified in the Estonian Workshop in 1998. The aim was to enhance the capacities of social partner organisations in the three Baltic States, to develop effective social dialogue practices in VET and to contribute to the shaping and implementation of VET policies in the Baltic States.

Output: A study visit for social partner representatives from the three Baltic States took place to Ireland.

161. After this stage the formal project ‘Social Partners Project’ was complete. However, ETF felt it was important to continue its involvement in the area of social partners in partner countries as part of its long-term support in this area. ETF now looks at social partners from a thematic angle and has incorporated the idea of social partners, for example, into an inventory on formal and non-formal learning. ETF has and will also be involved in a number of events which are outlined below:

12 10 – 12 representatives



• A seminar took place in Bratislava for trade union officials in candidate countries aimed at enhancing their involvement in vocational training policy.

• A Conference on Lifelong Learning and Social Dialogue took place in Malmo, Sweden in April 2001. The main objective of the conference was to provide a platform for the exchange of experience and information on the role of social partnership in the development of Lifelong learning as well as to further the understanding on the contribution of Lifelong learning to promoting economic performance and social progress. The discussions in the conference were also meant to provide an input to the current European consultation process organised by the EC on the six key messages of its Memorandum on Lifelong Learning. The conference brought together around 80 people representatives from the EC, the European Economic and Social Committee, European social partners, as well as social partners and officials from the Ministries of Labour and Education in the candidate countries, representatives from Sweden, other Member States and international organisations such as the OECD, ILO and CEDEFOP.

• The Greek Employers Association is planning to hold a conference that will involve social partners from Candidate Countries in 2003.

5.3 Social Partners: An Awareness-Raising Project 162. As the Social Partners Project was the first project to deal with social partners in the context of

VET many of the issues were new to project beneficiaries. The initiatives provided the stimulus for a discussion of social partners in VET, provided the platform for debate and thus increased awareness and knowledge of issues relating to VET and social partners.

163. Project reports provided the stimulus for the initial and further analysis of social partners in VET. The Estonian report suggests the preparation of the report helped to promote dialogue on the involvement of social partners in VET. In interviewing social partners they were able to make social partners think about the links between education and enterprise and to stimulate their interest in the developments that are currently taking place in vocational/professional education. However, one National Observatory explained how the team of writers for the report had suffered from the time conflicts and tight deadline for the delivery of the first draft.

164. The workshops were also important for the development of knowledge, awareness and future collaboration. Conclusions of the Romanian workshop suggested that the event had been a learning process in dialogue and confidence building among participants and the final documentation would serve as an action-oriented instrument to stimulate the social dialogue process in the area of VET. Bulgarian representatives asked what the most important developments in co-operation between stakeholders in VET had been in the previous three years. Increased awareness of the necessity of social dialogue between the partners was the main answer, along with clear national policy for development of the social dialogue and tripartite co-operation on an institutional level. One National Observatory stated that after the national workshop all VET activities had been subsequently viewed within a framework of social partners. Having said this, feedback from the Romanian workshop noted that only one Romanian social partner organisation had attended the whole duration of the seminar.

165. Feedback forms from the Baltic Study Visit to Ireland in 1999 demonstrate an awareness of the role of social partners in VET. Participants from Lithuania were able to use this information and relate it to proposed improvements in the VET system and social partnership in their own country. Over ten recommendations were prepared by this participant relating to the situation in Lithuania.13 Outcomes for the Study Visit explained how the project had enhanced the capacities of social partner organisations to develop effective social dialogue practices in VET and to contribute to the implementation of VET policies in their countries.

13 Although it is uncertain what became of these, the study visit seems to have been the stimulus for developing ideas.



166. Although it is hard to argue a direct cause-effect relationship with the Social Partners project, a representative from ETF believes the project could be considered as one of the factors which encouraged the following: the policy developments as regards the setting up of dedicated tripartite bodies in some countries (such as Czech Republic, Poland) and the high priority given to a new financing system for VET in Slovakia and in particular the increased awareness of the role of social partners in such new financing arrangements. The Slovakian National Observatory said that the workshop conclusions had influenced the Ministry of Education’s agenda (MoE Department of VET background documents and MoE measures towards SIOV on strengthening of involvement of social partners in VET).

167. In addition the project seemed to have a number of indirect outcomes which were not related directly to project objectives. One representative on the Baltic Study Visit noted the importance of specific industrial knowledge gained, other information and contacts. However, the importance of the role of social partners was not noted.

5.4 Dissemination of Project Outputs 168. Dissemination of relevant information and awareness-raising were one of the main objectives of

the Social Partners project and were mainly targeted at national stakeholders in partner countries. The main information products produced and disseminated through the Social Partner programme were the National and Cross-Country reports developed during Stage One of the project. These were published in the official country language and English and disseminated in partner countries via the network of National Observatories to conference and workshop participants as well as other stakeholders in the field of VET. National Reports were discussed in the national workshops during 1998-2000 and necessary adaptations were made in some cases as a result. EU practices and examples of experience of social partners in Member States were also discussed in these workshops. In addition, a discussion of the findings of the Cross-Country report took place at a conference in 1997, which key stakeholders from the EU Member States and partner countries attended. The conclusions of the workshops were also prepared and disseminated as separate documents. In addition, the Czech Republic made the minutes of the workshop and transcripts of presentations available. The Slovakian National Observatory made an electronic version of the documents which was e-mailed to interested people and placed on their website. Indication was made as to where these documents were disseminated. The national workshop in Poland suggested recommendations were to be forwarded to the public administration and all institutions involved in the development of VET programmes and the workshop in the Baltic indicated conclusions and proposals for actions would be presented to authorities for further discussion and possible action.

169. Other dissemination initiatives included a small-scale workshop held after the Study Visit of social partners from the Baltic States to Ireland had taken place to disseminate information gathered on the trip. One of the Estonian representatives of the study visit proposed that information gathered from the trip to Ireland would be available through a local social partner organisation.14 Additionally, information from the University of Cork in Ireland was to be sent to a number of Institutes and Universities in the Baltic States.

170. A representative from CEDEFOP stated that he had received national reports as well as other publications from the project in addition to information collected in the meetings he had attended. He explained that the information had subsequently been used in CEDEFOP presentations and workshops, as well as being a source for different CEDEFOP projects. The Slovakian National Observatory noted that CEDEFOP publications were also used by Comenius University within comparative education course in 2001.

171. Other feedback relates to the usefulness of the products. In the 1997 Cross-Country Conference delegates thought the National Reports and the Cross-Country Synthesis Report were very useful

14 The Estonian Meat Association



information tools. A questionnaire15 sent out in 2000 suggests the National and Synthesis Reports were helpful in terms of awareness-raising for public authorities and social partner organisations. The Polish National Observatory stated the National Report generated ‘a great interest.’ Feedback from the Baltic Study Visit indicates that the materials obtained would be used to assist the work of social partners, especially in terms of providing information on how social dialogue is carried out.

172. Having said this, there are a number of potential issues with documents and information disseminated. Firstly, although largely English translations of reports (useful for other partner countries and stakeholders in Member States) were generally good, there were sections of some documents where the English was difficult to understand. This could potentially lead to misunderstandings. Reports were long and only one had an executive summary (the inclusion of which may have assisted dissemination). One EU employee believes there is a lack of information on how social partners disseminate the information received. Without this additional information it may be difficult to keep track of the developing role of social partners.

173. There are a number of potential future needs with regards provision and dissemination of information. Many of the workshops16 called for additional information such as foreign case studies and details of good practice especially in terms of financing of VET initiatives. An internet or intranet site might provide the means to do this. Indeed representatives from ETF indicate that all relevant documentation will be made available on the new ETF web site.

5.5 Information Exchange and Stakeholder Cooperation 174. Country workshops provided an opportunity for representatives from partner countries and

Member States to exchange information on the role of social partners and VET development. A former EU representative describes the good exchange of information between social partners at the 1997 Cross-Country Conference in Turin. Results from the 2000 questionnaire point to the importance of these platforms for their ‘dialogic effect,’ discussion platforms and information exchange. It also explains how communication and exchange of experience links were established and/or strengthened between counterparts from the Member States and partner countries. Notes from the Romanian workshop indicate the importance of exchanging experience and especially of good practice. Feedback from the Baltic study visit highlights the importance of conversations between the representatives from the Baltic States which covered areas such as countries’ experiences and strengths and weaknesses. Cooperation between social partners was also promoted via the preparation of National Reports where experts from social partner organisations worked together in their preparation.

175. The collaboration and exchange of information during the Social Partners project proved useful for a number of participants. Feedback from the Slovak Republic’s seminar describes how the experiences provided from Hungary, Denmark and France could serve as a basis for the construction of the Slovak Republic’s VET Fund. The summary of the Romanian seminar explains the importance of exchanges of experience and good practice in the strengthening of social dialogue. Reports from the 1997 Conference considered the presentations and discussions during the seminar were very informative, giving new impetus to the improvement of VET systems in the partner countries and a strengthening of the relations between participants. A representative from CEDEFOP indicated how they have benefited in terms of their joint projects with ETF in the social partners field of VET policy.

15 ETF sent a questionnaire out in 2000, the results of which was summarised in the Document ETF Project to Support the

Development of Social Partners in VET Development – General Overview. The summary includes comments and statements from National Observatories and other counterp arts e.g. the Advisory Forum.

16 E.g. Hungary, Slovak Republic, Slovenia and Poland



176. The Slovak Republic’s report describes one caveat and potential area for future effort. It calls for a more systematic approach to the various events, conferences and seminars being organised,17 commissions and committees being created and measures proposed. They believe increased systemisation could bring increased synergies and harmonisation in the sphere of VET.

5.6 Networking

5.6.1 Links to the EU 177. The Social Partners project has links to the EU in terms of Candidate Country accession, ongoing

projects and mutual support.

178. A representative from ETF explained how the Social Partners Project fits within the wider EU priority of promoting structures and practice of social dialogue in the Candidate Countries which are key aspects of achieving the ‘acquis communautaire’. In some senses he suggests the project anticipated EU requirements (which have been explicitly articulated since 1998-99) towards these countries which advocate the incorporation of social partners in developing employment policies in line with the European Employment Strategy. Examples of partner countries understanding the need for social partners involvement in VET in the move towards accession can be found in National Reports and workshop accounts. These include the belief that social partners need knowledge of EU regulations (Hungary Report), the importance of using European Structural funds (seminar outcomes: Baltic Study Visit, Czech Republic workshop) and the recognition in the Estonian National Report that an internationally comparable employee qualification system is an important part of its preparations accession.

179. The Social Partners Project has links with DG Employment and Social Affairs and DG Education and Culture. In 1997 DG Employment had a unit dealing with social partner relations and a person in charge of social dialogue in Central and Eastern Europe. Representatives from DG V and DG XXII also were present at the 1997 Cross-Country Conference.

180. There are also a number of potentially synergistic links between the Social Partners project and other EU-funded projects and programmes. National Reports and workshop summaries have indicated connections with the Leonardo da Vinci Programme, and Phare VET Programmes. The issue of the social partners’ role in the Leonardo da Vinci programme was selected as one of the priority areas in which several partner countries decided to focus, which resulted in a presentation by an EU representative in the Czech Republic on the involvement of social partners in the Leonardo da Vinci programme.

181. Many countries from Central and Eastern Europe had previously signed agreements on participation in this programme. Conclusions and recommendations from the Slovak Republic include the possibility of establishing a tri-partite working group responsible for multi-source financing VET with possible international cooperation within the Leonardo da Vinci programme. With regard to Phare VET programmes, remarks from Slovenia highlight their links with a Phare programme for reconstruction of VET which included social partnership in VET at the regional level. In Estonia the work of the Foundation for Vocational Education Reform in Phare framework was seen to be significant, the Bulgarian workshop indicates the important role EU programmes have had in developing co-operation between stakeholders in VET in the past three years and results from the Czech seminar indicate a call for the enhancement of the role of social partners in the development of the Czech VET system (as part of a draft strategy for the further development of VET in their Phare VET Reform programme). The Slovenian National Report highlights the necessity for a certification system and the importance of the role of social partners in its development and introduction. Social partners participated in the preparation of the legal foundations for the certification system and in pilot projects which included the Phare MOCCA

17 This does not relate directly to the work of Social Partners and ETF.



Programme (which gives financial and professional assistance to the development of occupational qualifications system).

182. Support by the EU has been provided to the Social Partners Project and vice versa. One EU official provided contacts of European social partner organisations to ETF at project start-up. In addition a number of EU officials gave presentations at conferences and workshops. The Social Partners Project provided previously un-researched information on levels of social partnership in partner countries which was of use to the EU. It also promoted awareness of the importance of social partners in VET and raised the capacities of social partners themselves. All of which helped support EU initiatives in this area. An ETF representative explained how social partnership was one of the key EU priorities in the development of policies in VET and employment, and appears clearly in all-important EU documents governing employment, VET and Lifelong Learning issues.18 Social partners are also invited to take part and contribute actively in many processes at the EU level in the area of VET and employment, for example the consultation process on the Memorandum on Lifelong Learning in 2001.

183. Having said this one EU representative highlights the difficulties involved in enabling close coordination between the ETF in Turin and the diverse units of the EC in Brussels.

5.6.2 Links to ETF Projects and Programmes 184. It was suggested in the project proposal that the development and methodology of the Social

Partners Project should enable a synergy and cross fertilisation of activities with other ETF initiatives and networks. The following account provides an indication of such initiatives and networks to which there are acknowledged links.

185. It was suggested National Reports, in order to ensure synergy and avoid duplication, should draw as much as possible on information, material and experience made available by previous ETF activities (the work of the Advisory Forum sub group B, VET reform programmes funded by Phare and the Staff Development Programme). In fact the Staff Development Programme managed by ETF 1996-7, seems to have a number of similarities with the Social Partners project. It brought together national correspondents and social partners to take part in defining the content, extent, and aims of VET in relation to the needs of labour market.

5.6.3 Links to National Observatories 186. National Observatories in partner countries have played an important steering role in the Social

Partners Project. National Observatories have been involved in the writing, development and distribution of National Reports and the organisation of country workshops. The Lithuanian National Observatory was also the lead partner in the organisation of the Baltic Study Visit to Ireland. Additionally the Social Partners Project is seen to support the National Observatories. The 2000 questionnaire which was sent to National Observatories indicated the Social Partners Project had provided an opportunity to reinforce the profile and position of National Observatories in partner countries.

187. National Observatories at times carry out work which has links to those of the Social Partners project, for example in Estonia the National Observatory carried out a survey of employers. The reports and workshops indicate possible future links for social partners and the National Observatories. The Slovak National Observatory suggested it was ready (with ETF) to provide support to potential research groups in gathering information on foreign models of financing VET. It was suggested in the Romanian workshop that the National Observatories (and ETF) could play a valuable role supporting and monitoring the implementation of recommendations. The Hungarian workshop suggested the National Observatory introduce reconciling mechanisms

18 E.g. the European Employment Strategy, the Employment Guidelines, the EC Communication on Life long Learning, the

detailed Work Programme on the objectives for education and training systems in the EU, the conclusions of the policy conference on ‘increased cooperation in VET’ (Brussels June 2002) etc.



being used in EU countries to Hungary which would promote the willingness of the employers and employees to participate in the social dialogue. Having said this, it is unclear without further evaluation which of these activities took place.

188. Feedback from National Observatories indicates that the Social Partners Project assisted the work of the National Observatories in a number of ways. Involvement in the project had helped the National Observatories expand its list of experts and specialists (both national and foreign) thus providing an opportunity for the National Observatories to fulfil their primary task of developing networks of experts. This in turn contributed greatly to the implementation of the subsequent National Observatory tasks and projects, including the ones commissioned by the ETF. The Slovakian National Observatory described how networking among tripartite experts had been promoted. It stated the National Observatory had also made the Ministry of Education more sensitive to tripartite approach to tacking VET and labour market related issues and as a consequence the National Observatory had been appointed by the Ministry of Education to provide for important tasks with significant tripartite accent, for example, within the consultation process to the Memorandum on Lifelong Learning and the National Conference on VET in 2001 (which had social partner participation and social partnership was a topic of a specific workshop). Attendance at national and international conferences for social partners and experts involved in writing the National Reports seems to be on the increase. The Slovakian National Observatory was asked to participate or identify appropriate representatives for participation in conferences, for example, ‘Social Dialogue and Participation for the integration of CEECs in the European Union’ held in Budapest in June 2002. One of the Polish experts who prepared the National Report was invited to participate in the meeting organised jointly by ETF and CEDEFOP in October 1999 on the ‘Role of Social Partners in VET’. It enabled the expert to make international contacts, exchange experience and become acquainted with social dialogue in other countries.

5.6.4 Links to CEDEFOP 189. CEDEFOP has played an important role in the development of the Social Partners Project.

Specifically the country reports developed in 2000 for Slovenia, Slovakia, Poland and Bulgaria were developed in accordance with the structure of similar studies for four Member States, co-ordinated by CEDEFOP and ILO. It was thought this might result in a broadening of the context to which the partner countries could position their experiences. It was also believed it might promote a state of reflection as to the development of appropriate tools for benchmarking the social partners impact. It was also believed these reports might develop into a common publication, although this was not achieved. Also the proposal for the Baltic Study visit to Ireland suggests technical advice, assistance and monitoring would be provided by ETF in co-operation with CEDEFOP.

190. A representative from CEDEFOP indicated his involvement in the Social Partners project had consisted of attendance the 1997 Cross-Country Conference, at the Bulgarian workshop and the Malmo Conference. He regarded both as very successful as they brought useful knowledge to the ‘market’. As mentioned earlier CEDEFOP had used materials generated by the Socia l Partners Project in CEDEFOP presentations and workshops, as well as being a source for different CEDEFOP projects.

5.6.5 Links with other Organisations 191. The Social Partners Project has links and potential links to other organisations including ILO and

the World Bank. The 2000 National Reports as indicated earlier were put together in line with similar studies coordinated by ILO and CEDEFOP. Representatives from ILO also participated in the 1997 Cross-Country Conference. Recommendations from the study of the Slovak Republic called on using World Bank experience and potentially developing a HRD programme with the World Bank.



5.6.6 Cooperation between Social Partners 192. The Social Partners project has been successful in providing networking opportunities to social

partners. The project has facilitated communication and collaboration between social partners within and between partner countries and Member States.

• The 1997 Cross-Country Conference brought together representatives of employers’ organisations and trade unions from partner countries. Country workshops united social partners from within the country and on some occasions from other partner countries (especially the Baltic States). The Baltic Study Visit and subsequent workshop brought together social partners from the hospitality industry.

• Within the platforms provided by the project, social partners from partner countries were able to help develop ideas for VET policy and get involved in the EU accession process. Case study discussions and working groups in workshops gave participants the opportunity to discuss and exchange experiences. Feedback from the Baltic Study Visit describes how participants were able to have useful “businesslike conversations” with one another and hoped for the continuation of contacts.

• Involvement of social partner representatives from the Member States in the Social Partner project has generally taken the form of information dissemination and the sharing of experience. Representatives from UNICE, CEEP and ETUC were present as European Social Partners in the 1997 Conference. Presentations at the Czech Seminar suggested EU countries had provided the Czech Republic with significant assistance in terms of integration into European structures. Feedback from the Baltic Study Visit also indicates the importance of their dialogue with Member States social partners and in addition the necessity to maintain these links.

5.6.7 Continued Links 193. One important indicator of the success of the Social Partners Project is the continued

communication and cooperation of social partners after the end of ETF-funded initiatives. There is some indication that this has taken place. The 2000 questionnaire of National Observatories and other stakeholder groups indicated that in some cases the activities of the Social Partner project had triggered several subsequent meetings of social partners on VET specific issues. Certainly a number of links have been planned during project events. At the Czech Seminar, the Economic Chamber of the Czech Republic and the Ministry of Labour and Social Affairs indicated the possibility of future co-operation and representatives of the Ministries, employers, trade unions and other institutions declared they would promote the involvement of social partners in VET in the proposals for a new school act. Additionally, the Confederation of Industry of the CR, the Union of Entrepreneurs in Construction, the Union of Employers in Power Engineering, the Economic Chamber and the Czech-Moravian Confederation of Trade Unions expressed their readiness to participate in the development of professional standards. In addition feedback from the Baltic study visit indicates the mutual acquaintance of social partners from the three partner countries would be very important in further cooperation. However, without further evaluation it is difficult to know if this took place.

5.7 Management of the Social Partners Project 194. ETF maintains overall responsibility for the management of the Social Partners project. A project

manager based at ETF in Turin has been in place since the beginning of the Social Partners project in 1997 and has responsibility for a number of tasks. These include maintaining systematic cooperation and dialogue with the EC, CEDEFOP, ILO and EU social partners, identifying and monitoring the quality of the work of external experts (from the EU and CCs) who have been involved in drafting the reports and organising the agenda of the national seminars and monitoring the National Observatories work as regards preparation of the seminars



and monitoring the preparation and dissemination of conclusions. The Project Manager also ensures reports are written within structural guidelines determined by ETF. The Project Manager has attended all events organised as part of the Social Partners project and other ETF staff including Directors have participated where necessary. Support to the project has been provided by administrative staff in ETF.

195. A number of activities were sub-contracted during the course of the project. These included the preparation of the cross-country report in 1997-98 to an EU expert, the preparation of the National Reports by local experts through the National Observatories and the logistical organisation of country workshops to the National Observatories. The participation of external experts was also significant; experts included representatives from the EC (DG Employment and DG Education), EU social partner organisations and international organisations. At all times ETF maintained a managerial and monitoring role.

196. A specific budget was secured in 1997, 1998 and 2000 for this project as part of the ETF work programme. Within ETF the project was initially based in the Strategy Unit but after the re-organisation, moved to the CEN Department.

197. Regular reporting by the National Observatories on the developments in VET and the labour market in each country provide the basis for monitoring of the Social Partners project. ETF holds bi-annual meetings with the National Observatories and encourages a focus on social partners in their Annual Reports. Through these mechanisms ETF hopes to see how social partners issues are being tackled and identify potential progress. Evaluation questions are asked at workshops and in 2000 representatives from National Observatories took part in a questionnaire. No formal evaluation of the Social Partners Project has taken place.

198. National Observatory feedback suggested that ETF had provided the necessary managerial and organisational support for the project. This support was provided promptly and upon request. The Project Manager had helped in the preparation of the National Report and had helped suggest foreign experts for the country seminars. An EU official questioned about the organisation and management of the project felt the early contact ETF made with the EC made subsequent contact with the European social partners easier. He also felt the 1997 conference had been run very well.

199. The management of the Social Partners project has been responsive to the needs of project beneficiaries in a number of ways. The 1997 Cross-Country conference suggested the national workshops should be based around targeted themes, which subsequently took place. The Baltic States requested a combined workshop (instead of individual country workshops), which took place in Estonia in 1998. A joint proposal formulated as an outcome of the workshop in Estonia lead to the study visit to Ireland. Money saved on the organisation of the combined initial workshop was allocated to the study visit. In addition, a seminar held in Bratislava addressed to trade union officials was made at the request of the European Union Trade Union Confederation.

200. An ETF representative suggested the Social Partners project responded to the changing needs of partner countries by developing awareness on related EU policy and methodological approaches, by disseminating relevant EU practice in the field during the series of workshops and by the study visits organised in EU member states. He believes the changing needs of social partners should be seen in the light of the development of EU policy in VET and the enlargement process.

5.8 Added value 201. Added value can be viewed in terms of the added value the Social Partners Project brings and the

added value ETF brings to the management of the Social Partners Project.

202. The project is believed to be the first attempt at tackling the issue of social partners in vocational training in partner countries. It is the first time there has been serious analysis of the situation and the first time such a series of events has been organised in the participating countries in this area.



In addition the project has increased the visibility of the National Observatories and ETF in the partner countries, incorporated and institutions such as the EC, CEDEFOP and European social partners.

203. The Social Partners Project one ETF representative suggests, has to be considered within a framework of long-term involvement by ETF in the area of social partners. ETF’s involvement in social partners began around 1996 with the Advisory Forum working on the theme of social partners’ involvement in VET. A specific ETF project was funded as part of the Work Programme, and regular activities continue. It is this continuity of ETF involvement he believes is one of the main components making up ETF added value. He also suggests that ETF has access to in-house expertise in the field of VET and EU developments in the field. The collective ETF expertise in VET development in candidate countries is also seen as an added advantage. ETF was able where possible to cross social partner related issues with their other relevant knowledge on VET issues in the candidate countries. An EU expert suggested that Turin was a good place to bring together social partners from East and West Europe.

5.9 Social Partners links to a Centre of Expertise 204. An ETF representative suggested the Social Partners Project is a typical ‘Centre of Expertise’

project. This is because the project combines the gathering of relevant information from the countries through the network of National Observatories, analyses the situation and identifies gaps and challenges. The project also proposes further actions to be undertaken by the main actors or to be supported by EU assistance. The Social Partners Project uses specific ETF expertise in the field as well as general ETF expertise on VET and candidate countries combined with relevant EU specialised expertise and it prepares content related reports and delivers results in relevant workshops. Further, it disseminates EU policies and good practice and organises dialogue and exchanges between EU and Candidate Country experts and relevant key actors and beneficiaries.

205. Another ETF respondent agrees the Social Partners project is an example of a ‘Centre of Expertise’ project. He believes it raises the profile of ETF as an organisation which promotes the exchange of experience and information. The approach followed in the project (action oriented analysis) has been used also in subsequent ETF projects and the project has also triggered activities on the same topic in other geographical areas (for example, Tacis). In addition, due to its visibility gained through this project, ETF has been invited to develop and manage a social dialogue project in the Western Balkans supported by the European Economic and Social Committee.

Final Report (Vol. III) Case Study Annexes external... · Final Report (Vol.III) External...

Documents

Transcript of Final Report (Vol. III) Case Study Annexes external... · Final Report (Vol.III) External...