TR/05/B/F/ PP 178-009 “INTERNAL EVALUATION FINAL REPORT ” Prof. Lorenzo Fiorito.
-
date post
20-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of TR/05/B/F/ PP 178-009 “INTERNAL EVALUATION FINAL REPORT ” Prof. Lorenzo Fiorito.
TR/05/B/F/ PP 178-009
“INTERNAL EVALUATION FINAL REPORT ”
Prof. Lorenzo Fiorito
Specific aims of the project• To develop innovative approaches in Technology Education,• to adapt the content and supply of technology education to
suit new qualification needs through cooperation between educational and training bodies,
• to create a set of standards for European Technology Education Qualifications to be transparent and recognised at all levels and in all countries concerned,
• to develop new high quality curricula and training materials, • to realise international cooperation between educational and
training establishments and enterprises on national and European basis
• to enhance employment standards of Technology Education graduates and make them competitive in the job market by flexible training system and modules
• opening up Technology Education systems to a wider part of the world.
Expected results
1. Design of new curricula for Technology Education
2. New training material for the for Technology Education
3. Model training tools
4. Contact database of the national bodies, vocational schools, professional institutions and educational institutions regarding Technology Education.
5. Country reports about the current status and training/new qualification needs of Technology Education in partner countries
6. Printed copies of the project activities and results (in a form of handbook, in English
Workplan
The project is sub-divided into 8 work packages
1 Project Management and Coordination
2 Design of web based data acquisition medium-Set up database - Continuous update
3 Background analysis of technology education programs – Identification of current training needs and new qualification needs
4 Development of methodology
5Development of training modules-Determination of national exceptions - Integration of work related training and ICT
6 Development of training material and tools
7 Feedback and Evaluation
8 Valorisation
Time schedule of the project work plan (revised)
WPI
M1
II
III S1 C1
IV S2
V M2
S3
VI M3
S4
VII M4
VIII C1
11 12 1 2 3 4 5 6 7 8 910
11
12
1 2 3 4 5 6 7 8 9 1011
20052006
2007
Approach to evaluation
• The evaluation is intended to address the following questions:
• Did the outcomes meet the original objectives?• Analyse the differences between outcome and
objectives, identifying factors that have contributed to those differences.
• Evaluation of the strengths and improvements and make recommendations concerning good practice.
• To evaluate the productivity and tasks done in the project.
WP1 Project management and Coordination
• Effectiveness of communication system (extent to which partners were updated on project developments, efficiency of information flow, WP reports submitted in time, prompt response to problems/issues arising)
• Effectiveness of finance and administration systems (expenditure tracked efficiently, data collated and circulated regularly)
• Effectiveness of decision-making/planning process (evidence that workplan is flexible/adaptable to changing needs as the project develops, regular review of plans, equal participation of all partners in decision-making, documentation of decisions)
Performance indicators
WP1 Achieved results Project management and Coordination
• Establishment of project• Roles and work plan accepted• Communication channels established• Evaluation methods decided• Dissemination plan agreed• Basic concepts relevant to the development of
the common programme clarified
Organisational Structure
Czech Rep.
Germany
Italy
Bulgaria
Spain
Greece
Turkey
STEERING GROUP
PARTNERSHIPGROUP
WP1 Achieved results Project management and Coordination
Comments• Partnership agreement signed• Project leadership was agreed, with a responsible project
leader, a project management group• Sharing of duties through the splitting of the total project into
work packages (WP). Each WP was allocated a certain amount of work, shared between partners.
• Finance: Each partner was given responsibility for accounting, reporting and documenting their own expenses within the defined framework.
• Ownership and rights to the outcomes of the project, the IPR and other rights for joint products.
WP1 Achieved results Project management and Coordination
Kick off and Agreement: November 2005
WP1 Achieved results WP1: Communication and management
• On a weekly basis the Contractor informed the Partners about the latest developments and issues to be discussed within the project via e-mail. Activities and announcements have been located to the “partner announcement section of the project web-site and the partners are automatically informed via. e-mail about the location of the new announcement.
WP1 Achieved results WP1: Project management
Steering Committee Meetings
Meeting No
Type Venue Date Related Work Package
1 Steering Committee
Sofia, Bulgaria November 2005
WP1
2 Steering Committee
Naples, Italy October 2006
WP5
3 Steering Committee
Seville, Spain February 2007
WP6
4 Steering Committee
Prague Czech Republic
June 2007 WP7
WP1 Achieved results Project management
Conferences
Conference No
Type Venue Schedule Included Work
Package
1 Conference Greece June 2006 WP2
2 ValorisationConference
Ankara, Turkey
November 2007
WP8
WP2: Web site and contact database Performance Indicators
• Setting up project main web-site and sub-sites
• Effective data acquisition tools within the interactive web medium
• Instant and continuous monitoring of the results
• Effective internal and external monitoring of the results
WP2 achieved results:
Website; Completed: December 2005.
Country report database:added March 2006
• Comments: The webpage (www.modularte.gazi.edu.tr) created at the start of the project worked both as a communication tool with partners and stakeholders (for valorisation purposes), and a working platform of the project.
• In addition to the public area, a password-protected area on the web allowed the partners to access documents, reports, and the database.
• The web-site is built in a dynamic way and all databases are included within the site.
• Both public and private areas were constantly monitored
WP3: Web site and contact database Performance Indicators
• Positive feedback on usability • Varied sample of target group in partner
countries for contact database• Varied distributions of the questionnaires
in each partner country • Exhaustive reports about the status of
technology education in partner countries • Complete and on time final reports of the
partners on the overall WP tasks
WP3: Achieved resultsContact database
Completed: January/February 2006
Comments
• All partner countries created a contact database of the national bodies, vocational schools, professional institutions, educational institutions and enterprises related with technology education. Entries amount to a total of approx. 600.
WP3 Achieved results (2) Country Report Database Content:
Completed: June 2006 (Presented at the Conference in Larissa)
Comments The database is exhaustive as for:• Schools where technology education is taught• Units and applications regarding the technology
education or equivalent courses. • The National Educational system chart of the
partner countriesand relevant legislations • Current status of the Technology Education or
equivalent teacher training system in the partner countries
WP3 Achieved results Survey on Qualification needs of T. E.
Completed: June 2006 (presented at the conference in Larissa)
Comments:
• The 7 Questionnaires on the contact database of each
country aimed to analyse the situtation in :
Technology Education in Schools Technology Education
Teacher and Teacher Training System (teacher, teacher
training, programme) were well undesrtood and used
The rate of Questionnaire responses on overall contacts is
in the average of 50%
WP4 Development of methodology Performance Indicators
• Establishing a Common European work description of technology education
• Final report on methodology to be a basis for the development of training modules
• Clear tasks fro each partner to be accomplished according to key competences in the partner countries to be acceptable in European dimension.
WP4 Achieved resultsDevelopment of methodology
Concluded: October 2006 (Meeting in Naples)
• Establishment of agreed common requirements and description of five main modules
• Main Module 1 – Fundamentals • Main Module 2 – Systems and Processes• Main Module 3 – Information and
Communication• Main Module 4 – Sociotechnology• Main Module 5 – Didactics of technology Tasks assigned to each partner for developing
modules according to their competences
• Agreement on contents and structure of each module (both compulsory and elective)
• Translation of the all module contents into native languages including the national exceptions
WP5 Development of training modulesPerformance indicators
Main outcomes were:• Presentation of Main Modules• Overall Module System Partnership
Discussion, Changes and Revisions, Fixing of the Credits
• Structure of modules and Subject Oriented Areas for each lecture and on exemplary teaching materials
• Translations in native languages
WP5 Achieved resultsDevelopment of training modules
First part presented in Febr. 2007 (Meeting in Seville)Mostly Completed in June 2007 (Meeting in Prague)
some additional tasks completed in July
• Effective preview and usability of learning tools on website
• Effectiveness of the tools reviewed via identification of strengths and weaknesses via feedback from target group (WP7)
WP6 Development of training tools Performance indicators
• The learning tools were situated on the website: moreover, the interactive CD-ROMs containing the detailed training curricula and the training material of specified basic module were finalised.
• These were reviewed during evaluation workshops in each partner country (WP7): to date the response is positive
WP6 Achieved resultsDevelopment of training tools
Completed: July –September 2007
• Feedback from the target groups (evaluated on a national basis)
• Completed Internal evaluation report about the overall evaluation of the feedback
• External evaluation about the project activities and products Evaluation (from external experts of a European level)
WP7 Feedback and evaluationPerformance indicators
• External evaluation – Started in July. • Internal evaluation – Promoter deals with the
monitoring but the evaluation of the productivity and tasks done in the project was given as a task to the Italian partner.
• Feedbacks was obtained from target groups: teachers, academicians, students.
• The evaluation sheet was prepared by the Italian partner and sent to all partners, who organised workshops in October.
WP7 Achieved resultsFeedback and evaluation
Completed: November 2007
WP8 ValorisationPerformance Indicators
• Dissemination of materials and information to a range of relevant audiences and via a variety of appropriate methods/media at partner country and transnational level
• Organisation of events with participation of target groups and policy makers
• Effectiveness of the dissemination process in encouraging usage of learning materials and tools, also contributing to mainstreaming and sustainability (ex-post)
WP8 Achieved results
Valorisation activities- Started Nov. 2005 – ongoing
1. The main valorisation tool is the project web-site. All activities conducted within the project are either done by means of the project web-site or transferred to the web-site.
2. Press releases in local newspapers in partner countries: number of press releases within the partners countries: Turkey: 5 - Czech Republic: 3 - Spain: 1 - Greece: 2 Bulgaria: 2- Germany: 4 - Italy: 1
3. A summary of the project in each partner’s web-site
4. Local and national, workshops
5. Valorisation conferences, with the partecipation of policy makers, stakeholders and target groups
To end on:To end on:The opinion of the The opinion of the
partners partners MODULARTE MODULARTE
regarding the regarding the Overall ProcessOverall Process
A questionnaire was submitted to all partners in order to get their opinion concerning the different areas of the process
Partners’ Questionnaire
Not agree Partly agree
Agree Completely agree
The tasks assigned to your institution are not appropriate for the competence of your staff
Some of the tasks assigned to your institution are appropriate for the competence of your staff but something need to be changed
The tasks assigned to your institution are appropriate for the competence of your staff
The tasks assigned to your institution are appropriate for the competence of your staff you wouldn’t change anything
Appropriateness of the tasks assigned to each institution: did the participants collectively constitute a consortium of high quality? Were the participants well-suited and committed to the tasks assigned to them?
Poor Intermediate
Good Very good
the quality of the management is deficient
some aspects work but some others need to be changed
the quality of management is satisfactory
the quality of management is satisfactory and nothing need to be changed
Evaluation of the quality of the management: was the organisational structure well matched to the
complexity of the project and to the degree of integration required? Was the project management
demonstrably of high quality?
oThe rules for project activities implementation were clear, precise and acceptable for you.oThe tasks assigned for your institution corresponded to the project’s aims.oThe tasks assigned to your institution corresponded to the competence of your staff.oHow would you rate the quality and effectiveness of the project management?oThe results of the country report proved useful for the project’s aims
Which items we evaluated
oHow would you rate the effectiveness of the course evaluation in your country?oHow do you rate the effectiveness of the communication between partners during the life of the project?oHow would you rate the quality and effectiveness of the of the project web site. oHow would you rate the dissemination/valorisation events carried out so far by your organisation?oHow would you rate the meetings as for the results and the socialisation between the partners?
Which items we evaluated (2)
The highest scores were for: How would you rate the quality and effectiveness of the project management? (4)
The lowest scores were for :How do you rate the effectiveness of the communication between partners during the life of the project (3)•The tasks assigned for your institution corresponded to the project’s aims (3.2)
The overall opinion about the project process was positive.
•The partners judged the quality management very good•They were satisfied with the adequacy of tasks assigned to their institution and the tools for the ongoing of the project, such as the project website, rated effective, and the dissemination events, valued as fruitful. •The same opinion was valid for the meetings, helpful as for the results and the socialization and the rules for project activities implementation rated clear, precise and acceptable enough. •Some doubts were expressed about the effectiveness of the communication between partners during the life of the project and the correspondence of the tasks assigned to their institution for the project’s aims.
CLOSING REMARKS
1)Result orientation:The aims of the project were clear
2) Focus on beneficiaries:Always present. Too early to forecast impact on target groups
3) Partnership: Acknowledged by the project partners
4) People involvement: Generally high throughout the project
5) Leadership: Added value
CLOSING REMARKS
6) Strategy: Sound and well fitted
7)Processes: WPs aligned on the basis of a dynamic work model
8) People results: Good network to be sustained
9 )Impact: The project is liable to induce broader impact exceeding original aims of time, targets and partners through further investments to build on achievements and results
Thank you