Evaluation of Key Project Deliverables in Part Fulfilment ... 1.227 Making Qualit… · (NCFHE) to...
Transcript of Evaluation of Key Project Deliverables in Part Fulfilment ... 1.227 Making Qualit… · (NCFHE) to...
Operational Programme II – Cohesion Policy 2007-2013
Empowering People for More Jobs and a Better Quality of Life
Training part-financed by the European Union
European Social Fund (ESF)
Co-financing rate: 85% EU Funds; 15% National Funds
Investing in your future
Evaluation of Key Project Deliverables in Part Fulfilment of ESF Project 1.227 ‘Making Quality Visible’
Quantitative Report 2015
2
Table of Contents
Background 3
Findings and Analysis 5
Profile of Respondents 5
Role of the Respondents 5
CPD Award 6
Seminar for Prospective Expert Evaluators of the External Quality Audits 8
Seminar for Prospective Student Evaluators of the External Quality Audits 10
Net-QAPE Meetings 11
Quality Audit Process 12
Concluding Conference 13
Overall Project 14
Appendix I: Tables 16
Profile of Respondents 16
Role of the Respondents 19
CPD Award 20
Seminar for Prospective Expert Evaluators of the External Quality Audits 45
Seminar for Prospective Student Evaluators of External Quality Audits 65
Net-QAPE Meetings 85
Quality Audit Process 98
3
Background
Grant Thornton has been engaged by the National Commission for Further and Higher Education
(NCFHE) to carry out an evaluation of the deliverables forming part of the ESF Project 1.227 “Making
Quality Visible”. For the evaluation exercise, Grant Thornton utilised both quantitative and qualitative
research tools. The aim of this report is to present the results of the data gathered through the
quantitative research exercise.
The data for the quantitative exercise was collected through Computer Assisted Web Interviewing
(CAWI), whereby interviewees were invited to complete the survey online within a one-month period.
It is to be noted that there were three categories of participants who participated in the ESF Project
1.227 “Making Quality Visible”, that is, administrators, tutors and students. Thus, a different
questionnaire was specifically designed for each category of participants. However, some participants
had multiple roles in the project, for example, some participants were administrators and tutors, and
others were administrators, tutors, and students. Thus, in order to reduce inputting mistakes and to
minimise missing information, it was decided that an additional two distinct questionnaire were
developed specifically targeting those who had multiple roles in the Project. The finalised five versions
of the questionnaires were forwarded to the NCFHE for approval.
Subsequent to the attainment of this endorsement, the finalised questionnaires were uploaded on Grant
Thornton’s data entry system and the link was sent to the individuals listed in the database provided by
NCFHE via email. Every week, Grant Thornton sent an email reminding the participants to to fill in
the survey. Furthermore, the links of the surveys were sent to the three educational institutions (the
University of Malta, the Institute of Tourism Studies and the Malta College for Arts, Science and
Technology) involved in the project, as originally agreed upon with the Project Manager of the
evaluation, to be forwarded to their employees who participated in the Project. Initially, due to a number
of misunderstandings, some issues were encountered with these institutions, and the links were not
forwarded to the end participants. The quantitative research was launched during the first week of
September 2015 and was originally targeted to be completed by the last week of September 2015.
However, upon the end of September, Grant Thornton was informed that MCAST intended to finally
forward the link of the questionnaires to the participants within MCAST. Thus, it was decided that the
collection period be extended by another two weeks in order to improve the response rate. At the end
of the collection period, in total, 122 participants completed the on-line questionnaires. It is to be noted
that Grant Thornton could not obtain the number of participants to whom MCAST had sent the link to
the surveys, thus, it is unable to estimate the response rate.
4
The following sections present an analysis of the data collection from the quantitative exercise.
5
Findings and Analysis
Profile of Respondents
The majority of the respondents were mainly male (54.1%) and mostly were aged 46+ (46.7%). Tutors
and administrators were also asked about their highest level of education attained, and the vast majority
completed tertiary education (75.4%). The administrators were also asked about their current position
within the educational institutions - with most of them currently holding a combined position as an
administrator / manager / director (37.7%).
Role of the Respondents
The respondents were asked about their role in the ESF Project 1.227 “Making Quality Visible”. Indeed,
participation in the CPD Award, Expert Evaluators Seminar, Student Evaluators Seminar and Net-
QAPE Meetings was dependent on the position of the respondents within the educational institution.
On the other hand, the EQA and the Concluding Conference were open to all the participants. Focusing
on the specific activities, the majority of the respondents participated in the Net-QAPE Meetings
(36.1%) while for the general activities, the majority of the respondents participated in the Final
Conference (40.2%).
6
CPD Award
As shown in the previous graph 19.7% of the respondents participated in the CPD Award. These
participants were asked whether they received a certificate, and 18.0% of the respondents stated that
they did get a certificate. The main reasons for not completing the course principally relate to the change
in circumstances of the institution (0.8%), the lack of time (0.8%), and due to travelling (0.8%).
Overall, respondents believed that the course was well organised (10.6%). Indeed, they believed that
detailed information was provided on the course (14.8%), the learning outcomes of the course were
clearly stated (16.4%), and the information on the course was provided well in advance (12.3%).
Respondents were also asked about the course content. In this regard, the respondents described the
course as being practical (12.3%) and that it reflected projected External Quality Audit requirements
(13.1%). Furthermore, the respondents stated that lecturers allowed for active participation and
interaction (15.6%).
Respondents were asked additional questions on the lecturers. The majority of the respondents were
satisfied with the lecturers’ performance. Indeed, the respondents believed that they were well prepared
for the delivery of the courses (13.1%) and that they had good knowledge of the subject (16.4%).
Furthermore, they also believed that the lecturers managed to use the technology available effectively
(12.3%) and that they provided useful material (12.3%). The respondents were also asked to identify the
main strengths of the lecturers. In this regard, the respondents mainly mentioned their knowledge of
the subject, as well as their experience in Quality Assurance which lecturers used as examples during the
lectures.
A number of questions related to the learning resources made available during the delivery of the course.
Once again, the respondents were satisfied with the learning resources. The respondents believed that
the resources were enough to meet their needs (12.3%). For the respondents, the lecture room was of
an acceptable size (16.4%) and the equipment used was useful for their learning (14.8%). The educational
material made available also helped respondents to understand the topics (13.1%). The respondents
found the educational material to be up to date (15.6%), easy to understand (14.0%), well structured
(14.0%), and relevant (16.4%).
Overall, the respondents found the teaching methods to be good. The teaching methods mainly relate
to the lectures (13.9%), presentations (15.6%), group activities / discussions (13.3%), and problem
handling (10.7%).
7
In the survey, a number of questions were also asked to gauge the respondents’ commitment to the
course. From the feedback provided it seems that the respondents were committed to the course.
Indeed, it appears that they attended almost all the lectures (18.0%) and that they were involved in the
group discussions (18.1%). Over and above the lectures, the respondents also carried out the
recommended hours of self-study (17.2%) and made use of the hours made available to them by the
lectures outside the lecture room (12.3%).
When questioned on the assessment methods, the majority of the respondents were satisfied with the
assessment. Indeed, the majority regarded the assessment as being appropriate for the course (12.3%).
Furthermore, the respondents believed that the assessment gave them the opportunity to show their
knowledge (14.0%) and that they were given enough time for the assessment (15.6%). Participants also
agreed that they were informed about the evaluation criteria of the assessment (16.4%) and that they
were given clear feedback on their performance during the assessment (11.5%).
Overall, respondents were either very satisfied (4.1%) or satisfied (10.7%) with the CPD Course.
Respondents were also prompted to provide recommendations on how the course could be improved.
In this regard, a number of relevant recommendations were put forward and the following list
summarises the main recommendations mentioned in the on-line questionnaire:
1. Replacing Professor Suzanne Gatt (main tutor) on the programme
2. The workload of the course should be reflective of the credit value of the programme. Apart
from the direct lecturing and seminar hours, the amount of time as preparation for the
assignment and expected deliverables should have also be taken into consideration.
3. Workshops need to be effectively managed. Participants left the workshops when these were
still underway and it did not give the opportunity to participants to interact with individuals
from other institutions.
4. The NCFHE should not implement and run an accredited course due to conflict of interest.
The goal posts were continuously changing throughout the course and the certificate was also
issued by NCFHE which reserved the right to withdraw it unless participants continue to submit
the material it requests.
5. Shorter lectures and more frequent
6. More to the point and concise materials
7. The assessment was unfair on participants who are not academics
8. Provision of more individual feedback to participants
9. More involvement of foreign speakers
10. Dissemination of reading material prior to the lectures
11. Dissemination of a training manual to participants
8
12. Venue of the training should be in a central location
13. Lectures need to be more interactive
14. A timetable for the sessions should be in place with no short notice changes
15. Organisation of visits to institutions to show the practices of each other rather than just talking
about them.
Seminar for Prospective Expert Evaluators of the External Quality Audits
Around 19.7% of the respondents also participated in the Seminar for Prospective Expert Evaluators of
the External Quality Audits. A number of questions were asked on the overall organisation of this
seminar. In this regard, the respondents believed that the seminar was well organised (18.8%).
Respondents agreed that detailed information was provided on the workload and input required (18.0%),
and that the information was provided well in advance (16.4%). Furthermore, the learning outcomes of
the seminar were also clearly stated (18.0%).
The majority of the respondents were satisfied with the content of the seminar. In this regard, the
majority of the respondents agreed that the seminar was practical (18.9%) and that it reflected projected
External Quality Audit requirements (18.8%). Respondents also thought that the lecturers allowed for
active participation and interaction during the seminar (18.0%).
The respondents were asked a number of questions on the lecturers who delivered the seminar. Overall,
respondents agreed that the lecturers were well prepared (18.0%) and had good knowledge of the subject
(18.1%). Furthermore, the lecturers effectively used the technology available for the delivery of the
seminar (16.4%). The lecturers also provided useful material (17.2%) and managed to make the subject
interesting (18.1%). When asked to identify the main strengths of the lecturers, the respondents pointed
out the experience of the lecturers.
The respondents agreed that the learning resources made available for the seminar were enough to meet
their needs (18.1%). They found the equipment in the lecture room to be useful for their learning
experience (16.4%). The lecture room was deemed to be of an acceptable size for the number of
participants attending the seminar (18.0%). When asked about the educational material made available
to them, they agreed that it helped them to better understand the topics being tackled (18.1%). They
also found the educational material up to date (18.1%), easy to understand (17.2%), well-structured
(18.9%). and relevant (18.8%).
9
When asked about the teaching methods, most of the respondents found the lectures (10.7%),
presentations (11.5%) and problem handling (9.8%) as being good. However, they mostly appreciated
the group activities and discussions as they described them as being very good (13.1%).
Respondents were asked a number of questions on additional considerations related to the seminar. The
respondents were very involved in the group discussions that developed during the seminar (19.7%).
This was further encouraged due to the conducive environment which made the respondents feel
respected (18.0%) and at ease in the sharing of opinions and in asking questions (17.2%). The seminar
met the expectations of the participants (18.1%) and they strongly agreed that they would recommend
the seminar (18.1%). Furthermore, the respondents also stated that the seminar helped them gain in-
depth knowledge on the subject (18.1%).
Overall, the respondents were either very satisfied (9.0%) or satisfied (9.0%) with the seminar.
Nonetheless, the respondents provided a number of recommendations on how the seminar could be
improved. In this regard they mentioned:
1. Improving the first session of the European evolution of quality in education - should have
focused more on the local scenario.
2. More practical work and simulations
3. Improving the seating arrangements
4. Venue to be centrally located
5. Longer session or organisation of a follow up session
6. Simulation sessions to be organised in rooms rather than a hall
7. Offering more varied food
8. Information needs to be provided before
9. More subjects need to be discussed
10. Smaller groups
10
Seminar for Prospective Student Evaluators of the External Quality Audits
Focusing on the seminar for prospective student evaluators of the External Quality Audits, around
13.1% of the respondents attended this seminar. Overall, the respondents were satisfied with the
organisation of the seminar. Indeed, they stated that the seminar was well organised (10.7%). The
respondents also agreed that detailed information was provided on the seminar workload and input
required (9.8%), and that the learning outcomes of the seminar were clearly stated from the beginning
(8.2%). Whereas most of the respondents agreed that information on the seminar was provided well in
advance (5.7%), there was also a high percentage of respondents (4.9%) who did not have any opinion
in this regard.
When questioned on the seminar content, the majority of the respondents strongly agreed (7.4%) with
the statement that the lecturers allowed for active participation and interaction during the delivery of the
seminar. Also, the respondents agreed that the seminar was practical (9.1%) and that it reflected the
projected External Quality Audit requirements (9.1%).
The respondents were also asked questions on the lecturers who delivered the seminar. In this regard,
the respondents strongly agreed that the lecturers had good knowledge of the subject (8.2%) and that
they made the subject interesting (5.7%). There was also general agreement that the lecturers were well
prepared and fully resourced (11.5%), they used technology effectively (8.2%), and provided useful
material (9.9%). When respondents were asked to identify the main strength of the lecturers, they
mentioned their experience and knowledge of the subject.
Respondents were also asked to express their opinion on the learning resources used during the seminar.
Although there was a high percentage of respondents who did not have opinion on whether the
resources made available were enough to meet their needs (4.9%), the majority agreed that these were
enough (6.5%). Respondents believed that the size of the room was acceptable for the number of
persons attending the seminar (12.3%) and that the equipment used was useful during the delivery
(10.7%). When asked about the educational material, most of the respondents (6.6%) agreed that these
were useful in helping them understand the topics. However, there was another 4.9% who felt neutral
on this point. Despite this, there was a general consensus that the educational material was up to date
(9.8%), easy to understand (9.1%), well structured (8.2%), and relevant (12.3%).
The respondents answering this section stated that they were satisfied with the teaching methods.
Indeed, they stated that the lectures (10.6%), presentations (11.5%), group activities/discussions
(10.7%), and problem handling (12.3%) were either very good or good.
11
Respondents were also asked their opinion about a number of other aspects related to the delivery of
the seminar. Respondents were involved in the discussions that developed during the seminar (13.1%)
and they stated that the seminar met their expectations (9.9%). The respondents felt respected (12.7%)
and they strongly agreed that they felt at ease in sharing their opinions and in asking questions (6.6%).
Respondents also agreed that the seminar helped them gain in-depth knowledge of the subject (9.9%).
Thus, respondents felt that they would recommend the seminar if it was to be provided again in the
future (10.7%).
Overall, the respondents were satisfied with the seminar (10.7%). Nevertheless, they put forward a
number of recommendations for future reference:
1. Avoid the use of acronyms
2. Provision of more information on the procedure of the External Audit
3. Have a better mix of students from the institutions to improve interaction
4. Provision of more days for the delivery of the seminar
5. Provision of practical workshops and case studies on quality audits and examples of bad audits
6. Provision of information on the seminar
7. Organisation of a follow up session
8. Provision of information on the work that was already done by the EQA.
Net-QAPE Meetings
This section applied to 27.0% of the respondents who stated that they had a role in the Net-QAPE
Meetings. In this regard, 24.6% stated that they regularly attended the meetings, with the majority
(14.8%) claiming that they attended most of the meetings.
12
Overall, there was positive feedback on the Net-QAPE Meetings. Indeed, the respondents found these
meetings to be useful (23.8%) and these met their expectations (18.8%). The respondents found these
meetings to be quite relevant, both for their personal development (21.3%) and for their institution
(23.0%). As a result, 23.8% of the respondents stated that they will put into practise the information
that was provided during the Net-QAPE meetings. Furthermore, the respondents were also in general
satisfied with the speakers who stated that they have effectively delivered the information (20.5%).
Respondents were asked a number of questions relating to different aspects of the Net-QAPE Meetings.
The majority of the respondents described the content and the organisation of the meetings either as
very good (7.4%) or good (14.8%). Respondents also found the interaction among the participants
during the meetings and the use of visual aids as being good at 20.5%. With regards to the logistics of
the meetings, the respondents found the location and date and time to be good with 17.1% and 18.9%
respectively. Respondents were also satisfied with the meeting facility (20.5%) and registration process
(24.6%). While in general, the respondents found the parking to be either very good (3.3%) or good
(13.1%), there were some who had issues and they assessed this aspect as being either fair (6.6%), poor
(2.5%), or very poor (0.8%).
Quality Audit Process
Around 36.1% of the respondents participated in the Quality Audit Process. The majority of the
respondents felt that they were sufficiently involved in the External Quality Audit (29.5%). Respondents
13
also stated that they felt respected by the evaluation penal throughout the External Quality Audit
(32.0%). When asked about whether they had enough time to prepare for the audit, there was an equal
percentage between those who agreed and those who felt neutral on this point at 13.1%. Overall, there
was general unanimity that the External Quality Audit will help their respective institution to strengthen
the External Quality Audit system (23.0%). Furthermore, the respondents stated that the instructions
and documents provided were sufficiently enlightening (14.8%) and that they were involved in the
internal quality assurance of the institution (29.6%).
Students were also asked about whether taking part in the External Quality Audit made them feel
empowered as students, with the majority of them stating that they did (4.1%). Similarly, the majority
of the students (4.1%) claimed that the External Quality Audit was useful.
When asked about whether the respondents found the Quality Audit Process as being a stressful
experience, there was an equal percentage of those who found it to be stressful and those who have a
neutral opinion on this issue (12.3%).
Concluding Conference
Around 40.2% of the respondents participated in the concluding conference. The respondents were
asked to provide feedback on the contents of the conference. The respondents claimed that the
conference was useful (32.8%) and that it met their expectations (31.9%). The information provided
during the conference was relevant to the personal development of 28.7% of the respondents and they
also claimed that it was relevant for their respective institution. Around 30.3% of the respondents stated
that they will put into practice the information provided during the conference and 32.0% of the
respondents stated that the speakers at the conference clearly delivered the information.
Respondents were also questioned about different aspects of the conference. The majority of the
respondents claimed that the content of the conference was either very good (9.8%) or good (23.0%).
When asked about the organisation of the conference, 33.6% stated that the organisation was good. The
interaction among participants at the conference was also good for the majority of the participants
(24.6%). The use of the visual aids was also deemed as good to the respondents (31.3%). Respondents
were asked questions on the logistics of the conference. The location of the conference was good to
31.1% of the respondents while the date and time of the conference was good to 30.0% of the
respondents. Around 32.8% and 37.7% of the respondents respectively rated the conference facility and
14
registration as either very good or good. The parking for the conference was rated as good by 30.4% of
the respondents.
When asked about whether they would be interested in attending similar future events, 23.0% of the
respondents stated that they were very interested and another 13.1% of the respondents stated that they
were interested.
Overall Project
The respondents were asked to rate the usefulness of the ESF Project 1.227 “Making Quality Visible”
and its deliverables for the improvement of Quality Assurance in Further and Higher Education in Malta.
In this regard, 34.4% of the respondents stated that they rate its usefulness as being very good and
another 32.0% rated it as being good.
At the end of the quantitative questionnaire, the respondents were asked to provide any additional
comments. The following is a list of the main comments received:
1. All the higher education institutions should have been evaluated by the same team.
2. Should have organised a seminar to teach students what to do when they were called for an
audit.
3. Poor sound system at the Dolmen Hotel.
4. Unaware of the next step following the seminar.
5. Agreement with the quality audit in further and higher education as adults should have value for
the courses they are following and for what they are paying.
6. EQA is a very useful tool and should be carried out more frequently. Quality Assurance
improves the services and facilities on offer and at the end improves the quality of higher
education while making students attractive to prospective employers. There should be a sequel
of these sessions at least once a year, to keep updated, sustain the created forum and share
challenges/achievements.
15
7. Difficulty in how the details apply to much smaller institutions since both the seminar and the
conference were mostly related to the experiences of the three major institutions. Should be
informed about upcoming external quality audits in which it is possible to participate in.
8. Personnel should be chosen carefully.
16
Appendix I: Tables
Profile of Respondents
Gender:
Frequency Percent
Male 66 54.1
Female 56 45.9
Total 122 100.0
Education:
Frequency Percent
Not applicable
24 19.7
Secondary 6 4.9
Tertiary 92 75.4
Total 122 100.0
17
Age:
Percent Frequency
18-20 .8 1
21-30 22.1 27
31-45 30.3 37
46+ 46.7 57
Total 100.0 122
18
Status:
Percent Frequency
Not applicable 46.7 57
Administrator/director/manager 37.7 46
Quality assurance officer 12.3 15
Lecturer .8 1
Clerk .8 1
Counsellor .8 1
Other .8 1
Total 100.0 122
19
Role of the Respondents
Role within the ESF Project 1.227
Percentage Frequency
CPD Award 19.7 24
Expert Evaluators Seminar
19.7 24
Student Evaluators Seminar
13.1 16
Net-QAPE Meetings 27.0 33
EQA 36.1 44
Conference 40.2 49
None 21.3
26
20
CPD Award
CPD Certificate
Percent Frequency
N/A 80.3 98
Yes 18.0 22
No 1.6 2
Total 100.0 122
21
Reasons for not completing the course:
Percent Frequency
Change in employment 0 0
Change in circumstances in my institution 0.8 1
Not enough time 0.8 1
Course was not useful 0 0
Other 0.8 1
Organisation of the course:
Detailed information was provided on the course
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 10.7 13
Neutral 2.5 3
Disagree 1.6 2
Strongly Disagree
.8 1
Total 100.0 122
22
The learning outcomes of the course were clearly stated
Percent Frequency
N/A 80.3 98
Strongly Agree
4.9 6
Agree 11.5 14
Neutral 1.6 2
Disagree .8 1
Strongly Disagree
.8 1
Total 100.0 122
23
Information on the course was provided well in advance
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 8.2 10
Neutral 4.1 5
Disagree 2.5 3
Strongly Disagree
.8 1
Total 100.0 122
The course was well organised
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 9.0 11
Neutral 3.3 4
Disagree .8 1
Total 100.0 122
24
Course Content
The lecturers allowed for active participation and interaction
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 9.0 11
Neutral 3.3 4
Disagree .8 1
Total 100.0 122
25
The course content reflected projected External Quality Audit requirements
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 9.0 11
Neutral 4.1 5
Disagree .8 1
Strongly Disagree
1.6 2
Total 100.0 122
The course was practical
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 9.8 12
Neutral 4.1 5
Disagree 2.5 3
Strongly Disagree
.8 1
Total 100.0 122
26
Lecturers
The lecturers were well prepared and fully resourced
Percent Frequency
N/A 80.3 98
Strongly Agree
5.7 7
Agree 7.4 9
Neutral 4.9 6
Strongly Disagree
1.6 2
Total 100.0 122
27
The lecturers had a good knowledge on the subject
Percent Frequency
N/A 80.3 98
Strongly Agree
4.9 6
Agree 11.5 14
Neutral 1.6 2
Strongly Disagree
1.6 2
Total 100.0 122
28
The lecturers used technology effectively to advance my learning
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 9.8 12
Neutral 5.7 7
Strongly Disagree
1.6 2
Total 100.0 122
The lecturers made the subject interesting
Percent Frequency
N/A 80.3 98
Strongly Agree
3.3 4
Agree 10.7 13
Neutral 4.1 5
Strongly Disagree
1.6 2
Total 100.0 122
29
The lecturers provided useful material
Percent Frequency
N/A 80.3 98
Strongly Agree
4.9 6
Agree 7.4 9
Neutral 4.9 6
Disagree .8 1
Strongly Disagree
1.6 2
Total 100.0 122
30
Learning Resources
The resources were enough to meet my needs
Percent Frequency
No answer
.8 1
N/A 80.3 98
Strongly Agree
3.3 4
Agree 9.0 11
Neutral 4.1 5
Disagree 2.5 3
Total 100.0 122
The number of participants in relation to the room size was acceptable
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 12.3 15
Neutral 2.5 3
Disagree .8 1
Total 100.0 122
31
The equipment of the learning area was useful
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 10.7 13
Neutral 4.1 5
Disagree .8 1
Total 100.0 122
32
The educational material helped me understand the topics
Percent Frequency
N/A 80.3 98
Strongly Agree
3.3 4
Agree 9.8 12
Neutral 3.3 4
Disagree 3.3 4
Total 100.0 122
The educational material was up to date
Percent Frequency
Valid N/A 80.3 98
Strongly Agree
4.1 5
Agree 11.5 14
Neutral 4.1 5
Total 100.0 122
33
The educational material was easy to understand
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 11.5 14
Neutral 4.9 6
Disagree .8 1
Total 100.0 122
34
The educational material was well structured
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 11.5 14
Neutral 4.1 5
Disagree .8 1
Strongly Disagree
.8 1
Total 100.0 122
The educational material was relevant
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 12.3 15
Neutral 3.3 4
Total 100.0 122
35
Teaching Methods:
Lectures
Percent Frequency
N/A 80.3 98
Very Good
5.7 7
Good 8.2 10
Fair 3.3 4
Poor 1.6 2
Very Poor
.8 1
Total 100.0 122
36
Presentations
Percent Frequency
N/A 80.3 98
Very Good
4.1 5
Good 11.5 14
Fair 2.5 3
Poor 1.6 2
Total 100.0 122
37
Group activities / discussions
Percent Frequency
N/A 80.3 98
Very Good
4.9 6
Good 7.4 9
Fair 4.9 6
Poor 2.5 3
Total 100.0 122
Problem handling
Percent Frequency
No answer
.8 1
N/A 80.3 98
Very Good
2.5 3
Good 8.2 10
Fair 3.3 4
Poor 4.1 5
Very Poor
.8 1
Total 100.0 122
38
Commitment
I attended almost all of the lectures
Percent Frequency
N/A 80.3 98
Strongly Agree
13.9 17
Agree 4.1 5
Neutral .8 1
Disagree .8 1
Total 100.0 122
39
I was involved in the group discussion during the lectures
Percent Frequency
N/A 80.3 98
Strongly Agree
7.4 9
Agree 10.7 13
Neutral .8 1
Disagree .8 1
Total 100.0 122
I carried out the recommended hours of self-study
Percent Frequency
N/A 80.3 98
Strongly Agree
9.8 12
Agree 7.4 9
Neutral 1.6 2
Strongly Disagree
.8 1
Total 100.0 122
40
I made use of the hours made available by the lecturers outside the classroom
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 5.7 7
Neutral 4.1 5
Disagree .8 1
Strongly Disagree
1.6 2
Don't Know
.8 1
Total 100.0 122
41
Assessment Methods
The assessment methods were appropriate for the course
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 8.2 10
Neutral 4.1 5
Disagree 1.6 2
Strongly Disagree
1.6 2
Total 100.0 122
42
The assessment methods gave the opportunity to participants to show their knowledge
Percent Frequency
N/A 80.3 98
Strongly Agree
3.3 4
Agree 10.7 13
Neutral 4.9 6
Strongly Disagree
.8 1
Total 100.0 122
43
Participants were informed on the evaluation criteria
Percent Frequency
N/A 80.3 98
Strongly Agree
4.9 6
Agree 11.5 14
Neutral 1.6 2
Disagree .8 1
Strongly Disagree
.8 1
Total 100.0 122
Participants were given enough time for the assessment
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 11.5 14
Neutral 1.6 2
Disagree .8 1
Strongly Disagree
1.6 2
Total 100.0 122
44
The lecturers provided clear feedback on the participants' performance
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 7.4 9
Neutral 4.9 6
Disagree 3.3 4
Total 100.0 122
45
Seminar for Prospective Expert Evaluators of the External Quality Audits
Detailed information was provided on the seminar workload and input required
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 13.9 17
Neutral .8 1
Disagree .8 1
Total 100.0 122
The learning outcomes of the seminar were clearly stated
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 13.9 17
Neutral 1.6 2
Total 100.0 122
46
Information on the seminar was provided well in advance
Percent Frequency
N/A 80.3 98
Strongly Agree
4.9 6
Agree 11.5 14
Neutral 2.5 3
Strongly Disagree
.8 1
Total 100.0 122
47
The seminar was well organised
Percent Frequency
N/A 80.3 98
Strongly Agree
9.0 11
Agree 9.8 12
Neutral .8 1
Total 100.0 122
Seminar Content
The lecturers allowed for active participation and interaction
Percent Frequency
N/A 80.3 98
Strongly Agree
5.7 7
Agree 12.3 15
Neutral .8 1
Strongly Disagree
.8 1
Total 100.0 122
48
The seminar content reflected projected External Quality Audit requirements
Percent Frequency
N/A 80.3 98
Strongly Agree
5.7 7
Agree 13.1 16
Neutral .8 1
Total 100.0 122
49
The seminar was practical
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 14.8 18
Neutral .8 1
Total 100.0 122
Lecturers
The lecturers were well prepared and fully resourced
Percent Frequency
N/A 80.3 98
Strongly Agree
5.7 7
Agree 12.3 15
Neutral 1.6 2
Total 100.0 122
50
The lecturers had a good knowledge on the subject
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 11.5 14
Neutral 1.6 2
Total 100.0 122
51
The lecturers used technology effectively to advance my learning
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 12.3 15
Neutral 3.3 4
Total 100.0 122
The lecturers made the subject interesting
Percent Frequency
N/A 80.3 98
Strongly Agree
7.4 9
Agree 10.7 13
Neutral .8 1
Disagree .8 1
Total 100.0 122
52
The lecturers provided useful material
Percent Frequency
N/A 80.3 98
Strongly Agree
5.7 7
Agree 11.5 14
Neutral 2.5 3
Total 100.0 122
53
Learning resources
The resources were enough to meet my needs
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 15.6 19
Neutral .8 1
Disagree .8 1
Total 100.0 122
The number of participants in relation to the room size was acceptable
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 11.5 14
Neutral 4.1 5
Disagree 1.6 2
Total 100.0 122
54
The equipment of the learning area was useful
Percent Frequency
N/A 80.3 98
Strongly Agree
2.5 3
Agree 13.9 17
Neutral 3.3 4
Total 100.0 122
55
The educational material helped me understand the topics
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 11.5 14
Neutral 1.6 2
Total 100.0 122
The educational material was up to date
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 11.5 14
Neutral 1.6 2
Total 100.0 122
56
The educational material was easy to understand
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 13.1 16
Neutral 2.5 3
Total 100.0 122
57
The educational material was well structured
Percent Frequency
N/A 80.3 98
Strongly Agree
4.1 5
Agree 14.8 18
Neutral .8 1
Total 100.0 122
The educational material was relevant
Percent Frequency
N/A 80.3 98
Strongly Agree
4.9 6
Agree 13.9 17
Neutral .8 1
Total 100.0 122
58
Teaching Methods
Lectures
Percent Frequency
N/A 80.3 98
Very good
6.6 8
Good 10.7 13
Fair 1.6 2
Poor .8 1
Total 100.0 122
59
Presentations
Percent Frequency
N/A 80.3 98
Very good
4.9 6
Good 11.5 14
Fair 3.3 4
Total 100.0 122
Group activities / discussions
Percent Frequency
N/A 80.3 98
Very good
13.1 16
Good 5.7 7
Fair .8 1
Total 100.0 122
60
Problem handling
Percent Frequency
N/A 80.3 98
Very good
8.2 10
Good 9.8 12
Fair 1.6 2
Total 100.0 122
61
Additional considerations
I was involved in the group discussion during the seminar
Percent Frequency
N/A 80.3 98
Strongly Agree
10.7 13
Agree 9.0 11
Total 100.0 122
The seminar met my expectations
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 11.5 14
Neutral .8 1
Disagree .8 1
Total 100.0 122
62
I would definitely recommend this seminar if provided in the future
Percent Frequency
N/A 80.3 98
Strongly Agree
10.7 13
Agree 7.4 9
Neutral .8 1
Disagree .8 1
Total 100.0 122
63
I felt respected during the seminar
Percent Frequency
N/A 80.3 98
Strongly Agree
9.0 11
Agree 9.0 11
Disagree .8 1
Strongly Disagree
.8 1
Total 100.0 122
I felt at ease in sharing my opinions and ask questions in the classroom
Percent Frequency
N/A 80.3 98
Strongly Agree
9.0 11
Agree 8.2 10
Neutral 1.6 2
Disagree .8 1
Total 100.0 122
64
It helped me gain in-depth knowledge on the subject
Percent Frequency
N/A 80.3 98
Strongly Agree
6.6 8
Agree 11.5 14
Neutral 1.6 2
Total 100.0 122
65
Satisfaction
Percent Frequency
N/A 80.3 98
Very satisfied
9.0 11
Satisfied 9.0 11
Fair 1.6 2
Total 100.0 122
Seminar for Prospective Student Evaluators of External Quality Audits
Organisation of the Seminar
Detailed information was provided on the seminar workload and input required
Percent Frequency
N/A 86.9 106
Strongly Agree
1.6 2
Agree 8.2 10
Neutral 1.6 2
Disagree 1.6 2
Total 100.0 122
66
The learning outcomes of the seminar were clearly stated
Percent Frequency
N/A 86.9 106
Strongly Agree
.8 1
Agree 7.4 9
Neutral 4.9 6
Total 100.0 122
67
Information on the seminar was provided well in advance
Percent Frequency
N/A 86.9 106
Strongly Agree
.8 1
Agree 4.9 6
Neutral 4.9 6
Disagree 2.5 3
Total 100.0 122
The seminar was well organised
Percent Frequency
N/A 86.9 106
Strongly Agree
3.3 4
Agree 7.4 9
Neutral 1.6 2
Disagree .8 1
Total 100.0 122
68
Seminar Content
The lecturers allowed for active participation and interaction
Percent Frequency
N/A 86.9 106
Strongly Agree
7.4 9
Agree 4.1 5
Neutral 1.6 2
Total 100.0 122
69
The seminar content reflected projected External Quality Audit requirements
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 6.6 8
Neutral 4.1 5
Total 100.0 122
The seminar was practical
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 6.6 8
Neutral 4.1 5
Total 100.0 122
70
Lecturers
The lecturers were well prepared and fully resourced
Percent Frequency
N/A 86.9 106
Strongly Agree
4.1 5
Agree 7.4 9
Neutral 1.6 2
Total 100.0 122
71
The lecturers had a good knowledge on the subject
Percent Frequency
N/A 86.9 106
Strongly Agree
8.2 10
Agree 4.9 6
Total 100.0 122
The lecturers used technology effectively to advance my learning
Percent Frequency
N/A 86.9 106
Strongly Agree
3.3 4
Agree 4.9 6
Neutral 4.9 6
Total 100.0 122
72
The lecturers made the subject interesting
Percent Frequency
N/A 86.9 106
Strongly Agree
5.7 7
Agree 4.9 6
Neutral 2.5 3
Total 100.0 122
73
The lecturers provided useful material
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 7.4 9
Neutral 2.5 3
Disagree .8 1
Total 100.0 122
Learning resources
The resources were enough to meet my needs
Percent Frequency
N/A 86.9 106
Strongly Agree
.8 1
Agree 5.7 7
Neutral 4.9 6
Disagree .8 1
Strongly Disagree
.8 1
Total 100.0 122
74
The number of students in relation to the room size was acceptable
Percent Frequency
N/A 86.9 106
Strongly Agree
5.7 7
Agree 6.6 8
Neutral .8 1
Total 100.0 122
75
The equipment of the learning area was useful
Percent Frequency
N/A 86.9 106
Strongly Agree
4.1 5
Agree 6.6 8
Neutral 1.6 2
Disagree .8 1
Total 100.0 122
The educational material helped me understand the topics
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 4.1 5
Neutral 4.9 6
Disagree 1.6 2
Total 100.0 122
76
The educational material was up to date
Percent Frequency
N/A 86.9 106
Strongly Agree
4.1 5
Agree 5.7 7
Neutral 2.5 3
Disagree .8 1
Total 100.0 122
77
The educational material was easy to understand
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 6.6 8
Neutral 3.3 4
Strongly Disagree
.8 1
Total 100.0 122
The educational material was well structured
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 5.7 7
Neutral 4.1 5
Strongly Disagree
.8 1
Total 100.0 122
78
The educational material was relevant
Percent Frequency
N/A 86.9 106
Strongly Agree
3.3 4
Agree 9.0 11
Strongly Disagree
.8 1
Total 100.0 122
79
Teaching Methods
Lectures
Percent Frequency
N/A 86.9 106
Very good
4.9 6
Good 5.7 7
Fair 1.6 2
Poor .8 1
Total 100.0 122
Presentations
Percent Frequency
N/A 86.9 106
Very good
4.1 5
Good 7.4 9
Fair .8 1
Very poor
.8 1
Total 100.0 122
80
Group activities / discussions
Percent Frequency
N/A 86.9 106
Very good
4.1 5
Good 6.6 8
Fair 1.6 2
Poor .8 1
Total 100.0 122
81
Problem handling
Percent Frequency
N/A 86.9 106
Good 10.7 13
Fair 1.6 2
Don't know
.8 1
Total 100.0 122
Additional considerations on the seminar
I was involved in the group discussions during the seminar
Percent Frequency
N/A 86.9 106
Strongly Agree
5.7 7
Agree 7.4 9
Total 100.0 122
82
The seminar met my expectations
Percent Frequency
N/A 86.9 106
Strongly Agree
2.5 3
Agree 7.4 9
Neutral 3.3 4
Total 100.0 122
83
I would definitely recommend this seminar if provided in the future
Percent Frequency
N/A 86.9 106
Strongly Agree
3.3 4
Agree 7.4 9
Neutral 1.6 2
Don't Know
.8 1
Total 100.0 122
I felt respected during the seminar
Percent Frequency
N/A 86.9 106
Strongly Agree
5.7 7
Agree 6.6 8
Neutral .8 1
Total 100.0 122
84
I felt at ease in sharing my opinions and ask questions in the classroom
Percent Frequency
N/A 86.9 106
Strongly Agree
6.6 8
Agree 5.7 7
Neutral .8 1
Total 100.0 122
85
It helped me gain in-depth knowledge on the subject
Percent Frequency
N/A 86.9 106
Strongly Agree
3.3 4
Agree 6.6 8
Neutral 2.5 3
Disagree .8 1
Total 100.0 122
Net-QAPE Meetings
Attendance
Percent Frequency
N/A 73.0 89
Yes 24.6 30
No 2.5 3
Total 100.0 122
86
Rate of attendance
Percent Frequency
N/A 73.0 89
All meetings
11.5 14
Most meetings
14.8 18
Few meetings
.8 1
Total 100.0 122
87
Content
The meetings were useful
Percent Frequency
N/A 73.0 89
Strongly Agree
8.2 10
Agree 15.6 19
Neutral 3.3 4
Total 100.0 122
88
The meetings met my expectations
Percent Frequency
N/A 73.0 89
Strongly Agree
4.9 6
Agree 13.9 17
Neutral 5.7 7
Disagree 2.5 3
Total 100.0 122
The information provided was relevant for my personal development
Percent Frequency
No answer
.8 1
N/A 73.0 89
Strongly Agree
3.3 4
Agree 18.0 22
Neutral 3.3 4
Disagree 1.6 2
Total 100.0 122
89
The information provided was relevant for my institution
Percent Frequency
N/A 73.0 89
Strongly Agree
3.3 4
Agree 19.7 24
Neutral 3.3 4
Disagree .8 1
Total 100.0 122
90
I will put into practise the information provided
Percent Frequency
N/A 73.0 89
Strongly Agree
6.6 8
Agree 17.2 21
Neutral 3.3 4
Total 100.0 122
The speakers delivered the information
Percent Frequency
N/A 73.0 89
Strongly Agree
6.6 8
Agree 13.9 17
Neutral 5.7 7
Don't Know
.8 1
Total 100.0 122
91
Aspects of the Net-QAPE Meetings
Content
Percent Frequency
N/A 73.0 89
Very good
7.4 9
Good 14.8 18
Fair 2.5 3
Poor 2.5 3
Total 100.0 122
92
Organisation
Percent Frequency
N/A 73.0 89
Very good
7.4 9
Good 14.8 18
Fair 2.5 3
Poor 1.6 2
Very poor
.8 1
Total 100.0 122
93
Interaction with participants
Percent Frequency
N/A 73.0 89
Very good
5.7 7
Good 14.8 18
Fair 3.3 4
Poor 3.3 4
Total 100.0 122
The use of visual aids
Percent Frequency
N/A 73.0 89
Very good
6.6 8
Good 13.9 17
Fair 4.1 5
Poor 1.6 2
Very poor
.8 1
Total 100.0 122
94
Location of the meetings
Percent Frequency
N/A 73.0 89
Very good
4.1 5
Good 13.1 16
Fair 8.2 10
Poor 1.6 2
Total 100.0 122
95
Date and time
Percent Frequency
N/A 73.0 89
Very good
2.5 3
Good 16.4 20
Fair 5.7 7
Poor 1.6 2
Very poor
.8 1
Total 100.0 122
Meeting facility
Percent Frequency
N/A 73.0 89
Very good
4.9 6
Good 15.6 19
Fair 6.6 8
Total 100.0 122
96
Registration
Percent Frequency
N/A 73.0 89
Very good
8.2 10
Good 16.4 20
Fair 1.6 2
Don't know
.8 1
Total 100.0 122
97
Parking
Percent Frequency
N/A 73.0 89
Very good
3.3 4
Good 13.1 16
Fair 6.6 8
Poor 2.5 3
Very poor
.8 1
Don't know
.8 1
Total 100.0 122
98
Quality Audit Process
Aspects
I felt sufficiently involved in the External Quality Audit
Percent Frequency
N/A 63.9 78
Strongly Agree
12.3 15
Agree 17.2 21
Neutral 5.7 7
Don't Know
.8 1
Total 100.0 122
99
I felt respected by the External Quality Audit evaluation panel
Percent Frequency
N/A 63.9 78
Strongly Agree
14.8 18
Agree 17.2 21
Neutral 2.5 3
Don't Know
1.6 2
Total 100.0 122
I had enough time to prepare for the External Quality Audit
Percent Frequency
No answer
.8 1
N/A 68.9 84
Strongly Agree
3.3 4
Agree 9.8 12
Neutral 13.1 16
Disagree 2.5 3
Strongly Disagree
.8 1
Don't Know
.8 1
Total 100.0 122
100
It helped my institution to strengthen the External Quality Audit system
Percent Frequency
N/A 68.9 84
Strongly Agree
7.4 9
Agree 15.6 19
Neutral 6.6 8
Don't Know
1.6 2
Total 100.0 122
101
The instructions and documents provided were sufficiently enlightening
Percent Frequency
N/A 79.5 97
Strongly Agree
2.5 3
Agree 12.3 15
Neutral 3.3 4
Disagree .8 1
Strongly Disagree
.8 1
Don't Know
.8 1
Total 100.0 122
I was involved in the internal quality assurance of the institution
Percent Frequency
N/A 63.9 78
Strongly Agree
10.7 13
Agree 18.9 23
Neutral 4.1 5
Don't Know
2.5 3
Total 100.0 122
102
I felt empowered as a student during the External Quality Audit
Percent Frequency
N/A 95.1 116
Strongly Agree
2.5 3
Agree 1.6 2
Neutral .8 1
Total 100.0 122
103
The External Quality Audit was useful
Percent Frequency
N/A 95.1 116
Strongly Agree
2.5 3
Agree 1.6 2
Disagree .8 1
Total 100.0 122
Stressful
Percent Frequency
N/A 63.9 78
Very stressful
4.1 5
Stressful 8.2 10
Neutral 12.3 15
Not stressful
7.4 9
Not stressful at all
3.3 4
Don't know
.8 1
Total 100.0 122
104
Concluding Conference
The conference was useful
Percent Frequency
N/A 59.8 73
Strongly Agree
11.5 14
Agree 21.3 26
Neutral 4.9 6
Disagree 2.5 3
Total 100.0 122
105
The conference met my expectations
Percent Frequency
No answer
.8 1
N/A 59.8 73
Strongly Agree
9.8 12
Agree 22.1 27
Neutral 5.7 7
Disagree .8 1
Strongly Disagree
.8 1
Total 100.0 122
The information provided was relevant for my personal development
Percent Frequency
N/A 59.8 73.0
Strongly Agree
8.2 10.0
Agree 20.5 25.0
Neutral 9.0 11.0
Disagree 1.6 2.0
No answer
0.8 1.0
Total 100.0 122.0
106
The information provided was very relevant for my institution
Percent Frequency
N/A 59.8 73
Strongly Agree
9.8 12
Agree 18.9 23
Neutral 11.5 14
Total 100.0 122
107
I will put into practice the information provided
Percent Frequency
No answer
.8 1
N/A 59.8 73
Strongly Agree
9.8 12
Agree 20.5 25
Neutral 9.0 11
Total 100.0 122
The speakers clearly delivered the information
Percent Frequency
No answer
.8 1
N/A 59.8 73
Strongly Agree
12.3 15
Agree 19.7 24
Neutral 6.6 8
Disagree .8 1
Total 100.0 122
108
Aspects
Content
Percent Frequency
N/A 59.8 73
Very good
9.8 12
Good 23.0 28
Fair 4.9 6
Poor 2.5 3
Total 100.0 122
109
Organisation
Percent Frequency
N/A 59.8 73
Very good
15.6 19
Good 18.0 22
Fair 4.1 5
Very poor
.8 1
No answer
1.6 2
Total 98.4 120
Interaction with participants
Percent Frequency
N/A 59.8 73
Very good
8.2 10
Good 16.4 20
Fair 13.9 17
Poor 1.6 2
Total 100.0 122
110
The use of visual aids
Percent Frequency
N/A 59.8 73
Very good
9.8 12
Good 21.3 26
Fair 8.2 10
Poor .8 1
Total 100.0 122
111
Location of the conference
Percent Frequency
N/A 59.8 73
Very good
9.8 12
Good 21.3 26
Fair 5.7 7
Poor 2.5 3
Very poor
.8 1
Total 100.0 122
Date and time
Frequency Percent
Valid N/A 73 59.8
Very good
13 10.7
Good 27 22.1
Fair 7 5.7
Poor 1 .8
No answer
1 .8
Total 122 100.0
112
Conference facility
Percent Frequency
N/A 59.8 73
Very good
11.5 14
Good 21.3 26
Fair 7.4 9
Total 100.0 122
113
Registration
Percent Frequency
N/A 59.8 73
Very good
15.6 19
Good 22.1 27
Fair 1.6 2
Don't know
.8 1
Total 100.0 122
Parking
Percent Frequency
N/A 59.8 73
Very good
11.5 14
Good 18.9 23
Fair 4.1 5
Poor 2.5 3
Very poor
.8 1
Don't know
1.6 2
No answer
.8 1
Total 100.0 122
114
Interest in similar future events
Percent Frequency
N/A 59.8 73
Very interested
23.0 28
Interested 13.1 16
Fair 3.3 4
Not interested
.8 1
Total 100.0 122
115