Customer Satisfaction Report
-
Upload
viet-long-plaza -
Category
Documents
-
view
221 -
download
0
Transcript of Customer Satisfaction Report
-
8/3/2019 Customer Satisfaction Report
1/16
EUROCONTROL
A Centre of Excellence in ATM Training
IANSCustomer Satisfaction
Repor
Report
Sieu thi dien may Viet Long - www.vietlongplaza.com.vn
-
8/3/2019 Customer Satisfaction Report
2/16
ii Copyright 2006 EUROCONTROL
Copyright Notice
2006 European Organisation for the Safety of Air Navigation (EUROCONTROL). All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic,
mechanical, photocopying, recording or otherwise.
Any other use is subject to prior written consent by EUROCONTROL. Request shall be addressed to:
Director of the Institute of Air Navigation Services, 12, rue Antoine de Saint-Exupry, L-1432 Kirchberg/Luxembourg.
The EUROCONTROL Institute of Air Navigation Services aims to provide the services that you wantand to make your stay in the Institute as enjoyable as possible. All Institute personnel are there toensure that your stay at the Institute is successful. However, if you do have a complaint (or acompliment) please tell us. If you are not satisfied with the service we provide or you would like topropose an improvement then please fill out the form at http://www.eurocontrol.int/ians/complaint.html,
or contact [email protected] directly.
-
8/3/2019 Customer Satisfaction Report
3/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 3
Table of Contents
INTRODUCTION..................................................................................................................................... 4ATM TRAINING......................................................................................................................................5ATC TRAINING ....................................................................................................................................12E-LEARNING........................................................................................................................................ 13STAKEHOLDER FEEDBACK.............................................................................................................. 14COMPLAINTS ......................................................................................................................................15CONCLUSION......................................................................................................................................16
-
8/3/2019 Customer Satisfaction Report
4/16
Customer Satisfaction Report
4 Copyright 2006 EUROCONTROL
Introduction
This is the 7th
report that the Institute devotes to theassessment of the level of satisfaction of its
customers. The study on which this report is basedhas been conducted in the framework of the qualityimprovement programme implemented at theInstitute since end of 1996.
Measuring customer satisfaction on a regular basisis one of the key elements of the quality strategyimplemented by the Institute since 1997.
Our objective is to understand exactly how ourcustomers perceive the Institute and, based on theirlevel of satisfaction, to improve our services andproducts.
This report proposes a summary and an analysis ofthe data collected in 2005.
The customer segmentsassessed in 2005
IANS applies a policy to measure the satisfaction ofits customers with all its products and services.These include:
The ATM course participants
The ATC course participants
The E-Learning Students
Training Sponsors
Consultancy Bodies.
User Groups
Assessment Tools
The assessment is based on a set of assessmenttools and measurement methods. During 2005further work has been done to increase the integrityof the data collection.
Evaluation Forms
To measure the satisfaction of its customers withthe products and services IANS uses the followingevaluation forms:
A Student Evaluation Form filled in by thestudents at the end of a course. This evaluation
form is used for both the ATM courseparticipants and ATC course participants.
An E-Learning Evaluation Form filled in by astudent upon completion of an E-LearningModule.
A generic Stakeholder Feedback Form filled inby various stakeholders. The form is availableathttp://www.eurocontrol.int/ians/feedback.html.
Customer Visits
IANS visits a sample of its customers on a yearlybasis. The customer visits serve as a main source
of feedback on existing products and a source ofinformation for improvements and new products.The customer visits include visits to Air NavigationService Providers and civil and military StateAuthorities.
Complaint Procedure andComplaint Form
IANS uses an anonymous form, available on itsweb-site for the collection of complaints related toits products, services and staff. A reference to thecomplaint procedure and associated form is madein all products and services. The complaint form isavailable athttp://www.eurocontrol.int/ians/complaint.html.
-
8/3/2019 Customer Satisfaction Report
5/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 5
ATM Training
The Satisfaction of Students with the ATM Training Products is measured through the use of a Course EvaluationForm.
MeasurementMethod
At the end of each(classroom) course,students are presentedwith an evaluation formcontaining a number ofcriteria to be assessed.Each criterion can begiven a result between 1and 4 meaningunacceptable (1),unsatisfactory (2), good
(3) an excellent (4).
The complete set ofcriteria are grouped infive main categories:
Training Content
Instructional Team
Documentation
Equipment
Administration
The first three categoriesare directly related to the
course. The last twocategories are directlyrelated to the supportingenvironment andorganisation.
Figure 1 Student Evaluation Form
-
8/3/2019 Customer Satisfaction Report
6/16
Customer Satisfaction Report
6 Copyright 2006 EUROCONTROL
From Forms toResults
We map the marks of thestudents on a scale from
0 (all students markUnacceptable) to +100(all students markExcellent). When allstudents mark Goodthen the result is 75 andwhen students mark, onaverage, between Goodand Excellent then theresult will be around 85.Likewise, if students fillin 'all-Uncacceptable'then the result will be 0.If students fill in 'all-Un-Satisfactory' then theresult will be 25.
1Copyright 2003 EUROCONTROL
Measuring SatisfactionMeasuring Satisfaction
1
Unsatisfactory Satisfactory
2
Good
3
Excellent
4
Marks from Students
ResultResult
All 3All 3All 2All 2All 1All 1
2525 7575 10010000
All 4All 4
Figure 2 How we measure satisfaction
IANS aims to meet our objective of being a Centreof Excellence in ATM Training. For us this meanswe aim to achieve results on any criterion between75 and 100. As an example: the overall average (allcourses, all criteria, 3090 forms) for IANS in 2005 is87.5 compared to 85 in 2004. The highest resulton a single criterion is 95 (Subject Knowledge of theInstructor). The lowest result remains pre-courseinformation (76.5 compared to 70 in 2004).
Results
The results presented below are extracted from theautomated student satisfaction evaluation system.Results can be presented per category, per trainingdomain and per criterion. We have selected the
most significant results to be included in this report.
-
8/3/2019 Customer Satisfaction Report
7/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 7
0
500
1000
1500
2000
2500
3000
3500
Number of Forms 1937 1915 2624 3090
2002 2003 2004 2005
Figure 3 Number of Evaluation Forms
Size of thePopulation
This graph shows thenumber of evaluationforms received and
processed by IANS. In2005 the Institutecontinued to optimise theprocessing of the forms.
This led to anothersignificant increase ofthe number of processedforms in 2005.
Composition ofthe Population
Students indicate on theevaluation form whichtype of staff they are.
This chart shows that amajority of Students atIANS in 2005 areOperational andTechnical Staff.
These results areconsistent with theresults of the previousyears.
192
345
977
693
223
600
1298
824
0
200
400
600
800
1000
1200
1400
Administrative Managerial Operational Technical
20042005
Figure 4 Composition of the population (number of students)
-
8/3/2019 Customer Satisfaction Report
8/16
Customer Satisfaction Report
8 Copyright 2006 EUROCONTROL
50
55
60
65
70
75
80
85
90
95
100
Course Results 83 84,5 86 87,5
2002 2003 2004 2005
Figure 5 ATM Course Results
Satisfaction
This chart shows thesatisfaction of thestudents with the ATMcourses. The result isobtained by averagingthe results obtained onevery criterion in thecategories TrainingContent, InstructionalTeam andDocumentation.
A result of 87 indicatesthat, on average,students mark criteriabetween Good andExcellent.
The increase insatisfaction has been
achieved throughdetailed analysis of theresults of each courseand introducingcorrective actionwherever required.
50
55
60
65
70
75
80
8590
95
100
Content 81,5 83 84,5 86,5
Instructor 88 89 91 92
Documentation 80 80,5 83,5 85
2002 2003 2004 2005
Figure 6 ATM Course Results: per Category
Segmented View
This chart presents theresults per measurement
Category. These areTraining Content,Instructional Team andDocumentation.
IANS has improved theresults on each of thesethree categories duringthe last four years.
IANS will continue toanalyse the results andmake improvementswhere possible andpracticable.
-
8/3/2019 Customer Satisfaction Report
9/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 9
Best Results
Customers often praiseIANS for its staff andexcellent infrastructure.
This informal feedback is
confirmed by the courseevaluation results. Thecriteria InstructorSubject Knowledge andClassroom Equipmenthave consistentlyreceived the highestmarks from the studentsduring the last threeyears.
IANS will continue toinvest in the competenceof its instructors bycontinuation training and
competency assessmentand it will maintain thebuilding at its highstandard.
50
60
70
80
90
100
Instructor Subject
Knowledge
92,5 93 94,5 95,5
Classroom
Equipment
88,5 89 90,5 91
2002 2003 2004 2005
Figure 7 Criteria with Best Results
50
60
70
80
90
100
Pre Course
Information
61,5 64 70,5 78
Course Duration 71 74 76 76
Canteen 73,5 72,5 75 77,5
2002 2003 2004 2005
Figure 8 Criteria with less positive Results
Areas forImprovement
IANS continuously looksfor opportunities toimprove. Criteria with theless positive results arethe Pre-CourseInformation and theCourse Duration.
In 2004 IANS has put inplace an action plan toimprove the Pre-Courseinformation andsignificant progress hasbeen made in thisdomain in 2004 and2005.
The optimum duration foreach course will bereviewed on a course-by-course basis.
In 2005 IANS changedthe layout of the coffeearea and installed newequipment in the kitchento improve the flow ofstudents.
-
8/3/2019 Customer Satisfaction Report
10/16
Customer Satisfaction Report
10 Copyright 2006 EUROCONTROL
Focus on Safety
IANS is heavily involvedin the development anddelivery of SafetyCourses to supportSafety Regulation andSafety Management.
In 2005 the number ofcourses delivered in theSafety domain increasedfrom 36 to 50 and thenumber of participantsincreased from 614 to844.
We are pleased to notethat the results in thisrelatively new area forIANS equal the averageobtained in other, more
mature, trainingdomains.
It is a clear indicationthat the rigorousdevelopment approachat IANS leads toproducts that meetcustomer expectation.
50
55
60
65
70
75
80
85
90
95
100
Safety 86 88,5 90
Overall 84,5 86 87,5
2003 2004 2005
Figure 9 Safety Domain Course Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%100%
2004 91% 1% 8%2005 92% 1% 7%
Yes No Blank
Figure 10 Would recommend course to a colleague
Measuring
LoyaltyIn 2004 we introduced anew question on theevaluation form: Wouldyou recommend thecourse to a colleague.
In 2005, like in 2004,more than 90% of thestudents wouldrecommend the coursethat they attended to acolleague.
-
8/3/2019 Customer Satisfaction Report
11/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 11
0,00%
1,00%
2,00%
3,00%
4,00%
5,00%
6,00%
7,00%
8,00%
9,00%
10,00%
Percentage of'Unsatisfactory'
Results
0,91% 0,90% 0,83% 0,59%
2002 2003 2004 2005
Figure 11 Level of dissatisfaction
Measuringdissatisfaction
To measuredissatisfaction IANS usesthe percentage of
unsatisfactory marks onthe evaluation forms.
This chart shows theproportion of un-satisfactory marks on theevaluation forms for allcriteria.
The excellent results in2005 are spread over theentire spectrum of criteria.
Note that that anunsatisfactory mark onone criterion does not
necessarily mean that thestudent is not satisfiedwith the whole product.
-
8/3/2019 Customer Satisfaction Report
12/16
Customer Satisfaction Report
12 Copyright 2006 EUROCONTROL
ATC Training
The satisfaction of ATC ab-initio students ismeasured using the same evaluation as the systemused for measuring the satisfaction of the ATM
students. Satisfaction figures of ATM and ATCstudents must however not be compared. Thecontext of these courses, although delivered in thesame building, is completely different:
The duration of ATM courses is maximum oftwo weeks whereas ATC have a duration ofseveral months.
The student population of ATM courses arestaff members from service providers andmember states.
The student population of ATC courses are ab-inito ATC students
Last but not least, the ATC ab-initio studentsare evaluated at the end of the course. At theend of the evaluation they either pass or fail.
The outcome of the evaluation tends to have asignificant impact on the satisfaction of thestudents.
IANS believes that the critical success factor forATC students is their validation rate (i.e. theproportion of students that obtain an ATC license).We do however present the satisfaction data in thisreport as it is part of our quality management tools.
50
60
70
80
90
100
Training Content 79,5 79 75 78
Instructors 76,5 73 72,5 77
TrainingDocumentation
78 69 76 78
2002 2003 2004 2005
Figure 12 ATC Student Satisfaction
ATC Student
SatisfactionThis chart shows thesatisfaction of ATC ab-initio students.
The last few years ATCab-initio training has beencharacterised by highrotation of staff andcontinuous changes to thetraining content andstructure as a result ofvariations in demand andnew regulatory
requirements.In 2005, IANS significantlyimproved thedocumentation andtraining of the ATCInstructors.
Compared to 2004 thesatisfaction of thestudents has increased foreach segment.
-
8/3/2019 Customer Satisfaction Report
13/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 13
E-Learning
IANS introduced satisfaction measurement for its e-learning products in the middle of 2004. Studentsare given an opportunity to evaluate each e-learning
module at the end of the module. The figurespresented in this report cover all the modules on theEUROCONTROL IANS on-line TrainingZone,including the CFMU modules.
In 2005, students filled in 2774 forms (800 in 2004)evaluation forms were filled in. 90% of the studentsidentified to be satisfied with the modules and
more than 88% of the students indicated that themodule met the objectives.
These figures indicate that the overall satisfactionwith the e-learning modules is excellent.
1. Why are you studying this module? Personal reasons (e.g. to further career, out of personal interest etc). My employer decided I should. It was a prerequisite for a class-room based course or programme.
2. The module agree not sure-n/a disagree
followed a logical sequence, O O Owas not too easy or too difficult, O O Owas clear and easy to understand. O O O
3. The module was agree not sure-n/a disagree
easy to use, O O Ointeractive and interesting. O O O
4. In order to successfully access this module, did you have to
change your browser (Internet Explorer) settings? install new software (flash plug-in, java plug-in, internet explorer)? disable a popup-killer or anti-intrusion programme (e.g. Norton Internet
Security)?
5. The module agree not sure-n/a disagree
met the objectives set at the beginning,O O Oallowed me to check my understanding. O O O
6. Did you enjoy studying this module: O yes O no
7. Will this module
Help you directly in your current job? Help you in a future job? Increase your understanding of the industry without direct impact on your
current/future job? Help you personally (e.g. out of personal interest)?
E-LearningEvaluation Form
A customised form wasdeveloped to capturestudent satisfaction withthe e-learning modules.The form collectsinformation related to thereasons for studying, thecomplexity of the module,the objectives of themodule and the impact ofthe module on thestudents job.
0%
10%
20%
30%
40%
50%
60%70%
80%
90%
100%
2004 92% 73% 86%
2005 95% 80% 87%
structure level clarity of language
Figure 13 Structure of E-Learning Modules
Structure of E-Learning Modules
This chart shows thepercentage of students
that answered yes on thefollowing questions:
Structure of the modulemeets expectations.
Level of the modulemeets the expectations
Module uses clearlanguage.
These results indicate thatthe structure and contentof the E-Learning modulesmeet the needs of the
students
-
8/3/2019 Customer Satisfaction Report
14/16
Customer Satisfaction Report
14 Copyright 2006 EUROCONTROL
0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
Self-driven 43% 28%
Employer-driven 30% 39%
Prerequisite 27% 33%
2004 2005
Figure 14 Why did students study e-learning modules
Origin of E-LearningStudents
It is IANS strategy tointegrate e-learning in
well defined trainingprogrammes definedand implemented by thestudents employer - andas pre-requisite modulesfor classroom courses.
These results indicatethat the proportion ofstudents registeredthrough their employer orstudying as a pre-requisite module hassignificantly increasedcompared to the number
of students that study outof their own initiative.
The number of studentsthat see a direct impacton their job fromstudying e-learningmodules has increasedfrom 56% in 2004 to68% in 2005.
-
8/3/2019 Customer Satisfaction Report
15/16
Printed on 30/11/2004 Produced by the Business and Administration Unit 15
Stakeholder Feedback
Training Consultation Group
The Stakeholder Feedback Form was distributed tothe meeting participants at the end of each TrainingConsultation Group Meeting. The majority of thefeedback indicates that the overall perception of theInstitute lies between Good and Excellent (on ascale unsatisfactory satisfactory good excellent).
Compared to 2004 the results related tocommunication (content, format and timeliness)have improved. We will continue to review and
improve our communication with the members ofthe consultation group.
Other stakeholdersThe stakeholder evaluation form was alsodistributed to:
Participants of the training focus group.
The training partners at IATA.
The results indicate an equally high level ofsatisfaction of working together with IANS.
Customer VisitsThe Customer Visits are a process in which IANSmeets stakeholders in member states on a bi-lateralbasis. It is a mechanism to collect qualitativefeedback from various stakeholders. The countriesvisited in 2005 are: Austria, Belgium, BIH, Bulgaria,Greece, Hungary, Spain, Sweden and Turkey
During the meeting of the Training ConsultationGroup in September 2005 it was agreed that IANSwould include in its Customer Satisfaction Reportinformation gathered during the Customer Visits.
During these customer visits, stakeholders have in
general expressed a high degree of satisfaction withIANS, its staff and the services it offers. Thefollowing points have been raised:
OJTI: The delivery of OJTI courses throughpartnerships is often perceived as anexpensive solution (by stakeholders receivingthe services). The solution is not perceived asentirely satisfactory.
ATSEP Training: Many of the visitedstakeholders are currently in the process of
defining basic training for Air Traffic SafetyElectronics Personnel (ATSEP). Somecollaboration with EUROCONTROL and othermember states could avoid duplication of effort.
On-Site Courses: In general the demand foron-site course (i.e. courses at stakeholderpremises) exceeds the capacity of IANS.
Although most stakeholders are aware of theE-Learning products of the Institute many ofthem have not yet been able to assess the
feasibility to use the products available. ATC Training Notes: IANS should develop a
policy to distribute CCC aligned ATC TrainingNotes.
Complaints
Every training note distributed to students and all
pages on the On-Line Trainingzone contain aclearly visible reference to IANS complaintprocedure. The procedure enables stakeholders tosubmit complaints in an anonymous way or anidentified complaint through e-mail. The text is asfollows:
The EUROCONTROL Institute of Air NavigationServices aims to provide the services that you wantand to make your stay in the Institute as enjoyableas possible. All Institute personnel are there toensure that your stay at the Institute is successful.However, if you do have a complaint (or a
compliment) please tell us. If you are not satisfied
with the service we provide or you would like topropose an improvement then please fill out theform athttp://www.eurocontrol.int/ians/complaint.html, orcontact [email protected] directly.
Although the procedure is clearly marked in alldocumentation, IANS did not receive formalcomplaints in 2005.
-
8/3/2019 Customer Satisfaction Report
16/16
Customer Satisfaction Report
16 Copyright 2006 EUROCONTROL
Conclusion
IANS aims to be recognised by its stakeholders aCentre of Excellence in ATM Training. We
continuously improve our products and processes.We measure the satisfaction of our customers withthe products that we provide.
This document presented the results of thesatisfaction measurement in 2005. The satisfactionwith our ATM Training Products shows an upwardtrend in all domains for the fourth year in a row. The
satisfaction of the ATC Ab-initio students with theATC training started increasing again in 2005. In
2005 we have further exploited satisfactionmeasures for the E-Learning Products and for theStakeholders that we work with. The results in thesedomains show a very high degree of stakeholdersatisfaction.
END OF DOCUMENT.