Theory of Relativity · ―ATSEP.OR.300/305 Competence Assessment: A service provider employing...
Transcript of Theory of Relativity · ―ATSEP.OR.300/305 Competence Assessment: A service provider employing...
1
QUASAR New star in ATSEP assessment
Author Petr Dvořák
Version 1.8
Date February 13, 2020
Slides 23
2
slide 2
QUASAR
3
slide 3
…in ATSEP assessment
New star…
4
slide 4
Questionnaire
for
ATSEP
Standard
Assessment
Routines
5
slide 5
Context
6
slide 6
QUASAR Initiative
Inter-FAB cooperation
on ATSEP assessment harmonisation
Internationally recognised group
of experts on ATSEP Training
Ambitions to become European quasi-standard
Outstanding example
of both intra- and inter- FAB cooperation
Formal status coming from FAB CE HR SubC activities
and FABEC Training Task Force
What?
7
slide 7
Harmonisation of ATSEP training
Common training requirements / objectives are the same ― Eurocontrol ATSEP CCC Initial Training [SPEC-132]
― EU Regulation 2017/373 + EASA AMC/GM (under review of RMT.0719)
Training process may be different ― Training materials, duration, techniques, …
― Different local environment, …
― Different organisational structure, culture, etc.
Training output shall be the same ― Competent ATSEP on comparable level
Assessment shall be done ― “establish, implement and document”
Why?
8
slide 8
Harmonisation of ATSEP training
Common training material ― Not possible for various reasons
Common assessment ― Common final test (question database)
― Common standardised rules (Terms of Use)
― Tool for independent evaluation of training
QUASAR Examination
European-wide scope
How?
9
slide 9
Common database of final test questions for initial training ― Direct mapping to the defined training objectives
― Each objective covered by multiple questions
― Question complexity relates to taxonomy level
― Simple, understandable, unambiguous and consistent language (EN)
― Using multiple choice questions (single or multiple correct answers)
Questions cover all parts of ATSEP Initial Training
Outputs
Training part Total number of
questions
Number of training
objectives
Basic Training 621 198
Qualification Training – Shared 166 71
Qualification Training – COM 223 66
Qualification Training – NAV 312 149 (+34 MLS)
Qualification Training – SUR 255 100
Qualification Training – DPR 227 71
Qualification Training – SMC 61 40
TOTAL 1865 695
10
slide 10
Terms of Use ― How to generate and evaluate tests
― Defined standard pass mark (75 %)
― Rules for organising assessments
― Rules on quality monitoring (feedback) and maintenance
― Rules for handling of QUASAR material (confidentiality, copyright)
Outputs
11
slide 11
Complex development, review and approval process ― Collection of currently used material from participating organisations
― Review of questions and creation of new ones to cover all training
objectives
― Series of reviews incl. pilot runs, validation and feedback
― Release (common approval in QTF)
― Expert review (experts in participating organisations)
― Final approval (implementation of gathered comments)
― Working environment with interactive Access database
created by Austro Control and hosted by LPS SR
― Around 100 experts participated in the process
― Regular 2-3 day meetings every 2 months
― Offline work (”Homeworks”), distribution of tasks, use of SMEs
― Gathered a total of 2677 questions / 1865 approved
― About 30 % of gathered questions were dropped out
(not meeting high quality requirements)
Process
12
slide 12 01/2013 01/2014 01/2015 01/2016 01/2017 01/2018 01/2019 01/2020
QUASAR Task Force / Timeline
FAB CE (2013)
― ANS CR
― Austro Control
― BHANSA
― Croatia Control
― Hungarocontrol
― LPS SR
― Slovenia Control
FABEC (2015)
― DFS
― Skyguide
― ENAC/DSNA
― mandate from other FABEC members
(ANA Luxembourg, Belgocontrol, LVNL, MUAC)
NATS (2015)
AVINOR (2017)
12 core member organisations (not all participating all the time)
Who?
13
slide 13
QUASAR Task Force (QTF) *highlighted currently
active members
14
slide 14
Statistics
15
slide 15
Next Steps Continuous use of QUASAR examinations
Incorporation of changes in EU regulation 2017/373 and
EASA AMC/GM based on the work of EASA RMT.0719 ― Introduction of Information System Security subject to Shared
― Introduction of new topics (e.g. SWIM, Virtualisation)
― Move of Functional Safety to Shared
Maintenance of QUASAR material ― Keeping the database up to date
― Incorporation of feedback
― Updates of Terms of Use
Gathering feedback and statistics
Providing QUASAR examination to 3rd parties
16
slide 16
Use and Feedback Several validation runs performed (since 2016)
QUASAR Examination already in use (since 2017)
― as main means of assessment or for further validation
― Skyguide
― ENAC
― DFS
More than 240 participants in Basic Examination ― Average score of 83.7 %
― Average reached minimum 70.5 %, maximum 94.2 %
― Similar numbers for Shared Examination (more than 80 participants)
Very good feedback on quality and usability
― NATS
― Austro Control
― ANS CR
17
slide 17
Available to 3rd parties through QTF members ― Common administrative fee of 95 EUR per student
(applicable only if not combined with training provision)
― Electronic version or paper-form test
― On-site or remote provision
― Material remains confidential under supervision of QTF
― Non-disclosure agreement mandatory
QUASAR Examination
18
slide 18
Implementation of QUASAR examination in CANI LMS ― Full implementation of all parts according to ToU
― Allows detailed analysis of student’s performance
― Detailed test protocols for documentation and archival
― Provides group’s results for different parts / subjects
― Evaluation of training (training gap analysis)
― Available online
Local Implementation
19
slide 19
High quality standard examination
for ATSEP Initial Training
Transparency through clearly defined rules
and direct mapping of each question to defined objectives
Allows compliance with EU regulation 2017/373 ― ATSEP.OR.300/305 Competence Assessment:
A service provider employing ATSEP shall:
(a) establish, implement and document processes for:
(1) assessing the initial and ongoing competence of ATSEP;
…
(b) define the following criteria against which initial and ongoing competence
shall be assessed: …
(3) knowledge.
Available also to 3rd parties through QTF members
Summary
20
slide 20
Contacts
Petr Dvořák [email protected]
Robert Horak [email protected]
Thorsten Brikey [email protected]
Chris Cox [email protected]
… or any other QTF member
More info: quasar.cani.cz
21
slide 21
Working interactive demo available
at ANS CR / CANI booth #885
Unique possibility to check your ATSEP knowledge
and have hands-on experience with QUASAR examination
Demo
22
slide 22
Demo
23
slide 23
Invitation to participate in celebrating the success story
with core QUASAR members at CANI booth #885
today at 18:15
Invitation
“The power radiated by quasars is enormous.”
24
Thank you for your attention!
See you at CANI booth #885