LAB / Standardisation Check List – LSA/PEL (Authority) Meetings Seminars and... · 2014-05-28 ·...

Ref: LSA/PEL LAB / Standardisation Check List – LSA/PEL (Authority) For internal use only Name der Organisation:

Adresse (Straße, PLZ, Ort):

Training Organisation – Referenznummer:

Datum der Aufsicht:

Ort:

Aufsichtsteam:

Lead-Auditor: Co-Auditor:

Anmerkungen:

- Bemerkungen können auf zusätzlichen Formularen festgehalten und zu der Checkliste hinzugefügt werden - Wenn ein „nicht zufrieden stellendes“ Finding festgestellt wird, ist dieses schriftlich zu dokumentieren und dem Head

of Training (HT) bzw. dem verantwortlichen Geschäftsführers (vGF) der Zivilluftfahrerschule bekannt zu geben.

Reference: ICAO Doc 9835, Circ 323 / 318 and JAR-FCL 1.010 /2.010 (4) and related AMC Section 2 1 von 21 31.01.2011,© WIJ19:08:58

AUSTRO CONTROL Language Assessment Body - Audit Questions Number Reference Questions Rating Corrective Sign Sign

(successful yes or no)

incl. due Action (Date)

Auditee Finding Codes

Inspector Finding Codes

Reference: ICAO Doc 9835, Circ 323 / 318 and JAR-FCL 1.010 /2.010 (4) and related AMC Section 2

2

JAR‐FCL 1/2

1.010 / 2.010

Language Proficiency

JAR‐FCL 1/2

AMC Section 2 In order to be accepted, the language assessment bodies should demonstrate: a) Appropriate management and staffing, and b) Quality System established and maintained to ensure compliance with, and adequacy of, assessment requirements, standards and procedures.

Yes No N/A

JAR‐FCL 1/2

AMC Section 2 The assessment documentation and records should be kept for a period of time determined by the Authority and made available to the Authority, on request.

Yes No N/A







3

JAR‐FCL 1/2

AMC Section 2 The assessment documentation should include at least the following: a) assessment objectives b) assessment layout, time scale, technologies used, assessment samples, voice samples c) assessment criteria and standards (at least for the levels 4, 5 and 6 of the Rating Scale in the AMC No. 1 to JAR-FCL 1.010) d) documentation demonstrating the assessment validity, relevance and reliability e) assessment procedures and responsibilities - preparation of individual assessment - administration: location(s), identity check and invigilation, assessment discipline, confidentiality/security - reporting and documentation provided to the Authority and/or to the applicant, including sample certificate - retention of documents and records

Yes No N/A







4

JAR‐FCL 1/2

AMC Section 2 The assessment should determine the language skills of the applicant in the following areas: a) Pronunciation: - the extent to which the pronunciation, stress, rhythm and intonation are influenced by the applicant’s first language or national variations; and - how much they interfere with ease of understanding. b) Structure: - the ability of the applicant to use both basic and complex grammatical structures; and - the extent to which the applicant’s errors interfere with the meaning. c) Vocabulary: - the range and accuracy of the vocabulary used; and - the ability of the applicant to paraphrase successfully when lacking vocabulary d) Fluency: - tempo - hesitancy - rehearsed versus spontaneous speech - use of discourse markers and connectors e) Comprehension: - on common, concrete and work-related topics; and - when confronted with a linguistic or situational complication or an unexpected turn of events, Note: The accent or variety of accents used in the test material should be sufficiently intelligible for an international community of users. f) Interactions - quality of response (immediate, appropriate, and informative) - the ability to initiate and maintain exchanges: - on common, concrete and work-related topics; and - when dealing with an unexpected turn of events - the ability to deal with apparent misunderstandings by checking, confirming or clarifying.

Yes No N/A







5

JAR‐FCL 1/2

AMC Section 2 When the assessment is not conducted in a face-to-face situation, it should use appropriate technologies for the assessment of the applicant’s abilities in listening and speaking, and for enabling interactions (for example: simulated pilot/controller communication). Note: Digital / Interactive Test Program

Yes No N/A

PART T ICAO Circular 318‐AN/180

TESTING

1 ICAO Circular 318‐AN/180

AVIATION –SPECIFIC LANGUAGE TESTING ISSUES

1.1. Chapter 1.4.2 Does the language test used by the LAB focus on speaking and listening?

Yes No N/A

1.2. Chapter 1.4.3 Pilots and air traffic controllers need to be tested in content similar to that in which they work. Is the test content used in the LAB relevant to their work role?

Yes No N/A







6

1.3. Chapter 1.4.4. Radiotelephony communication requires not only the use of ICAO standardized phraseology but also the use of “plain” language? Does the test used by the LAB include also radiotelephony communications when standardized phraseology is not appropriate?

Yes No N/A

1.4 Chapter 1.4.5 Does the LAB use test contents that are in accordance with the provisions of the ICAO language proficiency requirements such as “ work‐related topics”, “work related context”, “routine work situation” and “vocabulary and comprehension which refers to work related topic”

Yes No N/A

1.4 Chapter 1.4.12 The level 6 descriptors in the ICAO rating scale refer to features of language use that go beyond the work related context indicated in the descriptors at lower levels. Is the assessment at Level 6 carried out by a trained and qualified rater?

Yes No N/A







7

2 ICAO Circular 318‐AN/180 Chapter 3

TEST DESIGN AND CONSTRUCT Yes No N/A

2.1. 2.1.1. Is the test used by the LAB designed to assess speaking and listening proficiency in accordance which each component of the ICAO language proficiency rating scale?

Yes No N/A

2.2. 2.1.1.6 Does the test used by the LAB include also voice only interaction?

Yes No N/A

2.3. 2.1.1.6 Does the assessment also include face‐to face‐ communication?

Yes No N/A

2.4. 2.1.3 The final score for each test taker should not be the average or aggregate of the ratings of each of the six ICAO language proficiency skills but the lowest of these six ratings. Does the test used by the LAB guarantee that the overall score for the test taker is the lowest of the six scores?

Yes No N/A







8


TEST VALIDITY AND RELIABILITY Yes No N/A

3.1. 2.2.1 A statement of evidence for test validity and reliability should be accessible to all decision makers, in plain layperson language. Does the LAB supply documented evidence for the validity and reliability of their testing methods?

Yes No N/A

4. ICAO Circular 318‐AN/180 Chapter 3

RATING

4.1. 3.3.1 Some speaking and listening tests rate performance during the test. Others record the test performance later. Is the rating process used explained in the test explained in the test documentation?

Yes No N/A







9

4.2. 3.3.2. To fulfil licensing requirements, rating should be carried out by a minimum of two raters. Best practise in language proficiency assessment calls for at least two trained and calibrated raters, at least one if whom is a language expert. Does the language assessment body work with qualified language experts?

Yes No N/A

4.3. 3.3.3 Initial and recurrent rater training should be documented, the rater training records should be maintained and audits of raters should be conducted and documented periodically. Does the LAB provide such documentation?

Yes No N/A







10


TEST ADMINISTRATION AND SECURITY

5.1 4.1.5 Is the testing location moderately comfortable, private and quiet?

Yes No N/A

5.2. 4.2.2. A full description of security measures required to ensure the integrity of the testing process should be documented. Does the LAB provide such documentation?

Yes No N/A


RECORD KEEPING

6.1. 4.3.1. Proficiency test of speaking ability involving interaction between a test taker and interlocutor during the test should be recorded on audio or video media.

Yes No N/A

6.2. 4.3.2. Are the evaluation sheets and the supporting documentation filed for a predetermined and documented period of time of sufficient duration to ensure that rating decision can no longer be appealed?

Yes No N/A







11


ORGANIZATIONAL INFORMATION AND INFRASTRUCTURE

7.1. 5.5.2 If a language assessment body is also a training provider there should be a clear and documented separation between the two activities.

Yes No N/A


TESTING TEAM QUALIFICATIONS

8.1. 6.3.5. Interlocutors should have appropriate aviation operational or language testing expertise, or both.

Yes No N/A

8.2 6.4.1. Raters should demonstrate language proficiency of at least ICAO extended level 5 in the language to be tested. If the rest is assigned to assess ICAO Level 6 proficiency, raters should demonstrate language proficiency to ICAO Expert Level 6

Yes No N/A







12

8.3 6.4.2 Raters should be familiar with aviation English and with any vocabulary and structures that are likely to be elicited by test prompts and interaction

Yes No N/A

Observation Observed Dummy / Actual Test procedure! Was this presentation acceptable?

Yes No N/A

Part II TRAINING 1. ICAO Circular

323‐AN/180 Chapter 1

AVIATION ENGLISH TRAINING DESIGN AND DEVELOMENT

1.1. 1.1.1. Use of operationally relevant, work‐related language. Between the narrow focus of standardized phraseology and the very wide focus of conversational aviation topics there is the domain of plain language in an operational context. Does the training organization provide training material that requires dealing effectively with all the non‐standard, abnormal or emergency situations which pilots encounter and for which standardized phraseology is insufficient?

Yes No N/A







13

1.2. 1.1.2. Does the language training provided by the training centre adopt an essentially communicative approach to language learning with the main focus on speaking, listening and interactive skills?

Yes No N/A

1.3. 1.1.3. Does the language training provided by he training centre contain activities that are designed to address all six language skill areas specified in the ICAO Rating Scale and holistic descriptors?

Yes No N/A

1.4. 1.1.4. Does the training ensure that students achieve proficiency at least at ICAO level 4 in all six skill areas?

Yes No N/A







14


JUSTIFICATION OF TRAINING APPROACHES

2.1. 1.3.1. The Doc 9835 advocates the use of content‐based language learning in aviation English for reasons of learning efficiency, relevance of the subject matter and motivation. Does the training programme follow a logical progression as regards not only language level, but also familiarity with basic information before more complex information is introduced?

Yes No N/A

2.2. 1.3.4. The ICAO language Proficiency Requirements (LPRs) are about oral communication. Proficient speakers shall communicate effectively in voice only (telephone/ radiotelephone) and in face to face situations.

Communicate in common, concrete and work related topics with accuracy and clarity

Yes No N/A







15

Use communicative strategies to

exchange messages and to recognize and resolve misunderstandings in a general and work ‐ related context

Does the training centre provide

interactive listening comprehension

exercises, classroom information exchange, practise of vocabulary and grammar

through oral use; using graphic and numerical data (such as scopes, instrument panels and charts, tables and displays) ?







16

2.3. 1.3.6 Although pilot and controller communications are mainly voice‐only radiotelephony, flight crew also have to manage face to face situations such as

Calming unruly or distressed passengers,

Handling hijackers and terrorists, liaising with fire crew and emergency services.

Does the training programme provided by the training organization is also include some of these topics.

Yes No N/A

2.4. 1.3.7. Language learning today, increasingly depends on the use of technology. Does the training organization use CBT or WBT materials?

Yes No N/A

2.5. 1.3.9. Blended training refers to the combination of computer‐based and classroom learning. Does the training centre provide such training?

Yes No N/A







17

Observation Observed Classroom Training Was this presentation acceptable?

Yes No N/A


AVIATION ENGLISH TRAINING DELIVERY

3.1. 2.1.5. It is preferable that training materials are designed so that there are multiple ways in which they can be delivered. Does the training centre provide a modular design, alternative lesson plans and alternative applications?

Yes No N/A


TRAINERS

4.1 2.2.1. Do the trainers in the training organization have academic qualifications, aviation communications familiarity and teaching experience?

Yes No N/A

4.2. 2.2.3. Does the training centre offer in tandem training in case the trainer’s degree of aviation familiarity requires the presence of an operational expert?

Yes No N/A







18


TRAINNG MATERIALS

5.1.. 2.4.1. Does the training centre provide an instructor manual or any other means of information tool that that contains examples on how each type of exercise is to be delivered and answer keys if answers are not self evident for trainers?

Yes No N/A


MONITORING, TESTING, and REMEDIAL TRAINING

6.1. 2.5.1. Does the training centre provide tests at the end of each major phase in the training to assess student’s readiness for the proficiency test?

Yes No N/A

6.2. 2.5.4. Does the training organization use a Learning Management System Software Tool designed to manage student’s learning?

Yes No N/A







19


QUALITY ASSURANCE

7.1. 2.6.2. Does the training organization offer a feedback loop in order to ensure quality and appropriate results?

Yes No N/A


AVIATION ENGLISH TRAINER AND BACKGROUND

8.1. 3.8.2. Is there at least one trainer in the training organization who is familiar with the operational environment of aviation ( e.g. airspace management, airport ground movements, flight deck exchanges etc.) ?

Yes No N/A


CLASS MANAGEMENT TECHNIQUES

9.1. 3.9.1. Does the language training organization have training personal with experience in class management techniques?

Yes No N/A







20


AVIATION ENGLISH TRAINER TRAINING

10.1. 4.1. Does the training organization employ trainers with sufficient familiarity with the operational environment of pilots and controllers?

Yes No N/A

10.2. 4.2. Does the organization employ trainers that are qualified as language teachers that have acquired a sufficient knowledge of aviation operational environments, radio transmissions and technology to be able to use aviation materials effectively for the purpose of language training and/ or does the organization offer in tandem training (language teachers and aviation specialists)?

Yes No N/A

10.2. 4.3.7. No aviation language trainer training programme can be satisfactory without an extended period of actual teaching practice. Does at least one of the trainers employed in the training centre provide sufficient training experience?

Yes No N/A

AUSTRO CONTROL Language Assessment Body - Audit Questions Number Reference Questions Rating


Corrective incl. due Action (Date)

Sign Auditee Finding Codes

Sign Inspector Finding Codes


21

10.4. 4.4. Does the training organization employ both: highly qualified trainers with a linguistic background as well all aviation experts?

Yes No N/A

11. INFRASTRUCTURE

11.1 Classroom Yes No N/A

11.2 PC‐ Working Station Minimum one CBT Working Station

Yes No N/A

11.3 Office Equipment – such as: Fax PC Phone

Storage

Yes No N/A

Language Assessment Body is acceptable for certification reasons: Yes No Lead Auditor: Co Auditor:

LAB / Standardisation Check List – LSA/PEL (Authority) Meetings Seminars and... · 2014-05-28 ·...

Documents

Transcript of LAB / Standardisation Check List – LSA/PEL (Authority) Meetings Seminars and... · 2014-05-28 ·...