Human Factors and Aerospace Safety 4(3) 153-167© 2004, Ashgate Publishing

FORMAL PAPERS

Cabin crew expected safetybehaviours

Peter Simpson, Christina Owens* and Graham Edkins**Cathay Pacific Airways Ltd, Hong Kong* Qantas Airways Ltd, Australia** Victorian Department of Infrastructure, Australia

Abstract

Expected Safety Behaviours (ESBs) or non-technical skills are becoming anaccepted aspect of pilot training, however, the concept has not been widelyapplied to cabin crew. This situation is changing, as regulatory authorities start torequire the assessment of cabin crew CRM and safety behaviours. Qantas hasdeveloped a set of expected safety behaviours for the training and assessment ofcabin crew. Unlike previous sets of behavioural markers, the cabin markers weredeveloped using a valid scientific methodology. The ESBs were derived from 80interviews conducted with experienced cabin crew using the critical decisionmethod protocol, which uses probing questions to explore critical decision events.Seven ESB categories were derived; situation awareness, passenger management,crew management, workplace safety, operational understanding, negotiation andinfluencing skills, and information and resource management.

Introduction

Crew Resource Management (CRM) has been considered an integral aspect ofpilot training since the early 1980's, and of cabin crew training since the early1990's. It is only in the last few years that CRM skills and competencies havebegun to be assessed as a regular part of pilot check and training. The concept ofCRM competency assessment may be relatively new to the cockpit, but it is even

Correspondence: Christina Owens. Qantas Airways Ltd. QCC4, Bourke Rd, Mascot, NSW 2020AUSTRALIA, or e-mail: cowens@qantas.com.au

154 Peter Simpson. Christina Owens and Graham Edkins

newer in the cabin. This paper describes the method that Qantas has used todevelop CRM competencies (also known as non-technical skills or expectedsafety behaviours - ESBs) for cabin crew, and the process Qantas is developingfor their use and assessment.

Expected safety behaviours (ESBs) and CRM competencies

CRM and human factors training have been required by regulatory authorities(e.g., CASA, FAA, JAA) as part of pilot licensing for several years. Cabin crew inAustralia do not require licensing, but most have been exposed to CRM andhuman factors training via company training programs. However, until now, theAustralian Civil Aviation Safety Authority (CASA) had not required thedevelopment or assessment of CRM competencies (ESBs) for cockpit or cabincrew; Civil Aviation Safety Regulation 121A will require this. CRM is finallystarting to be recognised as being as important as technical skills for the safe andefficient management of a flight.

Expected safety behaviours (ESBs) are generally thought of as observable, non­technical behaviours that contribute to effective performance within a specificwork environment. They are usually structured into categories, and the categoriesare constructed of several elements or behaviours. A behavioural marker should

describe a specific, observable behaviour, not an attitude or personality trait.These elements or behavioural markers can enable CRM performancemeasurement and assessment, and can also be used to build performancedatabases to identify norms and prioritise training needs, including the evaluationof training (Klampfer et aI., 2001).

In 2001, Qantas started a two-year research project to develop ESBs for cabincrew. Like other airlines, Qantas already had some experience in observing andscoring pilot CRM for check and training purposes, although this was not jeopardyassessment. Methodological guidelines for the development of behaviouralmarkers and CRM competencies are lacking. Programs such as NOTECHS (vanAvermeate and Krujisen, 1998; Flin et aI., 2003), LOSA (HeImreich, Klinect andWilhelm, in press), and GIHRE (Klampfer et aI., 2001) have developed their listsof pilot CRM skills and behaviours from training workshops and meetings, andthey are based on opinion and experience, using a variety of informal methods andtechniques, such as amalgamating existing marker systems. A number of airlineshave developed their own behavioural markers systems based on these programs(Flin and Martin, 2001), or using their methods and processes (e.g., Qantas).

Furthermore, there does not appear to be any published research into thedevelopment of behavioural markers for cabin crew, despite cabin crew CRMtraining being mandated in many countries. Numerous investigation reportsreinforce the need for cabin crew to take appropriate action to manage threats anderrors (for example, Air Ontario at Dryden, American Airlines AA132 atNashville, British Air Tours runway fire at Manchester, Qantas QFl at Bangkok,

Cabin crew expected safety behaviours 155

Singapore Airlines SQ6 at Taipei). Handling many of these threats and errors callsfor knowledge, skills and abilities quite different from those associated with theprovision of normal service duties, yet there is no consensus on which skills areneeded for effective cabin crew CRM (lCAO, 2002).

The cabin crew ESB project aimed to develop a valid and reliable method ofidentifying and developing ESBs, based on established human factors andbehavioural science methods, the primary method being the Critical DecisionMethod (CDM) (Klein, Calderwood and McGregor, 1989). The CDM is aretrospective interview strategy that applies a set of cognitive probes (see table 1)to non-routine incidents that requires expert judgement or decision making (Kleinet aI., 1989). A semi-structured interview format is used to probe different aspectsof the decision process and situation assessment. The CDM is a variant ofFlanagan's (1954) critical incident technique. The critical incident technique canbe applied for a variety of situations and uses in aviation, including: developingcritical requirements for evaluating the typical performance and safety behaviourof an operator, evaluating operator proficiency in a check and training situation,and providing an indication of the effectiveness of training programs (Flanagan,1954). These are three areas that the ESB project will cover at Qantas, and allthree areas are useful in meeting the upcoming Australian regulatoryrequirements.

The CDM is effective in revealing experts' knowledge, especially tacitknowledge, reasoning and decision strategies. Compared to other methods ofknowledge elicitation, the CDM yields more information, including a wider varietyof specific cognitive details, more information about underlying causal linkagesamong core subjects, and the revelation of tacit knowledge (Hoffman, Shadbolt,Burton and Klein, 1995). The reliability of the procedure is based on the idea thatexperts have clear memories of salient or unusual safety-related incidents (Hoffmanet aI., 1995). This method of knowledge elicitation has been used successfully innaturalistic environments such as fire fighting, para-medicine, nursing, helicopterflying, and military command and control (see Simpson, 2001 for a review). TheCDM is now established as a valid and reliable method of cognitive task analysisand knowledge elicitation (Hoffman, Crandall and Shadbolt, 1998; Taynor,Crandall and Wiggins, 1987). However, the limitations of this method are discussedby O'Hare, Wiggins, Batt and Morrison (1994), and Hamm (2004).

Method for developing the ESBs

Participants

The participants for the critical decision interviews consisted of eighty CustomerService Managers (CSMs) (known as Pursers or Cabin Supervisors in someairlines). The CSMs were a mixture of short haul (54%) and long haul (46%) from

156 Peter Simpsoll. Christilla Owells alld Graham Edkills

various bases across Australia. They had spent an average of 7.5 years operatingas a CSM (range = 1-25yrs, sd = 6.2yrs), and their mean age was 42 years (sd =7.0yrs).

The criteria for participation were that the person must be a CSM, and that theycould discuss, in detail, a recent (within 18 months) safety-related event that waschallenging. CSMs were paid for the ground duty time they spent being interviewed(usually 1.5 - 2 hours). Six people, who were also CSMs, conducted the interviews.Although their main role was interviewing, these six team members also helpedwith data analysis and development of the safety behaviours because they aresubject matter experts, with the domain knowledge area of cabin safety.

Desigll

The variables of home base location (Sydney, Melbourne, Brisbane, Perth) andoperation type (short or long haul) were controlled to ensure the CSMsproportionally represented the Qantas operation. The interview process was basedon the Critical Decision Method (CDM) described in the Introduction.

Procedure (stage J) - critical decisioll method (CDM) The procedure foremploying CDM is well documented (Klein et aI., 1989; Hoffman et aI., 1998),but the basic steps used in the interviewing process included:

1. Incident selection - CSMs select a recent, non-routine incident that waschallenging;

2. The interviewer obtains an unstructured free recall of the event;

3. Both participant and the interviewer establishes the sequence of decisionevents and constructs a time line;

4. Decision point identification - the interviewer identifies specific decisionsthat were made;

5. Decision point probing - probing/questioning techniques are used to identifyeffective decisions and resultant behaviours (see Table 1 for examples of theprobes);

6. Hypotheticals and 'what ifs?' The interviewer chooses several decisionpoints, and asks hypothetical questions based upon different event outcomes(eg., what would you do if the Captain didn't take your request seriously?).These queries serve to identify potential errors, alternative decision-actionpaths, and expert/novice differences (Hoffman et aI., 1998); and

7. Standard case study - CSMs are provided with a standard case studyl andrepeat steps 3 to 6 as if they were the CSM onboard that aircraft.

I The standard case study was a real Qantas incident on-board a B737. It involved smpke andfumes in the cabin. with many CRM issues and problems.

Cabin crew expected safety behaviours 157

Table 1 A sample of CDM probe questions (adapted from Hoffman et aI.,1998)

Probe typeCues

Knowledge

Analogues

Standardscenarios

Goals

Options

Mentalmodels

Experience

Decision

making

Aiding

Errors

Probe content

What were you seeing, hearing, smelling?

What info did you use in decision making and how was itobtained?

Were you reminded of any previous experience?

Does this case reflect a typical scenario or a scenario youwere trained in?

What were your specific goals and objectives at the time?

What other courses of action were considered or wereavailable?

Did you imagine the possible consequences of your action?

What specific training, experience or knowledge wasnecessary?

Was there time pressure? How long did it take to make thedecision?

What training, knowledge or experience could have helpedyou?

What mistakes are common at this point? How might anovice act?

Participants were informed that the interviews were anonymous. Interviewswere conducted in a quiet office and tape-recorded (if permission was granted).Most interviews lasted for 1.5-2 hours; requiring at least one hour for the firstcritical incident, and half to one hour for the repeated case-study incident.Verbatim transcripts were made from the tapes.

Procedure (stage 2) - development of expected safety behaviours (ESBs) Thesecond stage of the study involved the coding of the interviews and developmentof the ESBs. The procedure consisted of a number of steps:

1. Initial review and coding of the transcripts to develop behavioural markers.This was conducted by the research team in a workshop situation. Teninterviews were used to develop the initial ESB list. This critical step isexpanded in Appendix A.

158 Peter Simpson, Christina Owens and Graham Edkins

2. Improve code structure and markers with feedback from cabin crew subjectmatter experts (the six CSMs within the project team).

3. Test the improved ESB list by coding a further 10 interviews for refinement.The aim of this step was to determine whether the list of ESBs was able tocover a multitude of cabin situations that required both technical and non­technical skills.

4. Analysis of behavioural markers by general CSM population, includingwritten feedback requested from the entire CSM population to gain generalcomment and feedback on the list of ESBs, and brief interviews (20 mins)with random CSMs (n=20) in crew lounges to attain more detailedresponses. Both methods included rating the relative importance of eachbehavioural marker to the safe, efficient conduct of the flight.

5. Produce a master list of ESBs.

6. Code remainder of interviews (approximately 60).7. Attain a frequency count of behavioural marker elements occurring in each

critical incident.

8. Construct a 'training focus matrix' (frequency ESB occurring versusperceived importance ESB) for the training of expected safety behaviours(see figure I). This is based upon the results from steps 4 and 7 and allowsobjective prioritisation of training.

High

IM

PoR

T

AN

CE

Low

Low

Priority

High FREQUENCY Low

Figure 1 Expected safety behaviour training focus matrix

Achieving a high degree of inter-rater reliability is one of the challenges ofcoding transcripts. To avoid this problem completely, once this final ESB codingstructure was developed, a single person coded all the remaining interviews.

Cabin crew expected safety behaviours 159

Results and discussion

Critical decision method

The majority of critical incidents recalled by CSMs could be categorised underseven headings (see figure 2). Almost half of the. incidents were related todisruptive or drunk passengers (20%), in-flight medical emergencies (17%), orsecurity/terrorism threats (12%). The large number of security/terrorism threatshas only occurred since Sept I I th terrorist attack. It is interesting to note that manysafety issues involve the aircraft door and slides (10%). Aircraft technical issues(8%) refer to problems such as aborted take-offs, engine problems, and cockpitand cabin equipment malfunctions. The break down of all Qantas reported cabinsafety incidents for the 13 month period Jan 200 I to Feb 2002 is proportionallysimilar to those of this study (Qantas, 2002). This validates that the ESBs weredeveloped from critical incidents that reflected the real cabin incidents and eventsduring that time period.

smoke/fumes 9% / / security/terrorism 12%door/slide issue 10%

turbulence 4%

aircraft technical is----

drunk/disruptivepax 20%

medical 17%

Figure 2 Type of critical incidents recalled in the CSM interviews

A final list of seven ESBs was derived (see table 2). The behavioural markers(or elements) are the CRM competencies that can be used for training andchecking, and they can also be used to satisfy Australian regulatory requirements.The ESB list is similar to other existing, 'workshop-derived' lists. However, thisresearch appears to be the only published work on the topic, and it now validatessome of the CRM competencies that are being trained. It should be stressed thatthe ESBs are specifically for Qantas international and domestic cabin crew, andthey may not necessarily transfer to other airlines or cultures withoutmodification. Other Australian carriers have started to modify the Qantas ESBs tosuit their own operations.

160 Peter Simpson, Christina Dwens and Graham Edkins

Table 2 Expected safety behaviour categories and elements

Expected safetybehaviour categorySituationawareness

Passengermanagement

Workplace safety

Crew

management

Operationalunderstanding

Negotiation andinfluencing skills

Information andresource

management

Expected safety behaviour element/behaviour

• Demonstrates awareness of flight phase or situation

• Considers political and cultural context• Considers time constraint

• Recognises higher safety goals and priorities• Anticipates decision consequences

• Develops contingency plans

• Assesses passengers (boarding or in-flight)

• Monitors potentially threatening pax behaviour/condition

• Acts decisively to modifY passengerbehaviours/condition

• Considers passengers well-being• Presents a calm, controlled image to passengers

• Diffuses situation in a non-confronting manner• Minimises cabin disruption

• Pro actively manages OHandS situations

• Reactively manages OHandS situations• Follows-up OHandS situations

• Communicates OH andS importance• Displays role model OHandS behaviours

• Assesses crew

• Provides onboard coaching and training to modifY behaviour• Considers crew well-being

• Considers impact of non-routine events on crew performance• Allows and provides crew debrief

• Demonstrates Basic Aeronautical Knowledge (BAK)• Understands authority/duty of CSM

• Understands authority/duty of others

• Consults with others to develop a common strategy• Manages upwards 1 - identifies problem• Manages upwards 2 - expresses concern

• Manages upwards 3 - provides options• Manages upwards 4 - uses emergency language

• Identifies and utilises all resources

• Gathers information

• Confirms common understanding of information• Critically analyses information• Provides timely feedback to those who need to know• Prioritises tasks

Cabin crew expected safety behaviours 161

As a learning point from the study, other airlines would be advised to carefullyselect their interviewers. Although senior cabin supervisors are experienced incabin safety, they are not necessarily experienced in interviewing. Severalinterviews were not as detailed as they could have been, due to a focus on physicalevents rather than decisions, and many were 'war stories' rather than cognitiveanalysis. However, all the interviews were useful and have proven highly valuableas examples and support for training development.

The frequency of ESB elements was summed for the 80 critical incidents,yielding over 1,500 discrete occurrences of the elements. This allowed thedetermination of the safety behaviours most frequently used by CSMs in normaloperational events and incidents. Figure 3 shows a selection of the frequencyresults. The category of Information and Resource Management was the mostfrequently occurring safety behaviour. It is thought that the high frequency of thisbehaviour is unique to CSMs, because they are the cabin manager, and areresponsible for the control of information and resources within the cabin, andbetween the cockpit and cabin. The CSM is also the cabin member most oftenmaking decisions and solving problems for all the cabin crew.

The second most frequently displayed safety behaviour was the category ofSituation Awareness. Again, as the cabin manager, it is not surprising that theCSMs behaviour and decisions were very closely related with aspects of beingsituationally aware. All three levels or stages of situation awareness - perception,comprehension, and projection (Endsley, 1995) - were represented in the safetybehaviour elements. Projection was evident when the CSMs determined theconsequences of their decisions before implementing them, and their developmentof contingency plans in case situations and circumstances changed.

With the frequency count of the ESBs, and the rating of perceived importance bythe CSM population, Qantas now has information on what the most commonlydisplayed cabin safety behaviours were, as well as which were perceived to be themost important. It is interesting to note that the most frequent category, Informationand Resource Management, was considered to be the least important category.Situation Awareness was ranked highly both in frequency and perceived importance.This indicates that what crew perceive to be important safety behaviours, may notreflect reality. This may have an impact on the validity of behavioural marker listsderived from workshop and 'round-table' discussions that are based on experienceand perception, rather than valid and reliable scientific methodology.

Application of the ESBs

Use in training

Qantas now has 80 real, recent, and highly detailed safety events that can be usedas case studies and examples. They all detail actual threats encountered and errors

162 Peter SimpSOll, Clzristilla QwellS alld Gralzam Edkins

made on Qantas aircraft, and most are very good examples of threat and errormanagement. This is an excellent resource the company can use for many years.

CREW debrief

CREW impact

CREW wellbe.;ng F:"~"";'CREW coaching' ,CREW assess ,-' ." ,'"

PAX disruption

PAX diffuse f.:.:.:~PAXcalm f .-~

PAX wellbe;ng ~

::~~~:~r:+~--::~:~~t~:~~::f,:::t;:;:::.J.

~~~i.~;:~:~~~l:::'E:d~=:~SA time, ~:=-~"~:.',m'-~.· !SA political ~ .,~:. . ~_ I

;i~~~~~el r--'''' ",.".,..-=,~=.".~-.".....,.,..=~.._.="'"'~'-~'=====~

:~~~~~~o~~:~kr~::~':':,:~.:.,:::::.~.-~,.._-=.JINFO analyseINFO common

INFO gatherINFO resources

I

;::

I.~"".,.~"""'.""'""

o 20 40 60 80 100 120 140

frequency

Figure 3 Frequency count of selected expected safety behaviours

The ESBs are currently being used in cabin crew training as part of the annualCRM/Emergency Procedures refresher training (a two-day course that all cabin

Cabin crew expected safety behaviours 163

crew undergo annually). During a classroom session, cabin crew listened to threeof the original incident summaries that had been recorded onto audio file. Eachwas a few minutes long, and was recorded in the first person, as if the voice on therecording was that of the CSM recalling his/her own safety event. At the end ofeach incident summary, the cabin crew were asked to assess a specific category ofCRM from the ESB list (e.g., 'assess the passenger management competencies').A mock-up CRM assessment form was used along with the ESBs in table I. Thepurpose of this exercise was to introduce cabin crew to the concept of CRMassessment, and to give them some experience of the forms and categories thatmay be used to train and assess them in the future. Unlike pilots, CRM assessmentand CRM competencies are a new concept for cabin crew.

Currently the ESBs are not being used for assessment purposes in the cabin.Although, this situation will change in the next few years. It is expected thatinitially, the ESBs will be used for training and checking, but not necessarily forjeopardy assessment. This type of assessment will not occur until a level ofacceptance amongst crew and checker/trainer skill has been attained in assessingCRM competencies.

Development of ESB application guidelines

Broad guidelines for the appropriate application and use of ESBs and theirassessment are being developed across the Qantas group (i.e., international,domestic and regional operations) for both cockpit and cabin crew. Theseguidelines are based on the NOTECHS guidelines (van Avermaete and Krujisen,1998) and have been adapted to suit the operational and regulatory requirementsfaced by Qantas. The guidelines have been developed in conjunction withstakeholders (mainly Human Factors representatives) from the various flight andcabin operations areas of the Qantas group. The guidelines describe such issues as;

1. Introduction and definitions of ESB (e.g., categories, elements, behaviours).2. Philosophy behind the use of the ESBs (e.g., 'to train, reinforce and support

the skills and behaviours considered necessary to provide safe, effectiveoutcomes').

3. Preferred methods of assessment (e.g., 'Ratings should be done at the'category' level. An overall rating is given for the category, rather than forindividual components, but that category rating is based upon theperformance outcomes of the elementslbehaviours. ').

4. Process for failed competencies (e.g., Ineffective behaviour should beobserved repeatedly to conclude that it is ineffective. It is not the goal to failsomeone for failing to discuss a decision on a single occasion ... ').

5. Recommended evaluator training and qualifications (e.g., 'Complete initialtraining on ESB system, Formal assessment as competent and calibratedfollowing ESB training in classroom, Calibration in operationalenvironment, and Periodic re-calibration').

164 Peter Simpson, Christina Owens and Graham Edkins

Cabin crew LOSA

While behavioural marker systems can provide a focus for training goals andneeds, they are limited in that they cannot capture every aspect of normalperformance because of the infrequent occurrence of some behaviours (e.g.,sporadic behaviours such as aggression), especially during an assessmentsituation, such as a check flight. Hence, the University of Texas has developed amethod of normal operations, non-jeopardy, Line Operations Safety Audits(LOSA) in the cockpit. LOSA enable the collection of data on threats and errorsand their management (Helmreich, Klinect and Wilhelm, in press).

LOSA utilises trained observers to collect data about flight crew behaviour onnormal flights under non-jeopardy conditions. Observers record information basedon three worksheets. The first worksheet records a description of external threats(e.g., weather or ATC) that may influence crew performance and how thesethreats are managed. The second worksheet describes the errors made by the flightcrew and what strategies were used to detect and recover from these errors. Thefinal worksheet records the behavioural markers based on three categories;Planning, Execution and Review/Modify Plans. The behavioural markers are usedrepeatedly for every flight phase and rated on a four-point scale (poor ­outstanding) (Helmreich, Klinect and Wilhelm, in press).

LOSA is currently used only in cockpit situations. In the longer term, the cabincrew ESBs can be used in a similar manner to the way in which they are used forcockpit crew; the evaluation of CRM and non-technical skills in training andnormal operations. This could feedback into the training system, allowing therefinement and development of ESBs and the training program. The greatestchallenge is moving the evaluation and observation of safety behaviours out of thetraining environment and into normal line operations, in the form of a cabin crewLOSA program. There are many issues and problems to overcome before in-flightcabin observations can take place. Such problems include:

• Cabin environment is not as contained as a cockpit.• Double deck aircraft (B747 and A380).• All information goes through CSM.• Observers are more obtrusive in cabin.

• Errors tend to be less consequential in the cabin.• Impact on customers and service.• Multiple crew to observe, who cannot all be observed at once.• Less external threats to inflight safety in the cabin.Check and training for technical skills has been an accepted practice for

decades. Further, CRM and non-technical skills audit and evaluation is gainingacceptance in the cockpit, and a cockpit LOSA program is running at Qantas. Asyet, no airline has committed to a LOSA-style program for cabin crew. Qantas isconsidering incorporating LOSA style threat and error management within its

Cabin crew expected safety behaviours 165

inflight cabin observation program, which is currently being developed and duefor implementation mid-2004. The observation protocol has been developed inline with general LOSA principles, as far as possible taking into account the issuesabove. The protocol is currently under review, and the final version should beavailable January 2004. To our knowledge, the proposed Qantas project is the firstattempt to apply this program within the cabin environment.

Summary

As far as is known, the Qantas Cabin Crew ESB study is the first attempt todevelop behavioural markers in aviation using a robust, scientific method. TheCDM was a practical method for this purpose, as has been demonstrated in othernaturalistic environments. It is hoped that other authors will use similar, formalmethods for developing behavioural markers, rather than using the traditional,informal process of workshops and round-table discussions. The behaviouralmarkers developed from these informal processes are based on experience,opinion and perception, and as this current study indicated, the behaviours thatexperienced operators perceive to be important to safety may not reflectoperational reality. Hence, any of the current lists of safety behaviours andbehavioural markers being used in aviation and other industries require scientificvalidation.

The study has also provided a useful and extensive database of positive­outcome safety scenarios for use in cabin crew training. The case studies arespecific to the operating environments of Qantas, and are a good resource for thecurrent CRM focus of threat and error management. For these reasons, it is highlyrecommended that other airlines implement a similar program.

The LOSA program has proven successful in the cockpit, and its use in thecabin and other environments (e.g., ATC) appears to be a good concept in theory,yet logistical issues and other challenges continue to impede its implementation.No conclusion can be drawn from this current study regarding the use of the cabincrew LOSA. However, some form of non-jeopardy, normal operations observationis required to further understand the nature of threat and error management, andeffective safety behaviours in the cabin. Implementing such a program is now amajor challenge facing the aviation industry.

References

van Amermaete, J. and Krujisen, E. (Eds) (1998). NOTECHS: The evaluation ofnon-technical skills ofmulti-pilot aircrew in relation to JAR-FCL requirements.National Aerospace Laboratory NLR.

166 Peter Simpson. Christina Owens and Graham Edkins

Endsley, M. (1995). Towards a theory of SA in dynamic systems. Human Factors,37, 32-64.

Flanagan, J. (1954). The critical incident technique. Psychological Bulletin, 51,327-358.

Flin, R. and Martin, L. (2001). Behavioural markers for Crew ResourceManagement: A survey of current practice. International Journal of AviationPsychology, 11,95-118.

Flin, R., Martin, L., Goeters, K., Hormann, H., Amalberti, R., Valot, C. andNijhuis, H. (2003). Development of the NOTECHS (non-technical skills) forassessing pilots' CRM skills. Human Factors and Aerospace Safety, 3,97-119.

Hamm, R. (2004). Protocol analysis: The emperor needs a shave. MedicalDecision Making, 24, 681-686.

Helmreich, R., Klinect, J. and Wilhelm, J. (in press). System safety and threat anderror management: The Line Operations Safety Audit (LOSA). In, Proceedingsof the Eleventh International Symposium in Aviation Psychology. Columbus,OH: The Ohio State University.

Hoffman, R., Crandall, B. and Shadbolt, N. (1998). Use of the CDM to elicitexpert knowledge: A case study in the methodology of cognitive task analysis.Human Factors, 40,254-276.

Hoffman, R., Shadbolt, N., Burton, A. and Klein, G. (1995). Eliciting knowledgefrom experts: A methodological analysis. Organisational Behaviour andHuman Decision Processes. 62,129-158.

International Civil Aviation Organisation (ICAO). (2002). Draft Human factors

digest: Humanfactors in cabin safety. Montreal,9anada.Klampfer, et al. (2001). Behavioural markers workshop. Swissair Training Centre,

Zurich, 5-6 July.Klein, G., Calderwood, R. and McGregor, D. (1989). Critical decision method for

eliciting knowledge. IEEE Transactions on Systems, Man and Cybernetics, 19,462-467.

O'Hare, D., Wiggins, M., Batt, R. and Morrison, D. (1994) Cognitive failureanalysis for aircraft accident investigation. Ergonomics. 37, 1855-1869.

Qantas (2002). Safety analysis paper: Cabin safety incidents - causal factors (SAP06/1). Sydney, Australia: Qantas, Corporate Safety Dept.

Simpson, P. (2001). Naturalistic decision making in aviation environments.DSTO-GD-0279. Melbourne, Australia: DSTO, AMRL.

Taynor, J., Crandall, B. and Wiggins, S. (1987). The reliability of the criticaldecision method. KA-TR-863(b)-8707F. Yellow Springs, OH: Klein Associates.

Cabin crew expected safety behaviours 167

Appendix A

A list of safety behaviours and non-technical skills could be very lengthy.Developing the list is a compromise between having a practical, useable list foroperational purposes (training, assessing, checking, auditing, etc) and having along, exhaustive list to cover every behaviour and situation. The goal was toaccount for the majority of safety-related behaviours, not every individualbehaviour.

Despite the body of literature on verbal protocol analysis and CDM, there isvery little information explaining the practical aspects of coding the transcripts.The procedure developed for this project used a team of two psychologists, ahuman factors specialist and a cabin safety specialist to develop the ESB codes.The process was:

1. Ten interviews were selected that covered ten different situations.

2. All team members read each transcript, focussing on the decision points.The actions and behaviours surrounding the decisions were noted, and anattempt made to attach a recognised human factors / psychological term todescribe the behaviour.

3. After each transcript was worked on in isolation, the team met to discusstheir ideas and findings. As a group, they developed relevant descriptors forthe behavioural elements.

4. This process of coding in isolation then meeting together as a team occurredfor each interview transcript until the initial 10 transcripts were complete.

5. 'Diminishing returns' occurred, where the initial interviews yielded thelargest array of behaviours and elements, and by the tenth transcript, thecodes were being refined rather than new behaviours being uncovered.