Human Factors in Risk-Based Ship Design Methodology · 7th Framework Programme European Commission...

7th

Framework Programme European Commission

Human Factors in Risk-Based Ship Design Methodology

Project no 314817 01/10/12 – 01/10/15

Title: Results of physical experiments Deliverable n. 7.2 Task: 7.2 Results of experiments in bridge simulator WP: 7 Responsible: UoS – Stephen Butler Due delivery date: 2015-08-31 Actual delivery date: 2015-09-23 Dissemination level1: PU

1 PU = Public

PP = Restricted to other programme participants (including the Commission Services) RE = Restricted to a group specified by the consortium (including the Commission Services) CO = Confidential, only for members of the consortium (including the Commission Services)

Deliverable n. 7.2

2/44

Disclaimer The information contained in this report is subject to change without notice and should not be construed as a commitment by any members of the FAROS Consortium. In the event of any software or algorithms being described in this report, the FAROS Consortium assumes no responsibility for the use or inability to use any of its software or algorithms. The information is provided without any warranty of any kind and the FAROS Consortium expressly disclaims all implied warranties, including but not limited to the implied warranties of merchantability and fitness for a particular use.

COPYRIGHT 20012-2015 The FAROS Consortium. This document may not be copied, reproduced, or modified in whole or in part for any purpose without written permission from the FAROS Consortium. In addition, to such written permission to copy, acknowledgement of the authors of the document and all applicable portions of the copyright notice must be clearly referenced. All rights reserved.

Deliverable n. 7.2

3/44

Abstract Four naturalistic experimental bridge scenarios were designed based on each of a 200 m RoPax and on a 354 m Tanker. The aim was to investigate the influence of noise and task difficulty on navigation task performance, as measured by the passing distances from selected target vessels within the sea area, deviations from assigned track, and ratings by instructors of the timeliness and accuracy with which required actions were carried out. The scenarios varied in difficulty in that whilst each one involved out-of-course events occurring during the scenario, some required more actions to be taken by the mariner than others. The noise was the sound of a needle gun being used to remove rust from above the wheelhouse, and was accordingly varied rather than of constant intensity. The results indicated that there were no significant effects of noise across the scenarios in terms of navigation performance as expressed by either passing distances to target vessels or track deviation, except in the earliest trials within the sequence of tasks. However, there were some marginally significant effects of both task difficulty and noise on some of the instructors’ ratings of the mariners’ actions within the tasks.

Deliverable n. 7.2

4/44

Document Meta Data Author/s: Anthony Anderson, Madeleine Grealy, James Thomson, Stephen Butler,

Knud Benedict, Matthias Linnenbecker, Gerrit Tuschling

In-house reviewers (optional):

Reviewer 1:

Reviewer 2:

Other reviewers:

Nature of Deliverable : Report Prototype Demonstrator Other

Related FAROS Deliverables:

Partners involved No. Organisation

short name Organisation full name

Name email

5 UoS University of Strathclyde Stephen Butler [email protected]

6 HSW Hochschule Wismar – University of Applied Sciences: Technology, Business and Design

Knud Benedict [email protected]

3 DBL Deep Blue SRL Luca Save [email protected]

9 TLG Tallink Grupp AS Tanel Hinno [email protected]

8 AMC Alfa Marin Technikh Symvouleytikh Meleton Kai Ergon Epe

Philip Tscichlis [email protected]

11 NAP Naval Architecture Progress

George Pratikakis [email protected]

Document history Version Date of

delivery Changes Author(s)

Editor(s) Reviewed by

001 27.08.15 Basic document Gerrit Tuschling

002 31.08.15 Tables transformed into figures, summary and conclusions

Anthony Anderson

003 31.08.15 Bibliography and analysis Gerrit Tuschling

004 31.08.2015 Lay summary added Anthony Anderson

005 03.09.2015 Additional results added Anthony Anderson

006 04.09.2015 Summary amended Gerrit Tuschling

007 22.09.2015 Statistical values inserted Anthony Anderson

008 23.09.2015 Layout adjusted Gerrit Tuschling

Deliverable n. 7.2

5/44

1 EXECUTIVE SUMMARY ..................................................................................................... 6

1.1 PROBLEM DEFINITION ...................................................................................................... 6

1.2 TECHNICAL APPROACH .................................................................................................... 6

1.3 RESULTS AND ACHIEVEMENTS ....................................................................................... 6

1.4 CONTRIBUTION TO FAROS OBJECTIVES ........................................................................ 7

2 INTRODUCTION ................................................................................................................. 8

3 GENERAL METHODOLOGICAL APPROACH AND PARTICIPANTS ............................... 9

3.1 SCENARIO DESCRIPTION ................................................................................................. 9

3.2 DETAILED SCENARIO DESCRIPTION ..............................................................................13

3.3 MONITORING AND RECORDING OF SIMULATION TRIALS ............................................18

4 RESULTS ...........................................................................................................................24

4.1 ANALYSIS OF VESSEL NAVIGATION DATA ....................................................................24

4.2 ANALYSIS OF INSTRUCTOR RATINGS ...........................................................................27

5 SUMMARY AND CONCLUSIONS......................................................................................35

6 BIBLIOGRAPHY AND REFERENCES ...............................................................................39

7 INDEXES ............................................................................................................................40

7.1 INDEX OF TABLES ............................................................................................................40

7.2 INDEX OF FIGURES ..........................................................................................................40

7.3 ABBREVIATIONS ...............................................................................................................41

8 ANNEXES ..........................................................................................................................42

8.1 PUBLIC SUMMARY ............................................................................................................42

Deliverable n. 7.2

6/44

1 Executive Summary

1.1 Problem Definition

Maritime navigators have to be able to safely and efficiently process multiple streams of information via instrument screens and windows, verbal communications and auditory alarms. Attention, memory and response efficiency may be affected by global design aspects of the vessel. If these result in increased error and deteriorating performance, this will ultimately reduce societal safety. Research has suggested that global design factors (GDFs) including noise may have an adverse effect on attention and performance [1] [2]. Noise has also been found to increase fatigue and fatigue has been associated with increased human error [3]. Other laboratory-based research has confirmed the adverse effects of noise on the performance of various types of task, categorised broadly into perceptual, cognitive, motor and communication [4]. The adverse effects of noise have been confirmed in previous research using a vigilance task within a simulated voyage [5], although measures of navigation task performance were not reported. The primary objective of the present experiments was to assess the effect of noise on human performance in a simulated navigation task, as measured by the passing distance to a target ship or grounding risk, deviation from the required track, and reaction to on-board alarms within High Risk Events occurring during a simulated voyage.

1.2 Technical Approach

The experiments were conducted at the Maritime Simulation Centre Warnemünde (MSCW), part of the Warnemünde Department of Maritime Studies of the Hochschule Wismar, University of Applied Sciences: Technology, Business and Design. Four RoPax scenarios were conducted on the ferry “Mecklenburg-Vorpommern” and four tanker scenarios were conducted with the VLCC “Lagena” (as described in D3.1 and D4.2). Following an initial design and consultation phase from September 2014 to January 2015, the testing schedule ran from late February 2015 until May 2015. The span of the testing was due to the availability of the testing centre since it is heavily used as a training facility for much of the year. Analysis was then conducted from June 2015 to August 2015. Data from the simulator recordings on proximity to collision threats and course navigation was computed on SIMDAT software by the ISSIMS Gmbh2 [6]. Thirty-six experienced mariners were recruited to participate in four naturalistic bridge simulated scenarios on the basis of a pre-experiment screening questionnaire. Independent variables comprised two levels of noise and two levels of task difficulty.

1.3 Results and Achievements

The measures of navigation accuracy showed no significant effect of noise or of task difficulty: the mariners succeeded in sailing their voyages with similar distances to target vessels, grounding risks, and similar track deviations in both noisy and quieter conditions. However, in terms of the instructor ratings of the mariners’ task performance, there was a marginally significant drop in overall rated performance as a function of

2 www.issims-gmbh.com

Deliverable n. 7.2

7/44

noise, and several instances of significant or near-significant effects of noise on specific mariner actions within the scenarios, such as (for example) the time taken to reduce engine RPM within Scenario A (in which a High Risk event of a steering gear failure had been included). It should of course be pointed out that these detailed analyses involve making a large number of comparisons and when many statistical analyses are undertaken, a more stringent level of significance should be adopted. Considerable caution is therefore required in claiming the significance of any effects found. However, bearing in mind that the number of mariners being tested could ideally have been larger for the sake of statistical power, and the limitations imposed by the short duration of the scenarios, the fact that some significant differences are occurring, even marginal ones, should counsel caution against dismissing potential adverse effects of noise on mariner performance.

1.4 Contribution to FAROS objectives

As a result of the physical experiments, knowledge has been gained on the potential effect that noise as a global design factor might have in terms of slightly compromising mariners’ execution of tasks related to unexpected High Risk events taking place. Although many of the effects were non-significant, the tendency for the marginally significant ones to appear more often in the case of the difficult tasks might suggest the existence of a noise by task difficulty interaction in which the combination of noise and a particularly difficult situation for the mariner to deal with could result in some deterioration of performance. Whilst it is emphasised that the effects found were of marginal statistical significance, they do suggest that further investigation in future to tease out the precise circumstances in which effects are to be found would be a useful next step for research to take. Conducting the physical experiments can supplement knowledge gained through the development of risk models involving noise and motion by providing information on the effect in as naturalistic environment as testing of this nature would allow.

Deliverable n. 7.2

8/44

2 Introduction Following the outcomes of the physical experiments in Work Package 4 (see Deliverable D 4.4), it was decided to focus within WP7 and task 7.2 on one Global Design Factor, namely noise, because among all the Global Design Factors (GDFs), this could be most realistically simulated within the context of the bridge simulator. (Since the floor of the bridge simulator is fixed, the simulation of ship motion is limited to the visual modality rather than the visual and proprioceptive/kinaesthetic modalities that would be stimulated on real vessels; and likewise the simulation of vibration arising from vessel engine activity could not be realistically simulated within the bridge simulator). Noise was therefore selected as the Global Design Factor for investigation within Task 7.2. There is a considerable body of psychological literature attesting to the fact that noise disrupts the performance of various laboratory tasks, as reviewed by Szalma and Hancock in their 2011 meta-analysis of 242 studies [4]. Their analysis indicates that the effects of noise are not simple, and are moderated by the nature of the noise (e.g. continuous or intermittent), the type of noise (e.g. irrelevant speech versus non-speech) and the type of task (perceptual, cognitive, motor and communication). Their review suggested, for example, that whilst noise has no effect on visual perception tasks it has a moderately strong deleterious effect on cognitive and psychomotor tasks. Furthermore, when adverse effects of noise were evident, these tended to be on accuracy rather than on speed of performance. Furthermore, in some cases noise has a facilitative effect, whereas in others it has a debilitative effect. Szalma and Hancock explain these different effects in terms of noise increasing arousal, which can help with tasks like sustained vigilance, whereas noise can disrupt more complex cognitive tasks because it can lead to failures of attention. Many of the laboratory tasks in the psychological literature are traditional experimental ones such as reaction time tasks, pursuit rotor tracking tasks, signal detection tasks, mental arithmetic and word list recall tasks, with corresponding variability in their direct relevance to the types of activities engaged in by the mariner involved in navigation. Furthermore, many of the laboratory-based studies deploy quite intense levels of noise (e.g. 85-95 dB(A)) of a type (often white noise) that is unlike what would be encountered on a vessel at sea, and the noise within experiments is typically presented for a very short duration (seconds or minutes rather than hours). This contrasts with the effect of noise as a Global Design Factor in ships, in that in the latter case the noise is of lower intensity, of different physical characteristics than the noise used in experiments, and involving much more chronic exposure than in the laboratory tasks. There are therefore grounds for believing that noise would indeed be disruptive to marine navigation (as a cognitively complex task with various component activities), but it was considered unrealistic to deploy the types of noise, or the physical intensities of noise, that are typically used in laboratory experiments. The noise used therefore was the noise arising from the use of electric power tools during routine maintenance. The sounds of a needle gun were therefore recorded and played back to the mariners; this type of noise had the benefit (for the purposes of the study) of being non-constant and therefore difficult to habituate to, and from a frequency spectrum that the mariners reported to be irritating.

Deliverable n. 7.2

9/44

3 General methodological approach and participants

3.1 Scenario Description

Four scenarios were scripted (fuller details of the scenarios themselves are given below) with each conducted at a simulated time of 8 am. Participants completed one set of four RoPax or Tanker scenarios as appropriate to their experience. The scenarios lasted for 40 minutes to allow five minutes for hand over time on the bridge and adequate time to allow a potentially threatening situation to develop. Ethical approval was provided by the University of Strathclyde Ethics Committee. Thirty-six mariners each participated in all four scenarios at regular intervals within one day of testing. The mariners were all male, of average age 36.5, and the breakdown of their ranks was as follows: Masters (6); Chief Officers (17); Second Officer (10); 3rd Officer (2); unspecified (1). Scenarios were designed by Hochschule Wismar (HSW) in consultation with The University of Strathclyde (UoS). All scenarios were conducted on Bridge 1 which comprised a fully integrated replica bridge complete with a high-performance projector-based 360-degree real time visual display system to provide panoramic view of ultra-realistic scenarios. For the simulations in WP7 four scenarios were designed: two comparable scenarios with high difficulty and two comparable scenarios with moderate difficulty (see Table 3.1). Whilst scenarios A and B were always by definition more difficult and scenarios C and D were always by definition less difficult, all four scenarios were associated with high noise on half the trials and low noise on the other half, to avoid a noise by scenario confound. Table 3.1: Difficulty and noise levels of scenarios

Noise

high low

Dif

fic

ult

y

hig

h

mo

dera

te

As requested in D3.1 and previously described in D4.2 two different ships were used: a Ropax ferry (in scenario name marked “R”) and a VLCC tanker (in scenario name marked “T”). The ships themselves remained unchanged since the WP 4 experiments. The noise level was independently adjusted since now cluttered noise was used not coming directly from the machinery of the vessel. The main dimensions of the two vessels involved can be found in

Deliverable n. 7.2

10/44

Table 3.2.

Deliverable n. 7.2

11/44

Table 3.2: Ownships used in simulation trials

Vessel Ro-Pax Tanker

Name Mecklenburg-Vorpommern Lagena

Length over all: 199.95 m 353.50 m

Beam: 28.95 m 55.48 m

Draught: 6.20 m 23.00 m

Restrictions due to draught:

almost no restrictions in Dover Strait

mostly restricted to “Deep Draught Fairway”

Maximum speed: 22 kts 15 kts

Power: 2x 12,600 kW 26,478 kW

Manoeuvrability: very high moderate to low

Displacement: 22,720 tons 340,000 tons

The scenario name used later in this report is a combination of the respective ship (“R” or “T”) and the scenario (“A”, “B”, “C” or “D”, with “A” and “B” being of greater levels of difficulty and “C” and “D” being of more moderate levels of difficulty); e.g. a Ropax scenario with higher difficulty would have a name containing “RA” or “RB”, and a Ropax scenario with more moderate levels of difficulty would have a name containing “RC” or “RD”. All simulations done in WP 7 were located in the virtual exercise area of the Dover Strait. Consequently the mariners did not have to familiarise themselves with different sea areas. Furthermore the Dover Strait is one of the busiest international seaways of the world with more than 400 commercial vessels using the strait every day [7]. The strait is divided into one north-east bound and one south-west bound traffic separation scheme (Figure 3.1).

Deliverable n. 7.2

12/44

Figure 3.1: Overview Dover Strait with shipping lanes

Collisions and groundings were identified as major threats to shipping and risk models developed to quantify the risks (D4.6). One of these two threats was provoked in each scenario. The mariners were challenged to take actions preventing the imminent accident. The risk events embedded within the scenarios consisted of a combination of at least two possible but rare incidents challenging the mariner to take swift and correct actions. Table 3.3 below gives an overview of the events for the scenarios:

Deliverable n. 7.2

13/44

Table 3.3: Overview of events within scenarios

Criteria Scenario A Scenario B Scenario C Scenario D

Threat Collision Collision Collision Grounding

Wind east Bft 5 north Bft 5 east Bft 5 west Bft 5

Visibility good moderate, later poor good good

High Risk Event 1

External crossing vessel (OS

3 stand-on)

crossing vessel (OS give-way)

crossing vessel (OS give-way)

proximity to shallow water

Internal steering gear failure radar failure DOLOG break down

GPS break down

High Risk Event 2

Internal fire alarm bilge alarm rudder pump failure (1 pump still operative)

moving car on deck (ro-pax)/

oily water from bulkhead (tanker)

In contrast to the RPM dependent noise of the ship’s machinery used in WP4 experiments, in WP7 the cluttered noise of a needle gun while derusting a ship’s deck was utilised. This noise was rated as very annoying by the mariners. The variations in pattern and volume made it almost impossible to adapt to. The average loudness was 73.5 dB(A) with a maximum of 79.2 dB(A) measured at the centre of the bridge (Figure 3.2), which is considerably more than the maximum noise level of 65 dB(A) stipulated for navigation spaces [8, p. 17].

Figure 3.2: Sound level on simulator bridge

3 OS – Own Ship in simulations: vessel operated by mariner

Deliverable n. 7.2

14/44

3.2 Detailed Scenario Description

3.2.1 Scenario RA/TA

The vessel is in the south-west bound traffic lane north of Sandettie Bank slightly off-track due to previous manoeuvres. The next waypoint is 3 nm ahead. Also all other traffic follows the direction of the traffic lane and is not influencing the navigation of the vessel. All equipment is operative.

Figure 3.3: Overview scenario RA/TA

Steering the new course after approximately 9 minutes it is obvious that a small vessel (“Greta”) crossing the lane from the south is on a collision course. The mariner is according to the COLREGs4 obliged to maintain course and speed (stand-on vessel). The crossing vessel from portside must take early and substantial action to keep well clear (give-way vessel). In this simulation the give-way vessel does not take any action so that the mariner on the stand-on vessel has to take an appropriate action instead. This scenario was designed in this way to force both vessels to a close distance so that a collision risk exists. Six minutes before the potential collision the steering gear failed, not allowing the mariner to change the course away from the opponent vessel. Shortly before the potential collision a fire alarm is set off by the instructor.

4 COLREGs – Convention on the International Regulations for Preventing Collisions at Sea, 1972 by the

International Maritime Organization (IMO)

Deliverable n. 7.2

15/44

With substantial and early action the mariner should be able to stop the vessel. Additionally all relevant parties have to be informed and actions according to Emergency Response Procedures have to be initiated. Here is a selection of expected main actions:

– main engine stopped or reversed

– switching rudder pumps and engage follow-up mode

– advise engine control room and master

– set shapes and lights for “Not under command” vessel

– send safety/urgency message on DSC and VHF

– fix position and make relevant entries in bell book

– prepare emergency steering

– request bow thruster (on RoPax only)

– acknowledge fire alarm and inform squads (e.g. crew alert)

The evaluation of these actions is described in Error! Reference source not found..

3.2.2 Scenario RB/TB

The vessel follows the north-east traffic lane towards east of Colbart Bank. About 1 nm on the starboard side of the vessel there are three traffic ships with a lower speed to be overtaken within the next hour. There is one ship on the port side running a parallel course with a similar speed. The ferry corridor from Dover to Calais is about 10 nm ahead.

Figure 3.4: Overview scenario RB/TB

Deliverable n. 7.2

16/44

The S-band radar and the AIS are not operative. The visibility is moderate and deteriorating later due to heavy showers. After 14 minutes the visibility is poor. After 20 minutes also the remaining X-band radar loses its trigger so that no radar target information is available at all. A crossing ferry from Calais was visible before the radar failure increasing the pressure on the mariner to take some action. From now on the officer of the watch has to listen to fog signals being transmitted by other vessels and act according to COLREGs and company procedures (decreasing to safe speed, posting an additional lookout, adopting different bridge manning etc.). Some minutes later there is a bilge alarm to acknowledge and to handle, increasing the work load further. Some main tasks of the mariner are:

– adjust main engine’s RPM

– advise engine control room and master

– call additional personnel

– set fog signals

– send safety/urgency message on DSC and VHF


– acknowledge bilge alarm and inform relevant parties


3.2.3 Scenario RC/TC

The vessel is in the south-west bound traffic lane north of Sandettie Bank following this lane. There is no traffic in the closer vicinity influencing the ship. The next waypoint is 3 nm ahead. All equipment is operative.

Deliverable n. 7.2

17/44

Figure 3.5: Overview scenario RC/TC

After 10 minutes a car carrier on the starboard side alters its course crossing the traffic lane. The own ship is requested to give way to this vessel by COLREGs. There are two failures not having an influence on the collision avoidance and having a minor impact on the bridge watch: the Doppler log and one of the two rudder pumps fail in the course of the simulation. Some expected actions are:

– acknowledge the alarms

– switching rudder pumps

– advise engine control room and possibly master

– switch integrated navigation system to GPS based speed



3.2.4 Scenario RD/TD

Since the draught of the ro-pax ferry and the VLCC differ considerably the grounding risk had to be created in different spots of the Dover Strait: the risk of grounding for the scenario RD involving the ro-pax vessel is close to the Sandettie Bank, whereas for scenario TD (the VLCC vessel) it is close to the Varne Bank.

Deliverable n. 7.2

18/44

Figure 3.6: Overview scenario RD

Figure 3.7: Overview scenario TD

Due to overtaking vessels on one side the own ship is forced to stay longer on its course than intended by the voyage plan approaching shoal water ahead. Similar to scenario C

Deliverable n. 7.2

19/44

there are some low priority failures that the officer of the navigational watch has to attend to beside the situation handling navigating in restricted waters. In addition the work load was increased by internal and external communication. Examples for the expected actions are:

– acknowledge the alarms

– switch integrated navigation system to second GPS

– advise engine control room and possibly master



3.3 Monitoring and Recording of Simulation Trials

3.3.1 Evaluation Criteria

The above described actions were divided into “first minute actions” with a high priority and “follow-up actions” with a lower priority. For the evaluation a combination of a number based rating scheme and an expert rating was introduced. The number based rating scheme included the action itself and the time after the incident the action was started (Figure 3.8).

Figure 3.8: General rating scheme for actions in scenario A

After testing the rating scheme it was observed that in some cases it was necessary to adjust the score in a positive direction by one point because the mariners had executed

Score Score

<1 5 <2 3

<2 4 <4 2,5

<3 3 <6 2

<4 2 <8 1,5

>4 1 >8 1

0

1

2

3

4

5

General Scoring Tables

Scenario A

First minute actions scores: Follow up action scores:

Minutes Minutes

Reverse within 1 minute

M/ E RPM Change

RPM unchanged

Running ahead

Stop after 1 minute

Stop within 1 minute

Reverse after 1 minute

Deliverable n. 7.2

20/44

unexpected actions achieving a positive solution. In other cases, some actions were executed but in a wrong way, so the score was adjusted by reducing it. Figure 3.9 shows an example for a completed rating sheet for the primary incident in scenario A. The colour code on the side marks the “first minute actions” in red, the “follow-up actions” in yellow.

Figure 3.9: Example for rating scenario A for "High Risk Event 1"

The following two figures (Figure 3.10 and Figure 3.11) show the rating for the “High Risk Event 2” in the same scenario and the overall score from the instructor.

Figure 3.10: Example for rating scenario A for "High Risk Event 2"

High Risk Event 1:

Time Event Set Off: 08: 20 : 00

Response Actions

External Effect (Note Time of 1st Action): Elapsed Score Adjust Remarks (e.g. "Crash"):

M/ E command RPM: EOT % 4 4 steering with propellers!

Time of commanding: 08: 23 : 00 03:00 3

Internal Actions (Note Time of Actions): Remarks:

Check or try to switch pumps 08: 20 : 25 00:25 5

Switch to Follow-Up mode 08: 22 : 00 02:00 4

Advise E/ R 08: 20 : 45 00:45 5

Call Master 08: 24 : 00 04:00 2

NUC shapes/ lights 08: 23 : 45 03:45 2,5

Safety message DSC 08: :

Safety message VHF 08: 23 : 30 03:30 2,5

Fix position and bell book entry 08: :

Prepare emergency steering 08: 22 : 30 02:30 2,5 ordered helmsman

Request bow thruster 08: :

Vessel Crossing and Steering Gear Failure

-6

High Risk Event 2:

Time Event Set Off: 08: 25 : 15

Response Actions

Internal Actions (Note Time of Actions): Elapsed Score Adjust Remarks:

Acknowledge alarm 08: 25 : 24 00:09 5

Call ECR, Bosun or Watch Man 08: 25 : 50 00:35 5

Call Master 08: 35 : 00 09:45 1

G/ A or Crew Alert 08: :

Fix position and bell book entry 08: :

Fire Alarm

Deliverable n. 7.2

21/44

Figure 3.11: Example for rating scenario A for overall instructor rating

In case of a collision or grounding the respective “High Risk Event” was rated as having a “0” score. Most of the scoring was done whilst the simulation was taking place. For the exact times of some actions the recorded video records (see 3.3.2) and the post-processing data (see 3.3.3) had to be used. For further cross-scenario analyses it was necessary to normalize some data (passing distance to other ships/banks, track deviation) since due to the very different manoeuvring characteristics of the two simulation ships the absolute values varied such that there were designated patterns for the RoPax ferry and the tanker. Furthermore it is not legitimate to compare e.g. 0.8 nm passing distance from scenario A (manoeuvre of the stand-on vessel) with the passing distance from scenario B (manoeuvre of the give-way vessel). Therefore a weighted average for each ship type in each scenario was introduced on the basis of the achieved, expected, possible and common distances with a wider range, omitting extreme low and high values.

3.3.2 CCTV

To enable the evaluation of mariner action after completion of the scenario (where necessary) it was decided to install four CCTV cameras on the simulator bridge: two in the front for the port and starboard consoles, one at the steering console and one at the chart table (see Figure 3.12).

Overall performance from 0 (collision/ grounding) over 1 (poor) to 5 (excellent):

(consider quality of communication, bell book and chart entries, general attitude)4

Deliverable n. 7.2

22/44

Figure 3.12: CCTV cameras on the simulator bridge

The cameras could be monitored online and the timing of the entire simulation was recorded on all four cameras. Figure 3.13 shows the monitoring of the cameras. Due to the dimmed light on the bridge only infrared video was used. The two front cameras additionally recorded the audio signals from the bridge.

Deliverable n. 7.2

23/44

Figure 3.13: Monitoring of CCTV cameras

3.3.3 Post Processing with Evaluation Software SIMDAT

All simulations were recorded in a proprietary format for the post-processing in the software SIMDAT. The software package SIMDAT is designed to display and evaluate data from simulations and real-world experiments. It is possible to display combinations of various parameters e.g. RPM of the main engine and ship’s speed. Furthermore distances to other ships and objects (limit lines for banks) can be computed. All data is displayed on a kind of electronic navigational chart (Figure 3.14).

Deliverable n. 7.2

24/44

Figure 3.14: Example of data analysis with SIMDAT

All data and graphics can be exported for a more detailed view (see Figure 3.15 below, showing all tracks across all Ropax simulation runs in scenario D).

Deliverable n. 7.2

25/44

Figure 3.15: Example for tracks with end positions for scenario RD

4 Results

4.1 Analysis of vessel navigation data

The navigation by the mariners of their own vessel was measured by two variables, the passing distance to a relevant target ship in scenarios A-C or to a grounding risk in scenario D, and the deviation from the required track. Because the four scenarios varied in terms of where the target vessel was located relative to the mariner’s own ship (Mecklenburg-Vorpommern or Lagena, depending upon the scenario) at the outset, it was not appropriate to compare absolute passing distances (or indeed absolute track deviations) across scenarios, and therefore these distances were normalized to allow a valid comparison across scenarios (as explained above in section 3.3.1)

Deliverable n. 7.2

26/44

4.1.1 Analysis of passing distances

Figure 4.1 below shows the mean normalized passing distances across all four scenarios:

Figure 4.1: Mean normalized passing distances across all conditions of noise and task difficulty, expressed in nautical miles

Analysis of Variance of the above data yielded no significant effect of noise (passing distance under conditions of low noise mean = 0.92nm, SE = 0.67; passing distance in high noise mean = 0.88, SE = 0.66; F<1), no significant effect of task difficulty (passing distance under conditions of lower difficulty mean = 0.93nm, SE = 0.67; passing distance in low noise mean = 0.87, SE = 0.66; F<1) and no significant noise X difficulty interaction (F<1). This remained true even when the analysis was re-run using the experience measures as covariates: there was no significant effect of noise or task difficulty on the normalized passing distances. A detailed examination was undertaken of whether there was any effect of noise upon normalized passing distances within each scenario, in case there were more specific effects happening for particular scenarios. The relevant data are tabulated below in Figure 4.2:

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

Low Noise High Noise

Low Difficulty

High Difficulty

Deliverable n. 7.2

27/44

Figure 4.2: Mean normalized passing distances across all conditions of noise and task difficulty within each scenario, expressed in nautical miles

The basic patterns of data are inconsistent: for scenario A and B noise is associated with a lower passing distance but for scenarios C and D noise is associated with a greater passing distance (scenario A lower noise mean = 0.78, SE = 0.13, scenario A higher noise mean = 0.70nm, SE = 0.11; scenario B lower noise mean 1.10nm, SE = 0.20, scenario B higher noise mean = 0.94, SE = 0.18; scenario C lower noise mean = 1.0, SE = 0.07, scenario C higher noise mean = 1.04nm, SE = 0.07; and scenario D lower noise mean = 0.89, SE = 0.12, scenario D higher noise mean = 0.79nm, SE = 0.11). However, none of the differences involved are statistically significant. This remains true even when mariners’ self-rated experience of the relevant High Risk events within the scenarios are used as covariates. There is therefore no discernible effect of noise upon the passing distances between the mariners’ own and the target vessels.

4.1.2 Analysis of track deviations

Figure 4.3: Mean normalized track deviations across all conditions of noise and task difficulty, expressed in nautical miles

0

0.2

0.4

0.6

0.8

1

1.2

Scenario A Scenario B Scenario C Scenario D

Low Noise

High Noise

0.9

0.92

0.94

0.96

0.98

1

1.02

1.04


Low Difficulty

High Difficulty

Deliverable n. 7.2

28/44

Figure 4.3 shows the normalized track deviations across all conditions of noise and task difficulty. Analysis of Variance of the above data yielded no significant effect of noise (low noise mean = 0.99, SE = .05, high noise mean = 1.03, SE = 0.05, F<1), no significant effect of task difficulty (low difficulty mean = 0.99, SE = 0.07, high difficulty mean = 1.03, SE = 0.7, F<1) and no significant noise X difficulty interaction (F<1). This remained true even when the analysis was re-run using the experience measures as covariates: there was no significant effect of noise or task difficulty on the normalized passing distances.

Figure 4.4: Mean normalized track deviations across all conditions of noise and task difficulty within each scenario, expressed in nautical miles

Figure 4.4 breaks down the track deviations scenario by scenario. Again the basic patterns of data are inconsistent: for scenarios A, and B, high noise is associated with a greater passing distance but for scenarios C and D, high noise is associated with a lower passing distance. (The relevant descriptive statistics are: scenario A lower noise mean = 0.88, SE = 0.11, scenario A higher noise mean = 1.1.nm, SE = 0.9; scenario B lower noise mean = 0.94nm, SE = 0.14, scenario B higher noise mean = 1.14, SE = 0.13; scenario C lower noise mean = 1.02, SE = 0.11, scenario C higher noise mean = 0.94nm, SE = 0.10; and scenario D lower noise mean = 1.02, SE = 0.40, scenario D higher noise mean = 0.96nm, SE = 0.22).However, again, none of the differences involved are statistically significant, and again this remains true even when mariners’ self-rated experience of the relevant High Risk events within the scenarios are used as covariates. There is therefore no discernible effect of noise upon the size of the deviations from track in the mariners’ vessels.

4.2 Analysis of instructor ratings

4.2.1 Overall performance rating

The first analysis undertaken was of the instructors’ overall ratings of the task performance on a scale from 1 (poor) to 5 (excellent) – although in some extreme cases scores of zero were awarded. This is the only rating based on the subjective evaluation of the instructor (see 3.3.1). The data are shown in Figure 4.5 below.

0

0.2

0.4

0.6

0.8

1

1.2

Scenario A Scenario B Scenario C Scenario D

Low Noise

High Noise

Deliverable n. 7.2

29/44

Figure 4.5: Mean overall instructor rating across all conditions of noise and task difficulty within each scenario, expressed on a scale from 1 (poor) to 5 (excellent)

The data show the pattern of means that was initially predicted, namely higher overall ratings for performance in less noisy rather than noisier conditions, and higher overall ratings for performance in low difficulty compared to high difficulty tasks. The relevant descriptive statistics are: low noise overall mean = 3.65, SE = 0.14, high noise overall mean = 3.33, SE = 0.14, low difficulty overall mean = 3.64 SE = 0.14, high difficulty overall mean = 3.33,SE = 0.14. A 2 X 2 ANOVA showed a marginally significant effect of noise (F=2.54, p<0.06, 1 tailed), a marginally significant effect of task difficulty (F=2.32, p<0.065, 1 tailed) and a non-significant noise X difficulty interaction (F<1, ns). Rerunning the above analyses with self-rated experience of elements of the scenario as covariates revealed a significant effect of noise (F=2.61, p<0.05, 1 tailed), a non-significant effect of task difficulty (F=2.43, p<0.095, 1 tailed) and a non-significant noise X difficulty interaction (F<1, ns). There is therefore some support for the view that mariner performance under noisy conditions is less good than under less noisy conditions, as measured by instructors’ overall global ratings.

2.8

2.9

3

3.1

3.2

3.3

3.4

3.5

3.6

3.7

3.8


Low Difficulty

High Difficulty

Deliverable n. 7.2

30/44

4.2.2 Percentage performance in High Risk 1 events

Figure 4.6: Percentage performance across all task performance measures within High Risk Event 1 within the scenarios, across all conditions of noise and task difficulty

For this analysis, the mariners’ task performance scores (scores between 1 and 5 reflecting the rapidity and quality of mariners’ actions, as described in Figure 3.8 above) across all the individual actions were summed within each scenario and expressed as a percentage of the maximum possible score for that scenario. This made the scores comparable across different scenarios with differing required numbers of actions to be taken. The relevant means are shown in Figure 4.6 above. The low noise condition shows only a marginal increase in percentage task performance over the high noise condition (low noise mean = 49.4, SE = 1.98; high noise mean = 47.2, SE = 1.94) and it is not significantly different from it (F (1, 135)<1). The effect of task difficulty is in line with expectations (with more difficult tasks being associated with lower scores: low difficulty mean = 51.2, SE = 1.97; high difficulty mean = 45.4, SE = 1.96) and the difference between high and low difficulty tasks is significant (F (1,140)=5.08, p<0.026). The interaction between noise and difficulty is not significant (F<1). In contrast with the analysis above of overall instructor ratings, for High Risk Event 1 task difficulty has a significant effect, with low difficulty tasks being associated with significantly better instructor ratings than the high difficulty tasks. Noise, in contrast, has a non-significant effect overall, and (like the overall instructor ratings analysis above) the noise by difficulty interaction is non-significant. When the above analysis was re-run using the mariners’ self-rated experience of the elements within the scenario as covariates, the above effects were replicated (no significant effect of noise (F<1, ns), a significant effect of task difficulty (F=4.45, p<0.037) and a non-significant noise by difficulty interaction (F<1, ns). Regardless of whether mariner experience is taken into account or not, therefore, the effect of task difficulty is significant within High Risk Event 1, but not that of noise.

38

40

42

44

46

48

50

52


Low Difficulty

High Difficulty

Deliverable n. 7.2

31/44

4.2.3 Percentage performance in High Risk 2 events

As was the case for the analysis of scores for High Risk Event 1, for the analysis of scores in High Risk Event 2 the mariners’ task performance scores across all the individual actions that could be taken in the scenario were summed within each scenario and expressed as a percentage of the maximum possible score for that scenario. These data are shown in Figure 4.7 below.

Figure 4.7: Percentage performance across all relevant task performance scores within High Risk Event 2 within the scenarios, across all conditions of noise and task difficulty

The low noise condition shows a small increase in percentage task performance over the high noise condition overall (low noise mean = 52.7, SE = 1.69; high noise mean = 50.4, SE = 1.66) but it is not significant (F<1, ns). The effect of task difficulty shows very little overall difference between scores (low difficulty mean = 51.4, SE = 1.68; high difficulty mean = 51.7, SE = 1.67) and is non-significant (F<1, ns), and the noise X difficulty interaction is non-significant (F<1). This pattern of results was exactly replicated when mariner experience was used as covariates. There is thus no evidence of either noise or task difficulty affecting the mariners’ overall percentage performance scores within High Risk Event 2, nor was there any evidence of a noise X task difficulty interaction.

4.2.4 Percentage performance in High Risk 1and High Risk 2 events combined

In this analysis the grand total score for all actions undertaken across both High Risk Event 1 and High Risk Event 2 was expressed as a percentage of the total possible score across both High Risk Events. The relevant data are in Figure 4.8 below.

47

48

49

50

51

52

53

54

55


Low Difficulty

High Difficulty

Deliverable n. 7.2

32/44

Figure 4.8: Percentage performance across all relevant task performance scores within High Risk Events 1 and 2 combined, across all conditions of noise and task difficulty

The low noise condition shows very little difference in percentage task performance compared to the high noise condition (low noise mean = 52.7, SE = 1.71; high noise mean = 50.4, SE = 1.68) and again this small difference is not significant (F (1, 135)=1.044, ns). The effect of task difficulty shows a small difference between scores (low difficulty mean = 51.7, SE = 1.70; high difficulty mean = 51.3, SE = 1.69) and is non- significant (F (1,135=1.7, p<0.1, 1 tailed, ns), and the noise X difficulty interaction is non-significant (F(1,135)<1, ns). Re-running this analysis with the experience measures as covariates exactly replicated the above pattern of results: no significant effect of noise (F = 1.166, ns) a non-significant effect of task difficulty (F=1.822, p<0.09, 1 tailed, ns), and a non-significant noise X difficulty interaction (F<1, ns).

4.2.5 Detailed examination of mariner actions within scenarios

The task performance scores of individual mariner actions within each High Risk Event, scenario by scenario, were examined as a function of noise in case noise has the effect of impairing particular actions within scenarios. It should of course be pointed out that this involves making a large number of comparisons and when many statistical analyses are undertaken, more stringent level of significance should be adopted. Considerable caution is therefore required in claiming the significance of any effects found. As with all the previous analyses reported in section 4 above, in all cases Mariner 18’s data were excluded, because his performance of all tasks were idiosyncratically different from those of all other mariners, and he therefore constituted an outlier within the analysis. Within Scenario D, mariner 9’s data were additionally excluded because a note on the data log indicated that this particular scenario was not suitable for evaluation thanks to a vessel being missing within the scenario. The data for the task performance scores of mariners’ particular actions called for by the High Risk events in scenario A are summarised in Figure 4.9 below.

44

45

46

47

48

49

50

51

52


Low Difficulty

High Difficulty

Deliverable n. 7.2

33/44

Figure 4.9: Mean task performance scores for all particular actions undertaken by mariners within scenario A as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent)

It is clear that some actions are carried out rapidly and well by all mariners regardless of noise conditions (e.g. Acknowledging the alarm in High Risk Event 2) whereas others are carried out rarely and/or slowly, the extreme example being the General Alarm or Crew alert, which is carried out by none of the mariners. Some actions are attaining significant or near-significant differences between noisy and quieter conditions, and in the vast majority of cases in the predicted direction of poorer performance under conditions of noise, but these have to be interpreted cautiously given the large number of comparisons drawn. The significant or near-significant differences are in the variables: Time of RPM change command (low noise mean = 2.65, SE = 0.39, high noise mean = 4.13, SE = 0.45; p < 0.009, 1 tail); Call Master during High Risk Event 1 (low noise mean = 3.07, SE = 0.40, high noise mean = 2.2, SE = 0.35; p < 0.06 (1 tail)); Send a message via VHF (low noise mean = 1.5, SE = 1.26, high noise mean = 2.13, SE = 1.06; p < 0.06, 1 tail); Fix position and make a bell book entry (low noise mean = 1.67, SE = 0.27, high noise mean = 1.02, SE = 0.24; p < 0.04, 1 tail)). On the other hand, those non-significant results with F ratios of <1 (true for the variables Main Engine RPM, Switch pumps, Switch to Follow Up Mode, Advise Engine Room, utilise NUC lights/shapes, Message DSC, Prepare Emergency Steering, Activate Bow Thruster, Acknowledge Alarm in HRE2, Call Bosun in HRE2, Call Master in HRE2) are so far away from statistical significance that it is not really worth commenting even on the pattern of their means. The data for the ratings of mariners’ particular actions called for by the High Risk events in scenario B are summarised in Figure 4.10 below.

00.5

11.5

22.5

33.5

44.5

5

Low Noise

High Noise

Deliverable n. 7.2

34/44

Figure 4.10: Mean task performance scores for all particular actions undertaken by mariners within scenario B as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent)

As was the case with scenario A, some actions are carried out rapidly by all mariners (e.g. acknowledging the alarm in High Risk Event 2) whereas others are carried out much less rapidly (e.g. fixing the vessel’s position and making a bell book entry). The comparison between noisier and less noisy conditions for one action (closing water tight doors) appears to approach significance (p<0.08, 1 tail), whilst for Calling the Master during High Risk Event 2 the comparison reaches significance on a parametric test but subsequent checking via a non-parametric test (the variances are unequal and the use of the parametric result is therefore suspect) revealed a non-significant result. There is therefore no real evidence within the data from this scenario that noise was having any effect on mariners’ actions. The data for the ratings of mariners’ particular actions called for by the High Risk events in scenario C are summarised in Figure 4.11 below.

Figure 4.11: Mean task performance scores for all particular actions undertaken by mariners within scenario C as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent)

00.5

11.5

22.5

33.5

44.5

5

Low Noise

High Noise

00.5

11.5

22.5

33.5

44.5

5

Low Noise

High Noise

Deliverable n. 7.2

35/44

In scenario C there is again variability in the rapidity and quality with which the mariners carry out their actions overall, but absolutely no evidence of any effect of noise on those actions. All comparisons are statistically non-significant, and all but two (Switch to GPS and Switch Pumps) have F ratios of less than 1, meaning that they are very far away from being statistically significant. The data for the ratings of mariners’ particular actions called for by the High Risk events in scenario D are summarised in Figure 4.12 below.

Figure 4.12: Mean task performance scores for all particular actions undertaken by mariners within scenario D as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent)

Once again for scenario D, as with scenario C, there is no evidence of any effect of noise: all comparisons have an F ratio less than 1. Overall, what evidence there is for an effect of noise on mariners’ actions (and it has to be emphasised again that it is limited and weak) is occurring within the more challenging scenarios, particularly scenario A, and this might hint at a noise by task difficulty interaction such that more straightforward tasks can be carried out quite well even under conditions of noise, but more difficult scenarios might be showing some signs of impairment. Marginal though these effects are, bearing in mind that the number of mariners being tested could ideally have been larger for the sake of statistical power, and the limitations imposed by the short duration of the scenarios, the fact that any significant differences are occurring, even marginal ones, should counsel caution against dismissing potential adverse effects of noise on mariner performance.

4.2.6 Analysis of early scenarios within the testing sequence

The fact that the experiment used a repeated measures experimental design in which each mariner undertook a series of tasks raises the possibility that whilst they might find the high risk events surprising when they encounter them during the first scenario or two, they might, as the sequence of tasks progressed, begin to anticipate the possibility

0

1

2

3

4

5

Low Noise

High Noise

Deliverable n. 7.2

36/44

of such out-of-course events being likely to happen within the later scenarios. They would then be a little more alert and mentally prepared for them. This would imply that the occurrence of high risk events would be most surprising in the first one or two events in the sequence. It is possible that noise would be more disruptive in such circumstances. We therefore reanalysed the data looking at the effects of noise in the first two scenarios that each mariner encountered. These analyses confirmed the significant effect of noise on instructor ratings reported in Figure 4.5 above (F(1, 62)=2.98, p<0.045, 1 tail). However, they also showed a significant effect of noise on the distance from a target ship: mariners sailed significantly more closely to target vessels during the first two scenarios under conditions of noise (low noise mean = 0.99nm, SE = 0.08, high noise mean = 0.74, SE = 0.87; F(1, 65)=4.91, p<0.015, 1 tail; see Figure 4.13 below)

Figure 4.13: Mean normalized passing distances across all conditions of noise and task difficulty, expressed in nautical miles, for the first two scenarios undertaken by the mariners

The effect of task difficulty, however, was not significant (F<1), and the noise X task difficulty interaction was non-significant also (F<1). The finding that noise is disruptive to the maintenance of a safe distance from a target vessel within the first two scenarios (Figure 4.13 above) but not across all scenarios (Figure 4.1 above) implies that the mariners are adapting to the sequence of scenarios as they go through them, allowing them to counteract the effect of noise. It also suggests that noise might be most disruptive to task performance when it coincides with surprising unexpected events.

5 Summary and conclusions There are no significant effects of noise or task difficulty on either of the navigation measures (normalised distance to the relevant target ship, and normalised track deviation). However, when the instructors’ ratings of the rapidity/quality of mariners’ responses are considered, a number of significant or marginally significant effects of noise and task difficulty emerge. The overall, global ratings of task performance by instructors showed both a significant effect of task difficulty (which validated the task difficulty manipulation within the experiment) and a significant effect of noise such that higher levels of noise were associated with poorer performance. The effects of noise were also either significant, or close to statistical significance, in the case of several

0

0.2

0.4

0.6

0.8

1

1.2


Low Difficulty

High Difficulty

Deliverable n. 7.2

37/44

specific actions within scenario A (one of the two ‘high difficulty’ scenarios), suggesting that there might be an interaction between noise and task difficulty such that noise has little or no effect in situations where the task is easier, but might lead to impaired performance of a high difficulty task. This pattern of data fits well with Hancock and Warm’s [9] model of the effect of stress on sustained attention. They argue that the task itself should be considered as a stressor, in addition to any external stressor such as noise or temperature, and that where the combined level of stress from both sources is low, there is a buffering effect provided by our adaptive capability. (Indeed, where a task has very low levels of stress associated with it, some additional external stress may improve the performance of the task by increasing arousal). However, when the buffering capability of adaptive capacity is outstripped by the combined effect of task stress and external (e.g. noise) stress, psychological discomfort ensues, ultimately leading to a decrease in psychological capability. This fits closely with the present experiment’s findings: noise had no effect when the task itself was less difficult, but particularly within the high-difficulty scenario A, specific effects of noise began to emerge. It should of course be noted that Hancock and Warm’s model is of the effect of stressors specifically on sustained attention rather than task performance more generally; however, it is undoubtedly the case that sustained attention is one of the major task elements within marine navigation tasks, and it is therefore reasonable to expect that navigation tasks would be affected in the way that Hancock and Warm predict. The analysed data of the early scenarios showed that the situation itself (being thoroughly observed, using equipment that is not well-known, working within an exam situation etc.) might work as a stressor. If a severe second stressor (such as loud and varied noise) occurs in addition, a negative impact on the performance in high difficulty scenarios was apparent. Although there are some effects of noise found in the experiment, they are only marginally statistically significant, and future research could usefully investigate the effect of noise further, taking into account the following points concerning how the present experiments were designed and conducted: 1. The scenario duration might have been too short. Although the noise within the high-

noise conditions was irritating, it might be that the mariners were able to cope with it

knowing that the scenario would end after 40 minutes. Longer scenarios up to a full

navigational watch of four hours might help address this issue. However, this would

lengthen the amount of time for each experimental trial and therefore reduce the total

number of participants that could be tested within a given total amount of available

simulator time. Indeed, the need to test as many mariners as possible within the

available simulator time to maximise the statistical power of the analyses was one

major reason why the scenarios had been limited to 40 minutes in length in the

present experiment.

2. The use of two very different vessels in the experiment made it hard to treat the mariners’ actions and the task outcomes (passing distance, track deviation) as equivalent across vessel types. Normalized distance values were calculated to overcome these differences.

Deliverable n. 7.2

38/44

3. Mariners from varying professional backgrounds participated in the experiment. RoPax mariners from Estonia and tanker mariners from Greece would have different educational, cultural and social backgrounds from each other, and their respective regularly traversed sea areas, each with its habitual navigational practices influencing the mariners’ actions, could have had an effect on the measured outcomes. This would have served to increase variability in the mariners’ scores within the tasks and reduced the chances of finding significant differences across conditions of the experiment. Future experiments could reduce this variability by testing one group, or alternatively two groups of mariners who were more similar to each other than the two groups tested here.

4. Although all mariners had gone through a thorough familiarisation for some mariners the bridge equipment was new causing a learning effect during the four scenarios which might have been influencing the outcome.

5. The scenarios involving collision avoidance required the mariners to carry out sequences of well learned procedures (rather than engage in high level cognitive activity such as problem diagnosis) and could have left the mariners with too few alternatives to decide between, to properly show the effect of noise on task performance. As noted above, noise serves to increase mental stress; if the noise shows little or no effect even in difficult situations (for example, in the case of scenario B), this might suggest that the overall cognitive workload (which in this case was mainly comprised of executing well-learned procedures) could have been too low to properly show the potential effects of noise. Future studies could accordingly explore ways to enhance the mariners’ cognitive workload at the same time as introducing the noise stressor.

6. The mariners tested were on the whole, very experienced, which could lead to smaller differences between noise and no noise scenarios, because increased experience might be expected to increase an individual’s capacity to adapt to stressors. It might be useful to test mariners of a less senior level who had no or lesser experience of some of the faults and alarms used in the present experiments.

Since a large number of accidents result from actions taken at the “sharp end” (i.e. are caused by the operator or a team of operators) the extent to which noise has a negative influence in high stress situations with high cognitive workload remains a pressing issue for future research to address. Therefore we suggest that future experiments could incorporate into their designs the following considerations: 1. Testing reasonably large samples of Mariners (at least as large, or larger than, the

sample size in the current experiment) drawn from one background, or more comparable backgrounds than was the case in the current studies;

2. Utilizing only one reference vessel within the scenarios; 3. Using longer scenarios, making it harder for the mariners to cope with the noise; 4. Introducing greater cognitive workload by extending the scenarios to crisis

management; 5. Having a realistically-sized bridge team (e.g. 2 officers, 1 helmsman) dealing with the

problem; this would extend the possible effects of noise from one of impairing the cognitive processes within individual mariners to one of individual cognitive impairment plus degradation of inter-individual communication via masking of utterances by noise.

The scoring system used by the instructors within the present experiment worked well in practice. This utilised an expert instructor rating based on pre-defined objective criteria, with occasional adjustments being made to the recommended score to account for

Deliverable n. 7.2

39/44

actions being carried out better or slightly worse than expected. In future experiments we recommend that this approach to scoring task performance by expert instructors is developed further and extended by giving detailed consideration to how best to score mariner actions, and how much to adjust scores in the light of unexpected mariner task performance, within specific scenarios.

Deliverable n. 7.2

40/44

6 Bibliography and References

[1] C. D. Wickens and J. G. Hollands, Engineering Psychology and Human Performance, New Jersey: Prentice Hall, 2000.

[2] G. R. J. Hockey, “The effect of loud noise on attentional selectivity,” Quarterly Journal of Experimental Psychology, 22:1, pp. 28-36, 2007.

[3] Maritime Accident Investigation Branch, “Bridge Watchkeeping Safety Study,” Southampton, 2004.

[4] J. L. Szalma and P. A. Hancock, “Noise Effects on Human Performance. A Meta-Analytic Synthesis,” Psychological Bulletin, 137(4), pp. 682-707, July 2011.

[5] R. E. Kurt, O. Turan, O. Arslan, H. Khalid, D. Clelland and N. Gut, An Experimental Study Investigating the Effects of Noise on Seafarers’ Performance and Comfort, Glasgow, 2010.

[6] M. Kirchhoff, SIMDAT – Simulation Data Analysis Software, Rostock: ISSIMS GmbH, 2014.

[7] Maritime & Coastguard Agency, “Dover Strait crossings: channel navigation information service (CNIS),” 6 May 2014. [Online]. Available: https://www.gov.uk/government/publications/dover-strait-crossings-channel-navigation-information-service/dover-strait-crossings-channel-navigation-information-service-cnis.

[8] International Maritime Organisation – Maritime Safety Committee, “Resolution MSC.337(91),” 30 November 2012. [Online]. Available: http://www.imo.org/en/KnowledgeCentre/IndexofIMOResolutions/Documents/MSC%20-%20Maritime%20Safety/337%2891%29.pdf.

[9] P. A. Hancock and J. S. Warm, “A Dynamic Model of Stress and Sustained Attention,” Human Factors, 31, pp. 519-537, 1989.

Deliverable n. 7.2

41/44

7 Indexes

7.1 Index of Tables

Table 3.1: Difficulty and noise levels of scenarios ........................................................... 9 Table 3.2: Ownships used in simulation trials ................................................................ 10 Table 3.3: Overview of events within scenarios ............................................................. 12

7.2 Index of Figures

Figure 3.1: Overview Dover Strait with shipping lanes .................................................. 11 Figure 3.2: Sound level on simulator bridge .................................................................. 12 Figure 3.3: Overview scenario RA/TA ........................................................................... 13 Figure 3.4: Overview scenario RB/TB ........................................................................... 14

Figure 3.5: Overview scenario RC/TC ........................................................................... 16 Figure 3.6: Overview scenario RD ................................................................................. 17 Figure 3.7: Overview scenario TD ................................................................................. 17 Figure 3.8: General rating scheme for actions in scenario A ......................................... 18

Figure 3.9: Example for rating scenario A for "High Risk Event 1" ................................ 19

Figure 3.10: Example for rating scenario A for "High Risk Event 2" .............................. 19 Figure 3.11: Example for rating scenario A for overall instructor rating ......................... 20

Figure 3.12: CCTV cameras on the simulator bridge ..................................................... 21 Figure 3.13: Monitoring of CCTV cameras .................................................................... 22 Figure 3.14: Example of data analysis with SIMDAT ..................................................... 23

Figure 3.15: Example for tracks with end positions for scenario RD .............................. 24 Figure 4.1: Mean normalized passing distances across all conditions of noise and task difficulty, expressed in nautical miles ............................................................................. 25 Figure 4.2: Mean normalized passing distances across all conditions of noise and task difficulty within each scenario, expressed in nautical miles ........................................... 26 Figure 4.3: Mean normalized track deviations across all conditions of noise and task difficulty, expressed in nautical miles ............................................................................. 26 Figure 4.4: Mean normalized track deviations across all conditions of noise and task difficulty within each scenario, expressed in nautical miles ........................................... 27 Figure 4.5: Mean overall instructor rating across all conditions of noise and task difficulty within each scenario, expressed on a scale from 1 (poor) to 5 (excellent) .................... 28 Figure 4.6: Percentage performance across all task performance measures within High Risk Event 1 within the scenarios, across all conditions of noise and task difficulty ...... 29 Figure 4.7: Percentage performance across all relevant task performance scores within High Risk Event 2 within the scenarios, across all conditions of noise and task difficulty ...................................................................................................................................... 30 Figure 4.8: Percentage performance across all relevant task performance scores within High Risk Events 1 and 2 combined, across all conditions of noise and task difficulty .. 31

Figure 4.9: Mean task performance scores for all particular actions undertaken by mariners within scenario A as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent) ................................................................................................................... 32 Figure 4.10: Mean task performance scores for all particular actions undertaken by mariners within scenario B as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent) ................................................................................................................... 33 Figure 4.11: Mean task performance scores for all particular actions undertaken by mariners within scenario C as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent) ................................................................................................................... 33

Deliverable n. 7.2

42/44

Figure 4.12: Mean task performance scores for all particular actions undertaken by mariners within scenario D as a function of noise, expressed on a scale from 1 (poor) to 5 (excellent) ................................................................................................................... 34

Figure 4.13: Mean normalized passing distances across all conditions of noise and task difficulty, expressed in nautical miles, for the first two scenarios undertaken by the mariners......................................................................................................................... 35

7.3 Abbreviations

ANOVA Analysis of variance

Bft. Wind Beaufort scale

CCTV Closed-circuit television

dB(A) Decibel, A-weighted measurement

DOLOG Doppler Speed Log

DSC Digital Selective Call

ECR Engine Control Room

F F ratio (variance across conditions divided by error variance) associated with the statistical technique Analysis of Variance

GA General Alarm

GDF General design factor

GPS Global Positioning System

HRE High Risk Event

HSW Hochschule Wismar

INS Integrated Navigation System

MSCW Maritime Simulation Centre Warnemünde

nm Nautical mile

ns not statistically significant

NUC Not under command

OS Own Ship in simulations: vessel operated by mariner

RoPax Roll-on/Roll-off Passenger Carrying Vessel

RPM Revolutions per minute [-1]

UoS University of Strathclyde

VHF Very high frequency

VLCC Very Large Crude Carrier (oil tanker)

Deliverable n. 7.2

43/44

8 Annexes

8.1 Public summary

Research within psychology has indicated that environmental factors such as noise, temperature, vibration and vehicle motion can stress an individual and lead to poorer performance of work-related tasks. Mariners are subjected to these stressors routinely and sometimes for prolonged periods (depending, for example, on the length of a ship’s voyage) and concerns have been expressed that this might make mariners more prone to errors, which in turn could lead to accidents at sea. In the shipping industry these stressors are known as Global Design Factors, because their intensity can be reduced or altered by changing a ship’s design. The research reported within this deliverable examined one of these Global Design Factors, namely noise, to investigate whether it had a negative impact on mariner performance during simulated voyages carried out on the ship’s bridge simulator at the Hochschule Wismar (HSW) in Warnemünde, Germany. Staff at HSW, in collaboration with psychologists from the University of Strathclyde, Scotland, designed a series of challenging tasks to be run on the bridge simulator. 36 experienced mariners each took part in a series of four simulated voyage segments (‘scenarios’). Two of these scenarios were conducted in quiet conditions, and two were conducted in conditions where loud (75 decibels) noises from maintenance machinery were heard on the bridge, causing distraction from the main task of navigating the ship through a busy seaway (the Dover Strait). These tasks were made to be more or less difficult by including various unexpected events within the scenarios that the mariners had to deal with, in addition to performing the basic task of safe navigation. For example, a difficult task involved a sudden failure of the ship’s steering gear, requiring a series of actions from the mariner to make the situation safer, and a less difficult task involved a breakdown of the ship’s Global Positioning System, which required fewer actions on the part of the mariner. To measure task performance, two broad sets of measures were taken. The first concerned how close to or far from another ship or a grounding hazard the mariners sailed the vessel in the simulator. The second were measures of how quickly the mariners reacted to unexpected events taking place on the bridge, such as an alarm sounding, or an equipment failure occurring, as described above. The results of the experiment indicated that, whilst noise did not have an effect on how close the mariners sailed the ship to a collision or grounding hazard, it did have an adverse effect on their reaction to unexpected on-board events such as alarms and equipment failures by slowing down the time they took to respond. The effect of noise was most strong when it was combined with the more difficult tasks within the scenarios. This suggests that noise could potentially increase the level of hazard at sea if it coincided with a high level of task demand that the mariner had to deal with. Further information: please contact Anthony Anderson School of Psychological Sciences and Health University of Strathclyde Graham Hills Building

Deliverable n. 7.2

44/44

40 George Street Glasgow G1 1QE United Kingdom e-mail: [email protected] or Knud Benedict / Gerrit Tuschling Hochschule Wismar University of Applied Sciences: Technology, Business and Design Department of Maritime Studies Richard-Wagner-Strasse 31 D-18119 Rostock/Warnemünde Germany e-mail: [email protected] / [email protected]

Human Factors in Risk-Based Ship Design Methodology · 7th Framework Programme European Commission...

Documents

Transcript of Human Factors in Risk-Based Ship Design Methodology · 7th Framework Programme European Commission...