1

Validation of a driving simulator study on driver behaviour at passive 1

rail level crossings 2

Grégoire S. Larue ab

*, Christian Wullems ac

, Michelle Sheldrake a, Andry 3

Rakotonirainy a 4

aCentre for Accident Research and Road Safety - Queensland, Queensland University of 5

Technology (QUT), Brisbane, Australia; bAustralasian Centre for Rail Innovation, 6

Canberra, Australia; cCRC for Rail Innovation, Brisbane, Australia 7

*corresponding author: 8

g.larue@qut.edu.au 9

130 Victoria Park road, Kelvin Grove 4059, Australia 10

+61 7 3138 4644 11

12

Abstract 13

Objective: The behavioural validation of an advanced driving simulator for its use 14

in evaluating passive level crossing countermeasures was performed for stopping 15

compliance and speed profile. 16

Background: despite the fact that most research on emerging interventions for 17

improving level crossing safety is conducted in driving simulator, no study has validated 18

the use of simulator for this type of research. 19

Method: We monitored driver behaviour at a selected passive level crossing in the 20

Brisbane region in Australia for three months (N=916). The level crossing was then 21

replicated in an advanced driving simulator, and we familiarised participant drivers 22

(N=54) with traversing this crossing, characterised by low road and rail traffic. 23

Results: We established relative validity for the stopping compliance and the 24

approach speed. 25

Conclusion: This validation study suggests that driving simulators are an 26

appropriate tool to study the effects of interventions at passive level crossing with low 27

road and rail traffic, which are prone to reduced compliance due to familiarity. 28

2

Application: This study also provides support for the findings of previous driving 29

simulator studies conducted to evaluate compliance and approach speeds of passive level 30

crossings. 31

32

Précis 33

Railway level crossings with stop signs require interventions to reduce road users’ 34

complacency, which can result in catastrophic crashes with trains. This study validated 35

the use of driver simulators to evaluate changes in driver behaviour when designing 36

countermeasures for such crossings. 37

38

Keywords: driving simulation; validation; railway crossing; on-road study 39

Introduction 40

Driving simulators are valuable tools in road safety and human factors research and 41

have been used to assess a variety of driving performance (Mullen, Charlton, Devlin, & 42

Bédard, 2011), as they provide a safe environment for investigating driver behaviour 43

and new interventions ethically and effectively, both in terms of cost and time. It is 44

recognised however, that a simulator will never provide an accurate reproduction of 45

reality and each simulator produces specific compromises of a real-life situation (Espié, 46

Gauriat, & Duraz, 2005). In particular, participants may not drive as they normally 47

would in a simulator for reasons such as perceiving the driving task as a game, feeling 48

motion sickness and finding the driving task not realistic. Therefore, the validation of a 49

simulator is an important component of any research study and is required for individual 50

simulators (Godley, Triggs, & Fildes, 2002). Without validation, findings from 51

simulator studies may not translate to real roads, and this could result in limited funds 52

being wasted on ineffective interventions, or worse on interventions increasing risks on 53

the road. It has been argued that simulator validity is dependent on the particular 54

simulator due to differences in equipment (low, medium, high fidelity) (Hoskins & El-55

Gindy, 2006; Nilsson, 1993), the specific driving task and the realism of its 56

3

implementation (Kaptein, Theeuwes, & Van Der Horst, 1996), participant 57

characteristics (Blana, 1996), the variables selected to measure performance, and the 58

data collection measures (Blana, 1996; Kaptein et al., 1996). These issues do have 59

implications for the generalisability of findings from one simulator to another, and even 60

from one simulator task to another. Godley et al. (2002, p. 590) suggest that 61

“accumulated evidence from different driving simulators and a range of driving tasks 62

does add weight to the validity of simulator research”. 63

An area of road safety research where simulators are now being frequently used 64

is in the assessment of driver behaviour at railway level crossings. However, to date we 65

are not aware of any published simulator validation studies in this context. 66

Background 67

Use of simulators for rail level crossing research 68

Railway level crossings represent a complex road safety control environment, where the 69

potential for road vehicle collisions that can result in multiple fatalities means they carry 70

high social and economic costs, making them a priority for safety authorities. Level 71

crossings are divided into crossings with (i) passive and (ii) active controls (Standards 72

Australia, 2015). Passive crossings are equipped with a stop or give way sign, and 73

require road users to ensure no trains are approaching before traversing the crossing. 74

Active crossings have flashing lights and sometimes boom gates, and are activated by 75

the approach of a train. Road users are not allowed to enter the level crossing while 76

flashing lights are activated. 77

Analysis of crash history data adjusted for road and rail traffic indicates that passive 78

crossings are the least safe with the greatest proportion of fatal crashes occurring at such 79

crossings (Independent Transport Safety Regulator, 2011), human factors being the 80

4

major cause of these collisions (Australian Transport Safety Bureau, 2002; European 81

Level Crossing, 2011). Despite this, the majority of Australian level crossings (80%) are 82

equipped with passive controls, mainly due to their large number (~20,000 in Australia), 83

and the cost associated with upgrading passive crossings to active protection. 84

Studies that assess changes to driver behaviour in response to a new or modified 85

level crossing warning system, can pose significant safety risks if conducted at a live 86

level crossing. For this reason, researchers often use simulation to assess driver 87

response to the proposed road user interface. Several researchers have conducted studies 88

of rail level crossings using driving simulators to assess and compare driver response 89

with different warning systems including: traffic lights versus boom gate barriers 90

(Rudin-Brown, Lenné, Edquist, & Navarro, 2012); stop signs, flashing lights and traffic 91

lights (Lenné, Rudin-Brown, Navarro, Edquist, & Trotter, 2011; Tey, Wallis, Ferreira, 92

& Tavassoli Hojati, 2011); a range of different Intelligent Transport Systems (ITS) 93

(Kim, Larue, Ferreira, Tavassoli, & Rakotonirainy, 2013; Larue, Kim, Rakotonirainy, 94

Haworth, & Ferreira, 2015); and a number of innovative warning systems (Tey, Wallis, 95

Cloete, Ferreira, & Zhu, 2012). 96

The study by Tey, Wallis, et al. (2011) is the only one to include both a field 97

study using video observations at a range of level crossings, and a replication study in a 98

driving simulator. However, they did not take advantage of the data they collected to 99

provide a quantitative validation of their simulator study, and only focused on a high 100

level discussion of the high degree of consistency between their field and simulator 101

results: high compliance for active crossings, and low compliance for passive crossings. 102

Rudin-Brown et al. (2012) assessed driver response to different types of safety signals 103

at level crossings using an advanced simulator previously validated against real-world 104

driving measures (Godley et al., 2002), but point out it is not known if and how driver 105

5

responses to simulated level crossings might differ from their responses in the real 106

world. 107

This study aims therefore to investigate the validity of advanced driving 108

simulators to assess driver behaviour, particularly compliance, at railway level crossings 109

with passive controls. 110

Establishing simulator validity 111

Two types of validity have been identified as components of simulator validation: 112

physical and behavioural (Blaauw, 1982; Blana, 1996; Jamson, 1999; Rouzikhah, King, 113

& Rakotonirainy, 2010), also described as validation of vehicle response or driver 114

response (Hoskins & El-Gindy, 2006). Physical or vehicle response validity refers to the 115

extent to which the physical components of the simulator vehicle such as the layout, 116

visual displays, and dynamics (e.g., feel of braking and steering) correspond to an on-117

road vehicle (Blaauw, 1982; Hoskins & El-Gindy, 2006; Mullen et al., 2011). 118

Behavioural or driver response validity, also known as external validity (Kaptein et al., 119

1996; Reimer, D’Ambrosio, Coughlin, Kafrissen, & Biederman, 2006), refers to the 120

degree of correspondence between driving behaviours in the simulator and on real roads 121

(Blaauw, 1982; Jamson, 1999; Mullen et al., 2011). Blaauw (1982) states that 122

behavioural validity can be established in one of two ways: either through absolute 123

validity where the numerical values from the two environments are the same; or through 124

relative validity where the differences between the two environments are in the same 125

direction, and of the same or similar size (Godley et al., 2002). 126

The majority of validated measures to date only show relative validity however; 127

Törnros (1998) argues that relative validity is satisfactory when using a driving 128

simulator as a research tool, since research questions deal with matters relating to 129

effects of various independent variables. Overall it seems to be accepted that simulators 130

6

offer a safe alternative to on-road studies and most studies support the use of simulators 131

given that they approximate (relative validity) but not exactly replicate (absolute driving 132

behaviour) on-road driving behaviour. However, validation of individual simulators for 133

each specific driving task or scenario is also important. 134

Method 135

Field study 136

Observation site 137

An observation site was selected with the following criteria: the level crossing was 138

selected with a stop sign, low road and rail traffic (250 road vehicles and 27 trains per 139

day) and with long sighting distances (~3 km) to favour familiarity and complacency. 140

Data were collected for 3 months; during that time, less than ten vehicles would have 141

approached the crossing at the same time as a train. 142

Protocol 143

Four sets of pneumatic tubes were installed at the selected site (see Figure 1). Three 144

were installed prior to the crossing (North) and one after the crossing (South), as only 145

the southbound traffic was considered in this study. Prior to the crossings, tubed were 146

installed 150 meters, 40 meters and 20 meters in advance of the crossing. The south set 147

of tubes was installed 20 meters after the crossing. The set of tubes 150 meters to the 148

crossing was installed to increase the reliability of the classification of vehicles, as well 149

as estimating the cruising speed of vehicles on the road before approaching the crossing. 150

151

152

153

154

Measuring point to classify vehicles Measuring points for speed profile and compliance

7

155

156

Figure 1: Aerial view of the Lanefield level crossing site (adapted from GoogleMaps) 157

Dependent measures 158

Speed profile 159

Speed was measured with the pneumatic tubes 150, 40 and 20 meters away from the rail 160

track. 161

Stopping compliance 162

Speed measurements from the tubes were used to estimate the stopping 163

compliance at the crossing following the methodology developed by Larue and 164

Wullems (2017). A vehicle is considered to have not complied with the stop sign at the 165

crossing if its speed did not reach zero at that point. To evaluate this incompliance, we 166

compared the time taken by the vehicle to travel between the last set of tubes before the 167

crossing and the first set of tubes after the crossing, to a critical time . The 168

critical time, computed for each vehicle, is the minimum time necessary to travel 169

between these two points at the measured speeds, while stopping at the crossing. This 170

critical time is modelled and evaluated using linear motion equations (for more details 171

see Larue and Wullems (2017)) in conditions of a typical stopping brake to come to a 172

stop (decelerating at 3.4m/s2), followed by the typical acceleration that a car is capable 173

of (2.5m/s2) as in the American Association of State Highway and Transportation 174

Officials (2011). Despite the simplicity of the approach and the strong assumptions on 175

the constancy of acceleration/deceleration, compliance rates as computed with this 176

approach have been shown to be largely insensitive to small variations of the critical 177

time (Larue & Wullems, 2017). 178

8

Driving simulator study 179

Experiment design 180

During the driving simulator session, participants drove a 6km long route six times 181

(referred to as one scenario). Each scenario took five minutes to complete. The route 182

was a typical rural Australian road with three level crossings with passive controls, and 183

two intersections (one with a stop sign, one with a give-way sign). The first level 184

crossing encountered was the replication of the observation site, at the exception of the 185

T intersection. 186

During their first drive, participants encountered a train at one of the three level 187

crossings, the selected level crossing being counterbalanced between participants. This 188

drive was used to ensure that participants would be aware that trains could travel 189

through the replicated level crossings. All five following drives did not include any 190

trains, resulting in a very low probability to encounter a train (5% chance), in line with 191

the field trial. This allowed complacency to develop without any short term increased 192

attention at level crossings shortly after seeing a train, and in a consistent manner 193

between participants. 194

Instrument 195

An advanced driving simulator composed of a complete Holden Commodore vehicle 196

with automatic transmission and working controls and instruments was used in this 197

study (see Figure 2). The simulator used SCANeRstudio software version 1.4 from 198

Oktal with eight computers, projectors and a six degree of freedom (6DOF) motion 199

platform that can move and rotate in three dimensions. When seated in the simulator 200

vehicle, the driver is immersed in a virtual environment which includes a 180 degree 201

front field of view. 202

9

203

Figure 2 Advanced driving simulator used in this study 204

Geo-specific modelling in the simulator 205

The road and environment were developed to respect Australian Standards at rail level 206

crossings. The observation site was replicated in the simulator in terms of road/rail 207

geometry and visual environment, with exception of the T intersection south of the 208

crossing. While the T intersection was removed, a set of trees on the side of the road, a 209

curve and a hill in the far distance were added in the simulated environment. This was 210

used to reduce the visibility in the distance, as the actual site had a line of trees visible at 211

the intersection (see Figure 3), and hence limit any potential influence of this change in 212

the environment on the results. The road geometry was recreated by using a map of the 213

site when the road network was created in the simulator. The road had no grade at this 214

site, except for a small grade at the level crossing (less than half a meter) and hence 215

road elevation was not replicated. Important features of the observation site were 216

replicated in the driving simulator: number and size of lanes (two lanes of 3.25 meters 217

each), road width (7.10 meters), road markings, level crossing signage, and other type 218

10

of objects in the environment (trees, few farms, fences, bush). Figure 3 shows the 219

observation site and the site as it was replicated in the simulator. 220

221

222

Figure 3 Observation site and simulated geo-specific model 223

Participants 224

Sixty participants with a valid driving license and at least one year driving experience 225

were recruited for this study. They were recruited from the general public in the 226

Brisbane region of Queensland, Australia (location of the driving simulator). Power 227

calculation demonstrated that this sample size was required to attain a power of .9 at 228

level alpha .05 with medium size effects .25 with a correlation among repeated 229

measures of .5. 230

Assuming medium effect sizes for speeds (d=.5), a .80 power and .05 type I error, and a 231

ratio of 20 between the field and simulator samples, the required sample size was 232

calculated to be 60 for the simulator group and 1,000 for the field group. 233

Six participants were not able to complete the study due to motion sickness, resulting in 234

a final sample size of 54 participants (35 males, 19 females) with a mean age of 33 ± 11 235

years (range: 19-62). Each participant was paid AUD$100 for running the experiment 236

11

(they received this incentive at the end of a second driving session which was beyond 237

the scope of this study). 238

This research complied with the American Psychological Association Code of Ethics 239

and was approved by the Ethics Committee at the Queensland University for 240

Technology QUT. Informed consent was obtained from each participant. 241

Procedure 242

Upon arrival, participants read information about the study and provided their consent 243

to participate in the study. They were then introduced to the simulator and drove a 244

familiarisation drive. 245

Once the familiarisation phase was complete, participants were instructed to 246

drive to a given house as they normally would, located six kilometres away, 247

representing an everyday life scenario. Participants drove this route six times. The only 248

difference between the scenarios was a train was approaching the crossing as the driver 249

approached the level crossing in the first scenario. Data were analysed for the last 250

scenario only, once participants were familiarised with approaching level crossings with 251

low train traffic. 252

Dependent measures 253

Speed profile 254

The approach speed was extracted at the same three points of interest as the ones used 255

in the on-road study. 256

Stopping compliance 257

The driving behaviour at the crossing was divided in three categories: drivers who 258

stopped at the crossing, drivers who slowed down but did not stop completely, and 259

12

drivers who did not attempt to stop. Compliant drivers were the ones who stopped at the 260

crossing. 261

Driver behaviour was also statistically modelled using only data that were available in 262

the field data study in order to have an objective validation of the behaviour observed in 263

the simulator: speed at the tube locations, time taken to travel between tubes, and 264

modelled critical time. 265

Validation analysis 266

The validation analysis started with evaluating whether the simulator study resulted in 267

complacency. This was done by assessing whether speed increased the more 268

participants approached level crossings. The validation analysis included then a 269

validation of the stopping behaviour at the level crossing as well as a validation of the 270

approach speed. 271

Statistical analyses were performed on the driving simulator data to identify factors 272

which could predict the behaviour of the driver at the crossing. One factor was 273

identified as predicting the different types of behaviour observed in the simulator. The 274

distribution of this factor was modelled using both the simulator and field data, and 275

allowed to calibrate the model for the field study. Behaviour rates were then compared 276

using Generalised Linear Models (GLM). 277

The statistical analyses performed to compare the speed between the road and simulated 278

environment are Generalised Linear Mixed Models (GLMM) from the Gaussian family 279

to take into account the repeated measures (at the different points of interest). 280

All statistical analyses were performed with the R software environment 281

(version 3.4.1). 282

13

Results 283

The field data collection resulted in the observation of 1,016 car traversals of the level 284

crossing. Out of these traversals, 100 occurred during periods of the day with reduced 285

luminosity (after dusk and before dawn), and were removed from the dataset. This 286

filtering was done since the simulated environment only recreated day-time driving 287

conditions. This resulted in a sample size of 916 level crossing traversals at the 288

Lanefield site. 289

Approach speeds 290

The approach speed 150 meters away from the crossing was 76.0 km/h 291

(sd=11.5) in the simulator for participants who stopped at the crossing, and 73.4 km/h 292

(sd=11.6) for drivers who did not attempt to stop (see Figure 4). On the road, these 293

speeds were 62.8 km/h (sd=11.3) and 71.3 km/h respectively (sd=9.9). 294

In the approach section where drivers decelerate to stop, the speed for drivers 40 295

meters to the crossing was 50.5 km/h (sd=11.9) in the simulator, and 39.4 km/h (sd=6.8) 296

in the field when drivers were about to stop or almost stop. Twenty 20 meters to the 297

crossing, these speeds were 40.4 km/h (sd=11.5) and 30.4 km/h (sd=6.1) respectively. 298

For drivers who did not stop at the crossing, the speed 40 meters to the crossing was 299

57.8 km/h (sd=15.8) in the simulator, and 50.7 km/h (sd=9.2) in the field. Twenty 20 300

meters to the crossing, these speeds were 58.6 km/h (sd=16.3) and 48.8 km/h (sd=12.9) 301

respectively. For complying drivers, the resulting effect size was 1.17, 1.55 and 1.53 at 302

150, 40 and 20 meters respectively. The effect size was .21, .73 and .75 at these 303

distances in the case of drivers not stopping at the crossing. Large effects were found, 304

except for the approach speed 150 meters to the crossing for uncompliant drivers, for 305

which the effect size was small. 306

14

In the driving simulation, twenty meters to the crossing, drivers who did not 307

comply with the stop sign drove at 45.1 km/h (sd=21.1) for their first run without trains, 308

and at 57.3 km/h (sd=17.1) during their last drive. A Generalised Linear Mixed Model 309

(GLMM) was fitted and it showed that they drove 28.2 km/h faster than drivers who 310

stopped at level crossings (t=9.0, df=268, p<0.001). It also showed that their speed 311

increased by 2.0 km/h for each successive drive (t=3.0, df=1935, p=0.003), resulting in 312

an overall increase of 8 km/h between the second drive (first drive without any train), 313

and the last drive, showing a development of complacency. This increase in speed was 314

not found for participants who complied at the level crossing. 315

316

Figure 4: Comparison of speed measured on site by pneumatic tubes and in the driving simulator during the 317 approach of the crossing (150, 40 and 20 meters) with standard deviations 318

319

A GLMM was fitted to evaluate the effect of the environment, the speed 320

measurement location, stopping behaviour and their interaction on speed. All factors 321

had a statistically significant effect on speed, directly or indirectly through interactions 322

(see Table 1). 323

Before slowing down for the level crossing (150 meters to the crossing), speed 324

was 71.4 km/h. Getting closer to the crossing, speed reduced by 20.5 km/h 40 meters to 325

15

the crossing (t=-57.8, df=1935, p<0.001), and by a further 22.1 km/h 20 meters to the 326

crossing (t=-62.4, df=1935, p<0.001). For drivers who stopped or almost stopped at the 327

crossing, their speed was 8.4 km/h slower than drivers who did not attempt to stop (t=-328

12.9, df=1935, p<0.001), but it was 5.5 km/h higher in the simulator (t=4.5, df=1935, 329

p<0.001). Compared to field data, speed of complying participants was 5.0 km/h higher 330

(+11%) in the simulator 40 meters to the crossing (t=2.5, df=1935, p<0.001), and 1.8 331

km/h higher (4%) 20 meters to the crossing (t=3.8, df=1935, p<0.001). 332

16

333

Table 1: Statistical model of distance and environment on speed during the approach of the crossing (150, 40 and 334 20 meters) 335

Factor/Interaction Speed

(km/h)

Standard

error

df t-value p-value

Intercept 71.4 .43 1935 164.6 p<.001

Stopping at level crossing

or almost stopping (comply) -8.4 .65

1935 -12.9 p<.001

40 meters -20.5 .35 1935

-57.8 p<.001

20 meters -22.1 .35 1935

-62.4 p<.001

Simulator:20 meters 8.9 2.74 1935

3.2 0.001

Simulator:Comply 13.9 3.1 1935

4.5 p<.001

Comply:40meters -2.9 .53 1935

-5.4 p<.001

Comply:20meters -10.2 .53 1935

-19.3 p<.001

Simulator:40meters:Comply -22.5 2.75 1935

-8.2 p<.001

Simulator:20meters:Comply -41.1 2.75 1935

-14.9 p<.001

Stopping compliance 336

In the driving simulator study, 57.4% of drivers completely stopped at the crossing, 337

20.4% almost stopped, and 22.2% did not attempt to stop to proceed through the 338

crossing (see Table 2). 339

Using linear motion equations, we found that drivers who traversed the crossing in a 340

shorter amount of time than the critical time were the drivers who did not stop at the 341

crossing. For the remaining participants, a GLMM showed that the difference between 342

the time to traverse the crossing and the critical time (referred to as amount of time over 343

17

the critical time in the reminder of the paper) was statistically related to completely 344

stopping at the crossing or not (t=4.99, df=42, p<0.001), while the following other 345

factors did not: speed before or after the crossing, deceleration before the crossing or 346

acceleration after the crossing. This difference was apparent in the distribution of the 347

amount of time over the critical time, characterised by two distinct peaks. This 348

distribution was modelled as a Gaussian mixture model using the EM algorithm. We 349

found that drivers who completely stopped at the crossing represented 76% of such 350

drivers, and took on average 7.3 seconds longer than the critical time to traverse the 351

crossing (sd=2.3), while drivers who only slowed down at the crossing took 2.4 seconds 352

more than the critical time (sd=1.4). 353

We applied this approach to the data collected in the field, and we looked at the 354

distribution of the time over the critical time for drivers who could not have stopped at 355

the crossing. We found that this distribution also had two distinct peaks and we fitted a 356

Gaussian mixture model, showing that 68% of drivers (drivers completely stopping) 357

took 5.2 seconds longer than the critical time to proceed through the crossing (sd=2.3), 358

while the remaining 32% (drivers almost stopping) took only 1.3 seconds longer than 359

the critical time (sd=1.1), resulting in the behaviour rates for the field study presented in 360

Table 2. 361

GLM shows that there was a statistically significant difference between the road and the 362

simulation rates of drivers who did not attempt to stop at the crossing (z=-4.42,p <.001), 363

with field drivers being 2.5 times more likely to traverse the crossing without 364

attempting to stop than drivers in the simulated environment. A GLM also showed that 365

there was no statistically significant difference between the road and simulator studies 366

once the drivers who did not attempt to stop were not considered (z=.01,p =.992).367

18

368

Table 2: Behaviour rates at the level crossing 369

Behaviour Rate

Environment Percentage 95% confidence interval

Simulator Stopped 57.4 ±14.8

Almost stopped 20.4 ±12.1

Did not attempt to

stop

22.2 ±11.1

Road Stopped 30.0 ±4.2

Almost stopped 14.4 ±3.2

Did not attempt to

stop

55.6 ±3.2

Discussion 370

Validity of simulator for passive level crossing behaviour evaluation 371

The aim of the current study was to validate the use of advanced driving simulators for 372

research on rail level crossings with passive controls. This validation focused on 373

approach speeds and stopping compliance. Relative validation was considered the most 374

important for validating the results of past railway crossing experiments on the 375

simulator (Kim et al., 2013; Larue et al., 2015; Larue, Rakotonirainy, & Haworth, 2016; 376

Larue, Wullems, & Naweed, 2016; Lenné et al., 2011; Tey, Wallis, Cloete, & Ferreira, 377

2013), as well as future experiments of new interventions. 378

Speed was different for each measured location, which shows that the selected 379

points of interest accurately represent the zone where drivers slow down to stop at level 380

19

crossings, as shown by previous research (Luoma & Poutanen, 2011; Moon & Coleman, 381

1999; Ng & Saccomanno, 2010; Ward & Wilde, 1996). Speeds measured in the braking 382

zone of the level crossing were statistically significantly different and absolute validity 383

could not be found in this study. However, the approach speed profile, demonstrated 384

relative validity with speed changes replicated in the simulator at similar locations as 385

on-road this study, following the same decreasing trend. Speeds measured in the 386

simulator were however consistently higher than on the real road, consistent with other 387

research on speed behaviour in driving simulators (Bella, 2005; Bittner, Simsek, 388

Levison, & Campbell, 2002; Blaauw, 1982; Blana & Golias, 2002; Godley et al., 2002; 389

Klee, Bauer, Radwan, & Al-Deek, 1999; Törnros, 1998), which is one of the most 390

commonly studied measures of behavioural validity of driving simulators. 391

We designed the study to induce complacency by repeatedly driving through a 392

level crossing with a low chance to encounter a train, and it resulted in complacency, as 393

shown by the significant proportion of participants in the simulated drive who did not 394

attempt to stop at the crossing, and their increasing approach speeds the more they were 395

traversing the crossing, showing a deliberate decision to proceed through the crossing 396

disregarding the controls in place. This behaviour is in line with both the literature and 397

the observations conducted in the present road study: the observation study by Tey, 398

Ferreira, and Wallace (2011) shows that 59% of drivers did not comply at a passive 399

level crossing with a stop sign, with 41% of drivers only slowing down, while 18% did 400

not even slow down. Compliance was more pronounced at the level crossing selected 401

for this study, as indicated by Queensland Rail, which monitored this particular crossing 402

and reported that more than half of the drivers did not stop at that particular crossing. 403

This was also observed by the research team on the multiple occasions they visited the 404

site. This is in line with the literature, which shows that drivers tend to underestimate 405

20

speeds of large objects such as trains (Clark, Perrone, & Isler, 2013; Larue et al., 2018), 406

and report that they would enter level crossings even when train are very close to level 407

crossings when sighting distances are very long (Larue et al., 2018), as at the trial site 408

considered in this study. 409

While we selected participants in Brisbane city, located 50 km to the city of 410

Ipswich where the field site was selected, the proportion of drivers who did behave this 411

way was almost half in the simulated when compared to the road study, suggesting that 412

it is difficult to obtain absolute validity. This could be due to the fact that participants in 413

a driving simulator study know that their behaviour is observed. It could also be due to 414

the fact that the sample of participants used in the study was not representative of the 415

population of drivers at the actual site: users of the local level crossing are likely to be 416

regular users of the crossing, given the low traffic at that crossing and the fact that the 417

road was going to a specific location only. However this study highlighted that the 418

simulator was very effective at inducing the non-compliance characterised by not 419

completely stopping at the crossing, suggesting that a significant proportion of 420

participants became ‘regular drivers’ of the replicated level crossing. This behaviour 421

results in drivers being more likely to assess the situation at the crossing in conditions 422

that do not give them much margin of action in case something they do not expect 423

occurs (such as the arrival of a train). 424

Our results are consistent with the only other experiment which attempted to 425

validate compliance at level crossings: Tey, Wallis, et al. (2011)’s study also resulted in 426

relative validity and managed to recreate in the simulated environment the non-427

compliance characterised by not attempting to stop. Similar to our experiment, this 428

behaviour was two times more likely on the road than in the simulator. That study was 429

however unable to recreate the difference between drivers who stopped completely, and 430

21

the participants who only slowed down: most participants in their simulator study 431

completely stopped, while their field observations showed the same proportions for both 432

behaviours. Multiple reasons could play part in this result, including the lack of 433

replication of an actual level crossing in the simulator and the lower fidelity of the 434

simulator used (static simulator with one screen), or the lack of repetition of driving 435

through the level crossing to familiarise the participants with the crossing and trigger 436

complacent behaviour. 437

Our study has implications for the evaluation of interventions in driving 438

simulators. We have shown that the complacent driver behaviour at passive level 439

crossing can be recreated in the simulator, whether this behaviour is deliberate (driving 440

through without slowing down), or more as an error, failing to completely stop. This 441

shows that a range of driver behaviour issues can be evaluated using simulators, from 442

errors to actual violations. Deliberate violations are however less likely in a simulator as 443

compared to an actual level crossing, and this should be considered when designing the 444

sample size of a simulator study. 445

Strengths and limitations 446

The present study used a unique validation process for the use of driving 447

simulators to evaluate driver behaviour at passive level crossings through the replication 448

of a level crossing and naturalistic data collection in the field. This approach 449

complements the literature on the validity of simulators to a new driving task, and 450

provides supporting evidence to the research conducted in simulator for level crossings 451

interventions. A strength of this study is the use of a large sample size of participants in 452

the simulator study (N=54). Further, we used a methodology (pneumatic tubes) to 453

observe behaviour in the field which is unlikely to have had an effect on driver 454

compliance (Larue & Wullems, 2017), as this technology cannot be used for 455

22

enforcement and is largely perceived as a traffic monitoring device. This allowed us to 456

have an estimation of on-road compliance over a long period of time. 457

There were, however, some limitations of the present study that should be 458

considered when interpreting the results. First, we could not obtain demographics 459

information about the drivers in the field study. This does not allow confirming whether 460

our sample of simulator participants is representative of the drivers using that particular 461

level crossing. However, we used a sample of participants representing the general 462

driving population both in terms of gender and age. Further, participants in the 463

simulator were accustomed to the level crossing by driving multiple times through it, 464

resulting in complacency and driving behaviour similar to the one observed from 465

regular level crossing users. This study was also unable to validate the behaviour of 466

participants as a train approaches a level crossing. However, errors and violations when 467

trains are approaching are related to the complacency at level crossings (Edquist, 468

Stephan, & Wigglesworth, 2009), and therefore validating a simulator when no train is 469

approaching can be sufficient to determine whether interventions are likely to improve 470

behaviour at passive level crossings. The estimations of on-road compliance and speed 471

profile was also coarse with the approach used in this study. However, we collected 472

speed data at locations known to be where speed changes, and our results are in line 473

with other field studies at level crossings, with speed reductions around 50 meters to the 474

crossing (Luoma & Poutanen, 2011; Moon & Coleman, 1999; Ng & Saccomanno, 475

2010; Ward & Wilde, 1996). The estimation of compliance with pneumatic tubes, while 476

relying on a number of coarse assumptions, is still robust to variations of timing 477

measurements, as shown by sensitivity analysis (Larue & Wullems, 2017). 478

23

Future directions 479

This study has shown that simulators can be effective at inducing complacency, with a 480

significant number of participants disregarding controls in place. Further research is 481

needed to understand how the likelihood of seeing a train at level crossing impacts 482

complacency in the field, and how this translates in simulated driving. Future work 483

should also investigate how complacency develops when repeating the same action 484

repeatedly, and what amount of repetition is required in the simulator to result in a 485

similar behaviour, given that simulation is conducted over a much shorter period of time 486

compared to habituation in the field. Qualitative research is also warranted in order to 487

further understand the mental models of drivers both in the field and in the simulator, 488

which may explain partly the differences observed between field and simulated driving 489

in this study. 490

Conclusion 491

This study provides evidence that compliance and approach speeds obtained from an 492

advanced driving simulator are valid measures of driver behaviour at passive level 493

crossings. Such measures were found to be similar to the ones measured in-situ at the 494

level crossing replicated in the simulated environment. This validation study suggests 495

that driving through replicated level crossings with low chance of encounter with a train 496

in an advanced driving simulator is an adapted approach to study the effects of 497

interventions at level crossing with passive controls with low road and rail traffic, which 498

are prone to reduced compliance due to familiarity. This study also provides good 499

support for the previous studies conducted in this field with driving simulators. 500

Key points 501

Most research on emerging interventions for improving level crossing safety is 502

conducted in driving simulator 503

24

No study has validated the use of simulator for this type of research 504

We measured speed and compliance at a level crossing and its replica in a simulator 505

Our study shows relative validity for approach speed and compliance and validates the 506

use of simulators for passive level crossings 507

Acknowledgements 508

The authors gratefully acknowledge the CRC for Rail Innovation (established and supported 509

under the Australian Government’s Cooperative Research Centres program) and Queensland 510

Rail for the funding of this research. Project No. R2.195. 511

References 512

American Association of State Highway and Transportation Officials. (2011). A Policy 513

on Geometric Design of Highways and Streets (6th ed.) (6th ed.). Washington 514

D.C.: American Association of State Highway and Transportation Officials. 515

Australian Transport Safety Bureau. (2002). Monograph 10. level crossing accidents. 516

ACT: Commonwealth Department of Transport and Regional Services. 517

Bella, F. (2005). Validation of a Driving Simulator for Work Zone Design. 518

Transportation Research Record: Journal of the Transportation Research 519

Board, 1937, 136-144. doi:10.3141/1937-19 520

Bittner, A., Simsek, O., Levison, W., & Campbell, J. (2002). On-Road Versus 521

Simulator Data in Driver Model Development Driver Performance Model 522

Experience. Transportation Research Record: Journal of the Transportation 523

Research Board, 1803, 38-44. doi:10.3141/1803-06 524

Blaauw, G. J. (1982). Driving experience and task demands in simulator and 525

instrumented car: a validation study. Human Factors, 24(4), 473-486. 526

Blana, E. (1996). Driving Simulator Validation Studies: A Literature Review. Working 527

Paper. Institute of Transport Studies, University of Leeds, Leeds, UK. 528

Blana, E., & Golias, J. (2002). Differences between Vehicle Lateral Displacement on 529

the Road and in a Fixed-Base Simulator. Human Factors, 44(2), 303-313. 530

doi:10.1518/0018720024497899 531

Clark, H. E., Perrone, J. A., & Isler, R. B. (2013). An illusory size–speed bias and 532

railway crossing collisions. Accident Analysis & Prevention, 55(0), 226-231. 533

doi:http://dx.doi.org/10.1016/j.aap.2013.02.037 534

Edquist, J., Stephan, K., & Wigglesworth, L. M. (2009). A literature review of human 535

factors safety issues at Australian level crossings. Retrieved from Melbourne: 536

Espié, S., Gauriat, P., & Duraz, M. (2005). Driving simulators validation: The issue of 537

transferability of results acquired on simulator. Paper presented at the Driving 538

Simulator Conference, Orlando, US. 539

European Level Crossing, F. (2011). Background Information on Level Crossings. 540

About Level Crossings, 2011(28th April 2011). 541

Godley, S. T., Triggs, T. J., & Fildes, B. N. (2002). Driving simulator validation for 542

speed research. Accident Analysis & Prevention, 34(5), 589-600. 543

Hoskins, A. H., & El-Gindy, M. (2006). Technical report: Literature survey on driving 544

simulator validation studies. International Journal of Heavy Vehicle Systems 545

(IJHVS), 13(3). doi:10.1504/IJHVS.2006.010020 546

25

Independent Transport Safety Regulator. (2011). Transport Safety Bulletin: Level 547

crossing accidents in Australia. 548

Jamson, H. (1999). Curve negotiation in the Leeds driving simulator : a validation 549

study. Engineering psychology and cognitive ergonomics; 3; 351-358, 3. 550

Kaptein, N., Theeuwes, J., & Van Der Horst, R. (1996). Driving simulator validity: 551

Some considerations. Transportation Research Record: Journal of the 552

Transportation Research Board, 1550, 30-36. doi:10.3141/1550-05 553

Kim, I., Larue, G. S., Ferreira, L., Tavassoli, A., & Rakotonirainy, A. (2013, 2 - 4 554

October 2013). Evaluating ITS interventions at railway level crossings using a 555

driving simulator. Paper presented at the Australasian Transport Research 556

Forum, Brisbane, Australia. 557

Klee, H., Bauer, C., Radwan, E., & Al-Deek, H. (1999). Preliminary Validation of 558

Driving Simulator Based on Forward Speed. Transportation Research Record: 559

Journal of the Transportation Research Board, 1689, 33-39. 560

Larue, G. S., Filtness, A. J., Wood, J. M., Demmel, S., Watling, C. N., Naweed, A., & 561

Rakotonirainy, A. (2018). Is it safe to cross? Identification of trains and their 562

approach speed at level crossings. Safety Science, 103, 33-42. 563

doi:https://doi.org/10.1016/j.ssci.2017.11.009 564

Larue, G. S., Kim, I., Rakotonirainy, A., Haworth, N. L., & Ferreira, L. (2015). Driver’s 565

behavioural changes with new intelligent transport system interventions at 566

railway level crossings—A driving simulator study. Accident Analysis and 567

Prevention, 81, 74-85. doi:10.1016/j.aap.2015.04.026 568

Larue, G. S., Rakotonirainy, A., & Haworth, N. L. (2016). A simulator evaluation of 569

effects of assistive technologies on driver cognitive load at railway level 570

crossings. Journal of Transportation Safety & Security, 8(Supplement 1), 56-69. 571

doi:10.1080/19439962.2015.1055413 572

Larue, G. S., & Wullems, C. (2017). A new method for evaluating driver behaviour and 573

interventions for passive railway level crossings with pneumatic tubes. Journal 574

of Transportation Safety & Security, 0-0. doi:10.1080/19439962.2017.1365316 575

Larue, G. S., Wullems, C., & Naweed, A. (2016). Evaluation of a new level crossing 576

warning concept to improve safety of level crossings in remote locations. Paper 577

presented at the 11th World Congress on Railway Research, Milan, Italy. 578

http://eprints.qut.edu.au/95767/ 579

Lenné, M. G., Rudin-Brown, C. M., Navarro, J., Edquist, J., & Trotter, M. (2011). 580

Driver Behaviour at Rail Level Crossings: Responses to Flashing Lights, Traffic 581

signals and Stop Signs in Simulated Rural Driving. Applied Ergonomics, 42, 582

548-554. 583

Luoma, J., & Poutanen, M. (2011). How Drivers Understand Safe Behaviour and 584

Perceive Risks at Passive Railway-Road Level Crossings. The Open 585

Transportation Journal, 5, 88-91. doi:10.2174/1874447801105010088 586

Moon, Y., & Coleman, F. (1999). Driver's Speed Reduction Behavior at Highway-Rail 587

Intersections. Transportation Research Record: Journal of the Transportation 588

Research Board, 1692(1), 94-105. doi:10.3141/1692-11 589

Mullen, N., Charlton, J., Devlin, A., & Bédard, M. (2011). Simulator Validity 590

Handbook of Driving Simulation for Engineering, Medicine, and Psychology: 591

CRC Press. 592

Ng, O. K., & Saccomanno, F. F. (2010). Speed Reduction Profiles Affecting Vehicle 593

Interactions at Level Crossings with No Trains. Transportation Research 594

Record: Journal of the Transportation Research Board, 2149(1), 108-114. 595

doi:10.3141/2149-13 596

26

Nilsson, L. (1993). Behavioural research in an advanced driving simulator-experiences 597

of the VTI system. Paper presented at the Human Factors and Ergonomics 598

Society Annual Meeting, Seattle, US. 599

Reimer, B., D’Ambrosio, L. A., Coughlin, J. F., Kafrissen, M. E., & Biederman, J. 600

(2006). Using self-reported data to assess the validity of driving simulation data. 601

Behavior Research Methods, 38(2), 314-324. doi:10.3758/bf03192783 602

Rouzikhah, H., King, M., & Rakotonirainy, A. (2010). The validity of simulators in 603

studying driving behaviours. Paper presented at the 2010 Australasian Road 604

Safety Research, Policing and Education Conference, Canberra. 605

http://eprints.qut.edu.au/38011/ 606

Rudin-Brown, C. M., Lenné, M. G., Edquist, J., & Navarro, J. (2012). Effectiveness of 607

traffic light vs. boom barrier controls at road–rail level crossings: A simulator 608

study. Accident Analysis & Prevention, 45, 187-194. 609

doi:http://dx.doi.org/10.1016/j.aap.2011.06.019 610

Standards Australia. (2015). Manual of Uniform Traffic Control Devices, Part 7: 611

Railway Crossings. (AS 1742.7-2015). Sydney, Australia: Standards Australia. 612

Tey, L.-S., Ferreira, L., & Wallace, A. (2011). Measuring driver responses at railway 613

level crossings. Accident Analysis & Prevention, 43(6), 2134-2141. 614

Tey, L.-S., Wallis, G., Cloete, S., & Ferreira, L. (2013). Modelling driver behaviour 615

towards innovative warning devices at railway level crossings. Accident 616

Analysis & Prevention, 51(0), 104-111. 617

doi:http://dx.doi.org/10.1016/j.aap.2012.11.002 618

Tey, L.-S., Wallis, G., Cloete, S., Ferreira, L., & Zhu, S. (2012). Evaluating Driver 619

Behavior Toward Innovative Warning Devices at Railway Level Crossings 620

Using a Driving Simulator. Journal of Transportation Safety & Security, 5(2), 621

118-130. doi:10.1080/19439962.2012.731028 622

Tey, L.-S., Wallis, G., Ferreira, L., & Tavassoli Hojati, A. (2011, 28-30 September 623

2011). Using a driving simulator to assess driver compliance at railway level 624

crossings. Paper presented at the 34th Australasian Transport Research Forum 625

2011, Adelaide, SA, Australia. 626

Törnros, J. (1998). Driving behaviour in a real and a simulated road tunnel—a 627

validation study. Accident Analysis & Prevention, 30(4), 497-503. 628

doi:http://dx.doi.org/10.1016/S0001-4575(97)00099-7 629

Ward, N. J., & Wilde, G. J. S. (1996). Driver approach behaviour at an unprotected 630

railway crossing before and after enhancement of lateral sight distances: An 631

experimental investigation of a risk perception and behavioural compensation 632

hypothesis. Safety Science, 22(1-3), 63-75. doi:10.1016/0925-7535(96)00006-9 633

634

Biographies 635

Gregoire S. Larue, senior research fellow at CARRS-Q, PhD in mathematics from QUT 636

(Australia) obtaind in 2010 637

Christian Wullems, research fellow at CARRS-Q, PhD in computer science from QUT 638

(Australia) obtained in 2005 639

Michelle Sheldrake, senior research officer at CARRS-Q, MSc in psychology from 640

University of Otago in 1992

641

27

Andry Rakotonirainy, professor at CARRS-Q, PhD in computer science obtained from 642

Universite de Paris VI (France) in 1995 643