Modelling disaster resilience: enhancing student learning ... Meding_Report_2016.pdf · simulation...

Modelling disaster resilience: enhancing student learning through trans-disciplinary simulation of wicked scenarios (RES-SIM)

Final report 2016

Lead University: The University of Newcastle

Partner University: RMIT University

Dr Jason von Meding

Associate Professor Vanessa Cooper

Dr Sittimont Kanjanabootra

Dr Helen Giggins

Dr Giuseppe Forino

Jai Allison

Support for the production of this report has been provided by the Australian Government Office for Learning and Teaching. The views expressed in this report do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.

With the exception of the Commonwealth Coat of Arms, and where otherwise noted, all material presented in this document is provided under Creative Commons Attribution-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-sa/4.0/. The details of the relevant licence conditions are available on the Creative Commons website (accessible using the links provided) as is the full legal code for the Creative Commons Attribution-ShareAlike 4.0 International License http://creativecommons.org/licenses/by-sa/4.0/legalcode. Requests and inquiries concerning these rights should be addressed to: Learning and Teaching Support Student Information and Learning Branch Higher Education Group Department of Education and Training GPO Box 9880 Location code C50MA7 CANBERRA ACT 2601 <[email protected]> 2016 ISBN 978-1-76028-912-6 [PDF] ISBN 978-1-76028-911-9 [PRINT] ISBN 978-1-76028-913-3 [DOCX]

http://creativecommons.org/licenses/by-sa/4.0/

http://creativecommons.org/licenses/by-sa/4.0/legalcode

mailto:[email protected]


3

Acknowledgements We would like to thank the educators and practitioners that contributed to the various stages of our research by willingly giving of their time and sharing their experiences, as well as the organisations that hosted and facilitated our interviews, focus groups and workshops.


4

Glossary ● Concept map - A visual representation that illustrates the relationships between

concepts and ideas. Concepts are typically represented in circles or boxes and linked by words and phrases that explain the connection between the ideas.

● Developers/Software developers - Those who create a simulator from a brief provided to them by the RES-SIM team (such as Crisisworks).

● Dimension - One of the eleven dimensions provided in the conceptual framework. ● Framework - The framework based on Gaba (2004) and AFAC (2014). ● Keyword - As defined in VUE to ‘tag’ the interview maps. ● Map - 2D representation of attributes constituting something (e.g. an area of the

earth, a concept). ● Node - The ‘bits’ of the maps or systems in each visualisation. ● Project team – Academic investigators and research assistants on RES-SIM project. ● Scenario - An internally consistent view of what the future might turn out to be, not

a forecast. ● Simulation - A technique used to replace or amplify real experiences with guided

experiences that evoke or replicate substantial aspects of the real world. ● System - The conceptual representation of reality in its totality that participants in a

simulation should experience. The system includes five system components (see “System Components”).

● System components - The elements that interact to form a dynamic system for this project, including: Social, economic, environmental, infrastructure and governance.

● System designers - The designers of the system, in this case the RES-SIM team. The designers determine what is needed in the system for the purposes of the project through engagement with stakeholders and collaborate with software developers to create a virtual model of the system.

● System maps - The conceptual assemblage of different components that aim to represent a sub-set of, or the entire, reality that the participants will experience when using RES-SIM.


5

List of acronyms used AFAC Australasian Fire & Emergency Service Authorities Council

AIIMS Australasian Inter-Service Incident Management System

DRR Disaster Risk Reduction

GIS Geographic Information System

IT Information Technology

NGOs Non-government Organisations

NSW New South Wales

OLT Office for Learning and Teaching

RES-SIM Resilience Simulator

RFS Rural Fire Service

RMIT Royal Melbourne Institute of Technology

SES State Emergency Service (of NSW)

UN United Nations

UNISDR United Nations Office for Disaster Risk Reduction

UON The University of Newcastle

VUE Virtual Understanding Environment


6

Executive summary

Project Background and Rationale

The RES-SIM project is collaboration between The University of Newcastle and RMIT University. It aims to develop a conceptual model for a virtually distributed computer-based teaching and learning tool. Through an inter- and multi-discipline approach for both on and off campus students, this tool will enable students to acquire essential decision making skills for disaster management and to apply collaborative approaches through a dynamic disaster system simulation. This idea is mainly derived from established theories such as game theory and system theory and thinking.

A number of destabilising variables, ranging from the immediate impacts of disasters (of natural or anthropogenic origin) on various system components to the subsequent responses of decision-makers, increase the vulnerability of societal systems and subsystems (e.g. health systems, transport systems, political systems). In many fields, including disaster response, simulations are generally based on face-to-face resource intensive scenarios or involve ‘event-based’ simulations. However, these simulations fail to engage the full range of hazard affected social systems, which are essential for function (for example infrastructure, local economies or social networks). Therefore, while higher education provides students theoretical knowledge of complex systems, it lacks of effective ways to promote tangible experience. Phase 2 of the project (beyond the scope of this Seed project) will create a simulation tool that recognises these dynamics, while allowing the ‘game’ controller the flexibility to manipulate the conditions during the simulation itself to mimic the complex nature of disaster scenarios. This will allow for the creation of an environment that ensures students experience collaboration with an embedded conceptual learning.

In higher education, tools such as scenarios and problem-based learning environments are widely used to facilitate the consolidation of knowledge along the university programs and encourage students to apply critical thinking and synthesis to solve problems (Zoakou, Tzanavari, Popadopoulos & Sotiriou, 2007). Such tools also enable students to interact with each other and with the scenario context (Trillaud, Pham, Rabah, Estraillier & Malki, 2012; Taylor & Evans 2005) in achieving learning outcomes (Jinks, Norton, Taylor & Stewart, 2011). In a pedagogical context, scenario tools have been adopted by many disciplines, including health (Schultz, Koenig, Whiteside, Murray, & National Standardized All-Hazzard Disaster Core, 2012), business (Buytendijk, Hatch & Micheli, 2010), aviation (Schwaitzberg, Godinez, Kavic, Sutton, Worthington, Colburn & Park, 2009) and disaster management (Jinks Norton, Taylor & Stewart, 2011). This demonstrates that many attributes of these scenarios are universal and interchangeable across disciplines. Unlike some existing commercial emergency management ‘event’ simulators, RES-SIM takes a ‘whole-system’


7

approach to enable students to hone their judgement on decision-making in a safe environment and provides valuable feedback based on engineering, social and economic-based system dynamics. Therefore, RES-SIM greatly contributes to Australian higher education and emergency management training by promoting effective and efficient online learning. This prepares students for work and provides them with valuable decision-making skills to be applied in complex real-world scenarios.

Project Approach

Stage 1- Expert interviews with educators and practitioners: Semi-structured interviews were conducted with educators and practitioners in disaster management agencies. The interview findings primarily contributed to Outcomes 1-4 and informed the design of the focus groups for the scenario development and guided discussions with software developers.

Stage 2- System Mapping: To create the visual representations of concepts and ideas pertaining to the system components under different scenarios three workshops were undertaken with community and business leaders, representatives from the emergency management sector and academics with experience in disaster management in Newcastle, Sydney and Melbourne. These workshops contributed to development to Outcomes 1-5.

Stage 3- Scenario development: Three scenarios were developed through focus groups sessions with representatives from organisations working within that specific disaster context, namely 1) a flood scenario; 2) an economic shock and 3) a bushfire scenario. The scenario development focus groups primarily contributed to Outcomes 1, 3, 5 with secondary contributions to Outcomes 2 and 4.

Recommendations

• Scenarios in disaster resilience should be designed with the eleven scenario dimensions (AFAC 2014) in mind to maximise their effectiveness

• It is critical that system simulators emulate real-life experiences; such experiences can be supported via choices around scenarios design (e.g. use of real historical data); and type of technology employed to underpin the system simulator (e.g. virtual reality)

• Scenarios in disaster resilience are inherently complex and multi-disciplinary. They should incorporate the interdependencies of various system elements (i.e. social, economic, environment, infrastructure, governance)

• Disaster resilience simulations need to be flexible and customisable because even the same disaster type (e.g. bushfire), in the same location at a different point in time can


8

impact elements of the system differently. Simulation systems need to factor in these complexities and enable to game controller to vary the impact of student decision-making processes

• System simulators should be developed using an iterative rather than ‘big bang’ approach so that the way in which instructors and students use the system in practice can inform design improvements and ultimately ensure the simulation is better integrated into the classroom environment and wider curricula, including approaches to assessment and feedback.

Current and Future Benefits to stakeholders:

Educators: As the higher education sector evolves, educators in all disciplines must capture the imagination of students through new methods of teaching and engagement in a globally competitive environment. This project has developed the conceptual model that can be used to underpin the development of a virtual, trans-disciplinary tool that will deliver a blended learning experience of the highest calibre and which could be utilised by educators in multiple disciplines.

Students: The student of today needs more than a theoretical decision-making basis; he or she requires an experience that feels like the real thing and one that nurtures superior judgement to be applied in a professional context. The RES-SIM project has laid the foundation for a system that prepares students for work and equips them with valuable decision-making skills to that can be applied in complex real-world scenarios.

Industry: The project team has worked with industries to identify system and subsystem variables and the relationships between them, as well as acquired a deep understanding of the scenarios that may transpire in reality. The conceptual framework developed in this project will underpin the learning tool delivered in Phase 2 and in turn benefit industry by ensuring that graduates are adequately prepared and more capable in their chosen profession.


9

Table of Contents ACKNOWLEDGEMENTS 3

GLOSSARY 4

LIST OF ACRONYMS USED 5

EXECUTIVE SUMMARY 6

TABLES AND FIGURES 10

TABLES 10 FIGURES 10

CHAPTER 1 - CONTEXT 11

NEED FOR THE RES-SIM PROJECT 11 AIMS AND OBJECTIVES 11 LITERATURE REVIEW 12

CHAPTER 2 - APPROACH AND METHODS 16

EXPERT INTERVIEWS WITH EDUCATORS AND PRACTITIONERS 16 SYSTEM MAPS DEVELOPMENT VIA WORKSHOPS 17 SCENARIO DEVELOPMENT VIA FOCUS GROUPS 18

CHAPTER 3 - FINDINGS 19

STAGE 1: EXPERT INTERVIEWS WITH EDUCATORS AND PRACTITIONERS 19 STAGE 2 - RESULTS AND ANALYSIS OF WORKSHOPS 22 STAGE 3 - RESULTS AND ANALYSIS OF FOCUS GROUPS 24 MAPPING OF RESULTS AGAINST FRAMEWORK 26 CONCEPTUAL SYSTEM MODEL 28 KEY MESSAGE: THE NEED FOR REAL LIFE EXPERIENCES 29

CHAPTER 4 - EVALUATION 31

OUTCOMES 31 LIMITATIONS AND FUTURE WORK 33 DISSEMINATION 33 PROJECT IMPACT AND IMPLICATIONS FOR HIGHER EDUCATION 34

REFERENCES 36

APPENDIX A 38

APPENDIX B 39


10

Tables and figures

Tables Table 1 Scenario dimensions and design considerations (based on AFAC, 2014) ................. 28

Figures Figure 1 RES-SIM research approach ..................................................................................... 16

Figure 2 Example results of coding Simulation & Scenario theme from interviews .............. 19

Figure 3 Workshop participants present their system maps to the group ............................ 23

Figure 4 System dynamics model representing components affecting SES efficiency to complete call out jobs during a disaster ................................................................................ 24

Figure 5 A sample scenario developed from a focus group map ........................................... 25

Figure 6 Conceptual System Model........................................................................................ 29

Figure 7 Aspirational project outcomes ................................................................................. 31


11

Chapter 1 - Context

Need for the RES-SIM Project There is an expanding market for courses and degree programs at Australian institutions in the disaster realm, both at undergraduate and masters level, from various disciplinary perspectives such as health/humanitarian, emergency management and the built environment. Students operating in the disaster realm, as future disaster responders, will confront hazards that inevitably destabilise systems and subsystems. Students thus require an understanding of the relationships between system components, their value and their external influences in order to provide a basis for sound judgement and decision-making.

Yet current graduates emerging from university study are not doing so with the ability to make sense of complex systems (Wilensky and Stroup 2013) or operate efficiently in a trans-disciplinary environment (Remington-Doucette, Hillier Connell, Armstrong & Musgrove, 2013). This problem requires immediate attention given the potential life changing (or ending) outcomes of poor decision-making in the disaster realm. While learning in an interactive and collaborative environment is becoming an expectation across the education sector many disciplines are still underdeveloped in terms of the tools and techniques utilised for teaching and learning in the trans-disciplinary sphere. The RES-SIM project set out to address such challenges in the education and training of disaster responders, particularly in terms of their understanding of whole-of-system dynamics (Simonovic, 2011).

Aims and objectives The RES-SIM project was designed to develop the conceptual model for a virtually distributed scenario-based teaching and learning tool that enables students within and across disciplines (e.g. engineering, architecture, logistics), both on and off campus, to collaboratively acquire essential decision-making skills through immersion in a dynamic disaster system simulation. The concept stems from game theory, competition theory and system theory. Societal systems and subsystems (e.g. health systems, transport systems, political systems) are vulnerable to a range of destabilising variables, from the immediate impacts of disasters (natural or man-made) on various system components to the subsequent responses of decision-makers.

In many fields, including disaster response, simulations generally rely upon face-to-face, resource intensive scenarios or involve ‘event-based’ simulations, which fail to fully engage the systems of society that are impacted by shocks and hazards. Students are emerging from higher education with theoretical knowledge of complex systems but little in the way of tangible experience. Phase 2 of the RES-SIM project (beyond the scope of this grant) will create a simulation tool that recognises these dynamics, while allowing the ‘game’ controller the flexibility to manipulate the conditions during the simulation itself to mimic


12

the chaotic nature of disaster scenarios. This will create an environment that yields rich participatory experiences for students and embedded conceptual learning.

Literature Review Disasters are among the most urging pressures for social systems. The occurrence of disaster-triggering events reflects the nonlinear, emergent and sometimes chaotic behaviour of complex disaster (Davies 2015). According to UNISDR (2009, p. 9), disaster is “a serious disruption of the functioning of a community or a society involving widespread human, material, economic or environmental losses and impacts, which exceeds the ability of the affected community or society to cope using its own resources”. A disaster results from the combination of: exposure to a hazard; the conditions of vulnerability; and insufficient capacity or measures to reduce or cope with the potential negative consequences. Disaster impacts may include loss of life, injury, disease and other negative effects on human physical, mental and social well-being, together with damage to property, destruction of assets, loss of services, social and economic disruption and environmental degradation (Simonovic, 2011).

Given these impacts by disasters, several frameworks have been used to discuss disasters within policies and actions from international to the local level. Currently, one of the most common frameworks is resilience. Notwithstanding within disaster studies it is still a contested term with no clear definition, variables, or components, resilience has gained consensus among disaster scholars and practitioners over recent decades. While many definitions of disaster resilience exist, a common one has been provided by the UNISDR (2009). Accordingly, disaster resilience is “the ability of a system, community or society exposed to hazards to resist, absorb, accommodate to and recover from the effects of a hazard in a timely and efficient manner, including through the preservation and restoration of its essential basic structures and functions” (UNISDR, 2009, p. 24). Recent research trends are addressing disaster resilience in social systems in connections with place-based characteristics, variables, and resources that enable the system to not only cope with, but also adapt to current or expected disaster impacts (Coetzee, Van Niekerk & Raju, 2016). Social systems develop their own mechanisms that make these systems flexible enough to both adapt and respond to disasters. Therefore, incorporating adaption in disaster resilience has the potential to consider resilience as a flexible and continuous process, rather than a stable outcome, which adapts to changing conditions (Manyena 2006; Coetzee et al. 2016).

Disaster resilience can be analysed by adopting a Complex Adaptive System (CAS) perspective (Comfort, Sungu, Johnson, & Dunn, 2001; Coetzee et al. 2016) which describes behaviours of social systems in case of disaster and explains the interacting structures and patterns arising through simple but powerful rules driving change (Folke, 2006). Originally theorised and developed from ecological science (Levin 1998) and applied to socio-ecological systems (Walker, Holling, Carpenter & Kinzig, 2004; Folke, 2006), CAS finds space


13

also within disaster resilience framework. For Coetzee et al. (2016, pp. 204-205, and references therein) five characteristics of CAS can be translated within a disaster resilience framework. First, CAS is non-linear as the size of its inputs might not be proportional to expected outputs. A non-linearity lens to disaster resilience allows tracking potential impacts each variable has on the overall resilience in social systems. Second, through aggregation individuals organise themselves into sub-groups or hierarchal organisations establishing multiple interactions. In disaster resilience, aggregation allows focusing on the role, correlation, and contribution of coping mechanisms to the overall resilience. Third, properties, characteristics, and patterns of CAS emerge from the interaction between individual elements. The concept of emerging behaviour helps investigating the capacity by smaller aggregated variables of improving systems’ resilience. Fourth, feedback loops enhance, stimulate, detract, or inhibit elements in CAS. They allow for learning and adaptation within a dynamic environment thereby preventing the extinction of a system, and indicate the capacity of systems for improving resilience through learning from past events. Fifth, CAS is embedded within a unique context. Context is not static and can be altered by dynamic interactions between elements. Context-based responses explore how aggregated unique elements make a system resilient, and changes at lower levels can modify the wider context of resilience (Coetzee et al. 2016, pp. 204-205, and references therein).

Therefore, it is critical that higher education explores the multiple dimensions of disaster resilience to understand how social systems react to disasters. This allows that graduates, as future disaster responders, are prepared as individual and/or practitioners. The next years will pose new opportunities and challenges for teaching and learning disaster resilience in a higher education context (Haigh and Amaratunga 2015). This is particularly relevant for Australia, a country that faces significant and ongoing threat to life and property from a wide range of hazards.

Promoting a systems thinking able to address multiple and interconnected issues becomes essential for disaster-related disciplines in higher education. Richmond (1993) claims that the adoption of a systems thinking within the learning process allows students to understand how things work in the real-world and evolve as a consequence of shifts in the strengths of their dynamic interdependences. Systems can be defined as an interdependent group of items forming a unified pattern, so almost anything consisting of two or more components can be considered a system (e.g. a tree, an economy, a digestive system) (Simonovic 2011).

Digital tools such as computers, software, and virtual games are essential to support a learning process underpinned by systems thinking (Van Niekerk, Coetzee, Botha, Murphree, Fourie, Le Roux & Meyer, 2015). Digital tools allow compressing space and time and serve as personal theatres in which virtual realities can be played out. Students experience moving into space and time and are intellectually stimulated to explore


14

complexity and finding solutions to be potentially applied into the “real world” (Richmond 1993). Within digital tools, simulation technique represents a useful way to improve experience and decision-making capacities to be used in the real world. For Gaba (2004), simulation represents a technique that can be used to replace or amplify real experiences with guided experiences that evoke or replicate substantial aspects of the real world in a fully interactive manner. For Hopwood, Rooney, Boud, & Kelly (2014) simulation refers to any kind of model or device used to bring elements of one reality into another (e.g., drawings of anatomy simulate a body).

In higher education extant literature has explored the use of simulation for particular professional fields (Hopwood et al. 2014), as in management (Salas, Wildman & Piccolo, 2009), project management for engineering (Zwikael, Shtub & Chih,. 2013), surgery (Mann, Eidelson, Fukuchi, Nissman, Robertson & Jardines, 2002), as well as emergency management (Van Niekerk et al. 2015). Simulation represents an opportunity to integrate different aspects of curricular content that are learned separately elsewhere, to provide an environment where mistakes can be made safely with no negative consequences, to standardise learning experiences, and to address perceptions that university-based learning lacks authenticity or relevance in practice (Gonczi 2013). A survey on higher education students using simulation techniques in the field of homeland security reports that the majority of the students agreed that the simulation allowed them to improve their understanding of the course material, provided a hands on experience, mirrored a real world experience and enhanced their decision-making skills (Renda-Tanali and Abdul-Hamid 2011).

Simulations are commonly resource intensive scenarios, based on face-to-face techniques, which are widely used to facilitate the consolidation of knowledge from previous lessons and encourage students to critically apply knowledge to solve problems (Zoakou et al. 2007). Scenario-based learning engages learners in interactive scenarios as active participants. Students are required to make decisions, which can offer various future options in the scenario. Students are also allowed to better understand course material and policy implications, as well as to be able to demonstrate their application to real world situations (Renda-Tanali and Abdul-Hamid 2011).

A scenario is an internally consistent view of what the future might turn out to be, not a forecast (Buytendijk et al. 2010). Therefore, a scenario does not provide a “right” vision of the future or as a single plan to refer to. Rather, it helps thinking about the future as a range of options to be recognised, and could be used to be prepared for what might happen in the future (Buytendijk et al. 2010). For example, scenarios explore the joint impact of various uncertainties, change several variables at a time without necessarily keeping others constant, and go beyond objective analyses to include subjective interpretations. Scenarios enable learners to interact with each other within a real-world context (Jinks et al. 2011; Trillaud et al. 2012) and help learners to think creatively in order


15

to avoid future risks (Miller & Waller, 2003). In disaster studies, scenario-based training is able to place simulation participants into a specific situation that resembles a real life emergency and generates the scenario according to the decisions taken by participants (Van Niekerk et al. 2015).

Disaster scenarios are helpful as a tool to describe the impacts of the event on society and prepare institutions, practitioners, and communities for the effects of a specific event (Davies 2015). However, scenarios present some limitations. According to Miller and Waller (2003), four shortcomings emerge. First, scenarios may just have imaginative purposes, lacking of logical consistency and thorough examination. Second, some inputs to scenario analysis are not quantifiable in nature, so the output of the scenario may not be quantifiable. Third, the scenarios created may reflect current conditions and biases rather than future possibilities and the personalities of the scenario’s developers can potentially limit the possibilities considered. Finally, while many perspectives may be considered, eventually a lack of consensus in reaching a shared or common strategy is quite possible.

Based on the opportunities and challenges of simulation technique and disaster scenario-based training, the RES-SIM project aims to provide new insights about simulation and scenario-based learning for disasters, as described in the next chapters.


16

Chapter 2 - Approach and Methods To achieve the aims of the RES-SIM project the project team conducted a literature review to provide an overview of simulation and scenario-based learning in disaster management. A use case analysis interrogated the potential system options prior to field work. Following the granting of ethics approval for the project, data collection involved a) interviews, b) workshops and c) focus groups. The emerging data was analysed using multiple forms of cognitive (concept) mapping techniques (Wheeldon and Faubert, 2009) and system dynamics modelling to produce system maps, disaster scenarios and coding of interviews. Figure 1 outlines the basic research approach.

Figure 1 RES-SIM research approach

Expert Interviews with Educators and Practitioners The team investigated stakeholder needs and perceptions through interviews. Eight educators were interviewed together with eleven disaster management practitioners. Educators were purposively selected on the basis of their expertise in simulations (technology and non-technology based) and their disciplinary background (e.g. disaster management). Practitioners were selected on the basis of their experience in the disaster management sector across the preparedness, response and recovery stages of disaster. Participants represented organisations across the utilities sector (e.g. water, electricity), Non-government Organisations (NGOs) (e.g. Red Cross, State Emergency Service) and local governments. Interviewees were based in Australia (NSW, Tasmania, Victoria) as well as New Zealand, USA, and UK.

Interviews were semi-structured. Interviews with educators primarily explored topics around student engagement, assessment and their ideas and visions for “classrooms of the future” with reference to simulation techniques. Interviews with practitioners explored the process and phases in which stakeholders respond to disaster events, the key elements of the disaster context and the interactions of these, skills required in disaster scenarios, current approaches to disaster training (including simulations), and issues relating to data and information management.


17

The interview recordings were analysed via dialogue mapping using Virtual Understanding Environment (VUE)1 software. Each key point made by the respondent was summarised and represented as a node within the map. These nodes were connected according to logic of the discussion and interviewee chain of thought, according to Wheeldon and Faubert (2009). Next each interview map was coded according to keywords and categories as determined from the literature review and initial map review process by the project team. The keywords were based on two typologies:

● Type (1) are related to the development of the simulator and were correlated to the relevant dimensions of simulation framework; and

● Type (2) are operational concepts and primarily correspond to a component of the system model (see Appendix B for keywords and corresponding dimensions)

All interview maps were merged to produce 481 nodes that were coded with keywords according to their content and topic and correspondence to a specific dimension of the framework. The RES-SIM project built upon the dimensions of Gaba’s framework (2004), which was adapted to emergency management by AFAC (2014) with respect to the design of a simulator (See Appendix B).

System Maps Development via Workshops This stage comprised three workshops with community and business leaders, representatives of the emergency management sector and academics with disaster management experience. The intent was to confirm the proposed conceptual framework for RES-SIM and highlight areas for improvement. Participants were introduced to the RES-SIM project and presented with key findings for the first two stages and invited to comment. Confirmation of the conceptual model (e.g. the five elements of the system) and design specifications (e.g., characteristics of an effective simulation system from the interviews) occurred through participants participating in a mapping activity. Participants depicted their workplace as a system (e.g., their building and their organisations operating structure), highlighting the relevant sub-systems, variables and connections between them. By combining these (36) individual system maps with other participants, (13) teams were able to explain, connect and co-construct a more extensive and detailed representation of their reality. At the end of the interactive session, teams presented their systems to the remaining workshop participants (see Figure 3). The context of each system map contained the raw data for this activity, along with the audio and video recordings of the facilitated discussions and presentations (total of 10 hours of video and audio). The linkages and boundaries between systems were explored, along with educational and operational implications of using complex adaptive systems thinking in this type of work.

1 At its core, the VUE is a concept and content mapping application, developed to support teaching, learning and research and for anyone who needs to organise, contextualise, and access digital information. Using a simple set of tools and a basic visual grammar consisting of nodes and links, faculty and students can map relationships between concepts, ideas and digital content. Available from - http://vue.tufts.edu/index.cfm.

http://vue.tufts.edu/index.cfm


18

Scenario Development via Focus Groups Three focus groups were conducted with representatives from specific organisations dealing with disaster management. The intention was to create a scenario that involved disruption to a social system as defined by participants. The three specific scenarios were developed:

1. Flood scenario in the Hunter Valley; 2. Economic shock scenario; and 3. Bushfire recovery scenario.

These scenarios were selected as they represent a contrasting set of disasters that are relevant for diverse communities across Australia and would involve disaster responders from a wide variety of disciplinary backgrounds. Through describing a disaster event as it unfolds over time, the participants employed their individual and collective experience within the organisations to elaborate how key stakeholders, processes (e.g., communication, decision making) resources and information-based issues play out in that type of emergency. Participants then proceeded to draw connections between the elements of the disaster event and the five system variables (environment, social, economic, governance, infrastructure) as they intersect with the stages of the disaster event. The focus groups were recorded with audio and video, and included observation by the project team and scenario maps produced by the participants.


19

Chapter 3 - Findings This section presents the key results that emerged from each stage of the project. In Stage 1 (interviews), the analysis revealed five key learning areas to be discussed. In Stage 2 (workshops), the project team developed system models that represent the dynamic nature of participant’s worlds. It will present an example of these models and demonstrate their potential. Stage 3 (focus groups) enabled the project team to create disaster scenarios that can be utilised to design simulated experiences for students, one of which will be presented in this chapter. Finally, the findings have been mapped against the project’s framework, adapted from Gaba (2004) and AFAC (2014).

Stage 1: Expert Interviews with Educators and Practitioners

Figure 2 Example results of coding Simulation & Scenario theme from interviews

Figure 2, above, represents the key themes and subthemes that arose from iteratively analysing the 49 interview responses (nodes) which contain the keyword terms ‘simulation’ and ‘scenario’. The scale of the subtheme title reflects its frequency. This demonstrates the type of results that were used as considerations in the further stages of the project, including the design briefs for development of scenarios and the overall simulator. The scale of results produced from 54 other key words and over 480 interview data nodes is beyond this report but provides comprehensive insights for further design and cross referencing of the focus group and workshop data.


20

The value and future of simulation and scenario-based learning

The educators and practitioners interviewed for this study identified the value of simulation, similar to extant literature, as centring on the ability to provide a student experience that they could not otherwise have, due to issues such as lack of resources, the remote location of students or safety concerns. Interviewees noted that an important function of simulations is to provide students the chance for experimentation in a safe environment, particularly in the context of hazards where there is high-risk involved to themselves and others. Scenario-based learning supports experimentation and learning from mistakes, which students often try to avoid in order prevent “loss of marks” or “losing face”. Addressing this mindset was an important role of simulation-based learning. As stated by one participant,

“You learn more from your failures than your successes, you have to get things wrong, and that is part of it...a simulator provides a nice safe environment to do that” [Educator G]

Learning Context: Curricula, Assessment and Feedback

The importance of taking a holistic approach to the design of curricula, assessment and feedback was identified as important to ensure effective scenario-based learning. The purpose and aim of the simulation needs to be known in the first instance (e.g., education, assessment and/or demonstration). Integrating simulations into popular pedagogical approaches, such as the flipped classroom, was seen as important.

Participants acknowledged that students are often “assessment driven”. Many of the principles of assessment used in conventional classrooms were seen to be applicable in designing simulation systems and scenarios. Approaches to assessment such as peer-reviewed learning were raised as being useful in a simulation context. Some participants reported that assessments could be embedded in the scenario simulation. One participant believed, however, it was difficult to embed effective assessment within simulation systems and therefore assessment should be designed outside the system. One interviewee advocated for the simulation of professional dilemmas for students, suggesting the need to,

“Build in contradictory and inconsistent information so that there is conflicting group goals and expectations of participants.” [Educator A]

Comprehensive formative and summative feedback methods strengthen the effectiveness of simulation and scenario-based education. Notably, “after action reviews” were emphasised as important opportunities for reflection and learning. Here, the collection of information through logging activities during the simulation could be beneficial. Students could use this information as part of the reflection process and instructors could use the information to prompt students.


21

“A Simulation is only as good as it’s debrief. For a two-day scenario we would do a half-day feedback and debrief getting participants to review and reflect on how they responded is critical. Especially for the types of learning outcomes you’re talking about” [Practitioner A]

Simulation Participants: Learners and Instructors

Ensuring a student-centred approach is considered key and should be reflected in simulation system design and operation. For example, one participant noted that international students were sometimes less accustomed to experiential problem-based learning and required the instructor to give additional support and guidance. Language and gender issues were identified as considerations because instructors need to ensure there is a balance of contributions from participants. Maximising interaction is key to successful simulation systems.

Inclusion of team-based activities enhances learning and replicates the real world. Some observed that gamification could result in “competition” with the focus shifting from the process to the outcome in order to “win” and thus it was important for the instructor to emphasise the importance of collaboration vs competition in group activities. Educators had also to contend with technological challenges such as access to appropriate bandwidth. Further, learners (and some instructors) have varied levels of technology literacy thus manuals and support services were key. Therefore, as one educator asserted,

“The instructor for the simulation plays a very important role.” [Educator P]

Scenario Context

Interviewees provided insight into some developing disaster scenarios. A number of challenges for the development of disaster of scenarios were highlighted. These largely centred on the complexity of the context. The wide number of stakeholders involved, the different levels of urgency in disasters with some issues needing to take place in the same day, while others would unfold over weeks or months. To provide an authentic learning experience and achieve desired learning outcomes educators and practitioners will need to make decisions about whether their scenarios would run over a short period (e.g., 20 minutes) or be used as a basis of classes over an entire semester. There may also be decisions around whether the simulation would be live (synchronous) or asynchronous. That the characteristics of any one disaster can be so diverse is a challenge — the same type of disaster (e.g., bushfire) could occur in the same location and around the same time of year but 25 years later and the outcomes could be quite different. One suggestion to handle this challenge would be to base a scenario on historical data and emphasise these issues with learners. Further, educators need to remain mindful of the complexities of the disaster context and be careful that simulation systems do not result in students making generalisations. For example, a multi-disciplinary scenario may provide some insight into the roles other stakeholders play in the response to a disaster but this insight may be quite


22

limited based on the scope of the scenario. Despite the challenges educators saw the value for a system such as RES-SIM, with one participant asserting that,

“Some things really matter to us, but to other people they are not important. The challenge for skilled in a simulation is slowing down their thinking processes enough to understand the key steps to get them there [to what they’re trying to achieve]. That can be very effortful but it is an important thing to do.” [Educator P]

Simulation Technology and Design

In terms of simulation technology, the power of “visualisation” and “immersion” was considered important because when students “see it” and “experience it” they learn quickly. Increased use of gamification approaches, visualisation techniques and augmented reality, while still on the horizon for many, is set to become more commonplace. Well-designed simulations have the potential to provide an authentic experience. Some respondents noted that the use of real-life cases was ideal. It was highlighted that often in disaster scenarios decision-making is distributed, whereby decision makers are separated through space and time. Virtual simulation tools were actually seen to offer an advantage over face-to-face role-playing based simulations as such an environment more closely reflects the reality of the disaster context:

“Stakeholders make decisions on the run, often remotely, based on information provided on their phone, PDA, updated GIS maps and all sorts of different media need to be incorporated in simulation to make it realistic.” [Educator C]

It was identified that to develop a successful simulation system, software developers will need to elicit the requirements from the system designers and instructors. This was seen to require close cooperation and use of a development brief. Further, one educator advocated for incremental design — starting with a bare bones system and refining the design based on how students actually used the system rather than estimating how they would use it.

Stage 2 - Results and Analysis of Workshops Overall the workshops highlighted the need for a “real-life” experience and the complexities involved in providing this in an online simulation system. In particular, when analysing the model outputs and audio from the workshops, the complexity, non-linear feedback and multifaceted influence of system components such as governance and economic become apparent.


23

Figure 3 Workshop participants present their system maps to the group

Figure 4 is an example of system dynamics model that was produced from the workshops. This model depicts the larger scale considerations for State Emergency Service (SES) effectiveness in responding to calls for assistance in a storm or flood. The nested models (located within the macro model) exhibit the ‘pocket detail’ nature of system modelling, which will determine simulator characteristics. The nested models are those of the participants, one of which is a University lecturer, another a safety warden on campus and the other a SES volunteer. The participant conversation to develop this model elaborated on issues such as; the contextual similarities in challenges for higher education and emergency management to graduate capable participants with dwindling resources and increasing risk and complexity; and the trends of using technology to deliver content, including the lively concept of whether chainsaw skills could be taught online.


24

Figure 4 System dynamics model representing components affecting SES efficiency to complete call out jobs during a disaster

The workshops illustrated the amount of detail that is involved in mapping even a segment of a societal system. Each participant had a particular understanding of their environment and what emerged from the workshops was information leading to pockets of rich detail within a broader system model that continued to have many gaps. This is both frustrating and exciting for the research team, and represents an opportunity for further research in this area.

Stage 3 - Results and Analysis of Focus Groups The focus groups enabled the project team to engage with industry stakeholders to elaborate on the organisational and sectorial processes they use with the aim of building useful scenarios for student learning. The RES-SIM team also provided information to participating organisations on the innovative use of simulation in higher education, based on the visit of internationally renowned Disaster educator, Dr Ali Asgary (York University, Canada). From the focus groups, base knowledge was obtained about the significance and interactions of different system components during a disaster. The multiple, different perspectives provided valuable insights on the implications of decision-making after a disaster, and therefore shed light on how disaster managers can frame their approach. The 3 scenarios developed centred on 1) a flood scenario for the Hunter Valley, 2) an economic shock scenario and 3) a bushfire recovery scenario. An example output, from focus group 1, is provided in Figure 5.


25

Figure 5 A sample scenario developed from a focus group map

The sample scenario, depicted in Figure 5, demonstrates the complexity of a flood response for local SES unit controllers. This scenario is only an introduction to the many facets of their operating environment, their responsibilities and stakeholders, yet it provides a wealth of insights about the types of skills, knowledge and practices that students would need to attain to fulfil such roles. It is important to note that the elements and processes of the scenarios are transferable to other disasters and other stakeholders, the research already created value for project participants in discussing and developing not only their own scenarios but seeing others. The project team has conceived multiple logical frameworks and key learning objectives from the preliminary analysis of each scenario map. This analysis will be further enriched by the accompanying audio and video recordings, which give a depth of detail that can be used in scenario development for an online system. By integrating the conceptual system map into a graphical interface and then leading the participant through features of the


26

relevant scenario the project will have triangulated the three data sets. Combining the workshop results with those from the focus groups provide us the conceptual basis for developing agent based models (using AnyLogic2). Although the graphical interface is preliminary, this demonstrates the type of simulation challenges and user experience that can be created with the data from this project. A number of scenario characteristics were identified as important when developing scenarios for online learning simulations in disaster management:

• Identification of key actors • Incorporations of timelines • Representation of inter-dependencies • Recognise where and when decisions are made • Isolate key vulnerabilities of system • Awareness of restrictions, limitations, constraints • Create experiences • Feedback

Mapping of Results against Framework This section presents the mapping of the findings from Stages 1-3 against the framework. The analysis of the empirical results of the project (i.e., interviews, focus groups and workshops) led to the development of a number of maps that informed the project team’s thinking towards an online simulation system (e.g., Figure 4, Figure 5). Table 1 considers the findings of this project in relation to the scenario dimensions based on AFAC (2014), and highlights a number of design considerations for the online simulation system to be delivered in Phase 2 of the project.

Dimension 1: Purpose and aim of simulation a) RES-SIM needs to be customisable and flexible to accommodate scenarios designed for different

purposes (e.g. education, training, rehearsal, performance assessment and research). b) Education is likely be the primary purpose for which RES-SIM will be used and thus have the

greatest influence on scenario and system design and inform the nature of the guidelines provided to users on RES-SIM use.

c) Whether assessment capabilities are best built-in to the system or sit outside the system is dependent on the context where the system will be used

d) Scenarios used with RES-SIM will need to cater to multiple-disciplines as disasters present wicked problems that require a range of expertise to resolve.

2 AnyLogic is a proprietary software simulation tool that supports the most common simulation methodologies in place today: System Dynamics, Process-centric (AKA Discrete Event), and Agent Based modeling.

http://www.anylogic.com/system-dynamics

http://www.anylogic.com/discrete-event-simulation

http://www.anylogic.com/agent-based-modeling


27

Dimension 2: Unit of participation a) RES-SIM will accommodate a range of units of participation. These units could potentially include

individual students, teams of students, whole classes, or even students across courses or institutions, which might be particularly useful for multi-disciplinary problem-based learning.

b) RES-SIM needs to facilitate interaction whether this is between the individual and the system, between students and/or student(s) and the scenario facilitator.

c) If students are put into groups they are more likely to try out more variations in simulation and experiment thus this should be accommodated by RES-SIM

d) RES-SIM should be student-centred and consider issues such as gender (e.g. as women have historically taken ICT-based courses) and the needs of international students (e.g. who may be less accustomed to problem-based learning).

Dimension 3: Participant experience level a) RES-SIM will accommodate a broad range of users (both staff and students) who have different

skill levels in the use of digital/online scenarios and systems from those who are novices, have limited skills, to those who are skilled, proficient or even experts.

b) The use of problem-based learning was seen as ideal to cater to different experience levels. c) Including features in the system such as prompts, help in a variety of media formats and support

mechanisms were considered important; as was a classroom environment where students could be supported by the teacher as well as peers.

Dimension 4: Simulation domain a) RES-SIM should be able to accommodate simulation designs in a range of domains including

industrial, transport and infrastructure; civil disobedience and terrorism, search and rescue; health and well-being; and natural hazards.

b) The three scenarios designed in this project (bushfire, flood, economic shock) will serve as the base with other scenarios being developed at later points in time. In this way RES-SIM will be the “games console” and the scenarios will be the “game cartridge” providing for future development and a wide range of system uses.

c) The scenarios used by RES-SIM will encompass five system components including social, economic, environmental, infrastructure and governance; and consider relevant actors and system processes.

Dimension 5: Participant disciplines a) The RES-SIM project identified that students might be drawn from any discipline of study for this

simulated experience. b) The three scenarios developed in the project RES-SIM can be adapted for students at all levels of

study. c) The scenarios will emphasise the need for a systems thinking approach.

Dimension 6: Types of knowledge, skills, attributes or behaviours addressed a) The RES-SIM project identified that disaster scenarios can hone a range of skills; conceptual,

technical, attitudes, behaviours and decision making. b) Overall, the simulation and scenario designs of RES-SIM will build skills through problem-based

learning in a multi-disciplinary context with an emphasis on coordination, planning, communication and networking.


28

Dimension 7: Incident phase a) Stakeholder engagement during the simulation design of the RES-SIM project highlighted the

complexity of coordinating disaster incidents as they interact with a range of system components and system processes that unfold over time

b) RES-SIM can be developed for a range of incident phases including the preparation (initial), prevention, response, relief and recovery phases.

Dimension 8: Simulation technology a) Extant literature and stakeholder engagement during the RES-SIM project identified that

scenarios can involve a range of simulation technologies including verbal role-play, table-top, mixed role-play and computer-based; fully computer-based; and full immersion (e.g. virtual reality in Second Life)

b) From the analysis of Stages 1-3 of the project and initial communications with potential software developers either a fully computer-based or full immersion are the most likely choices for simulation technologies and design.

c) Regardless of which simulation technology is selected, RES-SIM will be developed with the wider learning context (e.g. curricula design and assessment) at the forefront.

Dimension 9: Simulation site a) Various simulation sites were identified including classrooms (e.g. universities), home/office (e.g.

students working remotely), dedicated sites (e.g. specialised rooms in universities) and incident control rooms and field-based simulations (e.g. in disaster management and other organisations).

b) The software developers will need to consider mixed sites in the development of RES-SIM and simulation design to enhance communication and interaction between actors.

Dimension 10: Extent of participation a) RES-SIM will be fully online and thus involve remote use. The simulation design for RES-SIM will

involve either full participation or immersive participation as these are likely to maximise interaction in a fully online environment – the final decision being made in Phase 2 of the project based on discussion with potential software developers.

b) RES-SIM will require the ability to change inputs into a scenario to allow students to observe the results of their decision-making and internalise knowledge and learning.

c) During participation, it was identified that students will required guidelines to support learning in a complex multi-disciplinary context involving a wide range of actors.

Dimension 11: Feedback methods a) While in some simulations no feedback is provided (e.g. in summative assessment situations),

feedback methods were considered paramount to the multi-disciplinary student learning experience in RES-SIM.

b) RES-SIM’s system and simulation design will accommodate multiple forms of feedback (e.g. written, audio, video) that might be provided by the instructor (e.g. online/offline) and/or automatically by the system (e.g. when the learner is experiencing difficulty). Feedback will be able to be provided in real-time during the simulation as well as after the simulation by the instructor.

Table 1 Scenario dimensions and design considerations (based on AFAC 2014)

Conceptual System Model The RES-SIM conceptual system model (Figure 6) has been developed based on the information derived from the multiple-stage data collection described above. The proposed model depicts the architectural components of a simulation learning system proposed in


29

Phase 2 of RES-SIM. The system comprises two main parts, (1) a user component, and (2) a simulation design module. The simulation design module contains a number of sub-components, including a scenarios domain that is intended to store information about specific characteristics relating to the various types of disaster events, for example, floods, fires and storms. The required attributes component contains the skills sets that students or learners expect to gain after they finish specific simulation lessons. These attributes also will be aligned with learning objectives. The simulation design module also contains other component parts including the back end (algorithms) component, a simulation technology component and a user interface component. Together, these will generate suitable learning scenarios, activities and tasks. These components will also align with interaction and feedback mechanisms to suit specifically targeted students as well as the simulation system moderator.

Figure 6 Conceptual System Model

Key Message: The need for Real Life Experiences The need for real life experience was the key theme that emerged from the RES-SIM project. It was highlighted that the “real” work of emergency management is beyond published policies and procedures. As a result, simulation is often used to introduce (and test) the ability of participants to make nuanced and complicated decisions in an emergency. The analysis identified that simulations need to expose and ideally force participants to experience what goes on behind the scenes of an emergency because people outside the context rarely understand what emergency managers have to do.

The RES-SIM project identified that while actors working in a disaster response are connected they are often unable to communicate effectively using ‘normal’ modes. Thus, a simulator will ideally capture how emergency managers operate under such conditions so students can experience this reality and its inherent complexities. The analysis revealed


30

that successful decision making towards realistic and appropriate goals requires multiple stakeholder involvement. Yet often stakeholders that are affected by early decisions will not be present or involved in the decision making process. For example, recovery organisations, such as the Red Cross, will rarely be involved in key decisions during the immediate response that impact system elements such as infrastructure, governance or the environment, yet these decisions will have significant implications for social recovery and the work of the Red Cross. Further, decisions made under difficult circumstances will affect later stages of an emergency. Such complexities need to be incorporated in Phase 2 of the project so that students experience the impact of their decisions on connected systems.

It is important to be able to move beyond an oversimplified understanding of root causes, opportunities and challenges. Uncertainty creates challenges for participants to know the best decision. This is extenuated by uncertainty about the context and the actual aim of an operation. These can have compounding effects on the ability of emergency managers to successfully conceptualise the logical processes being entrained. Future emergency responders need to learn about what causal processes might be involved and their implications. Good emergency managers listen for feedback within their personal and professional networks and then use their position and relationships to influence decision-making and outcomes. While the stakeholders engaged in the RES-SIM project acknowledged that training is needed, it is decision making under challenging circumstances that is most essential. Such concepts need to be catered for when designing simulation systems and scenarios in disaster management contexts.


31

Chapter 4 - Evaluation This chapter will discuss how both RES-SIM process and findings have led to the project outcomes and how the team has evaluated progress and quantified success, as well as outline the dissemination strategy. The chapter will finish by exploring current project impact and its future potential, including implications for and opportunities in higher education.

Outcomes This Seed project has delivered on five outcomes (Figure 7). Informed by the extant literature in disaster resilience and simulation in education, the activities undertaken have allowed the collection of a rich data set, networking with domain expert educators and practitioners. Analysis of the data set has enabled the formation of critical evaluation of strategies, processes and tools for scenario-based learning in the domain of disaster resilience that will inform the development of the simulation system in Phase 2 of RES-SIM.

Outcome 1 Conceptual System Model: The project set out to build a conceptual model for a virtually distributed computer-based teaching and learning tool to enable students from across disciplines, on and off campus, to acquire decision-making skills via disaster system simulations. Following a literature review, through a series of interviews, the project team identified the requirements of educators and disaster management practitioners for scenario and simulation systems (Chapter 3) with particular reference to the scenario dimensions adapted from the AFAC (2014) framework (Table 1). The project team also conducted three focus groups and three workshops creating multiple models of the participants’ ‘world’ (e.g., Figure 4) and disaster scenarios (e.g., Figure 5). Together these outputs informed the conceptual model (Figure 6) that will underpin discussions with potential software developers (e.g. Datalink) in Phase 2 (see also Outcome 2). Outcome 2 Design Specifications: Initial insights into the design specification for a disaster resilience simulation tool were gleaned from the expert interviews. In Stage 3, the project constructed a series of complex scenarios (e.g., Figure 5) to highlight the functionality that the underpinning simulation tool would need to support. The scenarios developed in the project highlight that the system being replicated is complex and therefore any simulation

Figure 7 Aspirational project outcomes


32

tool will have to focus on particular aspects of the entire system and use multiple assumptions to determine the boundary conditions. The project team will work with simulation system designers to synthesise the specifications form the interviews (e.g., Table 1), and scenarios (e.g., Figure 5) and maps (e.g., Figure 4) from the focus groups and workshops. As part of this process, initial discussions with a potential software development company, Datalink, that provides the Crisisworks disaster response software across Victoria, have been undertaken. It has been established that in the next phase of the RES-SIM project, it would be possible to use Datalink’s expertise, network, data and platform towards the development of a simulator. Notably the project team have identified that an iterative approach to system development would be employed, as this would enable the developers to improve the tool based on its actual use by students and instructors. Outcome 3 Identify and Engage Key Stakeholder Groups: The multiple methods used in the project provided the means to engage a wide range of stakeholders including eight educators and eleven practitioners during the expert interviews; fourteen practitioners from seven organisations during the scenario development focus groups; and four community and business leaders, thirteen disaster management practitioners and fifteen educators during the workshops. At the commencement of each round of data collection, the project team communicated the aims of the RES-SIM project and summarised the findings to date. The focus groups and workshops provided an interactive forum for participants to share and learn from their collective knowledge and experience in disaster resilience. The level of stakeholder engagement was evidenced through the rich exchange of ideas between participants, their willingness to volunteer for later stages of the project and interest in receiving the results of the project including use of the proposed simulation tool developed in Phase 2 of the project. Outcome 4 Educational Applications: Through application of the dimensions of the AFAC (2014) framework and data collection activities during Stages 1-3 of the project it is clear that there is potentially a wide range of education applications for RES-SIM. By targeting students across multiple disciplines the RES-SIM tool will need to cater to a wide range of students at undergraduate and postgraduate levels working both on and off campus. RES-SIM will need to engage single users as well as teams, classes and potentially students across education institutions. The RES-SIM tool could be applied for education, training, rehearsal, performance assessment and research, although education and training were the most likely applications. Further, during the data collection activities several participants identified that they had interest in use of such a tool in their organisations—this included, for example, use by other universities and also disaster management organisations.


33

Outcome 5 Base Knowledge about Societal Systems: The project has gained base information on societal systems through the identification of the system components which comprise five elements that interact, namely, social, economic, environmental, infrastructure and governance. However, these elements are essentially pieces of a multi-dimensional puzzle which can then become animated through the simulation of scenarios developed in the workshops and system maps developed in the workshops, according to the design principles as defined by practitioners and educators in the interviews. Further, the audio, video and field notes from the workshops and focus groups provide extensive supplementary data for rendering detail within the system model and animating scenarios.

Limitations and Future Work As an initial first phase of a wider project, the RES-SIM project was subject to a number of limitations. The project did not include elicit views from students due to resource and time constraints. Rather, the project focused on gathering expert perspectives from educators and practitioners. In future work, students would be engaged during the further design and implementation of the RES-SIM tool. In addition, interview and workshop results showed that it was impossible within the scope of this project to model the entire system in minute detail. The project team were, however, able to create accurate models of reality in specific segments of the overall system. The larger research project as envisaged for Phase 2 could begin to fill these gaps in detail as part of constructing the resilience simulator. Finally, it may appear odd that this project about ‘simulation’ did not develop any working simulations. In fact, the project set out to provide the groundwork for future simulation platform development and did not expect to develop simulations in this 12-month project. The basic simulation of a focus group scenario in AnyLogic demonstrated one potential avenue for the research.

Dissemination To date, the findings of the RES-SIM project have been disseminated via:

a) Seminars - Dr von Meding presented preliminary findings of the project in the School of Architecture & Built Environment open seminar on the 20th October 2015, at a seminar at Beijing Normal University School of System Science, on 21st January 2016 and at a seminar Universiti Sains Malaysia Engineering Campus, on 18th March 2016.

b) Blog – Numerous RES-SIM related posts have been contributed to the UON Disasters & Development blog, including: Developing RES-SIM - A game changer, Resilient futures: A game of high stakes, What type of game do you want to play?, RES-SIM: The challenges of addressing vulnerability in scenario design, Student

http://danddresearch.blogspot.com.au/

http://danddresearch.blogspot.com.au/2015/07/developing-res-sim-game-changer.html

http://danddresearch.blogspot.com.au/2015/08/resilient-futures-game-of-high-stakes.html

http://danddresearch.blogspot.com.au/2015/10/what-type-of-game-do-you-want-to-play.html

http://danddresearch.blogspot.com.au/2015/10/res-sim-challenges-of-addressing.html

http://danddresearch.blogspot.com.au/2015/11/student-learning-matters.html


34

Learning Matters and Slashed OLT represents lost opportunities for innovation in DRR.

Future planned dissemination:

a) Blog - All the findings provided by this report will be summarised within a series of posts to be published on the Disasters & Development blog.

b) School Seminars - The findings will be presented and discussed within the weekly seminars organised by the School of Architecture and Built Environment, UON, by members of the project team in the weeks and months ahead.

c) International Conferences - Results of the RES-SIM project are due for presentation as follows:

a. “Modelling disaster resilience: enhancing student learning through trans-disciplinary simulation of wicked scenarios (RES-SIM)” at Conference Åre - Risk Event 2016 “Resilience – Opportunities and Challenges for Societal Crisis Management and Individual Safety”, Åre, Sweden, 14-16 June 2016;

b. 6th Building Resilience Conference, Auckland, New Zealand, 7th-9th September 2016

d) Journal papers - Three papers in peer-reviewed academic journals focused on higher education and/or disaster studies are planned:

a. “Building Disaster Scenarios for Simulation-based Learning in Higher Education” for submission to Disaster Prevention and Management

b. “Towards a Simulation Platform for Multi-Disciplinary Disaster Resilience Education” for submission to Computers and Education

c. “Developing a System Model of Societal Resilience for Education and Training” for submission to International Journal of Disaster Risk Science

As noted in Outcome 3 (Stakeholder Engagement), preliminary results of the project have been shared with participants during the project. Some of these participants have expressed interest in receiving the final results of the project including the simulation tool that will be developed in Phase 2.

Project Impact and Implications for Higher Education The RES-SIM project has engaged education and practitioner experts in disaster resilience and simulation systems to develop a conceptual model and the design requirements for a system simulator using base knowledge about societal systems; that can be adopted for a wide range of educational applications both in higher education and the private sector. While Phase 1 of the project is exploratory and the actual RES-SIM tool will be developed in Phase 2, others might use these outcomes to inform the design of scenarios and simulators

http://danddresearch.blogspot.com.au/2015/11/student-learning-matters.html

http://danddresearch.blogspot.com.au/2016/05/slashed-olt-represents-lost.html

http://danddresearch.blogspot.com.au/2016/05/slashed-olt-represents-lost.html

http://danddresearch.blogspot.com.au/


35

in other contexts. In Phase 2, the system prototype will be developed and will be trialled in a range of classroom settings to demonstrate simulator process and outcomes.

In closing, the project team would like to stress the following recommendations to the higher education community:

• Scenarios in disaster resilience should be designed with the eleven scenario dimensions (after AFAC 2014) in mind to maximise their effectiveness

• It is critical that system simulators emulate real-life experiences; such experiences can be supported via choices around scenarios design (e.g. use of real historical data); and type of technology employed to underpin the system simulator (e.g. virtual reality)

• Scenarios in disaster resilience are inherently complex and multi-disciplinary. They should incorporate the interdependencies of various system elements (i.e. social, economic, environment, infrastructure, governance)

• Disaster resilience simulations need to be flexible and customisable because even the same disaster type (e.g. bushfire), in the same location at a different point in time can impact elements of the system differently. Simulation systems need to factor in these complexities and enable to game controller to vary the impact of student decision-making processes

• System simulators should be developed using an iterative rather than ‘big bang’ approach so that the way in which instructors and students use the system in practice can inform design improvements and ultimately ensure the simulation is better integrated into the classroom environment and wider curricula, including approaches to assessment and feedback.


36

References AFAC (2014). Building Capacity Through Simulation - Research Insights Into Good Practice (Research

Utilisation Resource). Retrieved from Australasian Fire and Emergency Service Authorities Council: Copy provided by contributing author.

Buytendijk, F., Hatch, T., & Micheli, P. (2010). Scenario-based strategy maps. Business Horizons, 53(4), 335-347.

Coetzee, C., Van Niekerk, D., & Raju, E. (2016). Disaster resilience and complex adaptive systems theory: Finding common grounds for risk reduction. Disaster Prevention and Management, 25(2), 196-211.

Comfort, L. K., Sungu, Y., Johnson, D., & Dunn, M. (2001). Complex systems in crisis: Anticipation and resilience in dynamic environments. Journal of contingencies and crisis management, 9(3), 144-158.

Davies, T. (2015). Developing resilience to naturally triggered disasters. Environment Systems and Decisions, 35(2), 237-251.

Folke, C. (2006). Resilience: The emergence of a perspective for social–ecological systems analyses. Global Environmental Change, 16(3), 253-267.

Gaba, D. M. (2004). The future vision of simulation in health care. Quality and safety in Health care, 13(suppl 1), i2-i10.

Gonczi, A. (2013). Competency-Based Approaches: Linking theory and practice in professional education with particular reference to health education. Educational Philosophy and Theory, 45(12), 1290-1306.

Haigh, R., & Amaratunga, D. (2015). Moving from 2015 to 2030: challenges and opportunities for higher education. International Journal of Disaster Resilience in the Built Environment, 6(3).

Hopwood, N., Rooney, D., Boud, D., & Kelly, M. (2014). Simulation in Higher Education: A sociomaterial view. Educational Philosophy and Theory, 1-14.

Jinks, A., Norton, G., Taylor, M., & Stewart, T. (2011). Scenario-Based Learning: Experiences in the Development. Professional Education Using E-Simulations: Benefits of Blended Learning Design: Benefits of Blended Learning Design, 346.

Levin, S. A. (1998). Ecosystems and the biosphere as complex adaptive systems. Ecosystems, 1(5), 431-436.

Mann, B. D., Eidelson, B. M., Fukuchi, S. G., Nissman, S. A., Robertson, S., & Jardines, L. (2002). The development of an interactive game-based tool for learning surgical management algorithms via computer. The American Journal of Surgery, 183(3), 305-308.

Manyena, S. B. (2006). The concept of resilience revisited. Disasters, 30(4), 434-450. Miller, K. D., & Waller, H. G. (2003). Scenarios, real options and integrated risk management. Long range

planning, 36(1), 93-107. Remington-Doucette, S. M., Hiller Connell, K. Y., Armstrong, C. M., & Musgrove, S. L. (2013). Assessing

sustainability education in a transdisciplinary undergraduate course focused on real-world problem solving: a case for disciplinary grounding. International Journal of Sustainability in Higher Education, 14(4), 404-433.

Renda-Tanali, I., & Abdul-Hamid, H. (2011). An assessment of the benefits of online scenario simulation tools in homeland security and emergency management education. Journal of Homeland Security and Emergency Management, 8(2), 1-16.

Richmond, B. (1993). Systems thinking: critical thinking skills for the 1990s and beyond. System dynamics review, 9(2), 113-133.


37

Salas, E., Wildman, J. L., & Piccolo, R. F. (2009). Using simulation-based training to enhance management education. Academy of Management Learning & Education, 8(4), 559-573.

Schultz, C. H., Koenig, K. L., Whiteside, M., Murray, R., & National Standardized All-Hazard Disaster Core Competencies Task Force. (2012). Development of national standardized all-hazard disaster core competencies for acute care physicians, nurses, and EMS professionals. Annals of emergency medicine, 59(3), 196-208.

Schwaitzberg, S. D., Godinez, C., Kavic, S. M., Sutton, E., Worthington, R. B., Colburn, B., & Park, A. (2009). Training and working in high-stakes environments: lessons learned and problems shared by aviators and surgeons.Surgical innovation, 16(2), 187-195.

Simonovic, S. P. (2011). Systems approach to management of disasters: methods and applications. John Wiley & Sons.

Taylor, J., & Evans, D. (2005). Pulling together: keeping track of pedagogy, design and evaluation through the development of scenarios—a case study.Learning, Media and Technology, 30(2), 131-145.

Trillaud, F., Pham, P. T., Rabah, M., Estraillier, P., & Malki, J. (2012, April). Situation-based scenarios for e-learning. In IADIS e-learning 2012.

UNISDR (2009). UNISDR terminology on disaster risk reduction. Geneva, Switzerland, May. Van Niekerk, D., Coetzee, C., Botha, D., Murphree, M. J., Fourie, K., Le Roux, T., ... & Meyer, S. (2015).

Planning and Executing Scenario Based Simulation Exercises: Methodological Lessons. Journal of Homeland Security and Emergency Management, 12(1), 193-210.

Walker, B., Holling, C. S., Carpenter, S. R., & Kinzig, A. (2004). Resilience, adaptability and transformability in social--ecological systems. Ecology and society, 9(2), 5.

Wheeldon, J., & Faubert, J. (2009). Framing experience: Concept maps, mind maps, and data collection in qualitative research. International Journal of Qualitative Methods, 8(3), 68-83.

Wilensky, U., & Stroup, W. M. (2013, April). Networked gridlock: Students enacting complex dynamic phenomena with the HubNet architecture. InProceedings of the 4th International Conference of the Learning Sciences (pp. 282-289). Erlbaum.

Zoakou, A., Tzanavari, A., Papadopoulos, G. A., & Sotiriou, S. (2007). A methodology for eLearning scenario development: the UNITE approach. InProceedings of the ECEL2007-European Conference on e-Learning, Copenhagen, Denmark. ACL publications (pp. 683-692).

Zwikael, O., Shtub, A., & Chih, Y. Y. (2013). Simulation-based training for project management education: Mind the gap, as one size does not fit all. Journal of Management in Engineering, 31(2), 04014035.

Appendix A

Certification by Deputy Vice-Chance"or (or equivalency

I certify that all parts of the final report for this OLT grant provide an accurate

representation of the implementation, impact and findings of the project, and that thereport is of publish able quality.

Name: ................. ... Dat^^ ..'2:0. .. I^A^...:^, D\!o

Modelling disaster resilience: enhancing student learning through trans-disciplinarysimulation of wicked scenarios (RES-SIM)

34


39

Appendix B

Modelling disaster resilience: enhancing student learning ... Meding_Report_2016.pdf · simulation...

Documents

Transcript of Modelling disaster resilience: enhancing student learning ... Meding_Report_2016.pdf · simulation...