Terrorism-related simulations have been used for a variety of purposes including testing and validating existing plans and procedures; evaluating performance; improving the capabilities and capacities of individuals and organisations to respond to real-life incidents; and identifying gaps in existing training, response plans, protocols, and procedures.
Simulation-based training exercises have been shown to produce short-term learning outcomes. Simulations can increase self-reported confidence in, and knowledge of, emergency response protocols and procedures, and can enhance technical and non-technical skills.
The longer-term impact of terrorism-related simulations is poorly understood. Only one study was identified that evaluated the impact of terrorism-related simulations on a real-life incident, and studies that evaluate the longer-term impact of such simulations are lacking.
There are several important factors to consider when designing and delivering simulations. Simulations must realistically recreate the complexity, uncertainty, and dynamic nature of real-life incidents, and test the technical and non-technical skills, such as collaboration and coordination between different agencies, that are crucial for an effective response.
Simulation exercises should include opportunities for feedback and debriefing to enhance learning. While not specific to terrorism, one study that analysed performance indicators from evaluations of 46 training exercises (e.g. simulator reports, game scores, hospital records, self-ratings, performance ratings) reported that effective debriefing improved performance against these metrics by approximately 25 per cent when compared to control groups that did not attend a debrief. As one of the few studies that uses a counterfactual to evaluate the impact of specific features of training exercises, this is one of the more robust findings in the literature.
Evaluating the longer-term effectiveness of simulations is challenging. The relative infrequency of terrorist attacks means that it is difficult to evaluate how effective simulations have been in preparing responders in dealing with real-life incidents. Evaluators also face practical challenges in accessing the data that could be used to evaluate responses to real-life attacks.
A range of different data collection and evaluation methodologies have been used to evaluate the impact of simulations on learning, behaviour, and outcomes. Common approaches include pre- and post-simulation surveys and follow-up interviews. While these methods are effective at recording shorter-term outcomes and claimed longer-term impacts, more consideration needs to be given to how best to capture longer-term impacts on behaviour and performance.
This report brings together the literature on simulation-based training by drawing on nine studies of terrorism-related simulations, as well as exercises used to simulate a range of natural and man-made disasters.
It discusses how simulations can be used to enhance learning that translates to real-life incidents and outlines the key principles for policymakers to consider when designing and evaluating simulation exercises.
It draws on case studies of terrorism-related simulations from the UK, Europe, and North America.
Aplin, D., & Rogers, M. B. (2020). ‘Alert not alarm’: The UK Experience of Public Counter-Terrorism Awareness and Training, with Explicit Reference to Project ARGUS. The Police Journal, 93:3, 167-182.
Chittaro, L. and Sioni, R. (2015). Serious Games for Emergency Preparedness: Evaluation of an Interactive vs. a Non-Interactive Simulation of a Terror Attack. Computers in Human Behavior, 50, 508-519.
Kim, H. (2014). Learning from UK Disaster Exercises: Policy Implications for Effective Emergency Preparedness. Disasters, 38:4, 846-857.
McElroy, J. A., Steinberg, S., Keller, J., and Falcone, R. E. (2019). Operation Continued Care: A Large Mass-Casualty, Full-Scale Exercise as a Test of Regional Preparedness. Surgery, 166:4, 587-592.
Murphy, J. P, Kurland, L., Rådestad, M. and Rüter, A., (2020). Hospital Incident Command Groups’ Performance During Major Incident Simulations: A Prospective Observational Study. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine, 28: 73.
Rüter, A., Kurland, L., Gryth, D., Murphy, J., Rådestad, M. and Djalali, A. (2016). Evaluation of Disaster Preparedness Based on Simulation Exercises: A Comparison of Two Models. Disaster Medicine & Public Health Preparedness, 10:4, 544-548.
Skryabina, E.A, Betts, N., Reedy, G., Riley, P., and Amlôt, R. (2020). The Role of Emergency Preparedness Exercises in the Response to a Mass Casualty Terrorist Incident: A Mixed Methods Study. International Journal of Disaster Risk Reduction, 46.
Swiech, A. et al. (2020). Terrorist Threat: Creating a Nationwide Damage Control Training Program for Non-Trauma Care Providers. Anaesthesia Critical Care & Pain Medicine, 39:1, 59-64.
van den Heuvel, C., Alison, L. and Crego, J. (2012). How Uncertainty and Accountability can Derail Strategic ‘Save Life’ Decisions in Counter‐Terrorism Simulations: A Descriptive Model of Choice Deferral and Omission Bias. Journal of Behavioral Decision Making, 25:2, 165-187.
Waring, S. (2019). Using Live Disaster Exercises to Study Large Multiteam Systems in Extreme Environments: Methodological and Measurement Fit. Organizational Psychology Review, 9:4, 219-244.a
No comments:
Post a Comment