WORKSHOP
Measuring aspects of situation awareness and understanding in safety-critical situations.
AT
18th International Conference on
Information Systems for Crisis Response and Management
https://www.drrm.fralinlifesci.vt.edu/iscram2021/index.php
INTRODUCTION TO THE WORKSHOP
Situation awareness is a key component of efficient and safe dynamic decision-making on the fireground. FireFront is a new tool developed under the EU Erasmus+ scheme to train and support both firefighters’ Actual Situation Awareness (ASA) and Actual Situation Understanding (ASU). The distinction that will be explored in this workshop is that ASA represents an awareness of aspects of the situation (where items of equipment are for example) whereas ASU represents an understanding of the relevance of aspects of the situation to successful completion of the task in hand.
The ability to assess both ASA and ASU within a training scenario (attendance at a road traffic collision (RTC) in this workshop) allows for a nuanced assessment of the performance of the participant. For example, if ASA is poor, then this might suggest that information-gathering activities may need greater support whereas if ASU is low it may suggest that further training is required. For example, we are exploring differences between experts and novices within a scenario. We predict that there will be little difference between novices and experts in ASA (they can both be equally aware of elements of the situation) but a far greater difference in ASU (knowing what is important to successfully complete the task).
The tool we will be demonstrating in this workshop also measures two other aspects of performance. The first is Perceived Situation Awareness (PSA) and Perceived Situation Understanding (PSU). In many ways, the match between ASA and PSA and between ASU and PSU is one of the most important aspects of performance in a particular scenario. Somebody that has a very poor awareness of what is going on (low ASA/ASU) but believes they have a very good awareness (high PSA/PSU) is more likely to be prone to potentially catastrophic errors.
The second additional measure is a measure of bias. Information Bias (IBias) that links with ASA, and Relevance Bias (RBias) – that links with ASU. Ibias reflects the tendency to accept large amounts of information when building ASA (with the risk of overload) or the tendency to reject information (with the risk of missing key information). RBias reflects the tendency to judge all aspects of the situation as important to the task – or not. We predict, for example, that experts would be less likely to judge everything as important to a task than novices.
This workshop will present the tool developed within the pan-European FireFront project, along with some of the data that have been collected so far – together with an opportunity for participants to try-out one of the training scenarios we have developed. The training scenario is a RTC (which should be perfectly accessible to non-firefighters) and has been implemented in XVR because we have been endeavouring to produce a scenario with as little culture-specific information as possible – within the FireFront project, the scenario is designed to be run in a number of different countries.
Although the scenario demonstrated is that of a RTC, the general approach has been successfully implemented in medical (obstetrics), military (command and control) and driving (autonomous vehicles) scenarios. The scenario runs entirely online and includes automated, but personalized, feedback to each participant on their performance within the scenario.
WORKSHOP STRUCTURE
00:00-00:20: Graham Edgar, Anna Figueras Masip and Stella Polikarpus introduce the project aim, team, and preliminary results.
00:20-00:40: Discussion with participants about measurement of SA and SU with the QASA (Quantitative Analysis of Situation Awareness) methodology and the introduction of FireFront tool.
00:40-00:50 Break
00:50-1:30: Participants take part in the online scenario on their own computers. The scenario should run in all browsers and can even be run on a mobile ‘phone or tablet if necessary.
1:30-2:00 Summary of the workshop and final discussion.
WORKSHOP CO-CHAIRS
Graham Edgar is Professor of Psychology and Applied Neuroscience at the University of Gloucestershire, UK. He has over 35 years research and consultancy experience both in academia and in industry (BAe Systems). Graham specialises in modelling and measuring situational awareness across a number of domains (military, health, firefighting, driving) and also conducts forensic investigations into the perceptual and cognitive contributors to road traffic collisions.
Ms Stella Polikarpus works as a senior lecturer in the Estonian Academy of Security Sciences. She has fifteen years’ experience in developing and implementing training protocols for firefighters. At the start of her career, she focused more on CBRN and hazmat training; however, in later years more on virtual simulation-based training for incident commanders. She is a certified Effective Command assessor and user of XVR On-Scene virtual reality software. She is also a PhD student at Tallinn University, where she studies the use of virtual simulations for training rescue incident commanders’ situational awareness. See more here.
Dr Anna Figueras Masip, is currently the manager of research in risk, emergencies, and civil protection at the Firefighters Research Unit and Civil Protection School of Catalonia, where she has worked since 2015. She has a degree in Psychology from the University of Barcelona, and a PhD in Clinical and Health Psychology. For the last 13 years, she has been involved as a psychologist in the field of security, emergencies, and civil society, and has been a trainer in the Police School of Catalonia and in the Firefighters and Civil Protection School of Catalonia.
THE FIREFRONT TEAM
The FireFront team are from Belgium, Denmark, Estonia, Netherlands, Spain and the UK. Thoelen, F. and Vastmans, J from PLOT (Provincial Centre for Education & Training) (Fire School) (Belgium); Blom Andersen, N., Bøhm, M., Holm, L.O.C.N. from University College Copenhagen (Department of Technology, Emergency and Risk Management) (Denmark); Arendtsen, B. from East Jutland Fire Service (Denmark); Polikarpus, S., Taukar, M., Kütt, T. from Estonian Academy of Security Services (Estonia); Fikke, R.C., Geertsema, T., Hazebroek, J.C., Tonnaer, C., Weewer, R. from IFV Institute for Safety (Netherlands); Figueras Masip, A., Fuste Castella, R. from Institute for Public Security of Catalonia (Firefighters and Civil Protection School) (Spain) and; Catherwood, D., Baker, S., Brookes, D., Edgar, G.K., Naughtie, C., Sallis, G., Silcock, G., Walker, S. from University of Gloucestershire (CRACKLE) (U.K.).
RELATED PUBLICATIONS
Arendtsen, B., Arendtsen, B., Baker, S., Bertels, M., Brookes, D., Catherwood, D., Christiansen, K., Cuypers, W., Edgar, G., Krawczynska, S., Maes, V., Sallis, G., Stegienko, K., Van Craybex, G., Vastmans, J., Vorenkamp, F., Weewer, R., Wenarski, G. (2016). Firemind: Trialling a new tool for training fire and rescue service decision-making. International Fire Professional: 14-17.
Catherwood, D., Edgar, G., Nikolla, D., Alford, A., Brookes, D., Baker, S. White, S. (2014). “Mapping brain activity during loss of situation awareness: an EEG investigation of a basis for top-down influence on perception.” Human Factors: The Journal of the Human Factors and Ergonomics Society 56(8): 1428-1452.
Catherwood, D., Edgar, G. K., Sallis, G., Medley, A., Brookes, D. (2012). “Fire Alarm or False Alarm?! Decision-making “Bias” of Firefighters in Training Exercises.” International Journal Emergency Services 1(2): 135-158.
Catherwood, D., Sallis, G., Edgar, G.K., Medley, A. (2011). “Understanding fireground ‘situation awareness’.” Fire 103(1333): 27-29.
Edgar, G. K., Catherwood, D., Baker, S., Sallis, G., Bertels, M., Edgar, H., Nikolla, D., Buckle, S., Goodwin, C., Whelan, A. (2018). “Quantitative Analysis of Situation Awareness (QASA): Modelling and Measuring Situation Awareness using Signal Detection Theory.” Ergonomics 61(6): 762-777.
Sallis, G., Catherwood, D., Edgar, G., Brookes, D., Medley, A., (2013). “The human brain in fireground decision-making: trustworthy firefighting equipment?” International Fire Professional(5): 21-24.
Thoelen, F., Vastmans, J., Blom Andersen, N., Bøhm, M., Holm, L.O.C.N., Arendtsen, B., Polikarpus, S., Taukar, M., Kütt, T., Fikke, R.C., Geertsema, T., Hazebroek, J.C., Tonnaer, C., Weewer, R., Figueras Masip, A., Fuste Castella, R., Catherwood, D., Baker, S., Brookes, D., Edgar, G.K., Naughtie, C., Sallis, G., Silcock, G., Walker, S. (2020). “FireFront: A new tool to support training in Fireground Situation Awareness, Situation Understanding and Bias.” International Fire Professional
A research paper describing the FireFront project has now been published in International Fire Professional (November 2020) Issue 34. It describes the theoretical basis of the tool underlying FireFront, including the novel conceptual framework of the measures on which the FireFront project is based:
