Dr Ian Greatbatch (Associate Professor, Kingston University London) and AC Iain Houseman (Area Commander, Surrey Fire and Rescue Service) explore the potential for augmented reality in the UK Fire and Rescue Services for training and operations.

You may have seen the recent news of the British firm ‘Improbable’ receiving investment equivalent to $1b (Cellan-Jones, 2017). The aim of founders Herman Narula and Rob Whitehead, was to harness VR technology to build massive virtual worlds for training, and to model large-scale infrastructure such as rail networks. There were a number of potential applications discussed in the media, ranging from traffic simulations, training military personnel, applications within higher and further education, to giant virtual worlds where simulations could be set up to solve some of the world’s hardest data problems.

Interestingly, one of the sectors not discussed at all was the Fire Service, which is a shame, because the potential for AR / VR has been discussed in trade publications for some time. Due to the drivers for change in the fire service (which has seen limited systemic change in its initial and ongoing training processes),  to link with the rapidly changing and complex risk nature of the operational environment, the service is a good fit for the assistive technology benefits and safety enhancements this could bring. This article will deal with the two most obvious (and arguably the two largest components of fire service activities) training and operations.

The suitability of virtual and augmented reality to the fire service comes from a number of factors. Firstly, the nature of the business is typically large-scale, dynamic situations that are hard to simulate in reality (at least cost-effectively). Fires, RTCs, floods etc are routinely simulated in reality in training centres and yards across the country, but come at a cost. They tend to be focussed on technical tool skills, often ignoring the potential to experiment or experience other tactics, and extremely unlikely to enable “resetting” and the ability to try something again if it didn’t work as planned the first time.

Secondly, there is a sector-wide trend of a reduction in all incidents, and in large fires. This means that there is some risk of an officer being in charge of their first large fire, with limited past experiences to draw upon. Virtual worlds enable a full range of situations to be made available to all, regardless of physical location, season, weather etc.

Thirdly, a lot of fire service work does involve equipment in combination or in dedicated usage, and it is not always easy or possible to create a physical space where training with them is easy. Yet virtual worlds, with their limitless possibilities, allow services to set up any scenario with any combination of tools to use.

Fourthly training.Recruits’ training especially can be expensive, and currently the fire services bare the entire cost of this training. It is clearly in the interests of the sector to explore means in which training costs can be reduced without losing its effectiveness.. For example, once scenarios have been developed, they can be distributed service-wide (and beyond), reducing the costs associated with running multiple physical simulations.

Training ‘cost’ is not only financial of course. We have an obligation to protect the environment, and training activities involving real fire, scrap vehicles, operating in and around water all have the potential to impair the environment, where a virtual simulation of that activity could be repeated many multiples of times and yet have a fraction of the impact.

We expect more of firefighters currently, and will continue to expect them to be more multi-skilled, technologically savvy and more resilient to rapidly changing environments in the future. Training in virtua allows us to match the training environment to the needs more fluidly, and more easily.

There is a need with all simulated training environments to be clear on the differences in the real and simulated worlds, and clarity of what is trying to be achieved is key to the success of this type of training.

The potential for this kind of simulation in the fire service is of course not entirely new, with products such as XVR for decision-making and tactical awareness (XVR, 2017), or Rosenbauer’s ERDS system (2017) already exploiting those advantages to great success.

However, there is considerable potential still yet to be widely rolled-out.

For example, one long-standing issue in all organisations is the capture and exploitation of informal knowledge; the skills, understanding, practices and behaviours that make professionals truly exceptional, yet are not routinely written into formal training or policy. We can probably all think of examples of “tricks of the trade” that we picked up from mentors in the early days of our career that we still do, but remain unknown to the organisation. The potential for training in virtua, and in an environment where experimentation, potential failure and rule-breaking are allowed, means that successful, non-canon techniques (the informal knowledge) might be uncovered, developed, or be provided as a menu of options through a data bank that crew members can call on in both the training and operational environment.

Furthermore, this informal knowledge gathering could potentially identify areas of lack-of-knowledge or skill-fade and tailor training to individual’s needs based on the personalised data which develops throughout their career, relating to training undertaken and operational attendance. The training could also be driven by emerging industry and risk trends, either known, or suspected. For example, there are ‘seasons’ for particular activities, and it is sensible to train before that season commences – so we might start to see an increase in training scenarios related to wildfire in the spring before the hot weather begins.

We might also see training being driven by potential, suspected, or nearly knowns, with big-data analysis systems scouring and crunching operations records, looking for emerging trends – response types that have new dimensions, and automatically feeding aspects of those activities into training scenarios.

The potential for AR systems in operational firefighting has been discussed elsewhere, and is certainly in development by a small group of major industry players. It is worth reiterating the major potential applications of the technology though.

Augmented Reality systems place an “overlay” of data-driven content on reality – in layman’s terms, you look through a visor, or glasses, or phone screen and you see additional information played in the right location of reality. The game Pokémon Go is built upon this technology, with monsters placed in a virtual space but in a real location, and players use a mobile device screen to “see” the monsters and collect them.

If we consider that the majority of fire operations involve some sort of locating activity (where something has to be found – address, casualty, seat of fire etc) and the application of some technical knowledge to resolve the situation, then a number of possibilities emerge. For example, one popular vision is to simply include thermal imaging as an augmented layer, via a heads-up display within a facemask visor. The ability of firefighters to see doors, or surfaces that are hotter than their surroundings, the location of casualties and generally through smoke, without needing a handheld device speaks for itself. Applying this technology to the virtual training scenarios above means that firefighters could train to diagnose fire behaviour and rapid search and rescue missions as a matter of course.

One of the original concepts of use for AR was for surgeons to see “schematics” of the human body – to effectively see under the hood of the patient, and so only make cuts or carry out procedures in the right place. This application is patently suitable to RTC operations. If the firefighter can “see” the location of airbags or batteries, explosive bolts and mechanisms as well as seeing “cut here” markers for the most effective cuts, we should see a reduction of time spent on cutting operations, and an increase in positive outcomes for trapped casualties.

In larger-scale and wide-area operations such as flooding we might be able to place virtual cordons, or zone-markers without the need to physically mark with tape – the advantage being that they can then be moved or updated without having to physically move the tape, or can be used in locations where there is nowhere to secure tape. The ability to have this information updated on each staff members visor display and alarm when in the risk areas is a significant step towards achieving a common recognised information and risk picture from the fire fighter through to the Strategic commander who may be remote from the scene.

In cases where a crime scene may exist, single paths of entry can be created, and potential evidence can be highlighted with operatives and shared with others for later, more detailed investigation.

The ability to have an interconnected team (with visor driven tactical and management options based on the knowledge bank referred to earlier) for the different roles being undertaken in the organisation, presents a step change in the management of incidents. Imagine the effectiveness of a team having risks presented and options for action through a visor driven information display, with an assessment of the impacts of actions on safety of all the crews attending. Now imagine these combined actions being coalesced into a commanders’ view of the real time activity picture. It represents a real change to safety on the incident ground.

The additional ability to share this tactical and organisation view with other emergency service commanders and responders will achieve the goal so long attempted, of joint and fully integrated incident management planning, something that so many post event enquiries and reviews have highlighted as significant challenges to overcome.

The use of the data in post event scenarios such as links to personal development training, organisational training, organisational risk management, health surveillance, etc is something that cannot be dismissed as a goal to be achieved.

Introducing change, and particularly this type of significant step, will have to be undertaken in a graduated manner to ensure staff understanding and how they all complete their part of any training and or operational picture. Eg a simple action such as removing a helmet to achieve a goal, or for a welfare purpose may reduce the effectiveness of the team and leave the wearer blind to the risks that they may be facing through the lack of visor transmitted risk information.

Costs for the equipment, the training, the upkeep, the data management and development of Artificial Intelligence (AI) game theory style update programmes to enhance wearer’s performance based on experience in training and operations will need to be understood. There may be efficiencies created in the future from the introduction of this style of interconnected AR/VR approach however initial investment will be a significant challenge.

Reliance in tech / networks and the ownership of data will be key issues. As the tech becomes accepted, having a common global platform to allow global trends and learning will be a significant issue to be overcome to prevent localist data and differences in tactical options menus forming. Free data access will be a key goal to create a uniformity of approach where different services are attending large incidents outside of their normal base of operations, eg USAR, DIM, Water operations, etc.

The Human factors in the introduction of this tech will need to be considered and learning from others such as the military will be invaluable. Safety to address non-integrated actors in the situations where the tech is used will also be a real challenge, eg dealing with shock / trauma victims may require the “visor” barrier to be removed, and good old Eye to Eye contact.

In conclusion, it would seem a sensible and clear-cut match of an emerging technology to a sector. The delaying factors could be the unique environments fire-fighters face; extreme heat, dust, vibration, steam and emersion in water – requiring an extremely robust and resilient system. The cost and the need to change culture to embrace this technology as a key safety system of the future, has been a challenge with most new but now accepted systems. We have touched on the risks of relying on a system that fails and so making the technology “job-proof” is clearly a priority. Once that step is achieved though, this technology has the potential to truly become invaluable to modern firefighting.

References

Cellan-Jones, R. 2017. UK virtual reality firm Improbable raises $500m. BBC. Available at: https://www.bbc.co.uk/news/business-39892251

Rosenbauer. 2017. Fire truck simulator & ERDS driving simulator. Available at: https://www.rosenbauer.com/en/int/rosenbauer-world/service/in-use-around-the-world/simulators.html

XVR, 2017 available at: https://www.xvrsim.com/

About the Authors

Dr Ian Greatbatch FRGS, MEPS, FHEA is an Associate Professor at Kingston University, London. He specialises in Search and Rescue (SAR), Fire and Rescue and applications of Geographical Information to those disciplines.

AC Iain Houseman is currently the Head of regulatory fire safety protection and prevention for Surrey fire and rescue service. He has held roles in the Local Authority Trading Company as a contract and business development manager, Head of Training, Cross service Support and operations resources manager creating new systems and processes to support change in the modern fire service.

The post Augmented Reality in the UK Fire and Rescue Services appeared first on Fire Product Search.

This content was originally published here.