A firefighter is attempting to rescue victims in a burning building, but, as the fire burns, his vision is clouded by thick smoke. What if he could wear glasses that allow him to see through the smoke, identifying the building’s floor plan, the location of other victims and the tasks need to assist in putting out the fire?
Regis Kopper, an assistant professor of computer science at UNC-Greensboro, is working to make that access to data a reality — and not just for firefighters, but for all first responders.
Kopper and his team in UNCG’s Interactive Realities Lab have been awarded a $1.8 million grant from the National Institutes of Standards and Technology to develop augmented reality interfaces for first responders.
Augmented reality (AR) is the integration of computer-generated content with the real world, creating an interactive experience for the user. AR enhances the natural environment while virtual reality creates an artificial one.
“AR displays information and images over a person’s real-world vision – think Google Glass meets Pokémon Go – and is the next technological frontier for improving public safety,” Kopper explained. “It will give EMS personnel, firefighters and law enforcement officers access to critical data when they need it the most, while they are in action in the field.”
See a video from UNCG’s Interactive Realities Lab on a firefighter using AR technology here.
The $1.8 million grant, part of NSIT’s Public Safety Innovation Accelerator program, will fund a three-year project called FirstModulAR. UNCG is one of eight awardees to receive part of NSIT’s $10 million funding for the accelerator program, with other awardees including Texas A&M University and the University of California, Irvine.
For FirstModulAR, Kopper and his team will work with public safety personnel and consultants to first create a database of common public safety tasks that can be improved by AR. Then, with that data, the researchers will develop a series of AR public safety modules, which will be tested with first responders.
In 2021, Kopper completed another NSIT-funded project – called Design, Prototyping and Evaluation of Next Generation Public Safety User Interfaces – aimed at the same goal of developing user interfaces for first responders.
In that first project, personnel at the Wake County EMS, the Durham Fire Department and the Hillsborough Police Department tested virtual reality environments that simulated soon-to-be-available AR technology. Tests included interfaces for firefighters needing directions in a burning building, police officers accessing driver information during a traffic stop and EMS personnel facing complex medical emergencies.
FirstModulAR will build on Kopper’s previous project by adapting and developing more public safety interfaces for use in current devices.
“Some of the devices we’ll test are not yet available commercially,” Kopper said. “We’ll be working with AR tech that uses spatial data, instead of heads-up units that simply display 2D information. This spatial AR tech is not yet rugged enough for widespread use but prototypes are available for testing.”
Partners on FirstModulAR include Jeronimo Grandi, an assistant professor of computer science at UNCG, and NextGen Interactions, a Raleigh-based company that specializes in both augmented and virtual reality experiences. The Hillsborough Police Department and the North Carolina Department of Information Technology’s FirstTech program will also collaborate on the project.
Kopper’s hope for FirstModulAR – and the use of AR by first responders – is to decrease response times, costs and risks to personnel and the public.
“We want to create plug-and-play modules that work across a variety of situations first responders encounter to increase situational awareness, reduce uncertainty and improve decisionmaking,” he said.
