JULY 17, 2023 – A research study at Naval Undersea Warfare Center, Division Keyport assessing participant engagement in virtual reality training compared to traditional in-person training is seeking recruits.
The Naval Innovative Science and Engineering-funded project, called the “Virtual Reality Training Engagement Metrics Study,” is actively recruiting from among the NUWC Division, Keyport workforce and aims to enlist at least 30 more participants with varied backgrounds in VR by mid-August, said principal investigator Kirsten Miskovich, an engineering psychologist at the command.
“The goal of our project is to identify a participant’s level of engagement while in a virtual training simulation,” said Miskovich. “By determining participant engagement, we’re able to demonstrate if the training is effectual. Hopefully, after the final stages of this project, we will be able to support that training in VR is at least as good as, if not better than, traditional classroom methods.”
Study participants take part in two sessions over a two-week period. During the first session, which lasts about 30 minutes, participants complete surveys and perform two rounds of a submarine maintenance procedure in a virtual environment. The chosen VR headset for this study is the HP Reverb G2 Ominicept Edition, said Miskovich.
Two weeks after the initial session, participants are asked to return for a follow-up session in which they perform the same procedure for the purpose of gauging how much they remember from before. This follow-up test is used to determine whether participants have retained their learning, and if so, to what extent, said Miskovich.
The training scenarios focus on maintenance procedures that are performed infrequently or that involve working in confined spaces.
“New sailors rarely get the chance to actually perform that type of maintenance,” said Fallon Orr, a NUWC Division, Keyport project lead and trained 3D artist who laid much of the groundwork for the virtual environment developed for this study. “Our goal was really to give them a way that they could practice that maintenance before they were expected to perform it in the field, especially if it had been several years since they last did it.”
Engagement is being used as a key metric in this study because it is a crucial factor in knowledge retention. The researchers are looking at various indicators of engagement, including eye tracking data to determine participants’ focus of attention, heart rate measurements to gauge stress levels, cognitive load as an additional measure of stress, and assessment of participants’ emotional states, said Miskovich.
If their hypothesis that VR training works at least as well as traditional classroom methods is supported by the study, the cost savings of switching from traditional methods to VR could be huge, given VR’s ability to provide realistic and immersive experiences without the need for physical resources or travel expenses, said Miskovich.
The study officially began in October 2022, and during its initial phase, a considerable amount of time was dedicated to establishing parameters to measure engagement and developing the virtual environment, said Miskovich.
A team of developers and artists created the virtual environment based on computer-aided design models. It was then tested by real-world submarine maintenance workers familiar with the simulated procedures, in order to ensure the simulations were sufficiently realistic, said Miskovich.
While the developers and artists aimed for realism, they also wanted to avoid the “uncanny valley” effect, which occurs when a simulation becomes eerily lifelike and uncomfortably close to resembling the real world without resembling it completely. For this reason, they decided to give the virtual environment a videogame-like aesthetic rather than a photorealistic one, said Miskovich.
“That’s another way to get participants more immersed: to make the environment just believable enough to where they’re not distracted by inconsistencies, but still not too heavily detailed,” said Miskovich.
“There is no real equation for it,” Miskovich added. “Luckily all of our developers are heavy gamers, and through years of playing and testing on their own, they have picked up on what works and what doesn’t work.”
Orr emphasized that the kinesthetic nature of VR training is well suited to those who learn best by physically performing tasks rather than reading or hearing about them in textbooks or lectures.
“Hopefully this will allow the sailors to be more engaged with the training so they get a better understanding without having to sit through hours of PowerPoints and hope they’re retaining something,” said Orr. “We wanted to create an environment where they could actually practice their skills and get a visual familiarity with the spaces that they would be working in.”
NUWC Division, Keyport computer scientist Michael Dean served as lead developer for the virtual environment, and one of his main tasks was creating the interface for tracking engagement metrics. Dean said due to a lack of existing resources and guidance, he found himself breaking new ground in this regard.
“What made it particularly challenging was the newness of the technology,” said Dean. “Because the technology was so new, there was a lack of guidance and readily available information on the internet to accomplish these tasks.”
The researchers plan to conduct a second phase of the study in which the software used in phase one will be brought into a traditional classroom in order to compare engagement, knowledge retention, and performance between traditional and VR classrooms, said Miskovich.
Miskovich stressed that study participants don’t need prior experience with VR or the maintenance procedures featured in the training. “We would prefer a wide range of people,” she said.
Employees of NUWC Division, Keyport who are interested in taking part in this study should reach out directly to Miskovich.
About Naval Undersea Warfare Center Division Keyport
NUWC Keyport provides advanced technical capabilities for test and evaluation, in-service engineering, maintenance and industrial base support, fleet material readiness, and obsolescence management for undersea warfare to expand America’s undersea dominance.
Story by Frank Kaminski
Naval Undersea Warfare Center Division Keyport