WASHINGTON (Army News Service, May 18, 2015) – Brainwave detection may one day make it easier for Army researchers to train computers to detect threats captured in digital images of the battlefield.
Jean Vettel, Ph.D., a neuroscientist with the Army Research Laboratory at Aberdeen Proving Ground, Maryland, was wearing headgear lined with brain wave-sensing EEG, or “electroencephalography” sensors, May 14, in the Pentagon courtyard as part of the display she manned at DOD Lab Day there.
She confirmed almost immediately that, in at least her part of the Army, researchers were not working on how Soldiers might one day control a combat vehicle or helicopter using only their brains.
“Evolution has evolved our hands to be very effective in doing that task,” she said. “So we are substantially slower and less accurate if we use brain signals – for a healthy individual – to do that task.”
What she and the translational neuroscience branch are working on, however, is just as interesting, if not more practical.
“What we are interested in doing is recording ongoing brain dynamics so we can have technology adapt to our Soldiers,” she said.
Vettel said that robotic assets throughout the Army collect “millions of images.” For those images to be useful to the Army, the image must be assessed to determine if it is of something that is a threat, or of something that is benign. The best way to do that now is for a human to look at the image and make that determination.
“But we don’t have sufficient time or resources to look though all those images,” she said.
So what Vettel and the neuroscience branch want to do is teach a computer to identify an image of a threat. For that to happen, there needs to be a sizable enough stack of both threat and benign images, which an algorithm can learn from.
Vettel and the branch plan to use EEGs hooked up to combat-experienced Soldiers to develop the training materials for those algorithms – those stacks of threatening and non-threatening images.
With EEG sensors on a Soldier’s head, Vettel can flash a series of images on a screen. The Soldier, without pressing any buttons or even providing a verbal response, will be able to tag the image as threatening or not using only his brain waves.
It is much, much quicker to tag images in this fashion than it would be to ask a Soldier to press a button or offer a verbal confirmation on their assessment of a photo, Vettel said.
“And then when we have images labeled, we can take those images and give it to a machine learning algorithm that can learn to distinguish between threatening or non-threatening images. We’ll have used Soldier expertise to train the algorithm,” she said.
It is not just computers that can learn from those images. Other, less combat-experienced Soldiers might also be able to learn from them as well, Vettel said.
“You have Soldiers who have deployed, some who haven’t,” she said. “You can then have the folks who have deployed, who have the gut instinct about whether something is a threat. You can start communicating that to somebody who hasn’t been deployed – not in semantics about what it is they are detecting was a threat, but by showing them a whole bunch of images that the combat-experienced Soldiers say are threatening, but that the non-combat-experienced Soldiers say are not threatening.”
SOLDIER AS A SENSOR
This type of brain wave-detecting technology will not remain stuck in the lab, either. Vettel’s branch has an Army combat helmet with EEG sensors built into the soft material inside. Soldiers wearing the helmet would have the sensors plugged into a computing device, a tablet or smart phone that they wear on their body. The entire setup makes use of what Vettel calls a “noisy sensor,” that is, a Soldier himself.
In a squad out on patrol, some of the Soldiers will be more experienced than others. Those more experienced Soldiers will be more in tune with threats in the environment around them. They will have a “gut instinct” for what is dangerous or threatening, Vettel said, something she said Soldiers have told her cannot be easily put into words. But they would not need to put that gut instinct into words to transmit their concerns or heightened awareness to their less-experienced teammates.
With such technology in their helmets, the heighted anxiety, fear, and tension of the more experienced Soldiers will be detected by their helmet sensor and can be, when appropriate, transmitted to their less-experienced teammates. An entire squad – those who are acutely aware of even the most subtle shifts in the environment, and those who are not yet skilled enough to recognize it – can all become aware at the same time that danger is afoot.
The technology would not just help those less-experienced Soldiers know what their more seasoned teammates are feeling, Vettel said, it might even provide an edge to those more-experienced Soldiers, who are fatigued.
“Not only is this good for units where you have novice Soldiers,” Vettel said, “but also for Soldiers who are sleep deprived.”
Vettel said she expects such brain-sensing technologies might be ready for the Army in the next 25 years.
“We are a future capability,” she said. “A lot of our research now focuses on reliably recording these signals in complex settings. It’s, overall, letting the technology adapt to the Soldier, rather than having the Soldier adapt to the technology.”