February 22, 2018
University of Toronto
A new technique developed by
neuroscientists can reconstruct images of what people perceive based on their
brain activity gathered by EEG.
A new technique developed by
neuroscientists at the University of Toronto Scarborough can, for the first
time, reconstruct images of what people perceive based on their brain activity
gathered by EEG.
The technique developed by Dan
Nemrodov, a postdoctoral fellow in Assistant Professor Adrian Nestor's lab at U
of T Scarborough, is able to digitally reconstruct images seen by test subjects
based on electroencephalography (EEG) data.
"When we see something,
our brain creates a mental percept, which is essentially a mental impression of
that thing. We were able to capture this percept using EEG to get a direct
illustration of what's happening in the brain during this process," says
Nemrodov.
For the study, test subjects
hooked up to EEG equipment were shown images of faces. Their brain activity was
recorded and then used to digitally recreate the image in the subject's mind
using a technique based on machine learning algorithms.
It's not the first time researchers
have been able to reconstruct images based on visual stimuli using neuroimaging
techniques. The current method was pioneered by Nestor who successfully
reconstructed facial images from functional magnetic resonance imaging (fMRI)
data in the past, but this is the first time EEG has been used.
And while techniques like fMRI
-- which measures brain activity by detecting changes in blood flow -- can grab
finer details of what's going on in specific areas of the brain, EEG has
greater practical potential given that it's more common, portable, and
inexpensive by comparison. EEG also has greater temporal resolution, meaning it
can measure with detail how a percept develops in time right down to
milliseconds, explains Nemrodov.
"fMRI captures activity
at the time scale of seconds, but EEG captures activity at the millisecond
scale. So we can see with very fine detail how the percept of a face develops
in our brain using EEG," he says. In fact, the researchers were able to
estimate that it takes our brain about 170 milliseconds (0.17 seconds) to form
a good representation of a face we see.
This study provides validation
that EEG has potential for this type of image reconstruction notes Nemrodov,
something many researchers doubted was possible given its apparent limitations.
Using EEG data for image reconstruction has great theoretical and practical
potential from a neurotechnological standpoint, especially since it's
relatively inexpensive and portable.
In terms of next steps, work
is currently underway in Nestor's lab to test how image reconstruction based on
EEG data could be done using memory and applied to a wider range of objects
beyond faces. But it could eventually have wide-ranging clinical applications
as well.
"It could provide a means
of communication for people who are unable to verbally communicate. Not only
could it produce a neural-based reconstruction of what a person is perceiving,
but also of what they remember and imagine, of what they want to express,"
says Nestor.
"It could also have
forensic uses for law enforcement in gathering eyewitness information on
potential suspects rather than relying on verbal descriptions provided to a
sketch artist."
The research, which will be
published in the journal eNeuro, was funded by the Natural Sciences and Engineering
Research Council of Canada (NSERC) and by a Connaught New Researcher Award.
"What's really exciting
is that we're not reconstructing squares and triangles but actual images of a
person's face, and that involves a lot of fine-grained visual detail,"
adds Nestor.
"The fact we can
reconstruct what someone experiences visually based on their brain activity
opens up a lot of possibilities. It unveils the subjective content of our mind
and it provides a way to access, explore and share the content of our
perception, memory and imagination."
Story Source:
Materials provided by University of Toronto. Note:
Content may be edited for style and length.
Journal Reference:
Dan Nemrodov, Matthias
Niemeier, Ashutosh Patel, Adrian Nestor. The Neural Dynamics of Facial
Identity Processing: insights from EEG-Based Pattern Analysis and Image
Reconstruction. eneuro, 2018; ENEURO.0358-17.2018 DOI: 10.1523/ENEURO.0358-17.2018
No comments:
Post a Comment