New Device Allows Man With Paralysis to Type by Imagining Handwriting
When the man visualizes his written messages, a pair of electrical sensors measure his brain activity and translate it into letters
/https://tf-cmsv2-smithsonianmag-media.s3.amazonaws.com/filer/d7/78/d7789122-be85-44e1-9160-604cb393b1bc/13-handwriting34.jpg)
An experimental brain-computer interface has allowed one man who is paralyzed below the neck to gain the ability to type by thinking about handwriting, according to research published on May 12 in the journal Nature.
The man became paralyzed after a spinal cord injury in 2007, and joined the research project at Stanford University and the Howard Hughes Medical Institute nine years later, Ian Sample reports for the Guardian. After years of development, the man can use the device to write up to 18 words per minute when connected to the system, with 94 percent accuracy on each letter. When the scientists added autocorrect to the program, its accuracy improved to 99 percent.
"We can also envision it being used by someone who might have had a spinal cord injury who wants to use email," says Stanford University neurosurgeon Jaimie Henderson to Jon Hamilton at NPR. "Or, say, a computer programmer who wants to go back to work."
The device uses two sensors called microelectrode arrays placed near the part of the brain used for handwriting, just under the surface of the brain. The arrays could measure signals from about 100 neurons.
First, the research team asked the man to imagine writing each letter by hand while the computer monitored the activity of his brain, which trained the program to recognize different neural activity for each letter. Similar-shaped letters, like lowercase “r,” “h” and “n” caused the most mistakes for the program. Eventually, he could practice writing words and sentences, and the scientists would ask questions for him to answer.
"He was very happy when he was able to write out messages in response to the questions we asked him,” says Henderson to NPR. "He was pretty excited about this."
University of Washington neural engineering specialist Amy Orsborn, who was not involved in the new study, tells the Guardian that it is “a remarkable advance” in the field.
The new device follows decades of work developing tools to help people who are paralyzed to communicate. Not all have relied on brain signals—Stephen Hawking operated his computer by tensing one muscle in his cheek. Previous experiments using brain-computer interfaces like the new one developed at Stanford asked participants to use their thoughts or eye movements to guide a cursor on a screen to point and click letters. However, those can demand a lot of energy and attention from the user.
“If you’re using eye tracking to work with a computer then your eyes are tied to whatever you’re doing,” says Henderson to New Scientist’s Matthew Sparkes. “You can’t look up or look around or do something else. Having that additional input channel could be really important.”
The handwriting-based system is much faster, possibly because participants already know how it feels to think about writing letters on paper. The man who used the device found he could type fastest when he imagined writing the letters smaller, says Howard Hughes Medical Institute neuroscientist Frank Willett to the Guardian.
Researchers may someday try signals other than handwriting to assist with communication, like typing on a keyboard. Or “instead of detecting letters, the algorithm would be detecting syllables, or rather phonemes, the fundamental unit of speech,” says Willett to the Guardian.
The device is not quite a prototype yet, and the alphabet that the computer could interpret was limited to lowercase letters and a few punctuation marks. The device also had to be recalibrated about once per week, notes John Timmer for Ars Technica.
"I was introduced to this concept over 10 years ago, and I thought it was quite a bit of science fiction. Then roughly about five years later it was shown to be not to be such science fiction after all. So I think we're seeing a progression. It's really quite exciting,” says John Ngai, director of the National Institutes of Health’s BRAIN Initiative which funded the research, to NPR. "But it was only on one subject in a laboratory setting. So at the moment it's a great demonstration of proof of principle."