Researchers Create Mind-Controlled Robotic Arm With Sense of Touch

In 2004, Nathan Copeland was almost entirely paralyzed by a car accident that broke his neck at the age of 18. But in 2016, after electing to participate in experimental research, Copeland used a robotic arm controlled by electrodes implanted in his brain to shake President Barack Obama’s hand.

Now, new research involving Copeland and a team of scientists has further refined this mind-controlled prosthetic, adding a sense of touch via brain stimulation that allows Copeland to complete tasks like pouring water into a cup more quickly, reports Jon Hamilton of NPR. The experiment is the first time a human has had electrodes implanted into the sensory region of the brain and marks a major advance in the realm of prosthetics as well as our ability to understand and manipulate the inner workings of the brain.

Previously, Copeland had to rely on visual cues to control the robotic arm. He tells NPR that with this iteration of the device he could “see that the hand had touched the object, but sometimes I would go to pick it up and it would fall out.”

But with his newly touch-sensitive robotic arm, described in a new paper published last week in the journal Science, Copeland was able to perform tasks like grasping a small cylinder and moving it to the other side of a table in half the time, reports Max G. Levy for Wired.

“In a sense, this is what we hoped would happen—but perhaps not to the degree that we observed,” says Jennifer Collinger, a biomedical engineer at the University of Pittsburgh and co-author of the research in a statement. “Sensory feedback from limbs and hands is hugely important for doing normal things in our daily lives, and when that feedback is lacking, people’s performance is impaired.”

To allow Copeland to feel what his prosthetic arm touched, the team placed electrodes in the part of his brain associated with processing sensations. These electrodes would work in tandem with the electrodes already in his brain that the brain-computer interface system used to control the robotic arm.

The electrodes that control the arm work by reading the electrical impulses created by Copeland’s brain when he thinks about moving his arm and sending them to a computer that translates them and manipulates the arm accordingly. The new set of electrodes create tactile sensations by stimulating parts of Copeland’s brain with pulses of electricity when the arm comes in contact with an object.

"It turned out that stimulating in the fingertip-related areas in the brain generated sensations that felt like they were coming from the participant's own hand," Collinger tells NPR.

Copeland says adding stimulation to the experience of controlling the prosthetic with his mind didn’t require much adjustment.

"I was already extremely familiar with both the sensations generated by stimulation and performing the task without stimulation. Even though the sensation isn't 'natural’—it feels like pressure and gentle tingle—that never bothered me," says Copeland in the statement. "There wasn't really any point where I felt like stimulation was something I had to get used to. Doing the task while receiving the stimulation just went together like PB&J."

Robert Gaunt, a biomedical engineer at the University of Pittsburgh and co-author of the study, tells Wired the sensory feedback catapulted Copeland’s performance on tasks using the robotic arm to a new level. “We're not talking about a few hundred milliseconds of improvement,” Gaunt tells Wired. “We're talking about a task that took him 20 seconds to do now takes 10 seconds to do.”

Gaunt tells Wired that the time savings is partly down to eliminating the hesitation and fumbling of trying to pick up an object without being able to feel if it’s been grasped securely.

Unlocking the sense of touch is a gateway to prosthetic limbs that can be used more intuitively and more effectively, Jeremy D. Brown, a biomedical engineer at Johns Hopkins University who was not involved in the research, tells NPR. "It's not just the ability to reach into your pocket and grab your keys," he tells NPR. "It's also the ability to hold a loved one's hand and feel that emotional connection."

However, Collinger tells Dennis Thompson of HealthDay that the technology won’t be ready for widespread use anytime soon. For now, the electrodes need to be connected to the brain-computer interface via wires and getting the electrodes in the brain in the first place requires invasive brain surgery. Additionally, each electrode has a limited lifespan meaning they need to be periodically surgically removed and replaced.

“We still have a long way to go in terms of making the sensations more realistic and bringing this technology to people’s homes,” says Gaunt in the statement, “but the closer we can get to recreating the normal inputs to the brain, the better off we will be.”

Alex Fox | | READ MORE

Alex Fox is a freelance science journalist based in California. He has written for the New York Times, National Geographic, Science, Nature and other outlets. You can find him at Alexfoxscience.com.