Researchers at the University of California, San Francisco, have enabled a participant who is paralyzed in all four limbs to control a computer cursor, using only brain activity, by tapping into the brain’s own natural learning system. Without tapping into that system, brain-computer interface (BCI) needs extensive daily retraining in order to work.

“It’s like asking someone to learn to ride a bike over and over again from scratch,” said study senior author Karunesh Ganguly, MD, PhD, an associate professor of in the UC San Francisco Department of Neurology. “Adapting an artificial learning system to work smoothly with the brain’s sophisticated long-term learning schemas is something that’s never been shown before in a paralyzed person.”

Nicholas Weiler, “First ‘Plug and Play’ Brain Prosthesis Demonstrated in Paralyzed Person” at UCSF News

The electronic device that provides the “plug and play” performance is an ECoG electrode array for BCI applications. A small pad of electrodes is placed surgically on the brain’s surface to allow long-term, stable recordings of neural activity (electrocorticography).

Earlier BCI efforts used arrays of sharp electrodes that penetrated the brain tissue like a pincushion. The signals tended to eventually shift position or lose the signal. The goal is a long-term implant that safely enables paralyzed subjects to use brain waves to control electronic devices like cursors and robotic equipment.

The researchers started the subject off with the traditional time-consuming and frustrating method:

Initially, the researchers followed the standard practice of resetting the algorithm each day. The participant would begin by imagining specific neck and wrist movements while watching the cursor move across the screen. Gradually the computer algorithm would update itself to match the cursor’s movements to the brain activity this generated, effective passing control of the cursor over to the user. However, starting this process over every day put a severe limit on the level of control that could be achieved. It could take hours to master control of the device, and some days the participant had to give up altogether.

Nicholas Weiler, “First ‘Plug and Play’ Brain Prosthesis Demonstrated in Paralyzed Person” at UCSF News

Then they switched over to the ECoG approach:

The researchers then switched to allow the algorithm to continue updating to match the participant’s brain activity without resetting it each day. They found that the continued interplay between brain signals and the machine learning-enhanced algorithm resulted in continuous improvements in performance over many days. Initially there was a little lost ground to make up each day, but soon the participant was able to immediately achieve top level performance.

Nicholas Weiler, “First ‘Plug and Play’ Brain Prosthesis Demonstrated in Paralyzed Person” at UCSF News

The researchers and the participant had to help the two systems—brain and computer—work together:

“We found that we could further improve learning by making sure that the algorithm wasn’t updating faster than the brain could follow — a rate of about once every 10 seconds,” said Ganguly, a practicing neurologist with UCSF Health and the San Francisco Veterans Administration Medical Center’s Neurology & Rehabilitation Service. “We see this as trying to build a partnership between two learning systems – brain and computer – that ultimately lets the artificial interface become an extension of the user, like their own hand or arm.”

Nicholas Weiler, “First ‘Plug and Play’ Brain Prosthesis Demonstrated in Paralyzed Person” at UCSF News

Over time, they reported, the participant’s brain sorted through patterns of neural activity, increasing the ones that worked well and eliminating the ones that didn’t. That’s how the brain is generally thought to learn any complex physical task. The new approach also seemed more natural to the brain:

They observed that the participant’s brain activity seemed to develop an ingrained and consistent mental “model” for controlling the BCI interface, something that had never occurred with daily resetting and recalibration. When the interface was reset after several weeks of continuous learning, the participant rapidly re-established the same patterns of neural activity for controlling the device – effectively retraining the algorithm to its former state.

“Once the user has established an enduring memory of the solution for controlling the interface, there’s no need for resetting,” Ganguly said. “The brain just rapidly convergences back to the same solution.”

Nicholas Weiler, “First ‘Plug and Play’ Brain Prosthesis Demonstrated in Paralyzed Person” at UCSF News

Another promising development was that the participant’s brain retained the new knowledge over time, in the same way that one might remember how to tie shoelaces even if one hasn’t worn shoes with laces for a while:

Eventually, once expertise was established, the researchers showed they could turn off the algorithm’s need to update itself altogether, and the participant could simply begin using the interface each day without any need for retraining or recalibration. Performance did not decline over 44 days in the absence of retraining, and the participant could even go days without practicing and see little decline in performance. The establishment of stable expertise in one form of BCI control (moving the cursor) also allowed researchers to begin “stacking” additional learned skills — such as “clicking” a virtual button — without loss of performance.

Nicholas Weiler, “First ‘Plug and Play’ Brain Prosthesis Demonstrated in Paralyzed Person” at UCSF News

The study’s senior author, Karunesh Ganguly, emphasizes that the important result is that the innovation actually be useful:

“We’ve always been mindful of the need to design technology that doesn’t end up in a drawer, so to speak, but which will actually improve the day-to-day lives of paralyzed patients,” Ganguly said. “These data show that ECoG-based BCIs could be the foundation for such a technology.”

University of California – San Francisco, “First ‘plug and play’ brain prosthesis demoed in paralyzed person” at ScienceDaily

The paper was published in Nature Biotechnology. Silversmith, D.B., Abiri, R., Hardy, N.F. et al. Plug-and-play control of a brain–computer interface through neural map stabilization. Nat Biotechnol (2020). https://doi.org/10.1038/s41587-020-0662-5 (restricted access):

Note: This milestone in electrocorticography (ECoG) is one of many that are giving hope to victims of paralysis, limb loss, or blindness:

Through AI, a paralyzed man has regained the sense of touch. In 2016, through advanced technology, he regained the ability to move individual fingers

Prosthetic hand controlled by thoughts alone? It’s here. Decades ago, no one could control a prosthesis only by thought. There is lots of room for the field to grow still.

High tech can help the blind see and amputees feel. It’s not a miracle; the human nervous system can work with electronic information.

and

New mind-controlled robot arm needs no brain implant. The thought-controlled device could help people with movement disorders control devices without the costs and risks of surgery.

Here’s a short presentation by Dr. Karunesh Ganguly from earlier this year on implantable brain-computer interfaces.