Neuroprosthetics: The role of neuroplasticity

Introduction

In 2017, 57.7 million people were living with limb amputation due to traumatic causes worldwide. Losing the ability to move an arm or a leg is an experience that deeply affects independence and quality of life. Imagine being able to regain the ability to move that limb through a robotic extension controlled entirely by your mind: advances in neuroprosthetics are turning this vision into reality.

Neuroprosthetics are devices designed to restore a motor, sensory or cognitive function that has been damaged as a result of an injury or a disease. In an amputee or tetraplegic patient, the limb may be either absent or disconnected from the brain. However the brain's motor areas that originally sent signals to move the limb remain functional. Signals from these areas of the brain can then be employed to control an external device, such as a prosthetic arm.

Neuroprosthetic devices link directly with the central or peripheral nervous systems through what are called brain machine interfaces (BMIs) – a direct communication pathway between the brain and a man-made computing device.

In these brain machine interfaces neuroplasticity plays a key role. Neuroplasticity, also known as neural plasticity or brain plasticity, is a process that involves adaptive structural and functional changes to the brain. A good definition is 'the ability of the nervous system to change its activity in response to intrinsic or extrinsic stimuli by reorganizing its structure, functions, or connections.'

The main focus of our work is, in fact, this very remarkable ability of our brain. Neuroplasticity allows for a 'retraining' of the nervous system, which, from a theoretical standpoint, should make it possible to integrate neural prostheses as functional extensions of the body, making the brain 'reprogram' itself to control prosthetic substitutes as it does one's own body parts.

In this article we will first illustrate the general functioning of neuroprosthetics and then focus on the core of our project: neuroplasticity and its potential role in this field.

Neural Decoding

In order to understand how neuroprosthetic works, we first focus on neural decoding. 'Neural decoding is a neuroscience field concerned with the hypothetical reconstruction of sensory and other stimuli from information that has already been encoded and represented in the brain by networks of neurons.' In other words, it involves interpreting brain signals to extract meaningful information, a crucial step in applications such as brain-machine interfaces.

In our brain, information is encoded through action potentials, or electrical impulses generated by neurons. In order to interpret them, neural signals must contain structured representations of the external world. For example, in vision, the brain interprets a stimulus, e.g. an image of a hat, by adapting neural responses to the statistical properties of that stimulus. This process is guided by the 'efficient coding hypothesis', which suggests that frequent patterns are processed more efficiently. Neural decoding thus uses these statistical patterns to reconstruct the original stimulus, creating a continuous feedback loop that lasts through thought and action.

This process shows the transition from implicit encoding (low-level features like edges and textures) to explicit encoding (recognizable objects, such as a hat). Higher-level brain areas process more abstract representations, simplifying the process; however, decoding complex information, such as recognizing a hat under different lighting conditions or angles, requires more sophisticated and non-linear models.

Neural decoding has a central relevance in motor control and decision-making. In particular, in the context of neuroprosthetics, sensors implanted in motor-related brain regions record neural activity which will be analyzed by AI-powered decoders to predict intended movements. For instance, when a person thinks about moving their hand, the decoder interprets the corresponding neural patterns and translates them into signals that control a prosthetic arm in real time.

Advances in machine learning and AI have significantly improved neural decoding models. These models can be broadly categorized into two types: fully observed models (e.g., Generalized Linear Models or GLMs), and latent variable models. The former analyzes direct interactions between neurons, while the latter uncovers hidden patterns in neural data using techniques like principal component analysis (PCA), variational autoencoders (VAEs), and contrastive learning.

Another distinction in models can be identified between task-driven models and data-driven models. The task-driven models focus on understanding why neural systems encode information in specific ways. These models assume that neural responses are shaped by learning constraints, brain architecture, and task objectives. Unlike data-driven models, which rely on large datasets, task-driven models require structured data, such as labeled images for object recognition.

In summary, neural decoding is the process of interpreting brain signals to generate outputs leading to motor control, perception, and learning. It represents the foundation of technologies like neuroprosthetics joining our brain to external devices.

Adaptive Learning

Adaptive learning (or teaching) is the expression referring to the ability of a system to improve over time based on feedback and experiences. This is the key difference between neuroprosthetic devices and traditional prosthetic devices: adaptive learning enables these systems to 'learn' and adapt to the user's unique neural patterns and intentions making their response finer and finer.

But how does this process really work? It all starts from brain-computer interfaces (BCIs), which interpret neural signals and translate them into commands for controlling external devices. BCIs use 'error signals', which are the differences between the intended and actual outcomes, to change and improve their decoding algorithms. This process is based on the so-called closed-loop-systems, where the device receives continuous feedback and makes real-time adjustments. We can identify four main steps:

1. signal detection: the BCI records neural signal from the user.

2. decoding: the system translates these signals into motor commands, moving the prosthetic limb accordingly.

3. error detection: the system detects an error signal when there are differences between the intended and actual outcomes.

4. recalibration: the system adapts and rewrites its decoding algorithm improving accuracy.

For instance, if the user tries to move the prosthetic arm, but the movement is not as they intended, the system detects the error and subsequently corrects, to better align with the user's will.

This adaptive learning process is crucial for the purpose of achieving precision, control, and ease of use, which are very desirable characteristics for a prosthetic device.

What is Neuroplasticity?

As previously mentioned, the term neuroplasticity refers to the ability of neural networks to change its activity in response to intrinsic or extrinsic stimuli by reorganizing its structure, functions, or connections.

The word plasticity means 'changeability' or 'moldability': 'plastic' comes from the Latin 'plasticus' which comes in turn from Greek 'plastikos', i.e. 'capable of change or of receiving a new direction'.

Neuroplasticity thus refers to change in nervous systems.

This is not something to be taken lightly or regarded as trivial: seventy years ago, the idea that nervous tissue can change was anathema to neuroscience. It was widely believed that the mature brain is a fixed structure.

Since then, a huge body of research has shown not only that the brain can change, but also that it changes continuously throughout life, in one way or another, in response to everything we do and every experience we have.

Neuroplasticity is an umbrella term referring to the many different ways in which the nervous system can change: neuroscientists use it to describe a wide variety of phenomena and it means different things to researchers in different subfields.

For the purpose of this discussion, we will focus on synaptic plasticity, while acknowledging that it does not fully encompass the broader concept of neuroplasticity.

Synaptic plasticity specifically refers to the change that occurs at synapses, the junctions between neurons that allow them to communicate. Synapses strengthen or weaken over time, in response to increases or decreases in their activity.

The synapse

The idea that synapses could change, and that this change depended on how active or inactive they were, was first proposed in the 1949 by Canadian psychologist Donald Hebb.

Activity-dependent synaptic plasticity is widely believed to be the basic phenomenon underlying learning, memory and is also thought to play a crucial role in the development of neural circuits.

Modeling Synaptic Plasticity

In order to better understand how neuroplasticity works we developed a Python simulation. The goal was to create a simple model of synaptic plasticity. We used Brian2 library, which allows to efficiently simulate spiking neural network models.

We started by creating a simple integrate-and-fire neuron. The integrate-and-fire neuron model is one of the most widely used models for analyzing the behavior of neural systems. It describes the membrane potential of a neuron in terms of the synaptic inputs and the injected current that it receives. We started by setting a resting potential of -70 mV and a threshold of -50 mV. We set external current of 1 nA to the neuron and membrane capacitance to 1 nF. Then we ran the simulation for 100 ms and plotted the membrane potential over time.

After that we tried to gradually increase the accuracy of our model: first we added a refractory period after the firing.

Firing pattern with refractory period

At this point we went on to simulate synaptic communication between two neurons.

Neurons communicate via a combination of electrical and chemical signals. Communication between neurons occurs at the synapses, where specialized parts of the two cells (i.e., the presynaptic and postsynaptic neurons) come within nanometers of one another to allow for chemical transmission. The presynaptic neuron releases a chemical (i.e., a neurotransmitter) that is received by the postsynaptic neuron's neurotransmitter receptors.

Hence, we created two neurons, a presynaptic one and postsynaptic one, and set up a Synapses object to connect the former to the latter. We started with a low synaptic weight (the weight of a synapse describes the strength of the connection) and gradually increased it. Plotting the membrane potentials of both neurons, we noted how the postsynaptic neuron's response changed as the synaptic weight increased.

After that we proceeded to simulate basic Hebbian learning. Hebbian learning is a classic theory of neuroscience. In 1949, Donald Hebb conjectured that if input from neuron A often contributes to the firing of neuron B, the synapse from A to B should be strengthened. The basic Hebb can be summarized as 'cells that fire together wire together'. This means that the synaptic weight between two neurons tends to increase whenever the two neurons both fire.

We created two connected neurons where the synaptic weight between them increased every time both neurons fired simultaneously. We initialized the weight at 5.0 and observed how it changed over time.

Hebb's original suggestion (what we referred to as basic Hebb rule) concerned only increases in synaptic strength, but it has been generalized to include decreases in strength arising from the repeated failure of neuron A to be involved in the activation of neuron B.

Spike timing dependent plasticity (STDP) is a phenomenon in which the precise timing of spikes affects the sign and magnitude of changes in synaptic strength. STDP is often interpreted as the comprehensive learning rule for a synapse.

With STDP, repeated presynaptic spike arrival a few milliseconds before postsynaptic action potentials leads in many synapse types to Long-Term Potentiation (LTP) of the synapses, whereas repeated spike arrival after postsynaptic spikes leads to Long-Term Depression (LTD) of the same synapse.

We implemented STDP by creating a synapse between two neurons with a weight that updates in the following way: if the presynaptic neuron fires before the postsynaptic neuron, the weight should increase slightly; if the reverse, it should decrease. We set up the neurons to spike regularly and observed the weight changes.

The presynaptic neuron spikes every 30 ms, while the postsynaptic neuron spikes every 40 ms. This creates a periodic pattern of relative spike timing that repeats every 120 ms, as 120 ms is the least common multiple of 30 ms and 40 ms. This periodic alternation of LTP and LTD based on the repeating spike intervals results in a cyclic pattern for the synaptic weight.

These simulations provide insight into how communication between neurons takes place and how it is connected to neuroplasticity. We believe that having a concrete, albeit simplified, model can give a better overall understanding and prepare to approach the next section.

Neuroplasticity in Prosthetics

We are going to consider the role of neuroplasticity in the context of neuroprostheses from two different perspectives: firstly as a phenomenon induced by the use of a prosthetic device, and secondly as a characteristic that allows to better integrate neural prostheses as functional extensions of the body.

It should be pointed out that these are not separate phenomena, but rather two faces of the same coin - the distinction is made solely for the purpose of the discussion.

Neuroplasticity Modulated by the Use of Prostheses

A limb amputation has, as a direct consequence, a peripheral and central nervous systems' reorganization.

Neuroplasticity thus comes into play in two main ways, at two different times after the injury.

Initially there is an activation of already existing (but 'silent') synaptic connections, which is due to the fact that the limb loss 'exposes' a new area. Subsequently there is a second mechanism, which can take weeks or months, linked to new connection creation and stabilization, as well as to an increasing amount of synaptic plasticity in the CNS (central nervous system) areas previously connected and adjacent to the one governing the lost limb.

Neuroplasticity in amputees seems to be influenced by the use of hand prostheses, both in the initial phase and in the long-term period, provided that the prosthesis is able to restore sensory inputs and to execute motor commands.

Following 'Neuroplasticity in amputees: Main implications on bidirectional interfacing of cybernetic hand prostheses' (G. Di Pino, E. Guglielmelli, P.M. Rossini, 2009), we report that:

(1) the level of CNS reorganization can be used as a parameter of the effectiveness achieved by the prosthetic device and its interfaces, in restoring the hand's physiological functionality

Roughly speaking, we thus could say that neuroplasticity is an indicator of how well the prosthesis performs.

(2) the prosthetic system could be seen as a neurorehabilitation tool, as it could induce reduction in aberrant plasticity and promote 'good' plasticity

Neuroplasticity can also evolve toward unpleasant effects like the phantom-limb pain syndrome. The use of the prosthetic device seems to mitigate those negative consequences and enhance positive neuroplasticity.

Neuroplasticity as a Tool to Better Integrate Neural Prostheses

As already highlighted in the previous paragraph, neuroplasticity is strictly linked with the effectiveness of the prosthetic device. Moreover there is evidence (Orsborn AL, Moorman HG, Overduin SA, Shanechi MM, Dimitrov DF, Carmena JM: 'Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control', Neuron, 2014) that neuroplasticity plays a critical role in developing robust, naturally controlled neuroprostheses.

Indeed beneficial neuroplasticity yields performance improvements, skill retention, and resistance to interference from native motor networks.

What do we mean by 'interference from native motor networks'?

Brain-machine interfaces (BMIs) create new functional neural circuits for actions that are distinct from the natural motor system. Many studies found that the relationship between neural activity and movement changes substantially between natural movements and BMI control (Taylor et al., 2002; Carmena et al., 2003; Ganguly and Carmena, 2009; Ganguly et al., 2011). These changes are likely due in part to key differences between the natural motor and BMI systems, such as different sensory feedback.

Much like our natural limbs, neuroprostheses will ultimately be used for a myriad of behaviors and in coordination with existing motor and cognitive functions. Tasks that activate brain areas near or overlapping with those used for BMI control, however, may cause performance disruptions: that is how native motor networks can undermine the performance of the prosthetic device.

Going back to the main point, as already mentioned, learning to control a BMI can induce neuroplasticity. Learning may also facilitate the formation of BMI-specific 'control' networks.

Neuroplasticity is thus found to generate a specialized BMI control network that allows skillful control.

The robust control attained via neuroplasticity is particularly useful for neuroprosthetic applications: learning (and the associated neural map formation) is critical for achieving performance that can transfer across contexts.

Summing up, neural plasticity can provide performance that is reliably recalled over days, and resistant to interference from native motor networks and is thus crucial for achieving robust, flexible neuroprosthetic control that can be maintained long term.

Real World Examples of Neuroprosthetics

Despite their futuristic appearance, there are actually real-world examples of neuroprosthetic devices. One important and revolutionary example is the LUKE arm.

LUKE is an acronym for 'Life Under Kinetic Evolution', but it takes its name from the famous series Star Wars, referring to Luke Skywalker and its iconic robotic arm. After eight years of research and study, in June 2017 the new neuroprosthetic arm created and perfected by Dean Kamen, engineer and inventor, and Defense Advanced Research Projects Agency (DARPA) in collaboration with Department of Veterans Affairs (VA), was first provided to two veterans who had lost their arm.

This device is connected to an external computer which gets the input from the user and gives the output to the device making it move. Usually the shoe of the user is employed as a controller: you move your foot and the device, which is the size of a piece of candy, sends the command to the prosthetic arm. Unlike traditional prosthetics, the LUKE Arm can conduct multi-joint movement and it is run by intuitive control.

Alternative control methods, such as electromyographic (EMG) sensors able to detect muscle activity, are also a matter of study in order to improve ease of use. This same technology has been adapted for different configurations, including the forearm and the hand. The peculiarity of this neuroprosthetic device is the 'feeling' hand: the sensory feedback system enables the user to feel the sensation of touching and this eases everyday tasks such as grabbing objects with precision and without hesitation. The arm is currently available from Mobius Bionics of Manchester but, nowadays, this kind of technology is quite expensive.

These kinds of achievements are possible thanks to years of research and investment. The DARPA organization is carrying on the Prosthetics Program - which aims to ease the life of veterans that have lost limbs during their time serving the nation. This project began in 2006 and ended in 2017, with LUKE arm.

Meanwhile, researchers at the University of Utah are pioneering a new technology that connects neuroprosthetic limbs directly to the user's nervous system. Their work focuses on linking prosthetic arms to functioning nerves in the shoulder or chest. Currently the research has connected a below-the-elbow arm to a remaining working nerve of the arm. This is possible thanks to Utah Slanted Electrode Array, invented by the biological engineer and Emeritus Distinguished Professor Richard A. Normann. This array is built of 100 microelectrodes and wires that are connected to the cut nerve and when receiving a stimulus it transmits it to a computer which rewrites it as a digital signal for the prosthetic arm. Moreover it can also give feedback in order to improve the accuracy.

There are tasks that require not only a stimulus to initiate movement, but also information about the amount of force or pressure needed to, e.g., grab an object. This kind of information is not available just from looking at the given object, but must be evaluated by touching and feeling it. To do so, the prosthetic hand has been equipped with sensors, which send signals to the array. These signals are then translated into a stimulus. The user then feels the sensation of touching and grabbing the object. This important step was achieved by biomedical engineering associate professor Gregory Clark and its team at the University of Utah and represents a great advancement in the field of neuroprosthesis, since it provides the user a 'natural' sense of touch and control over the artificial limb.

SENSY Leg

Another important neuroprosthetic device is the SENSY prosthetic leg. The SENSY leg is the result of years of research led by Francesco Petrini, founder of SensArs Neuroprosthetics, and Stanisa Raspopovic, professor of Biomedical Engineering. The project, started in 2017, became possible thanks to the financial support of Health Technologies Holding (HTH), Eurostars, German Federal Ministry of Education and Research (BMBF) and European Union's Horizon Research and Innovation Program, which made the SENSY leg one of the first European achievements in neuroprosthetics.

Back then, prosthetic limbs did not provide feedback: this implied risk of falls and low mobility. Moreover the prosthetic limb was perceived as an external object rather than an integrated part of the user's body. These were the main motivations that led the researchers to improve the obsolete prosthetic legs introducing a new method of delivering sensory feedback. Key to this innovative idea was the use of neural interfaces such as Utah's SEA, used for the prosthetic arm as well, the FINEs (Flexible Intraneural Electrodes) and TIMEs (Targeted Intraneural Microelectrodes). All of these interfaces work similarly: they create neural feedback by stimulating the nerve as the amputated limb would have done before.

The interface does not work alone, in order to send signals to the nerve it has to receive them from the prosthetic device itself. For this reason, the foot and the knee are improved with sensors for weight and equilibrium. This kind of technology helps the user to command the leg, to maintain equilibrium and to walk without risk of falls. Moreover its advanced design and integration of neural feedback systems contributes to a more natural feeling of the prosthetic, allowing users to integrate it into their physical and mental perception of their own body.

While the LUKE arm revolutionized upper-limb prosthetics with multi-joint movement and sensory feedback, the SENSY leg offered an important advancement in lower-limb prosthetics' filed with its integrated sensory systems that improve balance and walking. Despite their differences, not only both of them use similar technology but also, and more importantly, both ideas were driven by empathy and by the desire to help individuals with amputations lead more independent lives and have a more intuitive and natural relation with their prosthesis.

Conclusion

The aim of this article was to provide a foundational understanding of how the brain can adapt and change and what role this reorganizational ability plays in the context of neuroprostheses.

We started by providing two concepts that are very significant in neuroprosthetics: we looked into neural decoding, i.e. translating neural activity into motor commands, then moved on to adaptive learning, i.e. the ability of this new kind of prostheses to learn and improve accuracy.

Next, we introduced neuroplasticity and focused on its involvement in neuroprosthetics, starting from the changes in the brain that these devices induce and moving on to how this neural 'rewiring' allows for better control of artificial limbs.

Finally, we presented some of the most important real-world examples of neuroprostheses.

We conclude by quoting a fascinating article, 'Human-robotic interfaces to shape the future of prosthetics' (EBioMedicine, 2019 Aug) – which we encourage the reader to take a look at: 'We are living in exciting times and can expect a future in which human–robotic interfaces will become increasingly accessible, enabling people to live an increasingly healthy, high-quality life.

Bibliography

McDonald CL, Westcott-McCoy S, Weaver MR, Haagsma J, Kartin D. Global prevalence of traumatic non-fatal limb amputation. Prosthetics and Orthotics International. 2020;0(0). doi:10.1177/0309364620972258

Mateos-Aparicio P, Rodríguez-Moreno A. The Impact of Studying Brain Plasticity. Front Cell Neurosci. 2019;13:66.

Oweiss KG, Badreldin IS. Neuroplasticity subserving the operation of brain-machine interfaces. Neurobiol Dis. 2015 Nov;83:161-71. doi: 10.1016/j.nbd.2015.05.001. Epub 2015 May 9. PMID: 25968934; PMCID: PMC4639466.

Neural Decoding definition (link:https://en.wikipedia.org/wiki/Neural_decoding)

Weygandt Mathis, M., Perez Rotondo, A., Chang, E. F., Tolias, A. S., & Mathis, A. Decoding the brain: From neural representations to mechanistic models. 2024 https://doi.org/10.1016/j.cell.2024.08.051

Bockbrader, M. (Year). Upper limb sensorimotor restoration through brain–computer interface technology in tetraparesis. 2019 https://doi.org/10.1016/j.cobme.2019.09.002

Zubarev I, Zetter R, Halme HL, Parkkonen L. Adaptive neural network classifier for decoding MEG signals. Neuroimage. 2019 Aug 15;197:425-434. doi: 10.1016/j.neuroimage.2019.04.068. Epub 2019 May 4. PMID: 31059799; PMCID: PMC6609925.

Jayaram et al., (2016) Transfer Learning in Brain-Computer Interfaces

Ziat, Mounia & Bensmaia, Sliman. (2015). Neuroprosthetics. 10.1016/B978-0-08-097086-8.55057-0.

Mateos-Aparicio P, Rodríguez-Moreno A. The Impact of Studying Brain Plasticity. Front Cell Neurosci. 2019 Feb 27;13:66. doi: 10.3389/fncel.2019.00066. PMID: 30873009; PMCID: PMC6400842.

Costandi M. Neuroplasticity. The MIT Press; 2016.

Hughes JR (January 1958). 'Post-tetanic potentiation'. Physiological Reviews. 38 (1): 91–113. doi:10.1152/physrev.1958.38.1.91. PMID 13505117.

https://qbi.uq.edu.au/brain-basics/brain/brain-physiology/what-synaptic-plasticity#:~:text=Synaptic%20plasticity%20is%20change%20that,by%20Canadian%20psychologist%20Donald%20Hebb.

Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems, Peter Dayan and L.F. Abbott, 2001

Lovinger DM. Communication networks in the brain: neurons, receptors, neurotransmitters, and alcohol. Alcohol Res Health. 2008;31(3):196-214. PMID: 23584863; PMCID: PMC3860493.

Mobius Bionics Arm (https://mobiusbionics.com/luke-arm/)

Video: video segment by Colomba films: At the Edge of Fiction: The LUKE Arm (Colomba Films, Chile);

Video: Blue Tech Wave Media: The Luke Arm: A new era in AI-driven prosthetics

Video: DARPAtv: Inventor Dean Kamen Discusses DARPA's LUKE Arm

Vincent Horiuchi Prosthetic arm can move and feel: University of Utah engineers help develop motorized prosthetic arm that can sense touch, move with your thoughts. June 2019

Raspopovic, S., Valle, G., & Petrini, F. M. (Year). Sensory feedback for limb prostheses in amputees. 2021, https://doi.org/10.1038/s41563-021-00966-9

Petrini, F. M., Bumbasirevic, M., Valle, G., Ilic, V., Mijović, P., Čvančara, P., Barberi, F., Katic, N., Bortolotti, D., Andreu, D., Lechler, K., Lesic, A., Mazic, S., Mijović, B., Guiraud, D., Stieglitz, T., Alexandersson, A., Micera, S., & Raspopovic, S. (2019). Sensory feedback restoration in leg amputees improves walking speed, metabolic cost and phantom pain. Nature Medicine, 25(9), https://doi.org/10.1038/s41591-019-0567-3

The Brain ForumSensArs Neuroprosthetics, Francesco Petrini and Stanisa Raspopovic, Audience Award

https://www.cs.jhu.edu/~ayuille/JHUcourses/ProbabilisticModelsOfVisualCognition2020/Lec6/HebbianYuilleKersten.pdf

https://www.frontiersin.org/journals/computational-neuroscience/articles/10.3389/fncom.2010.00019/full

http://www.scholarpedia.org/article/Spike-timing_dependent_plasticity

Orsborn AL, Moorman HG, Overduin SA, Shanechi MM, Dimitrov DF, Carmena JM. Closed-loop decoder adaptation shapes neural plasticity for skillful neuroprosthetic control. Neuron. 2014

Di Pino G, Guglielmelli E, Rossini PM. Neuroplasticity in amputees: main implications on bidirectional interfacing of cybernetic hand prostheses. Prog Neurobiol. 2009

Mackenzie Weygandt Mathis, Adriana Perez Rotondo, Edward F. Chang, Andreas S. Tolias, Alexander Mathis: Decoding the brain: From neural representations to mechanistic models. p5814-5832, October 17 2024