Learning to “see” with language: a Russian scientist on the development of neural interface technologies and prosthetics

- Recently, Russian scientists have proposed new methods for intelligent analysis of human brain activity signals, we are talking about special non-invasive devices that are attached to the head and can read brain signals.

Such technologies are proposed to be used for the daily rehabilitation of patients who have lost motor functions.

Alexei Evgenievich, please tell us how such devices work? 

— Neurointerfaces allow you to establish communication with the human brain directly, bypassing natural channels.

Such devices read brain activity and turn it into commands for external devices. 

There has been a lot of excitement around this area for a long time, many research groups and companies are developing such devices.

However, the results are often far from the expectations of investors, who assume that such technologies will allow a person, using only the "power of thought", to control virtual limbs, etc. But in practice, there are a number of difficulties that non-specialists are not aware of. 

First, learning how to use such devices is very difficult.

Even patients who receive myographic prostheses learn to work with them within 3-4 months, and only 30% of people get it. 

In this case, we are talking about myographic prosthetics and interfaces that interpret the myogram (recording of electrical signals of a muscle or muscle group that occurs during their contraction. -

).

A person must tense his muscles in a certain way in order to give several discrete commands to the prosthesis to change the grip or mode.

The situation is even more complicated in the case of non-invasive neural interfaces, for example, in ideomotor (IMBMI) neural interfaces configured to decode the intention to perform a motor action.

Reading brain activity with non-invasive methods is much more difficult than reading muscle activity.

So far, it has been possible to generate 3-4 commands with the help of imBMC with average reliability, which, after decoding brain activity, can be transmitted to a prosthesis or other executing device.

Unfortunately, so far this technology is not very suitable for creating full-fledged control systems for prostheses with a large number of degrees of freedom.

Therefore, despite the romanticism and attractiveness of decoding signals from the brain, devices based on the decoding of neuromuscular signals are still the most relevant.

After all, even a person with a prosthesis or a hand whose function is impaired as a result of a stroke has residual muscle tension, and the patient can be taught to use it. 

For example, Stephen Hawking, although he had the most advanced technologies at his disposal, did not use a neural interface at all, but a special apparatus that read the deformation of the muscles on the face.

- It turns out that when headlines appear in the media that scientists have learned to read minds, these are just journalistic sensations?

- In most cases, yes.

Let me give you an example: recently they wrote that scientists allegedly learned to decipher human thoughts using fMRI, a device for recording the concentration of oxygenated hemoglobin in brain tissue.

A person was read some text, and scientists recorded the activity of his brain while listening, thanks to which they were able to determine which particular segment of the narrative was being reproduced.

Most likely, in this work, such recognition is based on the registration of emotions.

At the same time, the accuracy of guessing was very low, but in the media it was presented as news about "mind reading".

There is a lot of hype around neural interfaces, but much more “mundane” technologies are in demand for real medical applications. 

• Gettyimages.ru

— What developments are being carried out in this area in the world and in Russia?

- A non-invasive interface that allows a person to move either a virtual hand on a special screen, or using an exoskeleton, has already been invented.

There are such technologies in Russia.

However, now more attention is still paid to invasive interfaces, when sensors are implanted directly into the cerebral cortex.

For example, there are interfaces based on intracortical implants - these are special matrices whose contacts penetrate the thin shell of the cerebral cortex and can read the activity of individual neurons.

With the help of these technologies, patients really manage to control prototype prostheses with a large number of degrees of freedom.

It is also important that when developing such devices, scientists add the ability to receive feedback from the prosthesis, for example, through direct electrical stimulation of the sensory cortex of the brain in response to the movements of the prosthesis.

A person can feel such a prosthesis almost like his own hand.

True, the implementation of such technologies in practice is still far away.  

- That is, if the work is successful, a person with sensors implanted in the brain will be able to live a normal life? 

— Yes, it is quite possible.

True, over time, such implants can be covered with connective tissue, which interferes with signal transmission.

But this process is slow, and such devices may well turn out to be working.

In general, ear cochlear implants are already used, which function quite well.

They act directly on the auditory nerve and can compensate for hearing loss in some patients.

But implantation is not always successful - for example, sometimes a person hears sounds, but cannot form words from them.

This may be due to a violation of the process of functional integration of information by the brain, and in this case, new methods of teaching patients how to use implants are needed.

Significant progress is observed in the field of creating visual implants.

There are already retinal implants that stimulate the retina, and there are those that act directly on the visual primary cortex.

The difficulty here is that it is difficult to detect this zone in an already blind person.

However, with the help of such devices, a person begins to see the basic outlines of objects. 

In addition to such techniques that attempt to restore through interaction with the original visual mechanisms, systems based on sensory substitution are being created, which, for example, use a matrix of electrodes that stimulate nerve endings at the root of the tongue.

There are a lot of them, and a person can learn to “see” with the language.

• Gettyimages.ru

— You mentioned bionic prostheses.

On what principle do they operate?

— There are two types of prostheses.

Mechanical, which are controlled by accompanying movements of other parts of the body, for example, a hand prosthesis performs a grip when the injured arm is bent at the elbow.

Or prostheses controlled by signals from the electrical activity of the muscles.

In this case, the prosthesis deciphers the activity of the residual muscles and executes the necessary discrete commands.

Prostheses with natural control are currently being developed, when the electrical activity of the muscles is converted into smooth movements of the prosthesis.

But in this case, it is necessary to create a system for teaching patients how to use such devices, that is, to ensure the interaction between man and machine.

— Are there companies in the Russian Federation that are engaged in technologies in the field of bioprosthetics and how developed are such technologies?

Do we have our own scientific base for such work? 

— Yes, there are such companies, but they are few.

For example, Motorika, whose specialists develop a complex prosthesis.

She cooperates with the Far Eastern Federal University.

Together they are trying to create a technique for stimulating the nerves in the spine to feel the prosthesis, as well as to relieve patients of phantom pain. 

In Russia, there are companies involved in exoskeletons that can replace the functions of limbs.

The development of such devices is carried out, for example, by the ExoAtlet company.

Moreover, they create exoskeletons not only for adults, but also for children with chronic diseases, including cerebral palsy. 

There is a scientific base in Russia for such developments.

For example, the Center for Biometric Interfaces at the Higher School of Economics is busy creating the foundations for neural interfaces, including motor interfaces.

Our developments are tested by specialists in the clinic based at the FMBA Center for Brain and Neurotechnologies.

We are developing advanced algorithms for decoding muscle and brain activity together with the AIRI Institute of Artificial Intelligence.  

• Gettyimages.ru

I can give other examples.

For example, at Sirius University in St. Petersburg, under the guidance of Pavel Evgenievich Musienko, spinal cord implants are being made to restore spinal pathways disturbed by trauma.

If the spinal cord is seriously damaged, the pathways that conduct nerve impulses from the brain and back are destroyed, the consequences of a spinal injury are catastrophic.

An implant can help repair damaged nerve fibers.

Experiments on laboratory animals have already been crowned with success, they have learned to walk again. 

In addition, in St. Petersburg, under the leadership of Yuri Petrovich Gerasimenko, devices for transcutaneous spinal cord stimulation are being developed that can be used to trigger the locomotor cycle in patients with the corresponding disorders. 

— Can bionic prostheses and exoskeletons be used for non-medical purposes? 

- Yes, exoskeletons are already being created for the military and those who need to lift heavy weights.

Such devices are controlled by signals coming from the muscles. 

In addition, scientists recently attached a sixth mechanical finger to the hand, it was controlled using the movements of the big toe.

With the help of an additional finger, it was possible to take complex chords on the guitar and perform other previously impossible combinations of movements.

The most interesting thing is that when they began to study the activity in the brain of the experimental, it turned out that new connections arose in the areas responsible for the motor skills of the hands.

They survived even after the experience was over.

— Last year, Australian scientists created the DishBrain system in the laboratory, which consists of brain cells, cell culture and electrodes at the same time.

They taught these cells to play a primitive computer game.

Is it possible to use such developments in the field of neural interfaces? 

- In general, the technology of growing cell culture on matrices has been around for about ten years.

In Russia, for example, such studies were carried out at the University of Nizhny Novgorod in the group of Viktor Borisovich Kazantsev and Irina Vasilievna Mukhina.

The scientists took cells from the hippocampus of mice and placed them on special substrates to which electrodes were attached.

By stimulating the cells with electrodes, the researchers taught the neural network to perform the required tasks. 

There is another such technology - memristors.

This is a new type of conductor - an artificial object, similar to a biological synapse, like in our brain.

Such hybrid systems are of great interest for understanding and emulating learning algorithms that occur in living systems.

Perhaps this knowledge can be used to replace the natural nervous tissue.

However, so far these are only laboratory experiments.

You can also later turn these neurons into part of some kind of electronic circuit to create more natural interfaces.

Such work is being carried out in Germany and Russia.

However, we must not forget that such cells still need to be constantly maintained in a living state. 

• Gettyimages.ru

- There is a whole movement of transhumanists who believe that the human mind and body can be improved with the help of technology and in the possibility of "digital immortality" - consciousness can supposedly be transferred to artificial media after a person's death.

How do you rate these ideas? 

All of this has no scientific basis.

But as for the work to improve the functioning of the brain, there is no futurism here.

For example, impaired brain function can be restored by involving this organ in training in the neurofeedback paradigm, when a person is shown a certain aspect of his brain activity that deviates from the norm.

Thus, the patient can learn to control brain activity and normalize its function.

This is how epileptics are already being treated, and in the future such technologies can provide very great opportunities for the treatment of various neurological disorders.