The neurophysiologist discusses the Neuralink project and talks about the work of the brain “on the fingers”

_{Left - neurophysiologist Elena Belova, right - robot surgeon Neuralink}



In the summer of 2019, a presentation of the Neuralink startup was held , the purpose of which is to create a brain-machine interface. Elon Musk said that the company managed to work out for several years from the moment of its foundation. Introduced a robot surgeon, flexible threads for connecting chips to the brain and efficient signal processing algorithms. We met with a neurophysiologist to talk about what Neuralink is: business and marketing, or a real scientific breakthrough?



Elena Belova, a biochemist and bioinformatics by education, neurophysiologist and illustrator by occupation, senior researcher at the Laboratory of Cellular Neurophysiology of the Institute of Chemical Physics, patiently answered questions.



Stupid questions were asked by Ivan Zvyagin, who read several popular science books about brain function and evolution.



_{The audio version of the conversation is in the Habr Special podcast. You can listen to it not only on SoundCloud, but also on other platforms}

About Neuralink and its analogues

What's new in Neuralink threads and processors?

When in Russia people talk about the computer-brain interface, most often we are talking about a hat instead of a joystick and controlling the machine with the “power of thought”. And here everything is serious. Musk is going to expand the possibilities of treatment and rehabilitation, knowledge of how our brain works. It is very cool.





_{Thread A is 16 times thinner than a human hair}



When Musk presented this project, he mentioned things that already existed before Neuralink. Among these methods - deep (deep) stimulation of the brain, in English - Deep Brain Stimulation. This is the method that I do in my laboratory. There are also methods when they open the skull box and put an electrode on the surface of the brain to record potentials and electrical activity from the cerebral cortex. In the first case, we record deep subcortical nuclei, in the second - the cortex. Depending on what and where we write, different tasks can be solved.



They did not propose anything fundamentally new from the point of view of the approach. This is precisely a reduction in scale, an increase in the number of recording streams and acceleration of processing with the ability to analyze on the fly, without transferring data to supercomputers.

What is wrong with EEG caps?

Beanie is a very common available method. An electroencephalogram is one of the very first brain research methods. The problem is that we shoot a very weak and very generalized signal from the surface of the [head], which is very difficult to analyze and interpret. You can compare it with an attempt to understand what is happening, for example, somewhere in the city, taking off light from space. Very, very weak and very large signal. Of course, when a synchronous activity of a large number of neurons occurs somewhere, we will notice this on the electroencephalogram, but if more subtle mechanisms are of interest, then nothing will work out at such a distance and with such a sensitivity of the signal.

What is Deep Brain Stimulation?

DBS are very large and very invasive pieces. To imagine what it is, imagine that you untwisted a ballpoint pen and took a rod from there. The thickness and size of the electrode is approximately like that of a ballpoint pen — about 2 mm in diameter. Such a big thing, which is attached to the lining on the skull bone. Further with this electrode, a person walks deep in the brain for decades, until death. This technique allows him to significantly alleviate the symptoms of the disease with which he comes according to indications for surgery.







Maintenance consists in changing batteries in the device. Since this is electrical stimulation, you need a power source that must be replaced. While there is no enough miniature and powerful enough power supply for 30–40 years to last without replacement.



A wire sticks out of a person’s skull, and then surgeons conduct it under the skin to the sternum, where they put a stimulator. This is considered a safer option than outside the cranium. Firstly, it would not be very beautiful. And secondly, if you suddenly accidentally hit, then the sternum area is one of the most protected parts of the body.



This technology is almost 30 years old, and with such devices dozens, if not hundreds of thousands of people around the world go. She has proven herself, and now this is one of the ways to help people whom nothing else helps - neither medication, nor invasive.



The electrodes have a coating, insulation, and from 4 to 12 contacts, depending on the brand of a particular device and purpose. They differ in the length of the active region that can be stimulated. Contacts allow 30 years to get a good quality of stimulation, but there is no talk about recording. That is, there does not need a high resistance, which will allow you to send an electric signal to a small area. It is just a stimulation.

Why does the signal from the brain to the electrodes deteriorate over time?

There are many techniques that allow you to sensitively record a signal. The question is the size of the device that allows it to be recorded. Neuralink was able to make miniature electrodes and claim that for all its miniature size, the signal-to-noise ratio is enough for them to get a good record. A separate question will be how long they will be able to receive this recording without deterioration of the signal quality, because the brain is biochemically active and can affect the resistance of the sensor, the smaller the sensor, the greater the chance that something will go wrong.

Is Neuralink marketing and business, or is it true that science is moving forward?

As a scientist, I see here a gigantic resource for fundamental science.



There are two large areas in neurobilogy. In the first we study animals. But they are simpler, so many experiments are very difficult to carry out. Let’s say a monkey cannot articulate and pronounce words, and we don’t know how it arranges motor areas that are associated with the tongue and vocal cords during speech.





_{USB Nueralink Lab Rat}



When we talk about obtaining data about a person, we are faced with a huge amount of subjective experience and very stringent ethical requirements and very weak technologies that have been tested and approved for use in humans.



To learn something about the human brain, you can use the records of its electrical activity, which are done during almost all medical operations. Most often we are talking about motor disorders and deep brain stimulation.



Now they [Neuralink] say that the performance of their chips and the amount of data received are significantly increased. This is a very cool groundwork in order to get something really interesting.

How can Neuralink not damage the moving brain by introducing a bunch of threads?

_{For the impressionable: the picture is not the brain, but the jelly that simulates it. Real robot and threads}



The main problem with operations on the brain is that it is penetrated by blood vessels. It’s very bad to damage vessels, a brain hemorrhage is a very unpleasant thing that can lead to a micro stroke or death. Avoiding blood vessels is the first most important task that speaks about the safety of the entire procedure. Although she does not say anything about effectiveness.



Next, the question arises of fixing these electrodes. If we talk about deep stimulation, then the electrode is fixed directly on the skull - there is a drilled hole, a patch is screwed into it in a special way and the pad in which this electrode is absolutely rigidly fixed.



With flexible threads is not very clear. In an adult, brain mobility is limited. In young children, the brain can vary significantly in diameter. I suspect that near large vessels, the brain pulsates more than far away from them. Nevertheless, you can try to fix the electrode so that it is plus or minus in the same area. The good question is, how far can the electrodes that Mask offers be located near the neurons from which recording is being done?



When we [use the DBS method] write electrical activity inside the human brain, it happens that a person moves his hand or sneezes, and the tip of the electrode shifts just a little bit, and that’s all - we just “saw” the neurons, and now we stopped. Or, on the contrary, there is no neuron near the electrode, but then a person does something, and the neuron appears, and we see significant activity: the potentials of action on the record, everything is fine.



How much these processes can affect localization, fixing the flexible electrode at a certain point is a good question. Judging by what was at the presentation, they often write the same neuron from two or three contacts. This configuration allows, in which case, to stop writing this neuron from one of the contacts, but two more will remain. Density allows you to duplicate information.



When it comes to large electrodes, then, indeed, there is some damage to the brain near the electrode and an increase in connective tissue. As for these small, thinner hairs, we do not yet know how the brain reacts to this. But the smaller the device, the less changes will be directly related to the fact that it is somewhere near nerve tissues. There is some chance that the consequences of implanting such thin threads will be negligible compared to the opportunities that they can give.

Will Neuralink help interact with virtual objects on the screen directly, and not with the mouse cursor?

This option is not excluded. For this, a group of neurons must be isolated within the nervous system, which will specializedly control the movement of this object.

Say, when a person learns to handle tools, at a certain stage in the development of a skill they become, as it were, a continuation of the person’s hands. Specialized neurons appear in the motor cortex that control the fine motor skills necessary to work with such an instrument. If we assume that the Neuralink chip will allow you to read the signal from such neurons directly, then the hand movements are no longer needed, we can probably compare certain patterns of neuron activity and the movement of the instrument and control the instrument already using the chip.



When we move, motor-sensory feedback is very important to us. Inside the muscles there are a huge number of sensory inputs that indicate how much muscle is stretched now, how many muscular systems are tense, at what speed and what parts of our muscles are straining. When it comes to a virtual object, we can evaluate its speed only with our eyes. It is useful, but it is not the same as touch.

What diseases does DBS stop and can Neuralink potentially

It seems that the first disease that can be dealt with by electrical stimulation of the brain and approved by the FDA is Parkinson's disease. In people with this disease, dopamine neurons die. They hardly begin to move, they move very slowly and hard, they start tremor.



There is drug therapy, and when at some point it stops helping (and it stops), such an operation is indicated. In fact, quite simply, we are interfering in a place that makes life difficult for people. Without stimulation, the signal from this area simply does not enter, but it does so with interference, and it becomes easier for a person.



There are several more motor disorders that stop the stimulation of the same structures and also help. This is primarily dystonia, when people have the opposite pathological movements: contractions of certain muscles, twitching. If in this case the problem is not with the nerves, but with the brain, then such an operation can be performed.



The possibility of helping people with depression, obsessive-compulsive disorder, and those that are closer to the emotional sphere is being actively studied. But for now, these are theoretical studies, test protocols, that is, not a routine clinical procedure, but searches for methods of exposure and verification of activities.

Could Neuralink help Stephen Hawking?

When it comes to problems with the motor system, there are two different sources of problems: the nervous system and muscles. When the problem begins with the muscles, then gradually, most often due to the immune response, the muscles begin to degrade. Muscle tissue contracts, and it is impossible to grow further. This is not a problem of the nervous system, it is a muscle problem, that is, you need to grow another muscle from other genetic substrates. And this is a question for genetic engineering, and not for Neuralink. I hope that someday this problem can also be solved, but not now.

About neuro prostheses

How bionic prostheses work

All brain-machine interface systems that allow paralyzed people to move something are structured like this: there are machines that, in the process, learn to decode what a person wants. And a person learns to send signals from the area in which recording is as clear as possible. Neurons near these contacts can be rearranged depending on what kind of task the brain solves. If a bionic arm needs to be extended and a cup is taken, the brain begins to rebuild, even if initially these neurons were responsible for something else. This mutual adjustment, which allows a person to move a bionic hand, is somewhat similar to how he moved his own before it.

Are there neural prostheses with feedback?

In addition to Ilona Mask, a huge number of specialists are involved in the problems of people who have lost the ability to move. Bioprostheses, neuro prostheses are rapidly developing areas. Technologies allow us to do more and more new and more complex things, including testing of bionic hands with the ability to give feedback. That is, a person can use a bionic hand to touch the surface or grab an object, and the sensors from it give information back. But there the situation is such that healthy nerves remain, and sensors of the bionic arm are actually attached to them, and then information is sent through the nerves to the brain - through those channels that already exist, are predisposed, pre-trained and formed.





As far as I understood, Neuralink wants to place their devices in two places: in the motor and in the sensory cortex. And send a signal directly to the bark. In order for this to work, you need to select everything very well and test it very well. We now have a poor idea of ​​how and on which neuron to give stimulation, what pattern, how these electrical impulses should look so that the connection that the brain receives corresponds to what is happening in the real world. And how much the neuron is able to learn that this pattern is specified externally, rather than being generated from within the neural network.

Can a person feel the prosthesis as a real part of himself?

Yes, he [over time] begins to feel the prosthesis. I do not have such experience and I hope that I will never get it, but it seems to me that it is like a very, very numb hand, in which the sensitivity begins to gradually return: it is still so weak, still bad, but already something there is. This is much better than when the signal does not go at all and the person cannot see from what he sees: the bionic hand is now squeezing the bottle so that it falls out, or so that it crackes and breaks.

How long does it take to feel feedback from a neuro prosthesis?

It is very context sensitive. When it comes to a person who has just had his arm amputated (because of the danger of blood poisoning, for example), then if you immediately “lively” attach sensors to these nerves and immediately give feedback - this is the best option. These nerves are actually still built into the network, and how long they will have to be retrained will depend on how well a person is trained in principle. It seems to me that we are talking about a time period of the order of two or three months in order to learn how to use this prosthesis at least in some form, and in the region of the year - in order to use it already without significant stress. Perhaps I am mistaken, there are very few such studies so far and these are isolated cases in which we cannot tell any statistics.

Can the brain cope with deep body upgrades, like in Cyberpunk 2077? With a third hand, for example, or with the eye.

_{Shot from the game Cyberpunk 2077}



It will depend on how old a person is to attach this third hand. When the networks are mutually tuned, there are some critical windows when neurons find connections with other structures in other nuclei in order to efficiently carry out some goals, tasks, movements, and feedback flows constantly there. If something went wrong during this critical period, a person often simply loses the opportunity, for example, to control his hand if he is deprived of his hand. Or perceive and analyze visual information if he has problems with the retina or eyes. Turning everything back and returning a person’s vision, if he had no experience in perceiving and processing visual information in childhood, is almost impossible, as far as we know now. Maybe something will change in 2077, but now this is the case.



If you directly fantasize, then if you want four more bionic hands besides your two hands, then you need to do this very, very early - in infancy. But here many ethical problems arise, because the child is small, no one asks him, and it is not very clear what such justification should be in order to decide on such influences with not very clear consequences in the future.

About the brain

Is it possible to influence the emotional sphere with electricity?

Dystonia and Parkinson's disease are a violation of the activity of very deep brain structures that are responsible for the initiation of movement, the choice between several different motor programs. There are very close to the emotional areas that are associated with motivation: to choose and start a particular movement. In fact, having slightly shifted the position of the electrode, we can already influence the emotional sphere.



There are indications that some people with such an electrode begin to change in the sphere associated with emotions and habits. In general, Parkinson's disease can lead to a person becoming a pathologically gambler who goes to the casino, or he has episodes of pathological jealousy, outbursts of anger and mood changes. Sometimes this happens before treatment and after it is removed. It happens the other way around: everything was in order in a person, and then they put an electrode on him and began to influence the sphere, which is associated with impulsivity, compulsiveness, and reactions.



I said that in the big core of DBS there are several different contacts and several different stimulation programs. We can change the frequency of exposure and which specific contacts will be active. You can change contacts, slightly reduce the amplitude or frequency of stimulation - increase or decrease. But this is all shamanism, when by trial and error we try to choose what will help a person to move well and at the same time will not complicate his life.

Is there an exact map of the brain?

There are different accuracy. Inside the brain there are different structures: nuclei that have boundaries, and between them is white matter, which connects, in fact, one structure to another. This is an accumulation of nerve cells, and an error of 1 mm can significantly affect what effects we get from stimulation.



When we talk about the cortex, things are a little more complicated. There are sensorimotor areas in which there is a topography; there are areas that are associated with the movement or sensation of hands, touch, pain signals; have legs; have a face. And all this is marked along the cortex - both sensory and motor - and reconfiguration of these areas is possible. For those people who, for some reason, lose their hand, the area that was responsible for managing this hand can be rebuilt and begin to interact with some other areas, for example, with a part of the face.



If we are talking about the cortex, we first need to look at which specific zone and with what specific functions are those neurons that we can record. And then look: can we train them to respond as we need. That is, you can try to retrain these neurons.

Why do you need to be conscious during brain surgery?

When we insert an electrode into a person’s brain, we must make sure that he receives the effect he needs and does not receive any side effects. If you wake up a person and ask what he thinks, what he feels, and whether he has any side effects from stimulation, this allows you to choose a better location of the electrode, and it is simply better to help the person. Therefore, many brain operations are performed in the mind, in order not to make things worse. We immediately check the test stimulation in the process. At this moment, scientists have the opportunity to collect an invaluable set of information about how the human brain works and how it works. This is a mutually beneficial thing that helps both the patient and the scientist.



There are no pain receptors inside the brain - the main thing is that the patient has analgesia on the surface of the bone and skin. And inside, the maximum that happens is stimulation, and a person ceases to speak normally, “porridge” appears in his mouth. But we do not want a person with an electrode to have “porridge” in his mouth, and we are trying to rearrange the electrode to where there is no porridge.

Do scientists understand at least something about abstract and conceptual images?

Most often, biomedical research related to humans is aimed at helping those people in whom something happened. And since there is no biomedical task for studying abstractions, and this is rather just curiosity, ethical issues arise in such studies. Therefore, with abstractions, everything is very complicated. In comparison with the motor or visual systems, we do not understand them well: what is possible and what is impossible? And if we climb there, what consequences can this intervention cause?



There is a story about "Jennifer Aniston neurons" and "Holly Berry neurons." Sometimes it is possible to find such interesting neurons that respond to a specific person or to a specific image. This was discovered when patients with epilepsy were checked where the foci were. And very often they are located in the temporal lobe, which is associated with speech and perception of images. It seems like there was still a Star Wars neuron that reacted not to the specific image of Darth Vader, but to all the things that are related to Star Wars. If we imagine that we can stimulate conceptual neurons, I assume that we can also cause people to think about Jennifer Aniston, Holly Berry, or Star Wars. But we can say the same name in the same way, and in the same way this neuron will be excited, and it will be the same. Why to climb a person in the brain for this is not very clear.

Is it possible to generate sensations in the body? Let's say scratch your heel.

Depends on what's with that heel. If an insect crawls on it, then you can try to hammer this sensory channel, act on it and “say”: [there is no insect], now they just click hard on this place. But as soon as you remove this signal, most likely direct irritation on the heel will give a signal again, and the person will say: no, itchy again.

Is it possible not to spell words, but to immediately “think” them?

The question is partly philosophical. There are no studies that show the technical feasibility of doing this at the current level. Although there are probably people who are trying to do something in this area. There is an option, for example, to take signals from nerves that go to the vocal cords, to the tongue, to the lips, and see if we can understand something about what a person wants to say based on motor information from an articulation apparatus.



There is the Broca zone , there is the Wernicke zone , there are paths between them, and this whole system is very complicated for our current understanding.Unfortunately, this is the same story when we know how to break it, but not how to fix it. Breaking is easier.

How do people master the language?

My ideas about language and how it all works in the brain were formed on the basis of Stephen Pinker's book Language as Instinct. This is a rather old book, it was translated into Russian in 2009, and the original was published in 1994. Pinke promotes the idea that mastering the language is an instinct, that is, inside the brain there is a substrate on which the language is formed. Moreover, we know that if a person has lived outside his tongue until some critical age, he cannot master it. Mowgli, who lived somewhere in India with wolves until 10-12 years old, was simply not able to master the language as a communication technique after that. There are critical windows, and in order to master the language very well, you need to be in the context of speech and communication for up to three years. After three, it’s already getting worse, but the person is still able to master simple designs,although most likely he will not "breathe his tongue."



When a deadline sets in to master a language, it is not very clear. Individual characteristics can affect, for some, these windows close a little earlier, for others later. As far as I understand, in the region of 8–9 years the train leaves.

Is it possible to insert a flash drive into the brain and access it?

Memory is especially difficult. Given how much power has already been buried there and how much we know, it’s very difficult to bring everything into some kind of understandable concept that can be told.



Memory in the brain is a dynamic system in which the processes of memorization and forgetting are in some equilibrium and at the same time they are on different scales.

• You remember that the number dictated to you: you read it to yourself and type - this is temporary memory, you can remember 7 (plus or minus two) characters.
• , , , , .
• , . , , — .
• , . , — .

Memory is a very complex concept, especially if we are talking about the brain, behavior and how the brain interacts with the environment in all contexts that exist. But even when it comes to episodic memory and concepts, we very poorly understand how everything is implemented in the brain.



We have a hippocampus, and within this hippocampus there is a consolidation of long-term memory. When it comes to conditioned reflexes, we can only approximately describe the process of events. We cannot say anything about how the Jennifer Aniston neuron is formed. The fact that leads to the formation of the neuron Jennifer Aniston, but the neuron Leah Akhedzhakova - no, we also can not.



Fundamentally, in the brain there is no separation between the information carrier and the processor that processes this information. Each time a person remembers something, he changes a little the network within which this information is stored. At the same time, memory very much depends on the depth of information processing, on how much you have become accustomed to it. How to correlate all this with a flash drive is completely unclear.

About science

Is it interesting for a Russian scientist to work at Neuralink?

It would certainly be very interesting for me to work in such a project. But I don’t really understand yet how everything will be arranged for them in terms of organization and correlation of technical issues with fundamental ones. Because I love fundamental science, I really like the questions of “how” and “why”: “how it works” and “why it works like that”, and not “how to do something” and “why it doesn't work”. This is a technical project in which there are very ambitious in complexity problems and tasks that cannot be solved without understanding how all this [brain work] is fundamentally arranged at a basic level.



For example, how does the encoding of the same movements occur in the brain? When we talk about the motor cortex, this is what is called the higher motor neurons, those that are responsible for the concept of movements. Inside the brain there is still a certain number of motor zones that are responsible for ensuring that the signal reaches the specific muscle along the nerve. They are located in the cerebral cortex with a transition to the spinal cord. There, a signal is already specifically sent to a specific muscle: in order to bend a finger at 45º in such and such a phalanx.



In the cortex itself, a concept is implemented. And how does this concept look like, how do these many conflicting representations and motor programs relate? All this machinery is known to us very much in theory. How this is all implemented, and what are the possible solutions to so many technical problems in the brain, is a very interesting question.

Can Russian science offer Neuralink something?

We have not so much developed neurobiology, there are not many neurobiological laboratories. And the big problem is that the equipment of almost any laboratory loses to the equipment of an American or European laboratory. Our material and technical base is often substantially weaker.



I would say that Europe and Russia are strong in concepts, we have many who are involved in theory. America is strong in applied things, they can translate ideas and concepts into a working product that can be sold, capitalized, brought to the market, exchanges, and that’s all. They are financial geniuses.



At the same time, it seems to me that the flow of scientists in the USA is being recruited, most often from India, China, and Europe. They, of course, have a very young nation, scientists in the 4th-5th generation of Americans, I don’t know much. Almost all Americans who come are [ethnic] Chinese, Koreans, Indians, Irish.



As for us, Russia, we have an advantage. There are not such serious ethical committees. To conduct a study in Europe or the USA, it must be coordinated in 10 thousand instances at the initial stage. And with us it’s much easier to organize. We have far fewer institutions that deal with the observance of rights in every last comma and squiggle - that of animals, that of people. I don’t know how good or bad it is, but, nevertheless, for science this is some bonus. Because what is difficult and very expensive to carry out in Europe from the point of view of experimental research, it is much easier to organize in Russia.



The second point is theoretical work: concepts, checking ideas, working with open databases and checking ideas in the format: “We had such a great idea, we looked at what we did in the world and based on the data that we didn’t collect, found such an interesting thing. ”

How do animal experiments usually go to humans?

To conduct experiments even on animals, you need the approval of the commission. Almost every academic institution has an ethics committee. This applies even to mice. I don’t know about Drosophila, but it seems to me that with flies and small worms Caenorhabditis elegans, commissions are still not needed to conduct experiments on them. But if you want to insert an electrode into the brain of a mouse or rat and see how it will feel, you need the approval of the commission.



You must prove that you are ready to conduct these studies and you need them. That is, there are no other ways to get the information you need. You check and collect data on how safe it is, whether it causes any kind of inflammation or something else. It’s clear that you can’t insure yourself 100%, but you should have some reasons: what are you going to do, what can go wrong. The Commission reviews the arguments and decides that the study in this form can be approved.



The commission includes physicians - exactly the same fellow scientists who are guided by certain principles: whether the animal will suffer, how much this suffering is necessary, whether it can be reduced. Any operation on an animal is generally not a very pleasant experience for him. We cannot ask, but we guess.



When it comes to people, things get more complicated. The commission is no longer only internal, especially when it comes to innovative research. The same FDA is committed to considering technology and drugs: everything about people and their health, and evaluating risks. Is it possible to specifically conduct this study in the form in which it is? Is there enough animal information collected?



Further, depending on what is being tested, sometimes it all starts with healthy volunteers who are tested in a limited concentration of drugs. Sometimes when it comes to serious illnesses and a healthy person can’t verify anything, people with an injury are taken to the group. For example, with a fracture of the spine, which led to paralysis of the arms and legs. Or cancer patients - chemotherapy is often tested in humans, because this is their only chance of survival. In this case, it is much easier to get approval and permission, even with some doubts that it will be completely safe.

What to read about science and the brain if you are not a scientist, but are interested in?

It seems to me that acquaintance with the topic begins with Evgenia Timonova and “Everything is like animals”. Then people go through the funnel to the series of books “Elements”. And if you already understand everything on the “Elements” and would like to continue, you can go to the Biomolecule website - this is a popular science site for those who want to understand all the details. There are articles about neuro prosthetics and in the fall there will be an article about Neuralink - about how everything is arranged there and what is needed for it. In addition, there are many materials on molecular biology, genetics, biochemistry, neurobiology and medical applied things. For example, about the future of cochlear implants.