The surgeon is going to connect you to the internet through a brain implant

Erik Leuthardt believes that in the near future we will allow doctors to push electrodes into our brains so that we can communicate directly with computers and with each other

On Monday morning, following the premiere weekend of The Blade Runner 2049, Eric Leuthardt stands in the center of the light-flooded operating room, wearing a robe and mask, bending over an unconscious patient.



“I thought he was human, but I wasn’t sure,” says Lüthhardt to the resident surgeon standing next to him, while drawing a line on the patient’s shaved scalp where he is going to make the first incisions as part of a neurosurgical operation. “Did you think he was a replicant?”



“I definitely thought he was a replicant,” the resident answers, referring to the term from the film, meaning realistic androids.



“It seems to me very interesting that in the future flying cars are constantly shown,” says Lüthhardt, handing a marker to the ordinator and taking a scalpel in his hand. “They showed dystopia: all the time they were talking about biology, about replicants. But the big chunks of the future are missing. Where, for example, are neuroprostheses? ”



On this subject, Lyuthardt, a 44-year-old scientist and neurosurgeon, thought a lot. In addition to his work as a neurosurgeon at Washington University in St. Louis, he also published two novels and wrote an award-winning play, designed to “prepare society for the upcoming changes.” In the first novel, the techno-thriller "Red Devil 4" [RedDevil 4], 90% of the population chose to implant computer equipment directly into the brain. This made it possible to ensure communication between people and computers and a huge amount of sensation became available, right from the comfort of home. Leuthardt believes that over the next few decades such implants will become something like plastic surgery or tattoos, and they will be installed without hesitation.





Eric Lewhardt



“My job is to open people,” he says. “So it's pretty easy to imagine.”



But Lyuthardt did much more than just imagine the future. He specializes in operations for patients with intractable epilepsy, and they all have to spend several days before the main operation with electrodes implanted in their cortex, while computers collect information about the sequences of neurons that are activated in them before the seizures. At this time, they are tied to a hospital bed, and they are often very bored. About 15 years ago, Lüthardt had an epiphany: why not lure them to be tested in an experiment? This will dispel their boredom and help bring his dreams closer to reality.



Lyuthardt began to develop assignments for them. He then analyzed their brain signals to understand what he could learn about brain decoding of thoughts and intentions, and how such signals could be used to control external devices. Is there enough data to which it has access to describe the planned movement? Can I hear the inner monologue of a person? Is it possible to decipher the mind?



Although the answers to some of these questions were far from straightforward, they were inspiring. Inspiring enough to make Lyuthardt a true believer — those whose ideas might sound insane if he were not a neurosurgeon, working in an environment of life and death in the operating room, when there is no room for illusion or arrogance. Lüthardt knows better than anyone that neurosurgery is a dangerous, frightening, and difficult thing for the patient. But his understanding of the brain gave him a clear idea of ​​his inherent limitations — and the potential of technology to help overcome them. He insists that when the rest of the world understands all possibilities, and technology makes the necessary progress, the human race will do what it always did. Evolving. This time - with the help of chips implanted in the head.





The patient is prepared for minimally invasive laser surgery to treat a brain tumor. Such high precision operations have made implantation of electrodes safer and less intimidating.



“Real neurointegration is bound to happen,” says Lüthardt. It is the matter of time. It will take 10 or 100 years, in the long run it will be a significant development of humanity. ”



Leuthardt is not the only person who has exotic ambitions for creating brain-computer interfaces. Last March, Ilon Musk, the founder of Tesla and SpaceX, launched the Neuralink project, designed to create devices that facilitate the integration of the brain and machines. Mark Zuckerberg from Facebook expressed the same dreams, and this spring his company unveiled the fact that 60 of its engineers are working on creating interfaces that allow you to type with the power of thought. Brian Johnson, the founder of the Braintree payment system, uses his funds to support Kernel, a company developing neuroprostheses that he believes will someday improve intelligence, memory, and more.



But all these plans are in the early stages, and they have not been widely disseminated, therefore it is difficult to determine what progress has already been made, or whether the stated goals are realistic. Problems with interfaces brain-computer wagon. Those devices, which Mask and Zuckerberg are talking about, will require not only much better-quality equipment capable of maintaining continuous mechanical connection and data transfer between silicon computers and random gray matter of the human brain. They will need to have enough computing power to parse the array of data produced at any time by any number of the hundreds of billions of brain neurons. And besides, we still do not know the code used by the brain. In other words, we will have to learn to read people's minds.



But Lyuthardt believes that he will see it even during his lifetime. “Knowing the speed with which technology is changing, it is quite possible to imagine that for 20 years everything that is contained in a modern cell phone will fit in a grain of rice,” he says. “And this can already be put in your head in a minimally invasive way, and it can perform the necessary calculations in order to be a truly effective brain-computer interface.”

Decipher the brain

Scientists have long known that the activation of neurons allows us to move, feel and think. But to crack the code on which neurons communicate with each other and the rest of the body - to listen to this communication, to understand how brain cells allow us to function - has long been one of the most difficult tasks of neurobiology.







In the early 1980s, engineer Apostolos Georgopoulos [Apostolos Georgopoulos] from the Institute. Johns Hopkins, paved the way for today's brain-computer interface revolution. Georgopoulos identified the neurons in the high-level areas of the motor cortex that were activated before each movement, be it with a stroke of the brush to the right or pressure from top to bottom. This discovery was important because these signals could be recorded and used to predict the direction and strength of the movements. Some of these neuron activation schemes controlled the behavior of multiple low-level neurons, working together on the movement of individual muscles, and ultimately the limbs.



Using arrays of dozens of electrodes that track high-level signals, Georgopoulos showed that he can predict not only which way the monkey will move the joystick in three-dimensional space, but also the speed of movement and its change over time.



Such data, it seemed, could be used to enable the paralyzed patient to control the prosthesis with the power of thought. One of the protégés of Georgopoulos, Andrew Schwartz, in the 1990s took up this task. By the end of the 1990s, Schwartz, now a neuroscientist at the University of Pittsburgh, implanted electrodes into the brains of monkeys and began demonstrating that they could actually be trained to control robotic limbs with the power of thought.



Lyuthardt, who is preparing to apply for a job at the University of Washington in 1999, was inspired by this work: when he needed to decide how to spend a year-long research break, he understood what he wanted to do. Schwartz's first successes convinced Lüthhardt that science fiction was about to become a reality. Scientists finally made the first seductive steps to unite the person with the machine. Lyuthardt wanted to be part of the upcoming revolution.



He thought to devote his year to studying the problem of scarring in mice: over time, the electrodes implanted by Schwartz and others caused an inflammatory reaction or were covered with brain cells and stopped working. But when Luthardt and her curator set about developing a plan, they had a better idea. Why not explore the use of some other technology to record brain activity?



“It suddenly dawned on us:“ Hey, we always have people with electrodes! ”, Says Lüthardt. “Why don't we experiment on them?”





The surgeon is preparing to drill a hole in the patient's skull to place the laser probe.





The stereotactic frame, fixed on the skull, directs the laser probe, which points to the desired point of the brain.



Georgopoulos and Schwartz collected data based on a technique that uses microelectrodes located near the membranes of individual neurons, and tracking voltage changes. The electrodes used by Lüthardt, implanted in patients with epilepsy before surgery, were much larger and located on the surface of the cortex, under the scalp, on pieces of plastic, recording signals emitted by hundreds of thousands of neurons at the same time. To install them, Luthardt performed a primary operation in which the upper part of the skull was removed, a cut of the dura mater (the outermost membrane of the brain) was made, and the electrodes were located directly on the top of the brain. Then he connected them to a bundle of wires that went from the patient's head, and connected the wires to devices that analyze brain signals.



Such electrodes have been successfully used for decades to determine the exact source of epileptic seizures in a patient’s brain. After the first surgery, the patient stops taking medication for seizures, which ultimately causes a seizure - and the data on his physical source helps doctors, Lüthardt and others, decide which part of the brain needs to be resected to prevent seizures in the future.



But many people were skeptical that the electrodes could provide enough information to control the prosthesis. To find out, Lyuthardt brought in Gervin Schalk, an informatics specialist from the Wadsworth Center, a health lab at the New York Department of Health. Progress has been quick. After a few years of testing, Lüthardt patients demonstrated the ability to play Space Invaders — by moving a virtual spacecraft left and right — by the power of thought. Then they learned to move the cursor in three-dimensional space on the screen.



In 2006, following a report on his work at the conference, Elmar Schmeisser, head of the Department of the US Army Research, contacted Schalk. Schmeiser conceived something more complicated. He wanted to know whether it is possible to decode "mental speech" - words that a person does not voice, but speaks in his head. Schmeiser was fond of science fiction and had long dreamed of creating a “mental machine” capable of recognizing the soldier’s mental speech and transmitting it to his comrade’s earpiece.





Laser probe



Lüthhardt arranged for 12 bed-ridden patients with epilepsy who were in the rooms and bored waiting for seizures, and gave each of them 36 words with a simple “consonant-vowel-consonant” structure, such as “bet,” “bat , ““ Beat ”and“ boot ". He asked patients to first speak the words out loud, and then just imagine how they pronounce them, following the visual instructions on the computer screen without audio, and then again say the word out loud, without video tips, so that could identify incoming sensory signals. He sent Shalka for analysis.



The Schalka program works with sequence recognition algorithms — its system can be trained to recognize the activation schemes of groups of neutrons associated with a particular task or thought. In the presence of 50-200 electrodes, each of which produces 1000 values ​​per second, the program has to grind a huge number of variables. The more electrodes and the fewer neurons that fall on each electrode, the higher the chances of decoding meaningful sequences — if you draw in sufficiently large computational powers that can cope with the rejection of unnecessary noise.



“The higher the resolution, the better, but at a minimum we should have about 50,000 numbers per second,” says Schalk. - It is necessary to extract from them one thing that interests you. This is not a trivial task. ”



The results of Shalka were surprisingly reliable. As might be expected when patients voiced the word, the data spoke of activity in the areas of the motor cortex associated with muscles that reproduce speech. The sound cortex and its neighboring region, which had long been suspected in connection with speech processing, were also active at that moment. What is interesting is that when the subjects represented words without voicing them, the neuron activation schemes were very similar, although they were slightly different.



Schalk, Lyuthardt and other people involved in the project believe that they have discovered this quiet voice that we hear in our head when we imagine a speech. The system has never been perfect: for several years of work and improvements in algorithms, the Schalka program is able to guess words correctly in 45% of cases. But instead of trying to improve this result (quality should increase with improved sensors), Schalk and Lyuthardt concentrated on deciphering more and more complex components of speech.



In recent years, Schalk continues to improve the processing of real and mental speech (he can discern what the subject is mentally saying - Martin Luther King’s speech " I have a dream ", or Abraham Lincoln's Gettysburg speech . At that time, Lyuthardt is trying to go into another area: determine how the brain encodes intelligent concepts with different sections.



Data from these studies have not yet been published, but “to be honest, we are still trying to figure them out,” says Lüthardt. He admits that his laboratory may have approached the limit of what is possible at the current level of technology development.

Implanting the future

“As soon as we got evidence that we could decode intentions,” says Lüthardt, “I knew that things were going.”



Shortly after receiving the results, Lüthardt took a few days off to write articles, to imagine the future and to think about short-term and long-term goals. On the first lines of the to-do list, he decided to place the preparation of humanity for the approaching future - and this work is still very far from complete.





Lyuthardt drills a hole in the skull





Real-time laser movement is tracked on this screen.



Leaning back on a chair in his office after the operation, Lüthhardt argues that with proper funding, he could already create an implant-prosthesis for free sale that would allow a person to use a computer and control the cursor in three-dimensional space. Users could also do things like turn the lights on and off or adjust the temperature with a thought. They might even have experienced artificially induced tactile sensations and access to primitive ways of turning mental speech into sound. “With current technology, I could have an implant — but how many people need it?” He says. “I think it’s very important to take practical, small steps so that people take this path to a long-term goal.”



To this end, Leuthardt founded the company NeuroLutions, which must demonstrate that already today there is a market for rudimentary devices that connect the brain to the machine, and are at the beginning of the development of technologies to help people. NeuroLutions has already earned several million in investments, and a non-invasive brain-machine interface is already testing stroke victims who have lost control of one side of the body.



This device consists of electrodes that monitor the brain, located on the scalp and connected to an orthopedic apparatus. It is able to determine the characteristic neural activity associated with the intention to make a movement, before the signal reaches the motor area of ​​the brain.Neural signals come back from the side of the brain. It’s a little bit more than a little bit of a limb. And, importantly, this device can be used without interfering with the brain.



And although this technology is rather modest in comparison with Lyuthardt's Napoleonic plans for the future, he believes that this area is already capable of changing the lives of people. Each year, 700,000 new patients with stroke appear in the US, and hand paralysis is the most common injury. If you can find a way to help them restore body function - and demonstrate that this method is more efficient and fast - then this will demonstrate not only the power of the brain-machine interfaces, but also satisfy the huge demand that exists in medicine.





Lüthardt plans laser probe trajectory with Stereotactic System





Lüthardt Surgical Instruments



The use of non-invasive electrodes located on the scalp is not so confusing to patients, but it imposes serious limitations. The electrical signals of the nerve cells of the brain are strongly drowned out by the scalp, and also scattered as they pass through the bone. Because of this, they are harder to recognize and identify their source.



Leuthardt can do much more with implanted electrodes directly on the cortex. But on a very painful experience, he learned that it is harder to justify surgery, not only for patients, but also for investors.



When he and Schalk set up NeuroLutions in 2008, they hoped to return to paralyzed people the opportunity to move, bringing the interface to the market. But investors were not interested. Startups based by neuroscientists have been trying to create a brain-computer interface for a dozen years, but they have done little to turn technology into a working version of helping paralyzed patients. The number of potential patients is limited - compared to other problems that medical start-ups, developing various devices and competing for investments, are trying to solve. (In the US, 40,000 people suffer from paralysis of all limbs). And most of the tasks performed by such an interface can already be solved with non-invasive devices. Even the most difficult patients can blink their eyes or twitch with a finger.Methods based on these residual movements can be used to enter data or move a wheelchair without additional danger, recovery time, or funds needed to implant electrodes directly into the cerebral cortex.



So after they failed to get investment with the first project, Luthardt and Schalk shifted their attention to a more modest goal. They found that many patients continue to restore additional functions even after the removal of the orthopedic apparatus - this may be, for example, the restoration of fine finger motor skills. Often they require a very small incentive. After the new nerve pathways are strengthened, the brain continues to redo and expand them so that they transmit more complex motor commands to the hand.



The first success Lyuthardt expects from such patients is to inspire people to move to a more reliable invasive system. “In a couple of years, you can say:“ You know, a non-invasive version gives you these possibilities, but now, thanks to science and other knowledge, we can give you much more opportunities, ”he says. “We can further improve your functioning.”



Lyuthardt wants the world to share his passion for the potential transformative effects of this technology that he tried to inspire people through art. In addition to writing novels and a play, he also works on a podcast and videos on YouTube with his colleague, in which they discuss technology and philosophy for coffee with donuts.



In Lüthardt’s first book, The Red Devil 4, one character uses a cortical prosthesis to get the feeling of walking in the Himalayas while remaining on his couch. Another police detective, telepathically discusses with a colleague the interrogation of a suspect in the murder, while being next to him. Each character has instant access to all knowledge of world libraries - they can get information as quickly as a person generates any spontaneous thought. No one is forced to be alone, and our bodies no longer limit us. On the other hand, each person’s brain is exposed to computer viruses that can turn people into psychopaths.



Lyuthardt admits that now we still do not have the ability to record and stimulate as many neurons as necessary to recreate such things. But he claims that his conversations with investors from Silicon Valley only fuel his optimistic belief that we are on the verge of a blast of innovation.



Schalk is not so optimistic. He is skeptical that Facebook, Musk and others make some valuable contribution to finding the best interface.



“They will do nothing different from what the scientific community itself can do,” says Schalk. “Perhaps something will come of it, but not that they have something new that no one else would have.”



Schalk says that it is “quite obvious” that in the next 5-10 years, some forms of the brain-computer interface will be used to rehabilitate stroke victims, spinal cord injuries, chronic pain and other ailments. But he compares today's recording techniques with IBM computers from the 1960s, calling them “archaic.” In order for technology to achieve real long-term potential, he believes, a new type of brain scan is needed - something that can read a much larger number of neurons at a time.



“In fact, it is necessary to learn how to listen to the brain and talk with it so that it cannot distinguish it from its internal communications, and this we do not know now,” says Schalk. - At the moment we do not know how to do it. But it is also obvious to me that this will happen. And when this happens, our lives will change, and it is absolutely unprecedented. ”



When and where this breakthrough happens is unclear. After decades of research and progress, we still have the same technological problems. And yet, progress in neuroscience and computer equipment and programs makes the result inevitable.





A non-invasive brain-computer interface via an ECG uses electrode sets to help stroke victims recover limb performance.



At least, says Lyuthardt, the hype in Silicon Valley "has aroused great interest and made me really think about the brain-computer interfaces as a practical reality." And this, he says, "is something we have not seen before." And although he admits that if it turns out to be hype from scratch, it “can throw off development in this area by ten to twenty years,” nothing will stop us from achieving the ultimate goal: a technology that will allow us to overcome the cognitive and physical limitations that previous generations of people were taken for granted.



“It will happen,” he insists. “This idea has the potential to change the direction of the evolution of the human race.”



Adam Pyor is the author of the book “Body Makers: The Science of Human Engineering” [The Body Builders: About Biotechnology.