Scientists from the Russian group of companies "Neurobiotic" and the Laboratory of Neurobototechnics MIPT were able to recreate images of the electrical activity of the brain . True, we are only talking about working with images that are looked at by a person whose brain is being analyzed.
Actually, restoring the geometric shapes that a person is looking at at a certain point in time from the electrical activity of his nerve cells is not an end in itself. The main thing is to understand how the brain encrypts the information that we then store for many years (well, or minutes, which is no less important).
The work of Russian scientists intersects with the projects of foreign colleagues. But in most cases, other researchers use functional magnetic resonance imaging or analysis of electrical signals that are received from neurons. These two methods have serious limitations in use - in the hospital or at home (not now, of course - in the future).
On the contrary, in the work of domestic experts , an electroencephalogram is used, which is removed from the person’s head in the usual way . Then the obtained data is “fed” by a specially trained neural network. As a result, the latter determines what the subject is looking at, and builds a geometric figure by the configuration of his electric signal.
The experiment, which was conducted by experts, consisted of two parts. The first used videos from five different categories, including “abstractions”, “waterfalls,” people's faces, “speed.” So, the videos were videos shot from the first person on snowmobiles and other vehicles. The duration of the videos was 10 seconds, the session for each subject lasted about 20 minutes.
As a result, scientists were able to prove that the configuration of the electrical activity of the brain differs significantly depending on the category of videos watched.
After this, the second stage of the experiment began. For her, videos were also selected, not five, but three categories. Also in the second part of the experiment involved two neural networks. One of them generated arbitrary images from the demonstrated "noise", the second - generated "noise" from the EEG. After that, both neural networks were combined, training the system to reproduce the image using the EEG recorded by the device.
After that, the volunteers began to show videos of the same categories, recording EEG. The signal in real time was sent for processing to a binary system of neural networks. As a result, an image was reproduced, which in most cases was relatively exactly the same as the original. In general, the task was not to reproduce the exact video, but to understand what category the image generated by the neural network belongs to. This was done in 90% of cases.
“Encephalogram is a trace signal from the work of nerve cells, taken from the surface of the head. Previously, it was believed that to study the processes in the brain by EEG is the same as trying to recognize the structure of the engine of a steam locomotive by its smoke. We did not assume that it contains enough information to at least partially reconstruct the image that a person sees. However, it turned out that such a reconstruction is possible and demonstrates good results. Moreover, on its basis it is even possible to create a real-time interface “brain - computer”. This is very encouraging. Now the creation of invasive neurointerfaces, which Elon Musk talks about, rests on the complexity of the surgical operation and the fact that after a few months they fail due to oxidation and natural processes. We hope that in the future we will be able to make more affordable neural interfaces that do not require implantation, ”said Grigory Rashkov, one of the authors of the work, a junior researcher at the Moscow Institute of Physics and Technology and a programmer and mathematician at Neurobotics.