Imagine the problem - two people were lost in the forest. One of them is still mobile, the other lies in place and cannot move. The point where they were last seen is known. The search radius around it is 10 kilometers. It turns out an area of 314 km 2 . You have ten hours to search using the latest technology.
When I heard the condition for the first time, I thought: "pff, hold my beer." But then I saw how advanced solutions stumble on everything that is possible and impossible to take into account. In the summer, I wrote how about 20 engineering teams tried to solve the problem tens of times easier, but they did it to the limit, and only four teams did it. The forest turned out to be a territory of hidden tricks, where modern technology is powerless.
Then it was only the semi-finals of the Odyssey competition organized by the Sistema charitable foundation, the purpose of which was to come up with how to modernize the search for people lost in the wild. In early October, the Vologda Oblast hosted its final. Four teams faced the same challenge. I went to the place to watch one of the contest days. And this time he rode with the thought that the task is unsolvable. But I didn’t expect to see a “True Detective” for DIY electronics lovers.
Snow fell early this year, but if you live in Moscow and wake up late, you may not see it. That which does not melt itself will be scattered by the workers one hundred percent. It takes seven hours to travel from Moscow by train and a couple more hours by car — and you will see that winter has actually begun long ago.
The final took place in the Syamzhensky district near Vologda. Near the forest and a village of three and a half houses, the Odyssey organizers set up a field headquarters - large white tents with heat guns inside. Three teams have already searched in previous days. Nobody talked about the results, they were under the NDA. But according to facial expressions, it seemed that no one had done it.
While the last team was preparing for the test, the rest of the participants put up their equipment on the street for beautiful shots of local television, showed and explained how it works. The Nakhodka team from Yakutia rattled beacons so that reporters interviewing had to pause.
They passed the test the day before, and they got the worst weather. Snow and gusty wind did not even allow to launch the drone. Many lighthouses could not be set up because the transport was breaking. And when one of the devices finally worked, it turned out that a tree fell in the wind, and it pressed a button. However, the team is being watched with curiosity because they are the most experienced search engines.
- I have the whole team - hunters. They have long been waiting for the first snow. They will see the traces of any beast, as if they were catching it. I had to restrain them as guard dogs, ”says Nikolai Nakhodkin.
While combing the forest on foot, they probably could have found a trace of a person, but they would not have counted such a victory - this is a technology contest. Therefore, they relied only on their sound beacons with a powerful punching sound.
A unique device in fact. It can be seen that people with great experience did it. Technically, it is very simple - it is an ordinary air-coil with a LoRaWAN module and with a MESH network deployed on it. It can be heard for one and a half kilometers in the forest. Many others do not have this effect, although the volume level is about the same for everyone. But the right frequency and configuration give such results. I personally recorded a sound at a distance of about 1200 meters with a very good understanding that this is really the sound of a signal.
They look the most low-tech, and at the same time they have the simplest, most reliable and very effective solution, let’s say, but with their own limitations. We cannot take and with the help of these devices find a person who is unconscious, that is, these products are applicable only in a very narrow range of situations.
- Nikita Kalinovsky, technical expert of the competition
Today, the last of the four teams worked - MMS Rescue. These are ordinary guys, programmers, engineers, electronic engineers who had never before been engaged in searches.
Their idea was this - with the help of several aircraft-type drones, scatter a hundred or two small sound beacons over the forest. They connect into one network, where each unit is a radio signal repeater, and begin to make a loud sound. The lost person must hear it, find it, press the button and thus transmit a signal about his whereabouts.
UAVs are taking pictures at this time. Autumn forest is almost transparent during the day, so the team hoped to detect a lying person in the photo. At the base, they had a trained neural network through which all the pictures were driven.
In the semifinals of MMS Rescue, beacons scattered with ordinary quadrocopters - this was enough for four square kilometers. To cover 314 km 2 , you need an army of copters and, probably, several points to launch. Therefore, in the final, they teamed up with another team that had previously dropped out of the competition and used their Albatros aircraft.
The start of the search was scheduled for 10 in the morning. There was a terrible bustle in front of him in the camp. Journalists and guests walked around, the participants dragged equipment for technical inspection. Their tactics of planting woods with lighthouses no longer seemed an exaggeration when they brought and unloaded all the lighthouses - almost five hundred pieces.
“Each one is based on arduino, oddly enough.” Our programmer Boris made an amazing program that manages all attachments, says Maxim, a participant in MMS Rescue, “We have a LoRa, our own design board with attachments, mosfets, stabilizers, a GPS module, a rechargeable battery and a 12 V siren.
Each lighthouse costs about 3 thousand, despite the fact that the guys each ruble was on the account. The development and production was only two months. For most team members, the MMS Rescue project is not the main activity. Therefore, they returned from work and were engaged in preparation until late at night. When the parts arrived, they manually assembled and soldered all the equipment themselves. But the technical expert of the competition was not impressed:
“I like their solution the least.” I have great doubts that they will then collect the three hundred lighthouses that they brought here. Rather, how - we will force them to collect, but not the fact that it will turn out. The search itself will most likely work if you plant so many, but I did not like the dropping configuration or the configuration of the beacons themselves.
- Lighthouse technology reduces the number of kilometers traveled by feet. The lighthouses, which will be scattered now, suggest a further hike in the forest for collection. And it will be a distance that does not reduce the amount of human labor. That is, the technology itself is okay, but maybe you need to think out tactics how to scatter it, so that later it is easier to collect, says Grigory Sergeyev from Lisa Alert.
Two hundred meters from the camp, the drone crew equipped the launch pad. Five airplanes. Each takes off with the help of a slingshot, carries four lighthouses on board, scatters them in about 15 minutes, returns and lands with a parachute.
Missing Hunters
After the start of the search, the camp began to empty. The journalists parted, the organizers scattered around the tents. I decided to stay all day and see how the team would work. Some of the participants were still involved in the control of drones, some got into a car and drove through the forest to set up lighthouses along the roads manually. Maxim stayed in the camp to monitor how the network is deploying and receive signals from lighthouses. He told me more about this project.
- Now we are watching how the network of lighthouses unfolds, we see lighthouses that appeared on the network, what happened to them when we saw them for the first time, and what is happening now, we see their coordinates. The table is populated with data.
- Sitting and waiting for a signal?
- Roughly speaking, yes. We just have never scattered 300 lighthouses. Therefore, I look at how you can use the data from them.
- By what principle do you scatter them?
“We have a program that analyzes the terrain and considers where lighthouses should be dropped.” She has her own set of rules - so she looks into the forest and sees a path. First, she will offer to throw lighthouses along her, and then she will go into the forest, because the deeper, the less likely that a person is there. This is a practice voiced by rescue teams and people who were lost. I recently read that a missing boy was found 800 meters from his house. 800 meters - not 10 km.
Therefore, first we look as close as possible to the likely entry area. If a person got there, then he is most likely still there. If not, then we will incrementally expand the search boundary. The system simply grows around the likely point of human presence.
This tactic turned out to be the opposite of that used by experienced search engines from Nakhodka. On the contrary, they calculated the maximum distance that a person can go from the entry point, placed beacons around the perimeter, and then closed the ring, reducing the search radius. At the same time, the lighthouses were arranged so that a person cannot go beyond the ring without hearing them.
- What did you develop specifically for the finale?
- We have a lot of things that have changed. We conducted many tests, measured different antennas in the forest, measured the signal travel distance. In previous trials, we had three lighthouses. We carried them on foot and fastened to the trunks of trees at a short distance. Now the housing is adapted for resetting from a drone.
It falls from a height of 80-100 meters at a speed of 80-100 km / h drone, plus the wind. Initially, we planned to make the shape of the body in the form of a cylinder with a wing sticking up. They wanted to place the center of gravity in the form of batteries in the lower part of the hull, and the antenna would automatically rise up to achieve good indicators of communication between lighthouses in the forest.
“But they didn’t do it?”
- Yes, because the wing into which we inserted the antenna was a hindrance to the plane. Therefore, they came to the shape of a brick. Plus they tried to solve the issue of nutrition, because each element is heavy, it is necessary to cram the minimum mass into a small building while maintaining the maximum amount of energy so that the lighthouse does not die in an hour.
Modified software. 300 beacons on the same network can interrupt each other, so we made the spacing. There is a big complex task.
It is necessary that our 12 V sirens shout as needed, that the system lasts at least 10 hours, so that the arduino does not reboot when LoRa is turned on, so that there are no pickups from the squeaker, because there is a boost device that gives 40 V of 12.
- And what to do with the lying person?
- Unfortunately, no one gave a reliable answer to this question. It would seem wiser to search with dogs for smell along fallen trees. But it turned out that dogs are found by far fewer people. If the lost person lies somewhere on the windbreak, theoretically it can be photographed and recognized from the drone. We have two planes flying with such a system, we collect data in the air and analyze at the base.
- How will you analyze the photos? Watch everything with your eyes?
- No, we have a trained neural network.
- On what?
- On the data that we ourselves collected.
When the semi-finals passed, the experts said that a lot of work still needs to be done to search for people using photo analysis. Ideal when the drone analyzes images in real time on board using a neural network trained on a huge amount of data. In reality, teams had to spend a lot of time to download the footage to a computer, and even more to watch it, because no one really had a working solution then.
- Now neural networks are used in places, and they are deployed both on personal computers, on Nvidia Jetson boards, and on the aircraft themselves. But all this is so raw, so understudied, says Nikita Kalinovsky, - as practice has shown, the use of linear algorithms in these conditions worked much more efficiently than neural networks. That is, determining a person by the spot in the image from the thermal imager using linear algorithms according to the shape of the object gave a much greater effect. The neural network has found almost nothing.
- Because there was nothing to teach?
- They claimed to be taught, but the results were extremely controversial. Not even that controversial - they were almost nonexistent. There is a suspicion that they were either taught incorrectly or taught the wrong way. If you correctly apply neural networks in these conditions, then most likely they will give a good result, but you need to understand the entire search methodology.
- We recently launched a story with a Beeline neuron , says Grigory Sergeyev, - While I was here at the competition, this thing found a person in the Kaluga region. That is, here is a real application of modern technology, it is really useful for searching. But it is very important to have a medium that flies for a long time and allows you not to smear photos, especially at dawn and sunset, when there is practically no light in the forest, but you can see something else. If optics allows, this is a very good story. In addition, everyone is experimenting with thermal imagers. In principle, the trend is correct and the idea is correct - the issue of price is always worrying.
Three days before, on the first day of the finals, the search was led by the “Top” team - perhaps the most technologically advanced of the finalists. If everyone relied on beacons, the main weapon of this team was a thermal imager. To find a market model that is capable of delivering at least some kind of result, refine and customize it - all this was a separate adventure. As a result, something happened, and I heard enthusiastic whispers when a beaver and several moose were found with a thermal imager in the forest.
I really liked the decision of this team precisely by ideology - the guys are looking for technical means without attracting ground forces. They had a thermal imager plus a three-color camera. They searched only by flight, while they found people. I won’t say whether they found the one who was needed or not, but they found both people and animals. The object on the thermal imager and the object on the three-color camera were compared by coordinates, and it was determined that it was precisely according to two images.
I have questions about implementation - the synchronization of the thermal imager and the camera was done carelessly. Ideally, the system would work if there were a stereo pair in it: one monochrome camera, one three-color camera, a thermal imager, and all work in a single time system. This was not here. The camera worked in one system, the thermal imager in a separate system, and they had artifacts because of this. And if the speed of the letak is slightly higher, this would already give very strong distortions.
- Nikita Kalinovsky, technical expert of the competition
Most categorically about thermal imagers spoke Grigory Sergeev. When I asked his opinion about this in the summer, he said that thermal imagers were just a fantasy, and in ten years the search team had never found anyone with their help.
- Today I see a drop in prices and the emergence of Chinese models. But while it's still wildly expensive, dropping such a thing is twice as painful as the drone itself. A thermal imager that can show something decently costs more than 600 thousand. The second Mavik costs about 120. Moreover, the UAV can already show something, but the thermal imager needs specific conditions. If for one thermal imager we can buy six Maviks without a thermal imager, naturally we will act Mavikov. It’s not worth imagining that we will find someone under the crowns — we will not find anyone, the crowns are not transparent to the greenhouse.
While we were discussing all this, there was not much activity in the camp. UAVs took off and took off, somewhere in the distance the forest was overgrown with lighthouses, but there were no signals from them, although half the allotted time had already passed.
At six o’clock, I noticed that the guys began to actively talk on the walkie-talkie, Maxim sat down at the computer, very alarmed and serious. I tried not to meddle with questions, but after a few minutes he came up to me himself, quietly exhausted himself. A signal came from the lighthouses. But not from one, but from several at once. Over time, more than half of the units sounded the SOS signal.
In such a situation, I would think that these are software problems - the same mechanical failure cannot crawl out simultaneously on so many devices.
“We ran the tests two hundred times.” There were no problems. It cannot be software.
A few hours later, the database was filled with false signals and a bunch of unnecessary data. If at least one of the lighthouses worked by pressing, Max had no ideas how to determine it. Nevertheless, he sat down and began to manually sort through everything that came from the devices.
Theoretically, the real lost could find the lighthouse, take it with you and move on. Then, perhaps, the guys would detect movement on one of the units. How will the statistician depicting the lost behave? Also take or go to the base without a device?
About six o'clock the guys who were engaged in a drone came running to headquarters. They uploaded photographs and found on one of them very clear traces of a person.
The tracks passed a thin strip between the trees and hid outside the photograph. The guys looked at the coordinates, compared the photo with the map and saw that it was at the very edge of their flight zone. The tracks go north, where the drone did not fly. The photo was taken more than five hours ago. On the walkie-talkie, someone asked what time it was. They answered him: "now is the time of our flight."
Max continued to delve into the database and found that all the beacons were starting to signal after the same time. Something like delayed activation was laid in them. To prevent the button from working during a flight and a fall, it was deactivated during delivery. That is, the lighthouse was supposed to come to life and begin to make sounds half an hour after departure. But along with activation, the SOS signal also triggered for everyone.
The guys took out several lighthouses that they did not manage to send, picked up, and began to sort through all the electronics, trying to find what could go wrong. But not much could go. When the electronics were tested, it was not yet packaged in a housing that could withstand the discharge. The solution was found quite late, so several hundred lighthouses were collected by hand at the last moment.
Max at this time manually sorted through all the messages from the beacons in the database. There was one hour left until the end of the search.
Everyone was nervous, me too. Finally, Max left the tent and said:
- Write there in your article so that they never forget to screen.
Having disassembled several lighthouses, the guys clung to the theory. Since the housing for the lighthouses appeared very late, all the electronics had to be packaged more compactly than planned. And due to the fact that time was running out, the guys did not have time to shield the wires.
After a few minutes, a signal was found in the database from the device, which worked much later than the others. This lighthouse was not delivered to the forest on a drone, the guys brought it themselves and tied it to a tree next to one of the roads. The signal from him arrived at half past one, and now the clock was already half past seven. If the button was really pressed by the statistician, then due to noise the signal from it could not be recognized for several hours.
Nevertheless, the guys perked up, quickly wrote out the coordinates of the lighthouse and the activation time, and immediately ran to fix the find.
Much was at stake, and technical experts reacted to the find with distrust. How could one really work out among the heaps of broken lighthouses? The guys in a hurry tried to explain.
- Let's go back one step. Replacing the case has led to the fact that your signals stopped working after a fall?
- Not certainly in that way.
- Is it connected with the case?
- This is due to the fact that the SOS button worked until the moment it should work.
- She activated when falling?
- Not when falling, but when a sound signal is triggered. A sound signal gave a peak-peak, 12 V was converted to 40 V, a tip was given to the wire, and our controller thought that the button was pressed. These are assumptions so far, but very similar to the truth.
- Very strange. She can’t give such a tip. I strongly doubt it. The reason for the false positives in terms of circuitry?
- I'll explain now, everything is simple. Previously, the body was wider and the distance between the elements was greater. At the moment, some wires, including the wire from the button, pass right next to this thing.
- Is it a transformer?
- Yes. And not only with him. He raises to 40 V, this is a raise. Still nearby is a 1 watt antenna. When transmitting, we get a certain message, and then it goes into the SOS state.
- How is your button tied to a percent?
- Just hanged on the GPIO, with a lower lift.
- You hung the button directly on the port, pulled it down and any signal that passes through it immediately jumps up, right?
- Well, it turns out so.
“Then it seems to be true.”
- I, too, already understood what had to be pulled wrong.
- Have you tried wrapping the wires with foil?
- They tried. We have several such lighthouses.
“Well, you saw that when the signals go on the beep, and when the signal goes on the antenna, you have ...”
- Not certainly in that way. Not when the beep sounds, but when the time comes for activation of the beacon. The button is cut off so that it does not accidentally click on a branch or something else when it flies on an airplane. There is a certain time delay. When the time comes for turning on, activating the button, the entire beacon turns on, as if they had taken and closed the power for it. No delays, nothing, all the elements started to rise and work right away, and at that moment the button is triggered.
“Why then doesn’t everyone work like that?”
- Because there is an error.
“Then the next question.” How many products worked on false positives? More than a half?
- More.
- How did you single out one of them, which you submitted as the coordinates of the missing?
- Our captain drove by car to the most likely areas and carried the beacons manually. He took a box in which there was a separate batch of lighthouses, and in fact arranged those lighthouses that did not have such an error. We conducted an analysis of the data that we collected, identified from all those who did not start screaming SOS at the time when it should be activated and went out to the lighthouse, which started screaming SOS much later than 30 minutes.
“Do you admit that there was first no false positive, and then it could appear?”
- Well, you know, for more than 70 minutes since the revival of the lighthouse, he stood. We analyzed the coordinates - this is not far from the place where, according to legend, a man appeared.
Half an hour before the end of the search, the team finally took the coordinates of the missing person. It seemed like a real miracle. In the forest, a mountain of lighthouses, more than half broke. Worse, half of the lighthouses from the party that were placed manually broke too. And on the territory of 314 square kilometers, strewn with broken lighthouses, the statistician found a worker.
All that was needed was a check. But the team went to celebrate a possible victory, and I, after eleven hours in the cold, could leave the camp with a calm soul.
On October 21, about a week after the test, I received a press release.
„”, , «». , , „” 30 .
In addition to the Stratonauts, two more teams recognized as promising - “Nakhodka” from Yakutia and “Top” with their thermal imager. “Until the spring of 2020, teams together with rescue teams will continue to test their technical solutions, participating in search operations in the Moscow, Leningrad regions and Yakutia. This will allow them to refine their decisions for specific search tasks, ”the organizers write.
MMS Rescue was not mentioned in the press release. The coordinates they transmitted turned out to be incorrect - the statistician did not find this lighthouse, and did not press anything. Still, this was another false positive. And since the idea with continuous sowing of the forest did not find a response from experts, they abandoned it.
But the Stratonauts also did not cope with the task in the final. They were the best in the semifinals. Then, on an area of 4 square kilometers, the team found a person in just 45 minutes. Nevertheless, experts recognized their technological complex as the best.
Perhaps because their solution is the middle ground between everyone else. This is a communication balloon, drones for shooting, sound beacons and a system that tracks all search engines and all elements in real time in real time. And at least this system can be taken and equipped with real search squads.
“Search today is still a stone age with rare flashes of something new,” says Grigory Sergeyev, “unless we go with ordinary torches, but with LED ones. We are still not at the stage when people from Boston Dynamics walk through the woods, and we smoke at the edge of the forest and wait for them to bring us the missing grandmother. But if you do not move in this direction, do not move all scientific thought, nothing will happen. We need to excite the community - we need thinking people.