A few days ago, the Odyssey competition ended, in which engineering teams were looking for the best technology for finding people missing in the forest. In the summer I talked about the semi-finals , and yesterday I posted a great report from the finals .
The organizers set an enormously difficult task - to find two people in an area of 314 km 2 in 10 hours. The ideas were different, but (spoiler) no one managed. One of the technical experts of the competition was Nikita Kalinovsky. I discussed with the participants, their solutions, and also asked what other ideas were remembered throughout all stages of the competition.
If you have already read the report about the finale, some remarks will meet you here. This is just a complete interview with minimal editing.
If you have not read more than one article in this series, I will briefly recount the context.
In previous series
AFK Sistema Foundation launched the Odyssey contest to find ways to introduce modern technology in the search for people who are lost in the wild without means of communication. Of the 130 teams, four teams reached the final - only they could find people in the forest with an area of 4 km 2 twice in a row.
The Nakhodka team, founded by veterans of the Yakutia Rescue Service. These are search engines with great experience in real forest conditions, but perhaps the least advanced team in terms of technology. Their solution is a large sound beacon, which, with the help of a special signal configuration, is clearly audible at a distance of up to one and a half kilometers. A man comes to the sound and sends a signal to the rescuers from the lighthouse. The chip is not so much in technology as in tactics of its use. Search engineers use a minimum of lighthouses to shield the perimeter of the search and, gradually narrowing it down, find a person.
The “Top” team is the exact opposite of the “Find”. Engineers rely entirely on technology and do not use ground forces at all. Their solution is drones equipped with customized thermal imagers, cameras and loudspeakers. Search among the footage is also conducted by algorithms, not people. Despite the skepticism of many experts about the uselessness of thermal imagers and the low level of algorithms, “Vertex” several times found people in the semifinals and finals (but not those that were needed).
Stratonauts and MMS Rescue are two teams that use a whole range of solutions. Sound beacons, balloons for establishing communications on the territory, drones with photography and real-time search trackers. The stratonauts were the best in the semifinals, because they found the missing fastest.
Beacons have become the most effective and common solution, but with their help you can only find a person who is able to move. A lying person has almost no chance. It seems that it is best to look for it with a thermal imager, but the thermal imager does not see anything through the crowns, and also hardly distinguishes heat spots from people among all other objects in the forest. Photographing, algorithms and neural networks are promising technologies, but so far they are not showing themselves well. There were also exotic technologies, but each of them had more limitations than advantages.
The Nakhodka team, founded by veterans of the Yakutia Rescue Service. These are search engines with great experience in real forest conditions, but perhaps the least advanced team in terms of technology. Their solution is a large sound beacon, which, with the help of a special signal configuration, is clearly audible at a distance of up to one and a half kilometers. A man comes to the sound and sends a signal to the rescuers from the lighthouse. The chip is not so much in technology as in tactics of its use. Search engineers use a minimum of lighthouses to shield the perimeter of the search and, gradually narrowing it down, find a person.
The “Top” team is the exact opposite of the “Find”. Engineers rely entirely on technology and do not use ground forces at all. Their solution is drones equipped with customized thermal imagers, cameras and loudspeakers. Search among the footage is also conducted by algorithms, not people. Despite the skepticism of many experts about the uselessness of thermal imagers and the low level of algorithms, “Vertex” several times found people in the semifinals and finals (but not those that were needed).
Stratonauts and MMS Rescue are two teams that use a whole range of solutions. Sound beacons, balloons for establishing communications on the territory, drones with photography and real-time search trackers. The stratonauts were the best in the semifinals, because they found the missing fastest.
Beacons have become the most effective and common solution, but with their help you can only find a person who is able to move. A lying person has almost no chance. It seems that it is best to look for it with a thermal imager, but the thermal imager does not see anything through the crowns, and also hardly distinguishes heat spots from people among all other objects in the forest. Photographing, algorithms and neural networks are promising technologies, but so far they are not showing themselves well. There were also exotic technologies, but each of them had more limitations than advantages.
- What do you do out of competition?
- Group of companies INTEC, the city of Tomsk. The main area is industrial design, development of electronics and software, including embedded. We have our own small experimental and small-scale production, we help to bring the product from idea to mass production. One of our most famous projects is the NIMB project, which we have been developing since 2015. In 2018, we received the Red Dot Design Award with this project. This is one of the most prestigious awards in the world of industrial design.
- What is this thing doing?
- This is a security ring, an alarm button that the user presses when an alarm event occurs. It looks like an ordinary finger ring. On its lower part there is a button, inside a Bluetooth module for communication with a smartphone, a microelectromotor for tactile indication, a battery, a three-color LED. The base is a combined flexible-rigid board. The main part of the body is metal, the cover is plastic. This is a fairly well-known project. In 2017, they raised about 350 thousand dollars on Kickstarter.
- How do you like it here? Do teams live up to expectations?
- In some teams, people have extensive experience in searching, they have been to the forest more than once, and have conducted such events more than once. They understand well how to find a person in real conditions, but they understand very poorly in technology. In other teams, the guys are very well versed in technology, but have absolutely no idea how to move through the forest in summer, winter, autumn conditions.
- There is no middle ground?
- I have not seen it yet. The general opinion of all experts is this: if you combine all the teams, drive them into a single collaboration, force them to combine solutions, take the best of each and implement them, you will get a very cool set. It is naturally necessary to finish it, bring it to a sane grocery condition, and bring it to its final presentation. However, this will be a very cool decision that can really be used and that will really save lives.
But individually, each of the solutions is not fully effective. Somewhere lacking all-weather, somewhere lacking round-the-clock, someone is not looking for unconscious people. You always need to approach in a comprehensive way and most importantly, you always need to understand that there is a certain theory of people search and the complex should correspond to this theory.
Now the decisions are damp. Here you can see two classes of projects: the first - very simple and very reliable systems that work. Those sound beacons that the guys from Yakutia brought in, the Nakhodka team is a unique device. It can be seen that people with great experience did it. Technically, it is very simple, it is an ordinary pneumatic signal with a LoRaWAN module and with an MESH network deployed on it.
“What is so unique about her?”
- She can be heard for one and a half kilometers in the forest. For many others, this effect is not observed, although the volume level is about the same for everyone. But the right frequency and configuration of the pneumatic signal give such results. I personally recorded the sound at a distance of about 1200 meters, with a very good understanding that this is really the sound of the signal and the direction to it. In real life, this thing works fine.
- At the same time, she looks the most not technological.
- It really is. They are made from a piece of PVC pipe, with the most simple, reliable and very effective solution. But with its limitations. We cannot take and with the help of these devices find a person who is unconscious.
- The second class of projects?
- The second class is complex technical solutions that implement various specific search models - search using thermal imagers, combining thermal and three-color images, drones and more.
But, everything is very raw there. Locally used neural networks. They are deployed on personal computers, on nvidia jetson boards, on the aircraft themselves. But all this is still underexplored. And as practice has shown, the use of linear algorithms in these conditions worked much more efficiently than neural networks. That is, determining a person by the spot in the image from the thermal imager, using linear algorithms, by area and by the shape of the object, gave a much greater effect. The neural network did not find almost anything.
“Because there was nothing to teach her?”
- They claimed to be taught, but the results were extremely controversial. Not even that controversial - they were almost nonexistent. Neural networks have not shown themselves here. There is a suspicion that they were either not taught correctly or taught the wrong way. If you correctly apply neural networks in these conditions, then most likely they will give a good result, but you need to understand the entire search methodology.
- They say that neural networks are promising. If you make them cool, then they will work. About the thermal imager, on the contrary, they say that it is useless in any case.
- Nevertheless, the fact was recorded. The thermal imager is really looking for people. As with neural networks, you need to understand that we are talking about tools. If we take a microscope, then to examine small objects. If we hammer a nail, then it is better not to use a microscope. With the thermal imager and neural networks all the same. A properly tuned, correctly applied tool in the right conditions gives a good result. If we use the tool in the wrong and wrong way, it is natural that we will not get the result.
“Well, how to use a thermal imager if it is said here that even a rotting stump gives more heat than a missing grandmother?”
- Not more. Checked, watched - no more. Man has a clear pattern. You need to understand that a person is a very specific object. Moreover, at different times of the year these are different objects. If we are talking about summer, then this is a man in a light T-shirt or in a T-shirt or shirt, which glows with a powerful spot on the thermal imager. If we are talking about autumn, about winter, then we see a head covered with a hood with the rest of the heat trail that is knocked out from under the hood or from under the hat, luminous hands - everything else is hidden by clothes.
Therefore, a person can be seen in the thermal imager, clearly visible, I myself saw it with my own eyes. Another thing is that boars, elks, bears are clearly visible, and we need to filter very clearly, which we still observe. You just can’t get by with one thermal imager, you just can’t just take it, show it on the thermal imager and say that it will solve all our problems. No, there must be a complex. The complex should include a three-color camera, which gives a full-color image or monochrome backlit with LEDs. It should be with something else, because the thermal imager itself gives just spots.
- Of those teams that are now in the finals, who is the coolest?
- No pets, to be honest. I can throw a solid brick at everyone. Let's just say that I really liked the decision of the first “Top” team. They just had a thermal imager plus a three-color camera. I liked the ideology. The guys are searching by technical means without attracting ground forces, they did not have mobile crews at all, they searched only with drones, while they found people. I won’t say whether they found the one they needed or not, but they found people and found animals. If we compare the coordinates of the object on the thermal imager and the object on the three-color camera, then we can identify the object and determine whether the person is there.
I have questions about implementation, the synchronization of the thermal imager and the camera was done carelessly, it was practically not there at all. Ideally, the system should have a stereo pair, one monochrome camera, one three-color camera and a thermal imager, and they all work in a single time system. This was not here. The camera worked in a separate system, the thermal imager in a separate system, and they had artifacts because of this. If the speed of the drone is slightly higher, this would already give very strong distortion.
“Did they fly a copter or was there a plane?”
- No one was here copter. Rather, copters were launched by one of the teams, but it was a purely technical function to provide communication in the search area. The Lorovo repeater was hung on them, and it provided communication within a radius of 5 kilometers.
As a result, all search aircraft here are aircraft type. This entails its own problems, because it is not easy to take off and land. For example, yesterday, weather conditions did not allow the Nakhodka team to launch their drone. But I would say this - the drone, which was in their arsenal, would not have helped them in the form in which it is now configured.
- In the semifinals, they wanted to use the drone only for relay.
- The drone at Nakhodka was made for photo-video shooting and notification. There is an alarm siren, a thermal imaging camera and a color camera. At least this is what I heard from them. They didn’t even unpack it yesterday. As they brought it, it was packed. But even if they got it, most likely they would not have used it. They have a completely different tactic - they searched with their feet.
Today, the guys want to plant the forest with lighthouses and find a person with the help of them. I like this solution the least. I have great doubts that they will then be collected by the 350 lighthouses that they brought here. Or rather, we will force them to collect, but not the fact that they will collect everything. Most of all I liked the decision of the first team, because it involves a complete rejection of ground forces.
- Only because of this? After all, if you really take the amount of such a huge area, it may work.
- It most likely will work, but I didn’t like either the dropping configuration or the configuration of the beacons themselves.
- There was a brick for the Stratonauts.
“The Stratonauts have a cool solution.” If they did it the way they wanted to do, they would have succeeded. But they also had problems with flying vehicles.
They have a search group support system. The main emphasis is on mobile ground forces. They receive lighthouses, provide communication with groups and communicate with ground-based lighthouses to deploy search groups at the right points and in the right directions. They have balloons with repeaters that provide communication over the area. They have land-based stationary beacons, but there are very few of them, and they themselves admit that they did them at the last moment, and they do not have this basic tactical unit - they did them for the sake of testing. There are few of them and they have not made a special contribution to tactics.
The main tactic was that each search engine in the group has its own personal tracker, which is integrated into a single information network with the headquarters. They clearly see who is in what place. There is a combing in real time, the direction is adjusted.
- Everything looks so that you really want to combine into one.
- Yes, absolutely so. Grigory Sergeyev and I went, he looks and says - “Damn, what a cool thing, I would have it”, we come to others - “Damn, what a cool thing, I would have it”, we come to the third - “Damn, what a cool thing “I would have found a man there and there.”
Separately, they are sectorally good solutions for certain conditions. If you combine them, then it turns out a very good complex, which has a single communication field, there is a deployment of the system at a long range with the help of balloons, there is a tracking system and control of ground forces in real time, there are beacons that hit a sufficiently long range and can proper use and dissection of the search area into sectors to provide an alarm to a person so that he goes to them, and then everything turns into a matter of technology. There is flying weather - some forces are used, no flying weather - others, night - third.
“But it's all disastrously expensive.”
- Something expensive, something not.
“For example, one drone that is now taking off, probably stands like a Boeing.”
- Yes, their cost is quite high. But you need to understand that with proper use it is a one-time purchase. You need to buy once, and then just carry around the country and apply. Such a one-time investment in capable hands will last long enough if properly maintained and operated.
- When you watched the applications for the competition, was there something that you liked, but did not reach the final?
- There was a lot of funny things.
- What is the funniest thing to remember?
- The radar radar suspended on a balloon was very memorable. I laughed for a long time.
“It’s even scary to ask what it is.”
- The trick is that this is a really good way to determine. Bioradiolocation is aimed at determining exactly biological living objects against the background of everything else that is reflected. Usually used chest oscillation, pulse. For this, very high-frequency radars under 100 GHz are used, they shine at a fairly good distance and shine through the forest 150 to 200 meters deep.
“Then why is it funny?”
- Because this thing only works when installed permanently, and they wanted to hang it on a balloon. And they say - "this is a stationary object." Now we are looking at a balloon, it is constantly sausage, and they want to hang a thing on it that should be tightly screwed to the ground, otherwise there will be such a picture that nothing will be understood on it at all.
Still very funny were cardboard unmanned aerial vehicles.
- Cardboard?
“Yes, cardboard drones.” It was very funny. Aircraft glued from cardboard and painted with varnish. He flew just as God puts his soul. The guys wanted him to fly in one direction, and he flew anywhere, only not in the one that was needed, and eventually crashed, saving himself from torment.
Very funny was “a flying bagel that can be reconfigured into a flying sausage” - a real quote from the application. The outer braid of the fire hose is taken, the rubber is removed, inflated and becomes a long pipe twisted on both sides. They connect it, and it turns out a flying bagel, on which the camera is hung. And that a bagel can be easily transformed into a flying sausage - everyone was laughing at the sausage. Why, why sausage - it is not clear, but it was very funny.
- I heard about the cubes that are placed on the ground, and they read vibrations and steps.
- Yes, indeed, there were such things. You need to understand that the thing is actually quite working. I know several commercial products that work just that way. This is a security-tuned seismograph for perimeter support systems. But this thing is used exclusively for critical infrastructure and military facilities. I know that at gas pumping stations there are three-level access control systems, the first of which is just seismographs.
- Sounds like promising. Why not then?
- The fact is that it is one thing to protect the closed perimeter of the critical infrastructure object with a small area, and it is another to plant the whole forest with these seismographs. Their range is very small, and most importantly, you can hardly distinguish between a running boar, a running man and a running bear. Theoretically, you can of course, if you turn on the device correctly, but this all greatly complicates the technique, there are methods much easier, as it seems to me.
Everyone was recommended to go to the quarter finals, everyone was recommended to try their hand. Those we see here are those who really managed to find people. All other people did not find, so the competition, as it seems to me, is fairly objective. You can, for example, trust the opinion of experts, you can not trust, but the fact remains - found or not found.