Face recognition at a construction site

One bright sunny day, when the work at the construction site is in full swing, everything around is bright and positive, and I just want to fry the barbecue in the evening, a former employee of one of the contractors got into the facility. He walked past the entrance, waved his hand at the guard who knew him by sight (but did not know that he was fired), and went inside. There he went to his friends from Central Asia. Half an hour later they were already grilling kebab and enjoying life.



The nuance is that they lit a fire between two gas cylinders.



For some reason, one of the cylinders exploded. As they say, nothing portended, and for some reason the cylinder behaved atypically. If you saw how these cylinders are handled in the regions native to the heroes of the story, you can imagine their sincere perplexity from what happened.



All remained alive and even more or less intact (nothing irreversible). Shish kebab flew away, but did not enter orbit. Skewers did not pierce anyone below and from the side. But the police came to the failed astronauts, an ambulance and almost Sportloto.



I had to worry, but not as much as on the day when the inspection from the FMS came to the construction site. Then it turned out that so many work on the documents of the brothers. That is, one in the family makes himself a set of papers, gets a job on a construction site - and then a brother walks instead. Not a twin, but for a domestic neocortex almost indistinguishable.



For one such relative at a construction site, a fine of 500 thousand rubles or more is imposed, and construction can be suspended from 14 to 90 days. And it ends very, very sad for the general contractor and customer.



It was after this incident that they called us to solve the problem with IT tools.

Yes, yes, this is a continuation of the story about the terminator cat

The first experiments were in the form of turnstiles with infrared projectors, they did 3D scanning of the face (due to restoration of the geometry) and let those in the database pass.



Perhaps you have already read our post about the first implementation of the facial recognition system on walk-throughs by video. Brief story: a large store, several hundred people and about a third of this number - loaders, contractors, cleaners and other people from outstaff who are constantly changing. The task was to let only our own people pass and ensure that how many people went through the turnstile, there were so many passes. Many left for four of them for dinner, and one with four passes returned. On average, a worker cheated on an employer for 4 hours a week.



Then the recognition system gave a rather high percentage of false negative cases - up to 2.5% was on our part. These are dirty faces, fingals, waving hands in the face, overlapping, helmets, glasses and everything else. We could not control some of the factors, because in that part of the project we collected data from a conventional surveillance camera to first assess the extent of the problem.



Since then, technology has stepped far forward, and it has become possible to recognize faces accurately with a large overlap. The main thing is to have cameras of sufficient resolution and several frames with a face from which you can choose the best without overlapping. So we started catching shoplifters - this detective .



At this construction site it was necessary:

1. Allow only those who have a pass to the object. If the contractor’s team has a pass for a month, then exactly one month later the system will stop letting them in. Train.
2. Allow only those with a face that matches the model in the database to the object. When issuing a pass, the security officer of the object checks the documents and photographs the person on a webcam at the terminal (similar to receiving a pass to the Lenin Library in Moscow), and the person appears almost immediately in the database on the recognition kernel server.

Typical situations:

1. Another person is trying to pass through someone else's pass. Alarm, security stops it and understands manually.
2. A former employee is trying to get into the facility. Security does not let him in.
3. An employee who should not be trying to come to the facility: the guard asks for a pass and come with him.

Every evening, managers are sent a report on how many people were declared to work and how many came in fact. This is very important, because one of the biggest frauds of contractors is to declare 50 people, and in fact send 30 to the construction site. And then report about 50.



Also, these data on the actual number of people in the spirit of “Today should have been 600 people, and 450 came” help to understand that the deadlines are already breaking. Immediately, and not in a week, two, three-month.

What came before

Before that, there was a system of identification according to the pattern of veins in the palm of your hand. This is just a classic case, and we are already once again coming to remake for such systems. They work poorly even in sterile offices, where employees manage to smuggle cheburek. After contamination with a cheburek or any other fatty food, this very pattern changes.



At our construction site there were many ordinary people with fatty foods in the body, heroes with dirty hands and one superhero. No, he did not have authentication problems due to loss of hands. That would be the lesser of his problems. Everything is much more interesting: the electrician from the point of view of the system simply did not have this very picture. We don’t know how he achieved such a level of anonymization, but he played one of two factors: either the fact that he constantly twisted the wires with his bare hands, or the fact that these very twists quite regularly shocked him with fingers in response.



In general, there were only faces and irises.



Iris is slower and slightly more expensive. In the morning, buses arrive at the construction site, and from 7 to 8 you need to let in about 600 people.



We sat a couple of days on guard and recorded everything that could be useful for implementation. How workers behave when they come and go. How fast they walk through the checkpoint. What do they do at the moment of arrival and departure, do they go one after another, do they wear helmets, what is the weather like at the facility and how does it affect their appearance, do they overlap each other at the turnstile, how often do they return colorful after paintwork and so on.

Decision

Two cameras are installed on each turnstile: one looks at the incoming, the other at the exit.





Camera installation location.



On the territory of the object inside the perimeter there is a small server-room cooled by air conditioning, we have a rack there. In a rack - two servers and a switch. The first server receives a video stream of cameras and extracts from it the most successful frames with faces. The second server receives “clean” frames with faces and launches the recognition core on them, comparing these faces with the base of people.



Business logic spins on the first server, and turnstile management software is launched nearby. If there was only one person in the frame, and it has the right of passage, the turnstile opens. If something is wrong - the guard is notified.



For our task, the vendor stated 1 error (false positive and false negative) every 4 months. Here are their tests . We know that the system has been distinguishing twins for the past five years (tests for FSB officers, they asked if it is possible to go through a severed head - no, it’s impossible, the muscles “sag” and lose shape, and in general there are many interesting tests). It is clear that the vendor claims data for “greenhouse” conditions, but if you dig in the tables, you can see that good quality images are used there.



In practice, it is not always possible to obtain such images, however, we have created almost “greenhouse” conditions inside the change house with a walk-through.



All integration, managing software and image preparation is our development. An Internet solution is not needed, all communication via an ordinary copper cable inside the facility. But if you connect it, there will be constant monitoring, reports via e-mail and remote assistance support for second-line operators.



At the same time, the guard set a very tough check mode: now there is no “well, I just ask” everything through a pass, and any pass through entering a person into the database.

results

In practice, it was found that since the decoration had not yet begun, the “green men” did not leave the object. The maximum is dirt, but we already passed it at the World Cup, there was a fan coloring. Face recognition now focuses more on face geometry and builds a three-dimensional model inside the core. There were no problems with dirt. Slight bruises also did not affect: the main reference points are around the nose. One came at all all in solidarity, it seems, and we expected that he was not recognized. But nothing passed.



The lines in the morning almost disappeared. Previously, a large crowd gathered there, and now two turnstiles launch 700 people in an hour, taking into account the proceedings with guards and waving “well, I need it, the team is waiting there.”



The room had to be slightly modified so that there were no extra highlights of the face, the cameras should be arranged so that there were no overlaps. Hats (helmets) and glasses really represented a problem, but the rule was introduced that all this must be removed before going through the turnstile. They hung up signs with pictures.



The customer began to receive reports on the actual number of people at the construction site, which one of the contractors very upset.



I can not say anything about the payback of the system, since the main commercial case is the prevention of a FMS fine of 500 thousand and the closure of construction for 14-90 days. The system is cheaper than this fine, but checks happen irregularly, so it is difficult to calculate the economic effect. But old-timers say it's totally worth it.



Actually, the end of the story.

References

• Face recognition turnstiles (very old post)
• Number recognition (very old post)
• Terminator cat and adventures at the entrance
• How we recognized foremen and carts
• Different video analytics detectors
• Search shoplifters by faces
• Big Brother in production: detection of wearing helmets, gloves and cats on the machine
• My mail is CROCVBIACEL@croc.ru.