Tales from the data center: Halloween horror stories about diesels, diplomacy and self-tapping screws in the heater

My colleagues and I thought: why, before the beloved horror holiday, instead of successes and interesting projects, remember all kinds of horror movies that people encounter in the construction industry. So, turn off the lights, turn on the disturbing music, now there will be stories from which we still sometimes wake up in a cold sweat.







Ghost office



In one office-office building we made the server room, and all kinds of automation for climate systems, including curtains with drives. There is a weather station on the roof, determines which side the sun is shining on, and covers the curtains if it is too bright. The object was handed over and forgotten, after a while they called, asked:







“Could you please automate the curtains back?” We want to close everything ourselves.

- Why ?!

- Our cleaners are scared. Yes, and we too - it feels like there is a ghost.







This is exactly what the automation specialist’s terrible dream looks like: first, the customer wants automation and energy efficiency, you do it all happily, and then it turns out that the manager likes to manage it himself. And in a room where climate control is fully automated, in the end, everything works in manual mode.







We in the offices of all kinds of tops often make a touch panel with which you can control lighting scenarios, air conditioning, ventilation, curtains. One particularly conservative top said: I don’t want algorithms, I want two buttons: “turn on all” and “turn off all”. A programmer came, cried, removing the standard control interface, drew two buttons instead of him and went home to cry.







Where's the diesel?



On a dark-dark night, in the dark-dark room of technician Oleg, a cell phone rang.







- In our diesel installation, the temperature of the coolant is too high. Now I will send a printscreen.







It was the data center manager, which we surrendered a month ago with a pure soul. He was not confused by either the fact that it was three in the morning in the yard, or the fact that the system showed the same “temperature” in the diesel engine and indoors. Because it was not a temperature at all, but an error code “no connection with the sensor”. Oleg honestly told the dispatcher where he should go at night with such requests. Literally:







- Go to the diesels, look, most likely, the battery just died in the sensor. The battery module in diesel engines feeds this control panel, there is a switch there - if someone of yours has touched it, you need to turn it back.







In general, the dispatcher is just the person who needs to know the object as an experienced milling machine with his three fingers, but then an amazing question was raised:







- And diesel - where are they?

- Go up to the second floor, ask the electricians, they will take you.







For the next 20 minutes, Oleg worked remotely as a navigator, trying to reduce the dispatcher and electricians, who did not really want to arrange someone tours in the middle of the night.







Cycle of Stars in the Data Center



In the distant kingdom, the thirtieth state, we somehow covered ourselves with a chastnik in the chiller . Three hours before certification at the Uptime Institute. It’s a long story to tell what a chiller is and chastotniks, if you don’t know. So just believe: they must work like chimes, otherwise certification will turn into a pumpkin, and the customer will turn into an evil stepmother. And, the most annoying, it will be right, because for the commission’s visit, crazy money is taken, and if something goes wrong, no one will return it.







The vendor’s service worker rushed in, spread his arms and said that the patient was more likely dead, and wait at least a month to wait for a new payment. The commission is on the verge, there are few exits. The first is to unscrew the chastotnik from the “healthy” chiller and put it on the “sick” one, and then change places until the tests are over. This is not cheating, if that: according to the test regulations, anyway, one of the three chillers is reserve, so the scenario is quite working. The second way out is to try to find a new chastotnik for the remaining one and a half hours. We called our refrigerator in Moscow. The cooler scored a friend inside the Russian representative of the vendor. That, in turn, put pressure on the manufacturer’s representative office in Holland and ... after half an hour we already screwed a new board. Certification was a great success.







Imperfections



Whether it is long, short, but at the construction site there always comes a moment when all sub-workers leave and they remain: imperfections. This is about unfinished work, and not about those who left them, if that. Who is the last - he rakes.







We once made a data center in the basement: racks in two rows aligned on one of the edges so that the door opens. Because of this, a gap formed between one row and the wall, and the customer refused to sign the acceptance certificate because of this gap. In the early morning, Techman and the project manager drove to Leroy for polystyrene, paint, fasteners and conscientiously closed the gap to match the racks. Handed over.







And one day after the departure of the heating subcontractors, there was a discrepancy in the project: there should be 7 radiators, and there were 6. They went down, counted - that's right, they really did not reach one radiator in the corridor. It’s too late to drink Borjomi, everything has already been installed and tested. The customer sprinkles ash on his head, because the acts have already been signed. Leroy rescued again - they bought an electric heater there, pulled a group of cables into the corridor and mounted it on Sunday morning, the customer is happy.







Once we were left with magical nothing instead of a fire barrier. In the data center, the ventilation duct from the second floor goes to the basement, through the room, which according to the fire classification belongs to a different category than those above and below it. In practice, this means that there must be a fire-retardant valve in the duct, and a fire barrier around it. There was a hole around our shining duct with a fire-retardant valve, guess who and how closed it? Leroy Merlin did not sponsor this post, but a pity.







Dale Carnegie nervously smokes



Once upon a time, when the grass was greener, and the dollar was 30 with something, we made a data center for one bank in the historical center of Moscow. It was necessary to meet the deadlines. But the streets there are narrow, the openings near the building, too, it’s almost impossible to raise several tons of equipment to the desired floor by stairs. They told the customer that they need to load the crane directly onto the roof, the customer replied in the spirit of "you are a cowboy - you and jump." Well, that is a good idea, and now coordinate the arrival of a 120-ton crane with authorities such as the traffic police. And preferably yesterday. Good luck in your endeavors, do not have time - fine.







The situation is stalemate, time is running out, and we decided to take a chance, in the end, the traffic police fines compared to fines for non-compliance with deadlines are just flowers. On Saturday morning we drove a 16-meter crane, hoping that we will manage to do everything quickly. After a couple of hours, the district police officer arrived and timidly asked permission. Of course, we don’t have it. And it is not known how it would all end if a salesman with extraordinary diplomatic abilities hadn’t been with us.







He took the policeman to the side, explained something to him for 5 minutes, the policeman changed his face several times during this time, but eventually got into his UAZ and shouted that if something happened he would come and help us. What kind of arguments were there, the salesman is still not pricking.







Lift me, people!



Beyond the mountains, beyond the fields, but within the limits of the Third Transport, stood ... no, not a hut, but quite a serious government construction project. A difficult night shift, we ship the equipment. The last truck with a 15-ton piece of iron in the back goes to the only intersection on the territory, something bursts with a loud sound, and the colossus sits down in the mud. 5 am, the construction site comes to life a little, the driver of the concrete mixer behind our truck, remembering the fallen women, is clearly interested: are we willing to get out of the way? He, they say, concrete is getting cold. And we would be happy, because at 7 in the morning one of the many deputies of one of the ministers will come and, if he sees this disgrace, will fly to everyone: from engineers to tops.







Techman runs to the local crane operator, tearfully asks to lift a piece of iron from the body so that we can slip another truck under it. And that one in no way, even for money. Our engineer corrected the situation - agreed through the head of that crane operator. We brought the piece of iron to the right place, but already a little gray.







And then they became a lot of gray, at the same facility. The epicale happened in the most literal sense.







There is such a thing: a telescopic forklift. It is used when there is nowhere to turn around at a construction site, and the load needs to be lifted up and carefully placed. Using it, we needed to unload a 1.5-ton module into a window through a hole. Nothing boded: according to the specification, the car had to withstand 2 tons with a hook. But when exactly half a meter was left to the window, the "forks" at the loader broke off, and the piece of iron rolled from a height. There is nothing to do - we call the manufacturer to come and carry out restoration work. Their servicemen arrived and ... refused to enter the construction site. Because it was necessary to go there along the path, and an excavator stands on the path and digs. We are familiar: I waited 10 seconds while the arrow was turned in the opposite direction from the path, and ran. And the guys were shocked. I had to present this feature as a fascinating attraction, a kind of “Fort Boyard”, and they fixed the module for us.







Instant karma



Soon the fairy tale affects, but soon things are done, especially when it comes to signing acts of acceptance of work performed. We somehow built an excellent data center for one company. But the king-father-customer decided to give us the last test:

- You have written 100 nuts per tray in the specification, and I counted 97. Correct the specifications and estimates or I will not sign anything.







And we went each time to distant lands, and considered with the king-father the choir either fasteners for air ducts, nuts or bolts. Each time it turned out that it was not 97, but 99, etc. And there was no rest for us. As a result, so many internal costs dashed that our bosses could not stand it. They said: let them do what they want - no one else should go there. And left without signatures.







... And after a year, the customer comes by himself and politely asks where would he sign it? It turned out that the Accounts Chamber had come, and he had unaccounted equipment worth seven zeros.







Wingardium Leviosa!



Once upon a time there was one good customer, and he decided to buy an old building for himself under the data center. Only briefly did his joy last: something was wrong that started happening in the battery room. He then called us for help, for a marvelous look and consult. We come to visit, we go into the battery room, and there ... the walls above the floor levitate from three sides. Still: 4 tons of battery just put on the floor - he began to go underground. This is a common problem with batteries: it is important to correctly calculate the load on the structure and provide for unloading frames so that the floors do not form like a house of cards.







But the cherry on the cake of the architectural nightmare in this building was the complete lack of foundation. The walls stood stupidly on a sand screed, under which is not the most friendly soil. They began to think how to save the patient, in the end they proposed a complex system of silicatization: this is when the soil is drilled in several places and a strengthening solution is pumped there. This did not return the floor to its former heights, but at least it did not fail.







The battle of two yokozun



In a certain kingdom, in a certain office state, we did scheduling: for a data center on one of the floors, for climate control - for everything. Dozens of automation cabinets, kilometers of low-current cable!







A feature of that office was its own SPA complex with a sauna. When we were developing the project, it was assumed that the steam room would not be used often. But the engineer assumes, and the customer disposes: the management team was so drawn into the healthy lifestyle that the sauna was just turned off altogether just in case - it takes too long to warm up.







Bottom line: automatic detection detects a rise in temperature from the sauna - turns on the air conditioner harder to compensate. It helps so-so. The air conditioner continues to tear - the atmosphere is heating up, because the automation does not suspect that someone can soar so much. Condey falls into paranoia, decides: “It's not about them, it's about me. All efforts are in vain. It seems I’m broken, ”- which he reports to the dispatcher’s console. The dispatcher sighs and presses the button "task removed." And so every single day.







Flick of the wrist



There are many more stories in our bins, but not all of them fit here. Here are the last three tales: about crooked hands.







The first tale is about how we in one building, in addition to the server, made air supply to the elevator halls and shafts. They did, left the facility, waited until another contractor finished finishing work, returned to test the system. We start the backwater - the false ceiling is inflated sharply, the plates crash out onto the floor and everything is designed in a “loft” style in a second. It turned out that the guys who installed the ceiling forgot to put slabs with perforations for air. They didn’t guess at all about their existence, they simply took it from an open pack - and they didn’t pay attention to the packaging with perforated ones, they didn’t look at the project.







The second epos is also about the ceiling. In one data center there was a rather narrow corridor, and the air exchange in the room with the battery racks was supposed to be not weak, so there were communications along the entire ceiling. Good fellows, who made false ceilings, did not bother and ... screwed the suspension fasteners directly into our unfortunate ducts. Any air duct vibrates a little while the ventilation units are working, and when there are a lot of holes and irregularities in it, the sounds of the Apocalypse are provided to you. We weighed the glorious heroes-installers then of such life-giving cradles that the wounds on the air ducts miraculously healed. At their expense, of course.







The third tale happened when we did automation for a process air conditioning system. We handed over the temperature sensor to the customer’s contractor and, without a second thought, asked to put it on the air heater in the supply. In air handling units there is always a heater for outdoor air: either water or electric. This wonderful man fulfilled everything unquestioningly. That is, he took and screwed the self-tapping screw directly into the water heater tube, with all the consequences (literally) from here. It is good that there were air conditioners with MAPP gas at the facility - the flow was quickly soldered.







References:






All Articles