PDU and All-All-All: Rack Power Distribution

One of the racks of internal virtualization. They got confused with the color indication of the cables: orange indicates an odd input for power, green means even.



Here we most often talk about the “krupnyak” - chillers, diesel generator sets, main switchboards. Today we will talk about the “little things” - sockets in racks, they are also the Power Distribution Unit (PDU). In our data centers there are more than 4 thousand racks clogged with IT equipment, so in practice I saw a lot of things: classic PDUs, “smart” ones with monitoring and control, ordinary socket blocks. Today I’ll tell you which PDUs are available and what is better to choose in a particular situation.

What are PDUs

A simple block of outlets. Yes, the one who lives at every home or office.

Formally, this is not exactly a PDU in the sense of industrial use in racks with IT equipment, but these devices also have their fans. The only plus of this solution is low cost (the cost starts from 2 thousand rubles). They can also help out if you use open racks, where you can’t push a standard PDU in any way, and you don’t want to lose units for a horizontal PDU. This is again to the question of saving.



There are much more minuses: such devices do not always have internal protection against short-circuit and overload, monitor performance and even more so do not manage to manage sockets. Most often they will be placed at the bottom of the rack. This is not the most convenient position for sockets for disconnecting equipment.



In general, “pilots” can be used if:

• you have thousands of servers and you need to save
• you can afford to blindly connect equipment without understanding what is happening with real consumption,
• ready for downtime equipment.

We do not use this, but we have clients who practice this quite successfully. True, they build the infrastructure for their services in such a way that the failure of dozens of servers does not affect the performance of the client application.





Cheap and cheerful.





Vertical placement.



Dumb PDUs. Actually, this is a classic PDU for use in racks with IT equipment, and this is already good. They have the appropriate form factor for placement on the sides of the rack, due to which it is convenient to connect equipment to them. There is internal protection. Such PDUs do not have monitoring, which means we won’t know what equipment consumes how much, and what is going on inside. We have almost no such PDUs left, and on the whole they are gradually leaving mass use.



Such PDUs cost from 25 thousand rubles.







Smart PDUs with monitoring. These devices have brains, and they are able to track energy consumption parameters. There is a display where the main indicators are displayed: voltage, current and power. You can track them by separate groups of outlets: sections or banks. You can connect to such a PDU remotely, configure the sending of data to the monitoring system. They write logs by which you can see everything that happened to her, for example, when the PDU turned off.



They also know how to calculate the consumption (kW * h) for technical accounting in order to understand how much the rack consumes in a certain amount of time.



These are standard PDUs that we offer our customers for rent, and most of these PDUs are in our data centers.



If you buy, get ready to pay from 75 thousand rubles apiece.













Graph from our internal PDU monitoring.



Smart PDUs with control. These skills are supplemented with controls for the above skills. The coolest PDUs control and monitor each outlet: you can turn on / off, which happens in situations where there is a task to remotely restart the server on power. This is the charm and danger of such PDUs: an ordinary user can unknowingly go into the web interface, press something and restart / shut down the whole system in one fell swoop. Yes, the system will warn twice about the consequences, but practice shows that even alarms do not always protect against rash user actions.



The big problem with smart PDUs is overheating and controller and display failure. PDUs are usually mounted on the back of the rack where hot air is blown. It's hot there, and the controllers can't stand it. In this case, the PDU does not need to be changed entirely, the controller can be changed to hot.



Well, at a cost quite a bite - from 120 thousand rubles.





Controlled PDUs can be identified by the indications under each outlet.



In my opinion, the management function in the PDU is a matter of taste, but monitoring is a must have. Otherwise, it will not be possible to track consumption and load. Why this is important, I will tell a little later.

How to calculate the required PDU power?

At first glance, everything is quite simple: the power of the PDU is selected in accordance with the power of the rack, but there are nuances. Let's say you need a 10 kW rack. PDU manufacturers offer 3, 7, 11, 22 kW models. Choose 11 kW, and, unfortunately, you will be wrong. We will have to choose 22 kW. Why do we need such a large supply. I’ll explain everything now.



Firstly, manufacturers often indicate the power of PDUs in kilowatts rather than kilovolt-amperes, which is more correct, but not obvious to the average person.

Sometimes manufacturers themselves add additional confusion:



Here it first says about 11 kW,







And in the detailed description we are talking about 11000 VA:







If you are dealing with dummies and similar consumers, then there will be no difference between kW and kVA. A 10 kW rack with kettles will consume 10 kVA. But if we have IT equipment, then the coefficient (cos φ) appears there: the newer the equipment, the closer this coefficient to unity. The average hospital price for IT equipment is 0.93–0.95. Therefore, a 10 kW rack with IT will consume 10.7 kVA. Here is the formula by which we received 10.7 kVA.



Pfull = Pact./ Cos (φ)

10 / 0.93 = 10.7 kVA



Well, you ask a reasonable question, 10.7 is less than 11. Why do we need a 22 kW remote control? There is a second point: the level of power consumption of the equipment will vary depending on the time of day, day of the week. When distributing power, this moment must be taken into account and ~ 10% should be set aside for fluctuations and surges, so that at the time of increasing PDU consumption they do not go into overload and do not leave equipment without power.





Schedule of rack consumption 10 kW for 4 days.



It turns out that we must add another 10% to our 10.7 kW, and as a result, the remote control for 11 kW is no longer suitable for us.





A fragment of the power table of specific PDU models according to DataLine. Taking into account the conversion from kVA to kW and the reserve for horse racing during the day.

Mounting Features

It is most convenient to work with the PDU when it is mounted vertically, on the left and right of the rack. In this case, it does not occupy usable space. Regularly, up to four PDUs can be installed in the rack - two on the left and two on the right. Most often, they put one PDU on each side. Each PDU receives one power input.





The standard “kit” racks - 2 PDU and 1 ATS.



Sometimes in a rack there is no place for vertical PDUs, for example, if it is an open rack. Then horizontal PDUs come to the rescue. The only thing is that in this case you will have to accept the loss of 2 to 4 units in the rack, depending on the PDU model.





Here the PDU ate 4 units. This type of PDU is also used when you need to distinguish between two clients in the same rack. In this case, each client will have a separate pair of PDUs.



It happens that the rack is not selected deep enough, and the server sticks out, blocking the PDU. Here the saddest thing is not that some of the outlets will be idle, but that if such a PDU breaks, you will have to bury it right in the rack, or disconnect and pull out all the interfering equipment.





Do not do so - 1.





Do not do this - 2.

Equipment connection

Even the most sophisticated PDU will not help if the equipment is not connected correctly and there is no way to monitor consumption.



What could go wrong? A bit of equipment . Two power inputs come to each rack, two PDUs in a standard rack. It turns out that each PDU has its own input. If something happens to one of the inputs (read the PDU), then the rack continues to live on the second. For this scheme to work, you need to follow some rules. Here are the main ones (a complete list can be found here ):



Equipment must be connected to different PDUs. If the equipment has one power supply unit and one plug, then it is connected to the PDU via ATS (automatic transfer switch), or ATS (Automatic Transfer Switch). In case of problems with one of the inputs or the PDU itself, the ATS switches the equipment to a healthy PDU / input. The equipment will not feel anything.



Paired load on two inputs / PDUs. The backup input will save only if it can withstand the load of a fallen input. To do this, you need to leave a margin: load each input less than half of the rated power, and the total load on two inputs was less than 100% of the nominal. Only in this case, the remaining input will withstand double load. If you do not, then the focus with switching to the reserve will not take place - the equipment will remain without power. To prevent the worst, we monitor this parameter.



Load balancing between PDU sections. PDU sockets are grouped in sections. Usually 2 or 3 pieces. Each section has its own power limit. It is important not to exceed it and to distribute the load evenly across all sections. Well, the story with paired loads, which was discussed above, also works here.



To summarize

1. If possible, choose a PDU with a monitoring function.
2. When choosing a PDU model, leave a margin of power.
3. Mount the PDU so that you can replace it without disturbing IT equipment.
4. Connect correctly: connect the equipment to two PDUs, do not overload the sections and remember the pair loads.