SOC is people: the harsh everyday life of a service manager of a center for monitoring and response to cyber attacks

Talking about how the work of the center for monitoring and responding to cyber attacks (SOC) from the inside out, we have already talked about the first and second line engineers and about the analysts . Then we casually mentioned service managers. These are SOC employees who are responsible to the customer for the quality of the services provided. This short definition actually hides the following: the service manager determines the practical implementation of the service on the customer’s site, should be ready at any time to answer the customer’s call or notification of the monitoring engineer about a critical incident, assemble a response or investigation team and go to the site.



For the most part, Solar JSOC service managers are men over 30 who have seen species with diverse experience in information security: from designing information security systems to auditing processes. Be sure to have the skills of translation from bird to human technical to business and vice versa.

What and to whom does the service manager owe

The service manager does not require in-depth knowledge of specific information protection tools at the level of an operating engineer. But there is a broad outlook in the field of SPI and their manufacturers, minimal knowledge of network protocols and requirements for specific information objects such as AWS KBR, etc. - required. And as a mandatory requirement - an established and rooted aka baobab understanding that it is necessary to protect, first of all, not hosts, but business processes. In theory, everyone knows about this, but in practice, not everyone is able to follow this principle.



Why such a set of criteria? Service manager is a single window for the client. Together with the analyst they form a group that directly communicates with the customer. A sort of front office in the form as we imagine it. It is the service manager that determines how customer tasks will be solved at the application level. Such particulars, as a rule, are not spelled out in the contract.



It’s good if the client has a clear understanding of what he wants to get from the service (on this line my colleagues grinned). In reality, this phenomenon is almost as rare as meteor shower. And here just can not do without a person who ensures the development of information security in the framework of the project. For each specific client, he identifies critical data and processes, as well as means of their automation; potential points of compromise at the level of infrastructure and organization of the interaction of systems and people. And this is only at the stage of launching the service. And then work days come, during which the service manager constantly keeps his finger on the pulse of the customer’s infrastructure. It is necessary to remember and take into account a bunch of factors: the type of business, the network equipment and SZI used, the number and types of subcontractors, levels of access to sensitive information and much, much more.



A perfectly logical question: why is it so difficult? After all, you can connect customer’s systems to SOC as information sources and stop here. Probably someone does, but our experience shows that what is suitable for a bank is not applicable to a plant or a leasing company. Yes, and inside banks there is no identical approach to security, despite the fact that this is one of the most overregulated industries in our country. And factories in general have a quiet horror: a bunch of proprietary protocols, a prohibitive number of subcontractors connecting remotely to infrastructure elements (often without revealing this fact to the customer). And with all this knowledge it is necessary to work.



As mentioned above, under each contract we form a team of analysts and managers who are front-line. Teams are not permanent and vary from client to client. As a rule, one service manager serves from three to six customers, depending on their contract size and the level of IS maturity. And these are three or six different infrastructures, various faiths of approaches to information security and other “charms” of species diversity. In fairness, it’s worth saying that we have a number of large clients who have been allocated a personal service manager who works only with them (but this is not easier for him).



An individual approach to the customer is not big words, but one of the tasks in providing the service. Even different companies of the same infrastructure industry, approaches to security, information security policies and business processes are so different that, no matter how we want to unify the service, in reality this approach will lead to profanity and a decrease in the level of customer protection (in other words, it will be hacky Job). At the same time, it is still necessary to find a balance between the wishes of the customer, often strange, and the real usefulness of one or another action.



For example, there is an isolated WiFi guest segment without access to internal resources, but the customer wants that, if we fix the launch of the RAT (Remote Access Tool), he will receive a notification with the maximum level of criticality. However, in fact, upon receipt of such a notification, the customer does nothing, because he does not have playbooks and does not have enough resources to respond. And the satisfaction of such desires increases the burden on response engineers and does not benefit any of the participants in the process. As a result, we get nothing but increased "socialist obligations", i.e. High incident response and investigation process does not work.



Or vice versa: the customer, due to some obscure superstitions, does not want to monitor its billing system (which is the main business system and, in general, CII). And we have to methodically, in fact, explain with our fingers why we should protect this segment. Each of our SMs has a ton of similar stories, and if you publish them, you get a good book like that.



And on the other side of the coin are Solar JSOC processes: directly monitoring and identifying incidents, analytics, architecture, forensics, etc. etc. All this rather big and motley company works for customers. As in any living collective, it has its own development vectors, preferences, and personal communications. And this should not affect the provision of service. This is also SM’s headache: it is necessary to redistribute resources to solve a problem, prioritize implementation so that it does not affect other customers. A boring lesson is obtained.

Sleep is for dweebs

Unlike most Solar JSOC employees, the service manager “works around the clock": without breaks for the night, day off and lunch. All other services have attendants. This does not mean that the service manager, like a galley slave, is confined to the modern version of the oars - a table and a computer. It’s just that person who must answer the call from the customer or our internal services at any time of the day or night. In our understanding, the term “answer” conceals the following sequence of actions (we are for the process approach): to recognize the question / problem, make a decision on further actions and connect the required services within the company, monitoring the result.



The reasons for calls can be different - funny and not very. The most common and “favorite” reason for SMs is launched on the side of forensers. They find a new exploit, evaluate it in terms of possible threats and launch it into service managers (as it was recently with BlueKeep-2).

And then - a disco! Each of the SMs, just in case, looks at the dossiers of clients (although the majority remembers the infrastructure by heart), the head of these wonderful guys generates an alert text that is sent to customers through all available channels.



With night calls, there is another common story. At the start of the contract, the customer often asks that the incidents in a number of scenarios of the responsible persons be notified by voice at any time of the day or night. Accordingly, when an alert is triggered, the monitoring duty officer wakes up the SM and gives him all the necessary information on the incident. Then the SM wakes up the responsible person on the part of the customer and transfers the information to him already. Even if the client has his own response service, working around the clock, this does not prevent us from raising the responsible person out of bed. Calls occur in parallel with a notification to the customer about the incident. As a rule, after a couple of months, when the customer is convinced that the service is really round-the-clock and he pays for it in vain, we are asked to stop such a practice.



But there are serious reasons not to sleep at night. These clearly include an attack on the customer’s infrastructure. Such a rare but not exceptional situation also happens: a call rings at night, and they ask for assistance on the physical site from the handset. Over the years of the Solar JSOC's existence, the scheme has been run-in: “at the pace of a waltz” a team is assembled, in which the SM acts as a coordinator, and amicably drops to the client. Sometimes it happens that we drive up before the responsible persons who launched this chain.

Reports - for the strong in spirit

In addition to a sleepless night in such situations, there is another huge minus: all participants in the process can already get enough sleep, and the service manager needs to write a preliminary report on the situation indicating the timing of the completed actions, describe the actions themselves and their results. Reporting is not a tribute to formalities, but a real document, which is then disassembled to identify errors or, conversely, successful decisions. Absolutely all experience of all Solar JSOC specialists is taken into account, regardless of the position in which they work.



In general, reporting is an integral part of the work of a service manager. There are a lot of reports. No not like this. There are a lot of them. For every taste and color: from the minutes of meetings at which the further development of the service is recorded, to regular reporting to customers. Very often, reports are accompanied by presentations for top management, who wants to know about the money spent, but is not ready to delve into the particular service. Accordingly, the presentation should intelligibly confirm that the amount spent is not in vain and that the service is really useful. And all this is done absolutely for all customers by the hands of men over the age of 30. Yes, there is a certain level of automation, but we have not yet learned how to create presentations in automatic mode.





In general, a good SM can work with any audience: lucidity is our everything

But the main thing is different

Very often you can hear that a service manager is a shooting position and infernal work without the possibility of development. This is a very strong fallacy. One of our SMs says that "the result of the work is always visible, that is, you do not work in the trash can, and this is always cool."



We will talk about where the service manager is developing and how to grow a full-fledged CISO from it. So far, only - workdays through the keyhole.