Why do you need DevOps and who are DevOps specialists
When the application does not work, least of all I want to hear the phrase "the problem is on your side" from colleagues. As a result, users suffer - and they do not care which part of the team is responsible for the breakdown. DevOps culture emerged just to bring development and support together and unite them around common responsibility for the final product.
What practices are included in the concept of DevOps and why are they needed? What do DevOps engineers do and what should they be able to do? Experts from EPAM answer these and other questions: Kirill Sergeyev, systems engineer and DevOps evangelist, and Igor Boyko, leading systems engineer and coordinator of one of the company's DevOps teams.
Previously, there was a barrier between developers and support (the so-called operations). It sounds paradoxical, but they had different goals and KPIs, although they did a common thing. The development goal was to implement business requirements as quickly as possible and add them to a working product. Support was responsible for ensuring that the application worked stably - and any changes put stability at risk. There is a conflict of interest - DevOps has appeared to solve it.
The question is good - and controversial: finally, the world has not yet agreed on this. EPAM believes that DevOps combines technology, processes and a culture of interaction within the team. This association aims at the continuous delivery of values to end users.
Kirill Sergeyev : “Developers write the code, testers check it, and administrators install the final product on the production environment. For a long time, these parts of the team were somewhat fragmented, and then the idea came to combine them into a common process. That is how DevOps practices came about. ”
The day came when developers and system engineers became interested in each other's work. The barrier between production and support began to fade. This is how DevOps came into being, which includes practices, culture and teamwork.
The fact that the responsibility for the final result lies with each of the team members. The most interesting and difficult thing in DevOps philosophy is to understand that a particular person is not just responsible for his stage of work, but is responsible for how the whole product will work. The problem does not lie on any side - it is common, and each member of the team helps to solve it.
The most important point of the DevOps culture is to solve the problem, and not just apply DevOps practice. Moreover, these practices are not implemented “on someone’s side”, but in the whole product. The project does not need a DevOps engineer in itself - it needs a solution to the problem, and the role of a DevOps engineer can be distributed among several team members with different specializations.
DevOps practices cover all stages of the software life cycle.
Igor Boyko : “The ideal case is when we start using DevOps practices right at the initiation of the project. Together with architects, we plan what kind of application the architectural landscape will have, where it will be located and how to scale, we choose the platform. Now microservice architecture is in fashion - for it we choose an orchestration system: we need to be able to manage each element of the application individually and update it independently of others. Another practice is “infrastructure as code.” This is the name of the approach in which the project infrastructure is created and managed using code, rather than through direct interaction with servers.
Next, we move on to the development phase. Here one of the biggest practices is building CI / CD: you need to help developers integrate changes into the product quickly, in small portions, more often and more painlessly. CI / CD covers code verification, filling the wizard into the code base, and deploying the application to test and production environments.
In the CI / CD stages, the code goes through quality gates. With their help, they verify that the code that came out of the developer's workstation meets the specified quality criteria. Unit and UI testing is added here. For a quick, painless and focused product deployment, you can choose the appropriate type of deployment.
DevOps practices have a place at the stage of supporting the finished product. They are used for monitoring, feedback, security, implementation of changes. DevOps looks at all these tasks in terms of continuous improvements. We minimize repetitive operations, automate them. This also includes migrations, application expansion, and health support. ”
If we wrote a textbook on modern DevOps practices, there would be three points on its first page: automation, accelerated release, and quick user feedback.
Kirill Sergeev : “The first is automation. We can automate all interactions in the team: we wrote the code - rolled out - checked - installed - collected feedback, returned to the beginning. All this is automatic.
The second is to speed up release and even simplify development. It is always important for the customer that the product enter the market as soon as possible and begin to benefit sooner than the competitors ’analogues. The product delivery process can be infinitely improved: reduce time, add additional control marks, improve monitoring.
Third is speeding up user feedback. If he has comments, we can immediately make corrections and immediately update the application. ”
They intersect, but belong to slightly different areas.
Systems Engineer at EPAM is a position. They come in different levels: from junior to chief-specialist.
A building engineer is more likely a role that can be performed on a project. Now this is the name of the people responsible for CI / CD.
A DevOps engineer is a specialist who implements DevOps practices on a project.
To summarize all this, it turns out something like this: a person in the position of a system engineer performs the role of a build engineer on the project and is engaged in the implementation of DevOps practices there.
DevOps engineers put together all the parts that make up the project. They know the specifics of the work of programmers, testers, system administrators and help to simplify their work. They understand the needs and requirements of the business, its role in the development process - and build the process taking into account the interests of the customer.
We talked a lot about automation - DevOps engineers do it first. This is a very large point, which, among other things, includes the preparation of the environment.
Kirill Sergeyev : “Before introducing updates into the product, they need to be tested on a third-party environment. It is being prepared by DevOps engineers. They instill the DevOps culture as a whole on the project: they introduce DevOps practices on all layers of their projects. These three principles: automation, simplification, acceleration - they bring everywhere where they can reach. "
By and large, he should have knowledge from different areas: programming, working with operating systems, databases, build systems and configurations. To them is added the ability to work with cloud infrastructure, orchestration systems, and monitoring.
1. Programming languages
DevOps engineers know several basic languages for automation and can, for example, tell the programmer: “Come on, you will not install the code manually, but with the help of our script that automates everything? We will prepare a config-file for it, it will be convenient to read it for you and us - and we can change it at any time. And we will see who, when and why makes changes to it. ”
A DevOps engineer can learn one or more of these languages: Python, Groovy, Bash, Powershell, Ruby, Go. Knowing them at a deep level is not required - just the basics of syntax, the principles of OOP, the ability to write simple scripts for automation.
2. Operating systems
DevOps engineer must understand on which server the product will be installed, in which environment it will be launched, with which services it will interact. You can choose to specialize in the Windows or Linux family.
3. Version control systems
Without the knowledge of the version control system, the DevOps engineer is nowhere. Git is one of the most popular systems at the moment.
4. Cloud providers
AWS, Google, Azure - especially if we are talking about the Windows-direction.
Kirill Sergeyev : “Cloud providers provide us with virtual servers that fit perfectly on CI / CD tracks.
Installing ten physical servers requires about a hundred manual operations. Each server must be manually started, installed and configured the necessary operating system, install our application on these ten servers, and then double-check everything ten times. Cloud services replace this procedure with ten lines of code, and a good DevOps engineer should be able to handle them. So he saves time, effort and money - both for the customer and for the company. ”
5. Orchestration systems: Docker and Kubernetes
Kirill Sergeyev : “Virtual servers are divided into containers, in each of which we can install our application. When there are a lot of containers, you need to manage them: one to turn on, the other to turn off, somewhere to make backups. This is becoming a rather complicated affair for which an orchestration system is needed.
Previously, each application was engaged in a separate server - any changes in its operation could affect the health of the application. Thanks to containers, applications become isolated and run separately - each on its own virtual machine. If a failure occurs, there is no need to waste time searching for the cause. It’s easier to destroy the old container and add a new one. ”
6. Configuration systems: Chef, Ansible, Puppet
When you need to maintain a fleet of servers, you have to do a lot of the same operations. It is long and difficult, and even manual work increases the chance of error. This is where configuration systems come to the rescue. With their help, they create a script that is convenient to read by programmers, DevOps engineers, and system administrators. This script helps to carry out the same operations on servers automatically. So manual operations (and therefore errors) become less.
You can develop both horizontally and vertically.
Igor Boyko : “From the point of view of horizontal development, DevOps engineers now have the widest prospects. Everything is constantly changing, and you can build up skills in a variety of ways: from version control systems to monitoring, from configuration management to databases.
You can become a system architect if an employee is interested in understanding how the application works at all stages of its life cycle - from development to support. ”
What practices are included in the concept of DevOps and why are they needed? What do DevOps engineers do and what should they be able to do? Experts from EPAM answer these and other questions: Kirill Sergeyev, systems engineer and DevOps evangelist, and Igor Boyko, leading systems engineer and coordinator of one of the company's DevOps teams.
Why do I need DevOps?
Previously, there was a barrier between developers and support (the so-called operations). It sounds paradoxical, but they had different goals and KPIs, although they did a common thing. The development goal was to implement business requirements as quickly as possible and add them to a working product. Support was responsible for ensuring that the application worked stably - and any changes put stability at risk. There is a conflict of interest - DevOps has appeared to solve it.
What is DevOps?
The question is good - and controversial: finally, the world has not yet agreed on this. EPAM believes that DevOps combines technology, processes and a culture of interaction within the team. This association aims at the continuous delivery of values to end users.
Kirill Sergeyev : “Developers write the code, testers check it, and administrators install the final product on the production environment. For a long time, these parts of the team were somewhat fragmented, and then the idea came to combine them into a common process. That is how DevOps practices came about. ”
The day came when developers and system engineers became interested in each other's work. The barrier between production and support began to fade. This is how DevOps came into being, which includes practices, culture and teamwork.
What is the essence of DevOps culture?
The fact that the responsibility for the final result lies with each of the team members. The most interesting and difficult thing in DevOps philosophy is to understand that a particular person is not just responsible for his stage of work, but is responsible for how the whole product will work. The problem does not lie on any side - it is common, and each member of the team helps to solve it.
The most important point of the DevOps culture is to solve the problem, and not just apply DevOps practice. Moreover, these practices are not implemented “on someone’s side”, but in the whole product. The project does not need a DevOps engineer in itself - it needs a solution to the problem, and the role of a DevOps engineer can be distributed among several team members with different specializations.
What are DevOps practices?
DevOps practices cover all stages of the software life cycle.
Igor Boyko : “The ideal case is when we start using DevOps practices right at the initiation of the project. Together with architects, we plan what kind of application the architectural landscape will have, where it will be located and how to scale, we choose the platform. Now microservice architecture is in fashion - for it we choose an orchestration system: we need to be able to manage each element of the application individually and update it independently of others. Another practice is “infrastructure as code.” This is the name of the approach in which the project infrastructure is created and managed using code, rather than through direct interaction with servers.
Next, we move on to the development phase. Here one of the biggest practices is building CI / CD: you need to help developers integrate changes into the product quickly, in small portions, more often and more painlessly. CI / CD covers code verification, filling the wizard into the code base, and deploying the application to test and production environments.
In the CI / CD stages, the code goes through quality gates. With their help, they verify that the code that came out of the developer's workstation meets the specified quality criteria. Unit and UI testing is added here. For a quick, painless and focused product deployment, you can choose the appropriate type of deployment.
DevOps practices have a place at the stage of supporting the finished product. They are used for monitoring, feedback, security, implementation of changes. DevOps looks at all these tasks in terms of continuous improvements. We minimize repetitive operations, automate them. This also includes migrations, application expansion, and health support. ”
Why are DevOps practices useful?
If we wrote a textbook on modern DevOps practices, there would be three points on its first page: automation, accelerated release, and quick user feedback.
Kirill Sergeev : “The first is automation. We can automate all interactions in the team: we wrote the code - rolled out - checked - installed - collected feedback, returned to the beginning. All this is automatic.
The second is to speed up release and even simplify development. It is always important for the customer that the product enter the market as soon as possible and begin to benefit sooner than the competitors ’analogues. The product delivery process can be infinitely improved: reduce time, add additional control marks, improve monitoring.
Third is speeding up user feedback. If he has comments, we can immediately make corrections and immediately update the application. ”
How do the concepts “system engineer”, “build engineer” and “DevOps engineer” relate?
They intersect, but belong to slightly different areas.
Systems Engineer at EPAM is a position. They come in different levels: from junior to chief-specialist.
A building engineer is more likely a role that can be performed on a project. Now this is the name of the people responsible for CI / CD.
A DevOps engineer is a specialist who implements DevOps practices on a project.
To summarize all this, it turns out something like this: a person in the position of a system engineer performs the role of a build engineer on the project and is engaged in the implementation of DevOps practices there.
What exactly is a DevOps engineer doing?
DevOps engineers put together all the parts that make up the project. They know the specifics of the work of programmers, testers, system administrators and help to simplify their work. They understand the needs and requirements of the business, its role in the development process - and build the process taking into account the interests of the customer.
We talked a lot about automation - DevOps engineers do it first. This is a very large point, which, among other things, includes the preparation of the environment.
Kirill Sergeyev : “Before introducing updates into the product, they need to be tested on a third-party environment. It is being prepared by DevOps engineers. They instill the DevOps culture as a whole on the project: they introduce DevOps practices on all layers of their projects. These three principles: automation, simplification, acceleration - they bring everywhere where they can reach. "
What should a DevOps engineer know?
By and large, he should have knowledge from different areas: programming, working with operating systems, databases, build systems and configurations. To them is added the ability to work with cloud infrastructure, orchestration systems, and monitoring.
1. Programming languages
DevOps engineers know several basic languages for automation and can, for example, tell the programmer: “Come on, you will not install the code manually, but with the help of our script that automates everything? We will prepare a config-file for it, it will be convenient to read it for you and us - and we can change it at any time. And we will see who, when and why makes changes to it. ”
A DevOps engineer can learn one or more of these languages: Python, Groovy, Bash, Powershell, Ruby, Go. Knowing them at a deep level is not required - just the basics of syntax, the principles of OOP, the ability to write simple scripts for automation.
2. Operating systems
DevOps engineer must understand on which server the product will be installed, in which environment it will be launched, with which services it will interact. You can choose to specialize in the Windows or Linux family.
3. Version control systems
Without the knowledge of the version control system, the DevOps engineer is nowhere. Git is one of the most popular systems at the moment.
4. Cloud providers
AWS, Google, Azure - especially if we are talking about the Windows-direction.
Kirill Sergeyev : “Cloud providers provide us with virtual servers that fit perfectly on CI / CD tracks.
Installing ten physical servers requires about a hundred manual operations. Each server must be manually started, installed and configured the necessary operating system, install our application on these ten servers, and then double-check everything ten times. Cloud services replace this procedure with ten lines of code, and a good DevOps engineer should be able to handle them. So he saves time, effort and money - both for the customer and for the company. ”
5. Orchestration systems: Docker and Kubernetes
Kirill Sergeyev : “Virtual servers are divided into containers, in each of which we can install our application. When there are a lot of containers, you need to manage them: one to turn on, the other to turn off, somewhere to make backups. This is becoming a rather complicated affair for which an orchestration system is needed.
Previously, each application was engaged in a separate server - any changes in its operation could affect the health of the application. Thanks to containers, applications become isolated and run separately - each on its own virtual machine. If a failure occurs, there is no need to waste time searching for the cause. It’s easier to destroy the old container and add a new one. ”
6. Configuration systems: Chef, Ansible, Puppet
When you need to maintain a fleet of servers, you have to do a lot of the same operations. It is long and difficult, and even manual work increases the chance of error. This is where configuration systems come to the rescue. With their help, they create a script that is convenient to read by programmers, DevOps engineers, and system administrators. This script helps to carry out the same operations on servers automatically. So manual operations (and therefore errors) become less.
What career can a DevOps engineer build?
You can develop both horizontally and vertically.
Igor Boyko : “From the point of view of horizontal development, DevOps engineers now have the widest prospects. Everything is constantly changing, and you can build up skills in a variety of ways: from version control systems to monitoring, from configuration management to databases.
You can become a system architect if an employee is interested in understanding how the application works at all stages of its life cycle - from development to support. ”
How to become a DevOps engineer?
- Read the Phoenix Project books and DevOps Handbook. These are the true pillars of DevOps philosophy, the first being an artistic novel.
- Learn the technology from the list above: on your own or online courses.
- Join as a DevOps engineer for an open source project.
- Practice and offer DevOps practices on your personal and work projects.
All Articles