This is the third article in a series of publications on Kubernetes and service mesh technology (also known as the “microservice network” and the “microservice mesh network”). In a previous article, we learned the basics of working with Istio and found out how this tool helps set up and administer complex cloud architectures. So, with its help, you can configure the mesh-network of microservices and get some centralization capabilities in a distributed microservice environment. Today we’ll take a closer look at Istio’s features to appreciate the benefits of service mesh technology.
Istio is a powerful but quite sophisticated tool. Building a scalable mesh network of microservices that can handle heavy loads can be difficult. We hope that after reading this article you will better understand what the complexity of mesh networks is and what tools help to correctly configure them. Although in most cases you do not have to manually configure sidecar proxies or configure network interactions, having learned how to perform these steps, you will understand how the service mesh works. This is a very difficult topic. The deeper you know the capabilities of the tools, the better.
Therefore, today we will study in detail the four main components of Istio and their functions: traffic management (Envoy), control plane (Pilot), telemetry component (Mixer) and security component (Citadel). We will successively consider each of them and their role in the mesh network of microservices.
Envoy
You have probably already read about sidecar proxies in our latest publication on Istio and you know that they are added to each service definition and deployed in the same submission as the service itself. For us, the terms Envoy and sidecar are actually synonyms, although sidecar proxies can work with various plug-ins.
Sidecar proxies act as the main network gateways for the traffic of specific services that you define: the proxy receives incoming traffic directed to a particular service and routes it in accordance with the rules and policies specified in the general configuration.
Sidecar proxies are needed to process control data from two sources. The first of these is the user, or rather, the configuration deployed by him in the mesh network. Information about its modifications, for example, about changing load balancing parameters, adding new nodes, services, and network routing data, is transmitted to the Pilot component, which acts as the main source of information about the state of the application. Sidecar proxies periodically check with Pilot, get the latest configuration data and make the necessary changes to local rules.
The second source of management data is the applications to which sidecar proxies are connected. Envoy acts as a load balancer, constantly monitoring the state of the instances connected to it and sending requests to check their activity. The component monitors key indicators, such as response time, and makes sure that requests are being processed. Envoy removes bad instances from the pool so that damaged deployments or server errors do not cause the entire service to fail.
What other benefits do sidecar proxies provide? In addition to the built-in capabilities of load balancing and checking the status of instances, they allow you to configure traffic so that you can get an idea of the application. So, testing new versions with significant code changes at the development stage does not always help to form a detailed picture. In such cases, it is useful to direct a small amount of work traffic to the instance executing the new code in order to observe its behavior in real conditions.
Envoy allows you to create configurations that distribute the load between different versions. For example, you can start by redirecting only 5% of the traffic to new instances. As soon as the modified configuration data is transferred to the Pilot component, the load balancing in the sidecar proxy will change. Initially, the new service will receive a small amount of traffic, and you will be able to collect and process experimental data. Then it will be possible to gradually increase the volume from 5 to 100% and, if necessary, reduce it, until you are sure that the new version is working properly.
Sidecar proxy configuration
What does this configuration look like in practice? To redirect traffic to various target locations, you must set the weight parameter in the service configuration, for example, like this:
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: helloworld spec: hosts: - hello1 http: - route: - destination: host: hello1 subset: v1 weight: 75 - destination: host: hello1 subset: v2 weight: 25 --- apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: helloworld spec: host: hello1 subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2
We have one host, hello1, with two versions of the target location - v1 and v2. First, we assign 75% of the traffic for v1, and the remaining 25% for v2. After implementing the configuration, you can check the status by waiting for a predefined time, and change the parameters again.
In this way, Envoy provides excellent traffic management capabilities for specific targeted locations. If your application is characterized by traffic spikes that overload the equipment, you can inject delays using the
fault
parameter:
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: helloworld spec: hosts: - hello1 http: - fault: delay: percent: 10 fixedDelay: 3s route: - destination: host: hello1
Pilot
The main component that interacts with Envoy is Pilot , which centrally manages all Envoy instances in a mesh network. Pilot acts as a single source of environmental data. This is the local repository of all the rules for services and their interactions. It also contains information about the operations performed. Pilot is often identified with the Istio control plane, as it is responsible for administering and configuring all Envoy proxies (although technically other components like Citadel also belong to the control plane).
Of course, Pilot plays an important role in understanding how the service mesh works. But in practice, most of the operations associated with this component are performed by Envoy proxies. They periodically check with Pilot to get the latest configuration data, and are regularly updated. Any change to the mesh network configuration involves interaction with the control plane. This separation of tasks is a key principle of Istio: Envoy proxies interact with Pilot to create the services that make up the application. Knowledge of the interconnections between service mesh components plays a crucial role in understanding the essence of this technology.
In the first article, we talked about the benefits of microservice network abstraction, which allows the use of high-level commands to determine the interactions between services. The technology transforms these commands into exact configurations for services that it controls. Pilot is the component responsible for these key functions. As input, it receives service discovery rules and generates certain actions that are performed by Envoy proxies (or any other sidecar proxies compatible with the Envoy API).
Pilot component test drive
The best way to learn more about Pilot instances in a mesh network is to read the sidecar proxy status data. The list of services and the Pilot identifier that controls them can be obtained by running the following command:
istioctl proxy-status
A list of services and their identifiers is displayed with the corresponding Pilot ID and current synchronization status. Sidecar proxies with Synced or Synced status (100%) have been updated and received the latest configurations from the Pilot component. The status Not Sent means that the configuration has not changed recently, so there is nothing to synchronize. And the Stale status means that the sidecar proxy does not respond to changes.
If you encounter problems with synchronization, you can specify the identifier of a specific service in the proxy status command and view information about synchronization errors between Pilot and sidecar. For example, if the above list of services includes the identifier
hello-world-v2–5aa3f7abd-ppz2n.default
, you can run the command:
istioctl proxy-status hello-world-v2-686a3b641-4r52s.default
The output you get is the following:
Clusters Match Listeners Match Routes Match (RDS last loaded at Fri, 24 May 2019 20:22:14 UTC)
Please note that the type of output of proxy status commands differs in different versions of Istio (in particular, in versions 1.0.5, 1.1.2 and in the latest version 1.1.7). So, in earlier versions of Istio, the output contained the full JSON object, but since version 1.1.7 they look like the one shown above. You may encounter other output formats.
Pilot will show formatted differences between configuration changes and the current service status. So you can track which changes have not yet been implemented in the service and confirmed.
Mixer
Like Pilot, Mixer is an Istio component that works with traffic and applies custom rules. The main difference is that Mixer works at the level of the entire mesh-network and allows you to apply global rules. This means that it can be used to collect telemetry throughout the application or configure global restrictions on the use of certain services.
Now let's see how the work of Envoy and Mixer is different. The name sidecar proxy implies that Envoy is added to each service deployed in the application and works with traffic directed to this service. Mixer functions at the level of the entire application and applies the rules set for the environment as a whole.
How it works? Rules can be very different: from simple registration of requests with timestamps and IP addresses to the application of complex quotas and whitelists for more reliable protection. It is Mixer that provides the centralization inherent in monolithic applications. Want to configure billing based on the number of requests a user sends in a SaaS application? This can be done using the Mixer. Do I need to enable event logging in external services every time a user accesses confidential information? Mixer again.
Another useful feature of this component is integration with third-party plug-ins. The primary source of centralized application information should be compatible with tools for visualizing operations or viewing logs. Mixer allows you to add these features to Kubernetes and conveniently display the information you collect, often in real time. One such event monitoring platform (Prometheus) is already integrated.
Such tools will greatly facilitate your life. Mixer provides mesh data that was previously not available for microservice deployments. And since it is not easy to independently extract information about their activity, various specialized tools will come to the rescue.
Using Mixer
Istio comes with several Mixer policies, ready for deployment. To start collecting telemetry data, you just need to apply the YAML file with the configuration:
kubectl apply -f samples/bookinfo/telemetry/metrics.yaml
After that, incoming traffic will be registered and sent to the central instance of Mixer. If the service is active, you will immediately see the results. Otherwise, you can artificially direct traffic to the service.
Prometheus, the built-in tool for querying logs in Istio, is the easiest way to view traffic. Configure Prometheus port forwarding:
kubectl -n istio-system port-forward $(kubectl -n istio-system get pod -l app=prometheus -o jsonpath='{.items[0].metadata.name}') 9090:9090 &
Now Prometheus will run through port 9090 of the local computer - you can request logs from this location.
Citadel
In addition to the scalability provided by Envoy and the extensive knowledge provided by Mixer, one of the benefits of cloud mesh architecture is security. Istio uses Citadel , the primary security component, for this. It helps manage keys and certificates, which are an integral part of modern microservice deployments.
Managing the security of deploying microservices is not an easy task when switching to a service architecture. You will have to administer many separate, constantly changing, self-signed certificates (in the case of using mutual TLS authentication) or refuse encryption and hope for the best.
Citadel takes on a significant part of the work. This component automatically creates and stores keys based on service definitions and administers them using the built-in Kubernetes private key management infrastructure. By constantly interacting with Kubernetes, Citadel ensures that each new service will be assigned a certificate, and each new Envoy proxy will trust the certificate deployed with such a new service.
In other words, there is no longer any reason not to use TLS mutual authentication, which today is considered the best option for a service architecture. Each service receives TLS confirmation when exchanging data with other services, therefore, even if an attacker can view network traffic, the data will be encrypted and reliably protected.
Of course, support for self-signed certificates and TLS mutual authentication are just some of Citadel’s features. The component can interact with third-party certificate authorities and use alternative certificates if you have already implemented security measures based on them. It also allows you to apply more stringent policies if necessary, such as mutual authentication of TLS over HTTPS. As you can see, Citadel provides very flexible protection options.
Using Citadel
Although the security features of this component are quite complex, getting started with Citadel is a snap. As with Mixer, most of the work is done automatically. To start the process, you only need to activate the correct configuration.
For example, to enable global mutual TLS authentication, you need to change the mesh authentication policy. This can be done using kubectl by applying the following command:
kubectl apply -f - <<EOF apiVersion: "authentication.istio.io/v1alpha1" kind: "MeshPolicy" metadata: name: "default" spec: peers: - mtls: {} EOF
But that's not all. Services received a command to accept incoming traffic using TLS, but they are not configured to send outgoing requests through TLS, so data exchange is currently not possible. To apply TLS for outgoing requests, configure the rules for network interaction:
kubectl apply -f - <<EOF apiVersion: "networking.istio.io/v1alpha3" kind: "DestinationRule" metadata: name: "default" namespace: "istio-system" spec: host: "*.local" trafficPolicy: tls: mode: ISTIO_MUTUAL EOF
Then check the operation of the services. Now they can exchange data with each other, but encrypted traffic is used for this. With just two commands, your traffic is well protected and your network is safe.
Conclusion
Now you are much more knowledgeable about the capabilities of Istio and how they relate to the benefits of the service mesh in general. A clear distribution of tasks is one of the strengths of this tool. All components, such as the Envoy sidecar proxy or Citadel key management system, are isolated and standalone.
We advise you to devote some time to studying the principles of their functioning. Experiment with one or two components in a trial application or work through one of our examples. So you better understand the capabilities of service mesh technology.
Of course, it will be difficult for one developer to understand all the nuances of Istio. Technologies are constantly evolving, new opportunities are added every day, and old ones undergo changes - if you track them yourself, you will have to devote it all your time. Therefore, we have compiled visual examples to help you understand hidden processes. And the mentioned auxiliary tools will allow you to competently build work with Istio.