[Do not] use CDN
In almost any article or tool for optimizing the speed of sites there is a modest item “use CDN”. In general, a CDN is a content delivery network or content delivery network. We at Method Lab often encounter customer questions on this topic, some of which include CDNs on their own. The purpose of this article is to figure out what the CDN can give in terms of site loading speed, what problems can arise and in which cases the use of CDN is justified.
The delays in the picture are caused by the use of CDN.
Like many technologies, CDNs emerged as needed. With the development of Internet channels for Web users, online video services have appeared. Naturally, video content requires orders of magnitude greater throughput than conventional website content (pictures, text, and CSS or JS code).
When you try to broadcast a video stream in parallel to multiple clients from one server, the Internet channel of the server will most likely become a bottleneck. As a rule, several thousand threads are enough to clog a typical server channel. Of course, there may be other resource restrictions, but now they are not important. It is also important that expanding the server channel is too expensive (and sometimes impossible), and also impractical. The load on the channel during broadcasts will be cyclical.
The problem of channel limitation of a separate server is perfectly solved by CDN. Clients do not connect directly to the server, but to the nodes of the CDN network. In an ideal situation, the server gives one stream to the CDN, and then the network uses its own resources to deliver this stream to many users. From the point of view of the economy, we pay only for the actually consumed resources (this may be bandwidth or traffic) and we get excellent scalability of our service. Using a CDN to deliver heavy content is fully justified and logical. Although it is worth noting that the largest players in this area (for example, Netflix) build their CDNs instead of using large commercial CDNs (Akamai, Cloudflare, Fastly, etc.)
As the web has evolved, the web applications themselves have become more complex and heavier. The problem of download speed came to the fore. Site speed enthusiasts quickly spotted several major issues that led to slow site loading. One of them was network latency (RTT - round trip time or ping time). Delays affect many processes in loading a site: establishing a TCP connection, starting a TLS session, loading each resource separately (pictures, JS file, HTML document, etc.)
The problem was aggravated by the fact that when using the HTTP / 1.1 puncture (before SPDY, QUIC, and HTTP / 2 it was the only option), browsers open no more than 6 TCP connections to one host. All this led to a simple connection and inefficient use of channel bandwidth. The problem was partially solved by domain sharding - the creation of additional hosts to overcome the limit on the number of connections.
Here comes the second ability of CDN - delay reduction (RTT) due to the large number of points and the proximity of nodes to the user. The distance here plays a decisive role: the speed of light is limited (about 200,000 km / s in optical fiber). This means that every 1000 km of path adds 5 ms of delay or 10 ms to RTT. This is the minimum transmission time, since there are still delays on the intermediate equipment. Since a CDN usually can cache objects on its servers, we can benefit from loading such objects through a CDN. Prerequisites for this: the presence of an object in the cache, the proximity of the CDN point to the user in comparison with the web application server (origin server). It is important to understand that the geographical proximity of the CDN does not guarantee low latency. The routing between the client and the CDN can be designed in such a way that the client will connect to a host in another country, and possibly on another continent. Here, the relations between telecommunications operators and the CDN service (peering, the presence of interfaces, participation in IX, etc.) and the routing policy of the CDN traffic itself come into effect. For example, when using two initial plans (free and cheap), Cloudflare does not guarantee the delivery of content from the nearest site - the host will be selected to achieve the minimum cost.
Many leading Internet companies have attracted the interest of the public (web developers and service owners) in the subject of download speed and website performance. Among these companies are Yahoo (Yslow tool), AOL (WebPageTest) and Google (Page Speed Insights service), which develop their recommendations on website acceleration (primarily, they relate to client optimization). Later, new tools for testing the speed of sites appear, which also give tips on increasing the speed of sites. In each of these services or plugins there is a constant recommendation “Use CDN”. As an explanation of the effect of CDN, a reduction in network delays is generally indicated. Unfortunately, not everyone is ready to understand how exactly the acceleration effect from CDN is achieved and how it can be measured, therefore the recommendation is taken on faith and is used as a postulate. In fact, not all CDNs are equally useful.
To assess the utility of using CDNs, they need to be classified. What can be found now in practice (the examples in parentheses are certainly not exhaustive):
The key difference between these types is the following: what part of the traffic goes through the CDN. Types 1-3 is the delivery of only part of the content: from one request to several dozens (usually pictures). Types 4 and 5 are full proxy traffic through the CDN.
In practice, this means the number of connections that are used to download the site. When using HTTP / 2, we use one TCP connection to the host to process any number of requests. If we share resources on the main host (origin) and CDN, then it is necessary to separate requests across several domains and create several TCP connections. In the worst case, this is: DNS (1 RTT) + TCP (1 RTT) + TLS (2-3 RTT) = 6-7 RTT. This formula does not take into account delays in mobile networks on the activation of the device’s radio channel (if it was not active) and delays on the cell tower.
Here's what it looks like on a site’s waterfall download (delays are highlighted for connecting to a CDN with RTT 150 ms):
If the CDN covers all site traffic (except third-party services), then we can use a single TCP connection, saving delays in connecting to additional hosts. Of course, this applies to HTTP / 2 connections.
Further differences are determined by the functionality of a particular CDN - for the first type, this is just hosting a static file, for the fifth, it is changing several types of site content in order to optimize.
Let's describe the full range of CDN's website acceleration capabilities, without regard to the functionality of individual types of CDNs, and then see what is implemented in each of them.
The most basic feature and an understandable feature, however, is often poorly implemented. The presence of compression declare all CDN as their feature for acceleration. But if you look in more detail, then the flaws become clear:
If you use CDN, you should check these few points: take the file that came from the CDN, fix its size in compressed form and manually compress it for comparison (you can use some online service with brotli support, for example, compress.rf ).
Also a simple feature to speed up: put headers for caching content by the client (browser). The most current cache-control header, expired is expires. Additionally, Etag can be used. The main thing is that cache-control max-age should be large enough (from a month or more), if you are ready to cache the resource as hard as possible, you can add the immutable option.
CDNs can underestimate the max-age value, forcing the user to reload statics more often. What is the reason: with the desire to increase traffic on the network or increase compatibility with sites that do not know how to flush the cache - it is not clear. For example, the default caching time for Cloudflare headers is 1 hour, which is very small for immutable statics.
Since the CDN takes over the functions of caching and uploading pictures, it would be logical to optimize them on the side of the CDN and give them to users in this form. Let's make a reservation right away, this feature is available only for CDN types 2, 3, and 5.
You can optimize images in various ways: using advanced compression formats (for example, WebP), more efficient encoders (MozJPEG), or simply cleaning up unnecessary metadata.
In general, there are two types of such optimizations: with loss of quality and without loss of quality. CDNs usually strive to use lossless optimization - in order to avoid possible complaints from customers about the change in image quality. In such conditions, the gain will be minimal. In reality, often the quality level of JPEG is much higher than necessary and you can safely carry out recompression with a lower quality indicator, without compromising the perception of users. On the other hand, it is difficult to determine the level of quality and settings universally for all possible web applications, therefore CDNs use more conservative settings compared to those that can be applied taking into account the context (image assignment, type of web application, etc.)
Most of the traffic today is transmitted via TLS connections, which means that we spend additional time on TLS negotiation. Recently, new technologies have been developed to accelerate this process. For example, EC-cryptography, TLS 1.3, session cache and tickets (session tickets), hardware encryption acceleration (AES-NI), etc. Correct TLS settings can reduce the connection time to 0-1 RTT (not including DNS and TCP )
In the presence of modern software, it is not difficult to implement such practices at our own facilities.
Not all CDNs implement TLS best practices; this can be verified by measuring the time of the TLS connection (for example, in Webpagetest). Ideal for a new connection - 1RTT, 2RTT - medium level, 3RTT and more - bad.
It should also be noted that even when using TLS at the CDN level, the server with our web application must also handle TLS, but from the CDN side, because the traffic between the server and the CDN passes on the public network. In the worst case, we get double delays of the TLS connection (the first to the CDN host, the second between it and our server).
For some applications, you should pay attention to security issues: usually traffic is decrypted on CDN nodes, and this is a potential opportunity to intercept traffic. The option to work without disclosing traffic is usually offered in top-end tariff plans for a fee.
The main advantage of the CDN that everyone is talking about: low latency (less distance) between the CDN host and the user. This is achieved by creating a geographically distributed network architecture in which hosts are located at user concentration points (cities, traffic exchange points, etc.)
In practice, priorities for different networks may be located in specific regions. For example, Russian CDNs will have more points of presence in Russia. American will primarily develop a network in the United States. For example, one of the largest CDN Cloudflare has only 2 points in Russia - Moscow and St. Petersburg. That is, as much as possible, we can save about 10 ms of delay compared to direct accommodation in Moscow.
Most western CDNs have no points in Russia at all. By connecting to them you can only increase delays for your Russian audience.
The most complex and technological item. Changing content on delivery can be very risky. Even if we take the minification: reducing the source code (due to extra spaces, unimportant constructions, etc.) can affect its performance. If we talk about more serious changes - moving the JS-code to the end of HTML, merging files and the like - the risk of disrupting the functionality of the site is even higher.
Therefore, only some type 5 CDNs do this. Of course, it will not work to automate all the changes needed to accelerate - manual analysis and optimization is required. For example, the removal of unused or duplicate code relates precisely to manual tasks.
As a rule, all such optimizations are controlled by settings and the most dangerous ones are disabled by default.
So, let's see which of the potential acceleration options various types of CDNs provide.
For convenience, we repeat the classification.
Now compare the features and types of CDN.
In this table, “+” is used to indicate full support, “-” indicates lack, “+ -” indicates partial support. Of course, deviations from this table are possible in reality (for example, some general purpose CDN will introduce features for optimizing images), but it is useful for a general idea.
I hope that after reading this article you will have a clearer picture regarding the recommendation “use CDN” to speed up sites.
As in any business, one cannot believe the marketing promises of any service. The effect must be measured and verified in real conditions. If you are already using some kind of CDN, check it for effectiveness according to the criteria described in the article.
Perhaps using CDN is slowing down the loading of your site right now.
As a general recommendation, you can focus on the following: study your audience, determine its geographical scope. If your main audience is concentrated in a radius of 1-2 thousand kilometers, you do not need CDN for the main purpose - to reduce delays. Instead, you can place your server closer to users and configure it properly, getting most of the optimizations described in the article (free and permanent).
In case your audience is really geographically distributed (radius of more than 3000 kilometers), the use of high-quality CDN will really be useful. However, you need to understand in advance what exactly your CDN can speed up (see the table of features and their description). At the same time, website acceleration remains a complex task that cannot be solved by connecting a CDN. In addition to the indicated optimizations, the most effective acceleration tools remain overboard CDN: server part optimization, advanced client part changes (deleting unused code, optimizing the rendering process, working with content, fonts, adaptability, etc.)
The delays in the picture are caused by the use of CDN.
A bit of history
Like many technologies, CDNs emerged as needed. With the development of Internet channels for Web users, online video services have appeared. Naturally, video content requires orders of magnitude greater throughput than conventional website content (pictures, text, and CSS or JS code).
When you try to broadcast a video stream in parallel to multiple clients from one server, the Internet channel of the server will most likely become a bottleneck. As a rule, several thousand threads are enough to clog a typical server channel. Of course, there may be other resource restrictions, but now they are not important. It is also important that expanding the server channel is too expensive (and sometimes impossible), and also impractical. The load on the channel during broadcasts will be cyclical.
The problem of channel limitation of a separate server is perfectly solved by CDN. Clients do not connect directly to the server, but to the nodes of the CDN network. In an ideal situation, the server gives one stream to the CDN, and then the network uses its own resources to deliver this stream to many users. From the point of view of the economy, we pay only for the actually consumed resources (this may be bandwidth or traffic) and we get excellent scalability of our service. Using a CDN to deliver heavy content is fully justified and logical. Although it is worth noting that the largest players in this area (for example, Netflix) build their CDNs instead of using large commercial CDNs (Akamai, Cloudflare, Fastly, etc.)
As the web has evolved, the web applications themselves have become more complex and heavier. The problem of download speed came to the fore. Site speed enthusiasts quickly spotted several major issues that led to slow site loading. One of them was network latency (RTT - round trip time or ping time). Delays affect many processes in loading a site: establishing a TCP connection, starting a TLS session, loading each resource separately (pictures, JS file, HTML document, etc.)
The problem was aggravated by the fact that when using the HTTP / 1.1 puncture (before SPDY, QUIC, and HTTP / 2 it was the only option), browsers open no more than 6 TCP connections to one host. All this led to a simple connection and inefficient use of channel bandwidth. The problem was partially solved by domain sharding - the creation of additional hosts to overcome the limit on the number of connections.
Here comes the second ability of CDN - delay reduction (RTT) due to the large number of points and the proximity of nodes to the user. The distance here plays a decisive role: the speed of light is limited (about 200,000 km / s in optical fiber). This means that every 1000 km of path adds 5 ms of delay or 10 ms to RTT. This is the minimum transmission time, since there are still delays on the intermediate equipment. Since a CDN usually can cache objects on its servers, we can benefit from loading such objects through a CDN. Prerequisites for this: the presence of an object in the cache, the proximity of the CDN point to the user in comparison with the web application server (origin server). It is important to understand that the geographical proximity of the CDN does not guarantee low latency. The routing between the client and the CDN can be designed in such a way that the client will connect to a host in another country, and possibly on another continent. Here, the relations between telecommunications operators and the CDN service (peering, the presence of interfaces, participation in IX, etc.) and the routing policy of the CDN traffic itself come into effect. For example, when using two initial plans (free and cheap), Cloudflare does not guarantee the delivery of content from the nearest site - the host will be selected to achieve the minimum cost.
Many leading Internet companies have attracted the interest of the public (web developers and service owners) in the subject of download speed and website performance. Among these companies are Yahoo (Yslow tool), AOL (WebPageTest) and Google (Page Speed Insights service), which develop their recommendations on website acceleration (primarily, they relate to client optimization). Later, new tools for testing the speed of sites appear, which also give tips on increasing the speed of sites. In each of these services or plugins there is a constant recommendation “Use CDN”. As an explanation of the effect of CDN, a reduction in network delays is generally indicated. Unfortunately, not everyone is ready to understand how exactly the acceleration effect from CDN is achieved and how it can be measured, therefore the recommendation is taken on faith and is used as a postulate. In fact, not all CDNs are equally useful.
Using CDN Today
To assess the utility of using CDNs, they need to be classified. What can be found now in practice (the examples in parentheses are certainly not exhaustive):
- Free CDNs for distribution of JS-libraries (MaxCDN, Google. Yandex).
- CDN services for client optimization (for example, Google Fonts for fonts, Cloudinary, Cloudimage for pictures).
- CDN for statics and resource optimization in CMS (available in Bitrix, Wordpress and others).
- General purpose CDN (StackPath, CDNVideo, NGENIX, Megaphone).
- DNS for website acceleration (Cloudflare, Imperva, Airi).
The key difference between these types is the following: what part of the traffic goes through the CDN. Types 1-3 is the delivery of only part of the content: from one request to several dozens (usually pictures). Types 4 and 5 are full proxy traffic through the CDN.
In practice, this means the number of connections that are used to download the site. When using HTTP / 2, we use one TCP connection to the host to process any number of requests. If we share resources on the main host (origin) and CDN, then it is necessary to separate requests across several domains and create several TCP connections. In the worst case, this is: DNS (1 RTT) + TCP (1 RTT) + TLS (2-3 RTT) = 6-7 RTT. This formula does not take into account delays in mobile networks on the activation of the device’s radio channel (if it was not active) and delays on the cell tower.
Here's what it looks like on a site’s waterfall download (delays are highlighted for connecting to a CDN with RTT 150 ms):
If the CDN covers all site traffic (except third-party services), then we can use a single TCP connection, saving delays in connecting to additional hosts. Of course, this applies to HTTP / 2 connections.
Further differences are determined by the functionality of a particular CDN - for the first type, this is just hosting a static file, for the fifth, it is changing several types of site content in order to optimize.
CDN website acceleration features
Let's describe the full range of CDN's website acceleration capabilities, without regard to the functionality of individual types of CDNs, and then see what is implemented in each of them.
1. Compression of text resources
The most basic feature and an understandable feature, however, is often poorly implemented. The presence of compression declare all CDN as their feature for acceleration. But if you look in more detail, then the flaws become clear:
- low degrees can be used for dynamic compression - 5-6 (for example, for gzip maximum - 9);
- in static compression (files in the cache) no additional features are used (for example, zopfi or brotli with a power of 11)
- no support for efficient brotli compression (saving about 20% compared to gzip).
If you use CDN, you should check these few points: take the file that came from the CDN, fix its size in compressed form and manually compress it for comparison (you can use some online service with brotli support, for example, compress.rf ).
2. Setting Client Caching Headers
Also a simple feature to speed up: put headers for caching content by the client (browser). The most current cache-control header, expired is expires. Additionally, Etag can be used. The main thing is that cache-control max-age should be large enough (from a month or more), if you are ready to cache the resource as hard as possible, you can add the immutable option.
CDNs can underestimate the max-age value, forcing the user to reload statics more often. What is the reason: with the desire to increase traffic on the network or increase compatibility with sites that do not know how to flush the cache - it is not clear. For example, the default caching time for Cloudflare headers is 1 hour, which is very small for immutable statics.
3. Image optimization
Since the CDN takes over the functions of caching and uploading pictures, it would be logical to optimize them on the side of the CDN and give them to users in this form. Let's make a reservation right away, this feature is available only for CDN types 2, 3, and 5.
You can optimize images in various ways: using advanced compression formats (for example, WebP), more efficient encoders (MozJPEG), or simply cleaning up unnecessary metadata.
In general, there are two types of such optimizations: with loss of quality and without loss of quality. CDNs usually strive to use lossless optimization - in order to avoid possible complaints from customers about the change in image quality. In such conditions, the gain will be minimal. In reality, often the quality level of JPEG is much higher than necessary and you can safely carry out recompression with a lower quality indicator, without compromising the perception of users. On the other hand, it is difficult to determine the level of quality and settings universally for all possible web applications, therefore CDNs use more conservative settings compared to those that can be applied taking into account the context (image assignment, type of web application, etc.)
4. TLS connection optimization
Most of the traffic today is transmitted via TLS connections, which means that we spend additional time on TLS negotiation. Recently, new technologies have been developed to accelerate this process. For example, EC-cryptography, TLS 1.3, session cache and tickets (session tickets), hardware encryption acceleration (AES-NI), etc. Correct TLS settings can reduce the connection time to 0-1 RTT (not including DNS and TCP )
In the presence of modern software, it is not difficult to implement such practices at our own facilities.
Not all CDNs implement TLS best practices; this can be verified by measuring the time of the TLS connection (for example, in Webpagetest). Ideal for a new connection - 1RTT, 2RTT - medium level, 3RTT and more - bad.
It should also be noted that even when using TLS at the CDN level, the server with our web application must also handle TLS, but from the CDN side, because the traffic between the server and the CDN passes on the public network. In the worst case, we get double delays of the TLS connection (the first to the CDN host, the second between it and our server).
For some applications, you should pay attention to security issues: usually traffic is decrypted on CDN nodes, and this is a potential opportunity to intercept traffic. The option to work without disclosing traffic is usually offered in top-end tariff plans for a fee.
5. Reducing connection delays
The main advantage of the CDN that everyone is talking about: low latency (less distance) between the CDN host and the user. This is achieved by creating a geographically distributed network architecture in which hosts are located at user concentration points (cities, traffic exchange points, etc.)
In practice, priorities for different networks may be located in specific regions. For example, Russian CDNs will have more points of presence in Russia. American will primarily develop a network in the United States. For example, one of the largest CDN Cloudflare has only 2 points in Russia - Moscow and St. Petersburg. That is, as much as possible, we can save about 10 ms of delay compared to direct accommodation in Moscow.
Most western CDNs have no points in Russia at all. By connecting to them you can only increase delays for your Russian audience.
6. Content optimization (minification, structural changes)
The most complex and technological item. Changing content on delivery can be very risky. Even if we take the minification: reducing the source code (due to extra spaces, unimportant constructions, etc.) can affect its performance. If we talk about more serious changes - moving the JS-code to the end of HTML, merging files and the like - the risk of disrupting the functionality of the site is even higher.
Therefore, only some type 5 CDNs do this. Of course, it will not work to automate all the changes needed to accelerate - manual analysis and optimization is required. For example, the removal of unused or duplicate code relates precisely to manual tasks.
As a rule, all such optimizations are controlled by settings and the most dangerous ones are disabled by default.
Acceleration Support by CDN Type
So, let's see which of the potential acceleration options various types of CDNs provide.
For convenience, we repeat the classification.
- Free CDNs for distribution of JS-libraries (MaxCDN, Google. Yandex).
- CDN services for client optimization (for example, Google Fonts for fonts, Cloudinary, Cloudimage for pictures).
- CDN for statics and resource optimization in CMS (available in Bitrix, Wordpress and others).
- General purpose CDN (StackPath, CDNVideo, NGENIX, Megaphone).
- DNS for website acceleration (Cloudflare, Imperva, Airi).
Now compare the features and types of CDN.
| Possibility | Type 1 | Type 2 | Type 3 | Type 4 | Type 5 |
|---|---|---|---|---|---|
| Text compression | + - | - | + - | + - | + |
| Cache headers | + | + | + | + | + |
| Pictures | - | + - | + - | - | + |
| TLS | - | - | - | + - | + |
| Delays | - | - | - | + | + |
| Content | - | - | - | - | + |
In this table, “+” is used to indicate full support, “-” indicates lack, “+ -” indicates partial support. Of course, deviations from this table are possible in reality (for example, some general purpose CDN will introduce features for optimizing images), but it is useful for a general idea.
Summary
I hope that after reading this article you will have a clearer picture regarding the recommendation “use CDN” to speed up sites.
As in any business, one cannot believe the marketing promises of any service. The effect must be measured and verified in real conditions. If you are already using some kind of CDN, check it for effectiveness according to the criteria described in the article.
Perhaps using CDN is slowing down the loading of your site right now.
As a general recommendation, you can focus on the following: study your audience, determine its geographical scope. If your main audience is concentrated in a radius of 1-2 thousand kilometers, you do not need CDN for the main purpose - to reduce delays. Instead, you can place your server closer to users and configure it properly, getting most of the optimizations described in the article (free and permanent).
In case your audience is really geographically distributed (radius of more than 3000 kilometers), the use of high-quality CDN will really be useful. However, you need to understand in advance what exactly your CDN can speed up (see the table of features and their description). At the same time, website acceleration remains a complex task that cannot be solved by connecting a CDN. In addition to the indicated optimizations, the most effective acceleration tools remain overboard CDN: server part optimization, advanced client part changes (deleting unused code, optimizing the rendering process, working with content, fonts, adaptability, etc.)
All Articles