In the first part, we deployed a simple dynamic CDN for broadcasting WebRTC streams to two continents and made sure that the delays in such a CDN are really low, using the example of a countdown timer.
However, in addition to low latency, it is important to provide viewers with a good broadcast quality, because they pay for it. In real life, the channels between Edge servers and subscribers can be different in bandwidth and quality. For example, we publish a stream with a resolution of 720p with a bitrate of 2 Mbps, and the user plays it on an Android smartphone using a 3G connection in the area of insecure reception of the signal, and the maximum resolution at which the picture will be smooth, only 360p with a bitrate of 400 Mbps .
The devices and browsers used by viewers are very different. For example, we publish a WebRTC stream using the VP8 codec from the Chrome browser on a PC, and the viewer plays the stream in Safari on the iPhone, which only supports the H264 codec. Or vice versa, we publish the RTMP stream from OBS Studio, encoding video in H264, and audio in AAC, and the client uses a Chromium-based browser, which only supports VP8 or VP9 for video and opus for sound.
You may also need to improve the quality of the original publication. For example, we distribute the stream from the IP camera in some reserve, most of the time the picture is static, the camera gives it at a frequency of 1 frame per second. At the same time, we want the viewer to play 24 frames per second. What if it is impossible to replace the camera or change its settings?
In all these cases, transcoding of the stream on the server will be required, that is, decoding of each received frame and subsequent encoding with new parameters. Moreover, the parameters that must be changed are often known only on the client side. Let's see how it is possible to provide transcoding in CDN, balancing between broadcast quality and server loads.
Transcoding: how, where and why?
Suppose we know the flow parameters that the client wants to receive. For example, the viewer started playing a stream, and the number of frame losses in WebRTC statistics tells us that the resolution and bit rate must be reduced until the client has switched channels . In this case, by default, the stream will be transcoded on the Edge server to which the viewer is connected.
If the client does not support the codec used when publishing the stream, you can assign transcoding to both the Edge and the Origin server.
Both that, and another can only be a temporary solution, provided that the configuration of Origin and / or Edge servers were chosen with a margin. Transcoding is always performed frame by frame, so it is very demanding on processor resources. So, one processor core is able to transcode a very small number of threads:
| Resolution | Bitrate, Kbps | Number of threads |
|---|---|---|
| 360p | 1300 | 5 |
| 480p | 1800 | 3 |
| 720p | 3000 | 2 |
Even if we start the same transcoding process for all subscribers requiring the same media stream parameters, it is likely that several viewers with different parameters will select all the server’s resources.
Thus, the correct solution would be to select special servers in the CDN for transcoding tasks, and choose the server configuration based on these tasks.
Add Transcoder nodes to CDN
So, we will deploy one server in our CDN with the role of Transcoder in the European and American data centers
Configuring Transcoder Servers:
- Transcoder 1 EU
cdn_enabled=true cdn_ip=t-eu1.flashphoner.com cdn_point_of_entry=o-eu1.flashponer.com cdn_nodes_resolve_ip=false cdn_role=transcoder
- Transcoder 1 US
cdn_enabled=true cdn_ip=t-us1.flashphoner.com cdn_point_of_entry=o-eu1.flashponer.com cdn_nodes_resolve_ip=false cdn_role=transcoder
Stream transcoding parameters should be described on Edge servers as special profiles in the cdn_profiles.yml
file. As an example, consider three default profiles:
- transcoding to 640x360 resolution, 30 frames per second, a key frame is transmitted for every 90 frames, H264 video codec using OpenH264 encoder, Opus audio codec 48 kHz
-640x360: audio: codec : opus rate : 48000 video: width : 640 height : 360 gop : 90 fps : 30 codec : h264 codecImpl : OPENH264
- transcoding to resolution 1280x720, H264 video codec using OpenH264 encoder, without audio transcoding
-720p: video: height : 720 codec : h264 codecImpl : OPENH264
- transcoding to a resolution of 1280x720, 30 frames per second, a key frame is transmitted for every 90 frames, 2 Mbps bitrate, H264 video codec using an OpenH264 encoder, without audio transcoding
-720p-2Mbps: video: height : 720 bitrate : 2000 gop : 90 fps : 30 codec : h264 codecImpl : OPENH264
We publish the test
stream with a resolution of 720p on the o-eu1
and play this stream on e-eu1
, specifying the profile in the stream name, for example, test-640x360
The stream is transcoded!
Now we can describe a number of profiles on Edge servers, for example, -240p, -360p, -480p, and if on the client side, according to WebRTC statistics, a large number of lost frames are diagnosed, automatically re-request the stream with a lower resolution.
Group CDN nodes by continent
Now our Transcoder servers are equal. But what if we want to transcode streams by geography: for American viewers in America, for European viewers in Europe? This, by the way, will reduce the load on the transatlantic channels, since in this case only the original streams will go from Origin EU servers to America and vice versa, and not all transcoded versions.
In this case, in the settings of the Transcoder nodes, you must specify the group
- Transcoder 1 EU
cdn_enabled=true cdn_ip=t-eu1.flashphoner.com cdn_point_of_entry=o-eu1.flashponer.com cdn_nodes_resolve_ip=false cdn_role=transcoder cdn_groups=EU
- Transcoder 1 US
cdn_enabled=true cdn_ip=t-us1.flashphoner.com cdn_point_of_entry=o-eu1.flashponer.com cdn_nodes_resolve_ip=false cdn_role=transcoder cdn_groups=US
You must also add the group to the Edge server settings.
- Edge 1-2 EU
cdn_groups=EU
- Edge 1-2 US
cdn_groups=US
Restart the nodes with the new settings. We publish the test
stream with a resolution of 720p on the o-eu1
, we play this stream on e-eu1
with transcoding
Make sure that the stream is transcoded to t-eu
, for this we open the statistics page http://t-eu1.flashphoner.com:8081/?action=stat and see the video encoder and decoder in the Native resources
section
At the same time, there are no video encoders on t-us1
in statistics
More transcoders: load balancing
Let's say the number of viewers continues to grow, and the capacities of one Transcoder server on the continent are already not enough. Ok, add one more server
- Transcoder 2 EU
cdn_enabled=true cdn_ip=t-eu2.flashphoner.com cdn_point_of_entry=o-eu1.flashponer.com cdn_nodes_resolve_ip=false cdn_role=transcoder cdn_groups=EU
- Transcoder 2 US
cdn_enabled=true cdn_ip=t-us2.flashphoner.com cdn_point_of_entry=o-eu1.flashponer.com cdn_nodes_resolve_ip=false cdn_role=transcoder cdn_groups=US
However, now we are faced with the problem of load balancing across two transcoders. In order to prevent all threads from flowing through one server, we will limit the maximum allowable average processor load on Transcoder nodes
cdn_node_load_average_threshold=0.95
When the average processor load, divided by the number of available cores, reaches this value, the server will stop accepting requests for transcoding new threads.
You can also limit the maximum number of simultaneously running video encoders
cdn_transcoder_video_encoders_threshold=10000
When this amount is reached, the server will also cease to accept requests for transcoding streams, even if loading the processor still allows it.
In any case, the Transcoder server will continue to distribute to the Edge servers those streams that are already transcoded on it.
Ending follows
So, we have deployed dedicated servers for transcoding media streams in our CDN and, thus, we can provide our viewers with broadcast quality depending on the capabilities of their equipment and the quality of the channels. However, we still did not address the issue of restricting access to threads. We will consider it in the final part.
References
Low latency WebRTC streaming CDN is a Web Call Server-based content delivery network.