Clever Geek Handbook

About the network model in games for beginners

image






For the last two weeks I have been working on a network engine for my game. Before that, I did not know anything about network technologies in games, so I read many articles and conducted many experiments to understand all the concepts and be able to write my own network engine.



In this guide, I would like to share with you various concepts that you need to learn before writing your own game engine, as well as the best resources and articles for learning them.



In general, there are two main types of network architectures: peer-to-peer and client-server. In the peer-to-peer (p2p) architecture, data is transferred between any pair of connected players, and in the client-server architecture, data is transmitted only between the players and the server.



Although the peer-to-peer architecture is still used in some games, the standard is client-server: it is easier to implement, requires a smaller channel width and facilitates protection against cheating. Therefore, in this guide, we will focus on client-server architecture.



In particular, we are most interested in authoritarian servers: in such systems, the server is always right. For example, if a player thinks that he is in coordinates (10, 5), and the server tells him that he is in (5, 3), then the client should replace his position with the one transmitted by the server, and not vice versa. Using authoritarian servers makes cheater recognition easier.



There are three main components to gaming network systems:





It is very important to understand the role of each part and the difficulties associated with them.



Transport protocol



The first step is to choose a protocol for transporting data between the server and clients. There are two Internet protocols for this: TCP and UDP . But you can create your own transport protocol based on one of them or use the library in which they are used.



Comparing TCP and UDP



Both TCP and UDP are IP based. IP allows you to transfer a packet from the source to the recipient, but does not guarantee that the sent packet will sooner or later reach the recipient, that it will reach it at least once and that the sequence of packets will arrive in the correct order. Moreover, a packet may contain only a limited data size specified by the MTU value.



UDP is just a thin layer on top of IP. Therefore, it has the same limitations. In contrast, TCP has many features. It provides a reliable, ordered connection between two nodes with error checking. Therefore, TCP is very convenient and is used in many other protocols, for example, in HTTP , FTP and SMTP . But all these features come at a price: delay .



To understand why these functions can cause a delay, you need to understand how TCP works. When the sending node forwards the packet to the receiving node, it expects to receive an acknowledgment (ACK). If after a certain time he does not receive it (because the packet or confirmation was lost, or for some other reasons), then he resends the packet. Moreover, TCP ensures that packets are received in the correct order, therefore, until a lost packet is received, all other packets cannot be processed, even if they have already been received by the receiving node.



But as you probably understand, the delay in multiplayer games is very important, especially in such active genres as FPS. That is why many games use UDP with their own protocol.



A native UDP-based protocol can be more efficient than TCP for various reasons. For example, it may mark some packages as trusted and others as untrusted. Therefore, he does not care if the untrusted packet reached the receiver. Or it can process several data streams so that a packet lost in one stream does not slow down the remaining streams. For example, there may be a stream for player input and another stream for chat messages. If a chat message that is not urgent data is lost, it will not slow down the input, which is urgent. Or, a proprietary protocol can implement reliability differently than in TCP to be more efficient in video game environments.



So, if TCP is so crap, then we will create our own transport protocol based on UDP?



Everything is a bit more complicated. Even though TCP is almost suboptimal for gaming network systems, it can work just fine in your game and save your valuable time. For example, delay may not be a problem for a turn-based game or a game that can only be played on LANs, where packet delays and packet loss are much less than on the Internet.



Many successful games, including World of Warcraft, Minecraft, and Terraria, use TCP. However, most FPS use proprietary UDP-based protocols, so we'll talk more about them below.



If you decide to use TCP, then make sure that the Nagle algorithm is disabled, because it buffers packets before sending, which means it increases the delay.



To learn more about the differences between UDP and TCP in the context of multiplayer games, you can read the article by Glenn Fiedler UDP vs. TCP



Own protocol



So, you want to create your own transport protocol, but do not know where to start? You are lucky, because Glenn Fiedler wrote two amazing articles about this. You will find many clever thoughts in them.



The first article, Networking for Game Programmers 2008, is simpler than the second, Building A Game Network Protocol 2016. I recommend you start with an older one.



Keep in mind that Glenn Fiedler is a big proponent of using his own UDP protocol. And after reading his articles, you will surely get over his opinion that TCP has serious flaws in video games, and you want to implement your own protocol.



But if you are new to networking, do yourself a favor and use TCP or a library. To successfully implement your own transport protocol, you first need to learn a lot.



Network libraries



If you need something more efficient than TCP, but you do not want to bother with implementing your own protocol and go into many details, then you can use the network library. There are a lot of them:





I have not tried all of them, but I prefer ENet, because it is easy to use and reliable. In addition, she has clear documentation and a tutorial for beginners.



Transport Protocol: Conclusion



To summarize: there are two main transport protocols: TCP and UDP. TCP has many useful features: reliability, packet ordering, error detection. UDP does not have all this, but TCP, by its nature, has increased delays that are unacceptable for some games. That is, to ensure low latencies, you can create your own UDP-based protocol or use a library that implements the UDP transport protocol and is adapted for multi-player video games.



The choice between TCP, UDP and the library depends on several factors. Firstly, from the needs of the game: does it need low latencies? Secondly, from the requirements of the application protocol: does it need a reliable protocol? As we will see in the next part, you can create an application protocol for which an unreliable protocol is quite suitable. Finally, you need to take into account the experience of the developer of the network engine.



I have two tips:





At the end of this part, I recommend you read Brian Hook's Introduction to Multiplayer Game Programming , which covers many of the topics discussed here.



Application protocol



Now that we can exchange data between clients and the server, we need to decide which data to transfer and in which format.



The classic scheme is that clients send input or actions to the server, and the server sends the current game state to clients.



The server does not send a complete, but filtered state with entities that are next to the player. He does this for three reasons. First, the overall state may be too large for high frequency transmission. Secondly, customers are mainly interested in visual and audio data, because most of the game logic is simulated on the game server. Thirdly, in some games the player does not need to know certain data, for example, the opponent’s position at the other end of the map, because otherwise he can sniff packets and know exactly where to move to kill him.



Serialization



The first step is to convert the data we want to send (input or game state) into a format suitable for transmission. This process is called serialization .



The thought immediately comes to mind to use a human-readable format, such as JSON or XML. But it will be completely ineffective and in vain will occupy most of the channel.



Instead, it is recommended that you use a binary format that is much more compact. That is, packets will contain only a few bytes. Here you need to consider the problem of byte order , which may differ on different computers.



You can use a library to serialize data, for example:





Just make sure the library creates portable archives and takes care of the byte order.



An independent solution may be an independent implementation, it is not particularly difficult, especially if you use a data-oriented approach in the code. In addition, it will allow you to perform optimizations that are not always possible when using the library.



Glenn Fiedler has written two articles on serialization: Reading and Writing Packets and Serialization Strategies .



Compression



The amount of data transferred between clients and the server is limited by the bandwidth of the channel. Data compression allows you to transfer more data in each snapshot, increase the refresh rate, or simply reduce the channel requirements.



Bit packing



The first technique is bit packing. It consists in using exactly the number of bits that is necessary to describe the desired value. For example, if you have an enumeration that can have 16 different values, then instead of a whole byte (8 bits), you can use only 4 bits.



Glenn Fiedler explains how to implement this in the second part of the Reading and Writing Packets article.



Bit packing works especially well with sampling, which will be the topic of the next section.



Sampling



Discretization is a lossy compression technique that uses only a subset of the possible values ​​to encode a value. The easiest way to do this is to round off floating point numbers.



Glenn Fiedler (again!) Shows how to apply sampling in practice in his Snapshot Compression article.



Compression algorithms



The next technique will be lossless compression algorithms.



Here, in my opinion, the three most interesting algorithms you need to know:





Delta compression



The latest compression technique is delta compression. It lies in the fact that only differences between the current game state and the last state received by the client are transmitted.



It was first used in the Quake3 network engine. Here are two articles explaining how to use it:





Glenn Fiedler also used it in the second part of his Snapshot Compression article.



Encryption



In addition, you may need to encrypt the transfer of information between clients and the server. There are several reasons for this:





I strongly recommend using the library for this. I suggest using libsodium because it is especially simple and has great tutorials. Of particular interest is the key exchange tutorial, which allows you to generate new keys with every new connection.



Application Protocol: Conclusion



We will end with the application protocol. I believe that compression is completely optional and the decision to use it depends only on the game and the required bandwidth. Encryption, in my opinion, is mandatory, but in the first prototype you can do without it.



Application logic



Now we are able to update the state in the client, but we may encounter problems with delays. Having entered, the player needs to wait for the update of the game state from the server to see what impact he had on the world.



Moreover, between two state updates, the world is completely static. If the refresh rate of the states is low, then the movements will be very twitching.



There are several techniques to reduce the impact of this problem, and in the next section I will talk about them.



Delay Smoothing Techniques



All the techniques described in this section are discussed in detail in Gabriel Gambetta’s Fast-Paced Multiplayer Series . I highly recommend reading this great series of articles. It also has an interactive demo that lets you see how these techniques work in practice.



The first technique is to apply the input directly, without waiting for a response from the server. This is called client-side prediction . However, when the client receives the update from the server, he must make sure that his forecast was correct. If this is not so, then he just needs to change his state according to that received from the server, because the server is authoritarian. This technique was first used at Quake. You can read more about it in the article Quake Engine code review by Fabien Sanglar [ translation into Habré].



The second set of techniques is used to smooth the movement of other entities between two state updates. There are two ways to solve this problem: interpolation and extrapolation. In the case of interpolation, the last two states are taken and the transition from one to the other is shown. Its disadvantage is that it causes a small fraction of the delay, because the client always sees what happened in the past. Extrapolation is predicting where entities should now be based on the last state received by the client. Its disadvantage is that if the entity completely changes the direction of movement, then there will be a large error between the forecast and the actual position.



The last, most advanced technique, useful only in FPS is lag compensation . When using lag compensation, the server takes into account client delays when it shoots at a target. For example, if a player completed a headshot on his screen, but in reality his goal was due to a delay elsewhere, then it would be dishonest to deny a player the right to kill because of a delay. Therefore, the server rewinds time back to the moment when the player shot to simulate what the player saw on his screen and check the collision between his shot and the target.



Glenn Fiedler (as always!) Wrote a 2004 article in Network Physics (2004) , which laid the foundation for synchronizing physics simulations between a server and a client. In 2014, he wrote a new series of Networking Physics articles that described other techniques for synchronizing physics simulations.



There are also two articles on the Valve wiki, Source Multiplayer Networking and Latency Compensating Methods in Client / Server In-game Protocol Design and Optimization , which discuss delay compensation.



Cheating prevention



There are two main techniques to prevent cheating.



First: the complication of sending malicious packages by cheaters. As stated above, encryption is a good way to implement it.



Second: an authoritarian server should only receive commands / input / actions. The client should not be able to change the state on the server, except by sending input. Then, every time the input is received, the server must check it for validity before applying it.



Application Logic: Conclusion



I recommend that you implement a method of simulating large delays and low refresh rates in order to be able to test the behavior of your game in poor conditions, even when the client and server are running on the same computer. This will greatly simplify the implementation of delay smoothing techniques.



Other useful resources



If you want to explore other resources on network models, you can find them here:





More articles:


All Articles