Of course, we understand that for many this article will not be a revelation. But, since articles for beginning administrators appear from time to time on Habr, and also because this article appeared from a real letter to a real client, we will still share this information here. There is a high probability that it will be useful to someone.
Therefore, we explain in detail what happens between the server in the cloud and the office if they are connected by a site-to-site network. Note that in this case, part of the services is available only from the office, and part - from anywhere from the Internet.
We’ll immediately explain that our client wished that he could come to the server 192.168.A.1 from anywhere via RDP, connecting to AAA2: 13389 , and to the rest of the services only from the office (192.168.B.0 / 24) connected through VPN Also, the client was configured initially that the machine 192.168.B.2 in the office could also be accessed via RDP from anywhere, connecting to BBB1: 11111 . We helped organize IPSec connections between the cloud and the office, and the customer’s IT specialist began to ask questions about what would happen in this or that case. To answer all these questions, we, in fact, wrote him everything that you can read below.
Now consider these processes in more detail.
First position
When something is sent from 192.168.B.0 / 24 to 192.168.A.0 / 24 or from 192.168.A.0 / 24 to 192.168.B.0 / 24 , it gets into the VPN. That is, this packet is additionally encrypted and transmitted between BBB1 and AAA1 , but 192.168.A.1 sees the packet from 192.168.B.1 exactly. They can communicate with each other via any protocol. The return responses are also transmitted via VPN in the same way, which means that the packet from 192.168.A.1 for 192.168.B.1 will be sent as an ESP datagram from AAA1 to BBB1 , which the router will deploy on the other side, take that packet from it and send it to 192.168.B.1 as a packet from 192.168.A.1 .
Specific example:
1) 192.168.B.1 refers to 192.168.A.1 , wants to establish a TCP connection with 192.168.A.1: 3389 ;
2) 192.168.B.1 sends a request to establish a connection from 192.168.B.1: 55555 (he selects the port for feedback, hereinafter we will use the number 55555 as an example of the port number that the system selects when generating TCP- connections) to 192.168.A.1: 3389 ;
3) the operating system that runs on the computer with the address 192.168.B.1 decides to transfer this packet to the gateway address of the router ( 192.168.B.254 in our case), because there are other, more specific routes for 192.168.A.1 , it does not, therefore, it sends the packet along the default route (0.0.0.0/0);
4) for this, she tries to find the MAC address for the IP address 192.168.B.254 in the cache table of the ARP protocol. If it is not found, it sends from the address 192.168.B.1 a broadcast who-has request to the network 192.168.B.0 / 24 . When 192.168.B.254 sends back its MAC address to it, the system sends an Ethernet packet for it and stores this information in its cache table;
5) the router receives this packet and decides where to send it: it has a policy that it must transmit all packets between 192.168.B.0 / 24 and 192.168.A.0 / 24 over a VPN connection between BBB1 and AAA1 ;
6) the router forms an ESP datagram from BBB1 to AAA1 ;
7) the router decides to whom to send this packet, it sends it to, say, BBB254 (gateway of the Internet provider), because it does not have more specific routes to AAA1 than 0.0.0.0/0;
8) in the same way as already mentioned, it finds the MAC address for BBB254 and sends the packet to the gateway of the Internet provider;
9) Internet providers transmit via their networks the ESP datagram from BBB1 to AAA1 ;
10) the virtual router on AAA1 receives this datagram, decrypts it and receives a packet from 192.168.B.1: 55555 for 192.168.A.1: 3389 ;
11) the virtual router checks to whom it should be transferred, finds the network 192.168.A.0 / 24 in the routing table and sends it directly to 192.168.A.1 , since it has an interface 192.168.A.254 / 24 ;
12) for this, the virtual router finds the MAC address for 192.168.A.1 and passes this packet to it through the virtual Ethernet network;
13) 192.168.A.1 receives this packet on port 3389, agrees to establish a connection and forms a packet in response from 192.168.A.1: 3389 to 192.168.B.1: 55555 ;
14) his system sends this packet to the gateway address of the virtual router ( 192.168.A.254 in our case), because it does not have other, more specific routes for 192.168.B.1 , therefore, it must transmit the packet along the route default (0.0.0.0/0);
15) in the same way as in previous cases, the system that runs on the server with the address 192.168.A.1 finds the MAC address 192.168.A.254 , since it is on the same network with its interface 192.168.A.1 / 24 ;
16) the virtual router receives this packet and decides where to transfer it: it has a policy that it must transmit all packets between 192.168.A.0 / 24 and 192.168.B.0 / 24 over a VPN connection between AAA1 and BBB1 ;
17) the virtual router generates an ESP datagram from AAA1 to BBB1 ;
18) the virtual router decides to whom this packet should be sent, sends it to AAA254 (the gateway of the Internet provider, in this case, this is us too), because it does not have more specific routes to BBB1 than 0.0.0.0/0;
19) Internet service providers transmit on their networks the ESP datagram from AAA1 to BBB1 ;
20) the router on BBB1 receives this datagram, decrypts it, and receives a packet from 192.168.A.1: 3389 for 192.168.B.1: 55555 ;
21) he understands that it should be transmitted specifically to 192.168.B.1 , since he is on the same network with him, therefore, he has a corresponding entry in the routing table that forces him to send packets for the whole 192.168.B.0 / 24 directly;
22) the router finds the MAC address for 192.168.B.1 and sends it this packet;
23) the operating system on the computer with the address 192.168.B.1 receives a packet from 192.168.A.1: 3389 for 192.168.B.1: 55555 and initiates the following steps to establish a TCP connection.
This example is quite succinct and simplistic (and here you can recall a bunch of details) described what happens at levels 2-4. Levels 1, 5-7 are not considered.
Second position
If something is sent specifically to AAA2 from 192.168.B.0 / 24 , it goes not to the VPN, but directly. That is, if a user from address 192.168.B.1 accesses AAA2: 13389 , this packet fills from address BBB1 , passes to AAA2 , and there the router receives it and sends it to 192.168.A.1 . 192.168.A.1 does not know anything about 192.168.B.1 , he sees a packet from BBB1 , because he understood it. Therefore, the answer to this request goes along the general route, it exactly the same goes from the address AAA2 and goes to BBB1 , and that router gives this answer to 192.168.B.1 , it sees the answer from AAA2 , to which it addressed.
Specific example:
1) 192.168.B.1 refers to AAA2 , wants to establish a TCP connection with AAA2: 13389 ;
2) 192.168.B.1 sends a request to establish a connection from 192.168.B.1: 55555 (this number, as in the previous example, may be different) to AAA2: 13389 ;
3) the operating system that runs on a computer with the address 192.168.B.1 decides to transfer this packet to the gateway address of the router ( 192.168.B.254 in our case), because it doesn’t have other, more specific routes for AAA2 , which means that it sends the packet along the default route (0.0.0.0/0);
4) for this, she, as we mentioned in the previous example, tries to find the MAC address for the IP address 192.168.B.254 in the cache table of the ARP protocol. If it is not found, it sends from the address 192.168.B.1 a broadcast who-has request to the network 192.168.B.0 / 24 . When 192.168.B.254 sends back its MAC address to it, the system sends an Ethernet packet for it and stores this information in its cache table;
5) the router receives this packet and decides where to send it: it has a policy that it must forward (replacing the return address) all packets from 192.168.B.0 / 24 to other Internet nodes;
6) since this policy implies that the return address must coincide with the lowest address on the interface through which this packet will be transmitted, the router first decides to whom exactly this packet should be transmitted, and he, as in the previous example, should send it to BBB254 (gateway of the Internet provider), because it does not have more specific routes to AAA2 than 0.0.0.0/0;
7) therefore, the router replaces the return address of the packet, henceforth the packet from BBB1: 44444 (the port number, of course, may be different) to AAA2: 13389 ;
8) the router remembers what it has done, which means that when a response is received from AAA2: 13389 to BBB1: 44444 , it will know that it should change the address and port of the recipient to 192.168.B.1: 55555 .
9) now the router must transfer it to the network of the Internet provider via BBB254 , therefore, just like we already mentioned, it finds the MAC address for BBB254 and sends the packet to the gateway of the Internet provider;
10) Internet providers transfer a packet from BBB1 to AAA2 on their networks;
11) the virtual router on AAA2 receives this packet on port 13389;
12) there is a rule on the virtual router that stipulates that packets that came from any sender to this port should be sent to 192.168.A.1: 3389 ;
13) the virtual router finds the network 192.168.A.0 / 24 in the routing table and sends it directly to 192.168.A. 1, because it has an interface 192.168.A.254 / 24 ;
14) for this, the virtual router finds the MAC address for 192.168.A.1 and passes this packet to it through the virtual Ethernet network;
15) 192.168.A.1 receives this packet on port 3389, agrees to establish a connection and forms a packet in response from 192.168.A.1: 3389 on BBB1: 44444 ;
16) his system sends this packet to the gateway address of the virtual router ( 192.168.A.254 in our case), because it does not have other, more specific routes for BBB1 , therefore, it must transmit the packet along the default route (0.0. 0.0 / 0);
17) in the same way as in previous cases, the system that runs on the server with the address 192.168.A.1 finds the MAC address 192.168.A.254 , since it is on the same network with its interface 192.168.A.1 / 24 ;
18) The virtual router accepts this packet. It should be noted that he remembers that he received a packet from BBB1: 44444 on AAA2: 13389 and changed the recipient address and port to 192.168.A.1: 3389 , therefore, he changes the packet from 192.168.A.1: 3389 to BBB1: 44444 sender address on AAA2: 13389 ;
19) the virtual router decides to whom this packet should be sent, it sends it to AAA254 (the gateway of the Internet provider, in this case, this is us too), because it does not have more specific routes to BBB1 than 0.0.0.0/0;
20) Internet providers transfer a packet from AAA2 to BBB1 on their networks;
21) the router on BBB1 receives this packet and recalls that when it sent the packet from 192.168.B.1: 55555 to AAA2: 13389 , it changed its address and sender port to BBB1: 44444 , so this is the answer that needs to be transmitted on 192.168.B.1: 55555 (in fact, there are several more checks there, but we don’t go into it);
22) he understands that they should be sent directly to 192.168.B.1 , since he is on the same network with him, therefore, he has a corresponding entry in the routing table that forces him to send packets for the whole 192.168.B.0 / 24 directly;
23) the router finds the MAC address for 192.168.B.1 and sends it this packet;
24) the operating system on the computer with the address 192.168.B.1 receives a packet from AAA2: 13389 for 192.168.B.1: 55555 and initiates the following steps to establish a TCP connection.
It should be noted that in this case the computer with the address 192.168.B.1 does not know anything about the server with the address 192.168.A.1 , it only communicates with AAA2 . Similarly, the server with the address 192.168.A.1 does not know anything about the computer with the address 192.168.B.1 . He believes that they connected to him from the address BBB1 , and he knows nothing more, so to speak.
It should also be noted that if this computer accesses AAA2: 1540 , the connection will not be established because the forwarding of connections to port 1540 is not configured on the virtual router, even if on any servers in the virtual network 192.168.A.0 / 24 (for example, on a server with the address 192.168.A.1 ) and there are some services that are waiting for a connection on this port. If the user of the computer with the address 192.168.B.1 is absolutely necessary to establish a connection with this service, he must use a VPN, i.e. contact directly at 192.168.A.1: 1540 .
It should be emphasized that any attempts to establish a connection to AAA1 (except for the IPSec connection from BBB1 will not be successful. Any attempts to establish connections to AAA2 , except for connections to port 13389, will also fail.
Also note that if someone else calls on AAA2 (for example, UDP), everything that is indicated in paragraphs 10-20 will concern him too. What happens before and after that depends on what exactly is behind this. We do not own such information, therefore we advise you to consult the site administrators with the address of the address.
Third position
And vice versa, if something is sent from 192.168.A.1 to some port that is configured to forward internally to BBB1 (for example, 11111), it also doesn’t get into the VPN, but simply gets hooked by AAA1 and gets into BBB1 , and that one is already transmitting it somewhere in, say, 192.168.B.2: 3389 . He sees this package not from 192.168.A.1 , but from AAA1 . And, when 192.168.B.2 answers, the packet goes from BBB1 to AAA1, and later gets to the initiator of the connection - 192.168.A.1 .
Specific example:
1) 192.168.A.1 addresses BBB1 , wants to establish a TCP connection with BBB1: 11111 ;
2) 192.168.A.1 sends a request to establish a connection from 192.168.A.1: 55555 (this number, as in the previous example, may be different) on BBB1: 11111 ;
3) the operating system that runs on the server with the address 192.168.A.1 decides to transfer this packet to the gateway address of the router ( 192.168.A.254 in our case), because it doesn’t have other, more specific routes for BBB1 , therefore, it passes the packet along the default route (0.0.0.0/0);
4) for this, she, as we mentioned in the previous examples, tries to find the MAC address for the IP address 192.168.A.254 in the cache table of the ARP protocol. If it is not found, it sends from the address 192.168.A.1 a broadcast who-has request to the network 192.168.A.0 / 24 . When 192.168.A.254 sends back its MAC address to it, the system sends an Ethernet packet for it and puts this information into its cache table;
5) the virtual router receives this packet and decides where to transfer it: it has a policy that it must forward (replacing the return address) all packets from 192.168.A.0 / 24 to other Internet nodes;
6) since this policy assumes that the return address must coincide with the lowest address on the interface through which this packet will be transmitted, the virtual router first decides to whom exactly this packet should be transmitted, and he, as in the previous example, should send it on AAA254 (the gateway of the Internet provider, in this case, this is us too), because it does not have more specific routes to BBB1 than 0.0.0.0/0;
7) it means that the virtual router replaces the return address of the packet, henceforth it is a packet from AAA1: 44444 (the port number, of course, may be different) to BBB1: 11111 ;
8) the virtual router remembers what he did, therefore, when a response is received from BBB1: 11111 for AAA1: 44444 , he will know that he should change the address and port of the recipient to 192.168.A.1: 55555 .
9) now the virtual router must transfer it to the network of the Internet provider through AAA254 , which means that just as we already mentioned, it finds the MAC address for AAA254 and sends the packet to the Internet provider gateway;
10) Internet providers transfer on their networks a packet from AAA1 to BBB1 ;
11) the router on BBB1 receives this packet on port 11111;
12) there is a rule on the virtual router, which stipulates that packets received from any sender to this port should be transmitted to 192.168.B.2: 3389 ;
13) the router finds the network 192.168.B.0 / 24 in the routing table and sends it directly to 192.168.B.2 , since it has an interface 192.168.B.254 / 24 ;
14) for this, the virtual router finds the MAC address for 192.168.B.2 and passes this packet to it through the virtual Ethernet network;
15) 192.168.B.2 receives this packet on port 3389, agrees to establish a connection and forms a packet in response from 192.168.B.2: 3389 to AAA1: 44444 ;
16) his system sends this packet to the gateway address of the router ( 192.168.B.254 in our case), because it does not have other, more specific routes for AAA1 , therefore, it must pass the packet along the default route (0.0.0.0 / 0);
17) in the same way as in previous cases, the system that runs on the computer with the address 192.168.B.2 finds the MAC address 192.168.B.254 , since it is on the same network with its interface 192.168.B.2 / 24 ;
18) The router accept this packet. It should be noted that he remembers that he received a packet from AAA1 on BBB1: 11111 and changed the recipient address and port to 192.168.B.2: 3389 , therefore, he changes the sender address to a packet from 192.168.B.2: 3389 for AAA1: 44444 on BBB1: 11111 ;
19) The router decides to whom to forward this packet. He sends it to, say, BBB254 (the gateway of the Internet provider, whose exact address we do not know), because it does not have more specific routes to AAA1 than 0.0.0.0/0;
20) Internet providers transfer a packet from BBB1 to AAA1 on their networks;
21) the virtual router on AAA1 receives this packet and recalls that when it sent the packet from 192.168.A.1: 55555 to BBB1: 11111 , it changed its address and sender port to AAA1: 44444 . So this is the answer that needs to be transmitted to 192.168.A.1: 55555 (in fact, as we mentioned in the previous example, there are also a few more checks there, but this time we don’t go into them too);
22) he understands that it should be sent directly to 192.168.A.1 , since he is on the same network with him, which means that he has a corresponding entry in the routing table that forces him to send packets for the whole 192.168.A.0 / 24 directly;
23) the router finds the MAC address for 192.168.A.1 and sends it this packet;
24) the operating system on the server with the address 192.168.A.1 receives a packet from BBB1: 1111 1 for 192.168.A.1: 55555 and initiates the following steps to establish a TCP connection.
In the same way as in the previous case, in this case the server with the address 192.168.A.1 does not know anything about the computer with the address 192.168.B.1 , it communicates only with BBB1 . The computer with the address 192.168.B.1 also knows nothing about the server with the address 192.168.A.1 . He believes that they connected to him from the address AAA1 , and the rest is hidden from him.
Output
This is how it happens when connecting inside a VPN tunnel between the client’s office and the environment in the cloud, as well as when connecting outside the VPN tunnel. And if you still have questions or need our help in solving cloud problems, please contact 24x7.