Introduction
Modern corporate content filtering systems from such eminent manufacturers as Cisco, BlueCoat, FireEye have quite a lot in common with their more powerful counterparts - DPI systems, which are being intensively implemented at the national level. The essence of the work of both is to search for incoming and outgoing Internet traffic and, based on black / white lists, make a decision to ban the Internet connection. And since both of them rely on similar principles in the basics of their work, the ways to circumvent them will also have much in common.
One of the technologies that allows quite efficiently bypassing both DPI and corporate systems is domain-front-facing technology. Its essence lies in the fact that we go to a blocked resource, hiding behind another, public domain, with a good reputation, which obviously will not be blocked by any system, for example google.com.
A lot of articles have already been written about this technology and many examples have been given. However, DNS-over-HTTPS and encrypted-SNI technologies, as well as the recently discussed technologies, as well as the new version of the TLS 1.3 protocol, provide an opportunity to consider another variant of domain-front-facing.
We deal with technology
First, let's decide a little about the basic concepts so that everyone has an understanding of who is who and why all this is needed. We mentioned the eSNI mechanism, whose work will be discussed later. The eSNI (encrypted Server Name Indication) mechanism is a secure version of SNI, available only for the TLS 1.3 protocol. The main point is to encrypt including information about which domain the request is sent to.
Now let's look at how the eSNI engine works in practice.
Suppose we have an Internet resource that is blocked by a modern DPI solution (take, for example, the famous torrent tracker - rutracker.nl). When we try to access the site of the torrent tracker, we see the standard stub of the provider that the resource is blocked:
On the ILV website, this domain is indeed listed in the stop lists:
When you request whois, you can see that the domain itself is “hidden” behind the cloud provider Cloudflare.
But unlike the “specialists” from ILV, the more technically savvy employees from the beeline (or taught by the bitter experience of our famous regulator) did not stupidly ban the site by IP address, but entered the domain name in the stop list. This is easy to verify if you look at what other domains are hiding behind the same IP address, visit one of them and see that access is not blocked:
But how does it happen? How does the provider DPI find out which of the domains my browser goes to, because all communications take place via the https protocol, and we don’t seem to have noticed the substitution of https certificates from the beeline yet? Is he clairvoyant or is he being followed by me?
Let's try to answer this question by looking at traffic through wireshark
The screenshot shows that first the browser receives the IP address of the server via DNS, then there is a standard TCP handshake with the destination server, and then the browser tries to establish an ssl connection to the server. To do this, it sends the SSL Client Hello packet, which contains the source domain name in clear text. This field is required by the cloudflare front-end server in order to properly route the connection. This is where provider DPI catches us, breaking our connection. At the same time, we do not receive any stubs from the provider, and we see a standard browser error as if the site was down or simply does not work:
Now let's enable the eSNI mechanism in the browser, as described in the instructions for Firefox :
To do this, we open the Firefox configuration page about: config and activate the following settings:
network.trr.mode = 2; network.trr.uri = https://mozilla.cloudflare-dns.com/dns-query network.security.esni.enabled = true
After that, we will check the correctness of the settings on the cloudflare website using the link and try the focus with our torrent tracker again.
Voila. Our favorite tracker opened without any VPNs or proxies. Let's now look at the traffic dump in wireshark, what happened.
This time, the ssl client hello package does not explicitly contain the destination domain, but instead a new field appeared in the package - encrypted_server_name - this is where the value of rutracker.nl is contained, and only the front-end cloudflare server can decrypt this field. And if so, then the provider DPI has no choice but to wash their hands and allow such traffic. But there are no other options with encryption.
So, how the technology works in the browser - we looked. Now let's try to apply it for more specific and interesting things. And for starters, we will teach the same curl to use eSNI to work with TLS 1.3, and at the same time we'll see how the eSNI-based front-end domain itself works.
Domain Front End with eSNI
Due to the fact that curl uses the standard openssl library to connect via https, first of all, we need to provide eSNI support there. There is no support for eSNI in the openssl master branches yet, so we need to download the special openssl branch, compile and install it.
We clone the repository from the github and compile as usual:
$ git clone https://github.com/sftcd/openssl $ cd openssl $ ./config $ make $ cd esnistuff $ make
Next, we clone the repository with curl and configure its compilation using our assembled openssl library:
$ cd $HOME/code $ git clone https://github.com/niallor/curl.git curl-esni $ cd curl-esni $ export LD_LIBRARY_PATH=/opt/openssl $ ./buildconf $ LDFLAGS="-L/opt/openssl" ./configure --with-ssl=/opt/openssl --enable-esni --enable-debug
Here it is important to correctly indicate all the directories where openssl is located (in our case, it is / opt / openssl /) and make sure that the configuration process passes without errors.
In case of successful configuration, we will see the line:
WARNING: esni ESNI enabled but marked EXPERIMENTAL. Use with caution!
$ make
After successfully building the package, we will use a special openssl bash file to configure and run curl. Copy it to the directory with curl for convenience:
cp /opt/openssl/esnistuff/curl-esni
and execute a test https request to the cloudflare server, while simultaneously writing DNS and TLS packets to Wireshark.
$ ESNI_COVER="www.hello-rkn.ru" ./curl-esni https://cloudflare.com/
In the server’s response, in addition to a lot of debugging information from openssl and curl, we get an HTTP response with code 301 from cloudflare.
HTTP/1.1 301 Moved Permanently < Date: Sun, 03 Nov 2019 13:12:55 GMT < Transfer-Encoding: chunked < Connection: keep-alive < Cache-Control: max-age=3600 < Expires: Sun, 03 Nov 2019 14:12:55 GMT < Location: https://www.cloudflare.com/
which indicates that our request was successfully delivered to the destination server, heard and processed.
Now let's look at the traffic dump in wireshark, i.e. what the provider DPI saw in this case.
It can be seen that first curl turned to the DNS server for a public eSNI key for the cloudflare server - TXT DNS request to _esni.cloudflare.com (package No. 13). Then, using the openssl library, curl sent a TLS 1.3 request to the cloudflare server in which the SNI field was encrypted with the public key obtained in the previous step (package No. 22). But, in addition to the eSNI field, as part of the SSL-hello package, a field was also inserted with the usual - open SNI, which we can specify in any order (in this case - www.hello-rkn.ru ).
This field of open SNI was not taken into account when processing by cloudflare servers and was just a disguise for provider DPI. The cloudflare server accepted our ssl-hello package, decrypted the eSNI, extracted the original SNI from there and processed it as if nothing had happened (it did everything exactly as planned during the development of eSNI).
The only thing in this case, you can cling to in terms of DPI - the primary DNS query at _esni.cloudflare.com. But we made the DNS query open only to show how this mechanism works from the inside.
To finally knock the soil out of the DPI, we use the already mentioned DNS-over-HTTPS mechanism. A small explanation is DOH, a protocol that allows you to protect yourself from a man in the middle attack by sending a DNS query over HTTPS.
We will execute the request again, but this time we will get the public eSNI keys using the https protocol, not the DNS:
ESNI_COVER="www.hello-rkn.ru" DOH_URL=https://mozilla.cloudflare-dns.com/dns-query ./curl-esni https://cloudflare.com/
The request traffic dump is presented in the screenshot below:
It can be seen that first curl accesses the mozilla.cloudflare-dns.com server using the DoH protocol (https connection to the server 104.16.249.249) to get public key values ​​for SNI encryption from them, and then to the destination server, hiding under this domain www.hello-rkn.ru .
In addition to the mozilla.cloudflare-dns.com resolver DoH specified above, we can use other popular DoH services, for example, from the famous evil corporation.
We execute the following request:
ESNI_COVER="www.kremlin.ru" DOH_URL=https://dns.google/dns-query ./curl-esni https://rutracker.nl/
And we get the answer:
< HTTP/1.1 301 Moved Permanently < Date: Sun, 03 Nov 2019 14:10:22 GMT < Content-Type: text/html < Transfer-Encoding: chunked < Connection: keep-alive < Set-Cookie: __cfduid=da0144d982437e77b0b37af7d00438b1a1572790222; expires=Mon, 02-Nov-20 14:10:22 GMT; path=/; domain=.rutracker.nl; HttpOnly; Secure < Location: https://rutracker.nl/forum/index.php < CF-Cache-Status: DYNAMIC < Expect-CT: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" < Server: cloudflare < CF-RAY: 52feee696f42d891-CPH
In this case, we turned to the blocked server rutracker.nl, using the dns.google DoH resolver (there is no typo, now the famous corporation has its own first level domain) and covered ourselves with another domain, which is strictly forbidden by all DPI to block under pain of death. According to the answer you can understand that our request was successfully processed.
As an additional check that the provider DPI responds to the open SNI, which we pass as a cover - we can fulfill the request to rutracker.nl behind some other forbidden resource, for example, another “good” torrent tracker:
$ ESNI_COVER="rutor.info" DOH_URL=https://dns.google/dns-query ./curl-esni https://rutracker.nl/
We will not receive a response from the server, as our request will be blocked by the DPI system.
A small conclusion to the first part
So, we were able to show the health of eSNI using openssl and curl and test the operation of the domain-based front-end based on eSNI. In the same way, we can adapt our favorite tools using the openssl library to work “under cover” of other domains. More on this in our next articles.