HTTP Request smuggling - new approaches

On October 7, 2019, the director of research at PortSwigger (BurpSuite's manufacturer) published a study on new approaches to HTTP Request smuggling. With their help, he earned about $ 70,000 on bugbounty. In this article, we briefly find out the essence of the attack, tools, and also methods of researching web servers that are vulnerable to this vulnerability.

What is HTTP Request Smuggling

HTTP Request smuggling - an attack aimed at desynchronizing the frontend of the web server and the backend of the web server, as a result of which an attacker could smuggle an HTTP request past the frontend server. The picture from the original article is a good demonstration:

Such an attack can lead to a variety of consequences - the introduction of XSS in the session of other users, redirecting users to third-party resources, poisoning the server cache, similarity to SSRF, and a number of others.

In the 2019 Smuggling Incarnation of 2019, James Kettle exploited incorrect web server header processing

Transfer-Encoding: chunked
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

indicating that the message body will be transmitted in parts ( RFC ). Due to the fact that some web servers do not support chunked transmission, or handle the header differently, the frontend will “see” only one request, and the backend will recognize it as two. More details about the attack details can be found in the original article , there is also a practical task on which you can practice to find the vulnerability manually.

For a quick search, James developed a plug-in for BurpSuit, which accepts a request for input and creates a note on the service’s vulnerability (if any) at the output.

Examples of vulnerable web servers

I must say that the problem of smuggling and other vulnerabilities related to the operation of web servers has long been dealt with by another researcher under the nickname regilero . Over the past three years, he has published three articles describing the vulnerabilities he found in popular web servers, most of which are assigned medium and high criticality CVEs. Among the vulnerable servers are Apache Traffic Server, Jetty, Apsis.

In the wake of interest in the problem, another researcher, Nathan Davison, discovered a vulnerability in HAProxy that ignored an incorrectly formed header.

 Transfer-Encoding:[\x0b]chunked
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

and converted it to the following form:

 Transfer-Encoding: chunked
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

But the backend server - gunicorn, proxying the application on Flask, read the header, which provoked the vulnerability.

A little later, a number of other researchers discovered a vulnerability (assigned CVE-2019-16276) in the implementation of the http server golang - the server normalized the header if there was a space before the prelet.

Inquiry:

After processing by the server:

The vulnerability could be exploited if the frontend server ignored the header with a space, and used Content-Length to calculate the size of the request.

The Caddy web server written in Go was also vulnerable because it used the same net / http library. The developers confirmed that after updating GO and rebuilding the package, the problem disappears.

The author of this article found a similar problem in the lighthttpd server (no CVE was assigned). The screenshot shows that the server accepts and processes the header containing a space:

The developers do not completely agree with RFC 7230 (and the author, too), and believe that the responsibility for the incorrect processing of headers lies with proxies that forward requests without normalizing and checking them. However, in the new release the bug will be fixed:

By default, lighttpd parses (and normalizes) requests before reverse-proxying them to backends. Doing so thwarts the attacks mentioned in https://portswigger.net/research/http-desync-attacks-request-smuggling-reborn to servers upstream from lighttpd.

However, as mentioned by stbuehler above, proxies downstream from lighttpd might pass anything to lighttpd.

The change that will be made in the next release of lighttpd will be to reject requests with space or tab after field-name and before the colon, but only when lighttpd is configured in the (default) mode of strict http header parsing.

The same bug was found in the cheroot web server used by the cherrypy framework. This mini-framework is found in startups, it is often used to write APIs. The bug report hangs in the opened status.

Necessary and sufficient conditions for vulnerability

So, what are the necessary conditions for checking and exploiting the vulnerability:

• POST request. Although the RFC does not explicitly prohibit the use of Content-Length and Transfer-Encoding headers for GET requests, in fact they are used only in POST requests.
• The presence of frontend and backend servers - if there are none, there will be nothing to synchronize.
• Web servers must parse the Transfer-Encoding header differently, that is, one must "read" it, and the second should be ignored.

Laboratory testing

To better understand and troubleshoot existing web servers and proxies, it is wise to deploy a test environment locally using Docker.

An example of a test environment diagram:

Application Code:

 from flask import Flask, request, jsonify app = Flask(__name__) @app.route('/', methods=['GET', 'POST']) def main(): # the next line is required for Transfer-Encoding support in the request request.environ['wsgi.input_terminated'] = True headers = {} for header in request.headers: headers[header[0]] = header[1] print (request.data) print (headers) return jsonify(body=str(request.data), headers=headers)
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

File with frontend settings, for example, caddy:

 localhost:80 log ../access.log proxy / host.docker.internal:8888
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Then everything is simple, run the application:

 gunicorn --keep-alive 10 -k gevent --bind 0.0.0.0:8888 -w 4 backend:app
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And container:

 docker run -d -t --name caddy -p 80:80 -p 443:443 -v /Users/sun/work/caddyfile:/etc/Caddyfile abiosoft/caddy:latest
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

From the Burp store we install HTTP Request Smuggler and Logger ++ for debugging convenience. Next in Repeater we form a simple request, for example this:

 POST / HTTP/1.1 Host: localhost Content-Length: 8 Connection: close body=123
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

And send it to check that everything is configured correctly:

 HTTP/1.1 200 OK Content-Length: 202 Content-Type: application/json Date: Mon, 07 Oct 2019 13:17:18 GMT Server: Caddy Server: gunicorn/19.9.0 Connection: close {"body":"b'body=123'","headers":{"Accept-Encoding":"gzip","Connection":"close","Content-Length":"8","Host":"host.docker.internal:8888","User-Agent":"Go-http-client/1.1","X-Forwarded-For":"172.17.0.1"}}
      
      

        
        
        
      

    
        
        
        
      
      

        
        
        
      

    
     

Now launch Launch Smuggle Probe and look at the answers.

The most interesting begins at this moment. It is necessary to analyze requests and responses in order to understand if services are vulnerable or not. This part is left for the inquisitive reader.