Bypassing Web Filters Part 3: Domain Fronting

Bypassing Web Filters Part 3: Domain Fronting

/in

The last two blog posts in this series were about SNI spoofing and Host header spoofing. We also learned that the latter is addressed by some vendors with a technique called “Domain Fronting Detection”. But what exactly is domain fronting? This will be explained in this blog post.

Bypassing web filters blog post series overview:

Content Delivery Networks / Caching Servers

Let’s first discuss what a CDN is used for. A simple CDN could look like this:

Alice requests a file. The CDN server fetches it from the content servers and delivers it.

There are 3 content servers. These are used to serve content to users. Because these servers are not very powerful and only have a slow uplink, they would not be able to serve files to millions of users. Instead of adding more servers and more bandwidth to the content servers, a CDN server is placed in front of them, to offload the work.

The DNS records (e.g. b.example.net) of the content servers point to the CDN server (203.0.113.5). Now, when Alice wants to access a file from b.example.net, the request is sent to the CDN server’s IP address with both the SNI and Host header set to b.example.net. As long as this domain is registered at the CDN, the server then checks if the file is already in its cache and if not, fetches it from the respective content server.

If the same resource is requested again from the CDN server, the resource is directly loaded from the cache:

Bob requests the same file as Alice. The CDN server responds with the cached file.

This saves computing power and bandwidth on the content servers. The CDN servers can also exist simultaneously in multiple locations distributed over the world, for even better content delivery speed.

Bypassing Simple Web Filters Using Domain Fronting

Domain fronting is a technique used to bypass censorship and web filters1. This technique is not only useful to bypass web filters but also to enhance the legitimacy of traffic, allowing it to seamlessly blend into regular network activity.

It uses the fact that the hostname can be specified on two places in an HTTPS request. In the SNI during the TLS handshake, and in the Host header of the HTTP request, as we have already seen in the two previous blog posts.

As a short recap, in a legitimate CDN scenario, a user’s request contains the target hostname (the server behind the CDN) in both the SNI and in the Host header. With domain fronting, the SNI is set to a legitimate hostname (e.g. one that is allowed to be accessed), but the Host header is set to a server that should not be directly accessible.

For this to work, the server that handles this connection must be configured to correctly evaluate the hostname in the Host header and respond to it. This is where CDNs come into play, since that is essentially their job.

Imagine you have the following situation where Alice is able to access the CDN cdn.example.net which is used in turn to access legit.example.net. However, access to evil.example.com is blocked by the firewall based on the SNI evil.example.com:

Connection is blocked to evil.example.com

An attacker can now use the CDN network to “hide” their server behind it. Now, Alice (or a malware on her computer) can send a request to the allowed CDN server 203.0.113.5 with the legitimate SNI legit.example.net but with the Host header set to the blocked server evil.example.com. Because the CDN is configured accordingly, the request is forwarded to the evil.example.com server which allows Alice to access this otherwise blocked resource.

The evil system can be accessed via the CDN, using a legit SNI but evil hostname in Host header

The final IP, TLS and HTTP packet structure looks like this:

Domain Fronting

This looks similar to SNI spoofing (see the first part of this blog post series), but the connection is established to a legitimate CDN system and not directly to the blocked host.

Example Using Fastly CDN and curl

Choose Your CDN

Since domain fronting is a well-known technique, it does not work on all CDNs. There are several CDNs that check whether the hostnames in the SNI and Host header are different and then block such requests. This is the case for example on Azure2, Google3, or AWS4. In contrast, Fastly5 does (at the time of writing this blog) not block domain fronting.

Evil Server Configuration

This is the easiest part. A webserver was configured on evil.meta.securelogon.ch. that serves some content:

$ curl https://evil.meta.securelogon.ch
<h1>Hello from Compass</h1>

For this example, it is assumed that the access to this server is blocked.

Fastly Configuration

Let’s create a new CDN service in Fastly. Choose anything you want for the domain name, e.g. compass-test.global.ssl.fastly.net ①. This domain name is later used in the Host header. For the origin, the system evil.meta.securelogon.ch – which is under our control – is configured ②:

CDN service setup on Fastly

After this, you have a new domain name with the following DNS entry:

$ dig compass-test.global.ssl.fastly.net +noall +short
199.232.17.194

This domain name can now be used to access your system:

$ curl https://compass-test.global.ssl.fastly.net/
<h1>Hello from Compass</h1>

Side note: At this point we would already have achieved a simple bypass, as our newly registered domain may not be blocked by the web filter we try to bypass. However, this could easily be rectified by the operator of the filter, by simply blocking this new domain (for example because it is newly observed, or only accessed very infrequently).

But let’s get back to our setup. If we look at the request from the above curl command in details, you can see that both the SNI ① in the TLS handshake and the Host header ③ in the HTTP request ② contain the Fastly hostname compass-test.global.ssl.fastly.net:

Request to new CDN service

This is the typical way of accessing resources via a CDN and does not yet indicate, whether Fastly can be used for domain fronting. For this, we have to figure out, if Fastly also accepts requests where the SNI and Host header differ, and if serves the correct content.

Let’s use our previously configured hostname as Host header and set an arbitrary SNI (in this case even a non-valid domain for testing purposes). As you can from the response below, the server is indeed accessible:

$ curl -k -H "Host: compass-test.global.ssl.fastly.net" --connect-to invalid::199.232.17.194: https://invalid/
<h1>Hello from Compass</h1>

Note that in this curl request, certificate validation is disabled explicitly (-k). Since Fastly is not able to serve a valid certificate for https://invalid, our request would fail otherwise.

In Wireshark, the above request looks as follows. The SNI ① in the TLS handshake is set to invalid, but the the Host header ③ in the HTTP request ② contain the Fastly hostname compass-test.global.ssl.fastly.net:

Request to CDN via invalid SNI

This behavior tells us, that the Fastly CDN service:

  1. does not check whether the hostname in the SNI and Host header match
  2. delivers the content solely based on the Host header

Therefore, the Fastly CDN fulfills the basic requirements for domain fronting.

Finding Frontable Domains

As mentioned previously, domain fronting aims at bypassing web filters, but also at being stealthy. The stealth party mainly relates to the hostname used as SNI. Therefore, we now have to look for suitable domains.

Let’s define our requirements for these domains:

  • The hostname must be registered at the target CDN (in our case Fastly)
  • The hostname must not be blocked by the web filter in question
  • (Bonus) The hostname should not be suspicious in context of your target environment

If a domain uses a CDN to deliver its content, the DNS records for this domain point to one or more servers belonging to the CDN. There are several tools like FindFrontableDomain6 which help you to check if a domain is behind a known CDN, as well as lists that contain well-known frontable domains7. But let’s do this manually, so you can see the process behind.

First, you have to come up with existing hostnames that would suit the target infrastructure you are testing (remember that the domain should be allowed by the target web filter, and ideally should not raise suspicion). For a bank, you could e.g. enumerate the subdomains of some financial websites. As a starting point, you could use a public lists of widely used domains, for example from one8 of several GitHub repositories, or from DomCop9. To blend in with your target infrastructure, you could also enumerate hostnames of your target itself.

Next, we perform a DNS lookup on these hostnames and check if the DNS record points to the Fastly CDN. The following command uses dnsx10 to resolve the DNS entries. This is useful, since dnsx is also able to show which CDN is used:

$ dnsx -silent -resp -cname -a -cdn < possible-hostnames.txt
[...]
amazon.com [A] [205.251.242.103]                                                                                               
amazon.com [A] [54.239.28.85]                                                                                                  
creators.spotify.com [A] [151.101.194.133]  [fastly]
creators.spotify.com [A] [151.101.66.133]  [fastly]
creators.spotify.com [A] [151.101.2.133]  [fastly]
creators.spotify.com [A] [151.101.130.133]  [fastly]
dailymotion.com [A] [195.8.215.136]    
deezer.com [A] [13.224.103.100]  [cloudfront]                                                                                  
deezer.com [A] [13.224.103.83]  [cloudfront]                                                                                   
de.wikipedia.org [A] [185.15.58.224]                                                                                            
[...]                                                               
paypal.com [A] [151.101.195.1]  [fastly]                                                                                       
paypal.com [A] [151.101.3.1]  [fastly]                                                                                         
paypal.com [A] [162.159.141.96]  [fastly]                                                                                      
paypal.me [A] [151.101.65.21]  [fastly]                                                                                        
paypal.me [A] [162.159.141.96]  [fastly]                                                                                       
pbs.twimg.com [A] [104.18.37.127]  [fastly]                                                                                    
pbs.twimg.com [A] [172.64.150.129]  [fastly]                                                                                                                                              
trello.com [A] [3.165.190.47]                                                                                                  
[...]

Nice, there are several hostnames that resolve to the Fastly CDN. Sometimes, CDNs have multiple exposed systems and not all of them can be used to access the same targets. To check if our server with the domain compass-test.global.ssl.fastly.net can be reached, we can perform an HTTP request to all discovered hostnames and verify whether our self-hosted website (with the content “Hello”) can be accessed:

$ grep fastly dnsx_top10m_all.txt | cut -d ' ' -f 1 | sort -u | while read host
do
  curl -s -H "Host: compass-test.global.ssl.fastly.net" https://$host/ | grep -q "Hello" && echo $host
done
ameblo.jp
anchor.fm
creators.spotify.com
dictionary.com
linktr.ee
slate.com
smh.com.au
theatlantic.com

Note: These hostnames do not necessarily need to resolve to the exact same IP address as our Fastly domain, they just have to be part of the CDN:

$ dig creators.spotify.com +noall +answer
creators.spotify.com.   204     IN      A       151.101.130.133
creators.spotify.com.   204     IN      A       151.101.194.133
creators.spotify.com.   204     IN      A       151.101.66.133
creators.spotify.com.   204     IN      A       151.101.2.133

$ dig compass-test.global.ssl.fastly.net +noall +answer
compass-test.global.ssl.fastly.net. 36 IN A     199.232.17.194

Domain Fronting Using curl

Now, we can combine these elements to craft a request that includes a previously discovered hostname in the SNI while using our own Fastly domain in the Host header. Curl resolves creators.spotify.com to the IP address 151.101.2.133 and includes this hostname in the SNI. Since the CDN server manages this hostname, it provides a valid certificate for the TLS connection. However, the response we get is dictated by our malicious hostname in the Host header.

$ curl -v -H "Host: compass-test.global.ssl.fastly.net" https://creators.spotify.com/index.html
[...]

* SSL connection using TLS1.2 / ECDHE_RSA_CHACHA20_POLY1305
*   server certificate verification OK
*   server certificate status verification SKIPPED
*   common name: creators.spotify.com (matched)
    [...]
*   subject: CN=creators.spotify.com
    [...]
*   issuer: C=US,O=Certainly,CN=Certainly Intermediate R1

[...]
* Connected to creators.spotify.com (151.101.2.133) port 443
* using HTTP/1.x

> GET / HTTP/1.1
> Host: compass-test.global.ssl.fastly.net
[...]

< HTTP/1.1 200 OK
[...]

<h1>Hello from Compass</h1>

In Wireshark, we can retrace all steps of this connection. First, a DNS lookup is performed for the the legititimate hostname creators.spotify.com ①. Then the TLS connection is established with the legitimate hostname in the SNI ②. In the wrapped HTTP request ③, the Host header ④ has the value of compass-test.global.ssl.fastly.net:

Request to CDN using legitimate SNI but evil hostname in Host header

From a network perspective, the traffic appears legitimate for the destination host creator.spotify.com. However, it was possible to fetch a resource from another system which would otherwise be blocked.

Non-Working Example Using AWS CloudFront

As already mentioned, some CDNs block domain fronting. AWS CloudFront is one example.

To demonstrate this behavior, we use two hostnames that resolve to the same IP address of the AWS CloudFront CDN:

$ dnsx -silent -resp -cname -a -cdn < possible-hostnames.txt | sort -k3
[...]
marketwatch.com [A] [18.165.183.55]  [cloudfront]
nationalgeographic.com [A] [18.165.183.55]  [cloudfront]
[...]

When a request is made where the SNI does not match the hostname in the Host header, CloudFront detects this mismatch and blocks the request with the following error message:

$ curl --connect-to nationalgeographic.com::18.165.183.55: -H "Host: marketwatch.com." https://nationalgeographic.com/
[...]
<H1>421 ERROR</H1>
<H2>The request could not be satisfied.</H2>
The distribution does not match the certificate for which the HTTPS connection was established with.
[...]

Domain Fronting Protection and Limitations

Domain fronting can be detected by web filters – as with Host header spoofing – by comparing if the hostname in the SNI and in the Host header are identical. However, just by looking at the TLS traffic, it’s not possible to identify that another system is accessed, because both the destination IP address and the hostname in the SNI look legit, and even a valid certificate is used (the one matching the SNI).

In the previous part of this series about Host Header Spoofing, an example of this detection mechanism in Fortinet FortiGuard was already explained. Other products, such as Paloalto PAN-OS are also equipped with this feature11:

Domain fronting detection feature in PAN-OS

Takeaway

So this is another technique to bypass web filters that is worth testing in your environment or in your next penetration test.

Domain Fronting is quite similar to SNI spoofing, as it involves spoofing a legitimate hostname in the SNI. However, in this case, the connection is made to a legitimate system where the hostname in the SNI matches the Subject/SANs of the certificate, ensuring the presence of a valid certificate.

From an attacker’s perspective, it’s still no almighty bypass. If the proxy inspects the TLS traffic and performs proper checks, such attacks can be detected and prevented.

However, these prevention techniques are not golden and can sometimes be bypassed as well. This will be explained in the next blogpost.

References

  1. Blocking-resistant communication through domain fronting (paper, including presentation and video): https://www.bamsoftware.com/papers/fronting/ ↩︎
  2. Microsoft: Securing our approach to Domain Fronting within Azure: https://www.microsoft.com/en-us/security/blog/2021/03/26/securing-our-approach-to-domain-fronting-within-azure/ ↩︎
  3. Ars Technica: Google disables “domain fronting” capability used to evade censors: : https://arstechnica.com/information-technology/2018/04/google-disables-domain-fronting-capability-used-to-evade-censors/ ↩︎
  4. Enhanced Domain Protections for Amazon CloudFront Requests: https://aws.amazon.com/blogs/security/enhanced-domain-protections-for-amazon-cloudfront-requests/ ↩︎
  5. Fastly: https://www.fastly.com/ ↩︎
  6. GitHub: rvrsh3ll/FindFrontableDomain: https://github.com/rvrsh3ll/FindFrontableDomains/ ↩︎
  7. GitHub: vysecurity/DomainFrontingLists: https://github.com/vysecurity/DomainFrontingLists ↩︎
  8. GitHub: PeterDaveHello/top-1m-domains: https://github.com/PeterDaveHello/top-1m-domains ↩︎
  9. DomCop: https://www.domcop.com/top-10-million-websites ↩︎
  10. GitHub: projectdiscovery/dnsx: https://github.com/projectdiscovery/dnsx ↩︎
  11. Paloalto PAN-OS Documentation, Domain Fronting Detection: https://docs.paloaltonetworks.com/pan-os/10-2/pan-os-new-features/content-inspection-features/domain-fronting-detection ↩︎

Compass Security Blog – ​Read More