Exploring Hidden Attack Surfaces: The Secret To Finding More Subdomains

Exploring Hidden Attack Surfaces is a new blog post series curated by Nova Security for the community to help you on your way to discover any attack surfaces that most current tools are not capable of.

Part 1 is dedicated to finding more subdomains so that you can have that extra edge over other researchers and directly also increase your chances of finding more security vulnerabilities.

The secret is using and combining as many sources as possible. The more sources you use to find subdomains, the better.
But let's go over what this means as it proves to be difficult in practice.

There are certainly multiple methods of getting subdomains. From scraping Google search results and other indexed content to pulling data from APIs, to actively bruteforcing host names and performing reverse IP lookups.

Let's sort them all out and help you build your ultimate methodology for subdomain enumeration. And keep on reading because, at the end of this post, I will show you a simple managed solution to help you track your targets periodically to stay up to date on new changes!

Scraping:

Let's start with the basics. Scraping your target should be one of the first methods to have on your list.

You should make sure to look for subdomains:

• In Response Headers (headers like the Content-Security-Policy header often contain subdomains in CSP directives)
• Response bodies (buttons, linked content such as images and videos, commented-out sources, etc.)
• Common configuration files like robots.txt and sitemap.xml sometimes contain references to external resources (subdomains can be scraped here as well)

In addition to actively scraping your target, you can also passively scrape your target through external sources like Google, Bing and DuckDuckGo using the provided search syntaxis. Some search engines can provide different data that others may not be able to, that is why it is important to make sure you check all of them.

Internet Archives, such as the Wayback Machine, can also contain references to old and often forgotten assets and subdomains and should also be added to your list.

Public APIs:

Using public APIs like CRT.SH, Clouflare's Certificate Transparency Log API and EFF's CERTBot can help you discover new subdomains that have been just added and a new SSL certificate issued.

And although, this is a super effective way of discovering new potential subdomains linked to your target. You should always make sure to combine in other methods as well as not every new subdomain, app or service added by your target gets a new certificate issued.

Bruteforcing:

Another way to enumerate, and potentially also the best and most complete way to perform subdomain enumeration is bruteforcing host names.

As simple as it sounds, you can actually bruterforce subdomains and check if any resolve to an IP. If it does, it's best to run a port scan on the discovered host and look for any open ports listening for HTTP connections.

There are several tools and wordlists online that can help you get started. Simple Recon System's subdomain scanner now does provide support for subdomain bruteforcing!

Automation:

Now that we've covered all the ways to enumerate subdomains, there's one more thing to add.

Daily recurring scans should be a top priority to track changes and get notified immediately. Most fail to stay consistent and you can take advantage of that.

There are several ways to automate your entire recon process. You can craft a small shell script that would batch together the results of several tools and run it as a cronjob on your machine.

Or you can make use of a managed system like our web app pentesting suite. The moment you sign up you'll get the option to import your existing target definitions and set up recurring scans. All in less than 3 minutes.

Moreover, you can also set up notifications to be notified upon a new host change has been detected. But that's not everything, it has several other features such as live host probing, screenshotting, and much more! Check it out yourself now:

This post was part 1 of the new small post series "Exploring Hidden Attack Surfaces". If you've learned something new feel free to share this post with your friends and/or colleagues!

BLACKBIRD Technologies is a security company that provides a cloud-based web app pentesting platform with tens of manual and automated web app pentesting tools, including a fully automated web vulnerability scanner!