Website Finder - Search for websites by HTML, http codes and more

Own an SEO or Affiliate Marketing blog? - I'm looking to partner up with some SEO/IM influencers and give away my tools as lead magnets on their websites. Yes thats right, you can give this tool away for free. Learn more.

What is Website Finder?

Website Finder is a 100% free software tool for searching for websites by your set criteria. By default the homepage of each website is checked for the criteria but the URL that is returned from Google searches can also be checked.

What criteria is available?

  • HTML - does the website(s) contain certain HTML
  • HTTP codes - does server return a specific type of HTTP code e.g a 404 HTTP error code
  • HTTP error code type - does the website(s) return an certain error code type, eg failure, not allowed, redirect
  • Websites that have Google analytics installed - does the website(s) have the Google Analytics javascript on its homepage
  • Websites where meta data contains specific text - does the homepage title, keywords, and description contain specific text
  • Websites where title tag contains specific text
  • Websites where title tag is missing
  • Websites that are using WordPress
  • Websites that are using Joomla
  • Websites that are using Magento
  • Websites where domain contains specific text
  • Websites where URL contains specific text
  • Websites that are linking to specific outbound domains
  • Websites where page load time is over X amount of seconds
  • Websites where the HTTP responce header contains specific text - e.g. Apache 2.1 / PHP 4.3

How does this software work?

Website Finder has three methods of finding websites matching your criteria.

Crawl a list of websites for matching criteria

If you have a list of websites that you would like to check simply paste them in and Website Finder will check the homepage of each one for your set criteria.

Crawl search results by niche related keywords for matching websites/criteria

Enter a list of niche related keywords and Webiste Finder will search Google for all these niche related keywords and check every result (website) for your set criteria. There are two main options with this search, the criteria check can be applied to the result (URL) Google returns or the homepage of the result that is returned.

Endless crawl for matching websites

Enter your seed list of websites to start crawling from and Website Finder will check that inital seed website(s) for your criteria but then also crawl a specified amount of URLs on that website (30 by default) extracting all outbound links and then crawling these outbound websites. This process is repeated endlessly. Website Finder will just keep crawling from website to website checking to see if they match your set criteria.

Why should I use this tool?

List of possible uses are but not limited to...

  • Find websites with outdated plugins to market to
  • Find slow websites to sell faster hosting to
  • Find websites with security issues to offer security patching services to
  • Find websites to sell themes to, e.g use the Google search option to find all dog related websites running WordPress and sell them dog related WordPress themes

Tips for use

  • If you are checking websites that are being returned from a Google scrape make sure you have a sensible delay when sending requests to Google to avoid your IP adderess being blocked
  • See the most Pinterest pinned pages for a keyword.
  • More threads doesn't always mean faster, there will be a point depending on your internet connection and CPU/RAM where you'll hit your fastest speed. For me on village broadband with an i7 this is 3 threads per website and 40 concurrent websites.
  • When doing an endless crawl make sure you pick a website where many outbound links can be found to crawl next, such as reddit.

What does the software look like?

Website Finder

Watch Website Finder in action / Tutorial

Absolutely CRUSH your competitors on Google with SEMRush

Here is something I just have to share with you guys. SEMRush are giving away a free 7 day trial of their truly amazing SEO tool.

I highly recommend trying out their free trial. Here are just some of the awesome capabilities of this service.

  • Steal your competitors keywords - With SEMrush you can uncover your competitors organic and paid search strategies (keywords)
  • Steal your competitors backlinks - You can view your competitors' backlink profile so you can simply find where they are getting their links from and copy them!
  • Find new keywords - Find keywords you'd never of thought of that will bring you tons of traffic
  • Find search volume for keywords - See how many people are searching keywords before you start building pages for those keywords
  • Find keyword difficulty - Know if you are going to be able to rank for a keyword before you even start
  • Find toxic links - Find links that are pointing to your website that might be damaging your rankings or risking you getting removed from Google
  • On page SEO checker - Find ways to boost your pages rankings by tweaking them to search engine optimisation perfection
  • Social media tools - It doesn't just stop at SEO, they've now got some great social media tools too!

UPDATE: I've done video about SEMRush showing how you can do all of the above and more. How To Steal Your Competitors Web Traffic with SEMRush

Don't just take my word for it, enter your website in the box below and see just how powerful it is...

Change log

  • 31/07/2017 - Initial release