Blocklist for newly created scam, phishing, and malware domains automatically retrieved daily using Google Search API, automated detection, and public databases.
Since the project began, the blocklist has expanded to include not only scam websites but also malware domains.
This blocklist aims to be an alternative to blocking all newly registered domains (NRDs) seeing how many, but not all, NRDs are malicious. This is done by detecting new malicious domains within a short period of their registration date.
For blocking all NRDs, use xRuffKez's NRD Lists.
Sources include:
- Public databases
- Google Search indexing to find common scam site templates
- Detection of common cybersquatting techniques like typosquatting, doppelganger domains, and IDN homograph attacks using dnstwist and URLCrazy
- Domain generation algorithm (DGA) domain detection using DGA Detector
- Regex expression matching for phishing NRDs. See the list of expressions here
A list of all sources can be found in SOURCES.md.
The automated retrieval is done daily at 16:00 UTC.
| Format | Syntax |
|---|---|
| Adblock Plus | ||scam.com^ |
| Wildcard Domains | scam.com |
This blocklist is integrated into Hagezi's Threat Intelligence Feed (full version). For extended protection, please use that list instead.
Total domains: 212195
Light version: 19232
New domains after filtering:
Today | Monthly | %Monthly | %Filtered | Source
8 | 11642 | 7 % | 25 % | 165 Anti-fraud
4 | 283 | 0 % | 12 % | Artists Against 419
79 | 2039 | 1 % | 1 % | Cybersquatting
885 | 30703 | 20 % | 0 % | DGA Detector
9 | 2206 | 1 % | 23 % | Emerging Threats
0 | 2540 | 1 % | 19 % | FakeWebshopListHUN
31 | 910 | 0 % | 3 % | Google Search
274 | 21911 | 14 % | 10 % | Jeroengui Phishing feed
3 | 162 | 0 % | 8 % | Jeroengui Scam feed
641 | 44561 | 29 % | 25 % | PhishStats
189 | 10784 | 7 % | 0 % | PhishStats (NRDs)
3 | 216 | 0 % | 4 % | PuppyScams.org
607 | 29320 | 19 % | 1 % | Regex Matching
1 | 372 | 0 % | 7 % | Scam Directory
0 | 48 | 0 % | 32 % | ScamAdviser
0 | 8 | 0 % | 4 % | StopGunScams.com
88 | 88 | 0 % | 0 % | Verbraucherzentrale Hamburg
0 | 231 | 0 % | 0 % | ViriBack C2 Tracker
2633 | 149047 | 100 % | 21 % | All sources
- %Monthly: percentage out of total domains from all sources.
- %Filtered: percentage of dead, whitelisted, and parked domains.
Dead domains removed today: 2005
Resurrected domains added today: 986
Parked domains removed today: 413
Unparked domains added today: 136
Domains over time (days)
Courtesy of iam-py-test/blocklist_stats.
- Domains are filtered against an actively maintained whitelist
- Domains are checked against the Tranco Top Sites Ranking for potential false positives which are then vetted manually
- Common subdomains like 'www' are stripped
- Non-domain entries are removed
- Redundant rules are removed via wildcard matching. For example, 'abc.example.com' is a wildcard match of 'example.com' and, therefore, is redundant and removed. Wildcards are occasionally added to the blocklist manually to further optimize the number of entries
Entries that require manual verification/intervention are notified to the maintainer for fast remediations.
The full filtering process can be viewed in the repository's code.
Dead domains are removed daily using AdGuard's Dead Domains Linter.
Dead domains that are resolving again are included back into the blocklist.
Parked domains are removed daily. A list of common parked domain messages is used to automatically detect these domains. This list can be viewed here: parked_terms.txt.
Parked sites no longer containing any of the parked messages are assumed to be unparked and are included back into the blocklist.
For collated blocklists cautious about size, a light version of the blocklist is available in the lists directory. Sources excluded from the light version are marked in SOURCES.md.
Note that dead and parked domains that become alive/unparked are not added back into the light version due to limitations in how these domains are recorded.
A blocklist for NSFW domains is available in Adblock Plus format here: nsfw.txt.
Details
- Domains are automatically retrieved from the Tranco Top Sites Ranking daily
- Dead domains are removed daily
- Note that resurrected domains are not added back
- Note that parked domains are not checked for
This blocklist does not just include adult videos, but also NSFW content of the artistic variety (rule34, illustrations, etc).
For list maintainers interested in using the parked domains as a source, the list of parked domains can be found here: parked_domains.txt. This list is capped at 50,000 domains.
- AdGuard's Hostlist Compiler: simple tool that compiles hosts blocklists and removes redundant rules
- Elliotwutingfeng's repositories: various original blocklists
- Google's Shell Style Guide: Shell script style guide
- Grammarly: spelling and grammar checker
- Hagezi's DNS Blocklists: various curated blocklists including threat intelligence feeds
- Jarelllama's Blocklist Checker: generate a simple static report for blocklists or see previous reports of requested blocklists
- ShellCheck: static analysis tool for Shell scripts
- VirusTotal: analyze suspicious files, domains, IPs, and URLs to detect malware (also includes WHOIS lookup)
- iam-py-test/blocklist_stats: statistics on various blocklists