GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → T145 → black-mirror

T145 / black-mirror

Licence: AGPL-3.0 License
Automatically maintained malicious host blacklists and false-positive whitelists.

Programming Languages

shell
77523 projects
Dockerfile
14818 projects
python
139335 projects - #7 most used programming language
perl
6916 projects

Labels

dnsprivacywhitelistblacklistblacklist-datablocklistadblock-listwhitelistsblackhole-listsblacklist-ipsblacklist-scriptwhitelists-ipdaily-buildblacklist-feedblacklist-managementblacklist-domainsblacklist-servicewhitelists-hosts

Projects that are alternatives of or similar to black-mirror

unbound-dns-firewall
DNS-Firewall Python script for UNBOUND
Stars: ✭ 23 (-43.9%)
Mutual labels:  dns, whitelist, blacklist, blocklist
Filterlist-for-AdGuard-or-PiHole
A very aggressive filter-list that consolidates over 370 lists for use in AdGuard Home, Pi-Hole or similar.
Stars: ✭ 349 (+751.22%)
Mutual labels:  whitelist, blocklist, adblock-list
UnboundBL
🛑 DNSBL (adblock) on OPNsense with UnboundBL & Unbound DNS
Stars: ✭ 63 (+53.66%)
Mutual labels:  whitelist, blocklist, blacklist-management
WaGis-Mass-IP-Blacklister-Windows
This Tool blocks a Mass of IP Adresses via the Windows Firewall. You can run in Automatic-Mode using a Abusive IP List like blocklist.de
Stars: ✭ 34 (-17.07%)
Mutual labels:  blacklist, blocklist, blacklist-ips
Hosts Blocklists
Automatically updated, moderated and optimized lists for blocking ads, trackers, malware and other garbage
Stars: ✭ 1,749 (+4165.85%)
Mutual labels:  dns, blacklist, blocklist
accomplist
ACCOMPLIST - List Compiler
Stars: ✭ 51 (+24.39%)
Mutual labels:  whitelist, blacklist, blocklist
DNS-Sinkhole-Lists-A2
A DNS Sinkhole List for testing purposes. (not for use in production systems) - UUID: 0f90ca2c-4b0a-4fbe-b659-449ab30c4284
Stars: ✭ 19 (-53.66%)
Mutual labels:  dns, whitelist, blacklist
Spotify Adblock Linux
Spotify adblocker for Linux
Stars: ✭ 641 (+1463.41%)
Mutual labels:  dns, whitelist, blacklist
blackip
IP Blocklist for Ipset / Squid-Cache
Stars: ✭ 81 (+97.56%)
Mutual labels:  blacklist, blocklist, blacklist-ips
domainos
A laravel package for blacklisting or whitelisting specific email providers or domains
Stars: ✭ 22 (-46.34%)
Mutual labels:  whitelist, blacklist
hosts.whitelists
Collection of whitelists hosts. See also hosts.extras [ https://git.io/J3b5A ] and StevenBlack's hosts project [ https://git.io/Je0XR ]
Stars: ✭ 27 (-34.15%)
Mutual labels:  whitelist, whitelists-hosts
Windows.10.DNS.Block.List
Windows DNS Block List
Stars: ✭ 18 (-56.1%)
Mutual labels:  dns, blocklist
Nogo
A cross-platform network-wide ad/site blocker with a simple web control panel.
Stars: ✭ 143 (+248.78%)
Mutual labels:  dns, blacklist
1hosts
DNS filter-/blocklists | safe. private. clean. browsing!
Stars: ✭ 85 (+107.32%)
Mutual labels:  dns, blacklist
ipblacklist-database
Blacklist ip addresses caught scanning or bruteforcing hosts
Stars: ✭ 18 (-56.1%)
Mutual labels:  blacklist, blacklist-ips
jsCast
📻 An Audio Streaming Application written in JavaScript
Stars: ✭ 23 (-43.9%)
Mutual labels:  whitelist, blacklist
magento-2-security
Magento 2 Security extension FREE. Security extension gives store owners the ability to detect the IP addresses that are intentionally attacking their store at any given time. Therefore, they have timely measures to prevent this issue such as blocking those IP addresses or sending warning emails to store owners.
Stars: ✭ 40 (-2.44%)
Mutual labels:  whitelist, blacklist
AntiBot
Lightweight BungeeCord plugin that aims to stop attacks on your server quickly and efficiently.
Stars: ✭ 42 (+2.44%)
Mutual labels:  whitelist, blacklist
afdns
Ad free DNS server. A docker container with a DNS server configured to block advertisement hosts.
Stars: ✭ 27 (-34.15%)
Mutual labels:  dns, blacklist
Ioc2rpz
ioc2rpz is a place where threat intelligence meets DNS.
Stars: ✭ 67 (+63.41%)
Mutual labels:  dns, blacklist
View All Similar Projects ➔
logo

🌓 Reflection | 💿 Redundancy | ✅ Reliability

Automatically maintained malicious host blacklists and false-positive whitelists.

humane-tech last_updated visits release release release docker

🛡️ Privacy Protectors

🖋️ Manifesto

Defines the logic behind why a host is permitted or blocked. Please report any hosts that are wrongly blocked or sources that do not wholly align in an issue. Reference the contribution guidelines.

📋 Attributes

  1. Produced in domain-only, IPv4-only, IPv4-CIDR-only, and IPv6-only builds.
  2. Updates at 1:27 AM & PM UTC.
  3. No excess or trailing whitespace.
  4. No lingering webscraper garbage.
  5. Lines are terminated with lf.
  6. No blank lines.
  7. No comments.

🚚 Deliverables

⚓ Hyperlinks

List Name File Content Unique Entries File Size Mirror 1 Mirror 2
black_domain Domain entries 7,074,008 159M [✔️,🔗] [✔️,🔗]
black_ipv4 IPv4 addresses 738,750 10M [✔️,🔗] [✔️,🔗]
black_ipv4_cidr IPv4 CIDR blocks 23,096 387K [✔️,🔗] [✔️,🔗]
black_ipv6 IPv6 addresses 9,963 302K [✔️,🔗] [✔️,🔗]
white_domain Domain entries 21,964 334K [✔️,🔗] [✔️,🔗]
white_ipv4 IPv4 addresses 11,776 163K [✔️,🔗] [✔️,🔗]
white_ipv4_cidr IPv4 CIDR blocks 1,523 24K [✔️,🔗] [✔️,🔗]
white_ipv6 IPv6 addresses 2,799 107K [✔️,🔗] [✔️,🔗]

🧮 Checksum Evaluation

cat black_domain.txt | sha256sum -c black_domain.checksums --status && echo $?

A return code of 0 means the check was successful. The specific checksum command can be any of the following:

  • md5sum
  • b2sum
  • sha1sum
  • sha224sum
  • sha256sum
  • sha384sum
  • sha512sum

🐙 Fetching GitHub Releases

Provided below are some examples to fetch release artifacts leveraging the GitHub API.

Get all build artifacts

curl --proto '=https' --tlsv1.3 -H 'Accept: application/vnd.github.v3+json' -sSf https://api.github.com/repos/T145/black-mirror/releases/latest | jq -r '.assets[].browser_download_url'

Get a build artifact & its checksum

curl --proto '=https' --tlsv1.3 -H 'Accept: application/vnd.github.v3+json' -sSf https://api.github.com/repos/T145/black-mirror/releases/latest | jq -r '.assets[] | select(.name | startswith("black_domain")).browser_download_url'

Get a single build artifact

curl --proto '=https' --tlsv1.3 -H 'Accept: application/vnd.github.v3+json' -sSf https://api.github.com/repos/T145/black-mirror/releases/latest | jq -r '.assets[] | select(.name | startswith("black_domain")) | select(.name | endswith(".txt")).browser_download_url'

🛠️ Usage

Desktop OS Hosts File

mawk '{print "0.0.0.0 " $0}' black_domain.txt >>hosts
# mawk '{print ":: " $0}' black_domain.txt >>hosts
mawk '{print "0.0.0.0 " $0}' black_ipv4.txt >>hosts
mawk '{print ":: " $0}' black_ipv6.txt >>hosts

dnsmasq

Many popular platforms such as OpenWRT, DDWRT, and Pihole use DNSmasq as their choice TCP powerhouse. After inspecting many domain blocklists you'll inevitably run across a list in the dnsmasq.conf format. This list doesn't support it because you can use the addn-hosts parameter to add hosts in the list. Target a file that has the hosts in a format similar to the Desktop OS Hosts File format.

If you're using the RADVD daemon, prepend any listed hosts with ::. Otherwise, even if you have IPv6 support set up, prepend hosts with 0.0.0.0.

This has been tested across all the mentioned platforms using dig{6} on a small sample size and had each host null-routed successfully. DNSmasq's man page discusses configuration further, and DDWRT's ad blocking wiki page provides some examples.

Amazon EC2 DNS Resolver

Follow this guide to create a DNS server on a Amazon EC2 instance.

pihole

If you'd like to update when some sources do or not extract a production build, just use the single-line list sources.pihole. Note that this list only contains Pihole-compatible sources, and not every handled source. Some manual configuration may also be required.

unbound

Similar to dnsmasq, but requires more manual configuration. Name any products as a *.conf file. Then follow Steffinstanly's instructions on how to apply blocklists.

personalDNSfilter

Use the domain list.

📚 Sources

Please report any redundant sources in an issue!

⬛ Blacklists

⬜ Whitelists

Applied to generated blacklists.

🥢 Duplicates

Sources that contain duplicate and potentially deprecated data.

🧟 Zombies

Sources that are dead or deprecated and not included but may be worth mentioning.

🕵🏻 Lamers Unwelcome

📦 Big Data Lists

Typically used by other blacklist projects as whitelists.

List Name URL
Alexa https://s3.amazonaws.com/alexa-static/top-1m.csv.zip
Umbrella https://downloads.majestic.com/majestic_million.csv
Statvoo https://statvoo.com/dl/top-1million-sites.csv.zip
OpenPageRank https://www.domcop.com/files/top/top10milliondomains.csv.zip
Quantcast https://toplists.net.in.tum.de/archive/quantcast/quantcast-top-sites-2021-06-25_0900_UTC.txt.xz
Tranco list https://tranco-list.eu/

🌐 IP Block Providers

Simply provide IP blocks for entire geographic regions.

🎶 Notes

R Language

Docker installs

RUN apt-get -y install r-base

# install libarchive manually since libarchive-dev is at version 3.4.3
# https://github.com/libarchive/libarchive/wiki/BuildInstructions#using-configure-for-building-from-the-command-line-on-linux-freebsd-solaris-cygwin-aix-interix-mac-os-x-and-other-unix-like-systems
# https://www.zhouchun.net/blog/show/439 (run all commands together to prevent spawning subcontainers)
RUN aria2c https://github.com/libarchive/libarchive/releases/download/v3.5.2/libarchive-3.5.2.tar.gz \
&& tar xzf libarchive-3.5.2.tar.gz \
&& cd libarchive-3.5.2 \
&& ./configure \
&& make \
&& make check \
&& make install \
&& cd .. \
&& rm libarchive-3.5.2.tar.gz

# install R libarchive bindings
# https://github.com/r-lib/archive
RUN echo 'install.packages("archive", repos="https://cloud.r-project.org/")' | R --vanilla \
&& echo 'install.packages("data.table", repos="https://cloud.r-project.org/")' | R --vanilla

Boosting speeds

Need for speed

References

Together we'll make a better internet!

A project by T145 made with 💖
Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].