BlackIP

BlackIP is a project that collects and unifies public blocklists of IP addresses, to make them compatible with Squid and IPSET (Iptables Netfilter)

BlackIP es un proyecto que recopila y unifica listas públicas de bloqueo de direcciones IPs, para hacerlas compatibles con Squid e IPSET (Iptables Netfilter)

DATA SHEET

GIT CLONE

git clone --depth=1 https://github.com/maravento/blackip.git

HOW TO USE

blackip.txt is already optimized. Download it and unzip it in the path of your preference / blackip.txt ya viene optimizada. Descárguela y descomprimala en la ruta de su preferencia

Download

wget -q -N https://raw.githubusercontent.com/maravento/blackip/master/blackip.tar.gz && cat blackip.tar.gz* | tar xzf -

Checksum

wget -q -N https://raw.githubusercontent.com/maravento/blackip/master/checksum.md5
md5sum blackip.txt | awk '{print $1}' && cat checksum.md5 | awk '{print $1}'

Ipset Rules

Ipset allows us to perform mass filtering, at a processing speed far superior to other Solutions (See the benchmark) / Ipset permite realizar filtrado masivo, a una velocidad de procesamiento muy superior a otras soluciones (Vea el benchmark).

Edit your Iptables script and add the following lines: / Edite su script de Iptables y agregue las siguientes líneas:

ipset=/sbin/ipset
iptables=/sbin/iptables
$ipset -F
$ipset -N -! blockzone hash:net maxelem 1000000
for ip in $(cat /path_to/blackip.txt); do
    $ipset -A blockzone $ip
done
$iptables -t mangle -A PREROUTING -m set --match-set blockzone src -j NFLOG --nflog-prefix 'blackip'
$iptables -t mangle -A PREROUTING -m set --match-set blockzone src -j DROP
$iptables -A FORWARD -m set --match-set blockzone dst -j NFLOG --nflog-prefix 'blackip'
$iptables -A FORWARD -m set --match-set blockzone dst -j DROP

Ipset Advanced Rules (with IPDeny)

Additionally, you can use the above rule to include full IP ranges of countries with IPDeny adding the countries of your choice. For example: / Adicionalmente, puede usar la regla anterior para incluir rangos de IPs completos de países con IPDeny agregando los países de su elección. Por ejemplo:

# IPSET BLOCKZONE (select country to block and ip/range)
# http://www.ipdeny.com/ipblocks/
ipset=/sbin/ipset
iptables=/sbin/iptables

# Download Zones
zone=/path_to_folder/zones
if [ ! -d $zone ]; then mkdir -p $zone; fi
wget -q -N http://www.ipdeny.com/ipblocks/data/countries/all-zones.tar.gz
tar -C $zone -zxvf all-zones.tar.gz >/dev/null 2>&1
rm -f all-zones.tar.gz >/dev/null 2>&1

# Ipset Advanced Rules
$ipset -F
$ipset -N -! blockzone hash:net maxelem 1000000
# Zones (e.g. China, Russia) + BlackIP
for ip in $(cat $zone/{cn,ru}.zone /path_to/blackip.txt); do
    $ipset -A blockzone $ip
done
$iptables -t mangle -A PREROUTING -m set --match-set blockzone src -j NFLOG --nflog-prefix 'blackip'
$iptables -t mangle -A PREROUTING -m set --match-set blockzone src -j DROP
$iptables -A FORWARD -m set --match-set blockzone dst -j NFLOG --nflog-prefix 'blackip'
$iptables -A FORWARD -m set --match-set blockzone dst -j DROP

Flush

In case of error or conflict, execute: / En caso de error o conflicto, ejecute:

ipset flush blockzone # (or: ipset flush)

Log

NFLOG: /var/log/ulog/syslogemu.log

chown root:root /var/log
apt -y install ulogd2
if [ ! -d /var/log/ulog/syslogemu.log ]; then mkdir -p /var/log/ulog && touch /var/log/ulog/syslogemu.log; fi
usermod -a -G ulog $USER

Squid Rule

Edit:

/etc/squid/squid.conf

And add the following lines: / Y agregue las siguientes líneas:

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
acl blackip dst "/path_to/blackip.txt"
http_access deny blackip

Important about BlackIP

• Should not be used blackip.txt in IPSET and in Squid at the same time (double filtrate) / No debe utilizar blackip.txt en IPSET y en Squid al mismo tiempo (doble filtrado)
• blackip.txt is a list IPv4. Does not include CIDR / blackip.txt es una lista IPv4. No incluye CIDR
• blackip.txt has been tested in Squid v3.5.x and later / blackip.txt ha sido testeada en Squid v3.5.x y posteriores

Squid-Cache Advanced Rules

blackip contains millions of IP addresses, therefore it is recommended: / blackip contiene millones de direcciones IP, por tanto se recomienda:

• Use bipextra.txt to add IP/CIDR that are not included in blackip.txt (By default it contains some Block CIDR) / Use bipextra.txt para agregar IP/CIDR que no están incluidas en blackip.txt (Por defecto contiene algunos Block CIDR)
• Use allowip.txt (a allow list of IPv4 IP addresses like a Hotmail, Gmail, Yahoo. Etc) / Use allowip.txt (una lista blanca de direcciones IPs IPv4 tales como Hotmail, Gmail, Yahoo. etc)
• Use aipextra.txt to add allowlists of IP/CIDRs that are not included in allowip.txt / Use aipextra.txt para agregar listas blancas de IP/CIDR que no están incluidas en allowip.txt
• By default blackip.txt does not exclude private or reserved ranges RFC1918. Use IANA (iana.txt) to exclude these ranges / Por defecto blackip.txt no excluye rangos privados o reservados RFC1918. Use IANA (iana.txt) para excluir estos rangos
• To increase security, close Squid to any other request to IP addresses / Para incrementar la seguridad, cierre Squid a cualquier otra petición a direcciones IP

### INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS ###

## ALLOW IP/CIDR ##
# Allow IP list (Optional)
acl allowip dst "/path_to/allowip.txt"
http_access allow allowip
# Allow IP/CIDR list (not included in allowip) (Optional)
acl aipextra dst "/path_to/aipextra.txt"
http_access allow aipextra
# IANA list (not included in allowip) (Optional)
acl iana dst "/path_to/iana.txt"
http_access allow iana

## BLOCK IP/CIDR ##
# Block IP/CIDR list (not included in blackip) (Optional)
acl bipextra dst "/path_to/bipextra.txt"
http_access deny bipextra
# Blackip
acl blackip dst "/path_to/blackip.txt"
http_access deny blackip

## DENY ALL IP ##
acl no_ip url_regex -i [0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}
http_access deny no_ip

UPDATE

⚠️ WARNING: BEFORE YOU CONTINUE

This section is only to explain how update and optimization process works. It is not necessary for user to run it. This process can take time and consume a lot of hardware and bandwidth resources, therefore it is recommended to use test equipment / Esta sección es únicamente para explicar cómo funciona el proceso de actualización y optimización. No es necesario que el usuario la ejecute. Este proceso puede tardar y consumir muchos recursos de hardware y ancho de banda, por tanto se recomienda usar equipos de pruebas

Blackip Update

The update process of blackip.txt is executed in sequence by the script bipupdate.sh / El proceso de actualización de blackip.txt es ejecutado en secuencia por el script bipupdate.sh

wget -q -N https://raw.githubusercontent.com/maravento/blackip/master/bipupdate/bipupdate.sh && chmod +x bipupdate.sh && ./bipupdate.sh

Dependencies

Update requires python 3x and bash 5x / La actualización requiere python 3x y bash 5x

pkgs='wget git subversion curl libnotify-bin idn2 perl tar rar unrar unzip zip python-is-python3 squid ipset'
if ! dpkg -s $pkgs >/dev/null 2>&1; then
  apt -y install $pkgs
fi

Important about BLackip Update

• tw.txt containing IPs of teamviewer servers. By default they are commented. To block or authorize them, activate them in bipupdate.sh. To update it use tw.sh / tw.txt contiene IPs de servidores teamviewer. Por defecto están comentadas. Para bloquearlas o autorizarlas activelas en bipupdate.sh. Para actualizarla use tw.sh
• You must activate the rules in Squid before using bipupdate.sh / Antes de utilizar bipupdate.sh debe activar las reglas en Squid
• Some lists have download restrictions, so do not run bipupdate.sh more than once a day / Algunas listas tienen restricciones de descarga, entonces no ejecute bipupdate.sh más de una vez al día
• During the execution of bipupdate.sh it will request privileges when needed / Durante la ejecución de bipupdate.sh solicitará privilegios cuando los necesite

Check execution (/var/log/syslog):

BLackip: Done 06/05/2019 15:47:14

AllowIP Update

allowip.txt is already updated and optimized. The update process of allowip.txt is executed in sequence by the script aipupdate.sh / allowip.txt ya esta actualizada y optimizada. El proceso de actualización de allowip.txt es ejecutado en secuencia por el script aipupdate.sh

wget -q -N https://raw.githubusercontent.com/maravento/blackip/master/bipupdate/wlst/aipupdate.sh && chmod +x aipupdate.sh && ./aipupdate.sh

SOURCES

BLOCKLISTS

Active

• Abuse.ch Feodo Tracker
• adservers yoyo
• BBcan177 minerchk
• BL Myip
• Blocklist
• Cinsscore
• Emerging Threats Block
• Emerging Threats compromised
• Firehold Forus Spam
• Firehold
• Greensnow
• IPDeny
• Malwaredomain IP List
• MyIP BL
• Open BL
• opsxcq proxy-list
• Project Honeypot
• Public-Intelligence-Feeds
• Rulez BruteForceBlocker
• Spamhaus
• StopForumSpam 180
• StopForumSpam Toxic CIDR
• TOR BulkExitList
• TOR Node List
• UCEPROTECT IP Blocklists / BACKSCATTERER.ORG Blocklist (includes: Level 1, Level 2, Level3)
• Ultimate Hosts IPs Blocklist
• Zeustracker

Inactive, Discontinued or Private

Recovered by Wayback Machine, debugged and added to: oldip.txt

• Malc0de IP Blocklist
• Maxmind
• OpenBL
• Ransomwaretracker
• The LashBack UBL

ALLOWLISTS

Active

Debugged and added to: aipextra.txt

• Amazon AWS
• Microsoft Azure Datacenter

Inactive, Discontinued or Private

Recovered by EOP, debugged and added to: aipextra.txt

• O365IPAddresses (No longer support. Read me)

Debug

• Allow IP/CIDR extra
• Allow IPs
• Block IP/CIDR Extra
• IANA
• Old IPs
• Teamviewer IPs
• Allow URLs

Worktools

• cidr2ip
• Debug IPs
• Teamviewer Capture

CONTRIBUTIONS

We thank all those who contributed to this project. Those interested may contribute sending us new "Blocklist" links to be included in this project / Agradecemos a todos aquellos que han contribuido a este proyecto. Los interesados pueden contribuir, enviándonos enlaces de nuevas "Blocklist", para ser incluidas en este proyecto

Special thanks to: Jhonatan Sneider

DONATE

BTC: 3M84UKpz8AwwPADiYGQjT9spPKCvbqm4Bc

BUILD

maravento.com is licensed under a Creative Commons Reconocimiento-CompartirIgual 4.0 Internacional License.

OBJECTION

Due to recent arbitrary changes in computer terminology, it is necessary to clarify the meaning and connotation of the term blacklist, associated with this project: In computing, a blacklist, denylist or blocklist is a basic access control mechanism that allows through all elements (email addresses, users, passwords, URLs, IP addresses, domain names, file hashes, etc.), except those explicitly mentioned. Those items on the list are denied access. The opposite is a whitelist, which means only items on the list are let through whatever gate is being used.

Debido a los recientes cambios arbitrarios en la terminología informática, es necesario aclarar el significado y connotación del término blacklist, asociado a este proyecto: En informática, una lista negra, lista de denegación o lista de bloqueo es un mecanismo básico de control de acceso que permite a través de todos los elementos (direcciones de correo electrónico, usuarios, contraseñas, URL, direcciones IP, nombres de dominio, hashes de archivos, etc.), excepto los mencionados explícitamente. Esos elementos en la lista tienen acceso denegado. Lo opuesto es una lista blanca, lo que significa que solo los elementos de la lista pueden pasar por cualquier puerta que se esté utilizando.

Source Wikipedia

Therefore / Por tanto

blacklist, blocklist, blackweb, blackip, whitelist, etc.

are terms that have nothing to do with racial discrimination / son términos que no tienen ninguna relación con la discriminación racial

DISCLAIMER

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.