438 Open source projects that are alternatives of or similar to scrapy-cloudflare-middleware

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Common interface for data container classes

My Cloudflare Custom Pages ,It is Beautiful!

python+scrapy+splash 爬取拉勾全站职位信息

Raw scrapings of ARPEN https://transparencia.registrocivil.org.br/

Inventus is a spider designed to find subdomains of a specific domain by crawling it and any subdomains it discovers.

A scrapy middleware to use rotated proxy ip list.

虎扑步行街爬虫

Development repository for nginx-more package

No description or website provided.

Scraping the real estate agencies for up-to-date house listings as soon as they arrive!

Araignée LeBonCoin avec Scrapy et ElasticSearch

CloudFlare-DDNS-Script Support LEDE/Openwrt with IPv4/IPv6

使用asyncio和aiohttp开发的轻量级异步协程web爬虫框架

Download all video lectures from a MIT-OCW course with a single command.

A 1.1.1.1 DNS resolver built for Discord

Scrapy downloader middleware that stores response HTMLs to disk.

Analyze scraped data

Serverless Discord slash commands bot for Cloudflare's Workers.

A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)

Crawling zhihu, jobbole, lagou by Scrapy, and using Elasticsearch+Django to build a Search Engine website --- README_zh.md (including: implementation roadmap, distributed-crawler and coping with anti-crawling strategies).

Reference demo and modified PostgreSQL driver to connect Cloudflare Workers to a relational database.

Bash-powered DDNS client for Cloudflare-managed domain

🏡 A NodeJS server to scrape Pararius listings and show them in a dashboard 🏘️

DDNS with Cloudflare

The real cost of each TLD (top-level-domain). Find out how much your registrar marks up your domain prices.

Bug's feed is a local hosted portal where you can search for the latest news, videos, CVEs, vulnerabilities...

With Selenium headless browsing and CAPTCHA solving

Crawling the data from lazada, websosanh, compare.vn, cdiscount and cungmua with flexible configs

Raspagem de dados para iniciante usando Scrapy e outras libs básicas

python爬虫项目集合

A set of tools to work on Akamai v1 anti-bot solution. Current supported version: 1.70

Iota is a web scraper which can find all of the images and links/suburls on a webpage

💰用甚嚒服务器，ServerLess搭建一个图片分享站点！| 基于CloudFlareWorker无服务器函数和IPFS去中心化存储的图片分享网站

Dynamic configurable crawl (动态可配置化爬虫)

scrapy mysql pipeline

A script to update your Cloudflare DNS records at a glance.

Domain Name Search untuk mencari ketersedian nama domain.

Using scrapy to scrape tripadvisor in order to get users' reviews.

Classificador de poemas do Fernando Pessoa de acordo com os seus heterônimos

🚦 Block malicious crawlers with Cloudflare Firewall Rules

日常爬虫

Pagser is a simple, extensible, configurable parse and deserialize html page to struct based on goquery and struct tags for golang crawler

A Scrapy middleware for scraping time series data from Archive.org's Wayback Machine.

World’s single largest Internet domains dataset

Serverless API for requesting images from Unsplash's API, designed for use with a React frontend

A super lightweight router (1.3K) with middleware support and ZERO dependencies for CloudFlare Workers.

A chrome extension that purges the CF cache for the current URL of CloudFlare enabled websites

Helmut Hoffer von Ankershoffen experimenting with auto-provisioned RPi cluster running K8S on bare-metal

A Scrapy extension to log items coverage when the spider shuts down

CloudFlare ddns / 树莓派IPV6 DDNS解决方案

Automatically generated referrer bonuses for Cloudflare WARP (https://1.1.1.1)

Powerful DDoS Attack

Library and cli for extracting data from HTML via CSS selectors

Proxy server to bypass Cloudflare protection

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Cloudflare Tunnel Instructions and Template for Unraid

bing、google、baidu搜索引擎爬虫。python3.6 and scrapy

简单、免费、高效的百度地图poi采集和分析工具。

Distributed crawling/scraping, Kafka And Redis based components for Scrapy