1326 Open source projects that are alternatives of or similar to Free_proxy_website

🍿爬虫代理IP池(proxy pool) python🍟一个还ok的IP代理池

🥄 A package for building specific Proxy Pool for different Sites.

Python爬虫代理IP池(proxy pool)

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

B站用户爬虫 好耶~是爬虫

带你了解一下Golang的市场行情

Sample of using proxies to crawl baidu search results.

Golang爬虫 爬取豆瓣电影Top250

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

A high performance web crawler in Elixir.

Dark Web OSINT Tool

Html网页正文提取

Crawly, a high-level web crawling & scraping framework for Elixir.

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Python的基础练习代码与各种爬虫代码

Free Web Scraping Tool with Java

一个超级轻量的百度图片爬虫

🐾 Creeper - The Next Generation Crawler Framework (Go)

A Facebook crawler

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

自学入门 Python 优质中文资源索引，包含 书籍 / 文档 / 视频，适用于 爬虫 / Web / 数据分析 / 机器学习 方向

Powerful web scraping framework for Crystal

Golang爬虫 爬取汽车之家 二手车产品库

python crawler spider

爬取北大法宝网http://www.pkulaw.cn/Case/

OSINT Swiss Army Knife

🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

各种App、小程序、网站的请求签名或加密算法。 现已有：自如、小红书、蛋壳公寓、luckin coffee(瑞幸咖啡)、bangkokair(曼谷航空)

BaiduSpider，一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

A collection of awesome web crawler,spider in different languages

The Prime Cross Site Request Forgery (CSRF) Audit and Exploitation Toolkit.

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

API of DouYin for Humans used to Crawl Popular Videos and Musics

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

A multi-thread crawler framework with many builtin image crawlers provided.

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

姬长信API For Docker 一个基于多种编程语言开源免费不限制提供生活常用,出行服务,开发工具,金融服务,通讯服务和公益大数据的平台.

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic ignore patterns

Gospider - Fast web spider written in Go

a reliable high-level web crawling & scraping framework for Node.js.

💐 Full Amazon Automatic Download

Rust Web Crawler saving pages on Redis

Incredibly fast crawler designed for OSINT.

[DEPRECATED] Simple, flexible, delightful web crawler/spider package

hproxy - Asynchronous IP proxy pool, aims to make getting proxy as convenient as possible.(异步爬虫代理池)

Beanbun 是用 PHP 编写的多进程网络爬虫框架，具有良好的开放性、高可扩展性，基于 Workerman。

APIs for loginning some websites by using requests.

Scraping scripts for various websites.

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

Verify that a request is from Google crawlers using Google's DNS verification steps

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

抖音 SDK，数据采集，爬虫抓取不是梦

a puppeteer walker 🕷 🕸

GOPA, a spider written in Go.（NOTE: this project moved to https://github.com/infinitbyte/gopa ）

Async Python 3.6+ web scraping micro-framework based on asyncio

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion