644 Open source projects that are alternatives of or similar to proxi

Spin up Tor containers and then proxy HTTP requests via these Tor instances

A simple tool for fetching usable proxies from several websites.

Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS 🎭

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Simple but useful Python web scraping tutorial code.

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

🔥Free proxy servers list / Updated hourly!

Python编写的HTTP代理池，集成WEB管理界面，无外部数据库依赖，可直接运行，兼容Windows,Linux和macOS。Free Proxy Pool with Web UI on Windows, Linux, and macOS.

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Crawlera middleware for Scrapy

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

A curated list of free public proxy servers

An incredibly fast proxy checker & IP rotator with ease.

With Selenium headless browsing and CAPTCHA solving

Build and deploy a machine learning app from scratch 🚀

A framework for creating semi-automatic web content extractors

Creating Scrapy scrapers via the Django admin interface

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Antch, a fast, powerful and extensible web crawling & scraping framework for Go

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

Asynchronous proxy checker

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

a class that uses scraped proxies to make http GET/POST requests (Python requests)

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

A collection of awesome web scaper, crawler.

A Scrapy extension to log items coverage when the spider shuts down

Free proxy list, free proxy world - visit our website

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

免费 IP 代理池。Scrapy 爬虫框架插件

API for scrapping common meme sites

Parser and database to index the terpene profile of different strains of Cannabis from online databases

Raspagem de dados para iniciante usando Scrapy e outras libs básicas

JAVA实现的IP代理池，支持HTTP与HTTPS两种方式

This script is used to scrap images from the Internet to classify 5 common noodle "mee" dishes in Singapore. Wanton Mee, Bak Chor Mee, Lor Mee, Prawn Mee and Mee Siam.

Web crawling and scraping framework for Golang

Transparent filtering TLS proxy.

Your good mornings ☀️

📷 Automate full website screenshots and PDF generation with multiple viewport support.

No description or website provided.

A Node.js library to easily manage and rotate a pool of web browsers, using any of the popular browser automation libraries like Puppeteer, Playwright, or SecretAgent.

A Man-In-The-Middle Proxy as Fiddle and Charles, using Netty, JavaFX

Scrape and take screenshots of dynamic and static webpages

A tool for parsing Scrapy log files periodically and incrementally, extending the HTTP JSON API of Scrapyd.

An open source webapp for scraping: towards a public service for webscraping

Subdomain method that proxies websockets, XMLHttpRequests, and more.

Mastering the art of scrapping 🎓

[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3

Redirecting Load Balancer

The source code for animecenter

Command line program to download documents from web portals.

A PHP Scraping Class

Scrapy project boilerplate done right