888 Open source projects that are alternatives of or similar to torchestrator

Proxy [Finder | Checker | Server]. HTTP(S) & SOCKS 🎭

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

💪 Ruby / JRuby / TrufflleRuby gem & CLI for dealing with proxy lists from various sources

Hi there will be a lot of proxies here.

Asynchronous Library made using Python and aiohttp to get proxies from multiple services!

Auto-updated SOCKS5 proxy list + proxies for Telegram

A simple tool for fetching usable proxies from several websites.

proxy checker to check the status of the ip-port proxy list

HTTP(S) Rotating Residential proxies - Code examples & General information

Command line program to download documents from web portals.

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

（迫于压力，本项目停止维护，请尽快fork代码。1月1日之后删除项目）[免翻墙工具]A free and open-source youtube video proxy script [Written in PHP]

免费、高速的 V2Ray 代理和订阅。

Generate credentials for Opera's "browser VPN"

🐳 Tiny Docker(🤏 10MB) image as 🧅 Tor SOCKS5 proxy 🛡

Building a model to recognize incentives for landscape restoration in environmental policies from Latin America, the US and India. Bringing NLP to the world of policy analysis through an extensible framework that includes scraping, preprocessing, active learning and text analysis pipelines.

📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

A Python Tool For google Hacking

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

Crawlera middleware for Scrapy

Build and deploy a machine learning app from scratch 🚀

Scrape and take screenshots of dynamic and static webpages

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

A SOCKS, HTTP and DNS proxy for distributing traffic across multiple instances of Tor

A list of free, public, forward proxy servers. UPDATED DAILY!

🌈Wenshu_Spider-Scrapy框架爬取中国裁判文书网案件数据(2019-1-9最新版)

List of libraries, tools and APIs for web scraping and data processing.

An incredibly fast proxy checker & IP rotator with ease.

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

a class that uses scraped proxies to make http GET/POST requests (Python requests)

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

Containerized pool of multiple Tor instances with load balancing and HTTP proxy.

An easy to use open-source, multithreaded Proxy Checker. Allows you to quickly check HTTP and SOCKS proxies in a user friendly GUI, for Windows, Mac OS, Linux.

API for scrapping common meme sites

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

A framework for creating semi-automatic web content extractors

🔥Free proxy servers list / Updated hourly!

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Creating Scrapy scrapers via the Django admin interface

⭐️ A proxy scraper made using Protractor | Proxy list Updates every three hour 🔥

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

**[ARCHIVED]** website changes tracker 🔍

Python library for IP2Proxy database lookup. It can be used to find the IP addresses which are used as VPN anonymizer, open proxies, web proxies and Tor exits.

Tiny Library for get random proxy (free).

Simple but useful Python web scraping tutorial code.

With Selenium headless browsing and CAPTCHA solving

ProxyCrawl Python library for scraping and crawling

A Scrapy extension to log items coverage when the spider shuts down

IP2Proxy Java Component.It can be used to find the IP addresses which are used as VPN anonymizer, open proxies, web proxies and Tor exits.

Asynchronous proxy checker

Check if a IP is from tor or is a malicious proxy

Rules-based transparent HTTP/HTTPS proxy for the TOR server.

Declarative web scraping

A test suite of common scraper detection techniques. See how detectable your scraper stack is.