2287 Open source projects that are alternatives of or similar to Scrapoxy

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Node.js scraper to get data from Google Play

CORS compliant API to access Instagram's public data

获取免费socks/https/http代理的网站集合

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

News, full-text, and article metadata extraction in Python 3. Advanced docs:

Python library for retrieving free proxies (HTTP, HTTPS, SOCKS4, SOCKS5).

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

Dynamic meta tags in your AngularJS single page application

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

This repository contains all the code I use in my YouTube tutorials.

🥄 A package for building specific Proxy Pool for different Sites.

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Scrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok

pip安装的天眼查爬虫API，指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.

Elegant Scraper and Crawler Framework for Golang

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

A Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.

码云仓库链接:AJay13/ECommerceCrawlers Github 仓库链接:DropsDevopsOrg/ECommerceCrawlers 项目展示平台链接:http://wechat.doonsec.com

☁️ Modern & minimalistic load balancer for the Сloud era

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Araignée LeBonCoin avec Scrapy et ElasticSearch

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

An HTTP router and reverse proxy for service composition, including use cases like Kubernetes Ingress

Az összes számlád még egy helyen :)

PTT 網路版爬蟲

我的爬虫合集

Free proxy server, continuously crawling and providing proxies, based on Tornado and Scrapy. 免费代理服务器，基于Tornado和Scrapy，在本地搭建属于自己的代理池

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

swiss army knife for hackers

An extremely fast web scraper that parses megabytes of invalid HTML in a blink of an eye. PHP5.3+, no dependencies.

The Zipru scraper developed in the Advanced Web Scraping Tutorial.

spring-cloud-microservice-examples

Crawl BookCorpus

Creating Scrapy scrapers via the Django admin interface

Indonesia Index News Crawler, including 10 online media

use multiple proxies with Scrapy

Declarative web scraping

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

A framework for creating semi-automatic web content extractors

Golang pkg to quickly return a preview of a webpage (title/description/images)

Distributed crawler powered by Headless Chrome

An R web crawler and scraper

A high performance web crawler in Elixir.

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

Life is short, *****.

Parser and database to index the terpene profile of different strains of Cannabis from online databases

A scrapy spider that scraps times and prices from Voyages Sncf. It uses scrapyrt to provide an API interface.

Scraping scripts for various websites.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

A multi-thread crawler framework with many builtin image crawlers provided.

Update Version of weibo_terminator, This is Workflow Version aim at Get Job Done!

Simple but useful Python web scraping tutorial code.

PacBot (Policy as Code Bot)

hproxy - Asynchronous IP proxy pool, aims to make getting proxy as convenient as possible.(异步爬虫代理池)

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

A flexible commandline tool for template rendering. Supports lots of local and remote datasources.

A Cloud native multi-package registry