681 Open source projects that are alternatives of or similar to Scrapy Selenium

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

A Scrapy extension to log items coverage when the spider shuts down

A Scrapy Spider for downloading PDF files from a webpage.

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

An Instagram bot developed using the Selenium Framework

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Code for the second edition Web Scraping with Python book by Packt Publications

🚀电商网站爬虫合集，淘宝京东亚马逊等

小密圈备份，圈子/话题/图片/文件。

HTTP API for Scrapy spiders

爬虫学习仓库，适合零基础的人学习，对新手比较友好

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Python入门网络爬虫之精华版

With Selenium headless browsing and CAPTCHA solving

AlipaySpider on Scrapy(use chrome driver); 支付宝爬虫(基于Scrapy)

Simple but useful Python web scraping tutorial code.

cdp4j - Chrome DevTools Protocol for Java

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

No description or website provided.

Build and deploy a machine learning app from scratch 🚀

A curated list of awesome packages, articles, and other cool resources from the Scrapy community.

A complete test automation tool

swiss army knife for hackers

C#/.NET test automation framework for web

Crawly, a high-level web crawling & scraping framework for Elixir.

A non API python program to crawl public photos, posts or followers

use multiple proxies with Scrapy

ISP Data Pollution to Protect Private Browsing History with Obfuscation

A next generation, full-stack acceptance testing framework optimised for collaboration, speed and scale!

🎊 Design and implement of lightweight crawler framework.

Redis-based components for Scrapy.

Declarative web scraping

WebUI automation testing framework based on Selenium

Android in docker solution with noVNC supported and video recording

Scrapy Extension for monitoring spiders execution.

selenium破解滑动验证码

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

Generate Free Edu Mail(s) within minutes

Keep your Selenium WebDrivers updated automatically

A maven template for Selenium that will let you check out and go.

Web Extension for Firefox/Chrome/MS Edge and CLI tool to save a faithful copy of an entire web page in a single HTML file

PowerShell module to run a Selenium WebDriver.

豆瓣电影/豆瓣读书 Scarpy 爬虫

一个灵活、友好的爬虫框架

Crawlera middleware for Scrapy

滑动验证码破解示例，仅供学习使用。

Docker images for the Selenium Grid Server

A framework for creating semi-automatic web content extractors

基于 scrapy-redis 的通用分布式爬虫框架

An R client for Selenium Remote WebDriver

Stop stalking and start StopStalking 😉

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

A set of spiders and scrapers to extract location information from places that post their location on the internet.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

Selenium2 (webdriver) driver for Mink framework

Dockerfile for running Python Selenium in headless Chrome (Python 2.7 / 3.6 / 3.7 / 3.8 / Alpine based Python / Chromedriver / Selenium / Xvfb included in different versions)