817 Open source projects that are alternatives of or similar to Wswp

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

Schedules tweets using TweetDeck

Scrape Instagram content and stories anonymously, using a new technique based on the har file (No Token + No public API).

Build and deploy a machine learning app from scratch 🚀

Download all the ebooks with indexed csv of "allitebooks.com"

CHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages

Using Selenium, Neha scraped data about 35 top selling sneakers of Nike and Adidas from stockx.com. She used this data to draw insights about sneaker resales.

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

Scrape public FaceBook posts from any group or user into a .csv file without needing to register for any API access

Web scraper project to collect the useful Hong Kong weather data from HKO website

Python入门网络爬虫之精华版

AlipaySpider on Scrapy(use chrome driver); 支付宝爬虫(基于Scrapy)

A python web scraper for public email lists.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Scrapy middleware to handle javascript pages using selenium

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

Helpful Selenium functions to make web-scraping easier and faster

小密圈备份，圈子/话题/图片/文件。

🚀电商网站爬虫合集，淘宝京东亚马逊等

With Selenium headless browsing and CAPTCHA solving

web scraping in parallel with Selenium Grid and Docker

An image scraper that scraps images from unsplash.com

爬虫学习仓库，适合零基础的人学习，对新手比较友好

ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Creating Scrapy scrapers via the Django admin interface

A WebdriverIO plugin. Additional commands for taking screenshots with WebdriverIO.

An automation bot for free Udemy courses

(Python) Program to simulate multiple sessions on adidas queue pages.

Make your WebDriver based Selenium tests more readable, reusability and maintainable by using WebDriver Extensions!

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

Turn large Web sites into tables and charts using simple SQLs.

A bot for booking Glastonbury tickets using selenium

💻 command-line bootcamp adventure in your browser

Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner

可视化任务调度系统，精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)

🕶️ A curated list of resources around the topic: visual regression testing

土巴兔和谷居装修网站爬虫

AET - a system that detects visual changes on web sites and performs basic page health checks

An Instagram OSINT tool to collect all the geotagged locations available on an Instagram profile in order to plot them on a map, and dump them in a JSON.

InstantWP is a complete standalone, portable WordPress development environment.

研究资源列表 A curated list of research resources

Scrapy-based Crawlers for news of Taiwan

E2E test framework for Angular apps

Automation-friendly framework for Continuous Testing by

Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements

Commandline tool for automated downloads of echo360 videos hosted by university

Dockerfiles for Python 3.6/2.7 & Selenium in a headless Chrome or Firefox environment

An environment to define and run integration tests. It contains Fitnesse fixture (base) classes and a baseline FitNesse installation.

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Social media research and promotion, semi-autonomous CLI bot

lots of spider (很多爬虫）

Framework for writing maintainable Selenium and REST API tests in Java.

Perceptual image hashing for Node.js

This repository contains data and code used to get and clean data from https://github.com/CSSEGISandData/COVID-19 and https://www.worldometers.info/coronavirus/

A native Selenium WebDriver binding for ColdFusion

Amazon商品引流的 python 爬虫

OLX Scraper in Python Scrapy

capture pictures from website like sina, lofter, huaban and so on