818 Open source projects that are alternatives of or similar to SchweizerMesser

Continuous scalable web crawler built on top of Flink and crawler-commons

A collection of awesome web scaper, crawler.

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

A collection of awesome web crawler,spider in different languages

简单易用的Python爬虫框架，QQ交流群：597510560

扫描“微信读书”已购图书并下载本地PDF的爬虫

Cross Platform C# Web crawler framework, headless browser, parallel crawler. Please star this project! +1.

拼多多爬虫，爬取所有商品、评论等信息

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

徒手实现定时爬取知乎，从中发掘有价值的信息，并可视化爬取的数据作网页展示。

Turn large Web sites into tables and charts using simple SQLs.

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

Spider项目将会不断更新本人学习使用过的爬虫方法！！！

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

A web crawler for Go

NetDiscovery 是一款基于 Vert.x、RxJava 2 等框架实现的通用爬虫框架/中间件。

INFO-SPIDER 是一个集众多数据源于一身的爬虫工具箱🧰，旨在安全快捷的帮助用户拿回自己的数据，工具代码开源，流程透明。支持数据源包括GitHub、QQ邮箱、网易邮箱、阿里邮箱、新浪邮箱、Hotmail邮箱、Outlook邮箱、京东、淘宝、支付宝、中国移动、中国联通、中国电信、知乎、哔哩哔哩、网易云音乐、QQ好友、QQ群、生成朋友圈相册、浏览器浏览历史、12306、博客园、CSDN博客、开源中国博客、简书。

AlipaySpider on Scrapy(use chrome driver); 支付宝爬虫(基于Scrapy)

Rust Web Crawler saving pages on Redis

A simple distributed crawler for zhihu && data analysis

Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台

微博话题关键词,个人微博采集, 微博博文一键删除 selenium获取cookie,requests处理

千万级图片爬虫、视频爬虫 [开源版本] Image Spider

一些爬虫开发的技巧和案例

Python script which sends a custom morning message to your significant other every morning at a given time range on Facebook Messenger, WhatsApp, Telegram or SMS, for lazy people

抖音接口、抖音API、抖音数据爬虫、抖音直播数据、抖音直播Api、抖音视频Api、抖音爬虫、抖音去水印、抖音视频下载、抖音视频解析、抖音直播监控、抖音数据采集

Library packs for google cloud functions

极验滑动验证码研究报告

FofaMap是一款基于Python3开发的跨平台FOFA数据采集器，支持网站图标查询、批量查询和自定义查询FOFA数据，能够根据查询结果自动去重并生成对应的Excel表格。另外春节特别版还可以调用Nuclei对目标进行漏洞扫描，让你在挖洞路上快人一步。

💉🤖 Bot for the German "ImpfterminService - 116117"

Carina demo project.

boilerplate for nightwatch.js with selenium

An image scraper that scraps images from unsplash.com

一个基于Scrapy的数据采集爬虫代码库

小密圈备份，圈子/话题/图片/文件。

Extracting and parsing structured data with jQuery Selector, XPath or JsonPath from common web format like HTML, XML and JSON.

A tutorial-based introduction to web scraping with Python.

Instagram comment scraper using python and selenium. Save the comments into excel.

拟物校园，一个开源的高校教务移动化解决方案。

download document from book118 for free

虎扑网相册下载工具

Python script to control whatsapp web using terminal

A django admin site for scrapy

问卷星

基于 RabbitMQ 中间件的爬虫的 Ruby 实现 [Developing]

Python Program To Schedule Your Instagram Posts

A DHT Spider allows you to sniff the torrents and magnets.You can download them directly.

ZUCC正方教务系统抢课助手。针对ZUCC正方教务系统模拟登录，爬取课程信息，自动抓包发包抢课。具体实现流程可参考README中的实现原理链接

Extensions for Selenium WebDriver including jQuery/Sizzle selector support.

Implementation of the new headless chrome with chromedriver and selenium.

An AliExpress spider for Node

PrimeFaces testing support for Selenium

hCaptcha solver and bypasser for Python Selenium. Simple website to try to solve hCaptcha.

Robust Functional Tests leveraging WebDriver with flavour of neat AJAX-ready API

Script using requests module to register accounts to Slam Jam Socialism raffles.

解放双手，u校园的题再也不用写啦(暂停维护