1134 Open source projects that are alternatives of or similar to Gain

京东口罩库存监控爬虫(非selenium)，扫码登录、查价、加购、下单、秒杀

Web crawler.

web spider built by puppeteer, support task-queue and task-scheduling by decorators，support nedb / mongodb, support data visualization; 基于puppeteer的web爬虫框架，提供灵活的任务队列管理调度方案，提供便捷的数据保存方案（nedb/mongodb），提供数据可视化和用户交互的实现方案

Elegant Scraper and Crawler Framework for Golang

Linkedin Learning videos downloader

Web sessions for aiohttp.web

JavaScript + BeautifulSoup = JSSoup

Crawl some picture for fun

Asynchronous OAuth 2.0 framework and provider for Python 3

一些非常有趣的python爬虫例子,对新手比较友好,主要爬取淘宝、天猫、微信、豆瓣、QQ等网站。(Some interesting examples of python crawlers that are friendly to beginners. )

A framework to build Slack apps using Python

Golang爬虫 爬取豆瓣电影Top250

aiohttp_debugtoolbar is library for debugtoolbar support for aiohttp

The boilerplate for aiohttp. Quick setup for your asynchronous web service.

admin interface for aiohttp application http://aiohttp-admin.readthedocs.io

A Telegram bot that can stream Telegram files to users over HTTP.

🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。

Python3 asyncio based microframework for microservice architecture

asyncio client for neo4j

基于asyncio与aiohttp的异步协程爬虫框架 欢迎Star

A tool to create network tunnels over HTTP/S written in Python 3

💻 Microservice library / framework using Python's asyncio event loop with full support for HTTP + WebSockets, AWS SNS+SQS, RabbitMQ / AMQP, middleware, etc. Extendable for GraphQL, protobuf, gRPC, among other technologies.

Homework/task submit and review web app · based on React and Python aiohttp

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音|TikTok数据爬取工具，支持API调用，在线批量解析及下载。

BaiduSpider，一个爬取百度搜索结果的爬虫，目前支持百度网页搜索，百度图片搜索，百度知道搜索，百度视频搜索，百度资讯搜索，百度文库搜索，百度经验搜索和百度百科搜索。

The ultrarapid and multifunctional wrapper over QIWI and YooMoney

aiohttp middleware and helper utils for working with JSON web token.

Crawl sites for RSS, Atom, and JSON feeds.

A simple and flexible web crawler framework for java.

An asyncio-based client for sending metrics to StatsD with support of DogStatsD extension

High performance crawler for Nodejs

多线程知乎用户爬虫，基于python3

Traversal based router for aiohttp.web

Proxy (HTTP, SOCKS) connector for aiohttp

Binance chain SDK in Python

一个轻量级、快速、多线程、多管道、灵活配置的网络爬虫。

爬取 www.mzitu.com 全站图片，截至目前共5162个图集，16.5万多张美女图片，使用 asyncio 和 aiohttp 实现的异步版本只需要不到2小时就能爬取完成。按日期创建图集目录，保存更合理。控制台只显示下载的进度条，详细信息保存在日志文件中。支持异常处理，不会终止爬虫程序。失败的请求，下次再执行爬虫程序时会自动下载

磁力網站U3C3介紹以及域名更新

📰 A responsive interface of Hacker News with summaries and thumbnails.

🍰 一个可视化的爬虫平台

MM131网站图片爬取 🚨

微信公众号爬虫，公众号历史文章，文章评论，文章阅读及在看数据，可视化web页面，可部署于Windows服务器。基于Python3之flask/mysql/redis/mitmproxy/pywin32等实现，高效微信爬虫，微信公众号爬虫，历史文章，文章评论，数据更新。

Every web site provides APIs.

Python Docker API client based on asyncio and aiohttp

🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent

基于sanic的微服务基础架构

🌭💦 91porn爬虫在线无限制API接口（永久有效，口令每日更新） 及 在线web预览

We will try to find your visible basic footprint from social media as much as possible - 😤 more sites is comming soon

蓝天采集器是一款免费的数据采集发布爬虫软件，采用php+mysql开发，可部署在云服务器，几乎能采集所有类型的网页，无缝对接各类CMS建站程序，免登录实时发布数据，全自动无需人工干预！是网页大数据采集软件中完全跨平台的云端爬虫系统

微博超级话题爬虫，微博词频统计+情感分析+简单分类，新增肺炎超话爬取数据

新一代爬虫平台，以图形化方式定义爬虫流程，不写代码即可完成爬虫。

小说下载|小说爬取|起点|笔趣阁|导出Markdown|导出txt|转换epub|广告过滤|自动校对

💐Marmot | Web Crawler/HTTP protocol Download Package 🐭

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。

Demos for aiohttp project

💐 Full Amazon Automatic Download

a puppeteer walker 🕷 🕸

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

😱Full Automatic Amazon Distributed Spider | 亚马逊分布式四国际站采集选款产品|账号admin,密码adminadmin