1815 Open source projects that are alternatives of or similar to Awesome_crawl

慕课网python分布式爬虫源码-长期更新维护

🍰 Everything is RSSible

APIs for loginning some websites by using requests.

Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索

This is a sina weibo spider built by scrapy [微博爬虫/持续维护]

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

Python爬虫实战 - 模拟登陆各大网站 包含但不限于：滑块验证、拼多多、美团、百度、bilibili、大众点评、淘宝，如果喜欢请start ❤️

个人学习 SpringBoot2.x 写的一些示例程序，目前正在持续更新中.....

Korio: Kotlin cORoutines I/O : Virtual File System + Async/Sync Streams + Async TCP Client/Server + WebSockets for Multiplatform Kotlin 1.3

SAEA.Socket is a high-performance IOCP framework TCP based on dotnet standard 2.0; Src contains its application test scenarios, such as websocket,rpc, redis driver, MVC WebAPI, lightweight message server, ultra large file transmission, etc. SAEA.Socket是一个高性能IOCP框架的 TCP，基于dotnet standard 2.0；Src中含有其应用测试场景，例如websocket、rpc、redis驱动、MVC WebAPI、轻量级消息服务器、超大文件传输等

参照知乎网做的一个简易版问答网站，后台采用Java语言，及Spring，SpringMVC，MyBatis等框架，模版引擎使用FreeMarker。数据库用到MySQL，Redis。

Redis-based components for Scrapy.

模拟登录一些知名的网站，为了方便爬取需要登录的网站

A protocol gateway for the Web.

Python入门网络爬虫之精华版

Laravel 8.0 blog application with Vue.js, Homestead, Horizon, Telescope and Pusher

A Redis-backed notification server written in Go

video downloader

知乎模拟登录，支持提取验证码和保存 Cookies

This Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.

👽 OneBlog，一个简洁美观、功能强大并且自适应的Java博客

Open-source Back-end, self-hostable & ready to use - Real-time, storage, advanced search - Web, Apps, Mobile, IoT -

A Go websocket library with an API similar to Socket.IO... but not Socket.IO

A WebSocket message router based on Python/Redis/asyncio

Quick and Easy Web Development

微博爬虫：每天定时爬取微博热搜榜的内容，留下互联网人的记忆。

webchat via WebSockets/WebRTC that allows messaging/video call/screen sharing

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

Node.js based forum software built for the modern web

Photo sharing social media site built with Python/Django. Based on Instagram's design.

Rails in real time

turn based strategy game using django channels, redux, and react hooks

基于微博的数据挖掘与社交舆情分析

💕 build SSM from 0 👉🏽👉🏽 distributed micro service.

🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。

A lightweight and modular social sharing library

免费 IP 代理池。Scrapy 爬虫框架插件

A Facebook crawler

Real-time issue tracker optimized for small teams

Python爬虫代理IP池(proxy pool)

新闻抓取（微信、微博、头条...）

WebSocket client for Python

链喵 CMDB 本项目已停止开发！因长时间未对代码进行维护，可能会造成项目在不同环境上无法部署、运行BUG等问题，请知晓！项目仅供参考！

A simple high-performance Redis message queue for Node.js.

做个平台

DeviceHive Java Server

Hprose is a cross-language RPC. This project is Hprose 2.0 Client for HTML5

Dynamic file detection tool based on crawler 基于爬虫的动态敏感文件探测工具

syncClient，数据实时同步中间件（同步mysql到kafka、redis、elasticsearch、httpmq）！

webrtc 直播连麦

Build reactive applications with the django tooling you already know and love.

👾 Fast and simple video download library and CLI tool written in Go

Integrates your Laravel application with a redis manager

Download weibo images without logging-in

招聘网数据爬虫

spring boot 实践学习案例，是 spring boot 初学者及核心技术巩固的最佳实践。

Scrapy+Splash for JavaScript integration

Client and server-side state management library

A simple Flask boilerplate app with SQLAlchemy, Redis, User Authentication, and more.

基于 webmagic 的 Java 爬虫应用