2016 Open source projects that are alternatives of or similar to Scrapy Cluster

Possibly the best practice of Scrapy 🕷 and renting a house 🏡

A programmable, observable and distributed job orchestration system.

A web front-end for password cracking and analytics

A graph database with Cypher query language as a Redis module

Player tracking for Asheron's Call

Redis(<=5.0.5) RCE

使用Vue+VueX+ElementUI+SpringBoot的代码框架

两只蠢萌京东的分布式爬虫.

Redis memory profiler to find the RAM bottlenecks throw scaning key space in real time and aggregate RAM usage statistic by patterns.

This bot can help you in managing your group with rules, anti-flood, description, custom triggers, and much more!

ASP.NET Identity Redis Provider

End-to-end encrypted messaging and mutual authentication between cloud and edge-device applications

最全的Java操作Redis的工具类，使用StringRedisTemplate实现，封装了对Redis五种基本类型的各种操作！

Docker Compose for Node projects with Node, MySQL, Redis, MongoDB, NGINX, Apache2, Memcached, Certbot and RabbitMQ images

Tools for various online judges. Downloading sample cases, generating additional test cases, testing your code, and submitting it.

Free and open source log management

A music player that connects to your cloud/distributed storage.

Docs and files for ScrapydWeb, Scrapyd, Scrapy, and other projects

Laravel event broadcasting with Node.js, Redis & Socket.io

Analyze facebook copy of your data with ruby language. Download zip file from facebook and get info about friends ranking by message, vocabulary, contacts, friends added statistics and more

🔒A simple distributed tools based on Redis.

🚀 Web GUI for Kafka Cluster Management

High performance mqtt broker

A Python framework based on Flask microframework, with batteries included, and best practices in mind.

SiteWhere is an industrial strength open-source application enablement platform for the Internet of Things (IoT). It provides a multi-tenant microservice-based infrastructure that includes device/asset management, data ingestion, big-data storage, and integration through a modern, scalable architecture. SiteWhere provides REST APIs for all system functionality. SiteWhere provides SDKs for many common device platforms including Android, iOS, Arduino, and any Java-capable platform such as Raspberry Pi rapidly accelerating the speed of innovation.

ABP module-zero +AdminLTE+Bootstrap Table+jQuery+Redis + sql server+quartz+hangfire权限管理系统

Real-time monitor and web admin for Celery distributed task queue

2018年本科毕设项目，已更新所有开发和部署文档。基于Dubbo、SSM、Shiro、ELK、ActiveMQ、Redis等实现的一套高可用、高性能、高可扩展的分布式系统架构，实现可支持业务的基础公共服务,API使用Restful风格对外暴露。已经实现的包括：发送邮件服务、FastDFS文件存储服务、ELK实时日志查询服务、Redis缓存服务、Mybatis数据库、阿里短信推送、Goeasy消息推送、Druid监控、ActiveMQ消息队列、shiro权限认证、cas单点登录、权限配置web系统、移动端后台系统。持续更新中......

presto-redis is an experimental sql layer for redis

Hot sync two Redis servers using dumps.

Machinery is an asynchronous task queue/job queue based on distributed message passing.

Message queue system written in Go and backed by Redis

🍎 笔记本

Efficient video analysis at scale

Life is short, *****.

PPT I collected

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

A complete docker package for deploying django which is easy to understand and deploy anywhere.

基于springcloud全家桶开发分布式框架（支持oauth2认证授权、SSO登录、统一下单、微信公众号服务、Shardingdbc分库分表、常见服务监控、链路监控、异步日志、redis缓存等功能），实现基于Vue全家桶等前后端分离项目工程

🚀电商网站爬虫合集，淘宝京东亚马逊等

基于搜狗微信搜索的微信公众号爬虫接口

RedisView implements open source, cross-platform and high performance Redis interface tools through self-written RESP protocol parsing, self-written tree model and thread pool. RedisView业余爱好通过自写RESP协议解析、自写树模型、线程池实现开源、跨平台、高性能Redis界面图形化工具

A python web scraper for public email lists.

A generic dynamo implementation for different k-v storage engines

Scalable real-time messaging server in a language-agnostic way. Set up once and forever.

Custom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)

Modin: Speed up your Pandas workflows by changing a single line of code

WeDataSphere is a financial level one-stop open-source suitcase for big data platforms. Currently the source code of Scriptis and Linkis has already been released to the open-source community. WeDataSphere, Big Data Made Easy!

分布式任务调度平台(Distributed Job Schedule Platform)

An industrial IoT underlying architecture framework, focusing on the underlying technical communications and cross-platform, cross-language communication functions, to achieve a variety of mainstream PLC data reading and writing, to achieve modbus of various protocols read and write, and so on, to support the rapid construction of industrial upper computer software, configuration software, SCADA software, factory mes system, To help enterprise Industry 4.0 take-off, to achieve intelligent manufacturing, smart factory goals. The main PLC contains Siemens, Mitsubishi, Omron, Panasonic, Modbus, AB-PLC, Redis

General purpose redis client

An extensible distributed system for reliable nearline data streaming at scale

A swiss army knife CLI tool for interacting with Kafka, RabbitMQ and other messaging systems.

Real Time Analytics and Data Pipelines based on Spark Streaming

🐶 微信小程序-小商城后台（基于 Node.js、MongoDB、Redis 开发的系统...）

[Unmaintained] A simple and clean video/music/image downloader 👾

A job processing system that just verks! 🧛‍

一个整合SSM框架的高并发和商品秒杀项目,学习目前较流行的Java框架组合实现高并发秒杀API

计算机图书,java,mysql,架构类,web

Awesome list of distributed systems resources