1579 Open source projects that are alternatives of or similar to Scaleable Crawler With Docker Cluster

A lite distributed Java spider framework :-)

Broccoli - distributed task queues for ESP32 cluster

Celery Tasks Monitoring Tool

Build Docker clusters with Celery and RabbitMQ in 10 minutes

Kombu is a messaging library for Python.

🚀构建分布式即时聊天、消息推送系统。 Building distributed instant messaging, push notification systems.

Distributed Gatling

Small Flask app with scalable, asynchronous backend workers deployed on Kubernetes.

High-performance distributed sync service and atomic DB

Flask application development skeleton with docker-compose

An elixir implementation of Rtree, optimized for fast updates.

Celery client for Node.js

A to Z Sky Operating System / Microservice Chassis Framework

Minimal example utilizing fastapi and celery with RabbitMQ for task queue, Redis for celery backend and flower for monitoring the celery tasks.

Distributed web chat application base websocket built on akka.

The multi-node setup of TimescaleDB 🐯🐯🐯 🐘 🐯🐯🐯

pip install funboost，python全功能分布式函数调度框架,。支持python所有类型的并发模式和全球一切知名消息队列中间件，python函数加速器，框架包罗万象，一统编程思维，兼容50% python编程业务场景，适用范围广。只需要一行代码即可分布式执行python一切函数。旧名字是function_scheduling_distributed_framework

Documentation repo of nebula orchestration system

GUI based offensive penetration testing tool (Open Source)

DotnetSpider, a .NET standard web crawling library. It is lightweight, efficient and fast high-level web crawling & scraping framework

Java Library for Distributed Services

Clearly see and debug your celery cluster in real time!

Nebula is a powerful framwork for building highly concurrent, distributed, and resilient message-driven applications for C++.

CrateDB is a distributed SQL database that makes it simple to store and analyze massive amounts of data in real-time.

Real-time monitor and web admin for Celery distributed task queue

Redis-based components for Scrapy.

Library for distributed job/worker logic.

RabbitMQ-backed microservices supporting RPC, pubsub, automatic service discovery and scaling with no code changes.

🌊 .NET Standard 2.1 framework which makes easy to scaffold distributed systems and dispatch incoming load into units of work in a deterministic way.

Distributed Image Search Engine Crawler

Example Docker setup for a Django app behind an Nginx proxy with Celery workers

Coerce - an asynchronous (async/await) Actor runtime and cluster framework for Rust

🦀 Rust implementation of Celery for producing and consuming background tasks

LEMP stack in a Kubernetes cluster

The celery monitor app was written by Django.

🥄 A package for building specific Proxy Pool for different Sites.

The API endpoint that manages nebula orchestrator clusters

💐 Full Amazon Automatic Download

Django Celery Tutorial

gathers Python stack for DevOps, these are usually my basic templates use for my implementations, so, feel free to use it and evolve it! Everything is Docker!

😎Asynchronous tasks in Python with Celery + RabbitMQ + Redis

Online Judge for hosting coding competitions inside NIT Durgapur made by GNU/Linux Users' Group!

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

This is a prototype to schedule jobs in the backend based on some priority using Rabbitmq and Celery.

Small box of pandora to prototype your app with ready for use backend. This is just my compilation of different solutions occasionally applied in hackathons and challenges

A docker stack to create, test and benchmark a rabbitmq cluster in high availability configuration. HAProxy, php workers, node failures, network partition, persistent messages.

A HTTP Ruby API for Consul

Lets Map Your Network enables you to visualise your physical network in form of graph with zero manual error

Want to handle 100,000 messages in 90 seconds? Celery and Kombu are that awesome - Multiple publisher-subscriber demos for processing json or pickled messages from Redis, RabbitMQ or AWS SQS. Includes Kombu message processors using native Producer and Consumer classes as well as ConsumerProducerMixin workers for relay publish-hook or caching

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

Fairly basic redis-like hashmap implementation on top of a epoll TCP server.

Raspberry Pi Kubernetes cluster that runs HA/HP Drupal 8

A tiny wrapper over amqp exchanges and queues 🚌 ✨

🍭A web platform designed for mysql inception

Terraform configuration for creating RabbitMQ cluster on AWS.

🎯 λ Hexagonal Architecture + DDD + CQRS applied in Scala using Akka HTTP

STOMP client for Embarcadero Delphi and FreePascal.

Pure Rust RabbitMQ client

Repository for Docker Image of Apache-Superset. [Docker Image: https://hub.docker.com/r/abhioncbr/docker-superset]

Dafka is a decentralized distributed streaming platform