2334 Open source projects that are alternatives of or similar to Distributed Multi User Scrapy System With A Web Ui

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

💬 Real time Chat application built with Vue, Django, RabbitMQ and uWSGI WebSockets.

《Spring Boot教程》源码

Hexagon is a microservices toolkit written in Kotlin. Its purpose is to ease the building of services (Web applications, APIs or queue consumers) that run inside a cloud platform.

springboot和dubbo、netty的集成，redis mongodb的nosql模板， kafka rocketmq rabbit的MQ模板， solr solrcloud elasticsearch查询引擎

DDD+CQRS+Event-sourcing examples using EventFlow following CQRS-ES architecture. It is configured with RabbitMQ, MongoDB(Snapshot store), PostgreSQL(Read store), EventStore(GES). It's targeted to .Net Core 2.2 and include docker compose file.

all kinds of scrapy demo

基于python语言开发的千万级别搜索引擎

Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索

Python + Scrapy + MongoDB . 5 million data per day !!!💥 The world's largest website.

Seeker - another job board aggregator.

iHealth 项目的后台程序（一个基于 Django 和 MongoDB 的 Web 后端）

Example Docker setup for a Django app behind an Nginx proxy with Celery workers

UNMAINTAINED

mall项目是一套电商系统，包括前台商城系统及后台管理系统，基于SpringBoot+MyBatis实现，采用Docker容器化部署。 前台商城系统包含首页门户、商品推荐、商品搜索、商品展示、购物车、订单流程、会员中心、客户服务、帮助中心等模块。 后台管理系统包含商品管理、订单管理、会员管理、促销管理、运营管理、内容管理、统计报表、财务管理、权限管理、设置等模块。

用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

Yet another django based blog created in May 2015

Mongoengine support for Django Rest Framework

Machinery is an asynchronous task queue/job queue based on distributed message passing.

mall-swarm是一套微服务商城系统，采用了 Spring Cloud Hoxton & Alibaba、Spring Boot 2.3、Oauth2、MyBatis、Docker、Elasticsearch、Kubernetes等核心技术，同时提供了基于Vue的管理后台方便快速搭建系统。mall-swarm在电商业务的基础集成了注册中心、配置中心、监控中心、网关等系统功能。文档齐全，附带全套Spring Cloud教程。

about learning Spring Boot via examples. Spring Boot 教程、技术栈示例代码，快速简单上手教程。

Django Celery Tutorial

Aplicação que simula um banco digital, contendo a área do cliente e administrativa, permitindo depósitos e transferências entre contas do mesmo banco. | Application that simulates a digital bank, containing the customer and administrative areas, allowing deposits and transfers between accounts of the same bank.

mall学习教程，架构、业务、技术要点全方位解析。mall项目（40k+star）是一套电商系统，使用现阶段主流技术实现。涵盖了SpringBoot 2.3.0、MyBatis 3.4.6、Elasticsearch 7.6.2、RabbitMQ 3.7.15、Redis 5.0、MongoDB 4.2.5、Mysql5.7等技术，采用Docker容器化部署。

A lightweight task queue for Django using RabbitMQ

Vagrant configuration for PHP7, Phalcon 3.x and Zephir development.

☁️ 👀 💬 Visual Chatbot

Creating Scrapy scrapers via the Django admin interface

Symfony2/3/4/5 Queue Bundle (for background jobs) supporting Mongo (Doctrine ODM), Mysql (and any Doctrine ORM), RabbitMQ, Beanstalkd, Redis, and ... {write your own}

支持mysql、MongoDB数据变更订阅分发

一些基础服务的docker-compose配置文件，方便在一台新电脑上快速开始工作

spring、spring-boot、spring-cloud 常用整合用例

Backend para coleta e disponibilização dos dados 📜

CQRS, using Clean Architecture, multiple databases and Eventual Consistency

Universal set of diagnostic tests for PHP applications.

招聘网数据爬虫

Distributed Crawler Management Framework Based on Scrapy, Scrapyd, Django and Vue.js

Lets Map Your Network enables you to visualise your physical network in form of graph with zero manual error

🐂基于springcloud +dubbo构建的模拟秒杀项目，模块化设计，集成了分库分表、elasticsearch🔍、gateway、mybatis-plus、spring-session等常用开源组件

Management commands to help backup and restore your project database and media files

Container auto-configurations for spring-boot based integration tests

Sync data between persistence engines, like ETL only not stodgy

Microservices on .Net platforms which used Asp.Net Web API, Docker, RabbitMQ, MassTransit, Grpc, Ocelot API Gateway, MongoDB, Redis, PostgreSQL, SqlServer, Dapper, Entity Framework Core, CQRS and Clean Architecture implementation. Also includes Cross-Cutting concerns like Implementing Centralized Distributed Logging with Elasticsearch, Kibana and SeriLog, use the HealthChecks with Watchdog, Implement Retry and Circuit Breaker patterns with Polly and so on.. See Microservices Architecture and Step by Step Implementation on .NET Course w/ discount->

两只蠢萌京东的分布式爬虫.

Bifrost ---- 面向生产环境的 MySQL 同步到Redis,MongoDB,ClickHouse,MySQL等服务的异构中间件

全栈工程师手册

selenium异步爬取网页图片

SpringBoot Examples

Django and MongoDB database connector

🏭 An easy-to-use implementation of Creation Methods for Django, backed by Faker.

MuckRock's source code - Please report bugs, issues and feature requests to [email protected]

A Sample App built using Spring Boot, Angular and MongoDB

Reference applications for common web frameworks showing how to embed Metabase charts

SparkPost client library for Python

使用Docker构建云平台，Docker云平台系列共三讲，Docker基础、Docker进阶、基于Docker的云平台方案。OpenStack+Docker+RestAPI+OAuth/HMAC+RabbitMQ/ZMQ+OpenResty/HAProxy/Nginx/APIGateway+Bootstrap/AngularJS+Ansible+K8S/Mesos/Marathon构建/探索微服务最佳实践。

DataTables Example (server-side) - Python Django REST framework

Nuwber's events provide a simple observer implementation, allowing you to listen for various events that occur in your current and another application. For example, if you need to react to some event published from another API.