1699 Open source projects that are alternatives of or similar to Dotnetspider

Library for distributed job/worker logic.

A distributed web crawler framework.（分布式爬虫框架XXL-CRAWLER）

Redis-based components for Scrapy.

a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine

Build HTML Desktop Apps on .NET/.NET Core/.NET 5 using native GUI, HTML5, JavaScript, CSS

🐾 Creeper - The Next Generation Crawler Framework (Go)

This is a distributed task management system base on .Net Core platform .

一个基于 .NET Core构建的简单、跨平台、模块化的商城系统

SmartSql = MyBatis in C# + .NET Core+ Cache(Memory | Redis) + R/W Splitting + PropertyChangedTrack +Dynamic Repository + InvokeSync + Diagnostics

A cross platform XAML framework for .NET

Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

💐 Full Amazon Automatic Download

🛠「Steam++」是一个开源跨平台的多功能Steam工具箱。

Distributed Image Search Engine Crawler

💎 A collection of awesome training series, articles, videos, books, courses, sample projects, and tools for Microservices in .NET Core

Implementation of RAFT distributed consensus algorithm among TCP Peers on .NET / .NETStandard / .NETCore / dotnet

Cross platform file manager application for Windows, Mac and Linux operating systems. (planned mobile support)

Cross-platform Distributed Test Runner. Executes tests in parallel, time balanced on multiple machines.

A simple, cross platform, modularized ecommerce system built on .NET Core

一个基于 .NET Core构建的简单、跨平台、模块化的商城系统

The C++ Standard Library for Parallelism and Concurrency

💖 High available distributed ip proxy pool, powerd by Scrapy and Redis

A lite distributed Java spider framework :-)

🥄 A package for building specific Proxy Pool for different Sites.

IronScheme

The simple, easy to use command line web crawler.

Web Application Security Scanner Framework

🌋 A modern and easy-to-use library for the Vulkan API

An easy syntax to format your strings with colored fonts and backgrounds.

Library for building distributed, real-time collaborative web applications

circuits is a Lightweight Event driven and Asynchronous Application Framework for the Python Programming Language with a strong Component Architecture.

🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库

.NET Core Inter-process communication framework

A fast tool to fetch URLs from HTML attributes by crawl-in.

Recursive and multi-threaded broken link checker

📰 A responsive interface of Hacker News with summaries and thumbnails.

Easily create XML sitemaps for your website.

An opinionated HTTP library for Mobile Development

Observability Signaling for Distributed Computation

模拟登录各类网站，操作 API 完成各种不可描述的事情

✏️A modern, fast, beautiful note taking app, built on Electron and React

Java Library for Distributed Services

No description or website provided.

An R web crawler and scraper

HTTP Monitoring via API - Measure the performance of your servers

我的爬虫合集

Distributed Fieldaware Factorization Machines based on Parameter Server

line-bot-tutorial use python flask

Simple Weibo Scraper

A micro service development engine for node.js

A cross-platform python based utility to download courses from lynda.com for personal offline use.

[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn

CQRS-DDD-Actor framework for Node.js

Single header C++ logging library. It is extremely powerful, extendable, light-weight, fast performing, thread and type safe and consists of many built-in features. It provides ability to write logs in your own customized format. It also provide support for logging your classes, third-party libraries, STL and third-party containers etc.

python requests + Django+ nodejs koa+ mysql to crawl eastmoney fund and stock data,for data analysis and visualiaztion .

🐳 Sonar Scanner MsBuild Dockerfile for dotNet Projects

Simple, drag-and-drop, password-based file encryption

A Tool for Automatic Parallelization of Deep Learning Training in Distributed Multi-GPU Environments.

Just a crawler based on tg-cli for Telegram. Deprecated by now, please use telegram-export.