2535 Open source projects that are alternatives of or similar to Docs

豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章

爬虫学习仓库，适合零基础的人学习，对新手比较友好

Simple but useful Python web scraping tutorial code.

京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取

B站用户爬虫 好耶~是爬虫

Python related technologies used in work: crawler, data analysis, timing tasks, RPC, page parsing, decorator, built-in functions, Python objects, multi-threading, multi-process, asynchronous, redis, mongodb, mysql, openstack, etc.

ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个

python爬虫

HTTP API for Scrapy spiders

Bifrost ---- 面向生产环境的 MySQL 同步到Redis,MongoDB,ClickHouse,MySQL等服务的异构中间件

APIs for loginning some websites by using requests.

macOS development environment setup: Easy-to-understand instructions with automated setup scripts for developer tools like Vim, Sublime Text, Bash, iTerm, Python data analysis, Spark, Hadoop MapReduce, AWS, Heroku, JavaScript web development, Android development, common data stores, and dev-based OS X defaults.

A multi-thread crawler framework with many builtin image crawlers provided.

基于EF Core的Code First模式的DotNetCore快速开发框架，其中包括DBContext、IOC组件autofac和AspectCore.Injector、代码生成器（也支持DB First）、基于AspectCore的memcache和Redis缓存组件，以及基于ICanPay的支付库和一些日常用的方法和扩展，比如批量插入、更新、删除以及触发器支持，当然还有demo。欢迎提交各种建议、意见和pr~

Python + Scrapy + MongoDB . 5 million data per day !!!💥 The world's largest website.

基于appium的app自动遍历工具

深圳地铁大数据客流分析系统🚇🚄🌟

rabbitmq、redis、scheduled、socket、mongodb、Swagger2、spring data jpa、Thymeleaf、freemarker etc. (muti module spring boot project) (with spring boot framework，different bussiness scence with different technology。)

Analysis of the characteristics of different countries

mall-swarm是一套微服务商城系统，采用了 Spring Cloud Hoxton & Alibaba、Spring Boot 2.3、Oauth2、MyBatis、Docker、Elasticsearch、Kubernetes等核心技术，同时提供了基于Vue的管理后台方便快速搭建系统。mall-swarm在电商业务的基础集成了注册中心、配置中心、监控中心、网关等系统功能。文档齐全，附带全套Spring Cloud教程。

Netkiller Free ebook - 免费电子书

慕课网python分布式爬虫源码-长期更新维护

Symfony2/3/4/5 Queue Bundle (for background jobs) supporting Mongo (Doctrine ODM), Mysql (and any Doctrine ORM), RabbitMQ, Beanstalkd, Redis, and ... {write your own}

💪 A library that powers your app with GraphQL + Rest API

Docker image to periodically backup your database (MySQL, Postgres, or MongoDB) to S3 or local disk.

Parser and database to index the terpene profile of different strains of Cannabis from online databases

Custom Laravel and Lumen 5.6+ Log channel handler that can store log events to SQL or MongoDB databases. Uses Laravel/Monolog native logging functionality.

mall学习教程，架构、业务、技术要点全方位解析。mall项目（40k+star）是一套电商系统，使用现阶段主流技术实现。涵盖了SpringBoot 2.3.0、MyBatis 3.4.6、Elasticsearch 7.6.2、RabbitMQ 3.7.15、Redis 5.0、MongoDB 4.2.5、Mysql5.7等技术，采用Docker容器化部署。

Scrapy-based Crawlers for news of Taiwan

🔎Sniffing and parsing mysql,redis,http,mongodb etc protocol. 抓包截取项目中的数据库请求并解析成相应的语句。

🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。

scrapy专利爬虫（停止维护）

Everything about database,business.(Most for PostgreSQL).

两只蠢萌京东的分布式爬虫.

Life is short, *****.

Word2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索

Android应用市场网络爬虫

Database migrations. CLI and Golang library.

Scrapy as a Web Crawler for Azure Search Samples

TreeFrog Framework : High-speed C++ MVC Framework for Web Application

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架

基于搜狗微信搜索的微信公众号爬虫接口

node.js database migration tool

🔒 Node Access Control Lists (ACL).

Official GeoTools repository

🏀 Python3 网络爬虫实战（部分含详细教程）猫眼 腾讯视频 豆瓣 研招网 微博 笔趣阁小说 百度热点 B站 CSDN 网易云阅读 阿里文学 百度股票 今日头条 微信公众号 网易云音乐 拉勾 有道 unsplash 实习僧 汽车之家 英雄联盟盒子 大众点评 链家 LPL赛程 台风 梦幻西游、阴阳师藏宝阁 天气 牛客网 百度文库 睡前故事 知乎 Wish

Some scrapy and web.py exmaples

使用SpringCloud和Spring WebFlux开发的学生作业管理系统服务端，前后端分离项目，微服务架构。支持Docker集群化部署，Redis集群化缓存，文件在线预览 压缩包预览 打包上传下载

新浪微博相册大图多线程爬虫。

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

Customizations for Adminer, the best database management tool written in PHP.

DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

支持mysql、MongoDB数据变更订阅分发

Spring Boot 2.x code examples

GraphQuery is a query language and execution engine tied to any backend service.

A Facebook crawler

mall项目是一套电商系统，包括前台商城系统及后台管理系统，基于SpringBoot+MyBatis实现，采用Docker容器化部署。 前台商城系统包含首页门户、商品推荐、商品搜索、商品展示、购物车、订单流程、会员中心、客户服务、帮助中心等模块。 后台管理系统包含商品管理、订单管理、会员管理、促销管理、运营管理、内容管理、统计报表、财务管理、权限管理、设置等模块。

Vagrant configuration for PHP7, Phalcon 3.x and Zephir development.

Django based application that allows creating, deploying and running Scrapy spiders in a distributed manner