682 Open source projects that are alternatives of or similar to Fast Lianjia Crawler

备份豆瓣计划

那些年，我爬过的北科。一个由浅入深的定向爬虫教程。

Simple news aggregator displaying top stories in real time

Web Crawler written in C#

经济责任审计知识图谱：网络爬虫、关系抽取、领域词汇判定

CoCrawler is a versatile web crawler built using modern tools and concurrency.

Amazon商品引流的 python 爬虫

Crawler for leboncoin.fr

基于Scrapy的QQ音乐爬虫(QQ Music Spider)，爬取歌曲信息、歌词、精彩评论等，并且分享了QQ音乐中排名前6400名的内地和港台歌手的49万+的音乐语料

A simple but powerful web crawler library for .NET

Easy to use lightweight web crawler（易用的轻量化网络爬虫）

dynamic server-side rendering using headless Chrome to effortlessly solve the SEO problem for modern javascript websites

a scaleable and efficient crawelr with docker cluster , crawl million pages in 2 hours with a single machine

头条号爬虫案例

拼多多爬虫，爬取所有商品、评论等信息

JSpider会每周更新至少一个网站的JS解密方式，欢迎 Star，交流微信：13298307816

🕷️ A node crawler for github trending.

Scrapy as a Web Crawler for Azure Search Samples

（已跑路）百度贴吧吧务管理工具，自动扫描帖子并处理违规帖

A Python module to scrape several search engines (like Google, Yandex, Bing, Duckduckgo, ...). Including asynchronous networking support.

Go语言实例教程从入门到进阶，包括基础库使用、设计模式、面试易错点、工具类、对接第三方等

Sina stock options crawler with CSV output 新浪上证ETF期权数据爬虫

Crawl a website and run it through Google lighthouse

Distributed Image Search Engine Crawler

《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志

A collection of awesome web scaper, crawler.

Secret and/ credential patterns used for gf.

招聘网数据爬虫

Bilibili.com视频批量下载工具包

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

一些爬虫的代码

利用文本挖掘技术进行新闻热点关注问题分析

一个简单的豆瓣信息爬虫😄

[ Closed ] 🎨 image collector, which supports custom acquisition source configuration and is compatible with MacOS and Windows operating systems.

Script that generates a sitemap by crawling a given URL

A python3 package for writing spider more easily.

基于appium的app自动遍历工具

房价预测完整项目：1.爬取链家网数据 2.处理后，用sklearn中几个逻辑回归机器学习模型和keras神经网络搭建模型预测房价 最终结果神经网络效果更好，R^2值0.75左右

👧 美女写真套图爬虫（二）

使用Python3爬取煎蛋妹纸图片

A python web scraper for public email lists.

Viewers for statistics and dashboarding of Domain Search Engine data

The Last Web Recon Tool You'll Need

本仓库收集整理爬虫相关资源，开发语言以Java为主

🕷some website spider application base on proxy pool (support http & websocket)

Crawls a website, gets PageSpeed Insights data for each page, and exports an HTML report.

👨‍🎓 中国大学MOOC全课件（视频、文档、附件）下载器

scrapy专利爬虫（停止维护）

netease-music-spider is a sipder that you can find beautiful girlfriend or handsome boyfriend.

知乎用户公开个人信息爬虫, 能够爬取用户关注关系，基于Python、使用代理、多线程

Automatically constructing corpus for automatic speech recognition from YouTube videos

Web app for Scrapyd cluster management, Scrapy log analysis & visualization, Auto packaging, Timer tasks, Monitor & Alert, and Mobile UI. DEMO 👉

简单、易用、高效 一个有态度的开源.Net Http请求框架!可以用制作爬虫，api请求等等。

爬虫代理IP池服务，可供其他爬虫程序通过restapi获取

Simple, clear and fast Web Crawler framework build on python3.6+, powered by asyncio.

Touhou Project random music video generator/player, crawling image and video from websites to generate MV.

爬取CSDN上的博客文章

简简单单spider

BitTorrent DHT Protocol && DHT Spider.

抓取豆瓣，bilibili等中的电视剧、电影、动漫演员等信息