683 Open source projects that are alternatives of or similar to Lianjia Beike Spider

B站3亿用户信息爬虫（mid号，昵称，性别，关注，粉丝，等级）

经济责任审计知识图谱：网络爬虫、关系抽取、领域词汇判定

根据关键词抓取微博数据，再生成词云

A powerful dynamic crawler for web vulnerability scanners

🚇暗网中文网监控爬虫(DEEPMIX)

A simple but powerful web crawler library for .NET

News, full-text, and article metadata extraction in Python 3. Advanced docs:

👉一个基于spring boot 实现的java股票爬虫(仅支持A股)，如果你❤️请⭐️ . V2升级版正在开发中！

Crawl a website and run it through Google lighthouse

Web Scraping Framework

👨‍🎓 中国大学MOOC全课件（视频、文档、附件）下载器

Enjoy driving on a Javascriptive (originally Pythonic) way to Japanese AV!

用爬虫爬取小说网站上所有小说，存储到数据库中，并用爬到的数据构建自己的小说网站

哔咔漫画收藏夹下载程序

Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!

🤖 微信订阅机器人 | 🐟 微信订阅机器人之闲鱼二手商品监控

Some toolkits implements part of BT Protocol, like DHT spider.

create some tools use go lang.

Go process used to crawl websites

Sample of using proxies to crawl baidu search results.

Last-Statement-of-Death-Row, 人之将死，其言也善

Automatically constructing corpus for automatic speech recognition from YouTube videos

基于搜狗微信搜索的微信公众号爬虫接口

一个完整的nodeJs 串行爬虫 抓取3万多个页面。

最简单的91porn爬虫php版本

Simple, clear and fast Web Crawler framework build on python3.6+, powered by asyncio.

Leetcode Contest Ranking Searcher

房价预测完整项目：1.爬取链家网数据 2.处理后，用sklearn中几个逻辑回归机器学习模型和keras神经网络搭建模型预测房价 最终结果神经网络效果更好，R^2值0.75左右

This package is a complete tool for creating a large dataset of images (specially designed -but not only- for machine learning enthusiasts). It can crawl the web, download images, rename / resize / covert the images and merge folders..

简简单单spider

Crawl instagram photos, posts and videos for download.

owllook-小说搜索引擎

🐞 立马理财销售统计(爬虫+页面展示)

😩Tool For Taobao/Tmall| 儿时玩具已经过时

java framework for prerender

网易云爬虫解决方案

Doujinshi downloader 绅士漫画下载

GUI of AlipayOrdersSupervisor, implemented in Java and Swing

😊😊😊 知乎问题爬虫

Web 端反爬技术方案

🎬 基于Pyqt5的简单电影搜索工具

feapder是一款支持分布式、批次采集、任务防丢、报警丰富的python爬虫框架

Crawler for Nintendo Switch eShop

Scrapy-based Crawlers for news of Taiwan

Verify that a request is from Google crawlers using Google's DNS verification steps

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome or Chromium with or without a head

🏀 Python3 网络爬虫实战（部分含详细教程）猫眼 腾讯视频 豆瓣 研招网 微博 笔趣阁小说 百度热点 B站 CSDN 网易云阅读 阿里文学 百度股票 今日头条 微信公众号 网易云音乐 拉勾 有道 unsplash 实习僧 汽车之家 英雄联盟盒子 大众点评 链家 LPL赛程 台风 梦幻西游、阴阳师藏宝阁 天气 牛客网 百度文库 睡前故事 知乎 Wish

Viewers for statistics and dashboarding of Domain Search Engine data

Get the lyrics for the song currently playing on Spotify

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structured data from pages.

Social media (Weibo) comments analyzing toolbox in Chinese 微博评论分析工具, 实现功能: 1.微博评论数据爬取; 2.分词与关键词提取; 3.词云与词频统计; 4.情感分析; 5.主题聚类

📢 Ptt 文章通知機器人！Notify Ptt Article in Realtime

根据腾讯安全应急响应中心的架构编写的一款超强爬虫（广度优先搜索）

由Python编写的全异步实现的动漫之家(dmzj)漫画批量下载器（爬虫）

Python爬虫和Flask实现小说网站

基于NodeJS的基金数据爬虫，爬取的数据存于github的@nullpointer/fund-data。

Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt

A Strong, Fast and Flexible Pixiv Client based on .NET Core and WPF

Creating Scrapy scrapers via the Django admin interface

Polite, slim and concurrent web crawler.