Lodour / Weibo Album Crawler
Licence: mit
新浪微博相册大图多线程爬虫。
Stars: ✭ 83
Programming Languages
Projects that are alternatives of or similar to Weibo Album Crawler
Decryptlogin
APIs for loginning some websites by using requests.
Stars: ✭ 1,861 (+2142.17%)
Mutual labels: crawler, requests, weibo
WeiboCrawler
无cookie版微博爬虫，可以连续爬取一个或多个新浪微博用户信息、用户微博及其微博评论转发。
Stars: ✭ 45 (-45.78%)
Mutual labels: crawler, weibo
Bilili
🍻 bilibili video (including bangumi) and danmaku downloader | B站视频（含番剧）、弹幕下载器
Stars: ✭ 379 (+356.63%)
Mutual labels: crawler, requests
Weibopicdownloader
免登录下载微博图片 爬虫 Download Weibo Images without Logging-in
Stars: ✭ 247 (+197.59%)
Mutual labels: crawler, weibo
python3-concurrency
Python3爬虫系列的理论验证，首先研究I/O模型，分别用Python实现了blocking I/O、nonblocking I/O、I/O multiplexing各模型下的TCP服务端和客户端。然后，研究同步I/O操作（依序下载、多进程并发、多线程并发）和异步I/O（asyncio）之间的效率差别
Stars: ✭ 49 (-40.96%)
Mutual labels: requests, futures
Weibo Topic Spider
微博超级话题爬虫，微博词频统计+情感分析+简单分类，新增肺炎超话爬取数据
Stars: ✭ 128 (+54.22%)
Mutual labels: crawler, weibo
Easy Scraping Tutorial
Simple but useful Python web scraping tutorial code.
Stars: ✭ 583 (+602.41%)
Mutual labels: crawler, requests
Course Crawler
🎓 中国大学MOOC、学堂在线、网易云课堂、好大学在线、爱课程 MOOC 课程下载。
Stars: ✭ 611 (+636.14%)
Mutual labels: crawler, requests
Price Monitor
京东商品价格监控：监控用户设定商品价格，降价邮件/微信提醒。技术：Python爬虫/IP代理池/JS接口爬取/Selenium页面爬取
Stars: ✭ 634 (+663.86%)
Mutual labels: crawler, requests
Scrapingoutsourcing
ScrapingOutsourcing专注分享爬虫代码 尽量每周更新一个
Stars: ✭ 164 (+97.59%)
Mutual labels: crawler, requests
weibo topic
微博话题关键词,个人微博采集, 微博博文一键删除 selenium获取cookie,requests处理
Stars: ✭ 28 (-66.27%)
Mutual labels: requests, weibo
Python3 Concurrency Pics 02
爬取 www.mzitu.com 全站图片，截至目前共5162个图集，16.5万多张美女图片，使用 asyncio 和 aiohttp 实现的异步版本只需要不到2小时就能爬取完成。按日期创建图集目录，保存更合理。控制台只显示下载的进度条，详细信息保存在日志文件中。支持异常处理，不会终止爬虫程序。失败的请求，下次再执行爬虫程序时会自动下载
Stars: ✭ 275 (+231.33%)
Mutual labels: futures, requests
Social Scraper
Tổng hợp script crawl dữ liệu từ các mạng xã hội & website tiếng Việt
Stars: ✭ 47 (-43.37%)
Mutual labels: crawler, requests
Docs
《数据采集从入门到放弃》源码。内容简介：爬虫介绍、就业情况、爬虫工程师面试题 ；HTTP协议介绍； Requests使用 ；解析器Xpath介绍； MongoDB与MySQL； 多线程爬虫； Scrapy介绍 ；Scrapy-redis介绍； 使用docker部署； 使用nomad管理docker集群； 使用EFK查询docker日志
Stars: ✭ 118 (+42.17%)
Mutual labels: crawler, requests
Sina Weibo Album Downloader
Multithreading download all HD photos / pictures from someone's Sina Weibo album.
Stars: ✭ 125 (+50.6%)
Mutual labels: crawler, weibo
Weibo Analyst
Social media (Weibo) comments analyzing toolbox in Chinese 微博评论分析工具, 实现功能: 1.微博评论数据爬取; 2.分词与关键词提取; 3.词云与词频统计; 4.情感分析; 5.主题聚类
Stars: ✭ 430 (+418.07%)
Mutual labels: crawler, weibo
Weibo Album Crawler
新浪微博相册多线程爬虫。
Usage
-
安装
git clone [email protected]:Lodour/Weibo-Album-Crawler.git cd Weibo-Album-Crawler virtualenv env --python=python3.5 source ./env/bin/activate pip install -r requirements.txt mv settings.sample.py settings.py
-
设置
settings.py-
STORE_PATH下载目录 -
COOKIES任意用户微博的 Cookies，或设置为AUTO以自动获取（需安装 ChromeDriver） -
TARGETS目标用户的微博主页url
-
-
运行
python main.py
License
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].