LKI / Wescraper
依赖Scrapy和搜狗搜索微信公众号文章
Stars: ✭ 46
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Wescraper
Ecommercecrawlers
码云仓库链接:AJay13/ECommerceCrawlers
Github 仓库链接:DropsDevopsOrg/ECommerceCrawlers
项目展示平台链接:http://wechat.doonsec.com
Stars: ✭ 3,073 (+6580.43%)
Mutual labels: scrapy, wechat
Workmanger
laravel+vue.js 前后端分离实战项目(项目中wx端等已经存在,因为涉及业务较多,不再更新到github 需要参考可以私聊我)
Stars: ✭ 41 (-10.87%)
Mutual labels: wechat
Springboot Project
这是一个用SpringBoot+JPA打造的基于微信服务号平台的点餐系统的后端,前端是一个使用Vue.js构建的运行于微信公众号的WebApp。前后端完全分离,前后端之间通过RESTful风格的接口相连。
Stars: ✭ 1,034 (+2147.83%)
Mutual labels: wechat
Leavexchat Bot
Keep WeChat messages but leave WeChat. This Telegram bot is here to release you.
Stars: ✭ 32 (-30.43%)
Mutual labels: wechat
Kotlin Firebase Group Chat
Group and OneonOne chat using firebase built in Kotlin similar to whatsapp.
Stars: ✭ 44 (-4.35%)
Mutual labels: wechat
Docker Wxwork
DoWork is a Dockerized WeChat Work (盒装企业微信) PC Windows Client for Linux
Stars: ✭ 40 (-13.04%)
Mutual labels: wechat
Tom Vpr
how does voiceprint recognition work in wechat page
Stars: ✭ 37 (-19.57%)
Mutual labels: wechat
Crawlab
Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
Stars: ✭ 8,392 (+18143.48%)
Mutual labels: scrapy
Scrapymon
Simple Web UI for Scrapy spider management via Scrapyd
Stars: ✭ 35 (-23.91%)
Mutual labels: scrapy
Django Dynamic Scraper
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+2126.09%)
Mutual labels: scrapy
Mpvue Animated Number
微信小程序 mpvue 数字更新滚动动画组件 / An animated number component for mpvue
Stars: ✭ 33 (-28.26%)
Mutual labels: wechat
App comments spider
爬取百度贴吧、TapTap、appstore、微博官方博主上的游戏评论(基于redis_scrapy),过滤器采用了bloomfilter。
Stars: ✭ 38 (-17.39%)
Mutual labels: scrapy
WeScraper (WEchat SCRAPER)
本工具使用Python2.7和scrapy来搜索微信公众号文章。
使用教程
命令行直接查询
安装Scrapy,直接查询。
pip install scrapy
python wescraper/scraper.py account liriansu miawu > we.json # 查询liriansu和miawu相关的公众号
python wescraper/scraper.py key-day liriansu miawu > we.json # 查询liriansu和miawu相关的文章(一天内)
Web Server查询
安装Scrapy与Tornado,通过本地server查询:
pip install scrapy tornado
python wescraper/server.py
在server起来以后就可以通过http://localhost/account/foo/bar/baz...
来获取微信公众号文章列表了。
或者可以通过http://localhost/key-year/foo/bar/baz...
以关键字来查询公众号文章。
Python Code调用
参见scraper.py源码
详细说明
-
一些可配置的参数见config.py
-
查询公众号默认获取列表的第一个。
-
本工具有可能会被Ban,解决方案可以参考Scrapy: Avoiding getting banned (一般而言,换IP就可以解决问题了)
-
cookie.py内维护了一个Cookie池,会在n个Cookie中随机选取来访问,假如Cookie被ban了就会换一个Cookie。
-
欢迎在本代码基础上修改,记得跑一下单元测试噢:
python wescraper/test/test.py
-
本工具完全依赖搜狗微信搜索抓取文章,假如搜狗微信搜索接口什么的变了可能就会抓取失败。
版权/免责
代码版权归GitHub原作者 @LKI 所有。 严禁用于商业用途,其它转载/Fork随意。
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].