SitedorksSearch Google/Bing/Ecosia/DuckDuckGo/Yandex/Yahoo for a search term with a default set of websites, bug bounty programs or a custom collection.
Stars: ✭ 221 (-81.16%)
Xinahn Socket一个开源,高隐私,自架自用的聚合搜索引擎。 https://xinahn.com
Stars: ✭ 77 (-93.44%)
JsearchjSearch(聚搜) 是一款专注内容的chrome搜索扩展,一次搜索聚合多平台内容。
Stars: ✭ 193 (-83.55%)
Translators🌏🌍🌎Translators🌎🌍🌏 is a library which aims to bring free, multiple, enjoyable translation to individuals and students in Python. Translators是一个旨在用Python为个人和学生带来免费、多样、愉快翻译的库。
Stars: ✭ 295 (-74.85%)
scrapy helperDynamic configurable crawl (动态可配置化爬虫)
Stars: ✭ 84 (-92.84%)
devsearchA web search engine built with Python which uses TF-IDF and PageRank to sort search results.
Stars: ✭ 52 (-95.57%)
Alipayspider ScrapyAlipaySpider on Scrapy(use chrome driver); 支付宝爬虫(基于Scrapy)
Stars: ✭ 70 (-94.03%)
toutiao今日头条科技新闻接口爬虫
Stars: ✭ 17 (-98.55%)
ip proxy poolGenerating spiders dynamically to crawl and check those free proxy ip on the internet with scrapy.
Stars: ✭ 39 (-96.68%)
Search Engine ParserLightweight package to query popular search engines and scrape for result titles, links and descriptions
Stars: ✭ 216 (-81.59%)
BaiduSpider项目已经移动至:https://github.com/BaiduSpider/BaiduSpider !! 一个爬取百度搜索结果的爬虫,目前支持百度网页搜索,百度图片搜索,百度知道搜索,百度视频搜索,百度资讯搜索,百度文库搜索,百度经验搜索和百度百科搜索。
Stars: ✭ 29 (-97.53%)
NScrapyNScrapy is a .net core corss platform Distributed Spider Framework which provide an easy way to write your own Spider
Stars: ✭ 88 (-92.5%)
Web-IotaIota is a web scraper which can find all of the images and links/suburls on a webpage
Stars: ✭ 60 (-94.88%)
scrapy-distributedA series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
Stars: ✭ 38 (-96.76%)
python-spiderpython爬虫小项目【持续更新】【笔趣阁小说下载、Tweet数据抓取、天气查询、网易云音乐逆向、天天基金网查询、微博数据抓取(生成cookie)、有道翻译逆向、企查查免登陆爬虫、大众点评svg加密破解、B站用户爬虫、拉钩免登录爬虫、自如租房字体加密、知乎问答
Stars: ✭ 45 (-96.16%)
elves🎊 Design and implement of lightweight crawler framework.
Stars: ✭ 322 (-72.55%)
Elves🎊 Design and implement of lightweight crawler framework.
Stars: ✭ 315 (-73.15%)
Haipproxy💖 High available distributed ip proxy pool, powerd by Scrapy and Redis
Stars: ✭ 4,993 (+325.66%)
Python PiscesA tools to search for and download images by keywords using search engines: google/baidu/yahoo/bing. 使用google等搜索引擎搜索关键词并下载图片
Stars: ✭ 60 (-94.88%)
FbcrawlA Facebook crawler
Stars: ✭ 536 (-54.31%)
Python Spider豆瓣电影top250、斗鱼爬取json数据以及爬取美女图片、淘宝、有缘、CrawlSpider爬取红娘网相亲人的部分基本信息以及红娘网分布式爬取和存储redis、爬虫小demo、Selenium、爬取多点、django开发接口、爬取有缘网信息、模拟知乎登录、模拟github登录、模拟图虫网登录、爬取多点商城整站数据、爬取微信公众号历史文章、爬取微信群或者微信好友分享的文章、itchat监听指定微信公众号分享的文章
Stars: ✭ 615 (-47.57%)
Sou简单搜索,一个简单的前端界面。用惯了各种导航首页,满屏幕尽是各种不厌其烦的广告和资讯;尝试自己写个自己的主页。
Stars: ✭ 628 (-46.46%)
Image searchPython Library to download images and metadata from popular search engines.
Stars: ✭ 86 (-92.67%)
JspiderJSpider会每周更新至少一个网站的JS解密方式,欢迎 Star,交流微信:13298307816
Stars: ✭ 914 (-22.08%)
CrawlabDistributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
Stars: ✭ 8,392 (+615.43%)
163Music163music spider by scrapy.
Stars: ✭ 60 (-94.88%)
OpenScraperAn open source webapp for scraping: towards a public service for webscraping
Stars: ✭ 80 (-93.18%)
scrapy-adminA django admin site for scrapy
Stars: ✭ 44 (-96.25%)
scrapy facebookerCollection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (-98.12%)
PttImageSpiderPTT 圖片下載器 (抓取整個看板的圖片,並用文章標題作為資料夾的名稱 ) (使用Scrapy)
Stars: ✭ 16 (-98.64%)
Magic googleGoogle search results crawler, get google search results that you need
Stars: ✭ 247 (-78.94%)
AlltheplacesA set of spiders and scrapers to extract location information from places that post their location on the internet.
Stars: ✭ 277 (-76.39%)
Socialite Socialite is an OAuth2 Authentication tool. It is inspired by laravel/socialite, you can easily use it without Laravel.
Stars: ✭ 1,026 (-12.53%)
Crow TranslateA simple and lightweight translator that allows to translate and speak text using Google, Yandex and Bing.
Stars: ✭ 503 (-57.12%)
Happy Spiders🔧 🔩 🔨 收集整理了爬虫相关的工具、模拟登陆技术、代理IP、scrapy模板代码等内容。
Stars: ✭ 261 (-77.75%)
Web kg爬取百度百科中文页面,抽取三元组信息,构建中文知识图谱
Stars: ✭ 549 (-53.2%)
Search DeflectorA small program that forwards searches from Cortana to your preferred browser and search engine.
Stars: ✭ 620 (-47.14%)
SeekerSeeker - another job board aggregator.
Stars: ✭ 16 (-98.64%)
FunpyspidersearchengineWord2vec 千人千面 个性化搜索 + Scrapy2.3.0(爬取数据) + ElasticSearch7.9.1(存储数据并提供对外Restful API) + Django3.1.1 搜索
Stars: ✭ 782 (-33.33%)
App comments spider爬取百度贴吧、TapTap、appstore、微博官方博主上的游戏评论(基于redis_scrapy),过滤器采用了bloomfilter。
Stars: ✭ 38 (-96.76%)
MtransMulti-source Translation
Stars: ✭ 711 (-39.39%)
Dc SdkDC-SDK 是基于 Cesium 进行二次开发的2、3D一体 WebGis 应用框架,该框架优化了 Cesium 的使用方式和增添了一些额外功能,旨在为开发者快速构建 WebGis 应用。🌎
Stars: ✭ 206 (-82.44%)
Douban CrawlerUno Crawler por https://douban.com
Stars: ✭ 13 (-98.89%)
IcrawlerA multi-thread crawler framework with many builtin image crawlers provided.
Stars: ✭ 629 (-46.38%)
Reptile🏀 Python3 网络爬虫实战(部分含详细教程)猫眼 腾讯视频 豆瓣 研招网 微博 笔趣阁小说 百度热点 B站 CSDN 网易云阅读 阿里文学 百度股票 今日头条 微信公众号 网易云音乐 拉勾 有道 unsplash 实习僧 汽车之家 英雄联盟盒子 大众点评 链家 LPL赛程 台风 梦幻西游、阴阳师藏宝阁 天气 牛客网 百度文库 睡前故事 知乎 Wish
Stars: ✭ 1,048 (-10.66%)