JayBizzle / Laravel Crawler Detect
Licence: mit
A Laravel wrapper for CrawlerDetect - the web crawler detection library
Stars: ✭ 227
Projects that are alternatives of or similar to Laravel Crawler Detect
Arachnid
Powerful web scraping framework for Crystal
Stars: ✭ 68 (-70.04%)
Mutual labels: bot, crawler, spider
Avbook
AV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+3482.82%)
Mutual labels: crawler, spider, laravel
Scrapit
Scraping scripts for various websites.
Stars: ✭ 25 (-88.99%)
Mutual labels: bot, crawler, spider
Crawler Detect
🕷 CrawlerDetect is a PHP class for detecting bots/crawlers/spiders via the user agent
Stars: ✭ 1,549 (+582.38%)
Mutual labels: crawler, spider, detect
Ncov2019 data crawler
疫情数据爬虫,2019新型冠状病毒数据仓库,轨迹数据,同乘数据,报道
Stars: ✭ 175 (-22.91%)
Mutual labels: crawler, spider
Spoon
🥄 A package for building specific Proxy Pool for different Sites.
Stars: ✭ 173 (-23.79%)
Mutual labels: crawler, spider
Lianjia Beike Spider
链家网和贝壳网房价爬虫,采集北京上海广州深圳等21个中国主要城市的房价数据(小区,二手房,出租房,新房),稳定可靠快速!支持csv,MySQL, MongoDB,Excel, json存储,支持Python2和3,图表展示数据,注释丰富 ,点星支持,仅供学习参考,请勿用于商业用途,后果自负。
Stars: ✭ 2,257 (+894.27%)
Mutual labels: crawler, spider
Fooproxy
稳健高效的评分制-针对性- IP代理池 + API服务,可以自己插入采集器进行代理IP的爬取,针对你的爬虫的一个或多个目标网站分别生成有效的IP代理数据库,支持MongoDB 4.0 使用 Python3.7(Scored IP proxy pool ,customise proxy data crawler can be added anytime)
Stars: ✭ 195 (-14.1%)
Mutual labels: crawler, spider
Marmot
💐Marmot | Web Crawler/HTTP protocol Download Package 🐭
Stars: ✭ 186 (-18.06%)
Mutual labels: crawler, spider
Ok ip proxy pool
🍿爬虫代理IP池(proxy pool) python🍟一个还ok的IP代理池
Stars: ✭ 196 (-13.66%)
Mutual labels: crawler, spider
Linkedin Profile Scraper
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-24.67%)
Mutual labels: crawler, spider
Zhihu Crawler People
A simple distributed crawler for zhihu && data analysis
Stars: ✭ 182 (-19.82%)
Mutual labels: crawler, spider
Gain
Web crawling framework based on asyncio.
Stars: ✭ 2,002 (+781.94%)
Mutual labels: crawler, spider
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-16.3%)
Mutual labels: crawler, spider
Querylist
🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+953.74%)
Mutual labels: crawler, spider
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+6743.61%)
Mutual labels: crawler, spider
Jd mask robot
京东口罩库存监控爬虫(非selenium),扫码登录、查价、加购、下单、秒杀
Stars: ✭ 216 (-4.85%)
Mutual labels: crawler, spider
Laravel Crawler Detect
A Laravel wrapper for CrawlerDetect - the web crawler detection library
Installation
Run composer require jaybizzle/laravel-crawler-detect 1.*
or add "jaybizzle/laravel-crawler-detect": "1.*"
to your composer.json
file.
The last version compatible with Laravel 4 was v1.0.2 so if you need that, you will have to fix your composer.json
to that specific version.
Add the following to the providers
array in your config/app.php
file..
Jaybizzle\LaravelCrawlerDetect\LaravelCrawlerDetectServiceProvider::class,
...and the following to your aliases
array...
'Crawler' => Jaybizzle\LaravelCrawlerDetect\Facades\LaravelCrawlerDetect::class,
Laravel 5.5 uses Package Auto-Discovery, so doesn't require you to manually add the ServiceProvider.
Usage
use Crawler;
// Check current 'visitors' user agent
if(Crawler::isCrawler()) {
// true if crawler user agent detected
}
// Pass a user agent as a string
if(Crawler::isCrawler('Mozilla/5.0 (compatible; aiHitBot/2.9; +https://www.aihitdata.com/about)')) {
// true if crawler user agent detected
}
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].