All Projects → guyueyingmu → Avbook

guyueyingmu / Avbook

Licence: mit
AV 电影管理系统, avmoo , javbus , javlibrary 爬虫,线上 AV 影片图书馆,AV 磁力链接数据库,Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database

Programming Languages

PHP
23972 projects - #3 most used programming language
Blade
752 projects

Projects that are alternatives of or similar to Avbook

Core
🔞 JAVClub - 让你的大姐姐不再走丢
Stars: ✭ 2,728 (-66.46%)
Mutual labels:  spider, magnet, adult, javbus
Bt Btt
磁力網站U3C3介紹以及域名更新
Stars: ✭ 261 (-96.79%)
Mutual labels:  crawler, spider, magnet, magnet-link
Scrapit
Scraping scripts for various websites.
Stars: ✭ 25 (-99.69%)
Mutual labels:  crawler, spider, scraper
torrent-spider
基于DHT的p2p网络资源爬虫
Stars: ✭ 65 (-99.2%)
Mutual labels:  magnet-link, magnet, adult
Xcrawler
快速、简洁且强大的PHP爬虫框架
Stars: ✭ 344 (-95.77%)
Mutual labels:  crawler, spider, scraper
Querylist
🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (-70.59%)
Mutual labels:  crawler, spider, scraper
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+91.01%)
Mutual labels:  crawler, spider, scraper
arachnod
High performance crawler for Nodejs
Stars: ✭ 17 (-99.79%)
Mutual labels:  crawler, scraper, spider
Geziyor
Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (-84.68%)
Mutual labels:  crawler, spider, scraper
Crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (-94.59%)
Mutual labels:  crawler, spider, scraper
Gosint
OSINT Swiss Army Knife
Stars: ✭ 401 (-95.07%)
Mutual labels:  crawler, spider, scraper
Awesome Crawler
A collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (-41.07%)
Mutual labels:  crawler, spider, scraper
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-97.66%)
Mutual labels:  crawler, spider, scraper
Linkedin Profile Scraper
🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-97.9%)
Mutual labels:  crawler, spider, scraper
Laravel Crawler Detect
A Laravel wrapper for CrawlerDetect - the web crawler detection library
Stars: ✭ 227 (-97.21%)
Mutual labels:  crawler, spider, laravel
Not Your Average Web Crawler
A web crawler (for bug hunting) that gathers more than you can imagine.
Stars: ✭ 107 (-98.68%)
Mutual labels:  crawler, spider, scraper
Gmdb
GMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)
Stars: ✭ 189 (-97.68%)
Mutual labels:  scraper, magnet, magnet-link
Spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (-91.93%)
Mutual labels:  crawler, spider, scraper
Freshonions Torscraper
Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion
Stars: ✭ 348 (-95.72%)
Mutual labels:  crawler, spider, scraper
Fbcrawl
A Facebook crawler
Stars: ✭ 536 (-93.41%)
Mutual labels:  crawler, spider, scraper

关于 AVBook

AVBook 是基于 Laravel 开发的 Web 应用程序,通过 Artisan 控制台实现avmoo,javbus,javlibrary的爬虫采集数据入库mysql,前端页面模仿javbus,并添加一些类似javlibrary的功能。该系统相当于以上三个网站的结合体,能够使你更加方便地管理你的影片。

预览

(FBI WARNING:不要在公共场所打开以下图片)

🔞 首页截图

🔞 详情页截图

Get Started

git clone https://github.com/guyueyingmu/avbook.git

cd avbook;cp .env.example .env #复制 .env.example 为 .env 并设置数据库信息

composer install

php artisan migrate

#替换  avbook/config/urlconfig.php  文件中的域名为最新域名 #从被墙域名获取:avmoo.com ;javbus.com;javlib.com

php artisan avbook:avmoo  #启动avmoo爬虫

php artisan avbook:javbus --movie=1 --page=10 --magpage=10 #启动javbus爬虫

php artisan scandir --path='/moviefiles' #扫描指定目录 /moviefiles 中的文件,正则匹配番号,设置为已拥有状态

php artisan avbook:javlib --genre --movie  #javlibrary 爬虫

以上操作适用于Linux环境

For windows 用户

这里提供一份开箱即用的压缩包 avbook_laragon.7z

链接:https://pan.baidu.com/s/1JCfISyckxy9qNtYjo5RH-w 提取码:xug8

备用下载地址(慢):releases

压缩包中包含更新到2019.5.12的全部数据

使用方法:

1.解压到 D:\laragon
2.打开 D:\laragon\laragon.exe ,点击 启动所有
3.点击 网站 或者访问 http://avbook.test enjoy it

ps:
Ⅰ. 如果没有解压到 D:\laragon 需自行修改Nginx配置,php.ini配置,并在hosts文件新增一行 :127.0.0.1 avbook.test
Ⅱ. laragon终端启动爬虫前先 git pull 更新到最新版本。

其他注意事项

avmoo 有反爬虫机制,目前的反反爬虫方法是修改 User-Agent ,用的人多了应该会失效,建议下载avbook_laragon.7z导入数据库,进行增量更新。

windows 下 mysql 性能有限,需耐心等待,建议将数据库导入 homestead 使用。

在带宽够用的情况下 php artisan avbook:avmoo --max=500 将并发设置为500可在1小时内采集完 avmoo 全站30余万条电影信息,并发过大时建议使用 SMProxy 加一层连接池提高速度。

必须等 avmoo 采集完成,才能启动 javbus 爬虫。

最后,北京第三区交通委提醒您:道路千万条,安全第一条。飙车一时爽,一直飙一直爽。。。

2019.7.19 up,不好意思,你来晚了,爬虫已失效#issues/60,车暂时开不动了,重新发车时间待定,关注公众号:guyueyingmu (孤月盈木)接收通知。

License

The AVBook is open-source software licensed under the MIT license.

gyym.avbook.bt.01

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].