yan68 / Xcrawler
快速、简洁且强大的PHP爬虫框架
Stars: ✭ 344
Projects that are alternatives of or similar to Xcrawler
Avbook
AV 电影管理系统， avmoo , javbus , javlibrary 爬虫，线上 AV 影片图书馆，AV 磁力链接数据库，Japanese Adult Video Library,Adult Video Magnet Links - Japanese Adult Video Database
Stars: ✭ 8,133 (+2264.24%)
Mutual labels: crawler, spider, scraper
Spidr
A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+90.7%)
Mutual labels: crawler, spider, scraper
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-44.77%)
Mutual labels: crawler, spider, scraper
Awesome Crawler
A collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+1293.31%)
Mutual labels: crawler, spider, scraper
Scrapit
Scraping scripts for various websites.
Stars: ✭ 25 (-92.73%)
Mutual labels: crawler, spider, scraper
Crawly
Crawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+27.91%)
Mutual labels: crawler, spider, scraper
Linkedin Profile Scraper
🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (-50.29%)
Mutual labels: crawler, spider, scraper
Not Your Average Web Crawler
A web crawler (for bug hunting) that gathers more than you can imagine.
Stars: ✭ 107 (-68.9%)
Mutual labels: crawler, spider, scraper
Freshonions Torscraper
Fresh Onions is an open source TOR spider / hidden service onion crawler hosted at zlal32teyptf4tvi.onion
Stars: ✭ 348 (+1.16%)
Mutual labels: crawler, spider, scraper
Colly
Elegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+4415.99%)
Mutual labels: crawler, spider, scraper
Crawler
A high performance web crawler in Elixir.
Stars: ✭ 781 (+127.03%)
Mutual labels: crawler, spider, scraper
Geziyor
Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+262.21%)
Mutual labels: crawler, spider, scraper
Querylist
🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+595.35%)
Mutual labels: crawler, spider, scraper
arachnod
High performance crawler for Nodejs
Stars: ✭ 17 (-95.06%)
Mutual labels: crawler, scraper, spider
lightnovel epub
🍭 epub generator for (light)novels (轻) 小说 epub 生成器，支持站点：轻之国度、轻小说文库
Stars: ✭ 89 (-74.13%)
Mutual labels: crawler, scraper
galer
A fast tool to fetch URLs from HTML attributes by crawl-in.
Stars: ✭ 138 (-59.88%)
Mutual labels: crawler, spider
Weibo terminator workflow
Update Version of weibo_terminator, This is Workflow Version aim at Get Job Done!
Stars: ✭ 259 (-24.71%)
Mutual labels: crawler, scraper
XCrawler - 轻量级、易维护的PHP爬虫框架
特性
- 极易上手，具备完善的文档和示例
- 支持多并发爬取
- 支持失败重试、代理、断点续爬
- 完善的爬取进度日志
- 支持基于xpath、css选择器解析内容
- 支持模拟浏览器行为，如user-agent、cookie、提交表单
- 健壮的底层库，HTTP客户端基于guzzle，dom解析基于symfony/dom-crawler库
安装
composer require xcrawler/xcrawler
文档地址
查看文档: https://xcrawler.yanshuju.com/docs/
交流
XCrawler交流群: 790478771
开源协议 (License)
XCrawler是基于MIT开源协议发布的。
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].