FbcrawlA Facebook crawler
Stars: ✭ 536 (+2336.36%)
Mutual labels: scraper, facebook, spider, scrapy
Django Dynamic ScraperCreating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+4554.55%)
Mutual labels: scraper, spider, scraping, scrapy
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+1900%)
Mutual labels: scraper, spider, scraping
MailinglistscraperA python web scraper for public email lists.
Stars: ✭ 19 (-13.64%)
Mutual labels: scraper, spider, scrapy
scrapy-distributedA series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.
Stars: ✭ 38 (+72.73%)
Mutual labels: spider, scraping, scrapy
wget-luaWget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Stars: ✭ 52 (+136.36%)
Mutual labels: scraper, spider, scraping
LinkedinLinkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (+1304.55%)
Mutual labels: scraper, scraping, scrapy
OpenScraperAn open source webapp for scraping: towards a public service for webscraping
Stars: ✭ 80 (+263.64%)
Mutual labels: scraper, spider, scrapy
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+5563.64%)
Mutual labels: scraper, spider, scraping
SeleniumcrawlerAn example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (+431.82%)
Mutual labels: scraper, scraping, scrapy
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+677.27%)
Mutual labels: scraper, spider, scraping
facebook-discussion-tkA collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.
Stars: ✭ 33 (+50%)
Mutual labels: scraper, facebook, scraping
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+70513.64%)
Mutual labels: scraper, spider, scraping
Email ExtractorThe main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (+268.18%)
Mutual labels: scraper, scraping, scrapy
Goribot[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (+763.64%)
Mutual labels: scraper, spider, scrapy
crawler-chrome-extensions爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer
Stars: ✭ 53 (+140.91%)
Mutual labels: scraper, spider, scraping
V2EX SpiderV2EX爬虫
Stars: ✭ 21 (-4.55%)
Mutual labels: spider, scrapy
elves🎊 Design and implement of lightweight crawler framework.
Stars: ✭ 322 (+1363.64%)
Mutual labels: spider, scrapy
OLX Scraper📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.
Stars: ✭ 15 (-31.82%)
Mutual labels: scraper, scrapy
163Music163music spider by scrapy.
Stars: ✭ 60 (+172.73%)
Mutual labels: spider, scrapy