wget-luaWget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Stars: ✭ 52 (-1.89%)
Mutual labels: scraper, spider, scraping, crawl
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+2250.94%)
Mutual labels: scraper, spider, scraping
FbcrawlA Facebook crawler
Stars: ✭ 536 (+911.32%)
Mutual labels: scraper, spider, crawl
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+222.64%)
Mutual labels: scraper, spider, scraping
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+29211.32%)
Mutual labels: scraper, spider, scraping
scrapy facebookerCollection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (-58.49%)
Mutual labels: scraper, spider, scraping
diffbot-php-client[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library
Stars: ✭ 53 (+0%)
Mutual labels: scraper, scraping, crawl
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+730.19%)
Mutual labels: scraper, spider, scraping
Django Dynamic ScraperCreating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+1832.08%)
Mutual labels: scraper, spider, scraping
UdemycoursegrabberYour will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!
Stars: ✭ 137 (+158.49%)
Mutual labels: scraper, scraping
PhpscraperPHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+179.25%)
Mutual labels: scraper, scraping
SerpscrapSEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Stars: ✭ 153 (+188.68%)
Mutual labels: scraper, scraping
SeleniumcrawlerAn example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (+120.75%)
Mutual labels: scraper, scraping
Not Your Average Web CrawlerA web crawler (for bug hunting) that gathers more than you can imagine.
Stars: ✭ 107 (+101.89%)
Mutual labels: scraper, spider
Jsonframe Cheeriosimple multi-level scraper json input/output for Cheerio
Stars: ✭ 196 (+269.81%)
Mutual labels: scraper, scraping
Anime DlAnime-dl is a command-line program to download anime from CrunchyRoll and Funimation.
Stars: ✭ 190 (+258.49%)
Mutual labels: scraper, scraping
Email ExtractorThe main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (+52.83%)
Mutual labels: scraper, scraping
Goribot[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (+258.49%)
Mutual labels: scraper, spider
gospider⚡ Light weight Golang spider framework | 轻量的 Golang 爬虫框架
Stars: ✭ 183 (+245.28%)
Mutual labels: spider, crawl
Goose ParserUniversal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (+298.11%)
Mutual labels: scraper, scraping