OLX Scraper📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.
Stars: ✭ 15 (-73.21%)
AzurLaneWikiScrapersA console application that can scrape the Azur Lane wiki and export the data to Json files
Stars: ✭ 12 (-78.57%)
Scrapysharpreborn of https://bitbucket.org/rflechner/scrapysharp
Stars: ✭ 226 (+303.57%)
CascadiaGo cascadia package command line CSS selector
Stars: ✭ 67 (+19.64%)
CVparserCVparser is software for parsing or extracting data out of CV/resumes.
Stars: ✭ 28 (-50%)
Linkedin-ClientWeb scraper for grabing data from Linkedin profiles or company pages (personal project)
Stars: ✭ 42 (-25%)
GetsyA simple browser/client-side web scraper.
Stars: ✭ 238 (+325%)
Goose ParserUniversal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (+276.79%)
PhpscraperPHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+164.29%)
python3-malPython interface to MyAnimeList
Stars: ✭ 18 (-67.86%)
ScrapersA list of scrapers from around the web.
Stars: ✭ 366 (+553.57%)
pysub-parserLibrary for extracting text and timestamps from multiple subtitle files (.ass, .ssa, .srt, .sub, .txt).
Stars: ✭ 40 (-28.57%)
ScrapeMA monadic web scraping library
Stars: ✭ 17 (-69.64%)
JikanUnofficial MyAnimeList PHP+REST API which provides functions other than the official API
Stars: ✭ 531 (+848.21%)
Scrape Linkedin Selenium`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Stars: ✭ 239 (+326.79%)
extract-emailsExtract emails from a given website
Stars: ✭ 58 (+3.57%)
Link Preview JsParse and/or extract web links meta information: title, description, images, videos, etc. [via OpenGraph], runs on mobiles and node.
Stars: ✭ 240 (+328.57%)
Awesome CrawlerA collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+8458.93%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+1071.43%)
Anime DlAnime-dl is a command-line program to download anime from CrunchyRoll and Funimation.
Stars: ✭ 190 (+239.29%)
Goribot[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (+239.29%)
GmdbGMDB is the ultra-simple, cross-platform Movie Library with Features (Search, Take Note, Watch Later, Like, Import, Learn, Instantly Torrent Magnet Watch)
Stars: ✭ 189 (+237.5%)
Unhtml.rsA magic html parser
Stars: ✭ 180 (+221.43%)
DotGrokParse text with pattern. Inspired by grok filter.
Stars: ✭ 26 (-53.57%)
Annie👾 Fast and simple video download library and CLI tool written in Go
Stars: ✭ 16,369 (+29130.36%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+205.36%)
Thepiratebay💀 The Pirate Bay node.js client
Stars: ✭ 191 (+241.07%)
BlacksmithBlacksmith is a tool for viewing, extracting, and converting textures, 3D models, and sounds from Assassin's Creed: Odyssey/Origins/Valhalla and Steep.
Stars: ✭ 104 (+85.71%)
Novel基于 Laravel 5.2 的小说网站
Stars: ✭ 172 (+207.14%)
Skrape.itA Kotlin-based testing/scraping/parsing library providing the ability to analyze and extract data from HTML (server & client-side rendered). It places particular emphasis on ease of use and a high level of readability by providing an intuitive DSL. It aims to be a testing lib, but can also be used to scrape websites in a convenient fashion.
Stars: ✭ 231 (+312.5%)
Instagram CrawlerCrawl instagram photos, posts and videos for download.
Stars: ✭ 178 (+217.86%)
MangDLThe most inefficient Manga downloader for PC
Stars: ✭ 40 (-28.57%)
ReadablewebproxyRewriting web proxy and archival tool. At this point, it just tries to download all the things.
Stars: ✭ 172 (+207.14%)
Ruiji.netcrawler framework, distributed crawler extractor
Stars: ✭ 220 (+292.86%)
Scrapelib⛏ a library for scraping things
Stars: ✭ 164 (+192.86%)
Scrape Twitter🐦 Access Twitter data without an API key. [DEPRECATED]
Stars: ✭ 166 (+196.43%)
autumnA Java parser combinator library written with an unmatched feature set.
Stars: ✭ 112 (+100%)
Datmusic ApiAlternative for VK Audio API
Stars: ✭ 160 (+185.71%)
OpensanctionsAn open database of international sanctions data, persons of interest and politically exposed persons
Stars: ✭ 157 (+180.36%)
Covid19 mobilityCOVID-19 Mobility Data Aggregator. Scraper of Google, Apple, Waze and TomTom COVID-19 Mobility Reports🚶🚘🚉
Stars: ✭ 156 (+178.57%)
postcss-jsxPostCSS syntax for parsing CSS in JS literals
Stars: ✭ 73 (+30.36%)
CaptCCA tiny C compiler written purely in JavaScript.
Stars: ✭ 175 (+212.5%)
TwitterScraperScrape a User's Twitter data! Bypass the 3,200 tweet API limit for a User!
Stars: ✭ 80 (+42.86%)
Media ScraperScrapes all photos and videos in a web page / Instagram / Twitter / Tumblr / Reddit / pixiv / TikTok
Stars: ✭ 206 (+267.86%)
Instagram Scraperscrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot
Stars: ✭ 2,209 (+3844.64%)
DemeterDemeter is a tool for scraping the calibre web ui
Stars: ✭ 155 (+176.79%)
Tianyanchapip安装的天眼查爬虫API,指定的单个/多个企业工商信息一键保存为Excel/JSON格式。A Battery-included Scraper API of Tianyancha, the best Chinese business data and investigation platform.
Stars: ✭ 206 (+267.86%)
SerpscrapSEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Stars: ✭ 153 (+173.21%)
serlistSearch engine results page scraper
Stars: ✭ 12 (-78.57%)
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+27641.07%)
Scraperwiki PythonScraperWiki Python library for scraping and saving data
Stars: ✭ 146 (+160.71%)
Weibo terminaterFinal Weibo Crawler Scrap Anything From Weibo, comments, weibo contents, followers, anything. The Terminator
Stars: ✭ 2,295 (+3998.21%)
Google2csvGoogle2Csv a simple google scraper that saves the results on a csv/xlsx/jsonl file
Stars: ✭ 145 (+158.93%)
Youtube ProjectsThis repository contains all the code I use in my YouTube tutorials.
Stars: ✭ 144 (+157.14%)
PoliteBe nice on the web
Stars: ✭ 253 (+351.79%)
Querylist🕷️ The progressive PHP crawler framework! 优雅的渐进式PHP采集框架。
Stars: ✭ 2,392 (+4171.43%)