Splashr💦 Tools to Work with the 'Splash' JavaScript Rendering Service in R
Stars: ✭ 93 (+66.07%)
Hockey ScraperPython Package for scraping NHL Play-by-Play and Shift data
Stars: ✭ 93 (+66.07%)
HumanoidNode.js package to bypass CloudFlare's anti-bot JavaScript challenges
Stars: ✭ 88 (+57.14%)
DaftlistingsA library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.
Stars: ✭ 86 (+53.57%)
RvestSimple web scraping for R
Stars: ✭ 1,253 (+2137.5%)
Detect CmsPHP Library for detecting CMS
Stars: ✭ 78 (+39.29%)
ReaderExtract clean(er), readable text from web pages via Mercury Web Parser.
Stars: ✭ 75 (+33.93%)
Ping SmReceive an email or Telegram message as soon as Migros Sanalmarket is available for delivery in your neighborhood.
Stars: ✭ 71 (+26.79%)
ArachnidPowerful web scraping framework for Crystal
Stars: ✭ 68 (+21.43%)
CascadiaGo cascadia package command line CSS selector
Stars: ✭ 67 (+19.64%)
DecapitatedHeadless 'Chrome' Orchestration in R
Stars: ✭ 65 (+16.07%)
InstagoDownload/access photos, videos, stories, story highlights, postlives, following and followers of Instagram
Stars: ✭ 59 (+5.36%)
Scrapy CraigslistWeb Scraping Craigslist's Engineering Jobs in NY with Scrapy
Stars: ✭ 54 (-3.57%)
Project TauroA Router WiFi key recovery/cracking tool with a twist.
Stars: ✭ 52 (-7.14%)
Actor Google Search ScraperApify actor that crawls Google Search result pages (SERPs) and extracts a list of organic results, ads, related queries and more. It supports selection of custom country, language and location.
Stars: ✭ 38 (-32.14%)
SnoopSnoop — инструмент разведки на основе открытых данных (OSINT world)
Stars: ✭ 886 (+1482.14%)
WebmiddleNode.js framework for modular web scraping and data extraction
Stars: ✭ 13 (-76.79%)
Letterboxd recommendationsScraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username
Stars: ✭ 23 (-58.93%)
Youtube tutorialsCollection of scripts corresponding to LucidProgramming YouTube tutorials
Stars: ✭ 769 (+1273.21%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+1071.43%)
CoolqlcoolNextjs server to query websites with GraphQL
Stars: ✭ 623 (+1012.5%)
User AgentsA JavaScript library for generating random user agents with data that's updated daily.
Stars: ✭ 485 (+766.07%)
RpaUI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
Stars: ✭ 477 (+751.79%)
ScrappleA framework for creating semi-automatic web content extractors
Stars: ✭ 464 (+728.57%)
Awesome Web ScrapingList of libraries, tools and APIs for web scraping and data processing.
Stars: ✭ 4,510 (+7953.57%)
SelectolaxPython binding to Modest engine (fast HTML5 parser with CSS selectors).
Stars: ✭ 368 (+557.14%)
AutoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+7180.36%)
AcheACHE is a web crawler for domain-specific search.
Stars: ✭ 320 (+471.43%)
Gopa[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+394.64%)
Apify JsApify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Stars: ✭ 3,154 (+5532.14%)
Php Curl ClassPHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
Stars: ✭ 2,903 (+5083.93%)
comic-scraper[Python] Scraps comics and manga from various websites and creates cbz files from them
Stars: ✭ 16 (-71.43%)
Text-AnalysisExplaining textual analysis tools in Python. Including Preprocessing, Skip Gram (word2vec), and Topic Modelling.
Stars: ✭ 48 (-14.29%)
raspagem-de-dados-fatec📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí
Stars: ✭ 22 (-60.71%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-73.21%)
PaperScraperA web scraping tool to systematically extract the text of scientific papers and corresponding metadata from university accessible journals.
Stars: ✭ 63 (+12.5%)
linkextractorA Docker tutorial using a link extraction application example
Stars: ✭ 41 (-26.79%)
sp-subway-scraper🚆This web scraper builds a dataset for São Paulo subway operation status
Stars: ✭ 24 (-57.14%)
halfstaff🇺🇸 Is the US flag at half-staff?
Stars: ✭ 22 (-60.71%)
codechef-rank-comparatorWeb application hosted on Heroku cloud platform based on web scraping in python using lxml library (XML Path Language).
Stars: ✭ 23 (-58.93%)
investigation-youtube-ad-placementsData and code from our stories, "Google Has a Secret Blocklist that Hides YouTube Hate Videos from Advertisers—But It’s Full of Holes," and "Google Blocks Advertisers from Targeting Black Lives Matter YouTube Videos."
Stars: ✭ 27 (-51.79%)