kyrosPython wrapper for WhatsApp Web API websocket communication (based on https://github.com/sigalor/whatsapp-web-reveng)
Stars: ✭ 94 (+25.33%)
HumanoidNode.js package to bypass CloudFlare's anti-bot JavaScript challenges
Stars: ✭ 88 (+17.33%)
IdtImage Dataset Tool (idt) is a cli tool designed to make the otherwise repetitive and slow task of creating image datasets into a fast and intuitive process.
Stars: ✭ 202 (+169.33%)
softestRecording Browser Interactions And Generating Test Scripts.
Stars: ✭ 225 (+200%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+1561.33%)
Jsonframe Cheeriosimple multi-level scraper json input/output for Cheerio
Stars: ✭ 196 (+161.33%)
Email ExtractorThe main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (+8%)
MusoqUse SQL on various data sources
Stars: ✭ 252 (+236%)
ViewstateASP.NET View State Decoder
Stars: ✭ 77 (+2.67%)
Anime DlAnime-dl is a command-line program to download anime from CrunchyRoll and Funimation.
Stars: ✭ 190 (+153.33%)
google-scraperThis class can retrieve search results from Google.
Stars: ✭ 33 (-56%)
ArachnidCrawl all unique internal links found on a given website, and extract SEO related information - supports javascript based sites
Stars: ✭ 224 (+198.67%)
Geeksforgeeks.pdfTopic wise PDFs of Geeks for Geeks articles. (Last updated in October 2018)
Stars: ✭ 489 (+552%)
MechamlOCaml functional web scraping library
Stars: ✭ 60 (-20%)
MtntCode for the collection and analysis of the MTNT dataset
Stars: ✭ 48 (-36%)
Loconotion📄 Python tool to turn Notion.so pages into lightweight, customizable static websites
Stars: ✭ 237 (+216%)
ConfigsPublic, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores
Stars: ✭ 37 (-50.67%)
aws-pdf-textract-pipeline🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Stars: ✭ 141 (+88%)
Scrapy ClusterThis Scrapy project uses Redis and Kafka to create a distributed on demand scraping cluster.
Stars: ✭ 921 (+1128%)
SerpscrapSEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Stars: ✭ 153 (+104%)
WebhereHTML scraping for Objective-C.
Stars: ✭ 16 (-78.67%)
ReaperSocial media scraping / data collection tool for the Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs
Stars: ✭ 240 (+220%)
Imagescraper✂️ High performance, multi-threaded image scraper
Stars: ✭ 630 (+740%)
PhpscraperPHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+97.33%)
NewcrawlerFree Web Scraping Tool with Java
Stars: ✭ 589 (+685.33%)
clusteerClusteer is a Puppeteer wrapper written for Laravel, with the super-power of parallelizing pages across multiple browser instances.
Stars: ✭ 81 (+8%)
SqrapeSimple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)
Stars: ✭ 144 (+92%)
Gazpacho🥫 The simple, fast, and modern web scraping library
Stars: ✭ 525 (+600%)
Scrapysharpreborn of https://bitbucket.org/rflechner/scrapysharp
Stars: ✭ 226 (+201.33%)
Facebook data analyzerAnalyze facebook copy of your data with ruby language. Download zip file from facebook and get info about friends ranking by message, vocabulary, contacts, friends added statistics and more
Stars: ✭ 515 (+586.67%)
ScrappleA framework for creating semi-automatic web content extractors
Stars: ✭ 464 (+518.67%)
NickjsWeb scraping library made by the Phantombuster team. Modern, simple & works on all websites. (Deprecated)
Stars: ✭ 494 (+558.67%)
ChampA Telegram bot combined with python to serve some basic functions like weather, music charts, cricket score and much more.
Stars: ✭ 22 (-70.67%)
FerretDeclarative web scraping
Stars: ✭ 4,837 (+6349.33%)
UdemycoursegrabberYour will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!
Stars: ✭ 137 (+82.67%)
Search Engine ParserLightweight package to query popular search engines and scrape for result titles, links and descriptions
Stars: ✭ 216 (+188%)
DataflowkitExtract structured data from web sites. Web sites scraping.
Stars: ✭ 456 (+508%)
Torchbear🔥🐻 The Speakeasy Scripting Engine Which Combines Speed, Safety, and Simplicity
Stars: ✭ 128 (+70.67%)
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+486.67%)
JekyllJekyll-based static site for The Programming Historian
Stars: ✭ 387 (+416%)
LookylooLookyloo is a web interface that allows users to capture a website page and then display a tree of domains that call each other.
Stars: ✭ 381 (+408%)
Undetected ChromedriverCustom Selenium Chromedriver | Zero-Config | Passes ALL bot mitigation systems (like Distil / Imperva/ Datadadome / CloudFlare IUAM)
Stars: ✭ 365 (+386.67%)
privacysecI don't have anything to hide, but I don't have anything to show you either.
Stars: ✭ 110 (+46.67%)
Goose ParserUniversal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (+181.33%)
HtmlsqlhtmlSQL is a experimental PHP library which allows you to access HTML values by an SQL like syntax.
Stars: ✭ 120 (+60%)
Data ScienceCollection of useful data science topics along with code and articles
Stars: ✭ 315 (+320%)
CoronadatascraperCOVID-19 Coronavirus data scraped from government and curated data sources.
Stars: ✭ 372 (+396%)
Od DatabaseDistributed crawler, database and web frontend for public directories indexing
Stars: ✭ 121 (+61.33%)
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+20613.33%)
Comic DlComic-dl is a command line tool to download manga and comics from various comic and manga sites. Supported sites : readcomiconline.to, mangafox.me, comic naver and many more.
Stars: ✭ 365 (+386.67%)
KatanaA Python Tool For google Hacking
Stars: ✭ 355 (+373.33%)
SocialreaperSocial media scraping / data collection library for Facebook, Twitter, Reddit, YouTube, Pinterest, and Tumblr APIs
Stars: ✭ 338 (+350.67%)
SouqscraperSimple scriptes for Level UP your scraping Skills, and source code for Level UP playlist on Youtube
Stars: ✭ 118 (+57.33%)