Instagram-to-discordMonitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!
Stars: ✭ 113 (+841.67%)
ExDeMonA general purpose metrics monitor implemented with Apache Spark. Kafka source, Elastic sink, aggregate metrics, different analysis, notifications, actions, live configuration update, missing metrics, ...
Stars: ✭ 19 (+58.33%)
proxycrawl-pythonProxyCrawl Python library for scraping and crawling
Stars: ✭ 51 (+325%)
document-dlCommand line program to download documents from web portals.
Stars: ✭ 14 (+16.67%)
scrapmanRetrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs
Stars: ✭ 21 (+75%)
FerretDeclarative web scraping
Stars: ✭ 4,837 (+40208.33%)
websitechangesAlerts you via email about a website change.
Stars: ✭ 43 (+258.33%)
LeetCodeAt present contains scraped data from around 1500 problems present on the site. More to follow....
Stars: ✭ 45 (+275%)
OLX Scraper📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.
Stars: ✭ 15 (+25%)
Captcha-ToolsAll-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!
Stars: ✭ 23 (+91.67%)
TorScrapperA Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)
Stars: ✭ 24 (+100%)
ha-multiscrapeHome Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.
Stars: ✭ 103 (+758.33%)
scrapersscrapers for building your own image databases
Stars: ✭ 46 (+283.33%)
wget-luaWget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.
Stars: ✭ 52 (+333.33%)
copycatA PHP Scraping Class
Stars: ✭ 70 (+483.33%)
metafetchNodeJS package that fetches a given URL's title, description, images, links etc.
Stars: ✭ 21 (+75%)
imdb-scraper🎬 An attempt at the most complete IMDb API
Stars: ✭ 24 (+100%)
readability-cliA CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!
Stars: ✭ 41 (+241.67%)
ZeiverA Scraper, Downloader, & Recorder for static open directories.
Stars: ✭ 14 (+16.67%)
facebook-discussion-tkA collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.
Stars: ✭ 33 (+175%)
DataflowkitExtract structured data from web sites. Web sites scraping.
Stars: ✭ 456 (+3700%)
InformerA Telegram Mass Surveillance Bot in Python
Stars: ✭ 745 (+6108.33%)
Lulu[Unmaintained] A simple and clean video/music/image downloader 👾
Stars: ✭ 789 (+6475%)
Email ExtractorThe main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (+575%)
GeziyorGeziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.
Stars: ✭ 1,246 (+10283.33%)
diffbot-php-client[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library
Stars: ✭ 53 (+341.67%)
Pahe.ph-ScraperPahe.ph [Pahe.in] Movies Website Scraper
Stars: ✭ 57 (+375%)
Linkedin Profile Scraper🕵️♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.
Stars: ✭ 171 (+1325%)
Monitorfe🍉前端埋点监控,提供前端 JS 执行错误,第三方资源加载异常,Ajax 请求错误监控
Stars: ✭ 190 (+1483.33%)
scrapy facebookerCollection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (+83.33%)
papercutPapercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (+25%)
Scraper-Projects🕸 List of mini projects that involve web scraping 🕸
Stars: ✭ 25 (+108.33%)
bots-zooNo description or website provided.
Stars: ✭ 59 (+391.67%)
scraperNodejs web scraper. Contains a command line, docker container, terraform module and ansible roles for distributed cloud scraping. Supported databases: SQLite, MySQL, PostgreSQL. Supported headless clients: Puppeteer, Playwright, Cheerio, JSdom.
Stars: ✭ 37 (+208.33%)
LinkedinLinkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (+2475%)
Myperf4jHigh performance Java APM. Powered by ASM. Try it. Test it. If you feel its better, use it.
Stars: ✭ 2,281 (+18908.33%)
Anime DlAnime-dl is a command-line program to download anime from CrunchyRoll and Funimation.
Stars: ✭ 190 (+1483.33%)
CollyElegant Scraper and Crawler Framework for Golang
Stars: ✭ 15,535 (+129358.33%)
Imagescraper✂️ High performance, multi-threaded image scraper
Stars: ✭ 630 (+5150%)
CrawlyCrawly, a high-level web crawling & scraping framework for Elixir.
Stars: ✭ 440 (+3566.67%)
google-scraperThis class can retrieve search results from Google.
Stars: ✭ 33 (+175%)
PypatentSearch for and retrieve US Patent and Trademark Office Patent Data
Stars: ✭ 31 (+158.33%)
TradeTheEventImplementation of "Trade the Event: Corporate Events Detection for News-Based Event-Driven Trading." In Findings of ACL2021
Stars: ✭ 64 (+433.33%)
KatanaA Python Tool For google Hacking
Stars: ✭ 355 (+2858.33%)
SerpscrapSEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.
Stars: ✭ 153 (+1175%)
PhpscraperPHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+1133.33%)
Jsonframe Cheeriosimple multi-level scraper json input/output for Cheerio
Stars: ✭ 196 (+1533.33%)
UdemycoursegrabberYour will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!
Stars: ✭ 137 (+1041.67%)
Scrapysharpreborn of https://bitbucket.org/rflechner/scrapysharp
Stars: ✭ 226 (+1783.33%)
IglanceFree system monitor for OSX and macOS. See all system information at a glance in the menu bar.
Stars: ✭ 1,358 (+11216.67%)
AutoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+33875%)
SeleniumcrawlerAn example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (+875%)
Goose ParserUniversal scrapping tool, which allows you to extract data using multiple environments
Stars: ✭ 211 (+1658.33%)