Top 111 webscraping open source projects

🗽 NY Times is an Minimal News 🗞 iOS app 📱 built to describe the use of SwiftSoup and CoreData with SwiftUI🔥

Physical Bitcoin Stat Ticker

Web scraper project to collect the useful Hong Kong weather data from HKO website

Python wrapper for the unofficial scraped API of the satori testing system.

단 몇줄의 코드로 다양한 쇼핑 플랫폼에서 결제 내역을 긁어오자!

PHP Metacritic API - Mirrored by my GitLab

a work-in-progress guide to web scraping as an artistic and critical practice

An image scraper that scraps images from unsplash.com

API for fetching data from news websites.

Colossus Repository for Kodi Addons - Kodi is a registered trademark of the XBMC Foundation. We are not connected to or in any other way affiliated with Kodi - DMCA: [email protected]

News extraction and scraping. Article Parsing

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

Android Web Scraper is a simple library for android web automation. You can perform web task in background to fetch website data programmatically.

Blog do Brasil.IO

Projeto de calculo de Imposto de Renda em operacoes na bovespa automaticamente. Tags:canal eletronico do investidor, CEI, selenium, bovespa, IRPF, IR, imposto de renda, finance, yahoo finance, acao, fii, etf, python, crawler, webscraping, calculadora ir

CHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages

This repo contains regression and classification projects. Examples: development of predictive models for comments on social media websites; building classifiers to predict outcomes in sports competitions; churn analysis; prediction of clicks on online ads; analysis of the opioids crisis and an analysis of retail store expansion strategies using…

State of the art object detection in real-time using YOLOV3 algorithm. Augmented with a process that allows easy training of the classifier as a plug & play solution . Provides alert if an item in an alert list is detected.

An Introduction to Web Scraping

[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3

A Dragnet that also extract author, headline, date, keywords from context

R interface to Python requests module

This email crawler will visit all pages of a provided website and parse and save emails found to a csv file.

Pawn package manager for SA-MP

A VS code Extension to accelerate the process of solving problems on Codeforces.

Scrape all eBay sold listings to determine average/median pricing, plot listings over time with trend lines, and extract to excel

This program provides efficient web scraping services for Tor and non-Tor sites. The program has both a CLI and REST API.

extract videos from youtube in audio format using webscraping techniques 🎶

GUI app for downloading whole online courses with folder structure from one url

Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.

robots.txt file parsing and checking for R

Scrape public FaceBook posts from any group or user into a .csv file without needing to register for any API access

Automatically creates a Youtube channel dashboard

Using Selenium, Neha scraped data about 35 top selling sneakers of Nike and Adidas from stockx.com. She used this data to draw insights about sneaker resales.

web scraping in parallel with Selenium Grid and Docker

No description or website provided.

Web Scraping Framework

Helpful Selenium functions to make web-scraping easier and faster

Sometimes we need to scrap web data from our Android App. To achieve this goal jsoup library is a good option. I wrote a blog post on this topic in my personal blog. If you know Bengali language then you can visit this link.

Animeflv is a custom API that has the entire catalog of the animeflv.net website. You can enjoy all the content with subtitles in Spanish and the latest in the world of anime for free.

Scrap Medium Articles using tags.

🌎 🏨 Scrape Booking.com 🏨 🌎

Automated Python Script to retrieve vaccine slots availability and get notified when a slot is available.

[really old and probably doesn't work] Slack bot to post relevant Toronto apartment listings from Kijiji & Craigslist

Application for extracting large amounts of data from the Google search results page

A program which takes an Anime name or URL and downloads the specified range of episodes.

SerpApi client library for Node.js. Previously: Google Search Results Node.js.

A library to get images from social media

📚 A GoodReads.com Scraper script to get books reviews including text and rating.

Rotate between different NordVPN servers with ease. Works both on Linux and Windows without any required changes to your code!

🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript