DotnetcrawlerDotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c
Stars: ✭ 100 (+376.19%)
Sneakerbot AppApp that scrapes the Footlocker website to construct URLs for upcoming sneaker releases and adds the shoe to your cart if it is available. Uses Python and Selenium Webdriver. *Chrome and Chromedriver must be installed and Chromedriver must be on main path
Stars: ✭ 54 (+157.14%)
Tradingview Data ScraperExtract price and indicator data from TradingView charts to create ML datasets
Stars: ✭ 203 (+866.67%)
DatadoubleconfirmSimple datasets and notebooks for data visualization, statistical analysis and modelling - with write-ups here: http://projectosyo.wix.com/datadoubleconfirm.
Stars: ✭ 24 (+14.29%)
PoliteBe nice on the web
Stars: ✭ 253 (+1104.76%)
Covid 19 jhu data web scrap and cleaningThis repository contains data and code used to get and clean data from https://github.com/CSSEGISandData/COVID-19 and https://www.worldometers.info/coronavirus/
Stars: ✭ 80 (+280.95%)
toronto-apartment-finder[really old and probably doesn't work] Slack bot to post relevant Toronto apartment listings from Kijiji & Craigslist
Stars: ✭ 23 (+9.52%)
RedditsfinderArchive a reddit user's post history. Formatted overview of a profile, JSON containing every post, and picture downloads. Uses the pushshift API.
Stars: ✭ 28 (+33.33%)
DecryptrAn extensible API for breaking captchas
Stars: ✭ 154 (+633.33%)
Anirip🎬 A Crunchyroll show/season ripper
Stars: ✭ 127 (+504.76%)
MorphTake the hassle out of web scraping
Stars: ✭ 421 (+1904.76%)
NordVPN-switcherRotate between different NordVPN servers with ease. Works both on Linux and Windows without any required changes to your code!
Stars: ✭ 143 (+580.95%)
NytcrosswordAn exploration of New York Times crossword answers from 1994-2017, i.e. the Will Shortz era.
Stars: ✭ 117 (+457.14%)
Udemy botAn automation bot for free Udemy courses
Stars: ✭ 91 (+333.33%)
CrosslinkedLinkedIn enumeration tool to extract valid employee names from an organization through search engine scraping
Stars: ✭ 223 (+961.9%)
fBrowserHelpful Selenium functions to make web-scraping easier and faster
Stars: ✭ 16 (-23.81%)
FalkorOpen Source web scraping API. Falkor turns web pages into queryable JSON
Stars: ✭ 184 (+776.19%)
Sig To GooglecalendarA python script to get class schedules on UFLA's SIG and convert to a .CSV file to use in Google Calendar
Stars: ✭ 14 (-33.33%)
super-anime-downloaderA program which takes an Anime name or URL and downloads the specified range of episodes.
Stars: ✭ 26 (+23.81%)
Gazpacho🥫 The simple, fast, and modern web scraping library
Stars: ✭ 525 (+2400%)
StardoxGithub stargazers information gathering tool
Stars: ✭ 130 (+519.05%)
Proxy requestsa class that uses scraped proxies to make http GET/POST requests (Python requests)
Stars: ✭ 357 (+1600%)
GoodReadsScraper📚 A GoodReads.com Scraper script to get books reviews including text and rating.
Stars: ✭ 36 (+71.43%)
SoupWeb Scraper in Go, similar to BeautifulSoup
Stars: ✭ 1,685 (+7923.81%)
Php CrawlerA php crawler that finds emails on the internets
Stars: ✭ 119 (+466.67%)
aws-pdf-textract-pipeline🔍 Data pipeline for crawling PDFs from the Web and transforming their contents into structured data using AWS textract. Built with AWS CDK + TypeScript
Stars: ✭ 141 (+571.43%)
WswpCode for the second edition Web Scraping with Python book by Packt Publications
Stars: ✭ 112 (+433.33%)
iowebWeb Scraping Framework
Stars: ✭ 31 (+47.62%)
ImghashPerceptual image hashing for Node.js
Stars: ✭ 98 (+366.67%)
Nytimes App🗽 A Simple Demonstration of the New York Times App 📱 using Jsoup web crawler with MVVM Architecture 🔥
Stars: ✭ 246 (+1071.43%)
Clock可视化任务调度系统,精简到一个二进制文件 (Web visual task scheduler system , yes ! just one binary solve all the problems !)
Stars: ✭ 86 (+309.52%)
CoWin-Vaccine-NotifierAutomated Python Script to retrieve vaccine slots availability and get notified when a slot is available.
Stars: ✭ 102 (+385.71%)
InstagoDownload/access photos, videos, stories, story highlights, postlives, following and followers of Instagram
Stars: ✭ 59 (+180.95%)
R Web Scraping Cheat SheetGuide, reference and cheatsheet on web scraping using rvest, httr and Rselenium.
Stars: ✭ 207 (+885.71%)
Fifa Fut DataWeb-scraping script that writes the data of all players from FutHead and FutBin to a CSV file or a DB
Stars: ✭ 55 (+161.9%)
BrokenlinkhijackerA Fast Broken Link Hijacker Tool written in Python
Stars: ✭ 45 (+114.29%)
InstascrapePowerful and flexible Instagram scraping library for Python, providing easy-to-use and expressive tools for accessing data programmatically
Stars: ✭ 202 (+861.9%)
ConfigsPublic, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores
Stars: ✭ 37 (+76.19%)
google scraper live viewApplication for extracting large amounts of data from the Google search results page
Stars: ✭ 17 (-19.05%)
HuginnCreate agents that monitor and act on your behalf. Your agents are standing by!
Stars: ✭ 33,694 (+160347.62%)
Hq bot📲 Bot to help solve HQ trivia
Stars: ✭ 167 (+695.24%)
android-web-scraping-app-jsoupSometimes we need to scrap web data from our Android App. To achieve this goal jsoup library is a good option. I wrote a blog post on this topic in my personal blog. If you know Bengali language then you can visit this link.
Stars: ✭ 26 (+23.81%)
Youtube ProjectsThis repository contains all the code I use in my YouTube tutorials.
Stars: ✭ 144 (+585.71%)
SuckitSuck the InTernet
Stars: ✭ 429 (+1942.86%)
TiktokbotA TikTokBot that downloads trending tiktok videos and compiles them using FFmpeg
Stars: ✭ 126 (+500%)
Sneakers ProjectUsing Selenium, Neha scraped data about 35 top selling sneakers of Nike and Adidas from stockx.com. She used this data to draw insights about sneaker resales.
Stars: ✭ 32 (+52.38%)
animeflvAnimeflv is a custom API that has the entire catalog of the animeflv.net website. You can enjoy all the content with subtitles in Spanish and the latest in the world of anime for free.
Stars: ✭ 37 (+76.19%)
mirror-mirrorA library to get images from social media
Stars: ✭ 15 (-28.57%)
Ralgerralger makes it easy to scrape a website. Built on the shoulders of titans: rvest, xml2.
Stars: ✭ 130 (+519.05%)