ScrappleA framework for creating semi-automatic web content extractors
Stars: ✭ 464 (+439.53%)
Web ScrapingDetailed web scraping tutorials for dummies with financial data crawlers on Reddit WallStreetBets, CME (both options and futures), US Treasury, CFTC, LME, SHFE and news data crawlers on BBC, Wall Street Journal, Al Jazeera, Reuters, Financial Times, Bloomberg, CNN, Fortune, The Economist
Stars: ✭ 153 (+77.91%)
CascadiaGo cascadia package command line CSS selector
Stars: ✭ 67 (-22.09%)
Detect CmsPHP Library for detecting CMS
Stars: ✭ 78 (-9.3%)
PhpscraperPHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+72.09%)
Html MetadataMetaData html scraper and parser for Node.js (supports Promises and callback style)
Stars: ✭ 129 (+50%)
SoupWeb Scraper in Go, similar to BeautifulSoup
Stars: ✭ 1,685 (+1859.3%)
Bet On SibylMachine Learning Model for Sport Predictions (Football, Basketball, Baseball, Hockey, Soccer & Tennis)
Stars: ✭ 190 (+120.93%)
top-github-scraperScape top GitHub repositories and users based on keywords
Stars: ✭ 40 (-53.49%)
OLX Scraper📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.
Stars: ✭ 15 (-82.56%)
Project TauroA Router WiFi key recovery/cracking tool with a twist.
Stars: ✭ 52 (-39.53%)
Scrape Linkedin Selenium`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.
Stars: ✭ 239 (+177.91%)
Linkedin-ClientWeb scraper for grabing data from Linkedin profiles or company pages (personal project)
Stars: ✭ 42 (-51.16%)
Php Curl ClassPHP Curl Class makes it easy to send HTTP requests and integrate with web APIs
Stars: ✭ 2,903 (+3275.58%)
SpidrA versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.
Stars: ✭ 656 (+662.79%)
MediumScraperScraping articles of medium and providing audio versions 📑 to 🔊 using django
Stars: ✭ 12 (-86.05%)
grailerweb scraping tool for grailed.com
Stars: ✭ 30 (-65.12%)
Data-Wrangling-with-PythonSimplify your ETL processes with these hands-on data sanitation tips, tricks, and best practices
Stars: ✭ 90 (+4.65%)
ArachnidPowerful web scraping framework for Crystal
Stars: ✭ 68 (-20.93%)
Scrapy CraigslistWeb Scraping Craigslist's Engineering Jobs in NY with Scrapy
Stars: ✭ 54 (-37.21%)
ConfuciusA lightweight Java configuration library
Stars: ✭ 51 (-40.7%)
Node ConfigNode.js Application Configuration
Stars: ✭ 5,423 (+6205.81%)
Pitchfork🎶 Unofficial python API for pitchfork.com reviews.
Stars: ✭ 67 (-22.09%)
PropertiesThis library provides convinient way to work with properties. It can handle property-files on hard drive, in classpath or get values from system properties
Stars: ✭ 49 (-43.02%)
Awesome CrawlerA collection of awesome web crawler,spider in different languages
Stars: ✭ 4,793 (+5473.26%)
User AgentsA JavaScript library for generating random user agents with data that's updated daily.
Stars: ✭ 485 (+463.95%)
100projectsofcodeA list of practical knowledge-building projects.
Stars: ✭ 1,183 (+1275.58%)
Actor Google Search ScraperApify actor that crawls Google Search result pages (SERPs) and extracts a list of organic results, ads, related queries and more. It supports selection of custom country, language and location.
Stars: ✭ 38 (-55.81%)
RpaUI.Vision: Open-Source RPA Software (formerly Kantu) - Modern Robotic Process Automation with Selenium IDE++
Stars: ✭ 477 (+454.65%)
Awesome Web ScrapingList of libraries, tools and APIs for web scraping and data processing.
Stars: ✭ 4,510 (+5144.19%)
Monkey DlBulk download your favourite anime episodes from your favourite anime websites
Stars: ✭ 382 (+344.19%)
MechanicalsoupA Python library for automating interaction with websites.
Stars: ✭ 3,863 (+4391.86%)
SupremedropbotA supreme web bot, written in python, to grab a list of specified products, and checkout before they sell out!
Stars: ✭ 66 (-23.26%)
SelectolaxPython binding to Modest engine (fast HTML5 parser with CSS selectors).
Stars: ✭ 368 (+327.91%)
ScrapersA list of scrapers from around the web.
Stars: ✭ 366 (+325.58%)
SnoopSnoop — инструмент разведки на основе открытых данных (OSINT world)
Stars: ✭ 886 (+930.23%)
AutoscraperA Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+4640.7%)
Facebook Data Analyzer📊Python script to analyze the contents of your Facebook data export
Stars: ✭ 71 (-17.44%)
DecapitatedHeadless 'Chrome' Orchestration in R
Stars: ✭ 65 (-24.42%)
Sig To GooglecalendarA python script to get class schedules on UFLA's SIG and convert to a .CSV file to use in Google Calendar
Stars: ✭ 14 (-83.72%)
AcheACHE is a web crawler for domain-specific search.
Stars: ✭ 320 (+272.09%)
WebmiddleNode.js framework for modular web scraping and data extraction
Stars: ✭ 13 (-84.88%)
Docsify ThemeableA delightfully simple theme system for docsify.js. Features multiple themes with rich customization options, an improved desktop and mobile experience, and legacy browser support (IE10+).
Stars: ✭ 299 (+247.67%)
Turkce Python KaynaklariTürkçe olarak hazırlanmış Python programlama dili ile ilgili içeriklerin derlendiği sayfa.
Stars: ✭ 295 (+243.02%)
Letterboxd recommendationsScraping publicly-accessible Letterboxd data and creating a movie recommendation model with it that can generate recommendations when provided with a Letterboxd username
Stars: ✭ 23 (-73.26%)
I18n EditorGUI for editing your i18n translation files
Stars: ✭ 290 (+237.21%)
Gopa[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+222.09%)