🇷🇺 IOWeb Framework
A thing to build web crawlers.
Feedback
- t.me/grablab - English telegram chat about web scraping
- t.me/grablab_ru - Russian telegram chat about web scraping
lorien / ioweb
Licence: other
Web Scraping Framework
Stars: ✭ 31
Programming Languages
Labels
Projects that are alternatives of or similar to ioweb
Autoscraper
A Smart, Automatic, Fast and Lightweight Web Scraper for Python
Stars: ✭ 4,077 (+13051.61%)
Mutual labels: scraping, web-scraping, webscraping
ARGUS
ARGUS is an easy-to-use web scraping tool. The program is based on the Scrapy Python framework and is able to crawl a broad range of different websites. On the websites, ARGUS is able to perform tasks like scraping texts or collecting hyperlinks between websites. See: https://link.springer.com/article/10.1007/s11192-020-03726-9
Stars: ✭ 68 (+119.35%)
Mutual labels: scraping, webscraping, webcrawling
Apify Js
Apify SDK — The scalable web scraping and crawling library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
Stars: ✭ 3,154 (+10074.19%)
Mutual labels: scraping, web-scraping, web-crawling
zcrawl
An open source web crawling platform
Stars: ✭ 21 (-32.26%)
Mutual labels: scraping, web-crawling, webcrawling
Scrapple
A framework for creating semi-automatic web content extractors
Stars: ✭ 464 (+1396.77%)
Mutual labels: scraping, web-scraping
Gopa
[WIP] GOPA, a spider written in Golang, for Elasticsearch. DEMO: http://index.elasticsearch.cn
Stars: ✭ 277 (+793.55%)
Mutual labels: scraping, web-scraping
BookingScraper
🌎 🏨 Scrape Booking.com 🏨 🌎
Stars: ✭ 68 (+119.35%)
Mutual labels: web-scraping, webscraping
Django Dynamic Scraper
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+3203.23%)
Mutual labels: scraping, webscraping
raspagem-de-dados-fatec
📓 Minicurso de raspagem de dados web com Python ministrado na Semana de Tecnologia da FATEC Jundiaí
Stars: ✭ 22 (-29.03%)
Mutual labels: scraping, web-scraping
Gazpacho
🥫 The simple, fast, and modern web scraping library
Stars: ✭ 525 (+1593.55%)
Mutual labels: scraping, webscraping
Sqrape
Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)
Stars: ✭ 144 (+364.52%)
Mutual labels: scraping, web-scraping
schedule-tweet
Schedules tweets using TweetDeck
Stars: ✭ 14 (-54.84%)
Mutual labels: scraping, webscraping
Dotnetcrawler
DotnetCrawler is a straightforward, lightweight web crawling/scrapying library for Entity Framework Core output based on dotnet core. This library designed like other strong crawler libraries like WebMagic and Scrapy but for enabling extandable your custom requirements. Medium link : https://medium.com/@mehmetozkaya/creating-custom-web-crawler-with-dotnet-core-using-entity-framework-core-ec8d23f0ca7c
Stars: ✭ 100 (+222.58%)
Mutual labels: scraping, webscraping
Phpscraper
PHP Scraper - an highly opinionated web-interface for PHP
Stars: ✭ 148 (+377.42%)
Mutual labels: scraping, web-scraping
Configs
Public, free to use, repository with diggers configs for scraping / extracting data from various e-commerce websites and online stores
Stars: ✭ 37 (+19.35%)
Mutual labels: scraping, webscraping
PythonScrapyBasicSetup
Basic setup with random user agents and IP addresses for Python Scrapy Framework.
Stars: ✭ 57 (+83.87%)
Mutual labels: scraping, web-scraping
top-github-scraper
Scape top GitHub repositories and users based on keywords
Stars: ✭ 40 (+29.03%)
Mutual labels: scraping, web-scraping
papercut
Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.
Stars: ✭ 15 (-51.61%)
Mutual labels: scraping, web-scraping
Humanoid
Node.js package to bypass CloudFlare's anti-bot JavaScript challenges
Stars: ✭ 88 (+183.87%)
Mutual labels: scraping, web-scraping
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].