sangaline / Advanced Web Scraping Tutorial
The Zipru scraper developed in the Advanced Web Scraping Tutorial.
Stars: ✭ 384
Programming Languages
python
139335 projects - #7 most used programming language
Projects that are alternatives of or similar to Advanced Web Scraping Tutorial
Django Dynamic Scraper
Creating Scrapy scrapers via the Django admin interface
Stars: ✭ 1,024 (+166.67%)
Mutual labels: scraper, scrapy
Seleniumcrawler
An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site
Stars: ✭ 117 (-69.53%)
Mutual labels: scraper, scrapy
Warta Scrap
Indonesia Index News Crawler, including 10 online media
Stars: ✭ 57 (-85.16%)
Mutual labels: scraper, scrapy
OLX Scraper
📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.
Stars: ✭ 15 (-96.09%)
Mutual labels: scraper, scrapy
Voyages Sncf Api
A scrapy spider that scraps times and prices from Voyages Sncf. It uses scrapyrt to provide an API interface.
Stars: ✭ 7 (-98.18%)
Mutual labels: scraper, scrapy
Scrapoxy
Scrapoxy hides your scraper behind a cloud. It starts a pool of proxies to send your requests. Now, you can crawl without thinking about blacklisting!
Stars: ✭ 1,322 (+244.27%)
Mutual labels: scraper, scrapy
Mailinglistscraper
A python web scraper for public email lists.
Stars: ✭ 19 (-95.05%)
Mutual labels: scraper, scrapy
scrapy-LBC
Araignée LeBonCoin avec Scrapy et ElasticSearch
Stars: ✭ 14 (-96.35%)
Mutual labels: scraper, scrapy
Ruiji.net
crawler framework, distributed crawler extractor
Stars: ✭ 220 (-42.71%)
Mutual labels: scraper, scrapy
scrapy facebooker
Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.
Stars: ✭ 22 (-94.27%)
Mutual labels: scraper, scrapy
Email Extractor
The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url
Stars: ✭ 81 (-78.91%)
Mutual labels: scraper, scrapy
Goribot
[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler framework.一个轻量的分布式友好的 Golang 爬虫框架。
Stars: ✭ 190 (-50.52%)
Mutual labels: scraper, scrapy
OpenScraper
An open source webapp for scraping: towards a public service for webscraping
Stars: ✭ 80 (-79.17%)
Mutual labels: scraper, scrapy
Linkedin
Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy
Stars: ✭ 309 (-19.53%)
Mutual labels: scraper, scrapy
Xidel
Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
Stars: ✭ 335 (-12.76%)
Mutual labels: scraper
Post Tuto Deployment
Build and deploy a machine learning app from scratch 🚀
Stars: ✭ 368 (-4.17%)
Mutual labels: scrapy
Artistic Style Transfer
Convolutional neural networks for artistic style transfer.
Stars: ✭ 341 (-11.2%)
Mutual labels: tutorial-code
Javgo
JavGo是一个集合影片管理，影片刮削，视频处理，资源搜索等综合一体的全功能影音软件，支持爬取javbus，jav321，javdb，javlibrary进行刮削，支持db，bus的磁力搜索，支持获取library的影片评论。
Stars: ✭ 338 (-11.98%)
Mutual labels: scraper
Advanced Web Scraping Tutorial Project
This repository is a companion to the article Advanced Web Scraping: Bypassing captcha, "403 Forbidden," and more. Please refer to the article for further details.
This is a scrapy web scraper for the fictional Zipru torrent site. It is designed to bypass four distinct anti-scraping mechanisms:
- User agent filtering.
- Obfuscated javascript redirects.
- Captchas.
- Header consistency checks.
The scraper is not actually functional because Zipru is not a real site. The code, however, is otherwise complete and can easily be adapted to work on other sites.
Note that the project description data, including the texts, logos, images, and/or trademarks,
for each open source project belongs to its rightful owner.
If you wish to add or remove any projects, please contact us at [email protected].