Top 21 scrape open source projects

A Python module to bypass Cloudflare's anti-bot page.

scrapes medias, likes, followers, tags and all metadata. Inspired by instagram-php-scraper,bot

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

Retrieve years of imgur.com's data without any authentication.

No description or website provided.

Python Script to Scrape Pastebin with Regex

Scrape domain names from SSL certificates of arbitrary hosts

This is a web crawler for pubg.op.gg, written by Ruichong Liu. 绝地求生游戏数据抓取

Spider项目将会不断更新本人学习使用过的爬虫方法！！！

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

Scrape boards and threads from 4chan (8kun WIP). Downloads images, videos and HTML if desired.

A standalone package to scrape financial data from listed Vietnamese companies via Vietstock

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

Simply scrape / download all the media from an fansly account

A library use jQuery like API for html parsing & node selecting & node mutation, suitable for web scraping and html confusion.

scrapers for building your own image databases

Advanced python library to scrap Twitter (tweets, users) from unofficial API

Scrapers for vk, facebook, instagram and more

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

A webpage proxy that request through Chromium (puppeteer) - can be used to bypass Cloudflare anti bot / anti ddos on any application (like curl)