370 Open source projects that are alternatives of or similar to Web Scraping

Faster requests on Python 3

PHP Library for detecting CMS

Scape top GitHub repositories and users based on keywords

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

PHP Scraper - an highly opinionated web-interface for PHP

Web scraper for grabing data from Linkedin profiles or company pages (personal project)

A library that enables programmatic interaction with daft.ie. Daft.ie has nationwide coverage and contains about 80% of the total available properties in Ireland.

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

Go cascadia package command line CSS selector

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

A framework for creating semi-automatic web content extractors

A versatile Ruby web spidering library that can spider a site, multiple domains, certain links or infinitely. Spidr is designed to be fast and easy to use.

NBA Stats API via Basketball Reference

Powerful web scraping framework for Crystal

Web Scraping Craigslist's Engineering Jobs in NY with Scrapy

MetaData html scraper and parser for Node.js (supports Promises and callback style)

AKShare is an elegant and simple financial data interface library for Python, built for human beings! 开源财经数据接口库

PHP Curl Class makes it easy to send HTTP requests and integrate with web APIs

A Router WiFi key recovery/cracking tool with a twist.

Fetch user's data across social media

A Devtools driver for web automation and scraping

[DEPRECATED] A micro Web framework for Rust, based on Hyper and Tokio

Turn large Web sites into tables and charts using simple SQLs.

Open Data, more than 50 financial data. 提供超過 50 個金融資料(台股為主)，每天更新 https://finmind.github.io/

天勤量化开发包, 期货量化, 实时行情/历史数据/实盘交易

Stock Analysis Tutorial in Python

💦 Tools to Work with the 'Splash' JavaScript Rendering Service in R

Set up free and scalable Scrapyd cluster for distributed web-crawling with just a few clicks. DEMO 👉

work with Tehran stock exchange data in Python

Download SEC filings from the EDGAR database using Python

Automate the creation of Python Selenium Scripts by drawing coloured boxes on webpage elements

A Java API for Developing Automated Trading Applications for the Equity, Futures, and Currency Markets

Manage newsletters in Laravel

Scrapy crawler to collect data on the back catalog of songs listed for sale.

Web Scraper in Go, similar to BeautifulSoup

Python Package for scraping NHL Play-by-Play and Shift data

A free weekly newsletter about the Haskell programming language.

Simple Query Scraping with CSS and Go Reflection (MOVED to Gitlab)

Deutsche Boerse's Financial Trading Public Data Set

Node.js package to bypass CloudFlare's anti-bot JavaScript challenges

新浪微博相册大图多线程爬虫。

Apify actor that opens a web page in headless Chrome and analyzes the HTML and JavaScript objects, looks for schema.org microdata and JSON-LD metadata, analyzes AJAX requests, etc.

Simple web scraping for R

Tutorials for SimFin - Simple financial data for Python

Promise / Future library for Go

Asynchronous TLS/SSL streams for Tokio using Rustls.

一个开源的 android 平台期货行情交易终端

iexcloud api wrapper written in typescript (asynchronous interface)

A C++ stock market algorithmic trading bot

Learn Python for the next 30 (or so) Days.

A simple example server in asynchronous Rust for MMO games

Extract clean(er), readable text from web pages via Mercury Web Parser.

Zillow Scraper for Python using Selenium

(Repository is not maintained anymore) A Java wrapper to get stock data and stock indicators from the Alpha Vantage API

Extract financial data from the SEC's EDGAR database

Exchange Rest And WebSocket API For Golang Wrapper support okcoin,okex,huobi,hbdm,bitmex,coinex,poloniex,bitfinex,bitstamp,binance,kraken,bithumb,zb,hitbtc,fcoin, coinbene

⚡️ Ayakashi.io - The next generation web scraping framework

An Objective-C future implementation that aims to provide maximal type safety

A list of practical knowledge-building projects.

前端开发博客，分享互联网最精彩的前端技术，欢迎关注我微信公众号：前端开发博客，回复 1024，领取前端进阶资料包，回复 加群，与大神一起交流学习。