228 Open source projects that are alternatives of or similar to Scrapping

Instagram Photo Downloader

Tool to scrape linkedin

Provides a virtual web browser (a.k.a. "headless browser") appearing as a node.

👨‍👩‍👦 Social account detection and extraction in Python, e.g. for crawling/scraping.

Pahe.ph [Pahe.in] Movies Website Scraper

🦀 Scraping & crawling websites using Rust, and ReasonML

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

Kotlin Wrapper for Jsoup

⚽ Instantly find 🏆EURO 2016 live-streams & highlights, now a Web App!

A website dedicated to making recipes from any website easy to read.

Generate JSON representations of HTML tables

Generate a network graph of connections from your WhatsApp groups data

A library to read a YML file with Xpath or CSS Selectors and extract data from HTML pages using them

scrapers for building your own image databases

Browser automation API for repetitive web-based tasks, with a friendly user interface. You can use it to scrape content or do many other things like capture a screenshot, generate pdf, extract content or execute custom Puppeteer, Playwright functions.

A test suite of common scraper detection techniques. See how detectable your scraper stack is.

The simplest demo of chrome automation by python and selenium in AWS Lambda

🤖 A Versatile Telegram Bot

ProxyCrawl Python library for scraping and crawling

A CLI for Mozilla Readability. Get clean, uncluttered, ready-to-read HTML from any webpage!

Python (asyncio) wrapper for hackernews api

Basic setup with random user agents and IP addresses for Python Scrapy Framework.

extract data from html table

Tiny little ruby on rails website that crawls though your public github repos to find out what your favourite languages are.

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

A Selenium-driven tool for automated website interaction and scraping.

This class can retrieve search results from Google.

🔎Search LinkedIn profile by email address📧

MITM HTTP(S) proxy with integrated load-balancing, rate-limiting and error handling. Built for automated web scraping.

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

Python & command-line tool to gather text on the Web: web crawling/scraping, extraction of text, metadata, comments

Easily crawl news portals or blog sites using Storm Crawler.

📊 Python tool to scrape real-time information about ETFs from the web and mixing them together by proportionally distributing their assets allocation

Scrapping Facebook with JavaScript.

NBA Daily Fantasy Lineup Optimizer for FanDuel Using Python

Download images from reddit

**[ARCHIVED]** website changes tracker 🔍

An open source web crawling platform

Tutorial for web scraping / crawling with Node.js.

Projeto de monitoramento de publicações oficiais relacionadas a COVID-19 no Brasil.

Pillaging the seven seas for torrents, pieces of eight and other bounty.

A PHP Scraping Class

Chopper is a tool to extract elements from HTML by preserving ancestors and CSS rules

Overwatch API library for player details and career stats

Serverless selenium which dynamically execute any given code.

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

Simple app for scraping author profiles and tutorials from Scotch.io - https://scotch.io.

Web Scraping Framework

Machine learning for beginner(Data Science enthusiast)

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

.NET agile and extensible web searching API

A Scrapy extension to log items coverage when the spider shuts down

TV Series is a tool that scrapes Episode Synopsis' of popular TV Series' from websites like Wikipedia / IMDb and show in one place with a user-friendly navigation UI.

🕵‍♂ Bot detection tests for Puppeteer. Hide and seek!

With Selenium headless browsing and CAPTCHA solving

[partially working] Scrape and add anime episode stream URLs to uGet (Linux) or IDM (Windows) ~ Python3

Simple and friendly Bot for Instagram, using Selenium and Scrapy with Python.

Command-line utility to convert HTML tables into CSV files

youtube & tiktok analysis + youchoose recommendation custmizer. backend, extensions, and tooling

The 4CAT Capture and Analysis Toolkit provides modular data capture & analysis for a variety of social media platforms.