572 Open source projects that are alternatives of or similar to scraper

A Smart, Automatic, Fast and Lightweight Web Scraper for Python

PHP Scraper - an highly opinionated web-interface for PHP

An example using Selenium webdrivers for python and Scrapy framework to create a web scraper to crawl an ASP site

ProxyCrawl Python library for scraping and crawling

🕸 List of mini projects that involve web scraping 🕸

A collection of tools to (semi-)automatically collect and analyze data from online discussions on Facebook groups and pages.

Search for and retrieve US Patent and Trademark Office Patent Data

Creating Scrapy scrapers via the Django admin interface

Papercut is a scraping/crawling library for Node.js built on top of JSDOM. It provides basic selector features together with features like Page Caching and Geosearch.

scrapers for building your own image databases

A Scraper made 100% in Python using BeautifulSoup and Tor. It can be used to scrape both normal and onion links. Happy Scraping :)

Scraping the status of WhatsApp contacts

[Deprecated - Maintenance mode - use APIs directly please!] The official Diffbot client library

No description or website provided.

Monitor instagram user account and automatically post new images to discord channel via a webhook. Working 2022!

Distributed crawler powered by Headless Chrome

✂️ High performance, multi-threaded image scraper

Collection of scrapy spiders which can scrape posts, images, and so on from public Facebook Pages.

A Python Tool For google Hacking

Universal scrapping tool, which allows you to extract data using multiple environments

reborn of https://bitbucket.org/rflechner/scrapysharp

**[ARCHIVED]** website changes tracker 🔍

This class can retrieve search results from Google.

A PHP Scraping Class

No description or website provided.

Pahe.ph [Pahe.in] Movies Website Scraper

Command line program to download documents from web portals.

爬虫工程师常用的 Chrome 插件 | Chrome extensions used by crawler developer

🕵️‍♂️ LinkedIn profile scraper returning structured profile data in JSON. Works in 2020.

Anime-dl is a command-line program to download anime from CrunchyRoll and Funimation.

Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.

A Scraper, Downloader, & Recorder for static open directories.

Linkedin Scraper using Selenium Web Driver, Chromium headless, Docker and Scrapy

`scrape_linkedin` is a python package that allows you to scrape personal LinkedIn profiles & company pages - turning the data into structured json.

Declarative web scraping

Extract structured data from web sites. Web sites scraping.

[Unmaintained] A simple and clean video/music/image downloader 👾

Crawly, a high-level web crawling & scraping framework for Elixir.

Geziyor, a fast web crawling & scraping framework for Go. Supports JS rendering.

The main functionality is to extract all the emails from one or several URLs - La funcionalidad principal es extraer todos los correos electrónicos de una o varias Url

Your will to enroll in Udemy course is here, but the money isn't? Search no more! This python program searches for your desired course in more than [insert big number here] websites, compares the last updated date, and gives you the download link of the latest one back, but you also have the choice to see the other ones as well!

simple multi-level scraper json input/output for Cheerio

Elegant Scraper and Crawler Framework for Golang

SEO python scraper to extract data from major searchengine result pages. Extract data like url, title, snippet, richsnippet and the type from searchresults for given keywords. Detect Ads or make automated screenshots. You can also fetch text content of urls provided in searchresults or by your own. It's usefull for SEO and business related research tasks.

Retrieve real (with Javascript executed) HTML code from an URL, ultra fast and supports multiple parallel loading of webs

Wget-AT is a modern Wget with Lua hooks, Zstandard (+dictionary) WARC compression and URL-agnostic deduplication.

All-in-one Python (And now Go!) module to help solve captchas with Capmonster, 2captcha and Anticaptcha API's!

LinkedinScraper is an another information gathering tool written in python. You can scrape employees of companies on Linkedin.com and then create these employee names, titles and emails.

An api to check social media username availability on a variety of services

🚆This web scraper builds a dataset for São Paulo subway operation status

Checks USD/PYG exchange rate from several sites, with a calculator, RESTful API and a twitter bot

📝 Random quotes generator package. Available on npm and PyPi

Scraping the Un–scrapable™

Parse a human name string into salutation, first name, middle name, last name, suffix.

Scraps for publicly accessible MongoDB instances and dumps user passwords

Zyte Smart Proxy Manager (formerly Crawlera) middleware for Scrapy

Archive web pages with all relevant assets or save as a single file HTML

fetch data from Facebook, Instagram and LinkedIn

Perform lexical analysis on words, one word at a time.

네이버 영화 정보 및 사용자 작성 영화평/평점 데이터 수집기