572 Open source projects that are alternatives of or similar to scraper

A Python Oriented tool to Scrap WhatsApp Group Link using Google Dork it Scraps Whatsapp Group Links From Google Results And Gives Working Links.

Library to download subtitles from subscene.com

🚆This web scraper builds a dataset for São Paulo subway operation status

Instagram comment scraper using python and selenium. Save the comments into excel.

Checks USD/PYG exchange rate from several sites, with a calculator, RESTful API and a twitter bot

API for fetching data from news websites.

Parse a human name string into salutation, first name, middle name, last name, suffix.

Scraps for publicly accessible MongoDB instances and dumps user passwords

Perform lexical analysis on words, one word at a time.

네이버 영화 정보 및 사용자 작성 영화평/평점 데이터 수집기

bingip2hosts is a Bing.com web scraper that discovers websites by IP address

Transform unstructured document collections to structured Linked Data

Whisper of the arxiv: read comments in tex of papers

Archive web pages with all relevant assets or save as a single file HTML

News extraction and scraping. Article Parsing

fetch data from Facebook, Instagram and LinkedIn

Proxy pool. Finds and checks proxies with rest api for querying results. Can find over 25k proxies in under 5 minutes.

Scoopi Web Scraper is a heavy duty tool to extract data from HTML pages.

A web crawler that uses Firefox and js injection to interact with webpages and crawl their content, written in nodejs.

🌍 Dataset that shows the Internet affordability by country (a shocking reality!)

Completely free and open-source human-like Instagram bot. Powered by UIAutomator2 and compatible with basically any Android device 5.0+ that can run Instagram - real or emulated.

Instagram application for Django.

A web spider for shodan.io without using the Developer API.

A simple web scraping engine supporting concurrent and anonymous scraping

Screen scraping and web crawling framework

A crawler based on Phantom. Allows discovery of dynamic content and supports custom scrapers.

A series of distributed components for Scrapy. Including RabbitMQ-based components, Kafka-based components, and RedisBloom-based components for Scrapy.

A scraper to grab and publish TTC subway arrival times.

High performance crawler for Nodejs

Python script to download messages from a Facebook page to a CSV file

Twitter Troll & Fake News Hunter - Crawls news websites and twitter to identify fake news

百度贴吧分布式爬虫，用于贴吧数据挖掘。从贴吧维度和用户维度进行数据分析

NodeJS package that fetches a given URL's title, description, images, links etc.

Your good mornings ☀️

电视剧/番剧自动化重命名工具

Get Aliexpress product details in JSON

An open source webapp for scraping: towards a public service for webscraping

A GUI for Instaloader to scrape users and hashtags with on Instagram

CHeSF is the Chrome Headless Scraping Framework, a very very alpha code to scrape javascript intensive web pages

Create and **automatically** update a list of all videos on a YouTube channel (in txt/csv/md form) via YouTube bot with end-to-end web scraping - no API tokens required. Multi-threaded support for YouTube videos list updates.

API for scrapping common meme sites

A simple API for scraping Overwatch stats

🕸 Generates and delivers RSS feeds via HTTP. Create your own feeds or get started quickly with the included configs.

Extract all CSS from a webpage, packaged as a Now V2 Lambda

A library for fetching song lyrics & album art from genius.com 🎶🌈

A web scraper starter project

Access parts of your account unavailable through normal YouTube API access.

Interactive CLI Web Crawler

Web crawling and scraping framework for Golang

Download Impartus lectures, convert to mkv for offline viewing.

Scraper for the official dashboard with current Corona case numbers, traffic light indicators ("Corona-Ampel") and vaccination situation for Berlin.

A tutorial-based introduction to web scraping with Python.

📻 An OLX Scraper using Scrapy + MongoDB. It Scrapes recent ads posted regarding requested product and dumps to NOSQL MONGODB.

Scraps historical stock market data from Yahoo Finance (https://finance.yahoo.com/)

WIP Patreon attachment download written in TypeScript

No description or website provided.

scraper for Rensselaer Polytechnic Institute (RPI)'s Covid Dashboard

Web service and CLI tool for SEO site audit: crawl site, lighthouse all pages, view public reports in browser. Also output to console, json, csv, xlsx, Google Drive.

A POSIX shell script to parse HTML

🔎 A simple yet powerful module to retrieve organic search results and much more from Google.